This paper presents a novel object-centric contact representation ContactGen for hand-object interaction. The ContactGen comprises three components: a contact map indicates the contact location, a part map represents the contact hand part, and a direction map tells the contact direction within each part. Given an input object, we propose a conditional sequential generative model to predict ContactGen and adopt model-based optimization to predict diverse and geometrically feasible grasps. Experimental results demonstrate our method can generate high-fidelity and diverse human grasps for various objects.
Object-centric ContactGen consists of three maps, contact map, part map and direction map. All maps are defined on object points. Contact map represents the contact probability of the point. Part map indicate the hand part label in contact with the object point. Direction map records the direction of this point w.r.t. the hand part.
Conditioned on the input object point cloud, we decompose the joint distribution of ContactGen into three conditional probabilities by a sequential CVAE. The contact map is conditioned on object input; the part map is additionally conditioned on contact map; direction map additionally conditioned on part map. Each component is controlled by a latent code.
Given the sampled obejct-centric ContactGen from the generative model, we decode it into the corresponding hand grasp. To achieve this, we convert the MANO model into a piecewise SDF model to enhance the compatibility with the proposed ContactGen for grasp synthesis, formulate a model-based optimization to infer the pose and shape of the hand.
@inproceedings{liu2023contactgen,
title={ContactGen: Generative Contact Modeling for Grasp Generation},
author={Liu, Shaowei and Zhou, Yang and Yang, Jimei and Gupta, Saurabh and Wang, Shenlong},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
year={2023}
}