ManipTrans Efficiently Migrates Human Hands' Manipulative Skills; Dexterous Hands Can Squeeze Toothpaste Too!

Beijing General Artificial Intelligence Research Institute and other organizations proposed the ManipTrans method, which can efficiently transfer human two-handed operation skills to robot dexterous hands to realize complex operations such as screwing bottle caps and capping pens; ManipTrans adopts a two-phase approach, first mimicking human hand motions with a pre-training model, and then fine-tuning it with a residual learning module, which solves the problems of morphology differences and interaction accuracy; the research team also released a large-scale data set DexManipNet, which contains 3,300 operation sequences of 61 tasks and more than 3,000 objects, and has verified the feasibility on a real platform. The research team also released the large-scale dataset DexManipNet, which contains 3,300 operation sequences of 61 tasks and more than 1,200 objects, and has verified the feasibility on a real machine platform.