ms-mano.robotflow.ai Open in urlscan Pro
2606:50c0:8002::153  Public Scan

Form analysis
0 forms found in the DOM

Text Content

CVPR'24
MUSCULOSKELETAL-MANO
ENABLING HAND POSE TRACKING WITH BIOMECHANICAL CONSTRAINTS

Pengfei Xie1*
Wenqiang Xu2*
Tutian Tang2
Zhenjun Yu2
Cewu Lu2
1Southeast University
2Shanghai Jiao Tong University
*Equal Contribution
Read Paper
View Code
coming
soon
Watch Video
coming
soon
Supplementary Materials
coming
soon



ABSTRACT

This work proposes a novel learning framework for visual hand dynamics analysis
that takes into account the physiological aspects of hand motion. The existing
models, which are simplified joint-actuated systems, often produce unnatural
motions. To address this, we integrate a musculoskeletal system with a learnable
parametric hand model, MANO, to create a new model, MS-MANO. This model emulates
the dynamics of muscles and tendons to drive the skeletal system, imposing
physiologically realistic constraints on the resulting torque trajectories. We
further propose a simulation-in-the-loop pose refinement framework, BioPR, that
refines the initial estimated pose through a multi-layer perceptron (MLP)
network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR
is conducted in two separate parts. The accuracy of MS-MANO is compared with
MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale
public datasets and two recent state-of-the-art methods. The results demonstrate
that our approach consistently improves the baseline methods both quantitatively
and qualitatively.
The parametric MANO model
Musculoskeletal structure of hands
The proposed MS-MANO model


PIPELINE


RESULTS




CONTACT

If you have any inquiries or require further clarification regarding the paper,
its associated code, or any other related questions, we encourage you to get in
touch with us. Please feel free to send your questions via email to xiepf2002 #
gmail.com.