MyoDex: GENERALIZABLE REPRESENTATIONS FOR DEXTEROUS PHYSIOLOGICAL MANIPULATION

Abstract

The complexity of human dexterity builds on the coordinated actuation of a large number of muscles. Still, much is to be understood about how the control of such overactuated system for hand manipulation behaviors emerge and quickly and flexibly adapts to new behaviours. In this work we aim at learning generalizable representations for dexterous manipulation behaviors with a physiologically realistic hand model: MyoHand. In contrast to prior works demonstrating isolated postural and force control, here we demonstrate musculoskeletal agents (MyoDex) exhibiting contact-rich dynamic dexterous manipulation behaviors in simulation. Furthermore, to demonstrate generalization, we show that a single MyoDex agent can be trained to solve up-to 14 different contact-rich tasks. Aligned with human development, simultaneous learning of multiple tasks imparts physiological coordinated muscle contractions i.e., muscle synergies, that are not only shared amongst those in-domain tasks but are also effective to a large series of new outof-domain tasks. By leveraging these pre-trained manipulation synergies, we show generalization to 38 additional previously unsolved tasks. While physiological behaviors with large muscle groups (such as legged-locomotion, armreaching, etc) have been demonstrated before, to the best of our knowledge nimble behaviors of this complexity with smaller muscle groups and generalizable representations for the control of the overactuated human hand are being demonstrated for the first time.

1. INTRODUCTION

Human hands are astonishingly complex and require effective coordination of various muscle groups to impart effective manipulation abilities. Manipulation behaviors are incredibly sophisticated as, because of the overactuated musculoskeletal system, they evolve in a high-dimensional search space populated with intermittent contact dynamics between the hands' degrees of freedom and the object. Indeed, even in the field of robotics where joints and actuations are simpler, finding effective manipulation strategies nonetheless remains a challenge Kumar et al. (2016); Rajeswaran et al. (2018); Nagabandi et al. (2020) . The human hand consists of 29 bones, 23 joints, and more than 50 muscles Sobinov & Bensmaia (2021) . The complex multi-articular, multi-joint, pulling-only properties of the musculoskeletal system Sobinov & Bensmaia (2021) make physiological dexterous manipulation a very different and unique problem as opposed to joint based control typically adopted in robotics. In biology, the control of such complex musculoskeletal system is made possible by the fact that muscles are not activated in isolation, but rather, that different muscles are activated in a proportional way as a unit. This phenomenon is known as muscle synergy Bizzi & Cheung (2013) . Synergies allows the biological motor system -via the modular organization of the movements in the spinal cord Bizzi & Cheung (2013); Caggiano et al. (2016) -to simplify the control problem, solving tasks by building on a limited number of shared solutions d' Avella et al. (2003); d'Avella & Bizzi (2005) . Those shared synergies are suggested to be the fundamental building blocks for quickly learning new and more complex motor behaviours Yang et al. (2019); Dominici et al. (2011); Cheung et al. (2020) . Manipulation behaviors, the subject of this investigation, are further complicated because they unfold on a sequence of phases: reaching to the object, hand-object contact, and manipulation with object maneuvers. Before the hand-object contact, the human hand is pre-shaped to conform to the object such that it is often possible to predict the object that is going to be grasped just by observing the hand pose before hand-object contact Jeannerod ( 1988 2021) in a controlled setting with a physiologically realistic models of the hand, here we present MyoDex agents capable of dynamic dexterous contact rich manipulation behaviors with multiple objects and a variety of tasks e.g. drinking from a cup, playing with toys, etc. Furthermore, by jointly training multiple tasks, we capture reusable synergies in form of a general pre-trained policy that can be further fine-tuned to manipulate 38 previously unsolved tasks with non-trivial affordances. We provide a detailed analysis of emergent physiological details in our achieved behaviors. While we do not claim to have solved physiological dexterous manipulation, we emphasize that manipulation abilities demonstrated here significantly advance the state of the art of the bio-mechanics and neuroscience fields. Along these lines, this investigation is among the first to yield robust control policies exhibiting basic physiological constructs such as synergistic activations of muscle groups during dexterous manipulations. Nevertheless, further work is required to rigorously ground them in experimental validation. More specifically, our main contributions are: • We show for the first time that despite the high numbers of degrees of freedom, the multiarticular-multi-joint and the third order muscle dynamics of muscle control, it is possible to control a physiologically realistic musculoskeletal model of the hand to perform contact-rich skilled manipulation behaviors on up-to 14 different tasks. • We show that joint multi-task learning facilitates the learning of physiological representations that exploit muscle coordination in a lower-dimensional space of synergies to solve specific tasks. • Our framework MyoDex leverages joint multi-task learning to recover reusable representations (synergies) that allows for easier fine-tuning in both in-domain and out-ofdomain tasks (including one/few shot learning). Leveraging these synergies the MyoDex solves 38 previously unsolved tasks.

2. RELATED WORKS

Experimental studies of functional hand manipulations have been limited both by challenges in sensing, the discontinuous hand-object interactions and because of the limited ability to record many muscles of the hand simultaneously. 



);Santello et al. (2002);Thakur  et al. (2008);Yan et al. (2020). Contact and manipulation of the object are goal-driven so that the way the object is held depends on both the object affordance and the intermediate task goalsJeannerod  (1988).In this work, we seek to further our understanding of physiological dexterity by imparting dexterous manipulation ability to an anatomically realistic hand-fore-arm modelCaggiano et al. (2022). While prior works have not been able to scale beyond dexterous grasping McFarland et al. (2021); Mirakhorlo et al. (2018); Saito et al. (2021); Crouch & Huang (2015); Engelhardt et al. (

Musculoskeletal models of the hand McFarland et al. (2021); Lee et al. (2015); Saul et al. (2015) have been developed to overcome some of the experimental limitations and produce insights on the kinematic information of the muscles and joints. While musculoskelatal models of large muscle groups have been extensively developed and used Delp et al. (2007); Seth et al. (2018), models of the hand have been more challenging both because of the smaller muscle groups involved and the complexity of the behaviour they can produce. Indeed, simulations of the hand mostly focus on fingertips, pinch force McFarland et al. (2021), kinematic motion McFarland et al. (2021), and passive grasping McFarland et al. (2021). Furthermore, most of those studies are also limited by intensive computational needs and restricted contact forces. Those

