DAXBENCH: BENCHMARKING DEFORMABLE OBJECT MANIPULATION WITH DIFFERENTIABLE PHYSICS

Abstract

Deformable object manipulation (DOM) is a long-standing challenge in robotics and has attracted significant interest recently. This paper presents DaXBench, a differentiable simulation framework for DOM. While existing work often focuses on a specific type of deformable objects, DaXBench supports fluid, rope, cloth . . . ; it provides a general-purpose benchmark to evaluate widely different DOM methods, including planning, imitation learning, and reinforcement learning. DaXBench combines recent advances in deformable object simulation with JAX, a high-performance computational framework. All DOM tasks in DaXBench are wrapped with the OpenAI Gym API for easy integration with DOM algorithms. We hope that DaXBench provides to the research community a comprehensive, standardized benchmark and a valuable tool to support the development and evaluation of new DOM methods. The code and video are available online * .

1. INTRODUCTION

Deformable object manipulation (DOM) is a crucial area of research with broad applications, from household (Maitin-Shepard et al., 2010; Miller et al., 2011; Ma et al., 2022) to industrial settings (Miller et al., 2012; Zhu et al., 2022) . To aid in algorithm development and prototyping, several DOM benchmarks (Lin et al., 2021; Huang et al., 2021) have been developed using deformable object simulators. However, the high dimensional state and action spaces remain a significant challenge to DOM. Differentiable physics is a promising direction for developing control policies for deformable objects. It implements physical laws as differentiable computational graphs (Freeman et al., 2021; Hu et al., 2020) , enabling the optimization of control policies with analytical gradients and therefore improving sample efficiency. Recent studies have shown that differentiable physics-based DOM methods can benefit greatly from this approach (Huang et al., 2021; Heiden et al., 2021; Xu et al., 2022; Chen et al., 2023) . To facilitate fair comparison and further advancement of DOM techniques, it is important to develop a general-purpose simulation platform that accommodates these methods. We present DaXBench, a differentiable simulation framework for deformable object manipulation (DOM). In contrast with current single-task benchmarks, DaXBench encompasses a diverse range of object types, including rope, cloth, liquid, and elasto-plastic materials. The platform includes tasks with varying levels of difficulty and well-defined reward functions, such as liquid pouring, cloth folding, and rope wiping. These tasks are designed to support high-level macro actions and low-level controls, enabling comprehensive evaluation of DOM algorithms with different action spaces. We benchmark eight competitive DOM methods across different algorithmic paradigms, including sampling-based planning, reinforcement learning (RL), and imitation learning (IL). For planning methods, we consider model predictive control with the Cross Entropy Method (CEM-MPC) (Richards, 2005) , differentiable model predictive control (Hu et al., 2020) , and a combination of † These authors contributed equally. ‡ This work is partially completed at the SEA AI Lab. et al., 2020) . However, each simulator specializes in modeling a single type of deformable object and supports only a limited range of object-specific tasks. For example, PlasticineLab (Huang et al., 2021) and DiSECt (Heiden et al., 2021) model only the elasto-plastic objects, including tasks such as sculpting, rolling and cutting the deformable objects. Since the dynamics of each deformable object are vastly different, the physic engine specialized in modeling one type of deformable object cannot be easily extended to another. DaXBench bridges this gap by offering a general-purpose simulation framework that covers a wide range of deformable objects and the relevant tasks. We provide a fairground for comparing and developing all types of DOM methods, especially the differentiable ones.

2.2. DEFORMABLE OBJECT MANIPULATION ALGORITHMS

These challenges make it difficult to apply existing methods for rigid object manipulation directly to DOM. Methods designed for DOM must be able to handle a large number of degrees of freedom in the state space and the complexity of the dynamics. Despite these challenges, many interesting approaches have been proposed for DOM. Depending on the algorithmic paradigms, we categorize DOM methods into three groups: planning, Imitation Learning (IL), and Reinforcement Learning (RL). Our discussion below focuses on the general DOM methods for a standardized set of tasks. We refer to (Sanchez et al., 2018; Khalil & Payeur, 2010) for a more detailed survey on prior methods for robot manipulation of deformable objects. Reinforcement Learning algorithms using differentiable physics such as SHAC (Xu et al., 2022) and APG (Freeman et al., 2021) are also explored. These methods use the ground-truth gradients on the dynamics to optimize the local actions directly, thereby bypassing the massive sampling for the policy gradient estimation. Imitation Learning is another important learning paradigm for DOM. To overcome the "curse of history", the existing works such as (Seita et al., 2021; Sundaresan et al., 2020; Ganapathi et al., 



two. For RL domains, we consider Proximal Policy Optimization (PPO)(Schulman et al., 2017)   with non-differentiable dynamics, and Short-Horizon Actor-Critic (SHAC)(Xu et al., 2022)  and Analytic Policy Gradients (APG)(Freeman et al., 2021)  with analytical gradients. For IL, Transporter networks(Seita et al., 2021)  with non-differentiable dynamics and Imitation Learning via Differentiable physics (ILD) (Chen et al., 2023) are compared. Our experiments compare algorithms with and without analytic gradients on each task, providing insights into the benefits and challenges of differentiable-physics-based DOM methods.DaXBench provides a deformable object simulator, DaX, which combines recent advances in deformable object simulation algorithms(Xu et al., 2022; Chen et al., 2023)  with the high-performance computational framework JAX(Bradbury et al., 2018). This integration allows for efficient autodifferentiation and parallelization across multiple accelerators, such as multiple GPUs. All task environments are wrapped with the OpenAI Gym API(Brockman et al., 2016), enabling seamless integration with DOM algorithms for fast and easy development. By providing a comprehensive and standardized simulation framework, we aim to facilitate algorithm development and advance the state of the art in DOM. Moreover, our experimental results show that the dynamics model in DaXBench enables direct sim-to-real transfer to a real robot for rope manipulation, indicating the potential applicability of our simulation platform to real-world problems.

