MULTI-LAYERED 3D GARMENTS ANIMATION

Abstract

Figure 1: We propose a new large-scale 3D garment animation dataset LAYERS, which improves over previous datasets by considering multi-layered 3D garments and more driving factors for garment animation, e.g., environmental wind, besides human body movements. In (a)-(d) we show the new and realistic challenges covered in LAYERS but are omitted in previous datasets.

1. INTRODUCTION

3D garment animation has been an active and important topic in computer graphics and machine learning, due to its great potential in various downstream tasks, including virtual reality, virtual tryon, gaming and film production. While this topic has been extensively studied in the past, generating realistic and faithful animation remains an open research question. In particular, existing approaches are still limited in modeling diverse garments of different topologies and appearances. In addition, the complex interactions between the garment and the human body under the challenging setting of multi-layer garments and with external environmental factors remain much less explored in the literature. To support the development of data-driven approaches for 3D garment animation (Patel et al., 2020; Bertiche et al., 2020; Santesteban et al., 2021) , researchers have built various datasets on real-life scans and synthetic data generated by Physically Based Simulation (PBS) (Narain et al., 2012; Li et al., 2018) . However, most of existing datasets (Bertiche et al., 2020; Patel et al., 2020; Tiwari et al., 2020) consider only human bodies with single-layered garments, where each human body wears either a single dress or an upper t-shirt with lower pants that have limited overlap. The animation of multi-layered garments, such as a t-shirt with a jacket, that obey sophisticated physical dynamics, remain unexplored. In addition, in existing datasets, the moving human body is commonly regarded as the default and only driving factor in animating garments. Other factors, such as wind and friction, are left unconsidered. Such a simplification thus leads to a significant gap between experimental environments and real-world applications, making most approaches developed on top of these datasets less applicable in real life. To bridge the gap between experimental environments and real-world applications and facilitate the advance of 3D garment animation, this paper introduces a new challenging dataset called LAYERS, muLti-lAYerEd gaRmentS dataset, which is carefully generated based on a simulation engine. LAY-ERS focuses on the animation of multi-layered garments, while also taking the wind, another important driving factor besides the human body, into consideration. Specifically, in LAYERS, multilayered garments are prepared as combinations of inner and outer clothes, as shown in Figure 1 . The inner and outer garments adopt different attribute values, e.g., bend stiffness and frictions. All garments on the same human body will interact with each other, constrained by the laws of physics. They are also simultaneously affected by the wind with randomly sampled direction and strength. To demonstrate the quality of LAYERS, we further propose a novel data-driven method, dubbed as LayersNet, for multi-layered 3D garment animation. The core of LayersNet is a neural network based simulation system (Shao et al., 2022) that represents garments as unions of particles. Consequently, all kinds of interactions during garment animation, including the interactions between different parts of one garment, the interactions between different garments on the same body, and the interactions between garments and various driving factors, can be naturally and uniformly regarded as the interactions between particles. Hence, instead of being restricted to a specific driving factor (e.g., the human body) as previous methods, the proposed LayersNet possesses a strong generalization ability across diverse types of human body movements, multi-layered 3D garments, as well as driving factors. As the number of particles in LayersNet are considerably large when fine-grained details of garments and human bodies are preserved, we further exploit the redundancy of garments and extend LayersNet to establish a two-level structural hierarchy for garments where garments are made of patches, and patches are constituted of particles of a fixed configuration. Since the number of patches is much smaller than the number of particles, the interactions between all particles can be efficiently captured by the interactions of patches. Our contributions can be summarized as follows: 1. We propose LAYERS, a large-scale and new dynamic dataset for 3D garment animation. The dataset focuses on multi-layered 3D garments, introducing random wind and friction as additional driving factors besides human body movements. 2. On top of LAYERS, we further propose LayersNet, a novel method for 3D garment animation that uniformly captures interactions among garment parts, different garments, as well as garments against driving factors. The notion of unifying various interactions as particle-based simulations is novel in the literature.

2. RELATED WORK

3D Garment Datasets. Publicly available 3D garment datasets are in great need. Existing datasets are generated either from synthesis (Pumarola et al., 2019; Patel et al., 2020; Santesteban et al., 2021; Bertiche et al., 2020) or real-world scans (Zhang et al., 2017; Zheng et al., 2019; Ma et al., 2020; Tiwari et al., 2020; Cai et al., 2022) . For synthetic datasets, 3DPeople 



(Pumarola et al., 2019)   contains multi-view images including RGB, depth, normal, and scene flow data. TailorNet(Patel  et al., 2020)  provides a synthetic dataset with 20 different garments simulated in 1,782 static SMPL poses for nine body shapes. Santesteban et al. (2021) contributes a dataset composed of two different garments simulated on 56 human motion sequences with 17 body shapes.Cloth3D (Bertiche et al.,

