GALLERY: A DATASET AND ENVIRON-MENT FOR PROGRAMMATIC CAD RECONSTRUCTION Anonymous

Abstract

Parametric computer-aided design (CAD) is a standard paradigm used for the design of manufactured objects. CAD designers perform modeling operations, such as sketch and extrude, to form a construction sequence that makes up a final design. Despite the pervasiveness of parametric CAD and growing interest from the research community, a dataset of human designed 3D CAD construction sequences has not been available to-date. In this paper we present the Fusion 360 Gallery reconstruction dataset and environment for learning CAD reconstruction. We provide a dataset of 8,625 designs, comprising sequential sketch and extrude modeling operations, together with a complementary environment called the Fusion 360 Gym, to assist with performing CAD reconstruction. We outline a standard CAD reconstruction task, together with evaluation metrics, and present results from a novel method using neurally guided search to recover a construction sequence from a target geometry.

1. INTRODUCTION

The manufactured objects that surround us in everyday life are created in computer-aided design (CAD) software using common modeling operations such as sketch and extrude. With just these two modeling operations, a highly expressive range of 3D designs can be created (Figure 1 ). Parametric CAD files contain construction sequence information that is critical for documenting design intent, maintaining editablity, and downstream simulation and manufacturing. Despite the value of this information, it is often lost due to data translation or error and must be reverse engineered from geometry or even raw 3D scan data. The task of reconstructing CAD operations from geometry has been pursued for over 40 years (Shah et al., 2001) and is available in commercial CAD software using heuristic approaches (Autodesk, 2012; Dassault, 2019) . Recent advances in neural networks for 3D shape generation has spurred new interest in CAD reconstruction, due to the potential to generalize better and further automate this challenging problem. However, learning-based approaches (2020) . The absence of real world data has limited work on CAD reconstruction using common sketch and extrude modeling operations. Instead a focus has been on reconstruction from simple geometric primitives (Sharma et al., 2017; Tian et al., 2019; Ellis et al., 2019) that lack the rich parametric sketches commonly used in mechanical CAD (e.g. Figure 2 ). As there is no existing learning-based approach to reconstruct sketch and extrude sequences, we take a first step towards this goal by introducing data, a supporting software environment, and a novel action representation for reconstructing sketch and extrude designs. In this paper we present the Fusion 360 Gallery reconstruction dataset and environment for learning CAD reconstruction. The dataset contains 8,625 designs created by users of Autodesk Fusion 360 using a simple subset of CAD modeling operations: sketch and extrude. To the best of our knowledge this dataset is the first to provide human designed 3D CAD construction sequence data for use with machine learning. To support research with the dataset we provide an environment called the Fusion 360 Gym for working with CAD reconstruction. A key motivation of this work is to provide insights into the process of how people design objects. Furthermore, our goal is to provide a universal benchmark for research and evaluation of learning-based CAD reconstruction algorithms, bridging the gap between the computer graphics and machine learning community. To this end we describe a standard CAD reconstruction task and associated evaluation metrics with respect to the ground truth construction sequence. We also introduce a novel action representation for CAD reconstruction of sketch and extrude designs using neurally guided search. This search employs a policy, trained using imitation learning, consisting of a graph neural network encoding of CAD geometry. This paper makes the following contributions: • We present the Fusion 360 Gallery reconstruction dataset containing construction sequence information for 8,625 human-designed sketch and extrude CAD models. • We introduce an environment called the Fusion 360 Gym, standardizing the CAD reconstruction task in a Markov Decision Process formulation. • We introduce a novel action representation to enable neurally guided CAD reconstruction trained on real world construction sequences for the first time.

2. RELATED WORK

CAD Datasets Existing 3D CAD datasets have largely focused on providing mesh geometry (Chang et al., 2015; Wu et al., 2015; Zhou & Jacobson, 2016; Mo et al., 2019b; Kim et al., 2020) . However, the de facto standard for parametric CAD is the boundary representation (B-Rep) format, containing valuable analytic representations of surfaces and curves suitable for high level control of 3D shapes. B-Reps are collections of trimmed parametric surfaces along with topological information which describes adjacency relationships and the ordering of elements such as faces, loops, edges, and vertices (Weiler, 1986) . B-Rep datasets have recently been made available with both human-designed (Koch et al., 2019) and synthetic data (Zhang et al., 2018; Jayaraman et al., 2020; Starly, 2020) . Missing from these datasets is construction sequence information. We believe it is critical to understand not only what is designed, but how that design came about. Parametric CAD files contain valuable information on the construction history of a design. Schulz et al. ( 2014) provide a standard collection of human designs with full parametric history, albeit a limited set of 67 designs in a proprietary format. SketchGraphs (Seff et al., 2020) constrains the broad area of parametric CAD by focusing on the underlying 2D engineering sketches, including sketch construction sequences. In the absence of 3D human design data, learning-based approaches have instead leveraged synthetic CAD construction sequences (Sharma et al., 2017; Li et al., 2020) . The dataset presented in this paper is, to the best of our knowledge, the first to provide humandesigned 3D CAD construction sequence information suitable for use with machine learning.

CAD Reconstruction

The task of CAD reconstruction involves recovering the sequence of modeling operations used to construct a CAD model from geometry input, such as B-reps, triangle meshes, or point clouds. Despite extensive prior work (Shah et al., 2001) , CAD reconstruction remains a challenging problem as it requires deductions on both continuous parameters (e.g., ex-



Figure 1: Top: A subset of designs containing 3D CAD construction sequences from the Fusion 360 Gallery reconstruction dataset. Bottom: An example construction sequence using sketch and extrude modeling operations.

