NEURAL IMPLICIT SHAPE EDITING USING BOUNDARY SENSITIVITY

Abstract

Neural fields are receiving increased attention as a geometric representation due to their ability to compactly store detailed and smooth shapes and easily undergo topological changes. Compared to classic geometry representations, however, neural representations do not allow the user to exert intuitive control over the shape. Motivated by this, we leverage boundary sensitivity to express how perturbations in parameters move the shape boundary. This allows to interpret the effect of each learnable parameter and study achievable deformations. With this, we perform geometric editing: finding a parameter update that best approximates a globally prescribed deformation. Prescribing the deformation only locally allows the rest of the shape to change according to some prior, such as semantics or deformation rigidity. Our method is agnostic to the model its training and updates the NN in-place. Furthermore, we show how boundary sensitivity helps to optimize and constrain objectives (such as surface area and volume), which are difficult to compute without first converting to another representation, such as a mesh.

1. INTRODUCTION

A neural field is a neural network (NN) mapping every point in a domain of interest, typically of 2 or 3 dimensions, to one or more outputs, such as a signed distance function (SDF), occupancy probability, opacity or color. This allows to represent smooth, detailed, and watertight shapes with topological flexibility, while being compact to store compared to classic implicit representations (Davies et al., 2020) . When the NN is trained not on a single shape but instead an entire collection, each shape is encoded in a latent vector, which is an additional input to the NN (Park et al., 2019; Chen & Zhang, 2019; Mescheder et al., 2019) . As a result, neural fields are receiving increased interest as a geometric representation in numerous applications, such as shape generation (Park et al., 2019 ), shape completion (Chibane et al., 2020) , shape optimization (Remelli et al., 2020) , scene representation (Sitzmann et al., 2020), and view synthesis (Mildenhall et al., 2020) . Some pioneering works have also investigated geometry processing, like smoothing and deformation, on neural implicit shapes (Yang et al., 2021; Remelli et al., 2020; Mehta et al., 2022; Guillard et al., 2021) , but these can be computationally costly or resort to intermediate mesh representations. In part, this difficulty stems from the shape being available only implicitly as the sub-level set of the field. While intuitive (often synonymous with local) geometric control is a key design principle of classic explicit or parametric representations (like meshes, splines, or subdivision schemes), it is not trivial to edit even classic implicit representations, especially ones with global functions (Baerentzen & Christensen, 2002) . Previous works on neural implicit shape editing have focused on the shape semantics, i.e. changing part-level features based on the whole shape structure, but achieve this through tailored training procedures or architectures or resort to intermediate mesh representations. We instead propose a framework which unifies geometric and semantic editing and which is agnostic to the model and its training and modifies the given model in-place akin to classic representations. To treat the geometry, not the field, as the primary object we consider boundary sensitivity to relate changes in the function parameters and the implicit shape. This allows us to express and interpret a basis for the displacement space. In this framework, the user supplies a target displacement on (a part) of the shape boundary in the form of deformation vectors or normal displacements at a set of surface points. Employing boundary sensitivity we find the parameter update which best approximates the prescribed deformation. In geometric editing, we prescribe an exact geometric update on the entirety of the boundary. Akin to local control in classic representations, we especially study the case where the prescribed displacement is local and the rest of the boundary is fixed. In semantic editing only a part of the boundary is prescribed a target displacement. The remaining unconstrained displacement is determined by leveraging the generalization capability of the model as an additional prior, producing semantically consistent results on the totality of the shape. Another prior often used in shape editing is based on deformation energy, such as as-rigid-as-possible (Sorkine & Alexa, 2007) or as-Killing-as-possible (Solomon et al., 2011) , which generates physically plausible deformations minimizing stretch and bending. This is not trivially applicable to implicit surfaces, as there is no natural notion of stretch due to ambiguity in tangent directions. We discuss a few options to resolve this ambiguity and demonstrate that boundary sensitivity can be leveraged to optimize directly in the space of expressible deformations. Lastly, we use level-set theory and boundary sensitivity to constrain a class of functionals, which are difficult to compute without first converting to another representation, such as a mesh. As a specific use-case, we consider fixing the volume of a shape to prevent shrinkage during smoothing.

2. RELATED WORK

Implicit Shape Representations and Manipulation Implicit shape representations or level-sets have been widely used in fields such as computer simulation (Sethian & Smereka, 2003) , shape optimization (van Dijk et al., 2013), imaging, vision, and graphics (Vese, 2003; Tsai & Osher, 2003) . Classically, an implicit function is represented as a linear combination of either a few global basis functions, such as in blobby models (Muraki, 1991) , or many local basis functions supported on a (potentially adaptive) volumetric grid (Whitaker, 2002) . While few global bases use less memory, the task of expressing local displacements is generally ill-posed (Whitaker, 2002) . Hence, methods for interactive editing of implicit shapes are formulated for grid-supported level-sets (Museth et al., 2002; Baerentzen & Christensen, 2002) . These methods use a prescribed velocity field, for which the level-set-equation -a partial differential equation (PDE) modelling the evolution the surface -is solved using numerical schemes on the discrete spatiotemporal grid. Neural Fields In neural fields, a NN is used to represent an implicit function. Different from classic implicit representations, these are non-linear and circumvent the memory-expressivity dilemma Davies et al. (2020) . In addition, automatic differentiation also provides easy access to differential surface properties, such as normals and curvatures, useful for smoothing and deformation (Yang et al., 2021; Mehta et al., 2022; Atzmon et al., 2021) or more exact shape fitting (Novello et al., 2022) . Early works propose to use vanilla multilayer perceptrons (MLPs) to learn occupancy probability (Mescheder et al., 2019; Chen & Zhang, 2019) or signed distance functions (Park et al., 2019; Atzmon & Lipman, 2020) , whose level-sets define the shape boundary. Conditioning the NN on a latent code as an additional input allows to decode a collection of shapes with a single NN (Chen & Zhang, 2019; Mescheder et al., 2019; Park et al., 2019) . Later works build upon these constructions by introducing a spatial structure on the latent codes using (potentially adaptive) grids, which affords more spatial control when generating novel shapes (Ibing et al., 2021) and allows to reconstruct more complex shapes and scenes (Jiang et al., 2020; Peng et al., 2020; Chibane et al., 2020) . In this work, we develop a method to interactively modify shapes generated by any of these methods. Neural Shape Manipulation Although there are many previous works on the deformation of shapes with NNs, we focus only on methods that use neural implicit representations. These can roughly be sorted into two groups based on their guiding principle. Methods in the first group manipulate shapes based on a semantic principle. Hertz et al. ( 2022) create a generative framework with part-level control consisting of three NNs decomposing and mixing shapes in the latent space. Elsner et al. ( 2021) encourage the latent code to act as geometric control points, allowing to manipulate the geometry by moving the control points. Chen et al. (2021) demonstrate how to interpolate between shapes while balancing their semantics by choosing which layer's features to track. Similar to our method, this is agnostic to the training and the model. Hao et al. (2020) learn a joint latent space between an SDFs and its coarse approximation in the form of a union of spherical primitives. This way modifications of the spheres can be translated to the best matching change of the high-fidelity shape.

