BAYESIAN META-LEARNING FOR FEW-SHOT 3D SHAPE COMPLETION

Abstract

Estimating the 3D shape of real-world objects is a key perceptual challenge. It requires going from partial observations, which are often too sparse and incomprehensible for the human eye, to detailed shape representations that vary significantly across categories and instances. We propose to cast shape completion as a Bayesian meta-learning problem to facilitate the transfer of knowledge learned from observing one object into estimating the shape of another object. To facilitate the learning of object shapes from sparse point clouds, we introduce an encoder that describes the posterior distribution of a latent representation conditioned on the sparse cloud. With its ability to isolate object-specific properties from object-agnostic properties, our meta-learning algorithm enables accurate shape completion of newly-encountered objects from sparse observations. We demonstrate the efficacy of our proposed method with experimental results on the standard ShapeNet and ICL-NUIM benchmarks.

1. INTRODUCTION

The task of estimating 3D geometry from sparse observations, commonly referred to as shape completion, is a key perceptual challenge and an integral part of many mission-critical problems, including robotics (Varley et al., 2017) and autonomous driving (Giancola et al., 2019; Stutz & Geiger, 2018) . Recently, a series of methods (Mescheder et al., 2019; Park et al., 2019) have achieved great success by using the observations to infer the parameters of an implicit 3D geometric representation of the targets object. However, with some notable exceptions (Yuan et al., 2018) , such methods require relatively dense observations to achieve high accuracy, which is usually impractical in real situations. In this paper we introduce a novel methodology that enables state-of-the-art shape completion of previously unseen objects from highly sparse observations. Our insight comes from the following simple intuition: "Can we leverage the geometric information available in one object to improve shape completion results on another target object?" Meta-learning is an emerging field of study in machine learning that serves this very purpose. By training a model on multiple inter-related tasks, it learns how to learn new tasks efficiently from a small amount of observations. Recently proposed meta-learning methods often achieve this by parameterizing the input-output relationship with task-specific latent variables and training a separate, task-agnostic model/mechanism that can infer these task-specific variables from sparse observations of the target task (Chang et al., 2015; Finn et al., 2017; Garnelo et al., 2018) . We can cast the shape completion problem as a Bayesian meta-learning problem by treating each object as a task and its sparse observations as the corresponding contextual dataset. In popular Bayesian variants of meta-learning (Edwards & Storkey, 2017; Eslami et al., 2018; Garnelo et al., 2018) , the task-specific latent variables are treated as random variables, and the aforementioned task-agnostic model (i.e. the encoder) is represented as a posterior distribution of the latent variables conditioned on sparse observations. In this study, we combine probabilistic meta-learning with recent shape completion methods that represent the geometry of a given object with implicit parameters, such as the parameters of a signed distance function (SDF). By training an encoder that computes the posterior distribution of these implicit parameters conditioned against sparse observations, we develop a framework that enables the few-shot learning of implicit geometric functions. Under appropriate regularity conditions, the computation of correct posterior distribution leads to optimal prediction in the sense of Bayes Risk (Maeda et al., 2020) . Our proposed approach is a natural extension of many

