DEEP QUOTIENT MANIFOLD MODELING

Abstract

One of the difficulties in modeling real-world data is their complex multi-manifold structure due to discrete features. In this paper, we propose quotient manifold modeling (QMM), a new data-modeling scheme that considers generic manifold structure independent of discrete features, thereby deriving efficiency in modeling and allowing generalization over untrained manifolds. QMM considers a deep encoder inducing an equivalence between manifolds; but we show it is sufficient to consider it only implicitly via a bias-regularizer we derive. This makes QMM easily applicable to existing models such as GANs and VAEs, and experiments show that these models not only present superior FID scores but also make good generalizations across different datasets. In particular, we demonstrate an MNIST model that synthesizes EMNIST alphabets.

1. INTRODUCTION

Real-world data are usually considered to involve a multi-manifold structure by having discrete features as well as continuous features; continuous features such as size or location induce a smooth manifold structure in general, whereas discrete features such as digit-class or a new object in the background induce disconnections in the structure, making it a set of disjoint manifolds instead of a single (Khayatkhoei et al., 2018) . While this multiplicity makes modeling data a difficult problem, recently proposed deep generative models showed notable progresses by considering each manifold separately. Extending the conventional models by using multiple generators (Khayatkhoei et al., 2018; Ghosh et al., 2017; Hoang et al., 2018) , discrete latent variables (Chen et al., 2016; Dupont, 2018; Jeong and Song, 2019) , or mixture densities (Gurumurthy et al., 2017; Xiao et al., 2018; Tomczak and Welling, 2018) , they exhibit improved performances in image generations and in learning high-level features. There are, however, two additional properties little considered by these models. First, since discrete features are both common and combinatorial, there can be exponentially many manifolds that are not included in the dataset. For example, an image dataset of a cat playing around in a room would exhibit a simple manifold structure according to the locations of the cat, but there are also numerous other manifolds derivable from it via discrete variations-such as placing a new chair, displacing a toy, turning on a light or their combinations-that are not included in the dataset (see Fig. 1 ). Second, while the manifolds to model are numerous considering such variations, they usually have the same generic structure since the underlying continuous features remain the same; regardless of the chair, toy, or light, the manifold structures are equally due to the location of the cat. Considering these properties, desired is a model that can handle a large number of resembling manifolds, but the aforementioned models show several inefficiencies. They need proportionally many generators or mixture components to model a large number of manifolds; each of them requires much data, only to learn the manifolds having the same generic structure. Moreover, even if they are successfully trained, new discrete changes are very easy to be made, yet they cannot generalize beyond the trained manifolds. In this paper, we propose quotient manifold modeling (QMM)-a new generative modeling scheme that considers generic manifold structure independent of discrete features, thereby deriving efficiency in modeling and allowing generalization over untrained manifolds. QMM outwardly follows the multi-generator scheme (Khayatkhoei et al., 2018; Ghosh et al., 2017; Hoang et al., 2018) ; but it involves a new regularizer that enforces encoder compatibility-a condition that the inverse maps of the generators to be presented by a single deep encoder. Since deep encoders usually exhibit

