BRGANS: STABILIZING GANS' TRAINING PROCESS WITH BROWNIAN MOTION CONTROL

Abstract

The training process of generative adversarial networks (GANs) is unstable and does not converge globally. In this paper, we propose a universal higher-order noise-based controller called Brownian Motion Controller (BMC) that is invariant to GANs' frameworks so that the training process of GANs is stabilized. Specifically, starting with the prototypical case of Dirac-GANs, we design a BMC and propose Dirac-BrGANs, which retrieve exactly the same but reachable optimal equilibrium regardless of GANs' framework. The optimal equilibrium of our Dirac-BrGANs' training system is globally unique and always exists. Furthermore, we give theoretical proof that the training process of Dirac-BrGANs achieves exponential stability almost surely for any arbitrary initial value and derive bounds for the rate of convergence. Then we extend our BMC to normal GANs and propose BrGANs. We provide numerical experiments showing that our BrGANs effectively stabilize GANs' training process and obtain state-of-theart performance in terms of FID and inception score compared to other stabilizing methods.

1. INTRODUCTION

Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are popular deep learning based generative architecture. Given a multi-dimensional input dataset with unknown P real , GANs can obtain an estimated P model and produce new entries that are as close to indistinguishable as possible from the input entries. For example, GANs can be used to generate images that look real to the human eyes (Wang et al., 2017) . A GAN's architecture consists of two neural networks: a generator and a discriminator. The generator creates new elements that resemble the entries from the input dataset as closely as possible. The discriminator, on the other hand, aims to distinguish the (counterfeit) entries produced by the generator from the original members of the input dataset. The GAN's two networks can be modeled as a minimax problem; they compete against one another while striving to reach a Nash-equilibrium, an optimal solution where the generator can produce fake entries that are, from the point of view of the discriminator, in all respects indistinguishable from real ones. Many previous methods (Mescheder et al., 2018; Arjovsky & Bottou, 2017; Nagarajan & Kolter, 2017; Kodali et al., 2017) have investigated the causes of such instabilities and attempted to reduce them by introducing various changes to the GAN's architecture. However, as Mescheder et al. (2018) show in their study, where they analyze the convergence behavior of several GANs models, despite bringing significant improvements, GANs and its variations are still far from achieving stability in the general case. To accomplish this goal, in our work, we design a Brownian Motion Control (BMC) using control theory and propose a universal model, BrGANs, to stabilize GANs' training process. We start with the prototypical Dirac-GAN (Mescheder et al., 2018 ) and analyze its system of training dynamics. We then design a Brownian motion controller (BMC) on the training dynamic of Dirac-GAN in order to stabilize this system over time domain t. We generalize our BMC to normal GANs' setting and propose BrGANs. • We design Brownian Motion Controller (BMC), a universal higher order noise-based controller for GANs' training dynamics, which is compatible with all GANs' frameworks, and we give both theoretical and empirical analysis showing BMC effectively stabilizes GANs' training process. • Under Dirac-GANs' setting, we propose Dirac-BrGANs and conduct theoretical stability analysis to derive bounds on the rate of convergence. Our proposed Dirac-BrGANs are able to converge globally with exponential stability. • We extend BMC to normal GANs' settings and propose BrGANs. In experiments, our BrGANs converge faster and perform better in terms of inception scores and FID scores on CIFAR-10 and CelebA datasets than previous baselines in various GANs models. et al., 2016) , modifies the discriminator to achieve better stability.



Unfortunately, training GANs often suffers from instabilities. Previously, theoretical analysis has been conducted on GAN's training process. Fedus et al. (2018) argue that the traditional view of considering training GANs as minimizing the divergence of real distribution and model distribution is too restrictive and thus leads to instability. Arora et al. (2018) show that GANs training process does not lead generator to the desired distribution. Farnia & Ozdaglar (2020) suggest that current training methods of GANs do not always have Nash equilibrium, and Heusel et al. (2017a) is able to push GANs to converge to local Nash equilibrium using a two-time scale update rule (TTUR).

Figure 1: The gradient map and convergence behavior of Dirac-WGANs (first row) and Dirac-BrWGANs (second row), where the Nash equilibrium of both model should be at (0, 0) T .

RELATED WORKTo stabilize GANs training process, a lot of work has been done on modifying its training architecture.Karras et al. (2018)  train the generator and the discriminator progressively to stabilize the training process.Wang et al. (2021)  observe that during training, the discriminator converges faster and dominates the dynamics. They produce an attention map from the discriminator and use it to improve the spatial awareness of the generator. In this way, they push GANs' solution closer to the equilibrium.On the other hand, many work stabilizes GANs' training process with modified objective functions.Kodali et al. (2017)  add gradient penalty to their objective function to avoid local equilibrium with their model called DRAGAN. This method has fewer mode collapses and can be applied to a lot of GANs' frameworks. Other work, such as Generative Multi-Adversarial Network (GMAN)(Durugkar et al., 2017), packing GANs (PacGAN)(Lin et al., 2017)  and energy-based GANs (Zhao

