QRGAN: QUANTILE REGRESSION GENERATIVE AD-VERSARIAL NETWORKS

Abstract

Learning high-dimensional probability distributions by competitively training generative and discriminative neural networks is a prominent approach of Generative Adversarial Networks (GANs) among generative models to model complex real-world data. Nevertheless, training GANs likely suffer from non-convergence problem, mode collapse and gradient explosion or vanishing. Least Squares GAN (LSGANs) and Wasserstein GANs (WGAN) are of representative variants of GANs in literature that diminish the inherent problems of GANs by proposing the modification methodology of loss functions. However, LSGANs often fall into local minima and cause mode collapse. While WGANs unexpectedly encounter with inefficient computation and slow training due to its constraints in Wasserstein distance approximation. In this paper, we propose Quantile Regression GAN (QR-GAN) in which quantile regression is adopted to minimize 1-Wasserstein distance between real and generated data distribution as a novel approach in modification of loss functions for improvement of GANs. To study the culprits of mode collapse problem, the output space of discriminator and gradients of fake samples are analyzed to see if the discriminator guides the generator well. And we found that the discriminator should not be bounded to specific numbers. Our proposed QRGAN exposes high robustness against mode collapse problem. Furthermore, QRGAN obtains an apparent improvement in the evaluation and comparison of Frechet Inception Distance (FID) for generation performance assessment compared to existing variants of GANs.

1. INTRODUCTION

Deep learning-based data generation techniques have proved their successes in many real-world applications. Thanks to the rising of generative models, the generation of audio, images and videos, either unconditionally or conditionally, has achieved remarkable advancements in recent years. Text or structured data can be generated easily as well in many recent studies. Data generation techniques bring about efficency and creativity in human activities at every conner of the world. Among the most influent and successful methods for data generation experiments, Variational Autoencoders (VAE) Kingma & Welling (2014) and Generative Adversarial Networks (GANs) Goodfellow et al. ( 2014) are the fundamental representatives of generative models. Variational Autoencoders (VAE): VAEs regularize the encoder output to be a known distribution. This regularization is applied to each sample. For latent variable z and input x, p(z|x), not p(z), is pushed to the prior distribution. With the additional reconstruction loss, the two objectives may conflict each other. Usually, mean square error (MSE) loss is used for the reconstruction loss. Because if the global minimum of MSE loss is at the expected value of the distribution, the decoders generate blurry outputs. PixelVAE Gulrajani et al. Generative Adversarial Networks (GANs): GANs regularize the entire decoder input distribution to be a known distribution. In other words, GANs regularize p(z), not p(z|x). Therefore, generators can generate sharp outputs without the conflicting objectives in VAEs. A number of GAN variants has been presented in literature to improve the data generation including Least-square 1



(2016) fixes VAE's blurry output by replacing MSE loss by PixelCNN van den Oord et al. (2016) decoder.

