BLURRING DIFFUSION MODELS

Abstract

Recently, Rissanen et al. (2022) have presented a new type of diffusion process for generative modeling based on heat dissipation, or blurring, as an alternative to isotropic Gaussian diffusion. Here, we show that blurring can equivalently be defined through a Gaussian diffusion process with non-isotropic noise. In making this connection, we bridge the gap between inverse heat dissipation and denoising diffusion, and we shed light on the inductive bias that results from this modeling choice. Finally, we propose a generalized class of diffusion models that offers the best of both standard Gaussian denoising diffusion and inverse heat dissipation, which we call Blurring Diffusion Models.

1. INTRODUCTION

Diffusion models are becoming increasingly successful for image generation, audio synthesis and video generation. Diffusion models define a (stochastic) process that destroys a signal such as an image. In general, this process adds Gaussian noise to each dimension independently. However, data such as images clearly exhibit multi-scale properties which such a diffusion process ignores. Recently, the community is looking at new destruction processes which are referred to as deterministic or 'cold' diffusion (Rissanen et al., 2022; Bansal et al., 2022) . In these works, the diffusion process is either deterministic or close to deterministic. For example, in (Rissanen et al., 2022) a diffusion model that incorporates heat dissipation is proposed, which can be seen as a form of blurring. Blurring is a natural destruction for images, because it retains low frequencies over higher frequencies. However, there still exists a considerable gap between the visual quality of standard denoising diffusion models and these new deterministic diffusion models. This difference cannot be explained away by a limited computational budget: A standard diffusion model can be trained with relative little compute (about one to four GPUs) with high visual quality on a task such as unconditional CIFAR10 generation 1 . In contrast, the visual quality of deterministic diffusion models have been 1 An example of a denoising diffusion implementation https://github.com/w86763777/pytorch-ddpm 



(a) Diffusion (Sohl-Dickstein et al., 2015; Ho et al., 2020) (b) Heat Dissipation (Rissanen et al., 2022) (c) Blurring Diffusion

Figure 1: Comparison between standard diffusion, heat dissipation and blurring diffusion.

1

