ZERO-SHOT IMAGE RESTORATION USING DENOISING DIFFUSION NULL-SPACE MODEL

Abstract

Most existing Image Restoration (IR) models are task-specific, which can not be generalized to different degradation operators. In this work, we propose the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring. DDNM only needs a pre-trained off-the-shelf diffusion model as the generative prior, without any extra training or network modifications. By refining only the null-space contents during the reverse diffusion process, we can yield diverse results satisfying both data consistency and realness. We further propose an enhanced and robust version, dubbed DDNM + , to support noisy restoration and improve restoration quality for hard tasks. Our experiments on several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot IR methods. We also demonstrate that DDNM + can solve complex real-world applications, e.g., old photo restoration.



Figure 1: We use DDNM + to solve various image restoration tasks in a zero-shot way. Here we show some of the results that best characterize our method, where y is the input degraded image and x 0 represents the restoration result. Part (a) shows the results of DDNM + on image super-resolution (SR) from scale 2× to extreme scale 256×. Note that DDNM + assures strict data consistency. Part (b) shows multiple results of DDNM + on inpainting and colorization. Part (c) shows the results of DDNM + on SR with synthetic noise and colorization with real-world noise. Part (d) shows the results of DDNM + on old photo restoration. All the results here are yielded in a zero-shot way.

funding

* Equal contribution. work was supported in part by Shenzhen Research Project under Grant JCYJ20220531093215035 and Grant JSGGZD20220822095800001.

