KNOWLEDGE DISTILLATION BASED DEGRADATION ESTIMATION FOR BLIND SUPER-RESOLUTION

Abstract

Blind image super-resolution (Blind-SR) aims to recover a high-resolution (HR) image from its corresponding low-resolution (LR) input image with unknown degradations. Most of the existing works design an explicit degradation estimator for each degradation to guide SR. However, it is infeasible to provide concrete labels of multiple degradation combinations (e.g., blur, noise, jpeg compression) to supervise the degradation estimator training. In addition, these special designs for certain degradation, such as blur, impedes the models from being generalized to handle different degradations. To this end, it is necessary to design an implicit degradation estimator that can extract discriminative degradation representation for all degradations without relying on the supervision of degradation groundtruth. In this paper, we propose a Knowledge Distillation based Blind-SR network (KDSR). It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network. To learn the KDSR model, we first train a teacher network: KD-IDE T . It takes paired HR and LR patches as inputs and is optimized with the SR network jointly. Then, we further train a student network KD-IDE S , which only takes LR images as input and learns to extract the same implicit degradation representation (IDR) as KD-IDE T . In addition, to fully use extracted IDR, we design a simple, strong, and efficient IDR based dynamic convolution residual block (IDR-DCRB) to build an SR network. We conduct extensive experiments under classic and real-world degradation settings. The results show that KDSR achieves SOTA performance and can generalize to various degradation processes. The code is available at Github.

1. INTRODUCTION

Single image super-resolution (SISR) aims to recover details of a high-resolution (HR) image from its low-resolution (LR) counterpart, which has a variety of downstream applications (Dong et al., 2014; Zhang et al., 2019; Xia et al., 2022d; Fritsche et al., 2019; Xia et al., 2022c; b) . These stateof-the-art methods (Kim et al., 2016; Lim et al., 2017; Lai et al., 2017; Xia et al., 2022a; Wang et al., 2018b) usually assume that there is an ideal bicubic downsampling kernel to generate LR images. However, this simple degradation is different from more complex degradations existing in real-world LR images. This degradation mismatch will lead to severe performance drops. To address the issue, blind super-resolution (Blind-SR) methods are developed. Some Blind-SR works (Wang et al., 2021a; Luo et al., 2022) use the classical image degradation process, given by Eq. 1. Recently, some works (Cai et al., 2019; Bulat et al., 2018) attempted to develop a new and complex degradation process to better cover real-world degradation space, which forms a variant of Blind-SR called real-world super-resolution (Real-SR). The representative works include BSR-GAN (Zhang et al., 2021) and Real-ESRGAN (Wang et al., 2021b) , which introduce comprehensive degradation operations such as blur, noise, down-sampling, and JPEG compression, and control the severity of each operation by randomly sampling the respective hyper-parameters. To better simulate the complex degradations in real-world, they also apply random shuffle of degradation orders (Zhang et al., 2021) and second-order degradation (Wang et al., 2021b) respectively. Since Blind-SR faces almost infinite degradations, introducing prior degradation information to SR networks can help to constrain the solution space and boost SR performance. As shown in Fig. 1 , the way to obtain degradation information can be divided into three categories: (1) Several Non-Blind SR methods (Zhang et al., 2018a; Shocher et al., 2018; Zhang et al., 2020; Soh et al., 2020; Xu et al., 2020) directly take the known degradation information as prior (Fig. 1 In this paper, we aim to design an efficient implicit degradation representation (IDR) learning SR framework that can easily adapt to any degradation process. To this end, we develop a novel knowledge distillation based Blind-SR Network (KDSR). Specifically, as shown in Fig. 1 

2. RELATED WORK

2.1 BLIND SUPER-RESOLUTION In the past few years, numerous Non-Blind SISR methods (Dong et al., 2014; Lim et al., 2017; Zhang et al., 2018a; Ledig et al., 2017; Johnson et al., 2016; Ma et al., 2020; Xia et al., 2023) have



Figure 1: The illustration of different degradation estimators. (a) Non-blind SR methods directly use known degradation information to guide SR networks, such as SRMD (Zhang et al., 2018a). (b) Many Blind-SR methods estimate the explicit degradation with the supervision of ground-truth degradation. (c) Several methods use metric learning to distinguish degradation roughly. (d) Our knowledge distillation (KD) based implicit degradation estimator can estimate accurate implicit degradation representation to guide SR without ground-truth degradation supervision. Blind-SR methods (Gu et al., 2019; Luo et al., 2020; Wang et al., 2021a; Liang et al., 2022; Luo et al., 2022) adopt explicit degradation estimators, which are trained with ground-truth degradation (Fig. 1 (b)). However, these explicit degradation estimators are elaborately designed for specific degradation processes. The specialization makes them hard to transfer to handle other degradation processes. In addition, it is challenging to annotate precise ground-truth labels to represent the multiple degradation combination (Zhang et al., 2021; Wang et al., 2021b) for supervised degradation learning. Therefore, developing implicit degradation representation (IDR) based methods is important. (3) Recently, as shown in Fig. 1 (c), DASR (Wang et al., 2021a) and MM-RealSR (Mou et al., 2022) use metric learning to estimate IDR and quantize degradation severity respectively. However, metric learning methods roughly distinguish degradations by pushing away or pulling close features, which is unstable and cannot fully capture discriminative degradation characteristics for Blind-SR.

(d), KDSR uses a knowledge distillation based implicit degradation estimator (KD-IDE) to predict accurate IDR. Furthermore, we propose a strong yet efficient SR network based on our newly developed IDR based Dynamic Convolution Residual Blocks (IDR-DCRB) to reconstruct the HR image with the guidance of IDR. For the training process, we first input HR and LR images to the teacher KD-IDE T , which is optimized with the SR network together. Given the paired HR and LR images, teacher KD-IDE T can easily extract the latent degradation information in LR images. Then, we use a student KD-IDE S to learn to extract the same IDR as that of KD-IDE T from LR images directly. Extensive experiments can demonstrate the effectiveness of the proposed KDSR. Our main contributions are threefold:• We propose KDSR, a strong, simple, and efficient baseline for Blind-SR, can generalize to any degradation process, which addresses the weakness of explicit degradation estimation. • We propose a novel KD based implicit degradation representation (IDR) estimator. To the best of our knowledge, the design of IDR estimation has received little attention so far. Besides, we propose an efficient IDR-based SR network to fully utilize IDR to guide SR. • Extensive experiments show that the proposed KDSR can achieve excellent Blind-SR performance in different degradation settings from simple to complex.

