Computer Laboratory

Projects

Teaser figure
An illustration of our semi-supervised training scheme using the Transformation Consistency Regularization (TCR) for image colorization. In a semi-supervised setting we have a small set of labeled data i.e. image pairs (Ds) and a proportionally large set of unlabeled data (Dus). We in this work introduce diverse set of geometric transformations over the unlabeled data forcing the model's predictions to be invarient to the transformations. This allows us to leverage unsupervised data for the model training alongside the conventional Image to Image (I2I) supervision. The same method is used for Image Denoising and Single Image Super Resolution.

Abstract

Scarcity of labeled data has motivated the development of semi-supervised learning methods, which learn from large portions of unlabeled data alongside a few labeled samples. Consistency Regularization between model's predictions under different input perturbations, particularly has shown to provide state-of-the art results in a semi-supervised framework. However, most of these method have been limited to classification and segmentation applications. We propose Transformation Consistency Regularization, which delves into a more challenging setting of image-to-image translation, which remains unexplored by semi-supervised algorithms. The method introduces a diverse set of geometric transformations and enforces the model's predictions for unlabeled data to be invariant to those transformations. We evaluate the efficacy of our algorithm on three different applications: image colorization, denoising and super-resolution. Our method is significantly data efficient, requiring only around 10 - 20 % of labeled samples to achieve similar image reconstructions to its fully-supervised counterpart. Furthermore, we show the effectiveness of our method in video processing applications, where knowledge from a few frames can be leveraged to enhance the quality of the rest of the movie.

Video

Downloads

Results

Publication

Aamir Mustafa and Rafał K. Mantiuk. Transformation Consistency Regularization - A Semi Supervised Paradigm for Image-to-Image Translation. European Conference on Computer Vision (ECCV), 2020

Contact

Please contact Aamir Mustafa or Rafał K. Mantiuk with any questions regarding the method.

Related projects

Acknowledgement

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement N◦ 725253–EyeCode).