Intelligent image scaling to 4x resolution. An official PyTorch implementation of the SRResCGAN model as described in the paper Deep Generative Adversarial Residual Convolutional Networks for Real-World Super-Resolution. This work is participated in the CVPRW NTIRE 2020 RWSR challenges on the Real-World Super-Resolution.

Abstract

Most current deep learning based single image super-resolution (SISR) methods focus on designing deeper / wider models to learn the non-linear mapping between low-resolution (LR) inputs and the high-resolution (HR) outputs from a large number of paired (LR/HR) training data. They usually take as assumption that the LR image is a bicubic down-sampled version of the HR image. However, such degradation process is not available in real-world settings i.e. inherent sensor noise, stochastic noise, compression artifacts, possible mismatch between image degradation process and camera device. It reduces significantly the performance of current SISR methods due to real-world image corruptions. To address these problems, we propose a deep Super-Resolution Residual Convolutional Generative Adversarial Network (SRResCGAN) to follow the real-world degradation settings by adversarial training the model with pixel-wise supervision in the HR domain from its generated LR counterpart. The proposed network exploits the residual learning by minimizing the energy-based objective function with powerful image regularization and convex optimization techniques. We demonstrate our proposed approach in quantitative and qualitative experiments that generalize robustly to real input and it is easy to deploy for other down-scaling operators and mobile/embedded devices.

Video demo

BibTeX

@InProceedings{Umer_2020_CVPR_Workshops,
    author = {Muhammad Umer, Rao and Luca Foresti, Gian and Micheloni, Christian},
    title = {Deep Generative Adversarial Residual Convolutional Networks for Real-World Super-Resolution},
    booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month = {June},
    year = {2020}
    }

SRResCGAN Architecture

Overall Representative diagram

SR Generator Network

Quantitative Results

The NTIRE2020 RWSR Challenge Results (Track-1)

Visual Results

Validation-set (Track-1)

You can download all the SR resutls of our method on the validation-set from Google Drive: SRResCGAN, SRResCGAN+.

Test-set (Track-1)

You can download all the SR resutls of our method on the test-set from Google Drive: SRResCGAN, SRResCGAN+.

Real-World Smartphone images (Track-2)

You can download all the SR resutls of our method on the smartphone images from Google Drive: SRResCGAN, SRResCGAN+.

Run with Docker

Given that you have a folder of low-resolution images in the folder ./input, the following command saves high-resolution results to the folder ./output.

GPU

This model requires an NVIDIA GPU, compatible with CUDA 11.0.

docker run -it --rm --gpus all \
    -v $PWD/input:/code/LR \
    -v $PWD/output:/code/sr_results_x4 \
    us-docker.pkg.dev/replicate/raoumer/srrescgan:gpu

CPU

docker run -it --rm \
    -v $PWD/input:/code/LR \
    -v $PWD/output:/code/sr_results_x4 \
    us-docker.pkg.dev/replicate/raoumer/srrescgan:cpu

Code Acknowledgement

The training and testing codes are somewhat based on ESRGAN, DSGAN, and deep_demosaick.