Recent state-of-the-art super-resolution methods have achieved impressive performance on ideal datasets regardless of blur and noise. However, these methods always fail in real-world image super-resolution, since most of them adopt simple bicubic downsampling from high-quality images to construct Low-Resolution (LR) and High-Resolution (HR) pairs for training which may lose track of frequency-related details. To address this issue, we focus on designing a novel degradation framework for real-world images by estimating various blur kernels as well as real noise distributions. Based on our novel degradation framework, we can acquire LR images sharing a common domain with real-world images. Then, we propose a real-world super-resolution model aiming at better perception. Extensive experiments on synthetic noise data and real-world images demonstrate that our method outperforms the state-of-the-art methods, resulting in lower noise and better visual quality. In addition, our method is the winner of NTIRE 2020 Challenge on both tracks of Real-World Super-Resolution, which significantly outperforms other competitors by large margins.
If you are interested in this work, please cite our paper
@InProceedings{Ji_2020_CVPR_Workshops,
author = {Ji, Xiaozhong and Cao, Yun and Tai, Ying and Wang, Chengjie and Li, Jilin and Huang, Feiyue},
title = {Real-World Super-Resolution via Kernel Estimation and Noise Injection},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2020}
}
RealSR
Real-World Super-Resolution via Kernel Estimation and Noise Injection (CVPR 2020 Workshops)
Xiaozhong Ji, Yun Cao, Ying Tai, Chengjie Wang, Jilin Li, and Feiyue Huang
Tencent Youtu Lab
Our solution is the winner of CVPR NTIRE 2020 Challenge on Real-World Super-Resolution in both tracks.
(Official PyTorch Implementation)
Update - September 1, 2020
Update - May 26, 2020
./realsr-ncnn-vulkan -i in.jpg -o out.png-x- use ensemble-g 0- select gpu id.Introduction
Recent state-of-the-art super-resolution methods have achieved impressive performance on ideal datasets regardless of blur and noise. However, these methods always fail in real-world image super-resolution, since most of them adopt simple bicubic downsampling from high-quality images to construct Low-Resolution (LR) and High-Resolution (HR) pairs for training which may lose track of frequency-related details. To address this issue, we focus on designing a novel degradation framework for real-world images by estimating various blur kernels as well as real noise distributions. Based on our novel degradation framework, we can acquire LR images sharing a common domain with real-world images. Then, we propose a real-world super-resolution model aiming at better perception. Extensive experiments on synthetic noise data and real-world images demonstrate that our method outperforms the state-of-the-art methods, resulting in lower noise and better visual quality. In addition, our method is the winner of NTIRE 2020 Challenge on both tracks of Real-World Super-Resolution, which significantly outperforms other competitors by large margins.
If you are interested in this work, please cite our paper
and challenge report NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results
Visual Results
Quantitative Results Compared with Other Participating Methods
‘Impressionism’ is our team. Note that the final decision is based on MOS (Mean Opinion Score) and MOR (Mean Opinion Rank).
Qualitative Results Compared with Other Participating Methods
‘Impressionism’ is our team.
Dependencies and Installation
This code is based on BasicSR.
pip install numpy opencv-python lmdb pyyamlpip install tb-nightly futurepip install tensorboardXPre-trained models
Testing
Download dataset from NTIRE 2020 RWSR and unzip it to your path.
For convenient, we provide Corrupted-te-x and DPEDiphone-crop-te-x.
cd ./codesDF2K: Image processing artifacts
CUDA_VISIBLE_DEVICES=X python3 test.py -opt options/df2k/test_df2k.ymlDPED: Smartphone images
CUDA_VISIBLE_DEVICES=X python3 test.py -opt options/dped/test_dped.ymlTraining
Track 1
prepare training data
specify dataset paths in ‘./preprocess/path.yml’ and create bicubic dataset :
python3 ./preprocess/create_bicubic_dataset.py --dataset df2k --artifacts tdsrrun the below command to collect high frequency noise from Source :
python3 ./preprocess/collect_noise.py --dataset df2k --artifacts tdsrtrain SR model
CUDA_VISIBLE_DEVICES=4,5,6,7 python3 train.py -opt options/df2k/train_bicubic_noise.ymlTrack 2
prepare training data
Use KernelGAN to generate kernels from source images. Clone the repo here. Replace SOURCE_PATH with specific path and run :
specify dataset paths in ‘./preprocess/path.yml’ and generated KERNEL_PATH to kernel create kernel dataset:
python3 ./preprocess/create_kernel_dataset.py --dataset dped --artifacts clean --kernel_path KERNEL_PATHrun the below command to collect high frequency noise from Source:
python3 ./preprocess/collect_noise.py --dataset dped --artifacts cleantrain SR model
CUDA_VISIBLE_DEVICES=4,5,6,7 python3 train.py -opt options/dped/train_kernel_noise.yml