Skip to content

A PyTorch implementation of ESPCN based on CVPR 2016 paper "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network"

Notifications You must be signed in to change notification settings

leftthomas/ESPCN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

53 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Super Resolution

A PyTorch implementation of ESPCN based on CVPR2016 paper Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Requirements

conda install pytorch torchvision -c soumith
conda install pytorch torchvision cuda80 -c soumith # install it if you have installed cuda
  • PyTorchNet
pip install git+https://github.com/pytorch/tnt.git@master
  • tqdm
pip install tqdm

Datasets

Train、Val Dataset

The train and val datasets are sampled from VOC2012. Train dataset has 16700 images and Val dataset has 425 images. Download the datasets from here, and then extract it into data directory. Finally run

python data_utils.py

optional arguments:
--upscale_factor      super resolution upscale factor [default value is 3]

to generate train and val datasets from VOC2012 with given upscale factors(options: 2、3、4、8).

Test Dataset

The test dataset are sampled from | Set 5 | Bevilacqua et al. BMVC 2012 | Set 14 | Zeyde et al. LNCS 2010 | BSD 100 | Martin et al. ICCV 2001 | Sun-Hays 80 | Sun and Hays ICCP 2012 | Urban 100 | Huang et al. CVPR 2015. Download the dataset from here, and then extract it into data directory.

Usage

Train

python -m visdom.server & python train.py

optional arguments:
--upscale_factor      super resolution upscale factor [default value is 3]
--num_epochs          super resolution epochs number [default value is 100]

Visdom now can be accessed by going to 127.0.0.1:8097 in your browser, or your own host address if specified.

If the above does not work, try using an SSH tunnel to your server by adding the following line to your local ~/.ssh/config : LocalForward 127.0.0.1:8097 127.0.0.1:8097.

Maybe if you are in China, you should download the static resources from here, and put them on ~/anaconda3/lib/python3.6/site-packages/visdom/static/.

Test

python test.py

optional arguments:
--upscale_factor      super resolution upscale factor [default value is 3]
--model_name          super resolution model name [default value is epoch_3_100.pt]

The output high resolution images are on results directory.

Benchmarks

Adam optimizer were used with learning rate scheduling between epoch 30 and epoch 80.

Upscale Factor = 2

Epochs with batch size of 64 takes ~1 minute on a NVIDIA GeForce GTX TITAN X GPU.

Loss/PSNR graphs

Results

The left is low resolution image, the middle is high resolution image, and the right is super resolution image(output of the ESPCN).

  • Set5
  • Set14
  • BSD100
  • Urban100

Upscale Factor = 3

Epochs with batch size of 64 takes ~30 seconds on a NVIDIA GeForce GTX TITAN X GPU.

Loss/PSNR graphs

Results

The left is low resolution image, the middle is high resolution image, and the right is super resolution image(output of the ESPCN).

  • Set5
  • Set14
  • BSD100

Upscale Factor = 4

Epochs with batch size of 64 takes ~20 seconds on a NVIDIA GeForce GTX 1070 GPU.

Loss/PSNR graphs

Results

The left is low resolution image, the middle is high resolution image, and the right is super resolution image(output of the ESPCN).

  • Set5
  • Set14
  • BSD100
  • Urban100

About

A PyTorch implementation of ESPCN based on CVPR 2016 paper "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network"

Topics

Resources

Stars

Watchers

Forks

Languages