Eugene Siow commited on
Commit
24585ac
1 Parent(s): ca3844d

Add update to README with datasets training recipe.

Browse files
Files changed (1) hide show
  1. README.md +28 -22
README.md CHANGED
@@ -5,6 +5,10 @@ tags:
5
  - image-super-resolution
6
  datasets:
7
  - eugenesiow/Div2k
 
 
 
 
8
  metrics:
9
  - pnsr
10
  - ssim
@@ -42,8 +46,9 @@ preds = model(inputs)
42
  ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
43
  ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
44
  ```
 
45
  ## Training data
46
- The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
47
  ## Training procedure
48
  ### Preprocessing
49
  We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
@@ -51,24 +56,20 @@ Low Resolution (LR) images are created by using bicubic interpolation as the res
51
  During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
52
  Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
53
 
54
- The following code provides some helper functions to preprocess the data.
 
 
 
 
 
55
  ```python
56
- from super_image.data import EvalDataset, TrainAugmentDataset, DatasetBuilder
 
57
 
58
- DatasetBuilder.prepare(
59
- base_path='./DIV2K/DIV2K_train_HR',
60
- output_path='./div2k_4x_train.h5',
61
- scale=4,
62
- do_augmentation=True
63
- )
64
- DatasetBuilder.prepare(
65
- base_path='./DIV2K/DIV2K_val_HR',
66
- output_path='./div2k_4x_val.h5',
67
- scale=4,
68
- do_augmentation=False
69
- )
70
- train_dataset = TrainAugmentDataset('./div2k_4x_train.h5', scale=4)
71
- val_dataset = EvalDataset('./div2k_4x_val.h5')
72
  ```
73
  ### Pretraining
74
  The model was trained on GPU. The training code is provided below:
@@ -90,19 +91,21 @@ trainer = Trainer(
90
  model=model, # the instantiated model to be trained
91
  args=training_args, # training arguments, defined above
92
  train_dataset=train_dataset, # training dataset
93
- eval_dataset=val_dataset # evaluation dataset
94
  )
95
 
96
  trainer.train()
97
  ```
 
 
98
  ## Evaluation results
99
  The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
100
 
101
  Evaluation datasets include:
102
- - Set5 - [Bevilacqua et al. (2012)](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html)
103
- - Set14 - [Zeyde et al. (2010)](https://sites.google.com/site/romanzeyde/research-interests)
104
- - BSD100 - [Martin et al. (2001)](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
105
- - Urban100 - [Huang et al. (2015)](https://sites.google.com/site/jbhuang0604/publications/struct_sr)
106
 
107
  The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
108
 
@@ -122,6 +125,9 @@ The results columns below are represented below as `PSNR/SSIM`. They are compare
122
  |Urban100 |4x |23.14/0.6573 |**26.10/0.7857** |
123
 
124
  ![Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2](images/msrn_2_4_compare.png "Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2")
 
 
 
125
 
126
  ## BibTeX entry and citation info
127
  ```bibtex
 
5
  - image-super-resolution
6
  datasets:
7
  - eugenesiow/Div2k
8
+ - eugenesiow/Set5
9
+ - eugenesiow/Set14
10
+ - eugenesiow/BSD100
11
+ - eugenesiow/Urban100
12
  metrics:
13
  - pnsr
14
  - ssim
 
46
  ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
47
  ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
48
  ```
49
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Upscale_Images_with_Pretrained_super_image_Models.ipynb "Open in Colab")
50
  ## Training data
51
+ The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://huggingface.co/datasets/eugenesiow/Div2k), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
52
  ## Training procedure
53
  ### Preprocessing
54
  We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
 
56
  During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
57
  Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
58
 
59
+ We need the huggingface [datasets](https://huggingface.co/datasets?filter=task_ids:other-other-image-super-resolution) library to download the data:
60
+ ```bash
61
+ pip install datasets
62
+ ```
63
+ The following code gets the data and preprocesses/augments the data.
64
+
65
  ```python
66
+ from datasets import load_dataset
67
+ from super_image.data import EvalDataset, TrainDataset, augment_five_crop
68
 
69
+ augmented_dataset = load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='train')\
70
+ .map(augment_five_crop, batched=True, desc="Augmenting Dataset") # download and augment the data with the five_crop method
71
+ train_dataset = TrainDataset(augmented_dataset) # prepare the train dataset for loading PyTorch DataLoader
72
+ eval_dataset = EvalDataset(load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='validation')) # prepare the eval dataset for the PyTorch DataLoader
 
 
 
 
 
 
 
 
 
 
73
  ```
74
  ### Pretraining
75
  The model was trained on GPU. The training code is provided below:
 
91
  model=model, # the instantiated model to be trained
92
  args=training_args, # training arguments, defined above
93
  train_dataset=train_dataset, # training dataset
94
+ eval_dataset=eval_dataset # evaluation dataset
95
  )
96
 
97
  trainer.train()
98
  ```
99
+
100
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Train_super_image_Models.ipynb "Open in Colab")
101
  ## Evaluation results
102
  The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
103
 
104
  Evaluation datasets include:
105
+ - Set5 - [Bevilacqua et al. (2012)](https://huggingface.co/datasets/eugenesiow/Set5)
106
+ - Set14 - [Zeyde et al. (2010)](https://huggingface.co/datasets/eugenesiow/Set14)
107
+ - BSD100 - [Martin et al. (2001)](https://huggingface.co/datasets/eugenesiow/BSD100)
108
+ - Urban100 - [Huang et al. (2015)](https://huggingface.co/datasets/eugenesiow/Urban100)
109
 
110
  The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
111
 
 
125
  |Urban100 |4x |23.14/0.6573 |**26.10/0.7857** |
126
 
127
  ![Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2](images/msrn_2_4_compare.png "Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2")
128
+ You can find a notebook to easily run evaluation on pretrained models below:
129
+
130
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Evaluate_Pretrained_super_image_Models.ipynb "Open in Colab")
131
 
132
  ## BibTeX entry and citation info
133
  ```bibtex