Eugene Siow
commited on
Commit
•
24585ac
1
Parent(s):
ca3844d
Add update to README with datasets training recipe.
Browse files
README.md
CHANGED
@@ -5,6 +5,10 @@ tags:
|
|
5 |
- image-super-resolution
|
6 |
datasets:
|
7 |
- eugenesiow/Div2k
|
|
|
|
|
|
|
|
|
8 |
metrics:
|
9 |
- pnsr
|
10 |
- ssim
|
@@ -42,8 +46,9 @@ preds = model(inputs)
|
|
42 |
ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
|
43 |
ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
|
44 |
```
|
|
|
45 |
## Training data
|
46 |
-
The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://
|
47 |
## Training procedure
|
48 |
### Preprocessing
|
49 |
We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
|
@@ -51,24 +56,20 @@ Low Resolution (LR) images are created by using bicubic interpolation as the res
|
|
51 |
During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
|
52 |
Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
|
53 |
|
54 |
-
|
|
|
|
|
|
|
|
|
|
|
55 |
```python
|
56 |
-
from
|
|
|
57 |
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
do_augmentation=True
|
63 |
-
)
|
64 |
-
DatasetBuilder.prepare(
|
65 |
-
base_path='./DIV2K/DIV2K_val_HR',
|
66 |
-
output_path='./div2k_4x_val.h5',
|
67 |
-
scale=4,
|
68 |
-
do_augmentation=False
|
69 |
-
)
|
70 |
-
train_dataset = TrainAugmentDataset('./div2k_4x_train.h5', scale=4)
|
71 |
-
val_dataset = EvalDataset('./div2k_4x_val.h5')
|
72 |
```
|
73 |
### Pretraining
|
74 |
The model was trained on GPU. The training code is provided below:
|
@@ -90,19 +91,21 @@ trainer = Trainer(
|
|
90 |
model=model, # the instantiated model to be trained
|
91 |
args=training_args, # training arguments, defined above
|
92 |
train_dataset=train_dataset, # training dataset
|
93 |
-
eval_dataset=
|
94 |
)
|
95 |
|
96 |
trainer.train()
|
97 |
```
|
|
|
|
|
98 |
## Evaluation results
|
99 |
The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
|
100 |
|
101 |
Evaluation datasets include:
|
102 |
-
- Set5 - [Bevilacqua et al. (2012)](
|
103 |
-
- Set14 - [Zeyde et al. (2010)](https://
|
104 |
-
- BSD100 - [Martin et al. (2001)](https://
|
105 |
-
- Urban100 - [Huang et al. (2015)](https://
|
106 |
|
107 |
The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
|
108 |
|
@@ -122,6 +125,9 @@ The results columns below are represented below as `PSNR/SSIM`. They are compare
|
|
122 |
|Urban100 |4x |23.14/0.6573 |**26.10/0.7857** |
|
123 |
|
124 |
![Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2](images/msrn_2_4_compare.png "Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2")
|
|
|
|
|
|
|
125 |
|
126 |
## BibTeX entry and citation info
|
127 |
```bibtex
|
|
|
5 |
- image-super-resolution
|
6 |
datasets:
|
7 |
- eugenesiow/Div2k
|
8 |
+
- eugenesiow/Set5
|
9 |
+
- eugenesiow/Set14
|
10 |
+
- eugenesiow/BSD100
|
11 |
+
- eugenesiow/Urban100
|
12 |
metrics:
|
13 |
- pnsr
|
14 |
- ssim
|
|
|
46 |
ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
|
47 |
ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
|
48 |
```
|
49 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Upscale_Images_with_Pretrained_super_image_Models.ipynb "Open in Colab")
|
50 |
## Training data
|
51 |
+
The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://huggingface.co/datasets/eugenesiow/Div2k), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
|
52 |
## Training procedure
|
53 |
### Preprocessing
|
54 |
We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
|
|
|
56 |
During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
|
57 |
Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
|
58 |
|
59 |
+
We need the huggingface [datasets](https://huggingface.co/datasets?filter=task_ids:other-other-image-super-resolution) library to download the data:
|
60 |
+
```bash
|
61 |
+
pip install datasets
|
62 |
+
```
|
63 |
+
The following code gets the data and preprocesses/augments the data.
|
64 |
+
|
65 |
```python
|
66 |
+
from datasets import load_dataset
|
67 |
+
from super_image.data import EvalDataset, TrainDataset, augment_five_crop
|
68 |
|
69 |
+
augmented_dataset = load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='train')\
|
70 |
+
.map(augment_five_crop, batched=True, desc="Augmenting Dataset") # download and augment the data with the five_crop method
|
71 |
+
train_dataset = TrainDataset(augmented_dataset) # prepare the train dataset for loading PyTorch DataLoader
|
72 |
+
eval_dataset = EvalDataset(load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='validation')) # prepare the eval dataset for the PyTorch DataLoader
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
```
|
74 |
### Pretraining
|
75 |
The model was trained on GPU. The training code is provided below:
|
|
|
91 |
model=model, # the instantiated model to be trained
|
92 |
args=training_args, # training arguments, defined above
|
93 |
train_dataset=train_dataset, # training dataset
|
94 |
+
eval_dataset=eval_dataset # evaluation dataset
|
95 |
)
|
96 |
|
97 |
trainer.train()
|
98 |
```
|
99 |
+
|
100 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Train_super_image_Models.ipynb "Open in Colab")
|
101 |
## Evaluation results
|
102 |
The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
|
103 |
|
104 |
Evaluation datasets include:
|
105 |
+
- Set5 - [Bevilacqua et al. (2012)](https://huggingface.co/datasets/eugenesiow/Set5)
|
106 |
+
- Set14 - [Zeyde et al. (2010)](https://huggingface.co/datasets/eugenesiow/Set14)
|
107 |
+
- BSD100 - [Martin et al. (2001)](https://huggingface.co/datasets/eugenesiow/BSD100)
|
108 |
+
- Urban100 - [Huang et al. (2015)](https://huggingface.co/datasets/eugenesiow/Urban100)
|
109 |
|
110 |
The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
|
111 |
|
|
|
125 |
|Urban100 |4x |23.14/0.6573 |**26.10/0.7857** |
|
126 |
|
127 |
![Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2](images/msrn_2_4_compare.png "Comparing Bicubic upscaling against the models x4 upscaling on Set5 Image 2")
|
128 |
+
You can find a notebook to easily run evaluation on pretrained models below:
|
129 |
+
|
130 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Evaluate_Pretrained_super_image_Models.ipynb "Open in Colab")
|
131 |
|
132 |
## BibTeX entry and citation info
|
133 |
```bibtex
|