Model Details
Since the introduction of the Vision Transformer (ViT), researchers have sought to make ViTs more efficient by removing redundant information in the processed tokens. While different methods have been explored to achieve this goal, we still lack understanding of the resulting reduction patterns and how those patterns differ across token reduction methods and datasets. To close this gap, we set out to understand the reduction patterns of 10 different token reduction methods using four image classification datasets: ImageNet, NABirds, COCO, and NUS-WIDE.
We provide DeiT checkpoints (Tiny, Small, and Base) at four keep rates (0.9, 0.7, 0.5, and 0.25) for four classification datasets: ImageNet-1K, NABirds, COCO 2014, and NUS-WIDE.
Model Description
- Developed by: Joakim Bruslund Haurum, Sergio Escalera, Graham W. Taylor, and Thomas B. Moeslund
- Model type: Vision Transformers
- License: MIT License
More Resources
Model Zoo
Note: This repository does not host any checkpoints but contains links to all the model repositories. Each token reduction method repository contains the checkpoints for the four considered keep rates.
Baseline DeiT
| Model Name |
Dataset |
Weights |
| deit_base-nab |
NABirds |
link |
| deit_small-nab |
NABirds |
link |
| deit_tiny-nab |
NABirds |
link |
| deit_base-coco |
COCO 2014 |
link |
| deit_small-coco |
COCO 2014 |
link |
| deit_tiny-coco |
COCO 2014 |
link |
| deit_base-nus |
NUS-WIDE |
link |
| deit_small-nus |
NUS-WIDE |
link |
| deit_tiny-nus |
NUS-WIDE |
link |
Top-K:
| Model Name |
Dataset |
Weights |
| topk_base-im1k |
ImageNet-1K |
link |
| topk_small-im1k |
ImageNet-1K |
link |
| topk_tiny-im1k |
ImageNet-1K |
link |
| topk_base-nab |
NABirds |
link |
| topk_small-nab |
NABirds |
link |
| topk_tiny-nab |
NABirds |
link |
| topk_base-coco |
COCO 2014 |
link |
| topk_small-coco |
COCO 2014 |
link |
| topk_tiny-coco |
COCO 2014 |
link |
| topk_base-nus |
NUS-WIDE |
link |
| topk_small-nus |
NUS-WIDE |
link |
| topk_tiny-nus |
NUS-WIDE |
link |
EViT
| Model Name |
Dataset |
Weights |
| evit_base-im1k |
ImageNet-1K |
link |
| evit_small-im1k |
ImageNet-1K |
link |
| evit_tiny-im1k |
ImageNet-1K |
link |
| evit_base-nab |
NABirds |
link |
| evit_small-nab |
NABirds |
link |
| evit_tiny-nab |
NABirds |
link |
| evit_base-coco |
COCO 2014 |
link |
| evit_small-coco |
COCO 2014 |
link |
| evit_tiny-coco |
COCO 2014 |
link |
| evit_base-nus |
NUS-WIDE |
link |
| evit_small-nus |
NUS-WIDE |
link |
| evit_tiny-nus |
NUS-WIDE |
link |
DynamicViT
| Model Name |
Dataset |
Weights |
| dyvit_base-im1k |
ImageNet-1K |
link |
| dyvit_small-im1k |
ImageNet-1K |
link |
| dyvit_tiny-im1k |
ImageNet-1K |
link |
| dyvit_base-nab |
NABirds |
link |
| dyvit_small-nab |
NABirds |
link |
| dyvit_tiny-nab |
NABirds |
link |
| dyvit_base-coco |
COCO 2014 |
link |
| dyvit_small-coco |
COCO 2014 |
link |
| dyvit_tiny-coco |
COCO 2014 |
link |
| dyvit_base-nus |
NUS-WIDE |
link |
| dyvit_small-nus |
NUS-WIDE |
link |
| dyvit_tiny-nus |
NUS-WIDE |
link |
ATS
| Model Name |
Dataset |
Weights |
| ats_base-im1k |
ImageNet-1K |
link |
| ats_small-im1k |
ImageNet-1K |
link |
| ats_tiny-im1k |
ImageNet-1K |
link |
| ats_base-nab |
NABirds |
link |
| ats_small-nab |
NABirds |
link |
| ats_tiny-nab |
NABirds |
link |
| ats_base-coco |
COCO 2014 |
link |
| ats_small-coco |
COCO 2014 |
link |
| ats_tiny-coco |
COCO 2014 |
link |
| ats_base-nus |
NUS-WIDE |
link |
| ats_small-nus |
NUS-WIDE |
link |
| ats_tiny-nus |
NUS-WIDE |
link |
L1
| Model Name |
Dataset |
Weights |
| l1_base-im1k |
ImageNet-1K |
link |
| l1_small-im1k |
ImageNet-1K |
link |
| l1_tiny-im1k |
ImageNet-1K |
link |
| l1_base-nab |
NABirds |
link |
| l1_small-nab |
NABirds |
link |
| l1_tiny-nab |
NABirds |
link |
| l1_base-coco |
COCO 2014 |
link |
| l1_small-coco |
COCO 2014 |
link |
| l1_tiny-coco |
COCO 2014 |
link |
| l1_base-nus |
NUS-WIDE |
link |
| l1_small-nus |
NUS-WIDE |
link |
| l1_tiny-nus |
NUS-WIDE |
link |
L2
| Model Name |
Dataset |
Weights |
| l2_base-im1k |
ImageNet-1K |
link |
| l2_small-im1k |
ImageNet-1K |
link |
| l2_tiny-im1k |
ImageNet-1K |
link |
| l2_base-nab |
NABirds |
link |
| l2_small-nab |
NABirds |
link |
| l2_tiny-nab |
NABirds |
link |
| l2_base-coco |
COCO 2014 |
link |
| l2_small-coco |
COCO 2014 |
link |
| l2_tiny-coco |
COCO 2014 |
link |
| l2_base-nus |
NUS-WIDE |
link |
| l2_small-nus |
NUS-WIDE |
link |
| l2_tiny-nus |
NUS-WIDE |
link |
L-Infinity
| Model Name |
Dataset |
Weights |
| linf_base-im1k |
ImageNet-1K |
link |
| linf_small-im1k |
ImageNet-1K |
link |
| linf_tiny-im1k |
ImageNet-1K |
link |
| linf_base-nab |
NABirds |
link |
| linf_small-nab |
NABirds |
link |
| linf_tiny-nab |
NABirds |
link |
| linf_base-coco |
COCO 2014 |
link |
| linf_small-coco |
COCO 2014 |
link |
| linf_tiny-coco |
COCO 2014 |
link |
| linf_base-nus |
NUS-WIDE |
link |
| linf_small-nus |
NUS-WIDE |
link |
| linf_tiny-nus |
NUS-WIDE |
link |
ToMe
| Model Name |
Dataset |
Weights |
| tome_base-im1k |
ImageNet-1K |
link |
| tome_small-im1k |
ImageNet-1K |
link |
| tome_tiny-im1k |
ImageNet-1K |
link |
| tome_base-nab |
NABirds |
link |
| tome_small-nab |
NABirds |
link |
| tome_tiny-nab |
NABirds |
link |
| tome_base-coco |
COCO 2014 |
link |
| tome_small-coco |
COCO 2014 |
link |
| tome_tiny-coco |
COCO 2014 |
link |
| tome_base-nus |
NUS-WIDE |
link |
| tome_small-nus |
NUS-WIDE |
link |
| tome_tiny-nus |
NUS-WIDE |
link |
K-Medoids
| Model Name |
Dataset |
Weights |
| kmedoids_base-im1k |
ImageNet-1K |
link |
| kmedoids-small_im1k |
ImageNet-1K |
link |
| kmedoids_tiny-im1k |
ImageNet-1K |
link |
| kmedoids_base-nab |
NABirds |
link |
| kmedoids-small_nab |
NABirds |
link |
| kmedoids_tiny-nab |
NABirds |
link |
| kmedoids_base-coco |
COCO 2014 |
link |
| kmedoids-small_coco |
COCO 2014 |
link |
| kmedoids_tiny-coco |
COCO 2014 |
link |
| kmedoids_base-nus |
NUS-WIDE |
link |
| kmedoids-small_nus |
NUS-WIDE |
link |
| kmedoids_tiny-nus |
NUS-WIDE |
link |
DPC-KNN
| Model Name |
Dataset |
Weights |
| dpcknn_base-im1k |
ImageNet-1K |
link |
| dpcknn_small_im1k |
ImageNet-1K |
link |
| dpcknn_tiny-im1k |
ImageNet-1K |
link |
| dpcknn_base-nab |
NABirds |
link |
| dpcknn_small_nab |
NABirds |
link |
| dpcknn_tiny-nab |
NABirds |
link |
| dpcknn_base-coco |
COCO 2014 |
link |
| dpcknn_small_coco |
COCO 2014 |
link |
| dpcknn_tiny-coco |
COCO 2014 |
link |
| dpcknn_base-nus |
NUS-WIDE |
link |
| dpcknn_small_nus |
NUS-WIDE |
link |
| dpcknn_tiny-nus |
NUS-WIDE |
link |
SiT
| Model Name |
Dataset |
Weights |
| sit_base-im1k |
ImageNet-1K |
link |
| sit_small_im1k |
ImageNet-1K |
link |
| sit_tiny-im1k |
ImageNet-1K |
link |
| sit_base-nab |
NABirds |
link |
| sit_small_nab |
NABirds |
link |
| sit_tiny-nab |
NABirds |
link |
| sit_base-coco |
COCO 2014 |
link |
| sit_small_coco |
COCO 2014 |
link |
| sit_tiny-coco |
COCO 2014 |
link |
| sit_base-nus |
NUS-WIDE |
link |
| sit_small_nus |
NUS-WIDE |
link |
| sit_tiny-nus |
NUS-WIDE |
link |
PatchMerger
| Model Name |
Dataset |
Weights |
| patchmerger_base-im1k |
ImageNet-1K |
link |
| patchmerger_small_im1k |
ImageNet-1K |
link |
| patchmerger_tiny-im1k |
ImageNet-1K |
link |
| patchmerger_base-nab |
NABirds |
link |
| patchmerger_small_nab |
NABirds |
link |
| patchmerger_tiny-nab |
NABirds |
link |
| patchmerger_base-coco |
COCO 2014 |
link |
| patchmerger_small_coco |
COCO 2014 |
link |
| patchmerger_tiny-coco |
COCO 2014 |
link |
| patchmerger_base-nus |
NUS-WIDE |
link |
| patchmerger_small_nus |
NUS-WIDE |
link |
| patchmerger_tiny-nus |
NUS-WIDE |
link |
Sinkhorn
| Model Name |
Dataset |
Weights |
| sinkhorn_base-im1k |
ImageNet-1K |
link |
| sinkhorn_small_im1k |
ImageNet-1K |
link |
| sinkhorn_tiny-im1k |
ImageNet-1K |
link |
| sinkhorn_base-nab |
NABirds |
link |
| sinkhorn_small_nab |
NABirds |
link |
| sinkhorn_tiny-nab |
NABirds |
link |
| sinkhorn_base-coco |
COCO 2014 |
link |
| sinkhorn_small_coco |
COCO 2014 |
link |
| sinkhorn_tiny-coco |
COCO 2014 |
link |
| sinkhorn_base-nus |
NUS-WIDE |
link |
| sinkhorn_small_nus |
NUS-WIDE |
link |
| sinkhorn_tiny-nus |
NUS-WIDE |
link |
