

TITLE
Add PoolFormer Model

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 3
heart: 0
rocket: 0
eyes: 0

BODY
# 🌟 New model addition
I would like to add the recently announced [PoolFormer](https://github.com/sail-sg/poolformer) model to the Transformers library.

## Model description

PoolFormer model was proposed in the paper, “**MetaFormer is Actually What You Need for Vision**” by Sea AI Lab and the main argument behind this is that performance of transformer/MLP-like models primarily comes from the general architecture instead of the specific token mixers they are using (like Attention for example).

To show this, they have added a basic non-parametric Pooling operator which does basic token mixing. This suggested model outperforms DeiT and ResMLP.

Link to the Model Repo: [PoolFormer](https://github.com/sail-sg/poolformer)
Link to the Paper: [MetaFormer is Actually What You Need for Vision](https://arxiv.org/abs/2111.11418)

<!-- Important information -->

## Open-source status

* [x] the model implementation is available: In PyTorch on PoolFormer [repository](https://github.com/sail-sg/poolformer)
* [x] the model weights are available: In the same repository, [link](https://github.com/sail-sg/poolformer#2-poolformer-models)
* [x] who are the authors: [Sea AI Lab](https://github.com/sail-sg)
