

TITLE
[New Model] DocFormer: End-to-End Transformer for Document Understanding

COMMENTS
5

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🌟 New model addition

## Model description

See _"DocFormer: End-to-End Transformer for Document Understanding", Appalaraju et al (ICCV 2021)_ on [CVF](https://openaccess.thecvf.com/content/ICCV2021/papers/Appalaraju_DocFormer_End-to-End_Transformer_for_Document_Understanding_ICCV_2021_paper.pdf) and [arXiv](https://arxiv.org/abs/2106.11539)

DocFormer is a multi-modal transformer model for 2D/visual documents from Amazon (where, fair disclosure, I also currently work but not in research) - which I would characterize at a high level as being broadly along the same use cases as LayoutLMv2 (already in `transformers`), but achieving better (state-of-the-art) results with smaller datasets per the benchmarks in the paper. 

I've found this kind of multi-modal, spatial/linguistic model very useful in the past (actually released an [AWS sample](https://github.com/aws-samples/amazon-textract-transformer-pipeline) and [blog post](https://aws.amazon.com/blogs/machine-learning/bring-structure-to-diverse-documents-with-amazon-textract-and-transformer-based-models-on-amazon-sagemaker/) with Hugging Face LayoutLMv1 earlier this year) and would love the improvements from DocFormer could be available through HF Transformers.

## Open source status

* [X] the model implementation is available: (give details)
    * Looks like there's an (MIT-0) implementation at https://github.com/shabie/docformer
* [ ] the model weights are available: (give details)
    * Not currently as far as I can tell? 
* [X] who are the authors: (mention them, if possible by @gh-username)
    * Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha - all of AWS AI. Not sure of GitHub usernames
    * @shabie for the currently available implementation