

TITLE
Longformer2Roberta: global_attention_mask is never used

COMMENTS
6

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
I was following the Longformer2Roberta tutorial https://github.com/huggingface/transformers/blob/master/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md, it seems like 'global_attention_mask' is never used because this column is removed after https://github.com/huggingface/transformers/blob/9bdce3a4f91c6d53873582b0210e61c92bba8fd3/src/transformers/trainer.py#L301
so you have to add this column to the signature (whatever it is)
https://github.com/huggingface/transformers/blob/9bdce3a4f91c6d53873582b0210e61c92bba8fd3/src/transformers/trainer.py#L324
or set `remove_unused_columns=False` in TrainingArguments 

@patrickvonplaten
