

TITLE
Rework LongFormer to make it compatible with ONNX

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
- [x] Remove implicit `bool` -> `int` conversion while padding attention_mask with `False` value.
- [x] Remove call to `einsum` where `matmul` + `transpose` can be used (makes optimizations easier for ONNX)
- [ ] Diagonal matrix computation try to avoid ScatterND with negative steps / index
- [ ] Use `torch.div(a, b, rounding_mode='trunc')` instead of `floor_divide` 
- [ ] Check [function](https://github.com/huggingface/transformers/blob/master/src/transformers/models/longformer/modeling_longformer.py#L783) reporting `torch.Tensor` return type which doesn't return anything