

TITLE
[WIP] FSMT bart-like refactor

COMMENTS
4

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# What does this PR do?

This PR refactors `FSMT` to align it with other (bart-like) seq-2-seq models in the lib.

This PR refactors `FSMT` similar to `Bart` in that it moves the time dimension to be always at the 2nd place and the batch dimensions always in the first place. Also, the cache is refactored to consists of `tuples` instead of a `dict`. 

This refactor is very similar to  #10501.

I have verified that all slow-tets are passing and that all metrics (BLEU score) can be reproduced.  I ran the evaluation of the following four models and the results are similar to those reported in the [model cards](https://huggingface.co/facebook/wmt19-en-ru).
- en-ru: 33.42
- ru-en: 39.20
- en-de: 42.83
- de-en: 41.39

### Benchmarking

This PR however introduces some speed and memory regression, which I'm currently investigating.

On this PR:
```
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length     Time in s   
--------------------------------------------------------------------------------
     facebook/wmt19-en-ru            4               8             0.009     
     facebook/wmt19-en-ru            4               32             0.01     
     facebook/wmt19-en-ru            4              128            0.026     
     facebook/wmt19-en-ru            4              512            0.109     
--------------------------------------------------------------------------------

====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length    Memory in MB 
--------------------------------------------------------------------------------
     facebook/wmt19-en-ru            4               8              2172     
     facebook/wmt19-en-ru            4               32             2200     
     facebook/wmt19-en-ru            4              128             2306     
     facebook/wmt19-en-ru            4              512             2792     
--------------------------------------------------------------------------------
```

On master:

```
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length     Time in s   
--------------------------------------------------------------------------------
     facebook/wmt19-en-ru            4               8             0.007     
     facebook/wmt19-en-ru            4               32            0.007     
     facebook/wmt19-en-ru            4              128            0.013     
     facebook/wmt19-en-ru            4              512            0.046     
--------------------------------------------------------------------------------

====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length    Memory in MB 
--------------------------------------------------------------------------------
     facebook/wmt19-en-ru            4               8              2170     
     facebook/wmt19-en-ru            4               32             2176     
     facebook/wmt19-en-ru            4              128             2204     
     facebook/wmt19-en-ru            4              512             2356     
--------------------------------------------------------------------------------


```
