

TITLE
[Transfo-XL] Impossible to pass `attention_mask` to model

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
It is not possible to forward an `attention_mask` to transfo-xl because Transfo-XL's forward function does not accept an `attention_mask`. This makes it impossible to do batch generation with Transfo-XL for example.

IMO, this could be implemented quite easily.