

TITLE
Running seq2seq_trainer with iterable datasets 

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Hi
Looking into https://github.com/huggingface/transformers/blob/master/examples/seq2seq/seq2seq_trainer.py line 117, if the dataset is iterative, then the code does not return a distributed sampler so in case one wants to train on TPUs on multiple cores with iterable datasets, I was wondering if you could assist me how I can use the codes for iterable datasets on TPUs
thanks