

TITLE
[Trainer] add --max_train_samples  --max_val_samples  --max_test_samples 

COMMENTS
23

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
As we were planning to add `--max_train_samples  --max_val_samples  --max_test_samples`  to all examples https://github.com/huggingface/transformers/issues/10423, I thought is there any reason why we don't expand the Trainer to handle that?

It surely would be useful to be able to truncate the dataset at the point of Trainer to enable quick testing.

Another plus is that the metrics can then automatically include the actual number of samples run, rather than how it is done at the moment in examples.

That way this functionality would be built-in and examples will get it for free.

TODO:
1. [ ] port  `--max_train_samples  --max_val_samples  --max_test_samples`  to Trainer and remove the then unneeded code in `run_seq2seq.py`
2. [ ] extend metrics to report  the number of samples as it's done now in:

https://github.com/huggingface/transformers/blob/aca6288ff42cebded5421020f0ff088adeb446dd/examples/seq2seq/run_seq2seq.py#L590

so that all scripts automatically get this metric reported. Most likely it should be done here:

https://github.com/huggingface/transformers/blob/aca6288ff42cebded5421020f0ff088adeb446dd/src/transformers/trainer_utils.py#L224

@sgugger 
