

TITLE
OSError: [Errno 116] Stale file handle

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?): - 
- Using GPU in script?: - 
- Using distributed or parallel set-up in script?: - only 1 GPU

### Who can help
 Trainer: @sgugger
 examples/seq2seq: @patil-suraj

## Information
Hi, I am training finetune_trainer.py on on wmt dataset. I am getting the following error sometimes, do you have an idea what might cause it? thanks for any suggestion 

```
  File "finetune_trainer.py", line 342, in <module>
    main()
  File "finetune_trainer.py", line 256, in main
    if (os.path.isdir(training_args.output_dir) and not training_args.optimize_from_scratch) else None,
  File "/home/jim/trainer.py", line 814, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
  File "/home/jim/trainer.py", line 885, in _maybe_log_save_evaluate
    self._save_checkpoint(model, trial, metrics=metrics)
  File "/home/jim/trainer.py", line 916, in _save_checkpoint
    torch.save(self.optimizer.state_dict(), os.path.join(output_dir, "optimizer.pt"))
  File "/home/jim/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/serialization.py", line 374, in save
    _legacy_save(obj, opened_file, pickle_module, pickle_protocol)
  File "/home/jim/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/serialization.py", line 214, in __exit__
    self.file_like.close()
OSError: [Errno 116] Stale file handle
```
Model I am using T5.

## To reproduce
this is not happening all the time, but it does happen

## Expected behavior

to run the codes
