

TITLE
Insufficient memory occurs during finetune

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 📚 Migration

## Information

<!-- Important information -->

Model I am using (Bert,bert-base-chinese):

Language I am using the model on (Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## Details
* Here is my bash
```
export CUDA_VISIBLE_DEVICES=1,2
python run_mlm.py \
    --model_name_or_path bert-base-chinese \
    --train_file /home/xjyu/zhh/workspace/one-stop-ocr/char2char.allchars/quantize-train/data/scripts/wiki-part/wiki-part.split.nosign.txt \
    --validation_file /home/xjyu/zhh/workspace/one-stop-ocr/char2char.allchars/quantize-train/data/scripts/wiki-part/test.txt \
    --do_train \
    --do_eval \
    --output_dir ./test-mlm \
    --line_by_line \
```

When I performed finetune on bert->`run_mlm.py`, it took about 0.6 epoch，the output_dir file "test-mlm" took up 91G RAM，I am really surprised. It stands to reason that the output file cannot be so big，But i don't know where the problem is.
* here is my machine
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64       Driver Version: 440.64       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   32C    P8     9W / 250W |      0MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:03:00.0 Off |                  N/A |
|  0%   28C    P8     9W / 250W |      0MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 108...  Off  | 00000000:83:00.0 Off |                  N/A |
| 77%   80C    P2    73W / 250W |   4865MiB / 11177MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  GeForce GTX 108...  Off  | 00000000:84:00.0 Off |                  N/A |
|  0%   65C    P2    68W / 250W |   5574MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
<!-- A clear and concise description of the migration issue.
    If you have code snippets, please provide it here as well.
    Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
    Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
    -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
 
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

<!-- IMPORTANT: which version of the former library do you use? -->
* `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch):


## Checklist

- [x] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
  [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [x] I checked if a related official extension example runs on my machine.

@sgugger 