

TITLE
when the txt file has 5GB, a Killed prompt appears.

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
I am running run_language_modeling.py, 
python run_language_modeling.py \
	    --output_dir ${model_dir} \
	    --tokenizer_name $data_dir/wordpiece-custom.json \
	    --config_name $data_dir/config.json \
            --train_data_file "$data_dir/train.txt" \
            --eval_data_file $data_dir/valid.txt \
            --block_size=128 \
            --do_train \
            --per_device_train_batch_size 1 \
            --gradient_accumulation_steps 64 \
            --learning_rate 6e-4 \
            --weight_decay 0.01 \
            --adam_epsilon 1e-6 \
            --adam_beta1 0.9 \
            --adam_beta2 0.98 \
            --max_steps 500_000 \
            --warmup_steps 24_000 \
            --fp16 \
            --logging_dir ${model_dir}/tensorboard \
            --save_steps 1000 \
            --save_total_limit 20 \
            --seed 108 \
            --max_steps -1 \
            --num_train_epochs 20 \
            --overwrite_output_dir

when the txt file has 5GB, a Killed prompt appears.
