text: 
  
  TITLE
  Language model example hanging
  
  COMMENTS
  6
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ### System Info
  
  ```shell
  Transformer: 4.17
  PyTorch: 1.10.2
  
  Public Docker image I used:
  763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.10.2-transformers4.17.0-gpu-py38-cu113-ubuntu20.04-v1.0
  
  Hardware:
  1 AWS EC2 P4D instance, 8 A100 GPUs
  ```
  
  
  ### Who can help?
  
  _No response_
  
  ### Information
  
  - [X] The official example scripts
  - [ ] My own modified scripts
  
  ### Tasks
  
  - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
  - [ ] My own task or dataset (give details below)
  
  ### Reproduction
  
  Just run the examples using command in readme
  
  https://github.com/huggingface/transformers/tree/v4.17.0/examples/pytorch/language-modeling
  
  ```
  python run_mlm.py \
      --model_name_or_path roberta-base \
      --dataset_name wikitext \
      --dataset_config_name wikitext-2-raw-v1 \
      --per_device_train_batch_size 8 \
      --per_device_eval_batch_size 8 \
      --do_train \
      --do_eval \
      --output_dir /tmp/test-mlm
  ```
  
  It hangs after finish downloading the data.
  
  ```
  root/.cache/huggingface/datasets/wikitext/wikitext-2-raw-v1/1.0.0/a241db52902eaf2c6aa732210bead40c090019a499ceb13bcbfa3f8ab646a126/cache-7f695945617
  eacf7.arrow
  Grouping texts in chunks of 512: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████
  ██████████████████████████████████| 4/4 [00:00<00:00, 14.70ba/s]
  04/20/2022 18:15:03 - INFO - datasets.utils.file_utils - https://raw.githubusercontent.com/huggingface/datasets/1.18.4/metrics/accuracy/accuracy.py
  not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/tmp0f76eppv
  Downloading: 3.19kB [00:00, 3.66MB/s]
  ```
  
  
  ### Expected behavior
  
  ```shell
  run training as expected
  ```
  