text: 
  
  TITLE
  DeepSpeed: Exits with CUDA runtime error on A100 (requires recompiling DeepSpeed for NVIDIA 8.0 Arch)
  
  COMMENTS
  3
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  
  - `transformers` version: 4.3.0 (unofficial, off current main branch)
  - Platform: Linux
  - Python version: 3.7
  - PyTorch version (GPU?): 1.7.1
  - Using GPU in script?: Yes
  - Using distributed or parallel set-up in script?: Yes
  
  ## Information
  
  Model I am using (Bert, XLNet ...): T-5
  
  The problem arises when using:
  * [ ] the official example scripts: examples/seq2seq/finetune_trainer.py
  
  ## Issue
  
  In the hopes this saves others some time since it took a while for me to fix:  When running the new DeepSpeed mode in Transformers 4.3.0 on an A100 GPU, it will exit with a runtime error:
  
  RuntimeError: CUDA error: no kernel image is available for execution on the device
  
  For me, this was due to installing DeepSpeed from pip rather than source.  The A100 architecture appears not to be (as of this writing) installed in the default.  If you install from source as described in this post ( https://www.deepspeed.ai/tutorials/advanced-install/ ), the error goes away.  The post suggests selecting the architecture using the TORCH_CUDA_ARCH_LIST environment variable, but I found just using the install.sh script (which I am assuming auto-detects the architecture of your GPU) worked more successfully. 
  