RuntimeError: CUDA error: an illegal memory access was encountered

#5
by menglan - opened

run the demo on the page, meet this problem

RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Aborted

any idea to solve it?
thanks

Tencent org

Hi @menglan , this is likely to be an GPU OOM issue. You can try runining the fp8-quantized models

Sign up or log in to comment