torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 28.00 MiB. GPU

by weiminw - opened Jun 4

Jun 4

使用int4 为什么也会使用28G GPU memory, 在那里可以调整？

weiminw changed discussion status to closed Jun 4

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org Jun 4

单卡吗，没雨遇到过用了28G的情况，你对话长度多少了

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment