Does this pretrained model adequately work？
#1
by
Dtree07
- opened
Sorry to bother，I just wanted to try to deploy this model on a GPU which has a VRAM of 16 GB and this model could be fully loaded when I set ‘offload_per_layer’ as 4 or even greater. What puzzles me is that the output are totally messy codes. Did anyone ever try to run this model successfully, please offer me your valuable suggestions which are really important for me, thank u so much!