Run this model on multiple 24GB cards?
#9
by
LemonBranny
- opened
It seems that there is not enough video memory to run this model on a single 24GB card. Can I run this model on multiple 24GB cards to reduce the video memory usage of each card?
Any suggestions are greatly appreciated.
same
same here. tried to quantize it to 8bits no success, weird errors, 4bit will be unusable