vLLM help pls :(
1
#6 opened 3 days ago
by
fsaudm
How much cuda memory is needed to run this model?
2
#5 opened 10 days ago
by
JohnnyBoyzzz
Any chance of an int4 or quantised version?
2
#3 opened 11 days ago
by
smcleod