vLLM help pls :(
1
#6 opened 2 days ago
by
fsaudm
How much cuda memory is needed to run this model?
2
#5 opened 9 days ago
by
JohnnyBoyzzz
Any chance of an int4 or quantised version?
2
#3 opened 10 days ago
by
smcleod