Spaces:

ibm-granite
/

granite-guardian-3.1-8b

Running on Zero

grahamwhiteuk commited on 16 days ago

Commit

f466ad8

•

1 Parent(s): bce909e

fix: use_tqdm not used by the model error

Files changed (1) hide show

model.py CHANGED Viewed

@@ -161,7 +161,7 @@ def generate_text(messages, criteria_name):
     elif inference_engine == "VLLM":
         with torch.no_grad():
             # output = model.generate(chat, sampling_params, use_tqdm=False)
-            output = model.generate(chat, use_tqdm=False)
         label, prob_of_risk = parse_output(output[0])
     else:

     elif inference_engine == "VLLM":
         with torch.no_grad():
             # output = model.generate(chat, sampling_params, use_tqdm=False)
+            output = model.generate(chat)
         label, prob_of_risk = parse_output(output[0])
     else: