Spaces:

TIGER-Lab
/

TIGERScore

Running on Zero

Kaguya-19 commited on Oct 5, 2023

Commit

97a45f3

•

1 Parent(s): ec1da13

Update utils.py

Files changed (1) hide show

utils.py CHANGED Viewed

@@ -42,7 +42,8 @@ def load_tigerscore(model_size):
     tigerscore_model = AutoModelForCausalLM.from_pretrained(
         model_name,
         torch_dtype=torch.bfloat16,
-        device_map="auto"
     )
     tigerscore_tokenizer = AutoTokenizer.from_pretrained(
         model_name,

     tigerscore_model = AutoModelForCausalLM.from_pretrained(
         model_name,
         torch_dtype=torch.bfloat16,
+        device_map="auto",
+        offload="offload"
     )
     tigerscore_tokenizer = AutoTokenizer.from_pretrained(
         model_name,