Technoculture
/

Medorca-2x7b

Text Generation

Mixture of Experts

epfl-llm/meditron-7b

microsoft/Orca-2-7b

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

satyamt commited on Jan 14

Commit

597c5b2

•

1 Parent(s): 0c89558

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -79,7 +79,7 @@ tokenizer = AutoTokenizer.from_pretrained(model)
 pipeline = transformers.pipeline(
     "text-generation",
     model=model,
-    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
 )
 messages = [{"role": "user", "content": "Why am i feeling so tired this month?"}]

 pipeline = transformers.pipeline(
     "text-generation",
     model=model,
+    model_kwargs={"torch_dtype": torch.float16},
 )
 messages = [{"role": "user", "content": "Why am i feeling so tired this month?"}]