anslin-raj commited on
Commit
f01a66e
·
1 Parent(s): 370e6b8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -1,3 +1,16 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ language:
4
+ - en
5
+ library_name: transformers
6
  ---
7
+
8
+ ```
9
+ from transformers import AutoTokenizer, AutoModelForCausalLM
10
+
11
+ HF_MODEL_REPO = "anslin-raj/Llama-2-7b-chat-hf-8-bit"
12
+
13
+ tokenizer = AutoTokenizer.from_pretrained(HF_MODEL_REPO, device=device)
14
+
15
+ model = AutoModelForCausalLM.from_pretrained(HF_MODEL_REPO, device_map=device, load_in_8bit=True)
16
+ ```