chansurgeplus
commited on
Commit
•
d76d7b2
1
Parent(s):
5713e4b
Added usage instructions
Browse files
README.md
CHANGED
@@ -44,6 +44,39 @@ Notice that **no** end-of-sentence (eos) token is being appended.
|
|
44 |
|
45 |
*Note: The system prompt shown in the following figure is the one that the model has been trained on most of the time. However, you may attempt to use any other system prompt that is available in the [Orca](https://arxiv.org/abs/2306.02707) scheme.*
|
46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
## Limitations
|
48 |
|
49 |
- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
|
|
|
44 |
|
45 |
*Note: The system prompt shown in the following figure is the one that the model has been trained on most of the time. However, you may attempt to use any other system prompt that is available in the [Orca](https://arxiv.org/abs/2306.02707) scheme.*
|
46 |
|
47 |
+
### Usage
|
48 |
+
|
49 |
+
```python
|
50 |
+
from peft import PeftConfig, PeftModel
|
51 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig, AutoModelForSeq2SeqLM
|
52 |
+
|
53 |
+
checkpoint = "SurgeGlobal/OpenBezoar-SFT"
|
54 |
+
|
55 |
+
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
56 |
+
|
57 |
+
model = AutoModelForCausalLM.from_pretrained(
|
58 |
+
checkpoint,
|
59 |
+
load_in_4bit=True, # optionally for low resource environments
|
60 |
+
device_map="auto"
|
61 |
+
)
|
62 |
+
|
63 |
+
prompt = """### System:
|
64 |
+
Below is an instruction that describes a task, optionally paired with an input that provides further context following that instruction. Write a response that appropriately completes the request.
|
65 |
+
|
66 |
+
### Instruction:
|
67 |
+
{instruction}
|
68 |
+
|
69 |
+
### Response:""".format(
|
70 |
+
instruction="What is the world state in the year 1597."
|
71 |
+
)
|
72 |
+
|
73 |
+
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
74 |
+
|
75 |
+
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True)
|
76 |
+
|
77 |
+
print(tokenizer.decode(outputs[0]))
|
78 |
+
```
|
79 |
+
|
80 |
## Limitations
|
81 |
|
82 |
- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
|