chansurgeplus commited on
Commit
d76d7b2
1 Parent(s): 5713e4b

Added usage instructions

Browse files
Files changed (1) hide show
  1. README.md +33 -0
README.md CHANGED
@@ -44,6 +44,39 @@ Notice that **no** end-of-sentence (eos) token is being appended.
44
 
45
  *Note: The system prompt shown in the following figure is the one that the model has been trained on most of the time. However, you may attempt to use any other system prompt that is available in the [Orca](https://arxiv.org/abs/2306.02707) scheme.*
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## Limitations
48
 
49
  - The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
 
44
 
45
  *Note: The system prompt shown in the following figure is the one that the model has been trained on most of the time. However, you may attempt to use any other system prompt that is available in the [Orca](https://arxiv.org/abs/2306.02707) scheme.*
46
 
47
+ ### Usage
48
+
49
+ ```python
50
+ from peft import PeftConfig, PeftModel
51
+ from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig, AutoModelForSeq2SeqLM
52
+
53
+ checkpoint = "SurgeGlobal/OpenBezoar-SFT"
54
+
55
+ tokenizer = AutoTokenizer.from_pretrained(checkpoint)
56
+
57
+ model = AutoModelForCausalLM.from_pretrained(
58
+ checkpoint,
59
+ load_in_4bit=True, # optionally for low resource environments
60
+ device_map="auto"
61
+ )
62
+
63
+ prompt = """### System:
64
+ Below is an instruction that describes a task, optionally paired with an input that provides further context following that instruction. Write a response that appropriately completes the request.
65
+
66
+ ### Instruction:
67
+ {instruction}
68
+
69
+ ### Response:""".format(
70
+ instruction="What is the world state in the year 1597."
71
+ )
72
+
73
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
74
+
75
+ outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True)
76
+
77
+ print(tokenizer.decode(outputs[0]))
78
+ ```
79
+
80
  ## Limitations
81
 
82
  - The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.