Update the code format according to python syntax.
#4
by
imjunaidafzal
- opened
README.md
CHANGED
@@ -65,49 +65,49 @@ Any model can provide inaccurate or incomplete information, and should be used i
|
|
65 |
## How to Get Started with the Model
|
66 |
|
67 |
The fastest way to get started with BLING is through direct import in transformers:
|
68 |
-
|
69 |
-
|
70 |
-
|
71 |
-
|
72 |
-
|
73 |
Please refer to the generation_test .py files in the Files repository, which includes 200 samples and script to test the model. The **generation_test_llmware_script.py** includes built-in llmware capabilities for fact-checking, as well as easy integration with document parsing and actual retrieval to swap out the test set for RAG workflow consisting of business documents.
|
74 |
|
75 |
The DRAGON model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
|
76 |
-
|
77 |
-
|
78 |
-
|
79 |
The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
|
80 |
|
81 |
1. Text Passage Context, and
|
82 |
2. Specific question or instruction based on the text passage
|
83 |
|
84 |
To get the best results, package "my_prompt" as follows:
|
85 |
-
|
86 |
-
|
87 |
-
|
88 |
|
89 |
If you are using a HuggingFace generation script:
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
94 |
-
|
95 |
-
|
96 |
-
|
97 |
-
|
98 |
-
|
99 |
-
|
100 |
-
|
101 |
-
|
102 |
-
|
103 |
-
|
104 |
-
|
105 |
-
|
106 |
-
|
107 |
-
|
108 |
-
|
109 |
-
|
110 |
-
|
111 |
|
112 |
## Model Card Contact
|
113 |
|
|
|
65 |
## How to Get Started with the Model
|
66 |
|
67 |
The fastest way to get started with BLING is through direct import in transformers:
|
68 |
+
```python
|
69 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
70 |
+
tokenizer = AutoTokenizer.from_pretrained("dragon-yi-6b-v0")
|
71 |
+
model = AutoModelForCausalLM.from_pretrained("dragon-yi-6b-v0")
|
72 |
+
```
|
73 |
Please refer to the generation_test .py files in the Files repository, which includes 200 samples and script to test the model. The **generation_test_llmware_script.py** includes built-in llmware capabilities for fact-checking, as well as easy integration with document parsing and actual retrieval to swap out the test set for RAG workflow consisting of business documents.
|
74 |
|
75 |
The DRAGON model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
|
76 |
+
```python
|
77 |
+
full_prompt = "<human>: " + my_prompt + "\n" + "<bot>:"
|
78 |
+
```
|
79 |
The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
|
80 |
|
81 |
1. Text Passage Context, and
|
82 |
2. Specific question or instruction based on the text passage
|
83 |
|
84 |
To get the best results, package "my_prompt" as follows:
|
85 |
+
```python
|
86 |
+
my_prompt = {{text_passage}} + "\n" + {{question/instruction}}
|
87 |
+
```
|
88 |
|
89 |
If you are using a HuggingFace generation script:
|
90 |
+
```python
|
91 |
+
# prepare prompt packaging used in fine-tuning process
|
92 |
+
new_prompt = "<human>: " + entries["context"] + "\n" + entries["query"] + "\n" + "<bot>:"
|
93 |
+
|
94 |
+
inputs = tokenizer(new_prompt, return_tensors="pt")
|
95 |
+
start_of_output = len(inputs.input_ids[0])
|
96 |
+
|
97 |
+
# temperature: set at 0.3 for consistency of output
|
98 |
+
# max_new_tokens: set at 100 - may prematurely stop a few of the summaries
|
99 |
+
|
100 |
+
outputs = model.generate(
|
101 |
+
inputs.input_ids.to(device),
|
102 |
+
eos_token_id=tokenizer.eos_token_id,
|
103 |
+
pad_token_id=tokenizer.eos_token_id,
|
104 |
+
do_sample=True,
|
105 |
+
temperature=0.3,
|
106 |
+
max_new_tokens=100,
|
107 |
+
)
|
108 |
+
|
109 |
+
output_only = tokenizer.decode(outputs[0][start_of_output:],skip_special_tokens=True)
|
110 |
+
```
|
111 |
|
112 |
## Model Card Contact
|
113 |
|