mwitiderrick
commited on
Commit
•
d5e9f2f
1
Parent(s):
32f9347
Update README.md
Browse files
README.md
CHANGED
@@ -2,10 +2,6 @@
|
|
2 |
base_model: NousResearch/Llama-2-7b-hf
|
3 |
inference: false
|
4 |
model_type: llama
|
5 |
-
prompt_template: |
|
6 |
-
<s>[INST]
|
7 |
-
{prompt}
|
8 |
-
[/INST]
|
9 |
quantized_by: mwitiderrick
|
10 |
tags:
|
11 |
- deepsparse
|
@@ -24,24 +20,24 @@ Run in a [Python pipeline](https://github.com/neuralmagic/deepsparse/blob/main/d
|
|
24 |
```python
|
25 |
from deepsparse import TextGeneration
|
26 |
|
27 |
-
prompt = "
|
28 |
-
formatted_prompt = f"<s>[INST]{prompt}[/INST]"
|
29 |
|
30 |
model = TextGeneration(model_path="hf:nm-testing/Llama-2-7b-hf-pruned50-quant-ds")
|
31 |
|
32 |
-
print(model(
|
33 |
"""
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
"""
|
36 |
```
|
37 |
|
38 |
-
## Prompt template
|
39 |
-
```
|
40 |
-
<s>[INST]
|
41 |
-
<prompt>
|
42 |
-
[/INST]
|
43 |
|
44 |
-
```
|
45 |
## Sparsification
|
46 |
For details on how this model was sparsified, see the `recipe.yaml` in this repo and follow the instructions below.
|
47 |
|
|
|
2 |
base_model: NousResearch/Llama-2-7b-hf
|
3 |
inference: false
|
4 |
model_type: llama
|
|
|
|
|
|
|
|
|
5 |
quantized_by: mwitiderrick
|
6 |
tags:
|
7 |
- deepsparse
|
|
|
20 |
```python
|
21 |
from deepsparse import TextGeneration
|
22 |
|
23 |
+
prompt = "Once upon a time "
|
|
|
24 |
|
25 |
model = TextGeneration(model_path="hf:nm-testing/Llama-2-7b-hf-pruned50-quant-ds")
|
26 |
|
27 |
+
print(model(prompt, max_new_tokens=200).generations[0].text)
|
28 |
"""
|
29 |
+
1999
|
30 |
+
The first time I saw the movie Once Were Twice was when I was in my early teens.
|
31 |
+
I remember watching it with my brother and sister. I remember that I was very young and that I was not able to understand the movie.
|
32 |
+
I remember that I was very young and that I was not able to understand the movie. I remember that I was very young and that I was not able to understand the movie.
|
33 |
+
I remember that I was very young and that I was not able to understand the movie. I remember that I was very young and that I was not able to understand the movie.
|
34 |
+
I remember that I was very young and that I was not able to understand the movie. I remember that I was very young and that I was not able to understand the movie.
|
35 |
+
I remember that I was very young and that I was not able to understand the movie. I remember that I was very young and that I was not able to understand the movie.
|
36 |
+
I remember
|
37 |
"""
|
38 |
```
|
39 |
|
|
|
|
|
|
|
|
|
|
|
40 |
|
|
|
41 |
## Sparsification
|
42 |
For details on how this model was sparsified, see the `recipe.yaml` in this repo and follow the instructions below.
|
43 |
|