Update README.md
Browse files
README.md
CHANGED
@@ -41,18 +41,13 @@ Via pip: `pip install llm-rs`
|
|
41 |
|
42 |
#### Run inference
|
43 |
```python
|
44 |
-
from llm_rs import
|
45 |
-
from huggingface_hub import hf_hub_download
|
46 |
|
47 |
-
#
|
48 |
-
|
49 |
-
|
50 |
-
#Load the model
|
51 |
-
model = Mpt("mpt-7b-q4_0-ggjt.bin")
|
52 |
|
53 |
#Generate
|
54 |
print(model.generate("The meaning of life is"))
|
55 |
-
```
|
56 |
|
57 |
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
|
58 |
|
|
|
41 |
|
42 |
#### Run inference
|
43 |
```python
|
44 |
+
from llm_rs import AutoModel
|
|
|
45 |
|
46 |
+
#Load the model, define any model you like from the list above as the `model_file`
|
47 |
+
model = AutoModel.from_pretrained("Rustformers/mpt-7b-ggml",model_file="mpt-7b-q4_0-ggjt.bin")
|
|
|
|
|
|
|
48 |
|
49 |
#Generate
|
50 |
print(model.generate("The meaning of life is"))
|
|
|
51 |
|
52 |
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
|
53 |
|