Update README.md
Browse files
README.md
CHANGED
@@ -44,7 +44,7 @@ found in the model repo [here](https://huggingface.co/databricks/dolly-v2-3b/blo
|
|
44 |
Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
|
45 |
It is also fine to remove it if there is sufficient memory.
|
46 |
|
47 |
-
```
|
48 |
import torch
|
49 |
from transformers import pipeline
|
50 |
|
@@ -53,7 +53,7 @@ generate_text = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloa
|
|
53 |
|
54 |
You can then use the pipeline to answer instructions:
|
55 |
|
56 |
-
```
|
57 |
res = generate_text("Explain to me the difference between nuclear fission and fusion.")
|
58 |
print(res[0]["generated_text"])
|
59 |
```
|
@@ -61,7 +61,7 @@ print(res[0]["generated_text"])
|
|
61 |
Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/databricks/dolly-v2-3b/blob/main/instruct_pipeline.py),
|
62 |
store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
|
63 |
|
64 |
-
```
|
65 |
import torch
|
66 |
from instruct_pipeline import InstructionTextGenerationPipeline
|
67 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
@@ -77,7 +77,7 @@ generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokeniz
|
|
77 |
To use the pipeline with LangChain, you must set `return_full_text=True`, as LangChain expects the full text to be returned
|
78 |
and the default for the pipeline is to only return the new text.
|
79 |
|
80 |
-
```
|
81 |
import torch
|
82 |
from transformers import pipeline
|
83 |
|
@@ -87,7 +87,7 @@ generate_text = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloa
|
|
87 |
|
88 |
You can create a prompt that either has only an instruction or has an instruction with context:
|
89 |
|
90 |
-
```
|
91 |
from langchain import PromptTemplate, LLMChain
|
92 |
from langchain.llms import HuggingFacePipeline
|
93 |
|
@@ -109,13 +109,13 @@ llm_context_chain = LLMChain(llm=hf_pipeline, prompt=prompt_with_context)
|
|
109 |
|
110 |
Example predicting using a simple instruction:
|
111 |
|
112 |
-
```
|
113 |
print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())
|
114 |
```
|
115 |
|
116 |
Example predicting using an instruction with context:
|
117 |
|
118 |
-
```
|
119 |
context = """George Washington (February 22, 1732[b] – December 14, 1799) was an American military officer, statesman,
|
120 |
and Founding Father who served as the first president of the United States from 1789 to 1797."""
|
121 |
|
|
|
44 |
Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
|
45 |
It is also fine to remove it if there is sufficient memory.
|
46 |
|
47 |
+
```python
|
48 |
import torch
|
49 |
from transformers import pipeline
|
50 |
|
|
|
53 |
|
54 |
You can then use the pipeline to answer instructions:
|
55 |
|
56 |
+
```python
|
57 |
res = generate_text("Explain to me the difference between nuclear fission and fusion.")
|
58 |
print(res[0]["generated_text"])
|
59 |
```
|
|
|
61 |
Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/databricks/dolly-v2-3b/blob/main/instruct_pipeline.py),
|
62 |
store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
|
63 |
|
64 |
+
```python
|
65 |
import torch
|
66 |
from instruct_pipeline import InstructionTextGenerationPipeline
|
67 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
77 |
To use the pipeline with LangChain, you must set `return_full_text=True`, as LangChain expects the full text to be returned
|
78 |
and the default for the pipeline is to only return the new text.
|
79 |
|
80 |
+
```python
|
81 |
import torch
|
82 |
from transformers import pipeline
|
83 |
|
|
|
87 |
|
88 |
You can create a prompt that either has only an instruction or has an instruction with context:
|
89 |
|
90 |
+
```python
|
91 |
from langchain import PromptTemplate, LLMChain
|
92 |
from langchain.llms import HuggingFacePipeline
|
93 |
|
|
|
109 |
|
110 |
Example predicting using a simple instruction:
|
111 |
|
112 |
+
```python
|
113 |
print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())
|
114 |
```
|
115 |
|
116 |
Example predicting using an instruction with context:
|
117 |
|
118 |
+
```python
|
119 |
context = """George Washington (February 22, 1732[b] – December 14, 1799) was an American military officer, statesman,
|
120 |
and Founding Father who served as the first president of the United States from 1789 to 1797."""
|
121 |
|