ethzanalytics
/

dolly-v2-12b-sharded-8bit

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions Community

pszemraj commited on Apr 24, 2023

Commit

a69aed2

•

1 Parent(s): e9d6045

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -28,13 +28,13 @@ Refer to the [original model](https://huggingface.co/databricks/dolly-v2-12b) fo
 ## Basic Usage
-install `transformers`, `accelerate`, and `bitsandbytes`.
 ```bash
 pip install -U -q transformers bitsandbytes accelerate
 ```
-Load the model. As it is serialized in 8bit you don't need to do anything special.
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM

 ## Basic Usage
+install/upgrade `transformers`, `accelerate`, and `bitsandbytes`. For this to work **you must have** `transformers>=4.28.0` and `bitsandbytes>0.37.2`.
 ```bash
 pip install -U -q transformers bitsandbytes accelerate
 ```
+Load the model. As it is serialized in 8bit you don't need to do anything special:
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM