Update README.md

README.md

@@ -1,3 +1,44 @@
 ---
 license: mit
+datasets:
+- databricks/databricks-dolly-15k
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- dolly
+- dolly-v2
+- instruct
+- sharded
+inference: False
 ---
+
+# dolly-v2-12b: sharded **8bit** checkpoint 
+
+
+This is a sharded checkpoint (with ~4GB shards) of the `databricks/dolly-v2-12b` model **in `8bit` precision** using `bitsandbytes`.
+
+Refer to the [original model](https://huggingface.co/databricks/dolly-v2-12b) for all details w.r.t. to the model. For more info on loading 8bit models, refer to the [example repo](https://huggingface.co/ybelkada/bloom-1b7-8bit) and/or the `4.28.0` [release info](https://github.com/huggingface/transformers/releases/tag/v4.28.0).
+
+
+- this enables low-RAM loading, i.e. Colab :)
+
+## Basic Usage
+
+
+install `transformers`, `accelerate`, and `bitsandbytes`.
+
+```bash
+pip install -U -q transformers bitsandbytes accelerate
+```
+
+Load the model. As it is serialized in 8bit you don't need to do anything special.
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "ethzanalytics/dolly-v2-12b-sharded-8bit"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+model = AutoModelForCausalLM.from_pretrained(model_name)
+```