Update README.md

README.md

@@ -1,3 +1,50 @@
 ---
 license: apache-2.0
 ---
+
+# Sidrap-7B-v2-GPTQ
+
+Sidrap-7B-v2-GPTQ is an 8-bit quantized language model of Sidrap-7B-v2, which is one of the best open model LLM bahasa Indonesia available today. This model has been quantized using the [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) to get smaller model that allows us to run in low resource computer and faster inference. The quantization uses random subset of original training data to "calibrate" the weights resulting in an optimally compact model with minimall loss in accuracy.
+
+## Usage
+
+Here is an example code snippet for using Sidrap-7B-v2-GPTQ:
+
+```python
+from transformers import AutoTokenizer, pipeline
+from auto_gptq import AutoGPTQForCausalLM
+
+model_id = "robinsyihab/Sidrap-7B-v2-GPTQ"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
+
+model = AutoGPTQForCausalLM.from_quantized(model_id,
+                                           device="cuda:0", 
+                                           inject_fused_mlp=True,
+                                           inject_fused_attention=True,
+                                           trust_remote_code=True)
+
+chat = pipeline("text-generation", 
+                model=model,
+                tokenizer=tokenizer, 
+                device_map="auto")
+
+prompt = ("<s>[INST] <<SYS>>\nAnda adalah asisten yang suka membantu, penuh hormat, dan jujur. Selalu jawab semaksimal mungkin, sambil tetap aman. Jawaban Anda tidak boleh berisi konten berbahaya, tidak etis, rasis, seksis, beracun, atau ilegal. Harap pastikan bahwa tanggapan Anda tidak memihak secara sosial dan bersifat positif.\n\
+Jika sebuah pertanyaan tidak masuk akal, atau tidak koheren secara faktual, jelaskan alasannya daripada menjawab sesuatu yang tidak benar. Jika Anda tidak mengetahui jawaban atas sebuah pertanyaan, mohon jangan membagikan informasi palsu.\n"
+    "<</SYS>>\n\n"
+    "Siapa penulis kitab alfiyah? [/INST]\n"
+    )
+
+sequences = chat(prompt, num_beams=2, max_length=max_size, top_k=10, num_return_sequences=1)
+print(sequences[0]['generated_text'])
+```
+
+## License
+
+Sidrap-7B-v2-GPTQ is licensed under the Apache 2.0 License.
+
+## Author
+
+[] Robin Syihab ([@anvie](https://x.com/anvie))
+
+