Instructions to use kalisai/Nusantara-0.8b-Indo-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kalisai/Nusantara-0.8b-Indo-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kalisai/Nusantara-0.8b-Indo-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kalisai/Nusantara-0.8b-Indo-Chat")
model = AutoModelForCausalLM.from_pretrained("kalisai/Nusantara-0.8b-Indo-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use kalisai/Nusantara-0.8b-Indo-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kalisai/Nusantara-0.8b-Indo-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kalisai/Nusantara-0.8b-Indo-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kalisai/Nusantara-0.8b-Indo-Chat

SGLang

How to use kalisai/Nusantara-0.8b-Indo-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kalisai/Nusantara-0.8b-Indo-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kalisai/Nusantara-0.8b-Indo-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kalisai/Nusantara-0.8b-Indo-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kalisai/Nusantara-0.8b-Indo-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kalisai/Nusantara-0.8b-Indo-Chat with Docker Model Runner:
```
docker model run hf.co/kalisai/Nusantara-0.8b-Indo-Chat
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Description

Nusantara is a series of Open Weight Language Model of Bahasa Indonesia (Indonesia language). Nusantara is based from Qwen1.5 Language Model, finetuned by domain specific of datasets. As Chat-implemented language model, Nusantara is capable to do Question-Answering and respond to instructions given in Bahasa Indonesia. Due to limited resources, only 0.8B, 1.8B, 2.7B, 4B and 7B models are available. If you're interested in funding this project for further development, specific usage, or larger parameters, please contact us.

Finetuned by: Kalis AI
Funded by: Self-funded
Model type: transformer-based decoder-only language model
Language(s): Bahasa Indonesia (id), English (en)
License: Nusantara is licensed under Apache-2.0, but any usage of this model should comply with Qwen License
Finetuned from model: Qwen1.5-4B

Attentions!

Due to certain circumstances, models with <4B parameters tend to hallucinate easily. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Because this model is also trained with uncensored datasets, there is the possibility of negative impacts arising from using this model. All kinds of impacts that arise as a result of using this model are entirely the responsibility of the user. The model maker is not responsible for any risks incurred.

How to Get Started with the Model

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "kalisai/Nusantara-0.8B-Indo-Chat",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("kalisai/Nusantara-0.8B-Indo-Chat")

prompt = "Berikan saya resep memasak nasi goreng yang lezat."
messages = [
    {"role": "system", "content": "Kamu adalah Nusantara, asisten AI yang pintar."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Citation

If you use the Nusantara language model in your research or project, please cite it as:

@misc{zulfikar_aji_kusworo_2024,
  title={Nusantara: A Series of Versatile Open Weight Language Model of Bahasa Indonesia},
  author={Zulfikar Aji Kusworo},
  publisher={Hugging Face}
  journal={Hugging Face Repository},
  year={2024}
  url = {https://huggingface.co/kalisai}
}

Downloads last month: 106

Safetensors

Model size

0.8B params

Tensor type

BF16

Model tree for kalisai/Nusantara-0.8b-Indo-Chat

Quantizations

3 models

kalisai
/

Nusantara-0.8b-Indo-Chat

Model Description

Attentions!

How to Get Started with the Model

Citation

Model tree for kalisai/Nusantara-0.8b-Indo-Chat

Dataset used to train kalisai/Nusantara-0.8b-Indo-Chat