Instructions to use arcee-ai/Arcee-VyLinh with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arcee-ai/Arcee-VyLinh with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="arcee-ai/Arcee-VyLinh")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Arcee-VyLinh")
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Arcee-VyLinh")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use arcee-ai/Arcee-VyLinh with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arcee-ai/Arcee-VyLinh"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Arcee-VyLinh",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/arcee-ai/Arcee-VyLinh

SGLang

How to use arcee-ai/Arcee-VyLinh with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arcee-ai/Arcee-VyLinh" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Arcee-VyLinh",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arcee-ai/Arcee-VyLinh" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Arcee-VyLinh",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use arcee-ai/Arcee-VyLinh with Docker Model Runner:
```
docker model run hf.co/arcee-ai/Arcee-VyLinh
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Quantized Version: arcee-ai/Arcee-VyLinh-GGUF

Arcee-VyLinh

Arcee-VyLinh is a 3B parameter instruction-following model specifically optimized for Vietnamese language understanding and generation. Built through an innovative training process combining evolved hard questions and iterative Direct Preference Optimization (DPO), it achieves remarkable performance despite its compact size.

Model Details

Architecture: Based on Qwen2.5-3B
Parameters: 3 billion
Context Length: 32K tokens
Training Data: Custom evolved dataset + ORPO-Mix-40K (Vietnamese)
Training Method: Multi-stage process including EvolKit, proprietary merging, and iterative DPO
Input Format: Supports both English and Vietnamese, optimized for Vietnamese

Intended Use

Vietnamese language chat and instruction following
Text generation and completion
Question answering
General language understanding tasks
Content creation and summarization

Performance and Limitations

Strengths

Exceptional performance on complex Vietnamese language tasks
Efficient 3B parameter architecture
Strong instruction-following capabilities
Competitive with larger models (4B-8B parameters)

Benchmarks

Tested on Vietnamese subset of m-ArenaHard (CohereForAI), with Claude 3.5 Sonnet as judge:

Limitations

Might still hallucinate on cultural-specific content.
Primary focus on Vietnamese language understanding
May not perform optimally for specialized technical domains

Training Process

Our training pipeline consisted of several innovative stages:

Base Model Selection: Started with Qwen2.5-3B
Hard Question Evolution: Generated 20K challenging questions using EvolKit
Initial Training: Created VyLinh-SFT through supervised fine-tuning
Model Merging: Proprietary merging technique with Qwen2.5-3B-Instruct
DPO Training: 6 epochs of iterative DPO using ORPO-Mix-40K
Final Merge: Combined with Qwen2.5-3B-Instruct for optimal performance

Usage Examples

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Arcee-VyLinh")
tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Arcee-VyLinh")

prompt = "Một cộng một bằng mấy?"
messages = [
    {"role": "system", "content": "Bạn là trợ lí hữu ích."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=1024,
    eos_token_id=tokenizer.eos_token_id,
    temperature=0.25,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids)[0]
print(response)