Text Generation
Transformers
Safetensors
PyTorch
llama
facebook
meta
llama-3
conversational
text-generation-inference
Instructions to use meta-llama/Llama-3.2-1B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use meta-llama/Llama-3.2-1B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="meta-llama/Llama-3.2-1B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct") model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use meta-llama/Llama-3.2-1B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "meta-llama/Llama-3.2-1B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-1B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/meta-llama/Llama-3.2-1B-Instruct
- SGLang
How to use meta-llama/Llama-3.2-1B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "meta-llama/Llama-3.2-1B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-1B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "meta-llama/Llama-3.2-1B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-1B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use meta-llama/Llama-3.2-1B-Instruct with Docker Model Runner:
docker model run hf.co/meta-llama/Llama-3.2-1B-Instruct
finetuning
#176 opened 3 days ago
by
KarthikeyaPranav
Request: DOI
#175 opened 7 days ago
by
Baopro
Cross-architecture RYS sweep — Llama-3.2-1B-Instruct lifts reasoning 0%→76%
1
#174 opened 15 days ago
by
john-broadway
Access request
#173 opened 18 days ago
by
Jasonzggdsz
Request access
#172 opened 20 days ago
by
Kesline
Asking for request access
#171 opened 23 days ago
by
Rokaya-Akter
Request Access
#170 opened about 1 month ago
by
jepsonyang
Request Access
#169 opened about 1 month ago
by
henszv
Request for reconsideration of access to Llama 3.2
#168 opened about 1 month ago
by
SHENGGg
Access request rejected – need reset-
#167 opened about 1 month ago
by deleted
Request Access
#166 opened about 1 month ago
by
guiop
Student course assignment access request
#165 opened about 1 month ago
by
Charelse
Request for Reconsideration of Access to Llama 3.2 for Academic Research
#164 opened about 2 months ago
by
ysxiaogong
Request: DOI
#161 opened about 2 months ago
by
melvincz
Request Access
#159 opened 2 months ago
by
sammy520
Request Access
#158 opened 2 months ago
by
Huan0321
Request Access
#157 opened 2 months ago
by
Chang-Yue-Lin
Request access
#156 opened 2 months ago
by
vickywu555
Access Request for Llama 3.2-1B-Instruct - Agriculture
#155 opened 2 months ago
by
ajb-taranis
fix: set `clean_up_tokenization_spaces` to `false`
#154 opened 2 months ago
by
maxsloef
Access request rejected due to affiliation field mistake — please reconsider
#153 opened 2 months ago
by
cybercandyx1
Request Access
#152 opened 2 months ago
by
echo-mirror
Request Access
#151 opened 2 months ago
by
Xinghui0606
Install & run this model easily using llmpm
#150 opened 3 months ago
by
sarthak-saxena
Request for reconsideration of my access request
#149 opened 3 months ago
by
MASFAI
Request Access
2
#148 opened 3 months ago
by
xczou
Pull Request
#147 opened 3 months ago
by
JitendraParkar
Very long title with lots of characters to test the upper boundary limit validation - this title should be approaching the 200 character maximum length specified in the schema definition
#146 opened 4 months ago
by
121tester
Request: DOI
#145 opened 4 months ago
by
caiomercer
Request access
#144 opened 4 months ago
by
Dimpu00
Request access
#143 opened 4 months ago
by
eliao888
Update README.md
#142 opened 4 months ago
by
cherry0328
Request access
#140 opened 4 months ago
by
zhezhi233
Request access
#139 opened 4 months ago
by
llmler
Request Access
#138 opened 5 months ago
by
HabibiAhmad
Request access
#137 opened 5 months ago
by
ravenxone
Request Access
#136 opened 5 months ago
by
seachars
Safety Audit: GAE Score 9.87% (FAIL)
#135 opened 5 months ago
by
GAE-Auditor
Request Access
1
#134 opened 6 months ago
by
henac
Inquiries about the access
#133 opened 6 months ago
by
ssseoe
turbo_sft_training
#132 opened 6 months ago
by
rajeshSeemakurthi
Request: DOI
#131 opened 6 months ago
by
rokobasilisk1
Request: DOI
#130 opened 7 months ago
by
qasim418
Request Access: DOI
❤️ 1
#129 opened 7 months ago
by
h-gandomi
Request: DOI
#128 opened 7 months ago
by
ShInaa
Error: additional_chat_templates does not exist on "main"
3
#126 opened 8 months ago
by
Kaithryne
Request Access
#125 opened 8 months ago
by
yxstar2000