NbAiLab/borealis-1B-instruct-preview-gguf

borealis-1B-instruct-preview

Model creator: NbAiLab
Original model: NbAiLab/borealis-1B-instruct-preview
GGUF quantization: provided by versae using llama.cpp

Available Quantizations

• Q4_K_M
• Q8_0
• BF16

Special thanks

🙏 Special thanks to Georgi Gerganov and the whole team working on llama.cpp for making all of this possible.

Usage Examples

Q4_K_M

Ollama:

ollama run "hf.co/NbAiLab/borealis-1B-instruct-preview-gguf:Q4_K_M"

LM Studio:

lms load "NbAiLab/borealis-1B-instruct-preview-gguf/borealis-1B-instruct-preview-Q4_K_M.gguf"

llama.cpp CLI:

llama-cli --hf "NbAiLab/borealis-1B-instruct-preview-gguf:Q4_K_M" -p "The meaning to life and the universe is"

llama.cpp Server:

llama-server --hf "NbAiLab/borealis-1B-instruct-preview-gguf:Q4_K_M" -c 4096

Q8_0

Ollama:

ollama run "hf.co/NbAiLab/borealis-1B-instruct-preview-gguf:Q8_0"

LM Studio:

lms load "NbAiLab/borealis-1B-instruct-preview-gguf/borealis-1B-instruct-preview-Q8_0.gguf"

llama.cpp CLI:

llama-cli --hf "NbAiLab/borealis-1B-instruct-preview-gguf:Q8_0" -p "The meaning to life and the universe is"

llama.cpp Server:

llama-server --hf "NbAiLab/borealis-1B-instruct-preview-gguf:Q8_0" -c 4096

BF16

Ollama:

ollama run "hf.co/NbAiLab/borealis-1B-instruct-preview-gguf:BF16"

LM Studio:

lms load "NbAiLab/borealis-1B-instruct-preview-gguf/borealis-1B-instruct-preview-BF16.gguf"

llama.cpp CLI:

llama-cli --hf "NbAiLab/borealis-1B-instruct-preview-gguf:BF16" -p "The meaning to life and the universe is"

llama.cpp Server:

llama-server --hf "NbAiLab/borealis-1B-instruct-preview-gguf:BF16" -c 4096