requires updating llama.cpp to the latest version.

./llama-cli -m model.gguf -cnv --color -p "a chat between user and assistant" --in-prefix "user: " --in-suffix "assistant: " --reverse-prompt "user:"

GGUF

Model size

7.3B params

Architecture

olmo_1124

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference API

Unable to determine this model's library. Check the docs .

Model tree for Fishfishfishfishfish/OLMo-7B-1024-preview

Base model

Quantized

(1)

this model