Triangle104/Chronos-Gold-12B-1.0-Q4_K_S-GGUF

This model was converted to GGUF format from elinas/Chronos-Gold-12B-1.0 using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

Model details:

Chronos Gold 12B 1.0 is a very unique model that applies to domain areas such as general chatbot functionatliy, roleplay, and storywriting. The model has been observed to write up to 2250 tokens in a single sequence. The model was trained at a sequence length of 16384 (16k) and will still retain the apparent 128k context length from Mistral-Nemo, though it deteriorates over time like regular Nemo does based on the RULER Test

As a result, is recommended to keep your sequence length max at 16384, or you will experience performance degredation.

The base model is mistralai/Mistral-Nemo-Base-2407 which was heavily modified to produce a more coherent model, comparable to much larger models.

Chronos Gold 12B-1.0 re-creates the uniqueness of the original Chronos with significiantly enhanced prompt adherence (following), coherence, a modern dataset, as well as supporting a majority of "character card" formats in applications like SillyTavern.

It went through an iterative and objective merge process as my previous models and was further finetuned on a dataset curated for it.

The specifics of the model will not be disclosed at the time due to dataset ownership. Instruct Template

This model uses ChatML - below is an example. It is a preset in many frontends.

<|im_start|>system A system prompt describing how you'd like your bot to act.<|im_end|> <|im_start|>user Hello there!<|im_end|> <|im_start|>assistant I can assist you or we can discuss other things?<|im_end|> <|im_start|>user I was wondering how transformers work?<|im_end|> <|im_start|>assistant

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Chronos-Gold-12B-1.0-Q4_K_S-GGUF --hf-file chronos-gold-12b-1.0-q4_k_s.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Chronos-Gold-12B-1.0-Q4_K_S-GGUF --hf-file chronos-gold-12b-1.0-q4_k_s.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Chronos-Gold-12B-1.0-Q4_K_S-GGUF --hf-file chronos-gold-12b-1.0-q4_k_s.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Chronos-Gold-12B-1.0-Q4_K_S-GGUF --hf-file chronos-gold-12b-1.0-q4_k_s.gguf -c 2048