π Relay
Collection
Turning base LLMs into IRCs, for more natural and versatile chat applications.
β’
5 items
β’
Updated
This model page includes GGUF versions of relay-v0.1-Mistral-Nemo-2407. For more details about this model, please see that model page.
Note: If you have access to a CUDA GPU, it's highly recommended you use the main version (HF) of the model with the relaylm.py script, which supports better use of commands (e.g., system messages). The relaylm.py
script also supports 4bit and 8bit bitsandbytes quants.
To use these GGUF files with LM Studio, you should use this preset configuration. Relay models use ChatML, but not standard roles and system prompts.
After you select and download the GGUF version you want to use:
Base model
mistralai/Mistral-Nemo-Base-2407