|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
tags: |
|
- control-vector |
|
- creative-writing |
|
--- |
|
|
|
## gghfez/L3.3-70B-Euryale-v2.3-control-vectors |
|
|
|
Creative Writing control-vectors for [Sao10K/L3.3-70B-Euryale-v2.3](https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3) |
|
|
|
## Usage example |
|
|
|
### llama.cpp |
|
```sh |
|
llama-server --model L3.3-70B-Euryale-v2.3.Q5_K_M.gguf [other CLI arguments] \ |
|
--control-vector L3.3-70B-Euryale-v2.3-character_focus__debias.gguf \ |
|
--control-vector L3.3-70B-Euryale-v2.3-character_focus__dialogue.gguf |
|
``` |
|
|
|
### exllamav2 with exui (via exui fork) |
|
|
|
https://github.com/gapeleon/exui-control-vectors |
|
|
|
1. Set it up / install it the same as regular exui |
|
2. `pip install gguf` if you don't have it already |
|
3. Create a directory alongside the model but with `-vectors` appended |
|
4. Start exui `python server.py` |
|
5. When setting up the model on the model config page, add the vectors and their scaling/direction then click "load model" |
|
6. |
|
Trained on 1xH100NVL using jukofyork's [control-vectors](https://github.com/jukofyork/control-vectors) |
|
|
|
Based off Version v3.0 of [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/). |
|
|
|
See here for more info about control-vectors [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/) |
