metadata
library_name: transformers
license: cc-by-nc-4.0
tags:
- creative-writing
- creative-writer
- multiplicative-lora
An experimental model, fine-tuned using the "multiplicative-LoRA" method on c4ai-command-r-v01.
This model is nearly identical to creative-writer-v0.2-bravo-35b (ie: trained using the new v0.2 dataset which is around 15% larger and also more carefully curated to remove any weird formatting), but the pre-softmax logits were scaled by 1.25 during training (instead of 1.1).
NOTE: The model seems to be slightly broken as a result of the logit-scaling increase (ie: it doesn't follow instructions as well and is prone to outputting weirdly formatted text).
Please refer to creative-writer-v0.1-alfa-35b for full details on how to use this model.