Successfully merging with Miqu
#2
by
Ont
- opened
Miqu weights differ more than differences between many other 70B models. As demonstrated by this model, merges between models with greater differences may not make sense unless done carefully. For example, Sophosympatheia has shared a YAML configuration for a Miqu merge that resulted in a working model:
https://huggingface.co/sophosympatheia/Midnight-Miqu-70B-v1.0#configuration
A different Japanese model might possibly merge better (or worse). Here's another Japanese model:
https://huggingface.co/tokyotech-llm/Swallow-70b-NVE-instruct-hf
I hope this information helps if you're interested in trying again.