metadata
base_model:
- unsloth/Mistral-Small-Instruct-2409
library_name: transformers
tags:
- mergekit
- merge
LD-Zephyria-37b [EXPERIMENTAL]
Model Information
Base Model: unsloth/Mistral-Small-Instruct-2409
Strategy: Late Duplication
Total Layers: 55
Duplication Start: Layer 28 (50.9% of model)
Duplicated Layers: 21 (38.2% of model)
Unique Final Layers: 7 (12.7% of model)
Model Characteristics
- Models down_proj and o_proj layers have been nulled and will require healing
- Emphasizes complex feature extraction before duplication
- Smallest duplicated section among all strategies
- Ideal for tasks requiring extensive unique feature processing
- May excel in tasks that benefit from a wide range of unique features before refinement
Configuration Visualization
[ Unique ][ Duplicated ][ Unique ]
0 ------------------- 27 28 ------------ 48 49 --- 54
50.9% 38.2% 10.9%