MB-Zephyria-45b [EXPERIMENTAL]

Model Information

Base Model: unsloth/Mistral-Small-Instruct-2409

Strategy: Modified Balanced Approach with Extended Duplication

Total Layers: 55

Duplication Start: Layer 19 (34.5% of model)

Duplicated Layers: 30 (54.5% of model)

Unique Final Layers: 7 (11% of model)

Model Characteristics

• Models down_proj and o_proj layers have been nulled and will require healing
• Extends duplication further into later layers compared to the Balanced Approach
• Aims to enhance both understanding and creativity
• Maintains substantial unique initial layers for foundational processing
• Potentially suitable for complex reasoning and generative tasks

Configuration Visualization

[    Unique    ][        Duplicated        ][Unique]
0 ----------- 18 19 ------------------- 48 49 --- 54
     34.5%              54.5%              11%