File size: 3,436 Bytes
de9e8d3 88d3ae5 f31cd17 88d3ae5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 |
---
base_model:
- unsloth/Mistral-Small-Instruct-2409
library_name: transformers
tags:
- mergekit
- merge
---
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>BA-Zephyria-39b Data Card</title>
<link href="https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;600&display=swap" rel="stylesheet">
<style>
body, html {
height: 100%;
margin: 0;
padding: 0;
font-family: 'Quicksand', sans-serif;
background: linear-gradient(135deg, #0a1128 0%, #1c2541 100%);
color: #e0e1dd;
font-size: 16px;
}
.container {
width: 100%;
height: 100%;
padding: 20px;
margin: 0;
background-color: rgba(255, 255, 255, 0.05);
border-radius: 12px;
box-shadow: 0 4px 10px rgba(0, 0, 0, 0.3);
backdrop-filter: blur(10px);
border: 1px solid rgba(255, 255, 255, 0.1);
}
.header h1 {
font-size: 28px;
color: #4cc9f0;
margin: 0 0 20px 0;
text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
}
.update-section h2 {
font-size: 24px;
color: #7209b7;
}
.update-section p {
font-size: 16px;
line-height: 1.6;
color: #e0e1dd;
}
.info img {
width: 100%;
border-radius: 10px;
margin-bottom: 15px;
}
a {
color: #4cc9f0;
text-decoration: none;
}
a:hover {
color: #f72585;
}
.button {
display: inline-block;
background-color: #3a0ca3;
color: #e0e1dd;
padding: 10px 20px;
border-radius: 5px;
cursor: pointer;
text-decoration: none;
}
.button:hover {
background-color: #7209b7;
}
pre {
background-color: #1c2541;
padding: 10px;
border-radius: 5px;
overflow-x: auto;
}
code {
font-family: 'Courier New', monospace;
color: #e0e1dd;
}
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>BA-Zephyria-39b [EXPERIMENTAL]</h1>
</div>
<div class="info">
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/6W3orrbf8A68l-3p_JxN1.png">
<h2>Model Information</h2>
<p><strong>Base Model:</strong> unsloth/Mistral-Small-Instruct-2409</p>
<p><strong>Strategy:</strong> Balanced Approach</p>
<p><strong>Total Layers:</strong> 55</p>
<p><strong>Duplication Start:</strong> Layer 19 (34.5% of model)</p>
<p><strong>Duplicated Layers:</strong> 23 (41.8% of model)</p>
<p><strong>Unique Final Layers:</strong> 14 (25.5% of model)</p>
<h2>Model Characteristics</h2>
<ul>
<li>Models down_proj and o_proj layers have been nulled and will require healing</li>
<li>Combines benefits of early and mid duplication strategies</li>
<li>Balanced between unique initial layers, duplicated middle layers, and unique final layers</li>
<li>Versatile approach suitable for a wide range of tasks</li>
<li>Provides substantial unique layers at the end for task-specific adaptations</li>
</ul>
<h2>Configuration Visualization</h2>
<pre><code>
[ Unique ][ Duplicated ][ Unique ]
0 ----------- 18 19 ------------ 41 42 ---------- 54
34.5% 41.8% 23.7%
</code></pre>
</div>
</div>
</body>
</html> |