sometimesanotion
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -29,7 +29,7 @@ Lamarck 14B v0.6: A generalist merge focused on multi-step reasoning, prose, mu
|
|
29 |
|
30 |
Previous releases were based on a SLERP merge of model_stock+della branches focused on reasoning and prose. The prose branch got surprisingly good at reasoning, and the reasoning branch became a strong generalist in its own right. Some of you have already downloaded it as [sometimesanotion/Qwen2.5-14B-Vimarckoso-v3](https://huggingface.co/sometimesanotion/Qwen2.5-14B-Vimarckoso-v3).
|
31 |
|
32 |
-
A notable contribution to the middle to upper layers of Lamarck v0.6 comes from [Krystalan/DRT-o1-14B](https://huggingface.co/Krystalan/DRT-o1-14B). It has a fascinating research paper: [DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought](https://huggingface.co/papers/2412.17498). It is only a minor contribution, as I have not resolved IFEval issues with larger merges.
|
33 |
|
34 |
Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its multi-pronged merge strategies:
|
35 |
|
@@ -41,6 +41,56 @@ Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its mul
|
|
41 |
This approach selectively merges the strongest aspects of its ancestors. Lamarck v0.6 is my most complex merge to date. The LORA extractions alone pushed my hardware enough to be the building's sole source of heat for several winter days! By comparison, the SLERP merge below which finalized it was a simple step.
|
42 |
|
43 |
```yaml
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
name: Lamarck-14B-v0.6-rc4
|
45 |
merge_method: slerp
|
46 |
base_model: sometimesanotion/lamarck-14b-converge-della-linear
|
@@ -54,6 +104,7 @@ parameters:
|
|
54 |
parameters:
|
55 |
t:
|
56 |
- value: 0.30
|
|
|
57 |
slices:
|
58 |
- sources:
|
59 |
- model: sometimesanotion/lamarck-14b-converge-della-linear
|
|
|
29 |
|
30 |
Previous releases were based on a SLERP merge of model_stock+della branches focused on reasoning and prose. The prose branch got surprisingly good at reasoning, and the reasoning branch became a strong generalist in its own right. Some of you have already downloaded it as [sometimesanotion/Qwen2.5-14B-Vimarckoso-v3](https://huggingface.co/sometimesanotion/Qwen2.5-14B-Vimarckoso-v3).
|
31 |
|
32 |
+
A notable contribution to the middle to upper layers of Lamarck v0.6 comes from [Krystalan/DRT-o1-14B](https://huggingface.co/Krystalan/DRT-o1-14B). It has a fascinating research paper: [DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought](https://huggingface.co/papers/2412.17498). It is only a minor contribution, as I have not resolved IFEval issues with larger merges. Rigorously tested CoT has not yet arrived in Lamarck.
|
33 |
|
34 |
Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its multi-pronged merge strategies:
|
35 |
|
|
|
41 |
This approach selectively merges the strongest aspects of its ancestors. Lamarck v0.6 is my most complex merge to date. The LORA extractions alone pushed my hardware enough to be the building's sole source of heat for several winter days! By comparison, the SLERP merge below which finalized it was a simple step.
|
42 |
|
43 |
```yaml
|
44 |
+
---
|
45 |
+
name: lamarck-14b-v0.6-005-model_stock
|
46 |
+
merge_method: model_stock
|
47 |
+
base_model: sometimesanotion/Qwenvergence-14B-Base-v2
|
48 |
+
tokenizer_source: sometimesanotion/Abliterate-Qwenvergence
|
49 |
+
dtype: float32
|
50 |
+
out_dtype: bfloat16
|
51 |
+
parameters:
|
52 |
+
int8_mask: true
|
53 |
+
normalize: true
|
54 |
+
rescale: false
|
55 |
+
models:
|
56 |
+
- model: arcee-ai/Virtuoso-Small-qv64
|
57 |
+
- model: Krystalan/DRT-o1-14B-qv128
|
58 |
+
- model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3-qv64
|
59 |
+
- model: sometimesanotion/Qwenvergence-14B-v3-Prose-qv256
|
60 |
+
- model: sometimesanotion/Abliterate-Qwenvergence
|
61 |
+
---
|
62 |
+
name: lamarck-14b-converge-breadcrumbs
|
63 |
+
merge_method: breadcrumbs
|
64 |
+
base_model: sometimesanotion/lamarck-14b-v0.6-005-model_stock
|
65 |
+
tokenizer_source: base
|
66 |
+
dtype: bfloat16
|
67 |
+
out_dtype: bfloat16
|
68 |
+
parameters:
|
69 |
+
int8_mask: true
|
70 |
+
normalize: true
|
71 |
+
rescale: false
|
72 |
+
density: 0.95
|
73 |
+
weight: 1.00
|
74 |
+
gamma: 0.018
|
75 |
+
# Here there be dragons!
|
76 |
+
---
|
77 |
+
name: lamarck-14b-converge-della-linear
|
78 |
+
merge_method: della_linear
|
79 |
+
base_model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3
|
80 |
+
tokenizer_source: base
|
81 |
+
dtype: float32
|
82 |
+
out_dtype: bfloat16
|
83 |
+
parameters:
|
84 |
+
int8_mask: true
|
85 |
+
normalize: true
|
86 |
+
rescale: false
|
87 |
+
density: 0.95
|
88 |
+
weight: 1.00
|
89 |
+
epsilon: 0.018
|
90 |
+
lambda: 1.20
|
91 |
+
smoothing_factor: 0.07
|
92 |
+
# Yep, dragons.
|
93 |
+
---
|
94 |
name: Lamarck-14B-v0.6-rc4
|
95 |
merge_method: slerp
|
96 |
base_model: sometimesanotion/lamarck-14b-converge-della-linear
|
|
|
104 |
parameters:
|
105 |
t:
|
106 |
- value: 0.30
|
107 |
+
# Not so dragon-ish.
|
108 |
slices:
|
109 |
- sources:
|
110 |
- model: sometimesanotion/lamarck-14b-converge-della-linear
|