sometimesanotion commited on
Commit
c5c097e
·
verified ·
1 Parent(s): e820fce

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -1
README.md CHANGED
@@ -29,7 +29,7 @@ Lamarck 14B v0.6: A generalist merge focused on multi-step reasoning, prose, mu
29
 
30
  Previous releases were based on a SLERP merge of model_stock+della branches focused on reasoning and prose. The prose branch got surprisingly good at reasoning, and the reasoning branch became a strong generalist in its own right. Some of you have already downloaded it as [sometimesanotion/Qwen2.5-14B-Vimarckoso-v3](https://huggingface.co/sometimesanotion/Qwen2.5-14B-Vimarckoso-v3).
31
 
32
- A notable contribution to the middle to upper layers of Lamarck v0.6 comes from [Krystalan/DRT-o1-14B](https://huggingface.co/Krystalan/DRT-o1-14B). It has a fascinating research paper: [DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought](https://huggingface.co/papers/2412.17498). It is only a minor contribution, as I have not resolved IFEval issues with larger merges.
33
 
34
  Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its multi-pronged merge strategies:
35
 
@@ -41,6 +41,56 @@ Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its mul
41
  This approach selectively merges the strongest aspects of its ancestors. Lamarck v0.6 is my most complex merge to date. The LORA extractions alone pushed my hardware enough to be the building's sole source of heat for several winter days! By comparison, the SLERP merge below which finalized it was a simple step.
42
 
43
  ```yaml
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  name: Lamarck-14B-v0.6-rc4
45
  merge_method: slerp
46
  base_model: sometimesanotion/lamarck-14b-converge-della-linear
@@ -54,6 +104,7 @@ parameters:
54
  parameters:
55
  t:
56
  - value: 0.30
 
57
  slices:
58
  - sources:
59
  - model: sometimesanotion/lamarck-14b-converge-della-linear
 
29
 
30
  Previous releases were based on a SLERP merge of model_stock+della branches focused on reasoning and prose. The prose branch got surprisingly good at reasoning, and the reasoning branch became a strong generalist in its own right. Some of you have already downloaded it as [sometimesanotion/Qwen2.5-14B-Vimarckoso-v3](https://huggingface.co/sometimesanotion/Qwen2.5-14B-Vimarckoso-v3).
31
 
32
+ A notable contribution to the middle to upper layers of Lamarck v0.6 comes from [Krystalan/DRT-o1-14B](https://huggingface.co/Krystalan/DRT-o1-14B). It has a fascinating research paper: [DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought](https://huggingface.co/papers/2412.17498). It is only a minor contribution, as I have not resolved IFEval issues with larger merges. Rigorously tested CoT has not yet arrived in Lamarck.
33
 
34
  Lamarck 0.6 hit a whole new level of toolchain-automated complexity with its multi-pronged merge strategies:
35
 
 
41
  This approach selectively merges the strongest aspects of its ancestors. Lamarck v0.6 is my most complex merge to date. The LORA extractions alone pushed my hardware enough to be the building's sole source of heat for several winter days! By comparison, the SLERP merge below which finalized it was a simple step.
42
 
43
  ```yaml
44
+ ---
45
+ name: lamarck-14b-v0.6-005-model_stock
46
+ merge_method: model_stock
47
+ base_model: sometimesanotion/Qwenvergence-14B-Base-v2
48
+ tokenizer_source: sometimesanotion/Abliterate-Qwenvergence
49
+ dtype: float32
50
+ out_dtype: bfloat16
51
+ parameters:
52
+ int8_mask: true
53
+ normalize: true
54
+ rescale: false
55
+ models:
56
+ - model: arcee-ai/Virtuoso-Small-qv64
57
+ - model: Krystalan/DRT-o1-14B-qv128
58
+ - model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3-qv64
59
+ - model: sometimesanotion/Qwenvergence-14B-v3-Prose-qv256
60
+ - model: sometimesanotion/Abliterate-Qwenvergence
61
+ ---
62
+ name: lamarck-14b-converge-breadcrumbs
63
+ merge_method: breadcrumbs
64
+ base_model: sometimesanotion/lamarck-14b-v0.6-005-model_stock
65
+ tokenizer_source: base
66
+ dtype: bfloat16
67
+ out_dtype: bfloat16
68
+ parameters:
69
+ int8_mask: true
70
+ normalize: true
71
+ rescale: false
72
+ density: 0.95
73
+ weight: 1.00
74
+ gamma: 0.018
75
+ # Here there be dragons!
76
+ ---
77
+ name: lamarck-14b-converge-della-linear
78
+ merge_method: della_linear
79
+ base_model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3
80
+ tokenizer_source: base
81
+ dtype: float32
82
+ out_dtype: bfloat16
83
+ parameters:
84
+ int8_mask: true
85
+ normalize: true
86
+ rescale: false
87
+ density: 0.95
88
+ weight: 1.00
89
+ epsilon: 0.018
90
+ lambda: 1.20
91
+ smoothing_factor: 0.07
92
+ # Yep, dragons.
93
+ ---
94
  name: Lamarck-14B-v0.6-rc4
95
  merge_method: slerp
96
  base_model: sometimesanotion/lamarck-14b-converge-della-linear
 
104
  parameters:
105
  t:
106
  - value: 0.30
107
+ # Not so dragon-ish.
108
  slices:
109
  - sources:
110
  - model: sometimesanotion/lamarck-14b-converge-della-linear