BaseConfigA-ConfigC

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the SCE merge method using ./BaseConfigA as a base.

Models Merged

The following models were included in the merge:

• TheDrummer/Magidonia-24B-v4.3
• ApocalypseParty/Magi-24B-SFT-v4-1-DPO-2
• Delta-Vector/MS3.2-Austral-Winton
• Delta-Vector/Rei-24B-KTO

Configuration

The following YAML configuration was used to produce this model:

#Base Config A - Config C
models:
  - model: TheDrummer/Magidonia-24B-v4.3
  - model: ApocalypseParty/Magi-24B-SFT-v4-1-DPO-2
  - model: Delta-Vector/Rei-24B-KTO
  - model: Delta-Vector/MS3.2-Austral-Winton
select_topk: .15
merge_method: sce
base_model: ./BaseConfigA
out_dtype: bfloat16
dtype: float32
tokenizer:
  source: base