metadata

base_model:
  - RozGrov/NemoDori-v0.2-12B-MN-BT
  - unsloth/Mistral-Nemo-Instruct-2407
  - UsernameJustAnother/Nemo-12B-Marlin-v5
  - crestf411/nemo-sunfall-v0.6.1
library_name: transformers
tags:
  - mergekit
  - merge

NemoDori-v0.5-12B-MN-BT

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the breadcrumbs_ties merge method using RozGrov/NemoDori-v0.2-12B-MN-BT as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:


models:
  - model: crestf411/nemo-sunfall-v0.6.1
    parameters:
      weight: 0.33
  - model: UsernameJustAnother/Nemo-12B-Marlin-v5
    parameters:
      weight: 0.2
  - model: unsloth/Mistral-Nemo-Instruct-2407
    parameters:
      weight: 0.37
  - model: RozGrov/NemoDori-v0.2-12B-MN-BT
    parameters:
      weight: 1
merge_method: breadcrumbs_ties
base_model: RozGrov/NemoDori-v0.2-12B-MN-BT
parameters:
    density: 0.93
    gamma: 0.015
dtype: bfloat16