inferno-math-exp140

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using /home/ubuntu/tmp/models/miscii-14b-1225 as a base.

Models Merged

The following models were included in the merge:

/home/ubuntu/tmp/models/inferno-math-stage1-ckpt700
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1200
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt400
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1400
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1100
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt300
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt600
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt900
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt100
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1000
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt800
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1300
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt1484
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt500
/home/ubuntu/tmp/models/inferno-math-stage1-ckpt200

Configuration

The following YAML configuration was used to produce this model:

name:                exp-140
merge_method:        model_stock
base_model:          /home/ubuntu/tmp/models/miscii-14b-1225
tokenizer_source:    /home/ubuntu/tmp/models/miscii-14b-1225
parameters:
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt100
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt200
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt300
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt400
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt500
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt600
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt700  
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt800  
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt900
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1000
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1100
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1200
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1300
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1400
  - model:           /home/ubuntu/tmp/models/inferno-math-stage1-ckpt1484

dtype:               bfloat16