BigCodeLlama-169b / README.md
nisten's picture
Update README.md
ef972f7 verified
|
raw
history blame
1.03 kB
metadata
base_model: []
tags:
  - mergekit
  - merge
  - code
license: mit

BigCodeLLama LFG πŸš€

Experimental CodeLlaMA frankenstein to see how it benchmarks

Models Merged with base codellama/CodeLlama-70b-hf

The following models were included in the merge:

  • ../CodeLlama-70b-hf
  • ../CodeLlama-70b-Instruct-hf
  • ../CodeLlama-70b-Python-hf

Configuration

The following YAML configuration was used to produce this model:

dtype: bfloat16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 69]
    model:
      model:
        path: ../CodeLlama-70b-hf
- sources:
  - layer_range: [66, 76]
    model:
      model:
        path: ../CodeLlama-70b-Instruct-hf
- sources:
  - layer_range: [42, 66]
    model:
      model:
        path: ../CodeLlama-70b-hf
- sources:
  - layer_range: [13, 37]
    model:
      model:
        path: ../CodeLlama-70b-Python-hf
- sources:
  - layer_range: [10, 80]
    model:
      model:
        path: ../CodeLlama-70b-Instruct-hf

Stay tuned for GGUFs quants