---
base_model:
- appvoid/arco
- h2oai/h2o-danube3-500m-base
library_name: transformers
tags:
- mergekit
- merge

---
# arco+

This is an untrained passthrough model based on arco and danube as a first effort to train a small enough reasoning language model that generalizes across all kind of reasoning tasks.

#### Benchmarks

| Parameters | Model                          | MMLU  | ARC | HellaSwag | PIQA   | Winogrande | Average |
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------|
| 488m       | arco-lite                      | **23.22** | 33.45 | 56.55| 69.70 | **59.19**| 48.46 |
| 773m       | arco-plus                      | 23.06 | **36.43** | **60.09**|**72.36**| **60.46**| **50.48** |

#### Configuration

The following YAML configuration was used to produce this model:

```yaml
slices:
  - sources:
    - model: appvoid/arco
      layer_range: [0, 14]
  - sources:
    - model: h2oai/h2o-danube3-500m-base
      layer_range: [4, 16]

merge_method: passthrough
dtype: float16

```