yhavinga
/

Boreas-10.7B-step2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

yhavinga commited on Jun 6

Commit

1228364

•

1 Parent(s): 4baaecb

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -3,7 +3,7 @@ library_name: transformers
 tags: []
 ---
-# boreas-10_7b-v1
 This is the result of step 2 of the upscaling of [Boreas-7B](https://huggingface.co/yhavinga/Boreas-7B) with [mergekit](https://github.com/cg123/mergekit).
 It is trying to reproduce the upscaling described in the [SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling](https://arxiv.org/abs/2312.15166)
@@ -13,6 +13,7 @@ This model is the result after step 2 from the figure below:
 ![SOLAR 10.7B Depth up scaling](img_2.png)
 The model was continuously pretrained on a mix of Dutch and English for 20B tokens.
 ## Model Details

 tags: []
 ---
+# Boreas-10_7B-v1
 This is the result of step 2 of the upscaling of [Boreas-7B](https://huggingface.co/yhavinga/Boreas-7B) with [mergekit](https://github.com/cg123/mergekit).
 It is trying to reproduce the upscaling described in the [SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling](https://arxiv.org/abs/2312.15166)
 ![SOLAR 10.7B Depth up scaling](img_2.png)
 The model was continuously pretrained on a mix of Dutch and English for 20B tokens.
+It must be finetuned on an instruct or chat dataset to be useful.
 ## Model Details