jonabur commited on
Commit ·
b9b3e0f
1
Parent(s): 8f2c8dd
update readme
Browse files
README.md
CHANGED
|
@@ -16,8 +16,6 @@ language:
|
|
| 16 |
|
| 17 |
# Viking 13B
|
| 18 |
|
| 19 |
-
_**NOTE:** This is a **research checkpoint** of a model for which **training has not been completed.** It is being provided in its current state for research and testing purposes. **Care should be taken when using the outputs of the model.** Once pretraining has completed we intend to release additional instruction-tuned and chat-tuned varieties._
|
| 20 |
-
|
| 21 |
Viking 13B is a 13B parameter decoder-only transformer pretrained on Finnish,
|
| 22 |
English, Swedish, Danish, Norwegian, Icelandic and code. It is being trained
|
| 23 |
on 2 trillion tokens (1.3 trillion as of this release). Viking 13B is a fully open source model and is made available under the Apache 2.0 License.
|
|
@@ -38,7 +36,7 @@ Viking is the second set of models released by LumiOpen and is available at
|
|
| 38 |
[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
|
| 39 |
|
| 40 |
## Model Overview
|
| 41 |
-
_**NOTE:**
|
| 42 |
|
| 43 |
Viking is a generative pretrained transformer using a LLaMA-like GPT architecture, and makes use of rotary positional embeddings and flash attention.
|
| 44 |
|
|
@@ -102,6 +100,9 @@ Training checkpoints are available as branches in the repository. Checkpoints w
|
|
| 102 |
* [1500B](https://huggingface.co/LumiOpen/Viking-13B/tree/1500B)
|
| 103 |
* [1600B](https://huggingface.co/LumiOpen/Viking-13B/tree/1600B)
|
| 104 |
* [1700B](https://huggingface.co/LumiOpen/Viking-13B/tree/1700B)
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
The transformers library allows you to load a checkpoint from a branch as follows:
|
| 107 |
|
|
|
|
| 16 |
|
| 17 |
# Viking 13B
|
| 18 |
|
|
|
|
|
|
|
| 19 |
Viking 13B is a 13B parameter decoder-only transformer pretrained on Finnish,
|
| 20 |
English, Swedish, Danish, Norwegian, Icelandic and code. It is being trained
|
| 21 |
on 2 trillion tokens (1.3 trillion as of this release). Viking 13B is a fully open source model and is made available under the Apache 2.0 License.
|
|
|
|
| 36 |
[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
|
| 37 |
|
| 38 |
## Model Overview
|
| 39 |
+
_**NOTE:** This is a base model which needs further fine tuning for most use cases._
|
| 40 |
|
| 41 |
Viking is a generative pretrained transformer using a LLaMA-like GPT architecture, and makes use of rotary positional embeddings and flash attention.
|
| 42 |
|
|
|
|
| 100 |
* [1500B](https://huggingface.co/LumiOpen/Viking-13B/tree/1500B)
|
| 101 |
* [1600B](https://huggingface.co/LumiOpen/Viking-13B/tree/1600B)
|
| 102 |
* [1700B](https://huggingface.co/LumiOpen/Viking-13B/tree/1700B)
|
| 103 |
+
* [1800B](https://huggingface.co/LumiOpen/Viking-13B/tree/1800B)
|
| 104 |
+
* [1900B](https://huggingface.co/LumiOpen/Viking-13B/tree/1900B)
|
| 105 |
+
* [2000B](https://huggingface.co/LumiOpen/Viking-13B/tree/2000B)
|
| 106 |
|
| 107 |
The transformers library allows you to load a checkpoint from a branch as follows:
|
| 108 |
|