Spaces:
Runtime error
Runtime error
Commit
·
5709163
1
Parent(s):
f9cf95c
chore: remove footnotes header
Browse files- rag/rag.py +0 -2
rag/rag.py
CHANGED
|
@@ -65,8 +65,6 @@ The main mix of the training data for the Llama 3 405 billion parameter model is
|
|
| 65 |
|
| 66 |
Regarding the amount of data used to train the model, the snippets do not provide a specific total volume of data in terms of tokens or bytes. However, they do mention that the model was pre-trained on a large dataset containing knowledge until the end of 2023[^2^]. Additionally, the training process involved pre-training on 2.87 trillion tokens before further adjustments[^3^].
|
| 67 |
|
| 68 |
-
## Footnotes
|
| 69 |
-
|
| 70 |
[^1^]: "Scaling Laws for Data Mix," page 6.
|
| 71 |
[^2^]: "Pre-Training Data," page 4.
|
| 72 |
[^3^]: "Initial Pre-Training," page 14.
|
|
|
|
| 65 |
|
| 66 |
Regarding the amount of data used to train the model, the snippets do not provide a specific total volume of data in terms of tokens or bytes. However, they do mention that the model was pre-trained on a large dataset containing knowledge until the end of 2023[^2^]. Additionally, the training process involved pre-training on 2.87 trillion tokens before further adjustments[^3^].
|
| 67 |
|
|
|
|
|
|
|
| 68 |
[^1^]: "Scaling Laws for Data Mix," page 6.
|
| 69 |
[^2^]: "Pre-Training Data," page 4.
|
| 70 |
[^3^]: "Initial Pre-Training," page 14.
|