ringoreality
commited on
Commit
•
9d69232
1
Parent(s):
28f54fd
Update README.md
Browse files
README.md
CHANGED
@@ -26,7 +26,7 @@ After downloading the HF formatted model, the HF helpers should work out-of-the-
|
|
26 |
It is also possible to evaluate the model with https://github.com/EleutherAI/lm-evaluation-harness by plugging in the model identifier ```--model_args pretrained=climategpt/climategpt-70b```.
|
27 |
|
28 |
## Training
|
29 |
-
- For the Llama2 training data, we refer the user to https://huggingface.co/meta-llama/Llama-2-70b-
|
30 |
- For continued pretraining, 4.2B climate domain tokens (tokenized by the Llama tokenizer) are used.
|
31 |
- For instruction finetuning, about 1.1B instruction-finetuning tokens (both in the climate domain but also general domain) are used.
|
32 |
|
|
|
26 |
It is also possible to evaluate the model with https://github.com/EleutherAI/lm-evaluation-harness by plugging in the model identifier ```--model_args pretrained=climategpt/climategpt-70b```.
|
27 |
|
28 |
## Training
|
29 |
+
- For the Llama2 training data, we refer the user to https://huggingface.co/meta-llama/Llama-2-70b-hf.
|
30 |
- For continued pretraining, 4.2B climate domain tokens (tokenized by the Llama tokenizer) are used.
|
31 |
- For instruction finetuning, about 1.1B instruction-finetuning tokens (both in the climate domain but also general domain) are used.
|
32 |
|