Upload README.md
Browse files
README.md
CHANGED
|
@@ -28,6 +28,8 @@ To quote the paper *ProgressGym: Alignment with a Millennium of Moral Progress*:
|
|
| 28 |
|
| 29 |
ProgressGym-HistLlama3-70B-C013-instruct is one of the **36 historical language models** in the ProgressGym framework.
|
| 30 |
|
|
|
|
|
|
|
| 31 |
**ProgressGym-HistLlama3-70B-C013-instruct is a 13th-century historical language model.** Based on [Meta-Llama-3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B), It is continued-pretrained on the 13th-century text data from [ProgressGym-HistText](https://huggingface.co/datasets/PKU-Alignment/ProgressGym-HistText), using the following hyperparameters:
|
| 32 |
|
| 33 |
- learning_rate: 3e-06
|
|
|
|
| 28 |
|
| 29 |
ProgressGym-HistLlama3-70B-C013-instruct is one of the **36 historical language models** in the ProgressGym framework.
|
| 30 |
|
| 31 |
+
**ProgressGym-HistLlama3-70B-C013-instruct is under continual iteration.** Improving upon the current version, new versions of the model are currently being trained to reflect historical moral tendencies in ever more comprehensive ways.
|
| 32 |
+
|
| 33 |
**ProgressGym-HistLlama3-70B-C013-instruct is a 13th-century historical language model.** Based on [Meta-Llama-3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B), It is continued-pretrained on the 13th-century text data from [ProgressGym-HistText](https://huggingface.co/datasets/PKU-Alignment/ProgressGym-HistText), using the following hyperparameters:
|
| 34 |
|
| 35 |
- learning_rate: 3e-06
|