chore(readme): update instruction to set config to load from cache (#1030)
Browse files
README.md
CHANGED
@@ -935,8 +935,9 @@ accelerate launch -m axolotl.cli.train your_config.yml
|
|
935 |
You can optionally pre-tokenize dataset with the following before finetuning.
|
936 |
This is recommended for large datasets.
|
937 |
|
938 |
-
- Set `
|
939 |
-
-
|
|
|
940 |
|
941 |
```bash
|
942 |
python -m axolotl.cli.preprocess your_config.yml
|
|
|
935 |
You can optionally pre-tokenize dataset with the following before finetuning.
|
936 |
This is recommended for large datasets.
|
937 |
|
938 |
+
- Set `dataset_prepared_path:` to a local folder for saving and loading pre-tokenized dataset.
|
939 |
+
- (Optional): Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface.
|
940 |
+
- (Optional): Use `--debug` to see preprocessed examples.
|
941 |
|
942 |
```bash
|
943 |
python -m axolotl.cli.preprocess your_config.yml
|