Nanobit commited on
Commit
b31038a
1 Parent(s): c75f916

chore(readme): update instruction to set config to load from cache (#1030)

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -935,8 +935,9 @@ accelerate launch -m axolotl.cli.train your_config.yml
935
  You can optionally pre-tokenize dataset with the following before finetuning.
936
  This is recommended for large datasets.
937
 
938
- - Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface.
939
- - Use `--debug` to see preprocessed examples.
 
940
 
941
  ```bash
942
  python -m axolotl.cli.preprocess your_config.yml
 
935
  You can optionally pre-tokenize dataset with the following before finetuning.
936
  This is recommended for large datasets.
937
 
938
+ - Set `dataset_prepared_path:` to a local folder for saving and loading pre-tokenized dataset.
939
+ - (Optional): Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface.
940
+ - (Optional): Use `--debug` to see preprocessed examples.
941
 
942
  ```bash
943
  python -m axolotl.cli.preprocess your_config.yml