Update README.md
Browse files
README.md
CHANGED
@@ -12,8 +12,7 @@ For anyone that is new to coding and training Ai, all your really have to edit i
|
|
12 |
3. (alpaca_prompt =) Change the prompt format, this one is setup to meet llama-3-8b-instruct format, but match it to your specifications.
|
13 |
4. (dataset = load_dataset("Replete-AI/code-test-dataset", split = "train")) What dataset you are using from huggingface
|
14 |
5. (model.push_to_hub_merged("rombodawg/test_dataset_Codellama-3-8B", tokenizer, save_method = "merged_16bit", token = ""))
|
15 |
-
|
16 |
-
For the above you need to change "rombodawg" to your Hugginface name, "test_dataset_Codellama-3-8B" to the model name you want saved as, and in token = "" you need to put your huggingface write token so the model can be saved.
|
17 |
|
18 |
|
19 |
```Python
|
@@ -126,6 +125,7 @@ regular_params = [p for p in model.parameters() if id(p) not in id_galore_params
|
|
126 |
|
127 |
param_groups = [{'params': regular_params},
|
128 |
{'params': galore_params, 'rank': 64, 'update_proj_gap': 200, 'scale': 0.25, 'proj_type': 'std'}]
|
|
|
129 |
optimizer = GaLoreAdamW8bit(param_groups, lr=2e-5)
|
130 |
|
131 |
trainer = SFTTrainer(
|
|
|
12 |
3. (alpaca_prompt =) Change the prompt format, this one is setup to meet llama-3-8b-instruct format, but match it to your specifications.
|
13 |
4. (dataset = load_dataset("Replete-AI/code-test-dataset", split = "train")) What dataset you are using from huggingface
|
14 |
5. (model.push_to_hub_merged("rombodawg/test_dataset_Codellama-3-8B", tokenizer, save_method = "merged_16bit", token = ""))
|
15 |
+
6. For the above you need to change "rombodawg" to your Hugginface name, "test_dataset_Codellama-3-8B" to the model name you want saved as, and in token = "" you need to put your huggingface write token so the model can be saved.
|
|
|
16 |
|
17 |
|
18 |
```Python
|
|
|
125 |
|
126 |
param_groups = [{'params': regular_params},
|
127 |
{'params': galore_params, 'rank': 64, 'update_proj_gap': 200, 'scale': 0.25, 'proj_type': 'std'}]
|
128 |
+
|
129 |
optimizer = GaLoreAdamW8bit(param_groups, lr=2e-5)
|
130 |
|
131 |
trainer = SFTTrainer(
|