rombodawg commited on
Commit
a1fc191
1 Parent(s): 2f66107

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -12,8 +12,7 @@ For anyone that is new to coding and training Ai, all your really have to edit i
12
  3. (alpaca_prompt =) Change the prompt format, this one is setup to meet llama-3-8b-instruct format, but match it to your specifications.
13
  4. (dataset = load_dataset("Replete-AI/code-test-dataset", split = "train")) What dataset you are using from huggingface
14
  5. (model.push_to_hub_merged("rombodawg/test_dataset_Codellama-3-8B", tokenizer, save_method = "merged_16bit", token = ""))
15
-
16
- For the above you need to change "rombodawg" to your Hugginface name, "test_dataset_Codellama-3-8B" to the model name you want saved as, and in token = "" you need to put your huggingface write token so the model can be saved.
17
 
18
 
19
  ```Python
@@ -126,6 +125,7 @@ regular_params = [p for p in model.parameters() if id(p) not in id_galore_params
126
 
127
  param_groups = [{'params': regular_params},
128
  {'params': galore_params, 'rank': 64, 'update_proj_gap': 200, 'scale': 0.25, 'proj_type': 'std'}]
 
129
  optimizer = GaLoreAdamW8bit(param_groups, lr=2e-5)
130
 
131
  trainer = SFTTrainer(
 
12
  3. (alpaca_prompt =) Change the prompt format, this one is setup to meet llama-3-8b-instruct format, but match it to your specifications.
13
  4. (dataset = load_dataset("Replete-AI/code-test-dataset", split = "train")) What dataset you are using from huggingface
14
  5. (model.push_to_hub_merged("rombodawg/test_dataset_Codellama-3-8B", tokenizer, save_method = "merged_16bit", token = ""))
15
+ 6. For the above you need to change "rombodawg" to your Hugginface name, "test_dataset_Codellama-3-8B" to the model name you want saved as, and in token = "" you need to put your huggingface write token so the model can be saved.
 
16
 
17
 
18
  ```Python
 
125
 
126
  param_groups = [{'params': regular_params},
127
  {'params': galore_params, 'rank': 64, 'update_proj_gap': 200, 'scale': 0.25, 'proj_type': 'std'}]
128
+
129
  optimizer = GaLoreAdamW8bit(param_groups, lr=2e-5)
130
 
131
  trainer = SFTTrainer(