Hugging Face's logo

smangrul
/
peft-lora-starcoderbase7b-personal-copilot-A100-40GB-colab

PEFT
Generated from Trainer
Model card Files Community
smangrul commited on
Commit
d93452d
·
1 Parent(s): ef2a27d

End of training

Browse files
Files changed (1) hide show
  1. README.md +23 -13
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [bigcode/starcoderbase-7b](https://huggingface.co/bigcode/starcoderbase-7b) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.5210
19
 
20
  ## Model description
21
 
@@ -41,24 +41,34 @@ The following hyperparameters were used during training:
41
  - gradient_accumulation_steps: 2
42
  - total_train_batch_size: 16
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
- - lr_scheduler_type: cosine
45
  - lr_scheduler_warmup_steps: 30
46
- - training_steps: 1000
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | 0.737 | 0.1 | 100 | 0.6840 |
53
- | 0.7086 | 0.2 | 200 | 0.6227 |
54
- | 0.721 | 0.3 | 300 | 0.5667 |
55
- | 0.6248 | 0.4 | 400 | 0.5434 |
56
- | 0.5456 | 0.5 | 500 | 0.5394 |
57
- | 0.5964 | 0.6 | 600 | 0.5293 |
58
- | 0.5655 | 0.7 | 700 | 0.5220 |
59
- | 0.6456 | 0.8 | 800 | 0.5213 |
60
- | 0.5496 | 0.9 | 900 | 0.5206 |
61
- | 0.6356 | 1.0 | 1000 | 0.5210 |
 
 
 
 
 
 
 
 
 
 
62
 
63
 
64
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [bigcode/starcoderbase-7b](https://huggingface.co/bigcode/starcoderbase-7b) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.4253
19
 
20
  ## Model description
21
 
 
41
  - gradient_accumulation_steps: 2
42
  - total_train_batch_size: 16
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: constant
45
  - lr_scheduler_warmup_steps: 30
46
+ - training_steps: 2000
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
+ | 0.7335 | 0.05 | 100 | 0.6803 |
53
+ | 0.7072 | 0.1 | 200 | 0.6197 |
54
+ | 0.719 | 0.15 | 300 | 0.5627 |
55
+ | 0.6217 | 0.2 | 400 | 0.5389 |
56
+ | 0.5415 | 0.25 | 500 | 0.5378 |
57
+ | 0.5881 | 0.3 | 600 | 0.5238 |
58
+ | 0.5496 | 0.35 | 700 | 0.5092 |
59
+ | 0.6226 | 0.4 | 800 | 0.5070 |
60
+ | 0.5291 | 0.45 | 900 | 0.5037 |
61
+ | 0.612 | 0.5 | 1000 | 0.4923 |
62
+ | 0.5752 | 0.55 | 1100 | 0.4780 |
63
+ | 0.5921 | 0.6 | 1200 | 0.4620 |
64
+ | 0.4729 | 0.65 | 1300 | 0.4635 |
65
+ | 0.6316 | 0.7 | 1400 | 0.4600 |
66
+ | 0.4738 | 0.75 | 1500 | 0.4574 |
67
+ | 0.8665 | 0.8 | 1600 | 0.4555 |
68
+ | 0.4805 | 0.85 | 1700 | 0.4509 |
69
+ | 0.515 | 0.9 | 1800 | 0.4531 |
70
+ | 0.4674 | 0.95 | 1900 | 0.4402 |
71
+ | 0.5611 | 1.0 | 2000 | 0.4253 |
72
 
73
 
74
  ### Framework versions