Hugging Face's logo

smangrul
/
peft-lora-starcoder15B-personal-copilot-A100-40GB-colab

PEFT
Generated from Trainer
Model card Files Community
1
smangrul commited on
Commit
e76fa5f
•
1 Parent(s): 564a819

End of training

Browse files
Files changed (1) hide show
  1. README.md +23 -13
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [bigcode/starcoder](https://huggingface.co/bigcode/starcoder) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.4471
19
 
20
  ## Model description
21
 
@@ -41,24 +41,34 @@ The following hyperparameters were used during training:
41
  - gradient_accumulation_steps: 2
42
  - total_train_batch_size: 16
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
- - lr_scheduler_type: cosine
45
  - lr_scheduler_warmup_steps: 30
46
- - training_steps: 1000
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | 0.6622 | 0.1 | 100 | 0.5890 |
53
- | 0.6234 | 0.2 | 200 | 0.5315 |
54
- | 0.6612 | 0.3 | 300 | 0.4849 |
55
- | 0.5546 | 0.4 | 400 | 0.4655 |
56
- | 0.489 | 0.5 | 500 | 0.4620 |
57
- | 0.5305 | 0.6 | 600 | 0.4541 |
58
- | 0.5022 | 0.7 | 700 | 0.4492 |
59
- | 0.577 | 0.8 | 800 | 0.4479 |
60
- | 0.4889 | 0.9 | 900 | 0.4470 |
61
- | 0.5597 | 1.0 | 1000 | 0.4471 |
 
 
 
 
 
 
 
 
 
 
62
 
63
 
64
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [bigcode/starcoder](https://huggingface.co/bigcode/starcoder) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.3633
19
 
20
  ## Model description
21
 
 
41
  - gradient_accumulation_steps: 2
42
  - total_train_batch_size: 16
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: constant
45
  - lr_scheduler_warmup_steps: 30
46
+ - training_steps: 2000
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
+ | 0.6593 | 0.05 | 100 | 0.5847 |
53
+ | 0.6226 | 0.1 | 200 | 0.5292 |
54
+ | 0.6597 | 0.15 | 300 | 0.4814 |
55
+ | 0.5523 | 0.2 | 400 | 0.4617 |
56
+ | 0.4856 | 0.25 | 500 | 0.4597 |
57
+ | 0.5237 | 0.3 | 600 | 0.4505 |
58
+ | 0.4894 | 0.35 | 700 | 0.4398 |
59
+ | 0.5579 | 0.4 | 800 | 0.4377 |
60
+ | 0.4702 | 0.45 | 900 | 0.4322 |
61
+ | 0.5418 | 0.5 | 1000 | 0.4244 |
62
+ | 0.5159 | 0.55 | 1100 | 0.4133 |
63
+ | 0.524 | 0.6 | 1200 | 0.3977 |
64
+ | 0.4138 | 0.65 | 1300 | 0.3966 |
65
+ | 0.5572 | 0.7 | 1400 | 0.3936 |
66
+ | 0.4146 | 0.75 | 1500 | 0.3904 |
67
+ | 0.7927 | 0.8 | 1600 | 0.3905 |
68
+ | 0.4131 | 0.85 | 1700 | 0.3866 |
69
+ | 0.4552 | 0.9 | 1800 | 0.3881 |
70
+ | 0.3914 | 0.95 | 1900 | 0.3794 |
71
+ | 0.4945 | 1.0 | 2000 | 0.3633 |
72
 
73
 
74
  ### Framework versions