Hugging Face's logo

e-hossam96
/
arabic-nano-gpt-v1

Text Generation
Transformers
Safetensors
Arabic
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
e-hossam96 commited on
Commit
1965f36
•
1 Parent(s): 86c756d

End of training

Browse files
Files changed (3) hide show
  1. README.md +49 -20
  2. model.safetensors +1 -1
  3. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 8.5767
20
 
21
  ## Model description
22
 
@@ -35,33 +35,62 @@ More information needed
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
- - learning_rate: 0.0006
39
  - train_batch_size: 32
40
  - eval_batch_size: 32
41
  - seed: 42
 
 
42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
  - lr_scheduler_type: linear
44
- - num_epochs: 100
 
45
 
46
  ### Training results
47
 
48
- | Training Loss | Epoch | Step | Validation Loss |
49
- |:-------------:|:-------:|:----:|:---------------:|
50
- | 6.6922 | 6.6667 | 100 | 6.8460 |
51
- | 4.4109 | 13.3333 | 200 | 6.6570 |
52
- | 2.7038 | 20.0 | 300 | 6.9939 |
53
- | 1.427 | 26.6667 | 400 | 7.3622 |
54
- | 0.6413 | 33.3333 | 500 | 7.6590 |
55
- | 0.2914 | 40.0 | 600 | 7.9181 |
56
- | 0.1481 | 46.6667 | 700 | 8.1451 |
57
- | 0.0928 | 53.3333 | 800 | 8.2888 |
58
- | 0.069 | 60.0 | 900 | 8.3402 |
59
- | 0.055 | 66.6667 | 1000 | 8.4368 |
60
- | 0.0478 | 73.3333 | 1100 | 8.4684 |
61
- | 0.0399 | 80.0 | 1200 | 8.5143 |
62
- | 0.0363 | 86.6667 | 1300 | 8.5179 |
63
- | 0.0329 | 93.3333 | 1400 | 8.5729 |
64
- | 0.0306 | 100.0 | 1500 | 8.5767 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
 
67
  ### Framework versions
 
16
 
17
  This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 3.0267
20
 
21
  ## Model description
22
 
 
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
+ - learning_rate: 0.0002
39
  - train_batch_size: 32
40
  - eval_batch_size: 32
41
  - seed: 42
42
+ - gradient_accumulation_steps: 16
43
+ - total_train_batch_size: 512
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
+ - lr_scheduler_warmup_ratio: 0.01
47
+ - num_epochs: 24
48
 
49
  ### Training results
50
 
51
+ | Training Loss | Epoch | Step | Validation Loss |
52
+ |:-------------:|:-------:|:------:|:---------------:|
53
+ | 4.1743 | 0.5849 | 5000 | 3.6616 |
54
+ | 3.6165 | 1.1698 | 10000 | 3.4256 |
55
+ | 3.5241 | 1.7547 | 15000 | 3.3273 |
56
+ | 3.4341 | 2.3396 | 20000 | 3.2706 |
57
+ | 3.4023 | 2.9245 | 25000 | 3.2331 |
58
+ | 3.3652 | 3.5094 | 30000 | 3.2024 |
59
+ | 3.347 | 4.0943 | 35000 | 3.1826 |
60
+ | 3.3223 | 4.6791 | 40000 | 3.1637 |
61
+ | 3.3107 | 5.2640 | 45000 | 3.1526 |
62
+ | 3.2985 | 5.8489 | 50000 | 3.1370 |
63
+ | 3.2873 | 6.4338 | 55000 | 3.1296 |
64
+ | 3.2758 | 7.0187 | 60000 | 3.1190 |
65
+ | 3.2686 | 7.6036 | 65000 | 3.1105 |
66
+ | 3.2568 | 8.1885 | 70000 | 3.1042 |
67
+ | 3.2546 | 8.7734 | 75000 | 3.0982 |
68
+ | 3.248 | 9.3583 | 80000 | 3.0925 |
69
+ | 3.2431 | 9.9432 | 85000 | 3.0881 |
70
+ | 3.2371 | 10.5281 | 90000 | 3.0820 |
71
+ | 3.2346 | 11.1130 | 95000 | 3.0784 |
72
+ | 3.2273 | 11.6979 | 100000 | 3.0747 |
73
+ | 3.2207 | 12.2828 | 105000 | 3.0701 |
74
+ | 3.2191 | 12.8677 | 110000 | 3.0665 |
75
+ | 3.2148 | 13.4526 | 115000 | 3.0638 |
76
+ | 3.2132 | 14.0374 | 120000 | 3.0594 |
77
+ | 3.2079 | 14.6223 | 125000 | 3.0580 |
78
+ | 3.204 | 15.2072 | 130000 | 3.0549 |
79
+ | 3.2035 | 15.7921 | 135000 | 3.0512 |
80
+ | 3.1999 | 16.3770 | 140000 | 3.0473 |
81
+ | 3.2001 | 16.9619 | 145000 | 3.0462 |
82
+ | 3.1957 | 17.5468 | 150000 | 3.0432 |
83
+ | 3.1948 | 18.1317 | 155000 | 3.0417 |
84
+ | 3.19 | 18.7166 | 160000 | 3.0394 |
85
+ | 3.1873 | 19.3015 | 165000 | 3.0384 |
86
+ | 3.1848 | 19.8864 | 170000 | 3.0367 |
87
+ | 3.1826 | 20.4713 | 175000 | 3.0334 |
88
+ | 3.1839 | 21.0562 | 180000 | 3.0325 |
89
+ | 3.1818 | 21.6411 | 185000 | 3.0314 |
90
+ | 3.1775 | 22.2260 | 190000 | 3.0295 |
91
+ | 3.1747 | 22.8109 | 195000 | 3.0284 |
92
+ | 3.1724 | 23.3957 | 200000 | 3.0273 |
93
+ | 3.1757 | 23.9806 | 205000 | 3.0267 |
94
 
95
 
96
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e5b6cbff43b9e7c3e4ecc8514c85e7bf1985855621676169c39a859167c7055
3
  size 42555416
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6320e78744782619a6a28fdcd81e435c3bc43b9be00cc1dfc19429abd5c59d1a
3
  size 42555416
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3136559c3247ff5f3a551db29995826b7e6becd712db85d5f973849645b828fb
3
  size 5240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e2c7bd8cceec1b7f617905b6ef6d88624a19a55b7d14c1c915309fd01e54aa5
3
  size 5240