Hugging Face's logo

modelteam-ai
/
lop_dec2024

PEFT
Safetensors
Model card Files Community
obulikrish commited on
Commit
b98a7c2
•
1 Parent(s): 6802c7e

lop_dec2024

Browse files
Files changed (4) hide show
  1. README.md +202 -0
  2. adapter_config.json +36 -0
  3. adapter_model.safetensors +3 -0
  4. trainer_state.json +3830 -0
README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Salesforce/codet5p-770m
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.11.0
adapter_config.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Salesforce/codet5p-770m",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 64,
14
+ "lora_dropout": 0.1,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": [
18
+ "embed_tokens"
19
+ ],
20
+ "peft_type": "LORA",
21
+ "r": 32,
22
+ "rank_pattern": {},
23
+ "revision": null,
24
+ "target_modules": [
25
+ "v",
26
+ "lm_head",
27
+ "o",
28
+ "q",
29
+ "wo",
30
+ "k",
31
+ "wi"
32
+ ],
33
+ "task_type": "SEQ_2_SEQ_LM",
34
+ "use_dora": false,
35
+ "use_rslora": false
36
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f56f5db0055abf670aaa7b6258ebaf24ae65669bbb4e85a6b6fc5589d5372635
3
+ size 274252864
trainer_state.json ADDED
@@ -0,0 +1,3830 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.3201138377189636,
3
+ "best_model_checkpoint": "saved_model/lop_dec2024/checkpoint-5422",
4
+ "epoch": 0.9999538936788234,
5
+ "eval_steps": 500,
6
+ "global_step": 5422,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "grad_norm": NaN,
14
+ "learning_rate": 0.0,
15
+ "loss": 72.616,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.0,
20
+ "grad_norm": 15.978808403015137,
21
+ "learning_rate": 3e-06,
22
+ "loss": 72.0656,
23
+ "step": 10
24
+ },
25
+ {
26
+ "epoch": 0.0,
27
+ "grad_norm": 16.57687759399414,
28
+ "learning_rate": 7.000000000000001e-06,
29
+ "loss": 71.4774,
30
+ "step": 20
31
+ },
32
+ {
33
+ "epoch": 0.01,
34
+ "grad_norm": 17.607620239257812,
35
+ "learning_rate": 1.2e-05,
36
+ "loss": 70.0161,
37
+ "step": 30
38
+ },
39
+ {
40
+ "epoch": 0.01,
41
+ "grad_norm": 20.96339988708496,
42
+ "learning_rate": 1.7000000000000003e-05,
43
+ "loss": 66.9798,
44
+ "step": 40
45
+ },
46
+ {
47
+ "epoch": 0.01,
48
+ "grad_norm": 28.76253890991211,
49
+ "learning_rate": 2.15e-05,
50
+ "loss": 61.5471,
51
+ "step": 50
52
+ },
53
+ {
54
+ "epoch": 0.01,
55
+ "grad_norm": 34.237281799316406,
56
+ "learning_rate": 2.6500000000000004e-05,
57
+ "loss": 50.4159,
58
+ "step": 60
59
+ },
60
+ {
61
+ "epoch": 0.01,
62
+ "grad_norm": 37.09980773925781,
63
+ "learning_rate": 3.15e-05,
64
+ "loss": 30.1299,
65
+ "step": 70
66
+ },
67
+ {
68
+ "epoch": 0.01,
69
+ "grad_norm": 21.59116554260254,
70
+ "learning_rate": 3.65e-05,
71
+ "loss": 10.0018,
72
+ "step": 80
73
+ },
74
+ {
75
+ "epoch": 0.02,
76
+ "grad_norm": 4.791250705718994,
77
+ "learning_rate": 4.15e-05,
78
+ "loss": 1.4075,
79
+ "step": 90
80
+ },
81
+ {
82
+ "epoch": 0.02,
83
+ "grad_norm": 6.963352203369141,
84
+ "learning_rate": 4.6500000000000005e-05,
85
+ "loss": 0.6076,
86
+ "step": 100
87
+ },
88
+ {
89
+ "epoch": 0.02,
90
+ "grad_norm": 8.645480155944824,
91
+ "learning_rate": 5.1500000000000005e-05,
92
+ "loss": 0.6529,
93
+ "step": 110
94
+ },
95
+ {
96
+ "epoch": 0.02,
97
+ "grad_norm": 10.286620140075684,
98
+ "learning_rate": 5.65e-05,
99
+ "loss": 0.606,
100
+ "step": 120
101
+ },
102
+ {
103
+ "epoch": 0.02,
104
+ "grad_norm": 4.937899589538574,
105
+ "learning_rate": 6.15e-05,
106
+ "loss": 0.6646,
107
+ "step": 130
108
+ },
109
+ {
110
+ "epoch": 0.03,
111
+ "grad_norm": 5.505192279815674,
112
+ "learning_rate": 6.65e-05,
113
+ "loss": 0.5808,
114
+ "step": 140
115
+ },
116
+ {
117
+ "epoch": 0.03,
118
+ "grad_norm": 2.3285677433013916,
119
+ "learning_rate": 7.15e-05,
120
+ "loss": 0.4994,
121
+ "step": 150
122
+ },
123
+ {
124
+ "epoch": 0.03,
125
+ "grad_norm": 1.4964683055877686,
126
+ "learning_rate": 7.65e-05,
127
+ "loss": 0.4659,
128
+ "step": 160
129
+ },
130
+ {
131
+ "epoch": 0.03,
132
+ "grad_norm": 6.808905124664307,
133
+ "learning_rate": 8.15e-05,
134
+ "loss": 0.4923,
135
+ "step": 170
136
+ },
137
+ {
138
+ "epoch": 0.03,
139
+ "grad_norm": 1.631983757019043,
140
+ "learning_rate": 8.65e-05,
141
+ "loss": 0.4065,
142
+ "step": 180
143
+ },
144
+ {
145
+ "epoch": 0.04,
146
+ "grad_norm": 4.93166971206665,
147
+ "learning_rate": 9.15e-05,
148
+ "loss": 0.4412,
149
+ "step": 190
150
+ },
151
+ {
152
+ "epoch": 0.04,
153
+ "grad_norm": 1.9249387979507446,
154
+ "learning_rate": 9.65e-05,
155
+ "loss": 0.3778,
156
+ "step": 200
157
+ },
158
+ {
159
+ "epoch": 0.04,
160
+ "grad_norm": 3.0214664936065674,
161
+ "learning_rate": 9.999444650129582e-05,
162
+ "loss": 0.3697,
163
+ "step": 210
164
+ },
165
+ {
166
+ "epoch": 0.04,
167
+ "grad_norm": 0.8178681135177612,
168
+ "learning_rate": 9.997593483894854e-05,
169
+ "loss": 0.3376,
170
+ "step": 220
171
+ },
172
+ {
173
+ "epoch": 0.04,
174
+ "grad_norm": 3.6914970874786377,
175
+ "learning_rate": 9.995742317660126e-05,
176
+ "loss": 0.3389,
177
+ "step": 230
178
+ },
179
+ {
180
+ "epoch": 0.04,
181
+ "grad_norm": 1.8259241580963135,
182
+ "learning_rate": 9.993891151425399e-05,
183
+ "loss": 0.3186,
184
+ "step": 240
185
+ },
186
+ {
187
+ "epoch": 0.05,
188
+ "grad_norm": 2.6828951835632324,
189
+ "learning_rate": 9.99203998519067e-05,
190
+ "loss": 0.3228,
191
+ "step": 250
192
+ },
193
+ {
194
+ "epoch": 0.05,
195
+ "grad_norm": 1.0691156387329102,
196
+ "learning_rate": 9.990188818955942e-05,
197
+ "loss": 0.3243,
198
+ "step": 260
199
+ },
200
+ {
201
+ "epoch": 0.05,
202
+ "grad_norm": 1.3478268384933472,
203
+ "learning_rate": 9.988337652721215e-05,
204
+ "loss": 0.3061,
205
+ "step": 270
206
+ },
207
+ {
208
+ "epoch": 0.05,
209
+ "grad_norm": 1.275899887084961,
210
+ "learning_rate": 9.986486486486487e-05,
211
+ "loss": 0.297,
212
+ "step": 280
213
+ },
214
+ {
215
+ "epoch": 0.05,
216
+ "grad_norm": 3.873141050338745,
217
+ "learning_rate": 9.98463532025176e-05,
218
+ "loss": 0.3115,
219
+ "step": 290
220
+ },
221
+ {
222
+ "epoch": 0.06,
223
+ "grad_norm": 0.8778353929519653,
224
+ "learning_rate": 9.98278415401703e-05,
225
+ "loss": 0.3045,
226
+ "step": 300
227
+ },
228
+ {
229
+ "epoch": 0.06,
230
+ "grad_norm": 2.3832242488861084,
231
+ "learning_rate": 9.980932987782303e-05,
232
+ "loss": 0.3058,
233
+ "step": 310
234
+ },
235
+ {
236
+ "epoch": 0.06,
237
+ "grad_norm": 0.6602181196212769,
238
+ "learning_rate": 9.979081821547575e-05,
239
+ "loss": 0.2714,
240
+ "step": 320
241
+ },
242
+ {
243
+ "epoch": 0.06,
244
+ "grad_norm": 0.796381950378418,
245
+ "learning_rate": 9.977230655312848e-05,
246
+ "loss": 0.2911,
247
+ "step": 330
248
+ },
249
+ {
250
+ "epoch": 0.06,
251
+ "grad_norm": 1.6985359191894531,
252
+ "learning_rate": 9.97537948907812e-05,
253
+ "loss": 0.2827,
254
+ "step": 340
255
+ },
256
+ {
257
+ "epoch": 0.06,
258
+ "grad_norm": 0.9519887566566467,
259
+ "learning_rate": 9.973528322843392e-05,
260
+ "loss": 0.2828,
261
+ "step": 350
262
+ },
263
+ {
264
+ "epoch": 0.07,
265
+ "grad_norm": 0.771391749382019,
266
+ "learning_rate": 9.971677156608665e-05,
267
+ "loss": 0.274,
268
+ "step": 360
269
+ },
270
+ {
271
+ "epoch": 0.07,
272
+ "grad_norm": 1.5956106185913086,
273
+ "learning_rate": 9.969825990373937e-05,
274
+ "loss": 0.2509,
275
+ "step": 370
276
+ },
277
+ {
278
+ "epoch": 0.07,
279
+ "grad_norm": 0.8278146982192993,
280
+ "learning_rate": 9.967974824139208e-05,
281
+ "loss": 0.2644,
282
+ "step": 380
283
+ },
284
+ {
285
+ "epoch": 0.07,
286
+ "grad_norm": 3.159097671508789,
287
+ "learning_rate": 9.96612365790448e-05,
288
+ "loss": 0.2855,
289
+ "step": 390
290
+ },
291
+ {
292
+ "epoch": 0.07,
293
+ "grad_norm": 2.152933120727539,
294
+ "learning_rate": 9.964272491669753e-05,
295
+ "loss": 0.2755,
296
+ "step": 400
297
+ },
298
+ {
299
+ "epoch": 0.08,
300
+ "grad_norm": 1.9050334692001343,
301
+ "learning_rate": 9.962421325435023e-05,
302
+ "loss": 0.2837,
303
+ "step": 410
304
+ },
305
+ {
306
+ "epoch": 0.08,
307
+ "grad_norm": 1.3858613967895508,
308
+ "learning_rate": 9.960570159200297e-05,
309
+ "loss": 0.2584,
310
+ "step": 420
311
+ },
312
+ {
313
+ "epoch": 0.08,
314
+ "grad_norm": 0.629534125328064,
315
+ "learning_rate": 9.958718992965568e-05,
316
+ "loss": 0.2432,
317
+ "step": 430
318
+ },
319
+ {
320
+ "epoch": 0.08,
321
+ "grad_norm": 0.7582938075065613,
322
+ "learning_rate": 9.956867826730841e-05,
323
+ "loss": 0.242,
324
+ "step": 440
325
+ },
326
+ {
327
+ "epoch": 0.08,
328
+ "grad_norm": 0.5532256364822388,
329
+ "learning_rate": 9.955016660496113e-05,
330
+ "loss": 0.2828,
331
+ "step": 450
332
+ },
333
+ {
334
+ "epoch": 0.08,
335
+ "grad_norm": 0.8100008964538574,
336
+ "learning_rate": 9.953165494261385e-05,
337
+ "loss": 0.2391,
338
+ "step": 460
339
+ },
340
+ {
341
+ "epoch": 0.09,
342
+ "grad_norm": 1.3843342065811157,
343
+ "learning_rate": 9.951314328026658e-05,
344
+ "loss": 0.2513,
345
+ "step": 470
346
+ },
347
+ {
348
+ "epoch": 0.09,
349
+ "grad_norm": 2.4204509258270264,
350
+ "learning_rate": 9.94946316179193e-05,
351
+ "loss": 0.2378,
352
+ "step": 480
353
+ },
354
+ {
355
+ "epoch": 0.09,
356
+ "grad_norm": 1.645175576210022,
357
+ "learning_rate": 9.947611995557201e-05,
358
+ "loss": 0.2704,
359
+ "step": 490
360
+ },
361
+ {
362
+ "epoch": 0.09,
363
+ "grad_norm": 2.546315908432007,
364
+ "learning_rate": 9.945760829322473e-05,
365
+ "loss": 0.2762,
366
+ "step": 500
367
+ },
368
+ {
369
+ "epoch": 0.09,
370
+ "grad_norm": 3.5709307193756104,
371
+ "learning_rate": 9.943909663087746e-05,
372
+ "loss": 0.2753,
373
+ "step": 510
374
+ },
375
+ {
376
+ "epoch": 0.1,
377
+ "grad_norm": 0.6997337341308594,
378
+ "learning_rate": 9.942058496853018e-05,
379
+ "loss": 0.2637,
380
+ "step": 520
381
+ },
382
+ {
383
+ "epoch": 0.1,
384
+ "grad_norm": 0.6340448260307312,
385
+ "learning_rate": 9.94020733061829e-05,
386
+ "loss": 0.2447,
387
+ "step": 530
388
+ },
389
+ {
390
+ "epoch": 0.1,
391
+ "grad_norm": 1.1261272430419922,
392
+ "learning_rate": 9.938356164383563e-05,
393
+ "loss": 0.2345,
394
+ "step": 540
395
+ },
396
+ {
397
+ "epoch": 0.1,
398
+ "grad_norm": 0.7460525631904602,
399
+ "learning_rate": 9.936504998148834e-05,
400
+ "loss": 0.2356,
401
+ "step": 550
402
+ },
403
+ {
404
+ "epoch": 0.1,
405
+ "grad_norm": 0.7596022486686707,
406
+ "learning_rate": 9.934653831914106e-05,
407
+ "loss": 0.2497,
408
+ "step": 560
409
+ },
410
+ {
411
+ "epoch": 0.11,
412
+ "grad_norm": 0.7767930030822754,
413
+ "learning_rate": 9.932802665679378e-05,
414
+ "loss": 0.2332,
415
+ "step": 570
416
+ },
417
+ {
418
+ "epoch": 0.11,
419
+ "grad_norm": 0.36362847685813904,
420
+ "learning_rate": 9.930951499444651e-05,
421
+ "loss": 0.234,
422
+ "step": 580
423
+ },
424
+ {
425
+ "epoch": 0.11,
426
+ "grad_norm": 1.1732509136199951,
427
+ "learning_rate": 9.929100333209923e-05,
428
+ "loss": 0.2281,
429
+ "step": 590
430
+ },
431
+ {
432
+ "epoch": 0.11,
433
+ "grad_norm": 0.9395415782928467,
434
+ "learning_rate": 9.927249166975194e-05,
435
+ "loss": 0.2262,
436
+ "step": 600
437
+ },
438
+ {
439
+ "epoch": 0.11,
440
+ "grad_norm": 1.7397685050964355,
441
+ "learning_rate": 9.925398000740466e-05,
442
+ "loss": 0.2375,
443
+ "step": 610
444
+ },
445
+ {
446
+ "epoch": 0.11,
447
+ "grad_norm": 1.398878812789917,
448
+ "learning_rate": 9.923546834505739e-05,
449
+ "loss": 0.2408,
450
+ "step": 620
451
+ },
452
+ {
453
+ "epoch": 0.12,
454
+ "grad_norm": 1.213331937789917,
455
+ "learning_rate": 9.921695668271012e-05,
456
+ "loss": 0.2402,
457
+ "step": 630
458
+ },
459
+ {
460
+ "epoch": 0.12,
461
+ "grad_norm": 0.7745940685272217,
462
+ "learning_rate": 9.919844502036283e-05,
463
+ "loss": 0.259,
464
+ "step": 640
465
+ },
466
+ {
467
+ "epoch": 0.12,
468
+ "grad_norm": 0.9329665303230286,
469
+ "learning_rate": 9.917993335801556e-05,
470
+ "loss": 0.2457,
471
+ "step": 650
472
+ },
473
+ {
474
+ "epoch": 0.12,
475
+ "grad_norm": 0.9290269613265991,
476
+ "learning_rate": 9.916142169566827e-05,
477
+ "loss": 0.2236,
478
+ "step": 660
479
+ },
480
+ {
481
+ "epoch": 0.12,
482
+ "grad_norm": 1.089816689491272,
483
+ "learning_rate": 9.9142910033321e-05,
484
+ "loss": 0.2437,
485
+ "step": 670
486
+ },
487
+ {
488
+ "epoch": 0.13,
489
+ "grad_norm": 0.6095532178878784,
490
+ "learning_rate": 9.912439837097371e-05,
491
+ "loss": 0.2295,
492
+ "step": 680
493
+ },
494
+ {
495
+ "epoch": 0.13,
496
+ "grad_norm": 0.6958633065223694,
497
+ "learning_rate": 9.910588670862644e-05,
498
+ "loss": 0.231,
499
+ "step": 690
500
+ },
501
+ {
502
+ "epoch": 0.13,
503
+ "grad_norm": 0.5149015784263611,
504
+ "learning_rate": 9.908737504627916e-05,
505
+ "loss": 0.2567,
506
+ "step": 700
507
+ },
508
+ {
509
+ "epoch": 0.13,
510
+ "grad_norm": 0.9667665362358093,
511
+ "learning_rate": 9.906886338393189e-05,
512
+ "loss": 0.2324,
513
+ "step": 710
514
+ },
515
+ {
516
+ "epoch": 0.13,
517
+ "grad_norm": 0.8621126413345337,
518
+ "learning_rate": 9.90503517215846e-05,
519
+ "loss": 0.2349,
520
+ "step": 720
521
+ },
522
+ {
523
+ "epoch": 0.13,
524
+ "grad_norm": 0.7839102149009705,
525
+ "learning_rate": 9.903184005923732e-05,
526
+ "loss": 0.2395,
527
+ "step": 730
528
+ },
529
+ {
530
+ "epoch": 0.14,
531
+ "grad_norm": 0.9927177429199219,
532
+ "learning_rate": 9.901332839689005e-05,
533
+ "loss": 0.2253,
534
+ "step": 740
535
+ },
536
+ {
537
+ "epoch": 0.14,
538
+ "grad_norm": 1.7064800262451172,
539
+ "learning_rate": 9.899481673454276e-05,
540
+ "loss": 0.2178,
541
+ "step": 750
542
+ },
543
+ {
544
+ "epoch": 0.14,
545
+ "grad_norm": 1.4743956327438354,
546
+ "learning_rate": 9.897630507219549e-05,
547
+ "loss": 0.2186,
548
+ "step": 760
549
+ },
550
+ {
551
+ "epoch": 0.14,
552
+ "grad_norm": 1.73115873336792,
553
+ "learning_rate": 9.89577934098482e-05,
554
+ "loss": 0.2406,
555
+ "step": 770
556
+ },
557
+ {
558
+ "epoch": 0.14,
559
+ "grad_norm": 0.5700548887252808,
560
+ "learning_rate": 9.893928174750094e-05,
561
+ "loss": 0.2368,
562
+ "step": 780
563
+ },
564
+ {
565
+ "epoch": 0.15,
566
+ "grad_norm": 0.7140907049179077,
567
+ "learning_rate": 9.892077008515365e-05,
568
+ "loss": 0.2303,
569
+ "step": 790
570
+ },
571
+ {
572
+ "epoch": 0.15,
573
+ "grad_norm": 1.316851019859314,
574
+ "learning_rate": 9.890225842280637e-05,
575
+ "loss": 0.2271,
576
+ "step": 800
577
+ },
578
+ {
579
+ "epoch": 0.15,
580
+ "grad_norm": 1.3242087364196777,
581
+ "learning_rate": 9.88837467604591e-05,
582
+ "loss": 0.2213,
583
+ "step": 810
584
+ },
585
+ {
586
+ "epoch": 0.15,
587
+ "grad_norm": 0.3230711817741394,
588
+ "learning_rate": 9.886523509811182e-05,
589
+ "loss": 0.2366,
590
+ "step": 820
591
+ },
592
+ {
593
+ "epoch": 0.15,
594
+ "grad_norm": 1.613010287284851,
595
+ "learning_rate": 9.884672343576454e-05,
596
+ "loss": 0.2172,
597
+ "step": 830
598
+ },
599
+ {
600
+ "epoch": 0.15,
601
+ "grad_norm": 2.3217711448669434,
602
+ "learning_rate": 9.882821177341725e-05,
603
+ "loss": 0.2401,
604
+ "step": 840
605
+ },
606
+ {
607
+ "epoch": 0.16,
608
+ "grad_norm": 0.38820984959602356,
609
+ "learning_rate": 9.880970011106998e-05,
610
+ "loss": 0.2376,
611
+ "step": 850
612
+ },
613
+ {
614
+ "epoch": 0.16,
615
+ "grad_norm": 0.5649935007095337,
616
+ "learning_rate": 9.87911884487227e-05,
617
+ "loss": 0.2335,
618
+ "step": 860
619
+ },
620
+ {
621
+ "epoch": 0.16,
622
+ "grad_norm": 0.5969343781471252,
623
+ "learning_rate": 9.877267678637542e-05,
624
+ "loss": 0.228,
625
+ "step": 870
626
+ },
627
+ {
628
+ "epoch": 0.16,
629
+ "grad_norm": 1.4941283464431763,
630
+ "learning_rate": 9.875416512402814e-05,
631
+ "loss": 0.2388,
632
+ "step": 880
633
+ },
634
+ {
635
+ "epoch": 0.16,
636
+ "grad_norm": 0.7500633597373962,
637
+ "learning_rate": 9.873565346168087e-05,
638
+ "loss": 0.2203,
639
+ "step": 890
640
+ },
641
+ {
642
+ "epoch": 0.17,
643
+ "grad_norm": 0.6874450445175171,
644
+ "learning_rate": 9.871714179933358e-05,
645
+ "loss": 0.2397,
646
+ "step": 900
647
+ },
648
+ {
649
+ "epoch": 0.17,
650
+ "grad_norm": 1.4555752277374268,
651
+ "learning_rate": 9.86986301369863e-05,
652
+ "loss": 0.2418,
653
+ "step": 910
654
+ },
655
+ {
656
+ "epoch": 0.17,
657
+ "grad_norm": 1.051475167274475,
658
+ "learning_rate": 9.868011847463903e-05,
659
+ "loss": 0.2411,
660
+ "step": 920
661
+ },
662
+ {
663
+ "epoch": 0.17,
664
+ "grad_norm": 1.377054214477539,
665
+ "learning_rate": 9.866160681229175e-05,
666
+ "loss": 0.2312,
667
+ "step": 930
668
+ },
669
+ {
670
+ "epoch": 0.17,
671
+ "grad_norm": 0.4420924186706543,
672
+ "learning_rate": 9.864309514994447e-05,
673
+ "loss": 0.2179,
674
+ "step": 940
675
+ },
676
+ {
677
+ "epoch": 0.18,
678
+ "grad_norm": 1.9395197629928589,
679
+ "learning_rate": 9.862458348759718e-05,
680
+ "loss": 0.2286,
681
+ "step": 950
682
+ },
683
+ {
684
+ "epoch": 0.18,
685
+ "grad_norm": 0.31522631645202637,
686
+ "learning_rate": 9.860607182524991e-05,
687
+ "loss": 0.2267,
688
+ "step": 960
689
+ },
690
+ {
691
+ "epoch": 0.18,
692
+ "grad_norm": 0.4355124831199646,
693
+ "learning_rate": 9.858756016290263e-05,
694
+ "loss": 0.228,
695
+ "step": 970
696
+ },
697
+ {
698
+ "epoch": 0.18,
699
+ "grad_norm": 0.5540971159934998,
700
+ "learning_rate": 9.856904850055535e-05,
701
+ "loss": 0.2202,
702
+ "step": 980
703
+ },
704
+ {
705
+ "epoch": 0.18,
706
+ "grad_norm": 0.43327707052230835,
707
+ "learning_rate": 9.855053683820808e-05,
708
+ "loss": 0.2316,
709
+ "step": 990
710
+ },
711
+ {
712
+ "epoch": 0.18,
713
+ "grad_norm": 1.765080451965332,
714
+ "learning_rate": 9.85320251758608e-05,
715
+ "loss": 0.2302,
716
+ "step": 1000
717
+ },
718
+ {
719
+ "epoch": 0.19,
720
+ "grad_norm": 0.822860598564148,
721
+ "learning_rate": 9.851351351351353e-05,
722
+ "loss": 0.2213,
723
+ "step": 1010
724
+ },
725
+ {
726
+ "epoch": 0.19,
727
+ "grad_norm": 0.31268832087516785,
728
+ "learning_rate": 9.849500185116623e-05,
729
+ "loss": 0.2065,
730
+ "step": 1020
731
+ },
732
+ {
733
+ "epoch": 0.19,
734
+ "grad_norm": 1.3371509313583374,
735
+ "learning_rate": 9.847649018881896e-05,
736
+ "loss": 0.2261,
737
+ "step": 1030
738
+ },
739
+ {
740
+ "epoch": 0.19,
741
+ "grad_norm": 0.3734799325466156,
742
+ "learning_rate": 9.845797852647168e-05,
743
+ "loss": 0.2228,
744
+ "step": 1040
745
+ },
746
+ {
747
+ "epoch": 0.19,
748
+ "grad_norm": 0.45795735716819763,
749
+ "learning_rate": 9.843946686412441e-05,
750
+ "loss": 0.223,
751
+ "step": 1050
752
+ },
753
+ {
754
+ "epoch": 0.2,
755
+ "grad_norm": 0.4650135040283203,
756
+ "learning_rate": 9.842095520177711e-05,
757
+ "loss": 0.2301,
758
+ "step": 1060
759
+ },
760
+ {
761
+ "epoch": 0.2,
762
+ "grad_norm": 1.4706295728683472,
763
+ "learning_rate": 9.840244353942985e-05,
764
+ "loss": 0.2222,
765
+ "step": 1070
766
+ },
767
+ {
768
+ "epoch": 0.2,
769
+ "grad_norm": 0.3891691565513611,
770
+ "learning_rate": 9.838393187708256e-05,
771
+ "loss": 0.2103,
772
+ "step": 1080
773
+ },
774
+ {
775
+ "epoch": 0.2,
776
+ "grad_norm": 1.0710134506225586,
777
+ "learning_rate": 9.836542021473529e-05,
778
+ "loss": 0.2242,
779
+ "step": 1090
780
+ },
781
+ {
782
+ "epoch": 0.2,
783
+ "grad_norm": 0.9032573699951172,
784
+ "learning_rate": 9.834690855238801e-05,
785
+ "loss": 0.2124,
786
+ "step": 1100
787
+ },
788
+ {
789
+ "epoch": 0.2,
790
+ "grad_norm": 2.119032859802246,
791
+ "learning_rate": 9.832839689004073e-05,
792
+ "loss": 0.2289,
793
+ "step": 1110
794
+ },
795
+ {
796
+ "epoch": 0.21,
797
+ "grad_norm": 0.8338161110877991,
798
+ "learning_rate": 9.830988522769346e-05,
799
+ "loss": 0.2167,
800
+ "step": 1120
801
+ },
802
+ {
803
+ "epoch": 0.21,
804
+ "grad_norm": 2.242032527923584,
805
+ "learning_rate": 9.829137356534616e-05,
806
+ "loss": 0.2225,
807
+ "step": 1130
808
+ },
809
+ {
810
+ "epoch": 0.21,
811
+ "grad_norm": 0.4238918125629425,
812
+ "learning_rate": 9.827286190299889e-05,
813
+ "loss": 0.2205,
814
+ "step": 1140
815
+ },
816
+ {
817
+ "epoch": 0.21,
818
+ "grad_norm": 0.5318244099617004,
819
+ "learning_rate": 9.825435024065161e-05,
820
+ "loss": 0.222,
821
+ "step": 1150
822
+ },
823
+ {
824
+ "epoch": 0.21,
825
+ "grad_norm": 0.75505131483078,
826
+ "learning_rate": 9.823583857830434e-05,
827
+ "loss": 0.2311,
828
+ "step": 1160
829
+ },
830
+ {
831
+ "epoch": 0.22,
832
+ "grad_norm": 1.4188501834869385,
833
+ "learning_rate": 9.821732691595706e-05,
834
+ "loss": 0.2149,
835
+ "step": 1170
836
+ },
837
+ {
838
+ "epoch": 0.22,
839
+ "grad_norm": 0.6075870990753174,
840
+ "learning_rate": 9.819881525360978e-05,
841
+ "loss": 0.2233,
842
+ "step": 1180
843
+ },
844
+ {
845
+ "epoch": 0.22,
846
+ "grad_norm": 1.4305758476257324,
847
+ "learning_rate": 9.81803035912625e-05,
848
+ "loss": 0.2238,
849
+ "step": 1190
850
+ },
851
+ {
852
+ "epoch": 0.22,
853
+ "grad_norm": 0.5145848989486694,
854
+ "learning_rate": 9.816179192891522e-05,
855
+ "loss": 0.2217,
856
+ "step": 1200
857
+ },
858
+ {
859
+ "epoch": 0.22,
860
+ "grad_norm": 0.7991466522216797,
861
+ "learning_rate": 9.814328026656794e-05,
862
+ "loss": 0.2118,
863
+ "step": 1210
864
+ },
865
+ {
866
+ "epoch": 0.22,
867
+ "grad_norm": 1.0274105072021484,
868
+ "learning_rate": 9.812476860422066e-05,
869
+ "loss": 0.2178,
870
+ "step": 1220
871
+ },
872
+ {
873
+ "epoch": 0.23,
874
+ "grad_norm": 0.4420272409915924,
875
+ "learning_rate": 9.810625694187339e-05,
876
+ "loss": 0.219,
877
+ "step": 1230
878
+ },
879
+ {
880
+ "epoch": 0.23,
881
+ "grad_norm": 0.5142613053321838,
882
+ "learning_rate": 9.80877452795261e-05,
883
+ "loss": 0.2123,
884
+ "step": 1240
885
+ },
886
+ {
887
+ "epoch": 0.23,
888
+ "grad_norm": 0.32571473717689514,
889
+ "learning_rate": 9.806923361717882e-05,
890
+ "loss": 0.2115,
891
+ "step": 1250
892
+ },
893
+ {
894
+ "epoch": 0.23,
895
+ "grad_norm": 0.3837771713733673,
896
+ "learning_rate": 9.805072195483155e-05,
897
+ "loss": 0.1992,
898
+ "step": 1260
899
+ },
900
+ {
901
+ "epoch": 0.23,
902
+ "grad_norm": 1.0336960554122925,
903
+ "learning_rate": 9.803221029248427e-05,
904
+ "loss": 0.2109,
905
+ "step": 1270
906
+ },
907
+ {
908
+ "epoch": 0.24,
909
+ "grad_norm": 1.2124392986297607,
910
+ "learning_rate": 9.8013698630137e-05,
911
+ "loss": 0.2422,
912
+ "step": 1280
913
+ },
914
+ {
915
+ "epoch": 0.24,
916
+ "grad_norm": 1.8734978437423706,
917
+ "learning_rate": 9.79951869677897e-05,
918
+ "loss": 0.2264,
919
+ "step": 1290
920
+ },
921
+ {
922
+ "epoch": 0.24,
923
+ "grad_norm": 0.44908058643341064,
924
+ "learning_rate": 9.797667530544244e-05,
925
+ "loss": 0.2123,
926
+ "step": 1300
927
+ },
928
+ {
929
+ "epoch": 0.24,
930
+ "grad_norm": 0.4297716021537781,
931
+ "learning_rate": 9.795816364309515e-05,
932
+ "loss": 0.2249,
933
+ "step": 1310
934
+ },
935
+ {
936
+ "epoch": 0.24,
937
+ "grad_norm": 1.1651443243026733,
938
+ "learning_rate": 9.793965198074787e-05,
939
+ "loss": 0.2222,
940
+ "step": 1320
941
+ },
942
+ {
943
+ "epoch": 0.25,
944
+ "grad_norm": 0.3527076542377472,
945
+ "learning_rate": 9.792114031840059e-05,
946
+ "loss": 0.2206,
947
+ "step": 1330
948
+ },
949
+ {
950
+ "epoch": 0.25,
951
+ "grad_norm": 0.29916948080062866,
952
+ "learning_rate": 9.790262865605332e-05,
953
+ "loss": 0.2209,
954
+ "step": 1340
955
+ },
956
+ {
957
+ "epoch": 0.25,
958
+ "grad_norm": 0.24127215147018433,
959
+ "learning_rate": 9.788411699370604e-05,
960
+ "loss": 0.2167,
961
+ "step": 1350
962
+ },
963
+ {
964
+ "epoch": 0.25,
965
+ "grad_norm": 0.4984634220600128,
966
+ "learning_rate": 9.786560533135875e-05,
967
+ "loss": 0.2493,
968
+ "step": 1360
969
+ },
970
+ {
971
+ "epoch": 0.25,
972
+ "grad_norm": 0.36658474802970886,
973
+ "learning_rate": 9.784709366901149e-05,
974
+ "loss": 0.2203,
975
+ "step": 1370
976
+ },
977
+ {
978
+ "epoch": 0.25,
979
+ "grad_norm": 1.2130411863327026,
980
+ "learning_rate": 9.78285820066642e-05,
981
+ "loss": 0.2142,
982
+ "step": 1380
983
+ },
984
+ {
985
+ "epoch": 0.26,
986
+ "grad_norm": 1.1353245973587036,
987
+ "learning_rate": 9.781007034431693e-05,
988
+ "loss": 0.2157,
989
+ "step": 1390
990
+ },
991
+ {
992
+ "epoch": 0.26,
993
+ "grad_norm": 0.41700485348701477,
994
+ "learning_rate": 9.779155868196964e-05,
995
+ "loss": 0.2141,
996
+ "step": 1400
997
+ },
998
+ {
999
+ "epoch": 0.26,
1000
+ "grad_norm": 1.722000002861023,
1001
+ "learning_rate": 9.777304701962237e-05,
1002
+ "loss": 0.2137,
1003
+ "step": 1410
1004
+ },
1005
+ {
1006
+ "epoch": 0.26,
1007
+ "grad_norm": 0.7909780144691467,
1008
+ "learning_rate": 9.775453535727508e-05,
1009
+ "loss": 0.2136,
1010
+ "step": 1420
1011
+ },
1012
+ {
1013
+ "epoch": 0.26,
1014
+ "grad_norm": 1.2209943532943726,
1015
+ "learning_rate": 9.773602369492782e-05,
1016
+ "loss": 0.2075,
1017
+ "step": 1430
1018
+ },
1019
+ {
1020
+ "epoch": 0.27,
1021
+ "grad_norm": 0.9682492613792419,
1022
+ "learning_rate": 9.771751203258053e-05,
1023
+ "loss": 0.2206,
1024
+ "step": 1440
1025
+ },
1026
+ {
1027
+ "epoch": 0.27,
1028
+ "grad_norm": 0.36788633465766907,
1029
+ "learning_rate": 9.769900037023325e-05,
1030
+ "loss": 0.2204,
1031
+ "step": 1450
1032
+ },
1033
+ {
1034
+ "epoch": 0.27,
1035
+ "grad_norm": 1.2191195487976074,
1036
+ "learning_rate": 9.768048870788598e-05,
1037
+ "loss": 0.2103,
1038
+ "step": 1460
1039
+ },
1040
+ {
1041
+ "epoch": 0.27,
1042
+ "grad_norm": 0.6930559277534485,
1043
+ "learning_rate": 9.76619770455387e-05,
1044
+ "loss": 0.2157,
1045
+ "step": 1470
1046
+ },
1047
+ {
1048
+ "epoch": 0.27,
1049
+ "grad_norm": 1.2961112260818481,
1050
+ "learning_rate": 9.764346538319142e-05,
1051
+ "loss": 0.2244,
1052
+ "step": 1480
1053
+ },
1054
+ {
1055
+ "epoch": 0.27,
1056
+ "grad_norm": 0.7222216725349426,
1057
+ "learning_rate": 9.762495372084413e-05,
1058
+ "loss": 0.2161,
1059
+ "step": 1490
1060
+ },
1061
+ {
1062
+ "epoch": 0.28,
1063
+ "grad_norm": 0.9623996615409851,
1064
+ "learning_rate": 9.760644205849686e-05,
1065
+ "loss": 0.212,
1066
+ "step": 1500
1067
+ },
1068
+ {
1069
+ "epoch": 0.28,
1070
+ "grad_norm": 3.655571937561035,
1071
+ "learning_rate": 9.758793039614957e-05,
1072
+ "loss": 0.2302,
1073
+ "step": 1510
1074
+ },
1075
+ {
1076
+ "epoch": 0.28,
1077
+ "grad_norm": 0.7312277555465698,
1078
+ "learning_rate": 9.75694187338023e-05,
1079
+ "loss": 0.22,
1080
+ "step": 1520
1081
+ },
1082
+ {
1083
+ "epoch": 0.28,
1084
+ "grad_norm": 2.0226645469665527,
1085
+ "learning_rate": 9.755090707145502e-05,
1086
+ "loss": 0.2269,
1087
+ "step": 1530
1088
+ },
1089
+ {
1090
+ "epoch": 0.28,
1091
+ "grad_norm": 0.3837519884109497,
1092
+ "learning_rate": 9.753239540910775e-05,
1093
+ "loss": 0.2106,
1094
+ "step": 1540
1095
+ },
1096
+ {
1097
+ "epoch": 0.29,
1098
+ "grad_norm": 0.6145792007446289,
1099
+ "learning_rate": 9.751388374676046e-05,
1100
+ "loss": 0.2204,
1101
+ "step": 1550
1102
+ },
1103
+ {
1104
+ "epoch": 0.29,
1105
+ "grad_norm": 1.6149736642837524,
1106
+ "learning_rate": 9.749537208441318e-05,
1107
+ "loss": 0.218,
1108
+ "step": 1560
1109
+ },
1110
+ {
1111
+ "epoch": 0.29,
1112
+ "grad_norm": 1.113775610923767,
1113
+ "learning_rate": 9.747686042206591e-05,
1114
+ "loss": 0.2194,
1115
+ "step": 1570
1116
+ },
1117
+ {
1118
+ "epoch": 0.29,
1119
+ "grad_norm": 0.2693588137626648,
1120
+ "learning_rate": 9.745834875971863e-05,
1121
+ "loss": 0.2089,
1122
+ "step": 1580
1123
+ },
1124
+ {
1125
+ "epoch": 0.29,
1126
+ "grad_norm": 0.5227116346359253,
1127
+ "learning_rate": 9.743983709737135e-05,
1128
+ "loss": 0.2092,
1129
+ "step": 1590
1130
+ },
1131
+ {
1132
+ "epoch": 0.3,
1133
+ "grad_norm": 0.43677178025245667,
1134
+ "learning_rate": 9.742132543502406e-05,
1135
+ "loss": 0.2061,
1136
+ "step": 1600
1137
+ },
1138
+ {
1139
+ "epoch": 0.3,
1140
+ "grad_norm": 0.8750597834587097,
1141
+ "learning_rate": 9.74028137726768e-05,
1142
+ "loss": 0.2137,
1143
+ "step": 1610
1144
+ },
1145
+ {
1146
+ "epoch": 0.3,
1147
+ "grad_norm": 0.8018006086349487,
1148
+ "learning_rate": 9.738430211032951e-05,
1149
+ "loss": 0.2194,
1150
+ "step": 1620
1151
+ },
1152
+ {
1153
+ "epoch": 0.3,
1154
+ "grad_norm": 0.531554639339447,
1155
+ "learning_rate": 9.736579044798223e-05,
1156
+ "loss": 0.2077,
1157
+ "step": 1630
1158
+ },
1159
+ {
1160
+ "epoch": 0.3,
1161
+ "grad_norm": 0.5439698696136475,
1162
+ "learning_rate": 9.734727878563496e-05,
1163
+ "loss": 0.2153,
1164
+ "step": 1640
1165
+ },
1166
+ {
1167
+ "epoch": 0.3,
1168
+ "grad_norm": 1.296250343322754,
1169
+ "learning_rate": 9.732876712328768e-05,
1170
+ "loss": 0.1968,
1171
+ "step": 1650
1172
+ },
1173
+ {
1174
+ "epoch": 0.31,
1175
+ "grad_norm": 0.4914539158344269,
1176
+ "learning_rate": 9.73102554609404e-05,
1177
+ "loss": 0.2164,
1178
+ "step": 1660
1179
+ },
1180
+ {
1181
+ "epoch": 0.31,
1182
+ "grad_norm": 0.35670381784439087,
1183
+ "learning_rate": 9.729174379859311e-05,
1184
+ "loss": 0.2132,
1185
+ "step": 1670
1186
+ },
1187
+ {
1188
+ "epoch": 0.31,
1189
+ "grad_norm": 1.6645960807800293,
1190
+ "learning_rate": 9.727323213624584e-05,
1191
+ "loss": 0.2233,
1192
+ "step": 1680
1193
+ },
1194
+ {
1195
+ "epoch": 0.31,
1196
+ "grad_norm": 0.7455287575721741,
1197
+ "learning_rate": 9.725472047389856e-05,
1198
+ "loss": 0.2023,
1199
+ "step": 1690
1200
+ },
1201
+ {
1202
+ "epoch": 0.31,
1203
+ "grad_norm": 0.36129245162010193,
1204
+ "learning_rate": 9.723620881155128e-05,
1205
+ "loss": 0.2219,
1206
+ "step": 1700
1207
+ },
1208
+ {
1209
+ "epoch": 0.32,
1210
+ "grad_norm": 0.36295002698898315,
1211
+ "learning_rate": 9.721769714920401e-05,
1212
+ "loss": 0.2138,
1213
+ "step": 1710
1214
+ },
1215
+ {
1216
+ "epoch": 0.32,
1217
+ "grad_norm": 0.34044042229652405,
1218
+ "learning_rate": 9.719918548685672e-05,
1219
+ "loss": 0.2122,
1220
+ "step": 1720
1221
+ },
1222
+ {
1223
+ "epoch": 0.32,
1224
+ "grad_norm": 0.8785526156425476,
1225
+ "learning_rate": 9.718067382450946e-05,
1226
+ "loss": 0.2096,
1227
+ "step": 1730
1228
+ },
1229
+ {
1230
+ "epoch": 0.32,
1231
+ "grad_norm": 0.3848158121109009,
1232
+ "learning_rate": 9.716216216216216e-05,
1233
+ "loss": 0.2172,
1234
+ "step": 1740
1235
+ },
1236
+ {
1237
+ "epoch": 0.32,
1238
+ "grad_norm": 0.8510532975196838,
1239
+ "learning_rate": 9.714365049981489e-05,
1240
+ "loss": 0.2094,
1241
+ "step": 1750
1242
+ },
1243
+ {
1244
+ "epoch": 0.32,
1245
+ "grad_norm": 1.8068777322769165,
1246
+ "learning_rate": 9.712513883746761e-05,
1247
+ "loss": 0.2156,
1248
+ "step": 1760
1249
+ },
1250
+ {
1251
+ "epoch": 0.33,
1252
+ "grad_norm": 0.7489349246025085,
1253
+ "learning_rate": 9.710662717512034e-05,
1254
+ "loss": 0.2066,
1255
+ "step": 1770
1256
+ },
1257
+ {
1258
+ "epoch": 0.33,
1259
+ "grad_norm": 0.5769368410110474,
1260
+ "learning_rate": 9.708811551277304e-05,
1261
+ "loss": 0.2133,
1262
+ "step": 1780
1263
+ },
1264
+ {
1265
+ "epoch": 0.33,
1266
+ "grad_norm": 1.6159785985946655,
1267
+ "learning_rate": 9.706960385042577e-05,
1268
+ "loss": 0.2083,
1269
+ "step": 1790
1270
+ },
1271
+ {
1272
+ "epoch": 0.33,
1273
+ "grad_norm": 1.0204073190689087,
1274
+ "learning_rate": 9.705109218807849e-05,
1275
+ "loss": 0.213,
1276
+ "step": 1800
1277
+ },
1278
+ {
1279
+ "epoch": 0.33,
1280
+ "grad_norm": 0.4422318637371063,
1281
+ "learning_rate": 9.703258052573122e-05,
1282
+ "loss": 0.1996,
1283
+ "step": 1810
1284
+ },
1285
+ {
1286
+ "epoch": 0.34,
1287
+ "grad_norm": 0.9518413543701172,
1288
+ "learning_rate": 9.701406886338394e-05,
1289
+ "loss": 0.218,
1290
+ "step": 1820
1291
+ },
1292
+ {
1293
+ "epoch": 0.34,
1294
+ "grad_norm": 2.2408201694488525,
1295
+ "learning_rate": 9.699555720103666e-05,
1296
+ "loss": 0.2189,
1297
+ "step": 1830
1298
+ },
1299
+ {
1300
+ "epoch": 0.34,
1301
+ "grad_norm": 2.9319238662719727,
1302
+ "learning_rate": 9.697704553868939e-05,
1303
+ "loss": 0.2144,
1304
+ "step": 1840
1305
+ },
1306
+ {
1307
+ "epoch": 0.34,
1308
+ "grad_norm": 0.5640377998352051,
1309
+ "learning_rate": 9.695853387634209e-05,
1310
+ "loss": 0.2184,
1311
+ "step": 1850
1312
+ },
1313
+ {
1314
+ "epoch": 0.34,
1315
+ "grad_norm": 0.26673319935798645,
1316
+ "learning_rate": 9.694002221399482e-05,
1317
+ "loss": 0.2161,
1318
+ "step": 1860
1319
+ },
1320
+ {
1321
+ "epoch": 0.34,
1322
+ "grad_norm": 0.6571794152259827,
1323
+ "learning_rate": 9.692151055164754e-05,
1324
+ "loss": 0.207,
1325
+ "step": 1870
1326
+ },
1327
+ {
1328
+ "epoch": 0.35,
1329
+ "grad_norm": 0.20037458837032318,
1330
+ "learning_rate": 9.690299888930027e-05,
1331
+ "loss": 0.1985,
1332
+ "step": 1880
1333
+ },
1334
+ {
1335
+ "epoch": 0.35,
1336
+ "grad_norm": 0.9381949305534363,
1337
+ "learning_rate": 9.688448722695299e-05,
1338
+ "loss": 0.2118,
1339
+ "step": 1890
1340
+ },
1341
+ {
1342
+ "epoch": 0.35,
1343
+ "grad_norm": 1.4142616987228394,
1344
+ "learning_rate": 9.68659755646057e-05,
1345
+ "loss": 0.2155,
1346
+ "step": 1900
1347
+ },
1348
+ {
1349
+ "epoch": 0.35,
1350
+ "grad_norm": 0.7180021405220032,
1351
+ "learning_rate": 9.684746390225843e-05,
1352
+ "loss": 0.2024,
1353
+ "step": 1910
1354
+ },
1355
+ {
1356
+ "epoch": 0.35,
1357
+ "grad_norm": 0.6328707933425903,
1358
+ "learning_rate": 9.682895223991115e-05,
1359
+ "loss": 0.2107,
1360
+ "step": 1920
1361
+ },
1362
+ {
1363
+ "epoch": 0.36,
1364
+ "grad_norm": 1.0013030767440796,
1365
+ "learning_rate": 9.681044057756387e-05,
1366
+ "loss": 0.1973,
1367
+ "step": 1930
1368
+ },
1369
+ {
1370
+ "epoch": 0.36,
1371
+ "grad_norm": 0.9628821611404419,
1372
+ "learning_rate": 9.679192891521659e-05,
1373
+ "loss": 0.2074,
1374
+ "step": 1940
1375
+ },
1376
+ {
1377
+ "epoch": 0.36,
1378
+ "grad_norm": 1.5976811647415161,
1379
+ "learning_rate": 9.677341725286932e-05,
1380
+ "loss": 0.2076,
1381
+ "step": 1950
1382
+ },
1383
+ {
1384
+ "epoch": 0.36,
1385
+ "grad_norm": 1.5646719932556152,
1386
+ "learning_rate": 9.675490559052203e-05,
1387
+ "loss": 0.2291,
1388
+ "step": 1960
1389
+ },
1390
+ {
1391
+ "epoch": 0.36,
1392
+ "grad_norm": 1.701019287109375,
1393
+ "learning_rate": 9.673639392817475e-05,
1394
+ "loss": 0.2108,
1395
+ "step": 1970
1396
+ },
1397
+ {
1398
+ "epoch": 0.37,
1399
+ "grad_norm": 0.8562067747116089,
1400
+ "learning_rate": 9.671788226582747e-05,
1401
+ "loss": 0.2072,
1402
+ "step": 1980
1403
+ },
1404
+ {
1405
+ "epoch": 0.37,
1406
+ "grad_norm": 1.2539293766021729,
1407
+ "learning_rate": 9.66993706034802e-05,
1408
+ "loss": 0.215,
1409
+ "step": 1990
1410
+ },
1411
+ {
1412
+ "epoch": 0.37,
1413
+ "grad_norm": 1.9341269731521606,
1414
+ "learning_rate": 9.668085894113292e-05,
1415
+ "loss": 0.2184,
1416
+ "step": 2000
1417
+ },
1418
+ {
1419
+ "epoch": 0.37,
1420
+ "grad_norm": 0.4503909945487976,
1421
+ "learning_rate": 9.666234727878563e-05,
1422
+ "loss": 0.2129,
1423
+ "step": 2010
1424
+ },
1425
+ {
1426
+ "epoch": 0.37,
1427
+ "grad_norm": 0.4750733971595764,
1428
+ "learning_rate": 9.664383561643836e-05,
1429
+ "loss": 0.1996,
1430
+ "step": 2020
1431
+ },
1432
+ {
1433
+ "epoch": 0.37,
1434
+ "grad_norm": 0.7526803016662598,
1435
+ "learning_rate": 9.662532395409108e-05,
1436
+ "loss": 0.2152,
1437
+ "step": 2030
1438
+ },
1439
+ {
1440
+ "epoch": 0.38,
1441
+ "grad_norm": 0.45865398645401,
1442
+ "learning_rate": 9.66068122917438e-05,
1443
+ "loss": 0.204,
1444
+ "step": 2040
1445
+ },
1446
+ {
1447
+ "epoch": 0.38,
1448
+ "grad_norm": 0.4336209297180176,
1449
+ "learning_rate": 9.658830062939652e-05,
1450
+ "loss": 0.1989,
1451
+ "step": 2050
1452
+ },
1453
+ {
1454
+ "epoch": 0.38,
1455
+ "grad_norm": 2.891535520553589,
1456
+ "learning_rate": 9.656978896704925e-05,
1457
+ "loss": 0.2072,
1458
+ "step": 2060
1459
+ },
1460
+ {
1461
+ "epoch": 0.38,
1462
+ "grad_norm": 0.5239700675010681,
1463
+ "learning_rate": 9.655127730470196e-05,
1464
+ "loss": 0.2116,
1465
+ "step": 2070
1466
+ },
1467
+ {
1468
+ "epoch": 0.38,
1469
+ "grad_norm": 0.3464105725288391,
1470
+ "learning_rate": 9.653276564235468e-05,
1471
+ "loss": 0.2038,
1472
+ "step": 2080
1473
+ },
1474
+ {
1475
+ "epoch": 0.39,
1476
+ "grad_norm": 0.4590921401977539,
1477
+ "learning_rate": 9.651425398000741e-05,
1478
+ "loss": 0.2151,
1479
+ "step": 2090
1480
+ },
1481
+ {
1482
+ "epoch": 0.39,
1483
+ "grad_norm": 0.5390322208404541,
1484
+ "learning_rate": 9.649574231766013e-05,
1485
+ "loss": 0.2165,
1486
+ "step": 2100
1487
+ },
1488
+ {
1489
+ "epoch": 0.39,
1490
+ "grad_norm": 0.8248626589775085,
1491
+ "learning_rate": 9.647723065531286e-05,
1492
+ "loss": 0.199,
1493
+ "step": 2110
1494
+ },
1495
+ {
1496
+ "epoch": 0.39,
1497
+ "grad_norm": 0.5358192324638367,
1498
+ "learning_rate": 9.645871899296556e-05,
1499
+ "loss": 0.2159,
1500
+ "step": 2120
1501
+ },
1502
+ {
1503
+ "epoch": 0.39,
1504
+ "grad_norm": 1.0364553928375244,
1505
+ "learning_rate": 9.64402073306183e-05,
1506
+ "loss": 0.1947,
1507
+ "step": 2130
1508
+ },
1509
+ {
1510
+ "epoch": 0.39,
1511
+ "grad_norm": 0.628521740436554,
1512
+ "learning_rate": 9.642169566827101e-05,
1513
+ "loss": 0.2212,
1514
+ "step": 2140
1515
+ },
1516
+ {
1517
+ "epoch": 0.4,
1518
+ "grad_norm": 0.6107978224754333,
1519
+ "learning_rate": 9.640318400592374e-05,
1520
+ "loss": 0.2064,
1521
+ "step": 2150
1522
+ },
1523
+ {
1524
+ "epoch": 0.4,
1525
+ "grad_norm": 1.7244569063186646,
1526
+ "learning_rate": 9.638467234357646e-05,
1527
+ "loss": 0.2158,
1528
+ "step": 2160
1529
+ },
1530
+ {
1531
+ "epoch": 0.4,
1532
+ "grad_norm": 0.4745640158653259,
1533
+ "learning_rate": 9.636616068122918e-05,
1534
+ "loss": 0.2143,
1535
+ "step": 2170
1536
+ },
1537
+ {
1538
+ "epoch": 0.4,
1539
+ "grad_norm": 0.6135119795799255,
1540
+ "learning_rate": 9.634764901888191e-05,
1541
+ "loss": 0.2166,
1542
+ "step": 2180
1543
+ },
1544
+ {
1545
+ "epoch": 0.4,
1546
+ "grad_norm": 0.29639098048210144,
1547
+ "learning_rate": 9.632913735653463e-05,
1548
+ "loss": 0.2031,
1549
+ "step": 2190
1550
+ },
1551
+ {
1552
+ "epoch": 0.41,
1553
+ "grad_norm": 0.7129987478256226,
1554
+ "learning_rate": 9.631062569418734e-05,
1555
+ "loss": 0.2057,
1556
+ "step": 2200
1557
+ },
1558
+ {
1559
+ "epoch": 0.41,
1560
+ "grad_norm": 1.079046607017517,
1561
+ "learning_rate": 9.629211403184006e-05,
1562
+ "loss": 0.2012,
1563
+ "step": 2210
1564
+ },
1565
+ {
1566
+ "epoch": 0.41,
1567
+ "grad_norm": 0.61760014295578,
1568
+ "learning_rate": 9.627360236949279e-05,
1569
+ "loss": 0.198,
1570
+ "step": 2220
1571
+ },
1572
+ {
1573
+ "epoch": 0.41,
1574
+ "grad_norm": 0.36462926864624023,
1575
+ "learning_rate": 9.62550907071455e-05,
1576
+ "loss": 0.2003,
1577
+ "step": 2230
1578
+ },
1579
+ {
1580
+ "epoch": 0.41,
1581
+ "grad_norm": 0.37455418705940247,
1582
+ "learning_rate": 9.623657904479823e-05,
1583
+ "loss": 0.2234,
1584
+ "step": 2240
1585
+ },
1586
+ {
1587
+ "epoch": 0.41,
1588
+ "grad_norm": 0.3829789459705353,
1589
+ "learning_rate": 9.621806738245094e-05,
1590
+ "loss": 0.2177,
1591
+ "step": 2250
1592
+ },
1593
+ {
1594
+ "epoch": 0.42,
1595
+ "grad_norm": 0.760238528251648,
1596
+ "learning_rate": 9.619955572010367e-05,
1597
+ "loss": 0.2109,
1598
+ "step": 2260
1599
+ },
1600
+ {
1601
+ "epoch": 0.42,
1602
+ "grad_norm": 0.3408792316913605,
1603
+ "learning_rate": 9.618104405775639e-05,
1604
+ "loss": 0.1997,
1605
+ "step": 2270
1606
+ },
1607
+ {
1608
+ "epoch": 0.42,
1609
+ "grad_norm": 0.38549286127090454,
1610
+ "learning_rate": 9.616253239540911e-05,
1611
+ "loss": 0.213,
1612
+ "step": 2280
1613
+ },
1614
+ {
1615
+ "epoch": 0.42,
1616
+ "grad_norm": 1.2211161851882935,
1617
+ "learning_rate": 9.614402073306184e-05,
1618
+ "loss": 0.2104,
1619
+ "step": 2290
1620
+ },
1621
+ {
1622
+ "epoch": 0.42,
1623
+ "grad_norm": 0.7109599113464355,
1624
+ "learning_rate": 9.612550907071456e-05,
1625
+ "loss": 0.206,
1626
+ "step": 2300
1627
+ },
1628
+ {
1629
+ "epoch": 0.43,
1630
+ "grad_norm": 2.4902968406677246,
1631
+ "learning_rate": 9.610699740836727e-05,
1632
+ "loss": 0.2169,
1633
+ "step": 2310
1634
+ },
1635
+ {
1636
+ "epoch": 0.43,
1637
+ "grad_norm": 0.45785343647003174,
1638
+ "learning_rate": 9.608848574601999e-05,
1639
+ "loss": 0.201,
1640
+ "step": 2320
1641
+ },
1642
+ {
1643
+ "epoch": 0.43,
1644
+ "grad_norm": 0.5752105116844177,
1645
+ "learning_rate": 9.606997408367272e-05,
1646
+ "loss": 0.1957,
1647
+ "step": 2330
1648
+ },
1649
+ {
1650
+ "epoch": 0.43,
1651
+ "grad_norm": 1.4003032445907593,
1652
+ "learning_rate": 9.605146242132544e-05,
1653
+ "loss": 0.2131,
1654
+ "step": 2340
1655
+ },
1656
+ {
1657
+ "epoch": 0.43,
1658
+ "grad_norm": 0.7407622337341309,
1659
+ "learning_rate": 9.603295075897816e-05,
1660
+ "loss": 0.2069,
1661
+ "step": 2350
1662
+ },
1663
+ {
1664
+ "epoch": 0.44,
1665
+ "grad_norm": 0.6703731417655945,
1666
+ "learning_rate": 9.601443909663089e-05,
1667
+ "loss": 0.2068,
1668
+ "step": 2360
1669
+ },
1670
+ {
1671
+ "epoch": 0.44,
1672
+ "grad_norm": 0.8653967976570129,
1673
+ "learning_rate": 9.59959274342836e-05,
1674
+ "loss": 0.2001,
1675
+ "step": 2370
1676
+ },
1677
+ {
1678
+ "epoch": 0.44,
1679
+ "grad_norm": 3.6894798278808594,
1680
+ "learning_rate": 9.597741577193634e-05,
1681
+ "loss": 0.2133,
1682
+ "step": 2380
1683
+ },
1684
+ {
1685
+ "epoch": 0.44,
1686
+ "grad_norm": 1.5660738945007324,
1687
+ "learning_rate": 9.595890410958904e-05,
1688
+ "loss": 0.2186,
1689
+ "step": 2390
1690
+ },
1691
+ {
1692
+ "epoch": 0.44,
1693
+ "grad_norm": 1.0196833610534668,
1694
+ "learning_rate": 9.594039244724177e-05,
1695
+ "loss": 0.2096,
1696
+ "step": 2400
1697
+ },
1698
+ {
1699
+ "epoch": 0.44,
1700
+ "grad_norm": 0.25469130277633667,
1701
+ "learning_rate": 9.592188078489449e-05,
1702
+ "loss": 0.2086,
1703
+ "step": 2410
1704
+ },
1705
+ {
1706
+ "epoch": 0.45,
1707
+ "grad_norm": 0.517469584941864,
1708
+ "learning_rate": 9.59033691225472e-05,
1709
+ "loss": 0.2142,
1710
+ "step": 2420
1711
+ },
1712
+ {
1713
+ "epoch": 0.45,
1714
+ "grad_norm": 0.8714912533760071,
1715
+ "learning_rate": 9.588485746019992e-05,
1716
+ "loss": 0.1995,
1717
+ "step": 2430
1718
+ },
1719
+ {
1720
+ "epoch": 0.45,
1721
+ "grad_norm": 1.4467742443084717,
1722
+ "learning_rate": 9.586634579785265e-05,
1723
+ "loss": 0.1988,
1724
+ "step": 2440
1725
+ },
1726
+ {
1727
+ "epoch": 0.45,
1728
+ "grad_norm": 0.6827601790428162,
1729
+ "learning_rate": 9.584783413550537e-05,
1730
+ "loss": 0.2004,
1731
+ "step": 2450
1732
+ },
1733
+ {
1734
+ "epoch": 0.45,
1735
+ "grad_norm": 0.5829424262046814,
1736
+ "learning_rate": 9.582932247315809e-05,
1737
+ "loss": 0.2043,
1738
+ "step": 2460
1739
+ },
1740
+ {
1741
+ "epoch": 0.46,
1742
+ "grad_norm": 0.6504799127578735,
1743
+ "learning_rate": 9.581081081081082e-05,
1744
+ "loss": 0.2082,
1745
+ "step": 2470
1746
+ },
1747
+ {
1748
+ "epoch": 0.46,
1749
+ "grad_norm": 0.2879631221294403,
1750
+ "learning_rate": 9.579229914846354e-05,
1751
+ "loss": 0.2064,
1752
+ "step": 2480
1753
+ },
1754
+ {
1755
+ "epoch": 0.46,
1756
+ "grad_norm": 0.337453693151474,
1757
+ "learning_rate": 9.577378748611627e-05,
1758
+ "loss": 0.2195,
1759
+ "step": 2490
1760
+ },
1761
+ {
1762
+ "epoch": 0.46,
1763
+ "grad_norm": 0.7913995981216431,
1764
+ "learning_rate": 9.575527582376897e-05,
1765
+ "loss": 0.2002,
1766
+ "step": 2500
1767
+ },
1768
+ {
1769
+ "epoch": 0.46,
1770
+ "grad_norm": 0.33184346556663513,
1771
+ "learning_rate": 9.57367641614217e-05,
1772
+ "loss": 0.1974,
1773
+ "step": 2510
1774
+ },
1775
+ {
1776
+ "epoch": 0.46,
1777
+ "grad_norm": 1.3612488508224487,
1778
+ "learning_rate": 9.571825249907442e-05,
1779
+ "loss": 0.2081,
1780
+ "step": 2520
1781
+ },
1782
+ {
1783
+ "epoch": 0.47,
1784
+ "grad_norm": 1.3221415281295776,
1785
+ "learning_rate": 9.569974083672715e-05,
1786
+ "loss": 0.2142,
1787
+ "step": 2530
1788
+ },
1789
+ {
1790
+ "epoch": 0.47,
1791
+ "grad_norm": 0.7433180212974548,
1792
+ "learning_rate": 9.568122917437987e-05,
1793
+ "loss": 0.2087,
1794
+ "step": 2540
1795
+ },
1796
+ {
1797
+ "epoch": 0.47,
1798
+ "grad_norm": 0.3186396062374115,
1799
+ "learning_rate": 9.566271751203258e-05,
1800
+ "loss": 0.2161,
1801
+ "step": 2550
1802
+ },
1803
+ {
1804
+ "epoch": 0.47,
1805
+ "grad_norm": 0.9290895462036133,
1806
+ "learning_rate": 9.564420584968531e-05,
1807
+ "loss": 0.2165,
1808
+ "step": 2560
1809
+ },
1810
+ {
1811
+ "epoch": 0.47,
1812
+ "grad_norm": 0.9218959212303162,
1813
+ "learning_rate": 9.562569418733803e-05,
1814
+ "loss": 0.2077,
1815
+ "step": 2570
1816
+ },
1817
+ {
1818
+ "epoch": 0.48,
1819
+ "grad_norm": 0.6397104859352112,
1820
+ "learning_rate": 9.560718252499075e-05,
1821
+ "loss": 0.205,
1822
+ "step": 2580
1823
+ },
1824
+ {
1825
+ "epoch": 0.48,
1826
+ "grad_norm": 2.004181385040283,
1827
+ "learning_rate": 9.558867086264347e-05,
1828
+ "loss": 0.2093,
1829
+ "step": 2590
1830
+ },
1831
+ {
1832
+ "epoch": 0.48,
1833
+ "grad_norm": 0.29606181383132935,
1834
+ "learning_rate": 9.55701592002962e-05,
1835
+ "loss": 0.2145,
1836
+ "step": 2600
1837
+ },
1838
+ {
1839
+ "epoch": 0.48,
1840
+ "grad_norm": 2.1520352363586426,
1841
+ "learning_rate": 9.55516475379489e-05,
1842
+ "loss": 0.2165,
1843
+ "step": 2610
1844
+ },
1845
+ {
1846
+ "epoch": 0.48,
1847
+ "grad_norm": 1.1662155389785767,
1848
+ "learning_rate": 9.553313587560163e-05,
1849
+ "loss": 0.1981,
1850
+ "step": 2620
1851
+ },
1852
+ {
1853
+ "epoch": 0.49,
1854
+ "grad_norm": 0.5705534815788269,
1855
+ "learning_rate": 9.551462421325436e-05,
1856
+ "loss": 0.2004,
1857
+ "step": 2630
1858
+ },
1859
+ {
1860
+ "epoch": 0.49,
1861
+ "grad_norm": 0.2732492685317993,
1862
+ "learning_rate": 9.549611255090708e-05,
1863
+ "loss": 0.2137,
1864
+ "step": 2640
1865
+ },
1866
+ {
1867
+ "epoch": 0.49,
1868
+ "grad_norm": 2.1860570907592773,
1869
+ "learning_rate": 9.54776008885598e-05,
1870
+ "loss": 0.2057,
1871
+ "step": 2650
1872
+ },
1873
+ {
1874
+ "epoch": 0.49,
1875
+ "grad_norm": 0.5216954350471497,
1876
+ "learning_rate": 9.545908922621251e-05,
1877
+ "loss": 0.2175,
1878
+ "step": 2660
1879
+ },
1880
+ {
1881
+ "epoch": 0.49,
1882
+ "grad_norm": 0.7214330434799194,
1883
+ "learning_rate": 9.544057756386524e-05,
1884
+ "loss": 0.2104,
1885
+ "step": 2670
1886
+ },
1887
+ {
1888
+ "epoch": 0.49,
1889
+ "grad_norm": 1.1864840984344482,
1890
+ "learning_rate": 9.542206590151796e-05,
1891
+ "loss": 0.2168,
1892
+ "step": 2680
1893
+ },
1894
+ {
1895
+ "epoch": 0.5,
1896
+ "grad_norm": 0.7450833916664124,
1897
+ "learning_rate": 9.540355423917068e-05,
1898
+ "loss": 0.2064,
1899
+ "step": 2690
1900
+ },
1901
+ {
1902
+ "epoch": 0.5,
1903
+ "grad_norm": 1.97665274143219,
1904
+ "learning_rate": 9.53850425768234e-05,
1905
+ "loss": 0.2132,
1906
+ "step": 2700
1907
+ },
1908
+ {
1909
+ "epoch": 0.5,
1910
+ "grad_norm": 0.8283855319023132,
1911
+ "learning_rate": 9.536653091447613e-05,
1912
+ "loss": 0.217,
1913
+ "step": 2710
1914
+ },
1915
+ {
1916
+ "epoch": 0.5,
1917
+ "grad_norm": 0.8062860369682312,
1918
+ "learning_rate": 9.534801925212884e-05,
1919
+ "loss": 0.1985,
1920
+ "step": 2720
1921
+ },
1922
+ {
1923
+ "epoch": 0.5,
1924
+ "grad_norm": 1.2342814207077026,
1925
+ "learning_rate": 9.532950758978156e-05,
1926
+ "loss": 0.2043,
1927
+ "step": 2730
1928
+ },
1929
+ {
1930
+ "epoch": 0.51,
1931
+ "grad_norm": 2.1351473331451416,
1932
+ "learning_rate": 9.531099592743429e-05,
1933
+ "loss": 0.2145,
1934
+ "step": 2740
1935
+ },
1936
+ {
1937
+ "epoch": 0.51,
1938
+ "grad_norm": 1.2654520273208618,
1939
+ "learning_rate": 9.529248426508701e-05,
1940
+ "loss": 0.2067,
1941
+ "step": 2750
1942
+ },
1943
+ {
1944
+ "epoch": 0.51,
1945
+ "grad_norm": 0.3491131067276001,
1946
+ "learning_rate": 9.527397260273974e-05,
1947
+ "loss": 0.2008,
1948
+ "step": 2760
1949
+ },
1950
+ {
1951
+ "epoch": 0.51,
1952
+ "grad_norm": 1.2141033411026,
1953
+ "learning_rate": 9.525546094039244e-05,
1954
+ "loss": 0.1965,
1955
+ "step": 2770
1956
+ },
1957
+ {
1958
+ "epoch": 0.51,
1959
+ "grad_norm": 0.3704434037208557,
1960
+ "learning_rate": 9.523694927804518e-05,
1961
+ "loss": 0.2041,
1962
+ "step": 2780
1963
+ },
1964
+ {
1965
+ "epoch": 0.51,
1966
+ "grad_norm": 1.8885278701782227,
1967
+ "learning_rate": 9.521843761569789e-05,
1968
+ "loss": 0.2062,
1969
+ "step": 2790
1970
+ },
1971
+ {
1972
+ "epoch": 0.52,
1973
+ "grad_norm": 0.622947633266449,
1974
+ "learning_rate": 9.519992595335061e-05,
1975
+ "loss": 0.2023,
1976
+ "step": 2800
1977
+ },
1978
+ {
1979
+ "epoch": 0.52,
1980
+ "grad_norm": 2.0077390670776367,
1981
+ "learning_rate": 9.518141429100334e-05,
1982
+ "loss": 0.2025,
1983
+ "step": 2810
1984
+ },
1985
+ {
1986
+ "epoch": 0.52,
1987
+ "grad_norm": 0.7942656874656677,
1988
+ "learning_rate": 9.516290262865606e-05,
1989
+ "loss": 0.201,
1990
+ "step": 2820
1991
+ },
1992
+ {
1993
+ "epoch": 0.52,
1994
+ "grad_norm": 1.2571896314620972,
1995
+ "learning_rate": 9.514439096630879e-05,
1996
+ "loss": 0.206,
1997
+ "step": 2830
1998
+ },
1999
+ {
2000
+ "epoch": 0.52,
2001
+ "grad_norm": 0.8487832546234131,
2002
+ "learning_rate": 9.512587930396149e-05,
2003
+ "loss": 0.2079,
2004
+ "step": 2840
2005
+ },
2006
+ {
2007
+ "epoch": 0.53,
2008
+ "grad_norm": 0.46226075291633606,
2009
+ "learning_rate": 9.510736764161422e-05,
2010
+ "loss": 0.2048,
2011
+ "step": 2850
2012
+ },
2013
+ {
2014
+ "epoch": 0.53,
2015
+ "grad_norm": 0.33343935012817383,
2016
+ "learning_rate": 9.508885597926694e-05,
2017
+ "loss": 0.2082,
2018
+ "step": 2860
2019
+ },
2020
+ {
2021
+ "epoch": 0.53,
2022
+ "grad_norm": 0.5473183393478394,
2023
+ "learning_rate": 9.507034431691967e-05,
2024
+ "loss": 0.2033,
2025
+ "step": 2870
2026
+ },
2027
+ {
2028
+ "epoch": 0.53,
2029
+ "grad_norm": 0.767188310623169,
2030
+ "learning_rate": 9.505183265457238e-05,
2031
+ "loss": 0.2021,
2032
+ "step": 2880
2033
+ },
2034
+ {
2035
+ "epoch": 0.53,
2036
+ "grad_norm": 0.36617210507392883,
2037
+ "learning_rate": 9.50333209922251e-05,
2038
+ "loss": 0.2008,
2039
+ "step": 2890
2040
+ },
2041
+ {
2042
+ "epoch": 0.53,
2043
+ "grad_norm": 0.4321964979171753,
2044
+ "learning_rate": 9.501480932987782e-05,
2045
+ "loss": 0.2019,
2046
+ "step": 2900
2047
+ },
2048
+ {
2049
+ "epoch": 0.54,
2050
+ "grad_norm": 0.39318132400512695,
2051
+ "learning_rate": 9.499629766753055e-05,
2052
+ "loss": 0.2004,
2053
+ "step": 2910
2054
+ },
2055
+ {
2056
+ "epoch": 0.54,
2057
+ "grad_norm": 0.42340582609176636,
2058
+ "learning_rate": 9.497778600518327e-05,
2059
+ "loss": 0.1888,
2060
+ "step": 2920
2061
+ },
2062
+ {
2063
+ "epoch": 0.54,
2064
+ "grad_norm": 0.6964154243469238,
2065
+ "learning_rate": 9.495927434283599e-05,
2066
+ "loss": 0.1941,
2067
+ "step": 2930
2068
+ },
2069
+ {
2070
+ "epoch": 0.54,
2071
+ "grad_norm": 1.6123028993606567,
2072
+ "learning_rate": 9.494076268048872e-05,
2073
+ "loss": 0.2073,
2074
+ "step": 2940
2075
+ },
2076
+ {
2077
+ "epoch": 0.54,
2078
+ "grad_norm": 0.6400339007377625,
2079
+ "learning_rate": 9.492225101814142e-05,
2080
+ "loss": 0.2021,
2081
+ "step": 2950
2082
+ },
2083
+ {
2084
+ "epoch": 0.55,
2085
+ "grad_norm": 0.3889828026294708,
2086
+ "learning_rate": 9.490373935579415e-05,
2087
+ "loss": 0.197,
2088
+ "step": 2960
2089
+ },
2090
+ {
2091
+ "epoch": 0.55,
2092
+ "grad_norm": 0.2391413450241089,
2093
+ "learning_rate": 9.488522769344687e-05,
2094
+ "loss": 0.1988,
2095
+ "step": 2970
2096
+ },
2097
+ {
2098
+ "epoch": 0.55,
2099
+ "grad_norm": 1.3817205429077148,
2100
+ "learning_rate": 9.48667160310996e-05,
2101
+ "loss": 0.1995,
2102
+ "step": 2980
2103
+ },
2104
+ {
2105
+ "epoch": 0.55,
2106
+ "grad_norm": 0.47163134813308716,
2107
+ "learning_rate": 9.484820436875232e-05,
2108
+ "loss": 0.1899,
2109
+ "step": 2990
2110
+ },
2111
+ {
2112
+ "epoch": 0.55,
2113
+ "grad_norm": 0.9131165146827698,
2114
+ "learning_rate": 9.482969270640504e-05,
2115
+ "loss": 0.1944,
2116
+ "step": 3000
2117
+ },
2118
+ {
2119
+ "epoch": 0.56,
2120
+ "grad_norm": 0.6244738101959229,
2121
+ "learning_rate": 9.481118104405777e-05,
2122
+ "loss": 0.207,
2123
+ "step": 3010
2124
+ },
2125
+ {
2126
+ "epoch": 0.56,
2127
+ "grad_norm": 0.5568848848342896,
2128
+ "learning_rate": 9.479266938171048e-05,
2129
+ "loss": 0.1842,
2130
+ "step": 3020
2131
+ },
2132
+ {
2133
+ "epoch": 0.56,
2134
+ "grad_norm": 0.7403110861778259,
2135
+ "learning_rate": 9.47741577193632e-05,
2136
+ "loss": 0.1981,
2137
+ "step": 3030
2138
+ },
2139
+ {
2140
+ "epoch": 0.56,
2141
+ "grad_norm": 1.1462550163269043,
2142
+ "learning_rate": 9.475564605701592e-05,
2143
+ "loss": 0.1967,
2144
+ "step": 3040
2145
+ },
2146
+ {
2147
+ "epoch": 0.56,
2148
+ "grad_norm": 0.9351162910461426,
2149
+ "learning_rate": 9.473713439466865e-05,
2150
+ "loss": 0.1999,
2151
+ "step": 3050
2152
+ },
2153
+ {
2154
+ "epoch": 0.56,
2155
+ "grad_norm": 0.4262019693851471,
2156
+ "learning_rate": 9.471862273232137e-05,
2157
+ "loss": 0.1889,
2158
+ "step": 3060
2159
+ },
2160
+ {
2161
+ "epoch": 0.57,
2162
+ "grad_norm": 1.5111364126205444,
2163
+ "learning_rate": 9.470011106997408e-05,
2164
+ "loss": 0.2139,
2165
+ "step": 3070
2166
+ },
2167
+ {
2168
+ "epoch": 0.57,
2169
+ "grad_norm": 0.6351222991943359,
2170
+ "learning_rate": 9.468159940762682e-05,
2171
+ "loss": 0.1952,
2172
+ "step": 3080
2173
+ },
2174
+ {
2175
+ "epoch": 0.57,
2176
+ "grad_norm": 0.46668490767478943,
2177
+ "learning_rate": 9.466308774527953e-05,
2178
+ "loss": 0.1967,
2179
+ "step": 3090
2180
+ },
2181
+ {
2182
+ "epoch": 0.57,
2183
+ "grad_norm": 0.43086475133895874,
2184
+ "learning_rate": 9.464457608293226e-05,
2185
+ "loss": 0.2037,
2186
+ "step": 3100
2187
+ },
2188
+ {
2189
+ "epoch": 0.57,
2190
+ "grad_norm": 0.2739882171154022,
2191
+ "learning_rate": 9.462606442058497e-05,
2192
+ "loss": 0.2141,
2193
+ "step": 3110
2194
+ },
2195
+ {
2196
+ "epoch": 0.58,
2197
+ "grad_norm": 0.6408763527870178,
2198
+ "learning_rate": 9.46075527582377e-05,
2199
+ "loss": 0.1994,
2200
+ "step": 3120
2201
+ },
2202
+ {
2203
+ "epoch": 0.58,
2204
+ "grad_norm": 0.33041083812713623,
2205
+ "learning_rate": 9.458904109589042e-05,
2206
+ "loss": 0.1992,
2207
+ "step": 3130
2208
+ },
2209
+ {
2210
+ "epoch": 0.58,
2211
+ "grad_norm": 0.2604489326477051,
2212
+ "learning_rate": 9.457052943354313e-05,
2213
+ "loss": 0.2044,
2214
+ "step": 3140
2215
+ },
2216
+ {
2217
+ "epoch": 0.58,
2218
+ "grad_norm": 1.0766637325286865,
2219
+ "learning_rate": 9.455201777119585e-05,
2220
+ "loss": 0.2121,
2221
+ "step": 3150
2222
+ },
2223
+ {
2224
+ "epoch": 0.58,
2225
+ "grad_norm": 1.4685893058776855,
2226
+ "learning_rate": 9.453350610884858e-05,
2227
+ "loss": 0.2114,
2228
+ "step": 3160
2229
+ },
2230
+ {
2231
+ "epoch": 0.58,
2232
+ "grad_norm": 0.42889487743377686,
2233
+ "learning_rate": 9.45149944465013e-05,
2234
+ "loss": 0.2035,
2235
+ "step": 3170
2236
+ },
2237
+ {
2238
+ "epoch": 0.59,
2239
+ "grad_norm": 0.5236283540725708,
2240
+ "learning_rate": 9.449648278415402e-05,
2241
+ "loss": 0.1984,
2242
+ "step": 3180
2243
+ },
2244
+ {
2245
+ "epoch": 0.59,
2246
+ "grad_norm": 1.366331696510315,
2247
+ "learning_rate": 9.447797112180675e-05,
2248
+ "loss": 0.2003,
2249
+ "step": 3190
2250
+ },
2251
+ {
2252
+ "epoch": 0.59,
2253
+ "grad_norm": 0.7277652025222778,
2254
+ "learning_rate": 9.445945945945946e-05,
2255
+ "loss": 0.1902,
2256
+ "step": 3200
2257
+ },
2258
+ {
2259
+ "epoch": 0.59,
2260
+ "grad_norm": 0.5104700922966003,
2261
+ "learning_rate": 9.44409477971122e-05,
2262
+ "loss": 0.2041,
2263
+ "step": 3210
2264
+ },
2265
+ {
2266
+ "epoch": 0.59,
2267
+ "grad_norm": 0.307566374540329,
2268
+ "learning_rate": 9.44224361347649e-05,
2269
+ "loss": 0.1984,
2270
+ "step": 3220
2271
+ },
2272
+ {
2273
+ "epoch": 0.6,
2274
+ "grad_norm": 0.6505478620529175,
2275
+ "learning_rate": 9.440392447241763e-05,
2276
+ "loss": 0.2023,
2277
+ "step": 3230
2278
+ },
2279
+ {
2280
+ "epoch": 0.6,
2281
+ "grad_norm": 1.1189345121383667,
2282
+ "learning_rate": 9.438541281007035e-05,
2283
+ "loss": 0.1801,
2284
+ "step": 3240
2285
+ },
2286
+ {
2287
+ "epoch": 0.6,
2288
+ "grad_norm": 1.2170076370239258,
2289
+ "learning_rate": 9.436690114772308e-05,
2290
+ "loss": 0.2064,
2291
+ "step": 3250
2292
+ },
2293
+ {
2294
+ "epoch": 0.6,
2295
+ "grad_norm": 0.3985789716243744,
2296
+ "learning_rate": 9.43483894853758e-05,
2297
+ "loss": 0.2095,
2298
+ "step": 3260
2299
+ },
2300
+ {
2301
+ "epoch": 0.6,
2302
+ "grad_norm": 0.2721532881259918,
2303
+ "learning_rate": 9.432987782302851e-05,
2304
+ "loss": 0.2031,
2305
+ "step": 3270
2306
+ },
2307
+ {
2308
+ "epoch": 0.6,
2309
+ "grad_norm": 1.036005973815918,
2310
+ "learning_rate": 9.431136616068124e-05,
2311
+ "loss": 0.1997,
2312
+ "step": 3280
2313
+ },
2314
+ {
2315
+ "epoch": 0.61,
2316
+ "grad_norm": 0.7809330224990845,
2317
+ "learning_rate": 9.429285449833396e-05,
2318
+ "loss": 0.2047,
2319
+ "step": 3290
2320
+ },
2321
+ {
2322
+ "epoch": 0.61,
2323
+ "grad_norm": 1.0563770532608032,
2324
+ "learning_rate": 9.427434283598668e-05,
2325
+ "loss": 0.2095,
2326
+ "step": 3300
2327
+ },
2328
+ {
2329
+ "epoch": 0.61,
2330
+ "grad_norm": 1.421645998954773,
2331
+ "learning_rate": 9.42558311736394e-05,
2332
+ "loss": 0.2023,
2333
+ "step": 3310
2334
+ },
2335
+ {
2336
+ "epoch": 0.61,
2337
+ "grad_norm": 0.821112334728241,
2338
+ "learning_rate": 9.423731951129212e-05,
2339
+ "loss": 0.2007,
2340
+ "step": 3320
2341
+ },
2342
+ {
2343
+ "epoch": 0.61,
2344
+ "grad_norm": 0.28622138500213623,
2345
+ "learning_rate": 9.421880784894483e-05,
2346
+ "loss": 0.1934,
2347
+ "step": 3330
2348
+ },
2349
+ {
2350
+ "epoch": 0.62,
2351
+ "grad_norm": 0.34855708479881287,
2352
+ "learning_rate": 9.420029618659756e-05,
2353
+ "loss": 0.1912,
2354
+ "step": 3340
2355
+ },
2356
+ {
2357
+ "epoch": 0.62,
2358
+ "grad_norm": 0.5029919147491455,
2359
+ "learning_rate": 9.418178452425028e-05,
2360
+ "loss": 0.1977,
2361
+ "step": 3350
2362
+ },
2363
+ {
2364
+ "epoch": 0.62,
2365
+ "grad_norm": 0.6966719031333923,
2366
+ "learning_rate": 9.416327286190301e-05,
2367
+ "loss": 0.1929,
2368
+ "step": 3360
2369
+ },
2370
+ {
2371
+ "epoch": 0.62,
2372
+ "grad_norm": 0.6500990390777588,
2373
+ "learning_rate": 9.414476119955572e-05,
2374
+ "loss": 0.197,
2375
+ "step": 3370
2376
+ },
2377
+ {
2378
+ "epoch": 0.62,
2379
+ "grad_norm": 1.8321138620376587,
2380
+ "learning_rate": 9.412624953720844e-05,
2381
+ "loss": 0.1937,
2382
+ "step": 3380
2383
+ },
2384
+ {
2385
+ "epoch": 0.63,
2386
+ "grad_norm": 0.976116418838501,
2387
+ "learning_rate": 9.410773787486117e-05,
2388
+ "loss": 0.1971,
2389
+ "step": 3390
2390
+ },
2391
+ {
2392
+ "epoch": 0.63,
2393
+ "grad_norm": 1.2449440956115723,
2394
+ "learning_rate": 9.408922621251389e-05,
2395
+ "loss": 0.1915,
2396
+ "step": 3400
2397
+ },
2398
+ {
2399
+ "epoch": 0.63,
2400
+ "grad_norm": 0.3859243094921112,
2401
+ "learning_rate": 9.407071455016661e-05,
2402
+ "loss": 0.2041,
2403
+ "step": 3410
2404
+ },
2405
+ {
2406
+ "epoch": 0.63,
2407
+ "grad_norm": 1.1816600561141968,
2408
+ "learning_rate": 9.405220288781932e-05,
2409
+ "loss": 0.2041,
2410
+ "step": 3420
2411
+ },
2412
+ {
2413
+ "epoch": 0.63,
2414
+ "grad_norm": 0.6951285004615784,
2415
+ "learning_rate": 9.403369122547206e-05,
2416
+ "loss": 0.1971,
2417
+ "step": 3430
2418
+ },
2419
+ {
2420
+ "epoch": 0.63,
2421
+ "grad_norm": 1.083073377609253,
2422
+ "learning_rate": 9.401517956312477e-05,
2423
+ "loss": 0.1996,
2424
+ "step": 3440
2425
+ },
2426
+ {
2427
+ "epoch": 0.64,
2428
+ "grad_norm": 0.4412924647331238,
2429
+ "learning_rate": 9.399666790077749e-05,
2430
+ "loss": 0.2016,
2431
+ "step": 3450
2432
+ },
2433
+ {
2434
+ "epoch": 0.64,
2435
+ "grad_norm": 0.7372565865516663,
2436
+ "learning_rate": 9.397815623843022e-05,
2437
+ "loss": 0.1969,
2438
+ "step": 3460
2439
+ },
2440
+ {
2441
+ "epoch": 0.64,
2442
+ "grad_norm": 0.35396844148635864,
2443
+ "learning_rate": 9.395964457608294e-05,
2444
+ "loss": 0.2012,
2445
+ "step": 3470
2446
+ },
2447
+ {
2448
+ "epoch": 0.64,
2449
+ "grad_norm": 2.25356388092041,
2450
+ "learning_rate": 9.394113291373567e-05,
2451
+ "loss": 0.2023,
2452
+ "step": 3480
2453
+ },
2454
+ {
2455
+ "epoch": 0.64,
2456
+ "grad_norm": 1.400888442993164,
2457
+ "learning_rate": 9.392262125138837e-05,
2458
+ "loss": 0.1998,
2459
+ "step": 3490
2460
+ },
2461
+ {
2462
+ "epoch": 0.65,
2463
+ "grad_norm": 0.43919339776039124,
2464
+ "learning_rate": 9.39041095890411e-05,
2465
+ "loss": 0.1928,
2466
+ "step": 3500
2467
+ },
2468
+ {
2469
+ "epoch": 0.65,
2470
+ "grad_norm": 1.5835129022598267,
2471
+ "learning_rate": 9.388559792669382e-05,
2472
+ "loss": 0.2013,
2473
+ "step": 3510
2474
+ },
2475
+ {
2476
+ "epoch": 0.65,
2477
+ "grad_norm": 0.9350130558013916,
2478
+ "learning_rate": 9.386708626434654e-05,
2479
+ "loss": 0.1965,
2480
+ "step": 3520
2481
+ },
2482
+ {
2483
+ "epoch": 0.65,
2484
+ "grad_norm": 0.47554656863212585,
2485
+ "learning_rate": 9.384857460199926e-05,
2486
+ "loss": 0.1922,
2487
+ "step": 3530
2488
+ },
2489
+ {
2490
+ "epoch": 0.65,
2491
+ "grad_norm": 1.0913207530975342,
2492
+ "learning_rate": 9.383006293965199e-05,
2493
+ "loss": 0.2029,
2494
+ "step": 3540
2495
+ },
2496
+ {
2497
+ "epoch": 0.65,
2498
+ "grad_norm": 0.34091272950172424,
2499
+ "learning_rate": 9.381155127730472e-05,
2500
+ "loss": 0.1923,
2501
+ "step": 3550
2502
+ },
2503
+ {
2504
+ "epoch": 0.66,
2505
+ "grad_norm": 0.5650482773780823,
2506
+ "learning_rate": 9.379303961495742e-05,
2507
+ "loss": 0.2056,
2508
+ "step": 3560
2509
+ },
2510
+ {
2511
+ "epoch": 0.66,
2512
+ "grad_norm": 0.7033718228340149,
2513
+ "learning_rate": 9.377452795261015e-05,
2514
+ "loss": 0.203,
2515
+ "step": 3570
2516
+ },
2517
+ {
2518
+ "epoch": 0.66,
2519
+ "grad_norm": 0.5366137623786926,
2520
+ "learning_rate": 9.375601629026287e-05,
2521
+ "loss": 0.1995,
2522
+ "step": 3580
2523
+ },
2524
+ {
2525
+ "epoch": 0.66,
2526
+ "grad_norm": 1.450621485710144,
2527
+ "learning_rate": 9.37375046279156e-05,
2528
+ "loss": 0.2102,
2529
+ "step": 3590
2530
+ },
2531
+ {
2532
+ "epoch": 0.66,
2533
+ "grad_norm": 0.5831088423728943,
2534
+ "learning_rate": 9.37189929655683e-05,
2535
+ "loss": 0.2112,
2536
+ "step": 3600
2537
+ },
2538
+ {
2539
+ "epoch": 0.67,
2540
+ "grad_norm": 1.3041341304779053,
2541
+ "learning_rate": 9.370048130322103e-05,
2542
+ "loss": 0.2059,
2543
+ "step": 3610
2544
+ },
2545
+ {
2546
+ "epoch": 0.67,
2547
+ "grad_norm": 0.39401304721832275,
2548
+ "learning_rate": 9.368196964087375e-05,
2549
+ "loss": 0.2047,
2550
+ "step": 3620
2551
+ },
2552
+ {
2553
+ "epoch": 0.67,
2554
+ "grad_norm": 0.43465203046798706,
2555
+ "learning_rate": 9.366345797852648e-05,
2556
+ "loss": 0.1872,
2557
+ "step": 3630
2558
+ },
2559
+ {
2560
+ "epoch": 0.67,
2561
+ "grad_norm": 0.34132349491119385,
2562
+ "learning_rate": 9.36449463161792e-05,
2563
+ "loss": 0.2011,
2564
+ "step": 3640
2565
+ },
2566
+ {
2567
+ "epoch": 0.67,
2568
+ "grad_norm": 0.29983776807785034,
2569
+ "learning_rate": 9.362643465383192e-05,
2570
+ "loss": 0.1926,
2571
+ "step": 3650
2572
+ },
2573
+ {
2574
+ "epoch": 0.67,
2575
+ "grad_norm": 0.3644193410873413,
2576
+ "learning_rate": 9.360792299148465e-05,
2577
+ "loss": 0.2003,
2578
+ "step": 3660
2579
+ },
2580
+ {
2581
+ "epoch": 0.68,
2582
+ "grad_norm": 0.7472958564758301,
2583
+ "learning_rate": 9.358941132913736e-05,
2584
+ "loss": 0.2162,
2585
+ "step": 3670
2586
+ },
2587
+ {
2588
+ "epoch": 0.68,
2589
+ "grad_norm": 1.2634315490722656,
2590
+ "learning_rate": 9.357089966679008e-05,
2591
+ "loss": 0.2097,
2592
+ "step": 3680
2593
+ },
2594
+ {
2595
+ "epoch": 0.68,
2596
+ "grad_norm": 0.3091323971748352,
2597
+ "learning_rate": 9.35523880044428e-05,
2598
+ "loss": 0.1798,
2599
+ "step": 3690
2600
+ },
2601
+ {
2602
+ "epoch": 0.68,
2603
+ "grad_norm": 0.33748623728752136,
2604
+ "learning_rate": 9.353387634209553e-05,
2605
+ "loss": 0.1919,
2606
+ "step": 3700
2607
+ },
2608
+ {
2609
+ "epoch": 0.68,
2610
+ "grad_norm": 0.7219259142875671,
2611
+ "learning_rate": 9.351536467974825e-05,
2612
+ "loss": 0.2047,
2613
+ "step": 3710
2614
+ },
2615
+ {
2616
+ "epoch": 0.69,
2617
+ "grad_norm": 0.3145447075366974,
2618
+ "learning_rate": 9.349685301740096e-05,
2619
+ "loss": 0.2014,
2620
+ "step": 3720
2621
+ },
2622
+ {
2623
+ "epoch": 0.69,
2624
+ "grad_norm": 1.3122820854187012,
2625
+ "learning_rate": 9.34783413550537e-05,
2626
+ "loss": 0.1915,
2627
+ "step": 3730
2628
+ },
2629
+ {
2630
+ "epoch": 0.69,
2631
+ "grad_norm": 0.5373870134353638,
2632
+ "learning_rate": 9.345982969270641e-05,
2633
+ "loss": 0.1953,
2634
+ "step": 3740
2635
+ },
2636
+ {
2637
+ "epoch": 0.69,
2638
+ "grad_norm": 0.5105752348899841,
2639
+ "learning_rate": 9.344131803035913e-05,
2640
+ "loss": 0.2075,
2641
+ "step": 3750
2642
+ },
2643
+ {
2644
+ "epoch": 0.69,
2645
+ "grad_norm": 0.5081068277359009,
2646
+ "learning_rate": 9.342280636801185e-05,
2647
+ "loss": 0.1976,
2648
+ "step": 3760
2649
+ },
2650
+ {
2651
+ "epoch": 0.7,
2652
+ "grad_norm": 0.26259544491767883,
2653
+ "learning_rate": 9.340429470566458e-05,
2654
+ "loss": 0.1984,
2655
+ "step": 3770
2656
+ },
2657
+ {
2658
+ "epoch": 0.7,
2659
+ "grad_norm": 0.6756144165992737,
2660
+ "learning_rate": 9.33857830433173e-05,
2661
+ "loss": 0.1986,
2662
+ "step": 3780
2663
+ },
2664
+ {
2665
+ "epoch": 0.7,
2666
+ "grad_norm": 0.6026384234428406,
2667
+ "learning_rate": 9.336727138097001e-05,
2668
+ "loss": 0.1957,
2669
+ "step": 3790
2670
+ },
2671
+ {
2672
+ "epoch": 0.7,
2673
+ "grad_norm": 0.5947535037994385,
2674
+ "learning_rate": 9.334875971862273e-05,
2675
+ "loss": 0.195,
2676
+ "step": 3800
2677
+ },
2678
+ {
2679
+ "epoch": 0.7,
2680
+ "grad_norm": 0.6168169379234314,
2681
+ "learning_rate": 9.333024805627546e-05,
2682
+ "loss": 0.1951,
2683
+ "step": 3810
2684
+ },
2685
+ {
2686
+ "epoch": 0.7,
2687
+ "grad_norm": 1.3259109258651733,
2688
+ "learning_rate": 9.331173639392818e-05,
2689
+ "loss": 0.197,
2690
+ "step": 3820
2691
+ },
2692
+ {
2693
+ "epoch": 0.71,
2694
+ "grad_norm": 0.5684218406677246,
2695
+ "learning_rate": 9.32932247315809e-05,
2696
+ "loss": 0.198,
2697
+ "step": 3830
2698
+ },
2699
+ {
2700
+ "epoch": 0.71,
2701
+ "grad_norm": 1.8599364757537842,
2702
+ "learning_rate": 9.327471306923363e-05,
2703
+ "loss": 0.2114,
2704
+ "step": 3840
2705
+ },
2706
+ {
2707
+ "epoch": 0.71,
2708
+ "grad_norm": 0.33460474014282227,
2709
+ "learning_rate": 9.325620140688634e-05,
2710
+ "loss": 0.1891,
2711
+ "step": 3850
2712
+ },
2713
+ {
2714
+ "epoch": 0.71,
2715
+ "grad_norm": 2.3144965171813965,
2716
+ "learning_rate": 9.323768974453907e-05,
2717
+ "loss": 0.1831,
2718
+ "step": 3860
2719
+ },
2720
+ {
2721
+ "epoch": 0.71,
2722
+ "grad_norm": 0.7101608514785767,
2723
+ "learning_rate": 9.321917808219178e-05,
2724
+ "loss": 0.2025,
2725
+ "step": 3870
2726
+ },
2727
+ {
2728
+ "epoch": 0.72,
2729
+ "grad_norm": 0.42260223627090454,
2730
+ "learning_rate": 9.320066641984451e-05,
2731
+ "loss": 0.2095,
2732
+ "step": 3880
2733
+ },
2734
+ {
2735
+ "epoch": 0.72,
2736
+ "grad_norm": 0.473034143447876,
2737
+ "learning_rate": 9.318215475749723e-05,
2738
+ "loss": 0.1965,
2739
+ "step": 3890
2740
+ },
2741
+ {
2742
+ "epoch": 0.72,
2743
+ "grad_norm": 0.4022839665412903,
2744
+ "learning_rate": 9.316364309514994e-05,
2745
+ "loss": 0.1877,
2746
+ "step": 3900
2747
+ },
2748
+ {
2749
+ "epoch": 0.72,
2750
+ "grad_norm": 0.41458505392074585,
2751
+ "learning_rate": 9.314513143280267e-05,
2752
+ "loss": 0.1973,
2753
+ "step": 3910
2754
+ },
2755
+ {
2756
+ "epoch": 0.72,
2757
+ "grad_norm": 0.46193191409111023,
2758
+ "learning_rate": 9.312661977045539e-05,
2759
+ "loss": 0.2133,
2760
+ "step": 3920
2761
+ },
2762
+ {
2763
+ "epoch": 0.72,
2764
+ "grad_norm": 1.0600367784500122,
2765
+ "learning_rate": 9.310810810810812e-05,
2766
+ "loss": 0.1895,
2767
+ "step": 3930
2768
+ },
2769
+ {
2770
+ "epoch": 0.73,
2771
+ "grad_norm": 0.3262641429901123,
2772
+ "learning_rate": 9.308959644576083e-05,
2773
+ "loss": 0.1975,
2774
+ "step": 3940
2775
+ },
2776
+ {
2777
+ "epoch": 0.73,
2778
+ "grad_norm": 1.1876827478408813,
2779
+ "learning_rate": 9.307108478341356e-05,
2780
+ "loss": 0.1916,
2781
+ "step": 3950
2782
+ },
2783
+ {
2784
+ "epoch": 0.73,
2785
+ "grad_norm": 0.6535803079605103,
2786
+ "learning_rate": 9.305257312106627e-05,
2787
+ "loss": 0.1966,
2788
+ "step": 3960
2789
+ },
2790
+ {
2791
+ "epoch": 0.73,
2792
+ "grad_norm": 0.4089249074459076,
2793
+ "learning_rate": 9.3034061458719e-05,
2794
+ "loss": 0.195,
2795
+ "step": 3970
2796
+ },
2797
+ {
2798
+ "epoch": 0.73,
2799
+ "grad_norm": 1.0222320556640625,
2800
+ "learning_rate": 9.301554979637171e-05,
2801
+ "loss": 0.193,
2802
+ "step": 3980
2803
+ },
2804
+ {
2805
+ "epoch": 0.74,
2806
+ "grad_norm": 1.0429261922836304,
2807
+ "learning_rate": 9.299703813402444e-05,
2808
+ "loss": 0.2039,
2809
+ "step": 3990
2810
+ },
2811
+ {
2812
+ "epoch": 0.74,
2813
+ "grad_norm": 2.6728320121765137,
2814
+ "learning_rate": 9.297852647167716e-05,
2815
+ "loss": 0.1963,
2816
+ "step": 4000
2817
+ },
2818
+ {
2819
+ "epoch": 0.74,
2820
+ "grad_norm": 0.4279372990131378,
2821
+ "learning_rate": 9.296001480932989e-05,
2822
+ "loss": 0.1895,
2823
+ "step": 4010
2824
+ },
2825
+ {
2826
+ "epoch": 0.74,
2827
+ "grad_norm": 0.4307747781276703,
2828
+ "learning_rate": 9.29415031469826e-05,
2829
+ "loss": 0.1967,
2830
+ "step": 4020
2831
+ },
2832
+ {
2833
+ "epoch": 0.74,
2834
+ "grad_norm": 0.44748997688293457,
2835
+ "learning_rate": 9.292299148463532e-05,
2836
+ "loss": 0.1945,
2837
+ "step": 4030
2838
+ },
2839
+ {
2840
+ "epoch": 0.75,
2841
+ "grad_norm": 0.3396751284599304,
2842
+ "learning_rate": 9.290447982228805e-05,
2843
+ "loss": 0.1901,
2844
+ "step": 4040
2845
+ },
2846
+ {
2847
+ "epoch": 0.75,
2848
+ "grad_norm": 1.2306305170059204,
2849
+ "learning_rate": 9.288596815994076e-05,
2850
+ "loss": 0.1915,
2851
+ "step": 4050
2852
+ },
2853
+ {
2854
+ "epoch": 0.75,
2855
+ "grad_norm": 1.0502023696899414,
2856
+ "learning_rate": 9.286745649759349e-05,
2857
+ "loss": 0.2001,
2858
+ "step": 4060
2859
+ },
2860
+ {
2861
+ "epoch": 0.75,
2862
+ "grad_norm": 0.2933097779750824,
2863
+ "learning_rate": 9.28489448352462e-05,
2864
+ "loss": 0.1911,
2865
+ "step": 4070
2866
+ },
2867
+ {
2868
+ "epoch": 0.75,
2869
+ "grad_norm": 0.7432071566581726,
2870
+ "learning_rate": 9.283043317289894e-05,
2871
+ "loss": 0.1882,
2872
+ "step": 4080
2873
+ },
2874
+ {
2875
+ "epoch": 0.75,
2876
+ "grad_norm": 1.2065651416778564,
2877
+ "learning_rate": 9.281192151055165e-05,
2878
+ "loss": 0.2026,
2879
+ "step": 4090
2880
+ },
2881
+ {
2882
+ "epoch": 0.76,
2883
+ "grad_norm": 0.331061452627182,
2884
+ "learning_rate": 9.279340984820437e-05,
2885
+ "loss": 0.1941,
2886
+ "step": 4100
2887
+ },
2888
+ {
2889
+ "epoch": 0.76,
2890
+ "grad_norm": 0.819383978843689,
2891
+ "learning_rate": 9.27748981858571e-05,
2892
+ "loss": 0.1951,
2893
+ "step": 4110
2894
+ },
2895
+ {
2896
+ "epoch": 0.76,
2897
+ "grad_norm": 0.4251770079135895,
2898
+ "learning_rate": 9.275638652350982e-05,
2899
+ "loss": 0.1976,
2900
+ "step": 4120
2901
+ },
2902
+ {
2903
+ "epoch": 0.76,
2904
+ "grad_norm": 1.0525916814804077,
2905
+ "learning_rate": 9.273787486116254e-05,
2906
+ "loss": 0.1998,
2907
+ "step": 4130
2908
+ },
2909
+ {
2910
+ "epoch": 0.76,
2911
+ "grad_norm": 1.4899557828903198,
2912
+ "learning_rate": 9.271936319881525e-05,
2913
+ "loss": 0.1933,
2914
+ "step": 4140
2915
+ },
2916
+ {
2917
+ "epoch": 0.77,
2918
+ "grad_norm": 0.8893213272094727,
2919
+ "learning_rate": 9.270085153646798e-05,
2920
+ "loss": 0.1943,
2921
+ "step": 4150
2922
+ },
2923
+ {
2924
+ "epoch": 0.77,
2925
+ "grad_norm": 0.6278595924377441,
2926
+ "learning_rate": 9.26823398741207e-05,
2927
+ "loss": 0.1973,
2928
+ "step": 4160
2929
+ },
2930
+ {
2931
+ "epoch": 0.77,
2932
+ "grad_norm": 1.8130468130111694,
2933
+ "learning_rate": 9.266382821177342e-05,
2934
+ "loss": 0.1954,
2935
+ "step": 4170
2936
+ },
2937
+ {
2938
+ "epoch": 0.77,
2939
+ "grad_norm": 1.0211946964263916,
2940
+ "learning_rate": 9.264531654942615e-05,
2941
+ "loss": 0.1937,
2942
+ "step": 4180
2943
+ },
2944
+ {
2945
+ "epoch": 0.77,
2946
+ "grad_norm": 0.3028915226459503,
2947
+ "learning_rate": 9.262680488707887e-05,
2948
+ "loss": 0.208,
2949
+ "step": 4190
2950
+ },
2951
+ {
2952
+ "epoch": 0.77,
2953
+ "grad_norm": 0.33269938826560974,
2954
+ "learning_rate": 9.26082932247316e-05,
2955
+ "loss": 0.2032,
2956
+ "step": 4200
2957
+ },
2958
+ {
2959
+ "epoch": 0.78,
2960
+ "grad_norm": 0.9252434372901917,
2961
+ "learning_rate": 9.25897815623843e-05,
2962
+ "loss": 0.2018,
2963
+ "step": 4210
2964
+ },
2965
+ {
2966
+ "epoch": 0.78,
2967
+ "grad_norm": 1.3774528503417969,
2968
+ "learning_rate": 9.257126990003703e-05,
2969
+ "loss": 0.1895,
2970
+ "step": 4220
2971
+ },
2972
+ {
2973
+ "epoch": 0.78,
2974
+ "grad_norm": 1.9812060594558716,
2975
+ "learning_rate": 9.255275823768975e-05,
2976
+ "loss": 0.1951,
2977
+ "step": 4230
2978
+ },
2979
+ {
2980
+ "epoch": 0.78,
2981
+ "grad_norm": 1.7997092008590698,
2982
+ "learning_rate": 9.253424657534247e-05,
2983
+ "loss": 0.2014,
2984
+ "step": 4240
2985
+ },
2986
+ {
2987
+ "epoch": 0.78,
2988
+ "grad_norm": 0.35307908058166504,
2989
+ "learning_rate": 9.251573491299518e-05,
2990
+ "loss": 0.2026,
2991
+ "step": 4250
2992
+ },
2993
+ {
2994
+ "epoch": 0.79,
2995
+ "grad_norm": 0.906162440776825,
2996
+ "learning_rate": 9.249722325064791e-05,
2997
+ "loss": 0.1892,
2998
+ "step": 4260
2999
+ },
3000
+ {
3001
+ "epoch": 0.79,
3002
+ "grad_norm": 0.8749983906745911,
3003
+ "learning_rate": 9.247871158830063e-05,
3004
+ "loss": 0.2037,
3005
+ "step": 4270
3006
+ },
3007
+ {
3008
+ "epoch": 0.79,
3009
+ "grad_norm": 1.1504167318344116,
3010
+ "learning_rate": 9.246019992595335e-05,
3011
+ "loss": 0.1942,
3012
+ "step": 4280
3013
+ },
3014
+ {
3015
+ "epoch": 0.79,
3016
+ "grad_norm": 0.6674988865852356,
3017
+ "learning_rate": 9.244168826360608e-05,
3018
+ "loss": 0.1974,
3019
+ "step": 4290
3020
+ },
3021
+ {
3022
+ "epoch": 0.79,
3023
+ "grad_norm": 1.0109151601791382,
3024
+ "learning_rate": 9.24231766012588e-05,
3025
+ "loss": 0.1937,
3026
+ "step": 4300
3027
+ },
3028
+ {
3029
+ "epoch": 0.79,
3030
+ "grad_norm": 0.5439345836639404,
3031
+ "learning_rate": 9.240466493891153e-05,
3032
+ "loss": 0.1825,
3033
+ "step": 4310
3034
+ },
3035
+ {
3036
+ "epoch": 0.8,
3037
+ "grad_norm": 0.3471992015838623,
3038
+ "learning_rate": 9.238615327656423e-05,
3039
+ "loss": 0.1932,
3040
+ "step": 4320
3041
+ },
3042
+ {
3043
+ "epoch": 0.8,
3044
+ "grad_norm": 0.5230699181556702,
3045
+ "learning_rate": 9.236764161421696e-05,
3046
+ "loss": 0.1869,
3047
+ "step": 4330
3048
+ },
3049
+ {
3050
+ "epoch": 0.8,
3051
+ "grad_norm": 1.1102708578109741,
3052
+ "learning_rate": 9.234912995186968e-05,
3053
+ "loss": 0.1934,
3054
+ "step": 4340
3055
+ },
3056
+ {
3057
+ "epoch": 0.8,
3058
+ "grad_norm": 0.2299565076828003,
3059
+ "learning_rate": 9.233061828952241e-05,
3060
+ "loss": 0.1946,
3061
+ "step": 4350
3062
+ },
3063
+ {
3064
+ "epoch": 0.8,
3065
+ "grad_norm": 2.019484281539917,
3066
+ "learning_rate": 9.231210662717513e-05,
3067
+ "loss": 0.2005,
3068
+ "step": 4360
3069
+ },
3070
+ {
3071
+ "epoch": 0.81,
3072
+ "grad_norm": 0.49182599782943726,
3073
+ "learning_rate": 9.229359496482784e-05,
3074
+ "loss": 0.1862,
3075
+ "step": 4370
3076
+ },
3077
+ {
3078
+ "epoch": 0.81,
3079
+ "grad_norm": 1.1424809694290161,
3080
+ "learning_rate": 9.227508330248058e-05,
3081
+ "loss": 0.1857,
3082
+ "step": 4380
3083
+ },
3084
+ {
3085
+ "epoch": 0.81,
3086
+ "grad_norm": 0.5882079601287842,
3087
+ "learning_rate": 9.225657164013329e-05,
3088
+ "loss": 0.1866,
3089
+ "step": 4390
3090
+ },
3091
+ {
3092
+ "epoch": 0.81,
3093
+ "grad_norm": 0.8041399717330933,
3094
+ "learning_rate": 9.223805997778601e-05,
3095
+ "loss": 0.1882,
3096
+ "step": 4400
3097
+ },
3098
+ {
3099
+ "epoch": 0.81,
3100
+ "grad_norm": 1.2162193059921265,
3101
+ "learning_rate": 9.221954831543873e-05,
3102
+ "loss": 0.1893,
3103
+ "step": 4410
3104
+ },
3105
+ {
3106
+ "epoch": 0.82,
3107
+ "grad_norm": 0.516737163066864,
3108
+ "learning_rate": 9.220103665309146e-05,
3109
+ "loss": 0.1954,
3110
+ "step": 4420
3111
+ },
3112
+ {
3113
+ "epoch": 0.82,
3114
+ "grad_norm": 0.78972327709198,
3115
+ "learning_rate": 9.218252499074416e-05,
3116
+ "loss": 0.1959,
3117
+ "step": 4430
3118
+ },
3119
+ {
3120
+ "epoch": 0.82,
3121
+ "grad_norm": 1.6012400388717651,
3122
+ "learning_rate": 9.216401332839689e-05,
3123
+ "loss": 0.2035,
3124
+ "step": 4440
3125
+ },
3126
+ {
3127
+ "epoch": 0.82,
3128
+ "grad_norm": 0.8954785466194153,
3129
+ "learning_rate": 9.214550166604961e-05,
3130
+ "loss": 0.1922,
3131
+ "step": 4450
3132
+ },
3133
+ {
3134
+ "epoch": 0.82,
3135
+ "grad_norm": 0.6251717805862427,
3136
+ "learning_rate": 9.212699000370234e-05,
3137
+ "loss": 0.1898,
3138
+ "step": 4460
3139
+ },
3140
+ {
3141
+ "epoch": 0.82,
3142
+ "grad_norm": 1.7891993522644043,
3143
+ "learning_rate": 9.210847834135506e-05,
3144
+ "loss": 0.2006,
3145
+ "step": 4470
3146
+ },
3147
+ {
3148
+ "epoch": 0.83,
3149
+ "grad_norm": 0.3996617794036865,
3150
+ "learning_rate": 9.208996667900777e-05,
3151
+ "loss": 0.2078,
3152
+ "step": 4480
3153
+ },
3154
+ {
3155
+ "epoch": 0.83,
3156
+ "grad_norm": 1.112613320350647,
3157
+ "learning_rate": 9.20714550166605e-05,
3158
+ "loss": 0.2025,
3159
+ "step": 4490
3160
+ },
3161
+ {
3162
+ "epoch": 0.83,
3163
+ "grad_norm": 0.3055773079395294,
3164
+ "learning_rate": 9.205294335431322e-05,
3165
+ "loss": 0.193,
3166
+ "step": 4500
3167
+ },
3168
+ {
3169
+ "epoch": 0.83,
3170
+ "grad_norm": 0.9582087993621826,
3171
+ "learning_rate": 9.203443169196594e-05,
3172
+ "loss": 0.1899,
3173
+ "step": 4510
3174
+ },
3175
+ {
3176
+ "epoch": 0.83,
3177
+ "grad_norm": 0.7985404133796692,
3178
+ "learning_rate": 9.201592002961866e-05,
3179
+ "loss": 0.1868,
3180
+ "step": 4520
3181
+ },
3182
+ {
3183
+ "epoch": 0.84,
3184
+ "grad_norm": 1.6095155477523804,
3185
+ "learning_rate": 9.199740836727139e-05,
3186
+ "loss": 0.1899,
3187
+ "step": 4530
3188
+ },
3189
+ {
3190
+ "epoch": 0.84,
3191
+ "grad_norm": 0.6676468849182129,
3192
+ "learning_rate": 9.19788967049241e-05,
3193
+ "loss": 0.1969,
3194
+ "step": 4540
3195
+ },
3196
+ {
3197
+ "epoch": 0.84,
3198
+ "grad_norm": 1.045982003211975,
3199
+ "learning_rate": 9.196038504257682e-05,
3200
+ "loss": 0.1814,
3201
+ "step": 4550
3202
+ },
3203
+ {
3204
+ "epoch": 0.84,
3205
+ "grad_norm": 0.7359940409660339,
3206
+ "learning_rate": 9.194187338022955e-05,
3207
+ "loss": 0.1865,
3208
+ "step": 4560
3209
+ },
3210
+ {
3211
+ "epoch": 0.84,
3212
+ "grad_norm": 0.9083600044250488,
3213
+ "learning_rate": 9.192336171788227e-05,
3214
+ "loss": 0.2012,
3215
+ "step": 4570
3216
+ },
3217
+ {
3218
+ "epoch": 0.84,
3219
+ "grad_norm": 0.340991348028183,
3220
+ "learning_rate": 9.1904850055535e-05,
3221
+ "loss": 0.1892,
3222
+ "step": 4580
3223
+ },
3224
+ {
3225
+ "epoch": 0.85,
3226
+ "grad_norm": 0.2833189368247986,
3227
+ "learning_rate": 9.18863383931877e-05,
3228
+ "loss": 0.1888,
3229
+ "step": 4590
3230
+ },
3231
+ {
3232
+ "epoch": 0.85,
3233
+ "grad_norm": 0.6355910301208496,
3234
+ "learning_rate": 9.186782673084044e-05,
3235
+ "loss": 0.2041,
3236
+ "step": 4600
3237
+ },
3238
+ {
3239
+ "epoch": 0.85,
3240
+ "grad_norm": 0.7121712565422058,
3241
+ "learning_rate": 9.184931506849315e-05,
3242
+ "loss": 0.1783,
3243
+ "step": 4610
3244
+ },
3245
+ {
3246
+ "epoch": 0.85,
3247
+ "grad_norm": 0.8881193399429321,
3248
+ "learning_rate": 9.183080340614587e-05,
3249
+ "loss": 0.1981,
3250
+ "step": 4620
3251
+ },
3252
+ {
3253
+ "epoch": 0.85,
3254
+ "grad_norm": 1.2488733530044556,
3255
+ "learning_rate": 9.18122917437986e-05,
3256
+ "loss": 0.2016,
3257
+ "step": 4630
3258
+ },
3259
+ {
3260
+ "epoch": 0.86,
3261
+ "grad_norm": 0.283454567193985,
3262
+ "learning_rate": 9.179378008145132e-05,
3263
+ "loss": 0.1966,
3264
+ "step": 4640
3265
+ },
3266
+ {
3267
+ "epoch": 0.86,
3268
+ "grad_norm": 0.306062787771225,
3269
+ "learning_rate": 9.177526841910405e-05,
3270
+ "loss": 0.193,
3271
+ "step": 4650
3272
+ },
3273
+ {
3274
+ "epoch": 0.86,
3275
+ "grad_norm": 0.45514628291130066,
3276
+ "learning_rate": 9.175675675675675e-05,
3277
+ "loss": 0.192,
3278
+ "step": 4660
3279
+ },
3280
+ {
3281
+ "epoch": 0.86,
3282
+ "grad_norm": 0.6448143124580383,
3283
+ "learning_rate": 9.173824509440948e-05,
3284
+ "loss": 0.1996,
3285
+ "step": 4670
3286
+ },
3287
+ {
3288
+ "epoch": 0.86,
3289
+ "grad_norm": 0.8043811321258545,
3290
+ "learning_rate": 9.17197334320622e-05,
3291
+ "loss": 0.1973,
3292
+ "step": 4680
3293
+ },
3294
+ {
3295
+ "epoch": 0.86,
3296
+ "grad_norm": 0.3627445101737976,
3297
+ "learning_rate": 9.170122176971493e-05,
3298
+ "loss": 0.1864,
3299
+ "step": 4690
3300
+ },
3301
+ {
3302
+ "epoch": 0.87,
3303
+ "grad_norm": 1.8266276121139526,
3304
+ "learning_rate": 9.168271010736764e-05,
3305
+ "loss": 0.1939,
3306
+ "step": 4700
3307
+ },
3308
+ {
3309
+ "epoch": 0.87,
3310
+ "grad_norm": 0.8964821100234985,
3311
+ "learning_rate": 9.166419844502037e-05,
3312
+ "loss": 0.21,
3313
+ "step": 4710
3314
+ },
3315
+ {
3316
+ "epoch": 0.87,
3317
+ "grad_norm": 2.066237449645996,
3318
+ "learning_rate": 9.164568678267308e-05,
3319
+ "loss": 0.2042,
3320
+ "step": 4720
3321
+ },
3322
+ {
3323
+ "epoch": 0.87,
3324
+ "grad_norm": 0.5604624152183533,
3325
+ "learning_rate": 9.162717512032582e-05,
3326
+ "loss": 0.195,
3327
+ "step": 4730
3328
+ },
3329
+ {
3330
+ "epoch": 0.87,
3331
+ "grad_norm": 0.3185977041721344,
3332
+ "learning_rate": 9.160866345797853e-05,
3333
+ "loss": 0.1913,
3334
+ "step": 4740
3335
+ },
3336
+ {
3337
+ "epoch": 0.88,
3338
+ "grad_norm": 0.7183963060379028,
3339
+ "learning_rate": 9.159015179563125e-05,
3340
+ "loss": 0.1893,
3341
+ "step": 4750
3342
+ },
3343
+ {
3344
+ "epoch": 0.88,
3345
+ "grad_norm": 0.30005547404289246,
3346
+ "learning_rate": 9.157164013328398e-05,
3347
+ "loss": 0.2004,
3348
+ "step": 4760
3349
+ },
3350
+ {
3351
+ "epoch": 0.88,
3352
+ "grad_norm": 1.1535359621047974,
3353
+ "learning_rate": 9.15531284709367e-05,
3354
+ "loss": 0.1859,
3355
+ "step": 4770
3356
+ },
3357
+ {
3358
+ "epoch": 0.88,
3359
+ "grad_norm": 0.46643924713134766,
3360
+ "learning_rate": 9.153461680858941e-05,
3361
+ "loss": 0.198,
3362
+ "step": 4780
3363
+ },
3364
+ {
3365
+ "epoch": 0.88,
3366
+ "grad_norm": 0.45236846804618835,
3367
+ "learning_rate": 9.151610514624213e-05,
3368
+ "loss": 0.1911,
3369
+ "step": 4790
3370
+ },
3371
+ {
3372
+ "epoch": 0.89,
3373
+ "grad_norm": 0.8338435888290405,
3374
+ "learning_rate": 9.149759348389486e-05,
3375
+ "loss": 0.1944,
3376
+ "step": 4800
3377
+ },
3378
+ {
3379
+ "epoch": 0.89,
3380
+ "grad_norm": 1.7582203149795532,
3381
+ "learning_rate": 9.147908182154758e-05,
3382
+ "loss": 0.1973,
3383
+ "step": 4810
3384
+ },
3385
+ {
3386
+ "epoch": 0.89,
3387
+ "grad_norm": 1.3786252737045288,
3388
+ "learning_rate": 9.14605701592003e-05,
3389
+ "loss": 0.1867,
3390
+ "step": 4820
3391
+ },
3392
+ {
3393
+ "epoch": 0.89,
3394
+ "grad_norm": 0.376365602016449,
3395
+ "learning_rate": 9.144205849685303e-05,
3396
+ "loss": 0.1831,
3397
+ "step": 4830
3398
+ },
3399
+ {
3400
+ "epoch": 0.89,
3401
+ "grad_norm": 0.7689598798751831,
3402
+ "learning_rate": 9.142354683450575e-05,
3403
+ "loss": 0.1911,
3404
+ "step": 4840
3405
+ },
3406
+ {
3407
+ "epoch": 0.89,
3408
+ "grad_norm": 0.9973986744880676,
3409
+ "learning_rate": 9.140503517215846e-05,
3410
+ "loss": 0.1984,
3411
+ "step": 4850
3412
+ },
3413
+ {
3414
+ "epoch": 0.9,
3415
+ "grad_norm": 2.4425342082977295,
3416
+ "learning_rate": 9.138652350981118e-05,
3417
+ "loss": 0.1927,
3418
+ "step": 4860
3419
+ },
3420
+ {
3421
+ "epoch": 0.9,
3422
+ "grad_norm": 0.4806678295135498,
3423
+ "learning_rate": 9.136801184746391e-05,
3424
+ "loss": 0.1857,
3425
+ "step": 4870
3426
+ },
3427
+ {
3428
+ "epoch": 0.9,
3429
+ "grad_norm": 0.6703113913536072,
3430
+ "learning_rate": 9.134950018511663e-05,
3431
+ "loss": 0.1924,
3432
+ "step": 4880
3433
+ },
3434
+ {
3435
+ "epoch": 0.9,
3436
+ "grad_norm": 0.6484790444374084,
3437
+ "learning_rate": 9.133098852276935e-05,
3438
+ "loss": 0.1774,
3439
+ "step": 4890
3440
+ },
3441
+ {
3442
+ "epoch": 0.9,
3443
+ "grad_norm": 1.3424222469329834,
3444
+ "learning_rate": 9.131247686042206e-05,
3445
+ "loss": 0.1834,
3446
+ "step": 4900
3447
+ },
3448
+ {
3449
+ "epoch": 0.91,
3450
+ "grad_norm": 1.3998688459396362,
3451
+ "learning_rate": 9.12939651980748e-05,
3452
+ "loss": 0.1856,
3453
+ "step": 4910
3454
+ },
3455
+ {
3456
+ "epoch": 0.91,
3457
+ "grad_norm": 0.37422481179237366,
3458
+ "learning_rate": 9.127545353572751e-05,
3459
+ "loss": 0.1875,
3460
+ "step": 4920
3461
+ },
3462
+ {
3463
+ "epoch": 0.91,
3464
+ "grad_norm": 1.2003308534622192,
3465
+ "learning_rate": 9.125694187338023e-05,
3466
+ "loss": 0.1886,
3467
+ "step": 4930
3468
+ },
3469
+ {
3470
+ "epoch": 0.91,
3471
+ "grad_norm": 1.248398780822754,
3472
+ "learning_rate": 9.123843021103296e-05,
3473
+ "loss": 0.1918,
3474
+ "step": 4940
3475
+ },
3476
+ {
3477
+ "epoch": 0.91,
3478
+ "grad_norm": 0.6824427247047424,
3479
+ "learning_rate": 9.121991854868568e-05,
3480
+ "loss": 0.1795,
3481
+ "step": 4950
3482
+ },
3483
+ {
3484
+ "epoch": 0.91,
3485
+ "grad_norm": 0.5873004794120789,
3486
+ "learning_rate": 9.120140688633841e-05,
3487
+ "loss": 0.1958,
3488
+ "step": 4960
3489
+ },
3490
+ {
3491
+ "epoch": 0.92,
3492
+ "grad_norm": 0.7316002249717712,
3493
+ "learning_rate": 9.118289522399111e-05,
3494
+ "loss": 0.1794,
3495
+ "step": 4970
3496
+ },
3497
+ {
3498
+ "epoch": 0.92,
3499
+ "grad_norm": 0.40432649850845337,
3500
+ "learning_rate": 9.116438356164384e-05,
3501
+ "loss": 0.1845,
3502
+ "step": 4980
3503
+ },
3504
+ {
3505
+ "epoch": 0.92,
3506
+ "grad_norm": 1.2551237344741821,
3507
+ "learning_rate": 9.114587189929656e-05,
3508
+ "loss": 0.1847,
3509
+ "step": 4990
3510
+ },
3511
+ {
3512
+ "epoch": 0.92,
3513
+ "grad_norm": 1.0834324359893799,
3514
+ "learning_rate": 9.112736023694928e-05,
3515
+ "loss": 0.1942,
3516
+ "step": 5000
3517
+ },
3518
+ {
3519
+ "epoch": 0.92,
3520
+ "grad_norm": 0.5848122835159302,
3521
+ "learning_rate": 9.110884857460201e-05,
3522
+ "loss": 0.1909,
3523
+ "step": 5010
3524
+ },
3525
+ {
3526
+ "epoch": 0.93,
3527
+ "grad_norm": 0.34502893686294556,
3528
+ "learning_rate": 9.109033691225472e-05,
3529
+ "loss": 0.1954,
3530
+ "step": 5020
3531
+ },
3532
+ {
3533
+ "epoch": 0.93,
3534
+ "grad_norm": 0.7165638208389282,
3535
+ "learning_rate": 9.107182524990745e-05,
3536
+ "loss": 0.1856,
3537
+ "step": 5030
3538
+ },
3539
+ {
3540
+ "epoch": 0.93,
3541
+ "grad_norm": 1.3096377849578857,
3542
+ "learning_rate": 9.105331358756016e-05,
3543
+ "loss": 0.1865,
3544
+ "step": 5040
3545
+ },
3546
+ {
3547
+ "epoch": 0.93,
3548
+ "grad_norm": 2.7340123653411865,
3549
+ "learning_rate": 9.103480192521289e-05,
3550
+ "loss": 0.1957,
3551
+ "step": 5050
3552
+ },
3553
+ {
3554
+ "epoch": 0.93,
3555
+ "grad_norm": 0.3616611361503601,
3556
+ "learning_rate": 9.101629026286561e-05,
3557
+ "loss": 0.1967,
3558
+ "step": 5060
3559
+ },
3560
+ {
3561
+ "epoch": 0.94,
3562
+ "grad_norm": 0.4186195731163025,
3563
+ "learning_rate": 9.099777860051834e-05,
3564
+ "loss": 0.186,
3565
+ "step": 5070
3566
+ },
3567
+ {
3568
+ "epoch": 0.94,
3569
+ "grad_norm": 0.2687035799026489,
3570
+ "learning_rate": 9.097926693817105e-05,
3571
+ "loss": 0.1873,
3572
+ "step": 5080
3573
+ },
3574
+ {
3575
+ "epoch": 0.94,
3576
+ "grad_norm": 1.4722837209701538,
3577
+ "learning_rate": 9.096075527582377e-05,
3578
+ "loss": 0.1881,
3579
+ "step": 5090
3580
+ },
3581
+ {
3582
+ "epoch": 0.94,
3583
+ "grad_norm": 1.3331010341644287,
3584
+ "learning_rate": 9.09422436134765e-05,
3585
+ "loss": 0.1979,
3586
+ "step": 5100
3587
+ },
3588
+ {
3589
+ "epoch": 0.94,
3590
+ "grad_norm": 0.7769159078598022,
3591
+ "learning_rate": 9.092373195112922e-05,
3592
+ "loss": 0.1957,
3593
+ "step": 5110
3594
+ },
3595
+ {
3596
+ "epoch": 0.94,
3597
+ "grad_norm": 0.9742639064788818,
3598
+ "learning_rate": 9.090522028878194e-05,
3599
+ "loss": 0.1883,
3600
+ "step": 5120
3601
+ },
3602
+ {
3603
+ "epoch": 0.95,
3604
+ "grad_norm": 0.4907156527042389,
3605
+ "learning_rate": 9.088670862643465e-05,
3606
+ "loss": 0.1885,
3607
+ "step": 5130
3608
+ },
3609
+ {
3610
+ "epoch": 0.95,
3611
+ "grad_norm": 1.0384258031845093,
3612
+ "learning_rate": 9.086819696408739e-05,
3613
+ "loss": 0.183,
3614
+ "step": 5140
3615
+ },
3616
+ {
3617
+ "epoch": 0.95,
3618
+ "grad_norm": 0.6687269806861877,
3619
+ "learning_rate": 9.084968530174009e-05,
3620
+ "loss": 0.1986,
3621
+ "step": 5150
3622
+ },
3623
+ {
3624
+ "epoch": 0.95,
3625
+ "grad_norm": 0.49845072627067566,
3626
+ "learning_rate": 9.083117363939282e-05,
3627
+ "loss": 0.1897,
3628
+ "step": 5160
3629
+ },
3630
+ {
3631
+ "epoch": 0.95,
3632
+ "grad_norm": 0.3812813460826874,
3633
+ "learning_rate": 9.081266197704554e-05,
3634
+ "loss": 0.1959,
3635
+ "step": 5170
3636
+ },
3637
+ {
3638
+ "epoch": 0.96,
3639
+ "grad_norm": 1.7052239179611206,
3640
+ "learning_rate": 9.079415031469827e-05,
3641
+ "loss": 0.1955,
3642
+ "step": 5180
3643
+ },
3644
+ {
3645
+ "epoch": 0.96,
3646
+ "grad_norm": 0.8822137713432312,
3647
+ "learning_rate": 9.077563865235099e-05,
3648
+ "loss": 0.1809,
3649
+ "step": 5190
3650
+ },
3651
+ {
3652
+ "epoch": 0.96,
3653
+ "grad_norm": 1.1971176862716675,
3654
+ "learning_rate": 9.07571269900037e-05,
3655
+ "loss": 0.192,
3656
+ "step": 5200
3657
+ },
3658
+ {
3659
+ "epoch": 0.96,
3660
+ "grad_norm": 0.6374858021736145,
3661
+ "learning_rate": 9.073861532765643e-05,
3662
+ "loss": 0.1938,
3663
+ "step": 5210
3664
+ },
3665
+ {
3666
+ "epoch": 0.96,
3667
+ "grad_norm": 0.6527014970779419,
3668
+ "learning_rate": 9.072010366530915e-05,
3669
+ "loss": 0.1933,
3670
+ "step": 5220
3671
+ },
3672
+ {
3673
+ "epoch": 0.96,
3674
+ "grad_norm": 0.628654956817627,
3675
+ "learning_rate": 9.070159200296187e-05,
3676
+ "loss": 0.1794,
3677
+ "step": 5230
3678
+ },
3679
+ {
3680
+ "epoch": 0.97,
3681
+ "grad_norm": 0.7301254272460938,
3682
+ "learning_rate": 9.068308034061459e-05,
3683
+ "loss": 0.2051,
3684
+ "step": 5240
3685
+ },
3686
+ {
3687
+ "epoch": 0.97,
3688
+ "grad_norm": 1.1322051286697388,
3689
+ "learning_rate": 9.066456867826732e-05,
3690
+ "loss": 0.1811,
3691
+ "step": 5250
3692
+ },
3693
+ {
3694
+ "epoch": 0.97,
3695
+ "grad_norm": 0.5501244068145752,
3696
+ "learning_rate": 9.064605701592003e-05,
3697
+ "loss": 0.1868,
3698
+ "step": 5260
3699
+ },
3700
+ {
3701
+ "epoch": 0.97,
3702
+ "grad_norm": 0.9664620161056519,
3703
+ "learning_rate": 9.062754535357275e-05,
3704
+ "loss": 0.1907,
3705
+ "step": 5270
3706
+ },
3707
+ {
3708
+ "epoch": 0.97,
3709
+ "grad_norm": 0.3740132749080658,
3710
+ "learning_rate": 9.060903369122548e-05,
3711
+ "loss": 0.1796,
3712
+ "step": 5280
3713
+ },
3714
+ {
3715
+ "epoch": 0.98,
3716
+ "grad_norm": 0.9004091024398804,
3717
+ "learning_rate": 9.05905220288782e-05,
3718
+ "loss": 0.1814,
3719
+ "step": 5290
3720
+ },
3721
+ {
3722
+ "epoch": 0.98,
3723
+ "grad_norm": 0.5110592246055603,
3724
+ "learning_rate": 9.057201036653093e-05,
3725
+ "loss": 0.1897,
3726
+ "step": 5300
3727
+ },
3728
+ {
3729
+ "epoch": 0.98,
3730
+ "grad_norm": 0.6021327972412109,
3731
+ "learning_rate": 9.055349870418363e-05,
3732
+ "loss": 0.1848,
3733
+ "step": 5310
3734
+ },
3735
+ {
3736
+ "epoch": 0.98,
3737
+ "grad_norm": 1.157943844795227,
3738
+ "learning_rate": 9.053498704183636e-05,
3739
+ "loss": 0.175,
3740
+ "step": 5320
3741
+ },
3742
+ {
3743
+ "epoch": 0.98,
3744
+ "grad_norm": 0.6326828598976135,
3745
+ "learning_rate": 9.051647537948908e-05,
3746
+ "loss": 0.1884,
3747
+ "step": 5330
3748
+ },
3749
+ {
3750
+ "epoch": 0.98,
3751
+ "grad_norm": 0.37102219462394714,
3752
+ "learning_rate": 9.04979637171418e-05,
3753
+ "loss": 0.194,
3754
+ "step": 5340
3755
+ },
3756
+ {
3757
+ "epoch": 0.99,
3758
+ "grad_norm": 0.9475340247154236,
3759
+ "learning_rate": 9.047945205479452e-05,
3760
+ "loss": 0.1971,
3761
+ "step": 5350
3762
+ },
3763
+ {
3764
+ "epoch": 0.99,
3765
+ "grad_norm": 0.4836327135562897,
3766
+ "learning_rate": 9.046094039244725e-05,
3767
+ "loss": 0.1909,
3768
+ "step": 5360
3769
+ },
3770
+ {
3771
+ "epoch": 0.99,
3772
+ "grad_norm": 1.3864704370498657,
3773
+ "learning_rate": 9.044242873009996e-05,
3774
+ "loss": 0.1874,
3775
+ "step": 5370
3776
+ },
3777
+ {
3778
+ "epoch": 0.99,
3779
+ "grad_norm": 0.7823747992515564,
3780
+ "learning_rate": 9.042391706775268e-05,
3781
+ "loss": 0.1911,
3782
+ "step": 5380
3783
+ },
3784
+ {
3785
+ "epoch": 0.99,
3786
+ "grad_norm": 0.9502007365226746,
3787
+ "learning_rate": 9.040540540540541e-05,
3788
+ "loss": 0.1815,
3789
+ "step": 5390
3790
+ },
3791
+ {
3792
+ "epoch": 1.0,
3793
+ "grad_norm": 1.5369421243667603,
3794
+ "learning_rate": 9.038689374305813e-05,
3795
+ "loss": 0.1865,
3796
+ "step": 5400
3797
+ },
3798
+ {
3799
+ "epoch": 1.0,
3800
+ "grad_norm": 0.3976035416126251,
3801
+ "learning_rate": 9.036838208071086e-05,
3802
+ "loss": 0.1835,
3803
+ "step": 5410
3804
+ },
3805
+ {
3806
+ "epoch": 1.0,
3807
+ "grad_norm": 0.5696306824684143,
3808
+ "learning_rate": 9.034987041836356e-05,
3809
+ "loss": 0.1761,
3810
+ "step": 5420
3811
+ },
3812
+ {
3813
+ "epoch": 1.0,
3814
+ "eval_loss": 0.3201138377189636,
3815
+ "eval_runtime": 3918.2947,
3816
+ "eval_samples_per_second": 262.734,
3817
+ "eval_steps_per_second": 4.105,
3818
+ "step": 5422
3819
+ }
3820
+ ],
3821
+ "logging_steps": 10,
3822
+ "max_steps": 54220,
3823
+ "num_input_tokens_seen": 0,
3824
+ "num_train_epochs": 10,
3825
+ "save_steps": 500,
3826
+ "total_flos": 2.3125260009084027e+18,
3827
+ "train_batch_size": 6,
3828
+ "trial_name": null,
3829
+ "trial_params": null
3830
+ }