File size: 23,442 Bytes
4014562
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
2022-10-30 08:41:03,941 INFO    MainThread:1823586 [internal.py:wandb_internal():90] W&B internal server running at pid: 1823586, started at: 2022-10-30 08:41:03.939683
2022-10-30 08:41:03,942 INFO    WriterThread:1823586 [datastore.py:open_for_write():75] open: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/run-1hva11jp.wandb
2022-10-30 08:41:03,945 DEBUG   SenderThread:1823586 [sender.py:send():235] send: header
2022-10-30 08:41:03,945 DEBUG   SenderThread:1823586 [sender.py:send():235] send: run
2022-10-30 08:41:04,060 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: check_version
2022-10-30 08:41:04,067 INFO    SenderThread:1823586 [dir_watcher.py:__init__():205] watching files in: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files
2022-10-30 08:41:04,068 INFO    SenderThread:1823586 [sender.py:_start_run_threads():812] run started: 1hva11jp with start time 1667133661
2022-10-30 08:41:04,068 DEBUG   SenderThread:1823586 [sender.py:send():235] send: summary
2022-10-30 08:41:04,070 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file wandb-summary.json with policy end
2022-10-30 08:41:04,071 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: check_version
2022-10-30 08:41:04,115 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: run_start
2022-10-30 08:41:04,122 DEBUG   HandlerThread:1823586 [meta.py:__init__():35] meta init
2022-10-30 08:41:04,122 DEBUG   HandlerThread:1823586 [meta.py:__init__():49] meta init done
2022-10-30 08:41:04,122 DEBUG   HandlerThread:1823586 [meta.py:probe():209] probe
2022-10-30 08:41:04,129 DEBUG   HandlerThread:1823586 [meta.py:_setup_git():199] setup git
2022-10-30 08:41:04,154 DEBUG   HandlerThread:1823586 [meta.py:_setup_git():206] setup git done
2022-10-30 08:41:04,154 DEBUG   HandlerThread:1823586 [meta.py:_save_code():87] save code
2022-10-30 08:41:04,168 DEBUG   HandlerThread:1823586 [meta.py:_save_code():108] save code done
2022-10-30 08:41:04,168 DEBUG   HandlerThread:1823586 [meta.py:_save_patches():125] save patches
2022-10-30 08:41:04,318 DEBUG   HandlerThread:1823586 [meta.py:_save_patches():167] save patches done
2022-10-30 08:41:04,318 DEBUG   HandlerThread:1823586 [meta.py:_save_pip():53] save pip
2022-10-30 08:41:04,321 DEBUG   HandlerThread:1823586 [meta.py:_save_pip():67] save pip done
2022-10-30 08:41:04,321 DEBUG   HandlerThread:1823586 [meta.py:_save_conda():74] save conda
2022-10-30 08:41:05,072 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/conda-environment.yaml
2022-10-30 08:41:05,073 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/code/cleaned_code/main.py
2022-10-30 08:41:05,073 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/requirements.txt
2022-10-30 08:41:05,074 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-summary.json
2022-10-30 08:41:05,074 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/diff.patch
2022-10-30 08:41:05,075 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/code/cleaned_code
2022-10-30 08:41:05,075 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/code
2022-10-30 08:41:09,131 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:12,107 DEBUG   HandlerThread:1823586 [meta.py:_save_conda():84] save conda done
2022-10-30 08:41:12,108 DEBUG   HandlerThread:1823586 [meta.py:probe():247] probe done
2022-10-30 08:41:12,135 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/conda-environment.yaml
2022-10-30 08:41:12,135 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:12,135 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_created():256] file/dir created: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-metadata.json
2022-10-30 08:41:12,241 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: stop_status
2022-10-30 08:41:12,242 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: stop_status
2022-10-30 08:41:12,320 DEBUG   SenderThread:1823586 [sender.py:send():235] send: telemetry
2022-10-30 08:41:12,320 DEBUG   SenderThread:1823586 [sender.py:send():235] send: config
2022-10-30 08:41:12,446 DEBUG   SenderThread:1823586 [sender.py:send():235] send: files
2022-10-30 08:41:12,447 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file wandb-metadata.json with policy now
2022-10-30 08:41:12,448 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file code/cleaned_code/main.py with policy now
2022-10-30 08:41:12,449 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file diff.patch with policy now
2022-10-30 08:41:12,449 DEBUG   SenderThread:1823586 [sender.py:send():235] send: metric
2022-10-30 08:41:12,449 DEBUG   SenderThread:1823586 [sender.py:send():235] send: telemetry
2022-10-30 08:41:12,449 DEBUG   SenderThread:1823586 [sender.py:send():235] send: metric
2022-10-30 08:41:12,453 WARNING SenderThread:1823586 [sender.py:send_metric():905] Seen metric with glob (shouldn't happen)
2022-10-30 08:41:12,453 DEBUG   SenderThread:1823586 [sender.py:send():235] send: telemetry
2022-10-30 08:41:12,793 INFO    Thread-11 :1823586 [upload_job.py:push():132] Uploaded file /tmp/tmpgha_04txwandb/2d5q8r5e-wandb-metadata.json
2022-10-30 08:41:12,793 INFO    Thread-12 :1823586 [upload_job.py:push():132] Uploaded file /tmp/tmpgha_04txwandb/uempk45h-code/cleaned_code/main.py
2022-10-30 08:41:12,844 INFO    Thread-13 :1823586 [upload_job.py:push():132] Uploaded file /tmp/tmpgha_04txwandb/3w3tb6lz-diff.patch
2022-10-30 08:41:14,141 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:17,145 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:19,160 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:21,251 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:25,339 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:27,391 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:27,449 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: stop_status
2022-10-30 08:41:27,453 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: stop_status
2022-10-30 08:41:38,014 DEBUG   SenderThread:1823586 [sender.py:send():235] send: stats
2022-10-30 08:41:38,688 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/config.yaml
2022-10-30 08:41:42,569 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: stop_status
2022-10-30 08:41:42,569 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: stop_status
2022-10-30 08:41:43,473 DEBUG   SenderThread:1823586 [sender.py:send():235] send: telemetry
2022-10-30 08:41:45,794 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:46,151 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:46,151 DEBUG   SenderThread:1823586 [sender.py:send():235] send: exit
2022-10-30 08:41:46,151 INFO    SenderThread:1823586 [sender.py:send_exit():371] handling exit code: 1
2022-10-30 08:41:46,151 INFO    SenderThread:1823586 [sender.py:send_exit():373] handling runtime: 42
2022-10-30 08:41:46,153 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file wandb-summary.json with policy end
2022-10-30 08:41:46,153 INFO    SenderThread:1823586 [sender.py:send_exit():379] send defer
2022-10-30 08:41:46,154 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:46,154 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,155 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 0
2022-10-30 08:41:46,155 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,155 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 0
2022-10-30 08:41:46,155 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 1
2022-10-30 08:41:46,155 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,155 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 1
2022-10-30 08:41:46,352 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:46,352 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,352 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 1
2022-10-30 08:41:46,353 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 2
2022-10-30 08:41:46,353 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:46,353 DEBUG   SenderThread:1823586 [sender.py:send():235] send: stats
2022-10-30 08:41:46,354 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,354 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 2
2022-10-30 08:41:46,354 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,354 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 2
2022-10-30 08:41:46,355 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 3
2022-10-30 08:41:46,355 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,355 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 3
2022-10-30 08:41:46,355 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,356 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 3
2022-10-30 08:41:46,356 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 4
2022-10-30 08:41:46,356 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,357 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 4
2022-10-30 08:41:46,357 DEBUG   SenderThread:1823586 [sender.py:send():235] send: summary
2022-10-30 08:41:46,385 INFO    SenderThread:1823586 [sender.py:_save_file():949] saving file wandb-summary.json with policy end
2022-10-30 08:41:46,385 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,385 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 4
2022-10-30 08:41:46,385 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 5
2022-10-30 08:41:46,386 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:46,386 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 5
2022-10-30 08:41:46,386 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:46,387 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 5
2022-10-30 08:41:46,511 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:46,795 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:46,795 INFO    Thread-7  :1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-summary.json
2022-10-30 08:41:50,346 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 6
2022-10-30 08:41:50,346 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:50,347 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:50,347 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 6
2022-10-30 08:41:50,348 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:50,348 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 6
2022-10-30 08:41:50,348 INFO    SenderThread:1823586 [dir_watcher.py:finish():336] shutting down directory watcher
2022-10-30 08:41:50,448 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:50,807 INFO    SenderThread:1823586 [dir_watcher.py:_on_file_modified():269] file/dir modified: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/config.yaml
2022-10-30 08:41:50,807 INFO    SenderThread:1823586 [dir_watcher.py:finish():366] scan: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files
2022-10-30 08:41:50,808 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/conda-environment.yaml conda-environment.yaml
2022-10-30 08:41:50,808 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/config.yaml config.yaml
2022-10-30 08:41:50,812 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/diff.patch diff.patch
2022-10-30 08:41:50,812 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-metadata.json wandb-metadata.json
2022-10-30 08:41:50,816 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/requirements.txt requirements.txt
2022-10-30 08:41:50,816 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log output.log
2022-10-30 08:41:50,820 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-summary.json wandb-summary.json
2022-10-30 08:41:50,832 INFO    SenderThread:1823586 [dir_watcher.py:finish():380] scan save: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/code/cleaned_code/main.py code/cleaned_code/main.py
2022-10-30 08:41:50,832 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 7
2022-10-30 08:41:50,833 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:50,833 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:50,833 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 7
2022-10-30 08:41:50,834 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:50,834 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 7
2022-10-30 08:41:50,834 INFO    SenderThread:1823586 [file_pusher.py:finish():145] shutting down file pusher
2022-10-30 08:41:50,934 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:50,935 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,037 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:51,037 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,139 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:51,139 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,142 INFO    Thread-14 :1823586 [upload_job.py:push():132] Uploaded file /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/conda-environment.yaml
2022-10-30 08:41:51,149 INFO    Thread-16 :1823586 [upload_job.py:push():132] Uploaded file /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/requirements.txt
2022-10-30 08:41:51,160 INFO    Thread-18 :1823586 [upload_job.py:push():132] Uploaded file /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/wandb-summary.json
2022-10-30 08:41:51,167 INFO    Thread-17 :1823586 [upload_job.py:push():132] Uploaded file /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/output.log
2022-10-30 08:41:51,241 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:51,241 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,281 INFO    Thread-15 :1823586 [upload_job.py:push():132] Uploaded file /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/files/config.yaml
2022-10-30 08:41:51,343 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:51,343 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,445 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:51,445 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:51,482 INFO    Thread-6  :1823586 [sender.py:transition_state():392] send defer: 8
2022-10-30 08:41:51,482 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:51,482 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 8
2022-10-30 08:41:51,483 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:51,483 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 8
2022-10-30 08:41:51,547 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:52,896 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 9
2022-10-30 08:41:52,897 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:52,897 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:52,898 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 9
2022-10-30 08:41:52,898 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:52,898 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 9
2022-10-30 08:41:52,899 INFO    SenderThread:1823586 [sender.py:transition_state():392] send defer: 10
2022-10-30 08:41:52,899 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: defer
2022-10-30 08:41:52,899 DEBUG   SenderThread:1823586 [sender.py:send():235] send: final
2022-10-30 08:41:52,900 INFO    HandlerThread:1823586 [handler.py:handle_request_defer():164] handle defer: 10
2022-10-30 08:41:52,900 DEBUG   SenderThread:1823586 [sender.py:send():235] send: footer
2022-10-30 08:41:52,901 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: defer
2022-10-30 08:41:52,901 INFO    SenderThread:1823586 [sender.py:send_request_defer():388] handle sender defer: 10
2022-10-30 08:41:52,999 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: poll_exit
2022-10-30 08:41:52,999 DEBUG   SenderThread:1823586 [sender.py:send_request():249] send_request: poll_exit
2022-10-30 08:41:53,000 INFO    SenderThread:1823586 [file_pusher.py:join():150] waiting for file pusher
2022-10-30 08:41:53,244 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: sampled_history
2022-10-30 08:41:53,245 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: get_summary
2022-10-30 08:41:53,246 DEBUG   HandlerThread:1823586 [handler.py:handle_request():141] handle_request: shutdown
2022-10-30 08:41:53,246 INFO    HandlerThread:1823586 [handler.py:finish():806] shutting down handler
2022-10-30 08:41:53,900 INFO    WriterThread:1823586 [datastore.py:close():279] close: /n/fs/nlp-pranjal/SemSup-LMLC/cleaned_code/wandb/run-20221030_084100-1hva11jp/run-1hva11jp.wandb
2022-10-30 08:41:54,143 INFO    SenderThread:1823586 [sender.py:finish():1111] shutting down sender
2022-10-30 08:41:54,143 INFO    SenderThread:1823586 [file_pusher.py:finish():145] shutting down file pusher
2022-10-30 08:41:54,143 INFO    SenderThread:1823586 [file_pusher.py:join():150] waiting for file pusher
2022-10-30 08:41:54,146 INFO    MainThread:1823586 [internal.py:handle_exit():80] Internal process exited