| 2021-07-15 00:28:10,840 INFO MainThread:620043 [internal.py:wandb_internal():88] W&B internal server running at pid: 620043, started at: 2021-07-15 00:28:10.839711 | |
| 2021-07-15 00:28:10,842 INFO WriterThread:620043 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/run-22keyfxg.wandb | |
| 2021-07-15 00:28:10,842 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-15 00:28:10,843 DEBUG SenderThread:620043 [sender.py:send():179] send: header | |
| 2021-07-15 00:28:10,843 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-15 00:28:10,882 DEBUG SenderThread:620043 [sender.py:send():179] send: run | |
| 2021-07-15 00:28:11,065 INFO SenderThread:620043 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files | |
| 2021-07-15 00:28:11,065 INFO SenderThread:620043 [sender.py:_start_run_threads():716] run started: 22keyfxg with start time 1626308890 | |
| 2021-07-15 00:28:11,065 DEBUG SenderThread:620043 [sender.py:send():179] send: summary | |
| 2021-07-15 00:28:11,065 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-15 00:28:11,066 INFO SenderThread:620043 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 00:28:12,069 INFO Thread-8 :620043 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-summary.json | |
| 2021-07-15 00:28:12,177 DEBUG HandlerThread:620043 [meta.py:__init__():39] meta init | |
| 2021-07-15 00:28:12,177 DEBUG HandlerThread:620043 [meta.py:__init__():53] meta init done | |
| 2021-07-15 00:28:12,177 DEBUG HandlerThread:620043 [meta.py:probe():210] probe | |
| 2021-07-15 00:28:12,178 DEBUG HandlerThread:620043 [meta.py:_setup_git():200] setup git | |
| 2021-07-15 00:28:12,205 DEBUG HandlerThread:620043 [meta.py:_setup_git():207] setup git done | |
| 2021-07-15 00:28:12,205 DEBUG HandlerThread:620043 [meta.py:_save_pip():57] save pip | |
| 2021-07-15 00:28:12,206 DEBUG HandlerThread:620043 [meta.py:_save_pip():71] save pip done | |
| 2021-07-15 00:28:12,206 DEBUG HandlerThread:620043 [meta.py:probe():252] probe done | |
| 2021-07-15 00:28:12,209 DEBUG SenderThread:620043 [sender.py:send():179] send: files | |
| 2021-07-15 00:28:12,209 INFO SenderThread:620043 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-15 00:28:12,217 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:28:12,217 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:28:12,363 DEBUG SenderThread:620043 [sender.py:send():179] send: config | |
| 2021-07-15 00:28:12,363 DEBUG SenderThread:620043 [sender.py:send():179] send: config | |
| 2021-07-15 00:28:12,364 DEBUG SenderThread:620043 [sender.py:send():179] send: config | |
| 2021-07-15 00:28:12,702 INFO Thread-11 :620043 [upload_job.py:push():137] Uploaded file /tmp/tmpkt2gzisdwandb/opsxvzfj-wandb-metadata.json | |
| 2021-07-15 00:28:13,068 INFO Thread-8 :620043 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/requirements.txt | |
| 2021-07-15 00:28:13,068 INFO Thread-8 :620043 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:28:13,069 INFO Thread-8 :620043 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-metadata.json | |
| 2021-07-15 00:28:27,074 INFO Thread-8 :620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:28:27,365 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:28:27,366 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:28:40,252 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:28:42,080 INFO Thread-8 :620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/config.yaml | |
| 2021-07-15 00:28:42,496 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:28:42,497 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:28:57,628 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:28:57,628 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:29:10,324 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:29:12,761 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:29:12,762 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:29:27,895 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:29:27,896 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:29:40,392 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:29:43,029 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:29:43,029 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:29:58,161 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:29:58,161 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:30:10,458 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:30:13,295 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:30:13,296 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:30:28,431 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:30:28,431 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:30:40,524 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:30:43,564 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:30:43,564 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:30:58,695 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:30:58,695 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:31:10,591 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:31:13,829 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:31:13,829 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:31:28,959 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:31:28,960 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:31:40,658 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:31:44,094 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:31:44,095 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:31:59,226 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:31:59,227 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:32:10,721 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:32:14,364 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:32:14,364 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:32:29,496 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:32:29,497 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:32:40,792 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:32:44,631 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:32:44,632 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:32:59,762 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:32:59,762 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:33:10,860 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:33:14,895 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:33:14,895 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:33:30,027 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:33:30,027 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:33:40,933 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:33:45,162 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:33:45,162 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:34:00,292 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:34:00,293 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:34:11,005 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:34:15,424 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:34:15,425 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:34:30,556 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:34:30,556 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:34:41,080 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:34:45,687 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:34:45,688 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:34:57,238 INFO Thread-8 :620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:35:00,957 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:35:00,957 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:35:11,161 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:35:16,118 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:35:16,119 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:35:31,255 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:35:31,255 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:35:41,236 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:35:46,389 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:35:46,390 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:36:01,522 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:36:01,522 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:36:11,313 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:36:16,658 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:36:16,658 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:36:19,273 INFO Thread-8 :620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:36:20,774 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:20,775 DEBUG SenderThread:620043 [sender.py:send():179] send: telemetry | |
| 2021-07-15 00:36:20,775 DEBUG SenderThread:620043 [sender.py:send():179] send: exit | |
| 2021-07-15 00:36:20,775 INFO SenderThread:620043 [sender.py:send_exit():287] handling exit code: 1 | |
| 2021-07-15 00:36:20,775 INFO SenderThread:620043 [sender.py:send_exit():295] send defer | |
| 2021-07-15 00:36:20,776 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:20,776 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:20,776 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 0 | |
| 2021-07-15 00:36:20,777 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:20,777 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 0 | |
| 2021-07-15 00:36:20,777 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 1 | |
| 2021-07-15 00:36:20,777 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:20,777 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 1 | |
| 2021-07-15 00:36:20,837 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:20,837 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 1 | |
| 2021-07-15 00:36:20,837 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 2 | |
| 2021-07-15 00:36:20,837 DEBUG SenderThread:620043 [sender.py:send():179] send: stats | |
| 2021-07-15 00:36:20,843 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:20,843 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 2 | |
| 2021-07-15 00:36:20,843 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:20,844 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 2 | |
| 2021-07-15 00:36:20,844 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 3 | |
| 2021-07-15 00:36:20,844 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:20,844 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 3 | |
| 2021-07-15 00:36:20,844 DEBUG SenderThread:620043 [sender.py:send():179] send: summary | |
| 2021-07-15 00:36:20,845 INFO SenderThread:620043 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 00:36:20,846 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:20,846 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 3 | |
| 2021-07-15 00:36:20,846 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 4 | |
| 2021-07-15 00:36:20,847 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:20,847 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 4 | |
| 2021-07-15 00:36:20,847 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:20,847 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 4 | |
| 2021-07-15 00:36:20,878 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,021 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 5 | |
| 2021-07-15 00:36:21,021 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,022 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:21,022 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 5 | |
| 2021-07-15 00:36:21,022 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:21,022 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 5 | |
| 2021-07-15 00:36:21,023 INFO SenderThread:620043 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-15 00:36:21,124 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,275 INFO SenderThread:620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:36:21,275 INFO SenderThread:620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/config.yaml | |
| 2021-07-15 00:36:21,275 INFO SenderThread:620043 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-summary.json | |
| 2021-07-15 00:36:21,275 INFO SenderThread:620043 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files | |
| 2021-07-15 00:36:21,276 INFO SenderThread:620043 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/requirements.txt requirements.txt | |
| 2021-07-15 00:36:21,276 INFO SenderThread:620043 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log output.log | |
| 2021-07-15 00:36:21,276 INFO SenderThread:620043 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-15 00:36:21,276 INFO SenderThread:620043 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/config.yaml config.yaml | |
| 2021-07-15 00:36:21,280 INFO SenderThread:620043 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-summary.json wandb-summary.json | |
| 2021-07-15 00:36:21,281 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 6 | |
| 2021-07-15 00:36:21,281 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,284 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:21,284 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 6 | |
| 2021-07-15 00:36:21,288 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:21,288 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 6 | |
| 2021-07-15 00:36:21,288 INFO SenderThread:620043 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:36:21,386 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,386 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,488 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,488 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,590 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,590 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,691 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,692 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,711 INFO Thread-12 :620043 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/requirements.txt | |
| 2021-07-15 00:36:21,719 INFO Thread-13 :620043 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/output.log | |
| 2021-07-15 00:36:21,731 INFO Thread-14 :620043 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/config.yaml | |
| 2021-07-15 00:36:21,759 INFO Thread-15 :620043 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/files/wandb-summary.json | |
| 2021-07-15 00:36:21,793 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,793 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,895 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:21,895 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:21,959 INFO Thread-7 :620043 [sender.py:transition_state():308] send defer: 7 | |
| 2021-07-15 00:36:21,960 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:21,960 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 7 | |
| 2021-07-15 00:36:21,960 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:21,960 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 7 | |
| 2021-07-15 00:36:21,997 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:22,240 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 8 | |
| 2021-07-15 00:36:22,241 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:22,241 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:22,241 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 8 | |
| 2021-07-15 00:36:22,242 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:22,242 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 8 | |
| 2021-07-15 00:36:22,242 INFO SenderThread:620043 [sender.py:transition_state():308] send defer: 9 | |
| 2021-07-15 00:36:22,242 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:36:22,242 INFO HandlerThread:620043 [handler.py:handle_request_defer():141] handle defer: 9 | |
| 2021-07-15 00:36:22,243 DEBUG SenderThread:620043 [sender.py:send():179] send: final | |
| 2021-07-15 00:36:22,243 DEBUG SenderThread:620043 [sender.py:send():179] send: footer | |
| 2021-07-15 00:36:22,243 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:36:22,243 INFO SenderThread:620043 [sender.py:send_request_defer():304] handle sender defer: 9 | |
| 2021-07-15 00:36:22,342 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:36:22,343 DEBUG SenderThread:620043 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:36:22,343 INFO SenderThread:620043 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:36:22,344 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: get_summary | |
| 2021-07-15 00:36:22,345 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: sampled_history | |
| 2021-07-15 00:36:22,345 DEBUG HandlerThread:620043 [handler.py:handle_request():124] handle_request: shutdown | |
| 2021-07-15 00:36:22,345 INFO HandlerThread:620043 [handler.py:finish():638] shutting down handler | |
| 2021-07-15 00:36:23,243 INFO WriterThread:620043 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210715_002810-22keyfxg/run-22keyfxg.wandb | |
| 2021-07-15 00:36:23,343 INFO SenderThread:620043 [sender.py:finish():945] shutting down sender | |
| 2021-07-15 00:36:23,344 INFO SenderThread:620043 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:36:23,344 INFO SenderThread:620043 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:36:23,346 INFO MainThread:620043 [internal.py:handle_exit():78] Internal process exited | |