| 2021-07-14 23:55:55,747 INFO MainThread:613304 [internal.py:wandb_internal():88] W&B internal server running at pid: 613304, started at: 2021-07-14 23:55:55.746857 | |
| 2021-07-14 23:55:55,749 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-14 23:55:55,749 INFO WriterThread:613304 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/run-y01xq728.wandb | |
| 2021-07-14 23:55:55,750 DEBUG SenderThread:613304 [sender.py:send():179] send: header | |
| 2021-07-14 23:55:55,750 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-14 23:55:55,788 DEBUG SenderThread:613304 [sender.py:send():179] send: run | |
| 2021-07-14 23:55:55,953 INFO SenderThread:613304 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files | |
| 2021-07-14 23:55:55,954 INFO SenderThread:613304 [sender.py:_start_run_threads():716] run started: y01xq728 with start time 1626306955 | |
| 2021-07-14 23:55:55,954 DEBUG SenderThread:613304 [sender.py:send():179] send: summary | |
| 2021-07-14 23:55:55,954 INFO SenderThread:613304 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 23:55:55,954 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-14 23:55:56,957 INFO Thread-8 :613304 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-summary.json | |
| 2021-07-14 23:55:57,055 DEBUG HandlerThread:613304 [meta.py:__init__():39] meta init | |
| 2021-07-14 23:55:57,056 DEBUG HandlerThread:613304 [meta.py:__init__():53] meta init done | |
| 2021-07-14 23:55:57,056 DEBUG HandlerThread:613304 [meta.py:probe():210] probe | |
| 2021-07-14 23:55:57,057 DEBUG HandlerThread:613304 [meta.py:_setup_git():200] setup git | |
| 2021-07-14 23:55:57,086 DEBUG HandlerThread:613304 [meta.py:_setup_git():207] setup git done | |
| 2021-07-14 23:55:57,087 DEBUG HandlerThread:613304 [meta.py:_save_pip():57] save pip | |
| 2021-07-14 23:55:57,087 DEBUG HandlerThread:613304 [meta.py:_save_pip():71] save pip done | |
| 2021-07-14 23:55:57,087 DEBUG HandlerThread:613304 [meta.py:probe():252] probe done | |
| 2021-07-14 23:55:57,090 DEBUG SenderThread:613304 [sender.py:send():179] send: files | |
| 2021-07-14 23:55:57,091 INFO SenderThread:613304 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-14 23:55:57,098 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:55:57,098 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:55:57,230 DEBUG SenderThread:613304 [sender.py:send():179] send: config | |
| 2021-07-14 23:55:57,231 DEBUG SenderThread:613304 [sender.py:send():179] send: config | |
| 2021-07-14 23:55:57,231 DEBUG SenderThread:613304 [sender.py:send():179] send: config | |
| 2021-07-14 23:55:57,551 INFO Thread-11 :613304 [upload_job.py:push():137] Uploaded file /tmp/tmp_7vrmrnzwandb/30914hmv-wandb-metadata.json | |
| 2021-07-14 23:55:57,956 INFO Thread-8 :613304 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/requirements.txt | |
| 2021-07-14 23:55:57,956 INFO Thread-8 :613304 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-metadata.json | |
| 2021-07-14 23:55:57,956 INFO Thread-8 :613304 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-14 23:56:11,962 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-14 23:56:12,233 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:56:12,233 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:56:13,962 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-14 23:56:25,133 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:56:26,968 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/config.yaml | |
| 2021-07-14 23:56:27,371 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:56:27,372 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:56:42,505 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:56:42,505 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:56:55,203 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:56:57,640 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:56:57,641 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:57:12,787 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:57:12,788 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:57:25,275 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:57:27,922 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:57:27,922 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:57:43,054 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:57:43,055 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:57:55,351 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:57:58,186 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:57:58,187 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:58:13,317 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:58:13,318 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:58:25,419 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:58:28,450 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:58:28,450 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:58:43,580 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:58:43,581 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:58:55,493 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:58:58,713 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:58:58,713 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:59:13,843 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:59:13,844 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:59:25,562 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:59:28,973 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:59:28,974 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:59:44,104 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:59:44,104 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:59:55,632 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-14 23:59:59,242 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:59:59,243 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:00:14,374 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:00:14,375 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:00:25,703 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:00:29,505 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:00:29,505 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:00:44,637 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:00:44,637 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:00:55,773 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:00:59,774 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:00:59,774 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:01:14,907 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:01:14,907 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:01:25,847 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:01:30,041 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:01:30,042 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:01:45,174 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:01:45,174 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:01:55,920 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:02:00,307 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:02:00,308 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:02:15,449 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:02:15,449 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:02:25,995 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:02:30,588 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:02:30,588 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:02:45,719 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:02:45,720 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:02:56,066 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:03:00,850 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:03:00,850 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:03:15,978 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:03:15,978 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:03:26,139 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:03:31,111 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:03:31,112 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:03:46,244 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:03:46,244 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:03:56,128 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-15 00:03:56,212 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:04:01,389 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:04:01,390 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:04:16,550 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:04:16,551 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:04:26,288 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:04:31,686 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:04:31,686 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:04:46,826 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:04:46,826 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:04:56,371 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:05:01,959 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:05:01,959 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:05:17,092 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:05:17,092 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:05:22,163 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-15 00:05:23,289 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:23,289 DEBUG SenderThread:613304 [sender.py:send():179] send: telemetry | |
| 2021-07-15 00:05:23,289 DEBUG SenderThread:613304 [sender.py:send():179] send: exit | |
| 2021-07-15 00:05:23,289 INFO SenderThread:613304 [sender.py:send_exit():287] handling exit code: 1 | |
| 2021-07-15 00:05:23,291 INFO SenderThread:613304 [sender.py:send_exit():295] send defer | |
| 2021-07-15 00:05:23,291 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:23,292 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,292 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 0 | |
| 2021-07-15 00:05:23,292 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,292 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 0 | |
| 2021-07-15 00:05:23,292 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 1 | |
| 2021-07-15 00:05:23,293 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,293 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 1 | |
| 2021-07-15 00:05:23,339 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,339 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 1 | |
| 2021-07-15 00:05:23,339 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 2 | |
| 2021-07-15 00:05:23,339 DEBUG SenderThread:613304 [sender.py:send():179] send: stats | |
| 2021-07-15 00:05:23,340 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,340 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 2 | |
| 2021-07-15 00:05:23,340 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,340 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 2 | |
| 2021-07-15 00:05:23,340 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 3 | |
| 2021-07-15 00:05:23,341 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,341 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 3 | |
| 2021-07-15 00:05:23,341 DEBUG SenderThread:613304 [sender.py:send():179] send: summary | |
| 2021-07-15 00:05:23,342 INFO SenderThread:613304 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 00:05:23,342 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,342 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 3 | |
| 2021-07-15 00:05:23,342 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 4 | |
| 2021-07-15 00:05:23,342 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,342 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 4 | |
| 2021-07-15 00:05:23,343 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,343 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 4 | |
| 2021-07-15 00:05:23,393 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:23,512 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 5 | |
| 2021-07-15 00:05:23,512 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:23,513 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:23,513 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 5 | |
| 2021-07-15 00:05:23,513 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:23,514 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 5 | |
| 2021-07-15 00:05:23,514 INFO SenderThread:613304 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-15 00:05:23,615 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,163 INFO Thread-8 :613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/config.yaml | |
| 2021-07-15 00:05:24,164 INFO SenderThread:613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-15 00:05:24,164 INFO SenderThread:613304 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-summary.json | |
| 2021-07-15 00:05:24,164 INFO SenderThread:613304 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files | |
| 2021-07-15 00:05:24,165 INFO SenderThread:613304 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/requirements.txt requirements.txt | |
| 2021-07-15 00:05:24,165 INFO SenderThread:613304 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log output.log | |
| 2021-07-15 00:05:24,165 INFO SenderThread:613304 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-15 00:05:24,165 INFO SenderThread:613304 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/config.yaml config.yaml | |
| 2021-07-15 00:05:24,165 INFO SenderThread:613304 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-summary.json wandb-summary.json | |
| 2021-07-15 00:05:24,166 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 6 | |
| 2021-07-15 00:05:24,166 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,167 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:24,167 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 6 | |
| 2021-07-15 00:05:24,170 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:24,170 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 6 | |
| 2021-07-15 00:05:24,170 INFO SenderThread:613304 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:05:24,275 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,275 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,377 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,378 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,479 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,480 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,582 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,582 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,625 INFO Thread-15 :613304 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/wandb-summary.json | |
| 2021-07-15 00:05:24,632 INFO Thread-12 :613304 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/requirements.txt | |
| 2021-07-15 00:05:24,643 INFO Thread-13 :613304 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/output.log | |
| 2021-07-15 00:05:24,651 INFO Thread-14 :613304 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/files/config.yaml | |
| 2021-07-15 00:05:24,684 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,684 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,786 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:24,786 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:24,851 INFO Thread-7 :613304 [sender.py:transition_state():308] send defer: 7 | |
| 2021-07-15 00:05:24,852 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:24,852 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 7 | |
| 2021-07-15 00:05:24,852 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:24,852 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 7 | |
| 2021-07-15 00:05:24,888 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:25,136 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 8 | |
| 2021-07-15 00:05:25,136 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:25,137 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:25,137 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 8 | |
| 2021-07-15 00:05:25,137 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:25,137 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 8 | |
| 2021-07-15 00:05:25,138 INFO SenderThread:613304 [sender.py:transition_state():308] send defer: 9 | |
| 2021-07-15 00:05:25,138 DEBUG SenderThread:613304 [sender.py:send():179] send: final | |
| 2021-07-15 00:05:25,138 DEBUG SenderThread:613304 [sender.py:send():179] send: footer | |
| 2021-07-15 00:05:25,138 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:05:25,139 INFO HandlerThread:613304 [handler.py:handle_request_defer():141] handle defer: 9 | |
| 2021-07-15 00:05:25,139 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:05:25,139 INFO SenderThread:613304 [sender.py:send_request_defer():304] handle sender defer: 9 | |
| 2021-07-15 00:05:25,238 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:05:25,239 DEBUG SenderThread:613304 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:05:25,239 INFO SenderThread:613304 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:05:25,240 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: get_summary | |
| 2021-07-15 00:05:25,241 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: sampled_history | |
| 2021-07-15 00:05:25,242 DEBUG HandlerThread:613304 [handler.py:handle_request():124] handle_request: shutdown | |
| 2021-07-15 00:05:25,242 INFO HandlerThread:613304 [handler.py:finish():638] shutting down handler | |
| 2021-07-15 00:05:26,139 INFO WriterThread:613304 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_235555-y01xq728/run-y01xq728.wandb | |
| 2021-07-15 00:05:26,239 INFO SenderThread:613304 [sender.py:finish():945] shutting down sender | |
| 2021-07-15 00:05:26,239 INFO SenderThread:613304 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:05:26,240 INFO SenderThread:613304 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:05:26,242 INFO MainThread:613304 [internal.py:handle_exit():78] Internal process exited | |