| 2021-07-15 00:52:29,330 INFO MainThread:624595 [internal.py:wandb_internal():88] W&B internal server running at pid: 624595, started at: 2021-07-15 00:52:29.329937 | |
| 2021-07-15 00:52:29,332 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-15 00:52:29,333 INFO WriterThread:624595 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/run-361cvuow.wandb | |
| 2021-07-15 00:52:29,333 DEBUG SenderThread:624595 [sender.py:send():179] send: header | |
| 2021-07-15 00:52:29,333 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-15 00:52:29,368 DEBUG SenderThread:624595 [sender.py:send():179] send: run | |
| 2021-07-15 00:52:29,550 INFO SenderThread:624595 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files | |
| 2021-07-15 00:52:29,550 INFO SenderThread:624595 [sender.py:_start_run_threads():716] run started: 361cvuow with start time 1626310348 | |
| 2021-07-15 00:52:29,550 DEBUG SenderThread:624595 [sender.py:send():179] send: summary | |
| 2021-07-15 00:52:29,551 INFO SenderThread:624595 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 00:52:29,551 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-15 00:52:30,553 INFO Thread-8 :624595 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-summary.json | |
| 2021-07-15 00:52:30,640 DEBUG HandlerThread:624595 [meta.py:__init__():39] meta init | |
| 2021-07-15 00:52:30,640 DEBUG HandlerThread:624595 [meta.py:__init__():53] meta init done | |
| 2021-07-15 00:52:30,640 DEBUG HandlerThread:624595 [meta.py:probe():210] probe | |
| 2021-07-15 00:52:30,641 DEBUG HandlerThread:624595 [meta.py:_setup_git():200] setup git | |
| 2021-07-15 00:52:30,670 DEBUG HandlerThread:624595 [meta.py:_setup_git():207] setup git done | |
| 2021-07-15 00:52:30,670 DEBUG HandlerThread:624595 [meta.py:_save_pip():57] save pip | |
| 2021-07-15 00:52:30,671 DEBUG HandlerThread:624595 [meta.py:_save_pip():71] save pip done | |
| 2021-07-15 00:52:30,671 DEBUG HandlerThread:624595 [meta.py:probe():252] probe done | |
| 2021-07-15 00:52:30,674 DEBUG SenderThread:624595 [sender.py:send():179] send: files | |
| 2021-07-15 00:52:30,674 INFO SenderThread:624595 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-15 00:52:30,680 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:52:30,680 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:52:30,812 DEBUG SenderThread:624595 [sender.py:send():179] send: config | |
| 2021-07-15 00:52:30,812 DEBUG SenderThread:624595 [sender.py:send():179] send: config | |
| 2021-07-15 00:52:30,813 DEBUG SenderThread:624595 [sender.py:send():179] send: config | |
| 2021-07-15 00:52:31,126 INFO Thread-11 :624595 [upload_job.py:push():137] Uploaded file /tmp/tmp7sk_pd_0wandb/2r26ralk-wandb-metadata.json | |
| 2021-07-15 00:52:31,551 INFO Thread-8 :624595 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-metadata.json | |
| 2021-07-15 00:52:31,552 INFO Thread-8 :624595 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/requirements.txt | |
| 2021-07-15 00:52:31,552 INFO Thread-8 :624595 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:52:45,557 INFO Thread-8 :624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:52:45,814 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:52:45,814 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:52:58,723 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:53:00,564 INFO Thread-8 :624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/config.yaml | |
| 2021-07-15 00:53:00,947 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:53:00,947 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:53:16,080 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:53:16,080 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:53:28,803 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:53:31,216 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:53:31,216 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:53:46,347 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:53:46,347 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:53:58,879 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:54:01,496 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:54:01,496 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:54:16,630 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:54:16,630 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:54:28,953 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:54:31,760 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:54:31,761 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:54:46,892 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:54:46,893 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:54:59,029 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:55:02,026 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:55:02,027 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:55:17,159 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:55:17,160 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:55:29,107 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:55:32,292 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:55:32,292 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:55:47,422 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:55:47,422 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:55:59,186 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:56:02,555 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:56:02,556 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:56:17,688 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:56:17,688 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:56:29,261 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:56:32,824 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:56:32,824 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:56:47,955 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:56:47,955 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:56:59,339 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:57:03,086 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:57:03,086 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:57:18,219 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:57:18,219 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:57:29,416 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:57:33,350 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:57:33,350 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:57:48,482 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:57:48,483 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:57:59,490 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:58:03,614 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:58:03,614 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:58:17,674 INFO Thread-8 :624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:58:18,763 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:58:18,763 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:58:29,567 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:58:33,909 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:58:33,910 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:58:49,058 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:58:49,059 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:58:59,648 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:59:04,188 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:59:04,188 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:59:19,321 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:59:19,322 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:59:29,729 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:59:34,455 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 00:59:34,455 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 00:59:41,706 INFO Thread-8 :624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:59:43,394 DEBUG SenderThread:624595 [sender.py:send():179] send: telemetry | |
| 2021-07-15 00:59:43,394 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:43,395 DEBUG SenderThread:624595 [sender.py:send():179] send: exit | |
| 2021-07-15 00:59:43,395 INFO SenderThread:624595 [sender.py:send_exit():287] handling exit code: 1 | |
| 2021-07-15 00:59:43,395 INFO SenderThread:624595 [sender.py:send_exit():295] send defer | |
| 2021-07-15 00:59:43,395 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:43,396 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,396 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 0 | |
| 2021-07-15 00:59:43,396 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,396 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 0 | |
| 2021-07-15 00:59:43,396 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 1 | |
| 2021-07-15 00:59:43,396 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,396 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 1 | |
| 2021-07-15 00:59:43,467 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,467 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 1 | |
| 2021-07-15 00:59:43,467 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 2 | |
| 2021-07-15 00:59:43,467 DEBUG SenderThread:624595 [sender.py:send():179] send: stats | |
| 2021-07-15 00:59:43,467 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,467 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 2 | |
| 2021-07-15 00:59:43,468 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,468 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 2 | |
| 2021-07-15 00:59:43,468 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 3 | |
| 2021-07-15 00:59:43,468 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,468 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 3 | |
| 2021-07-15 00:59:43,468 DEBUG SenderThread:624595 [sender.py:send():179] send: summary | |
| 2021-07-15 00:59:43,468 INFO SenderThread:624595 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 00:59:43,468 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,469 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 3 | |
| 2021-07-15 00:59:43,469 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 4 | |
| 2021-07-15 00:59:43,469 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,469 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 4 | |
| 2021-07-15 00:59:43,469 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,469 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 4 | |
| 2021-07-15 00:59:43,498 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:43,675 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 5 | |
| 2021-07-15 00:59:43,675 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:43,676 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,676 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 5 | |
| 2021-07-15 00:59:43,676 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,676 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 5 | |
| 2021-07-15 00:59:43,676 INFO SenderThread:624595 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-15 00:59:43,707 INFO SenderThread:624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/config.yaml | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-summary.json | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/requirements.txt requirements.txt | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log output.log | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/config.yaml config.yaml | |
| 2021-07-15 00:59:43,708 INFO SenderThread:624595 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-summary.json wandb-summary.json | |
| 2021-07-15 00:59:43,709 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 6 | |
| 2021-07-15 00:59:43,709 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:43,715 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 6 | |
| 2021-07-15 00:59:43,720 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:43,720 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 6 | |
| 2021-07-15 00:59:43,721 INFO SenderThread:624595 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:59:43,778 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:43,778 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:43,880 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:43,880 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:43,982 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:43,982 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,084 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,084 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,146 INFO Thread-13 :624595 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/output.log | |
| 2021-07-15 00:59:44,161 INFO Thread-15 :624595 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/wandb-summary.json | |
| 2021-07-15 00:59:44,186 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,187 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,194 INFO Thread-12 :624595 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/requirements.txt | |
| 2021-07-15 00:59:44,198 INFO Thread-14 :624595 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/files/config.yaml | |
| 2021-07-15 00:59:44,289 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,289 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,391 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,391 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,399 INFO Thread-7 :624595 [sender.py:transition_state():308] send defer: 7 | |
| 2021-07-15 00:59:44,399 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:44,399 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 7 | |
| 2021-07-15 00:59:44,399 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:44,400 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 7 | |
| 2021-07-15 00:59:44,493 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,686 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 8 | |
| 2021-07-15 00:59:44,687 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,687 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:44,687 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 8 | |
| 2021-07-15 00:59:44,687 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:44,688 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 8 | |
| 2021-07-15 00:59:44,688 INFO SenderThread:624595 [sender.py:transition_state():308] send defer: 9 | |
| 2021-07-15 00:59:44,688 DEBUG SenderThread:624595 [sender.py:send():179] send: final | |
| 2021-07-15 00:59:44,688 DEBUG SenderThread:624595 [sender.py:send():179] send: footer | |
| 2021-07-15 00:59:44,688 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-15 00:59:44,689 INFO HandlerThread:624595 [handler.py:handle_request_defer():141] handle defer: 9 | |
| 2021-07-15 00:59:44,689 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: defer | |
| 2021-07-15 00:59:44,689 INFO SenderThread:624595 [sender.py:send_request_defer():304] handle sender defer: 9 | |
| 2021-07-15 00:59:44,789 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-15 00:59:44,789 DEBUG SenderThread:624595 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-15 00:59:44,789 INFO SenderThread:624595 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:59:44,791 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: get_summary | |
| 2021-07-15 00:59:44,791 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: sampled_history | |
| 2021-07-15 00:59:44,792 DEBUG HandlerThread:624595 [handler.py:handle_request():124] handle_request: shutdown | |
| 2021-07-15 00:59:44,792 INFO HandlerThread:624595 [handler.py:finish():638] shutting down handler | |
| 2021-07-15 00:59:45,689 INFO WriterThread:624595 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210715_005228-361cvuow/run-361cvuow.wandb | |
| 2021-07-15 00:59:45,789 INFO SenderThread:624595 [sender.py:finish():945] shutting down sender | |
| 2021-07-15 00:59:45,790 INFO SenderThread:624595 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-15 00:59:45,790 INFO SenderThread:624595 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-15 00:59:45,792 INFO MainThread:624595 [internal.py:handle_exit():78] Internal process exited | |