| 2021-07-15 13:29:56,917 INFO MainThread:690242 [internal.py:wandb_internal():88] W&B internal server running at pid: 690242, started at: 2021-07-15 13:29:56.916849 | |
| 2021-07-15 13:29:56,919 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-15 13:29:56,919 INFO WriterThread:690242 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/run-37l0zsn1.wandb | |
| 2021-07-15 13:29:56,920 DEBUG SenderThread:690242 [sender.py:send():179] send: header | |
| 2021-07-15 13:29:56,920 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-15 13:29:56,960 DEBUG SenderThread:690242 [sender.py:send():179] send: run | |
| 2021-07-15 13:29:57,133 INFO SenderThread:690242 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files | |
| 2021-07-15 13:29:57,133 INFO SenderThread:690242 [sender.py:_start_run_threads():716] run started: 37l0zsn1 with start time 1626355796 | |
| 2021-07-15 13:29:57,133 DEBUG SenderThread:690242 [sender.py:send():179] send: summary | |
| 2021-07-15 13:29:57,133 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-15 13:29:57,134 INFO SenderThread:690242 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-15 13:29:58,138 INFO Thread-8 :690242 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/wandb-summary.json | |
| 2021-07-15 13:29:58,227 DEBUG HandlerThread:690242 [meta.py:__init__():39] meta init | |
| 2021-07-15 13:29:58,227 DEBUG HandlerThread:690242 [meta.py:__init__():53] meta init done | |
| 2021-07-15 13:29:58,227 DEBUG HandlerThread:690242 [meta.py:probe():210] probe | |
| 2021-07-15 13:29:58,228 DEBUG HandlerThread:690242 [meta.py:_setup_git():200] setup git | |
| 2021-07-15 13:29:58,258 DEBUG HandlerThread:690242 [meta.py:_setup_git():207] setup git done | |
| 2021-07-15 13:29:58,258 DEBUG HandlerThread:690242 [meta.py:_save_pip():57] save pip | |
| 2021-07-15 13:29:58,259 DEBUG HandlerThread:690242 [meta.py:_save_pip():71] save pip done | |
| 2021-07-15 13:29:58,259 DEBUG HandlerThread:690242 [meta.py:probe():252] probe done | |
| 2021-07-15 13:29:58,262 DEBUG SenderThread:690242 [sender.py:send():179] send: files | |
| 2021-07-15 13:29:58,262 INFO SenderThread:690242 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-15 13:29:58,270 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:29:58,270 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:29:58,400 DEBUG SenderThread:690242 [sender.py:send():179] send: config | |
| 2021-07-15 13:29:58,401 DEBUG SenderThread:690242 [sender.py:send():179] send: config | |
| 2021-07-15 13:29:58,401 DEBUG SenderThread:690242 [sender.py:send():179] send: config | |
| 2021-07-15 13:29:58,737 INFO Thread-11 :690242 [upload_job.py:push():137] Uploaded file /tmp/tmps2g86y4ywandb/37km806o-wandb-metadata.json | |
| 2021-07-15 13:29:59,137 INFO Thread-8 :690242 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:29:59,138 INFO Thread-8 :690242 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/wandb-metadata.json | |
| 2021-07-15 13:29:59,138 INFO Thread-8 :690242 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/requirements.txt | |
| 2021-07-15 13:30:13,740 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:30:13,740 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:30:15,144 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:30:17,144 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:30:26,310 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:30:28,149 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/config.yaml | |
| 2021-07-15 13:30:28,945 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:30:28,945 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:30:44,077 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:30:44,077 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:30:56,384 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:30:59,224 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:30:59,225 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:31:14,355 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:31:14,355 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:31:26,457 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:31:29,490 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:31:29,490 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:31:44,622 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:31:44,623 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:31:56,528 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:31:59,753 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:31:59,754 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:32:14,887 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:32:14,887 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:32:26,603 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:32:30,023 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:32:30,024 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:32:45,158 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:32:45,158 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:32:56,677 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:33:00,291 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:33:00,291 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:33:15,424 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:33:15,425 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:33:26,741 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:33:30,559 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:33:30,559 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:33:45,691 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:33:45,692 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:33:56,798 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:34:00,825 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:34:00,825 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:34:15,959 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:34:15,959 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:34:26,856 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:34:31,092 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:34:31,093 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:34:46,236 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:34:46,236 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:34:56,922 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:35:01,377 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:35:01,377 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:35:16,510 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:35:16,511 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:35:26,998 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:35:31,642 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:35:31,642 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:35:45,257 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:35:46,786 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:35:46,787 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:35:57,072 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:36:01,936 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:36:01,936 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:36:17,084 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:36:17,084 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:36:27,147 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:36:32,219 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:36:32,219 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:36:47,351 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:36:47,351 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:36:57,222 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:37:02,481 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:37:02,482 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:37:17,616 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:37:17,617 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:37:27,306 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:37:32,749 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:37:32,749 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:37:45,303 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:37:47,971 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:37:47,971 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:37:57,376 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:38:03,134 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:38:03,135 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:38:18,276 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:38:18,277 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:38:27,456 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:38:33,418 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:38:33,419 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:38:48,552 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:38:48,553 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:38:57,533 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:39:03,683 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:39:03,683 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:39:18,813 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:39:18,814 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:39:27,608 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:39:34,015 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:39:34,015 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:39:49,151 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:39:49,152 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:39:57,698 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:40:01,356 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:03,357 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:04,295 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:40:04,296 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:40:05,357 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:07,358 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:09,359 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:12,361 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:14,362 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:16,362 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:18,363 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:19,564 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:40:19,564 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:40:20,364 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:22,365 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:24,366 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:26,367 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:27,772 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:40:28,367 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:30,368 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:32,369 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:34,371 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:34,696 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:40:34,697 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:40:36,372 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:38,373 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:40,374 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:42,374 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:44,375 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:46,376 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:48,377 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:49,831 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:40:49,831 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:40:50,378 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:52,379 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:54,380 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:56,381 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:40:57,856 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:40:58,382 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:00,383 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:02,384 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:04,385 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:04,961 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:41:04,961 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:41:06,386 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:08,387 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:10,388 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:12,389 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:14,390 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:16,391 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:18,392 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:20,092 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:41:20,092 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:41:20,393 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:22,394 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:24,395 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:26,396 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:27,927 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:41:28,396 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:30,397 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:32,398 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:34,399 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:35,222 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:41:35,222 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:41:36,400 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:38,401 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:40,402 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:42,403 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:44,404 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:46,405 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:48,406 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:50,352 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:41:50,352 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:41:50,407 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:52,408 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:54,409 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:56,409 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:41:57,995 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:41:58,410 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:00,411 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:02,412 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:04,413 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:05,483 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:42:05,484 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:42:06,413 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:08,414 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:10,415 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:12,416 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:14,417 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:16,418 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:18,418 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:20,419 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:20,615 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:42:20,616 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:42:22,420 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:24,421 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:26,422 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:28,066 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:42:28,422 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:30,423 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:32,424 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:34,425 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:35,983 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:42:35,983 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:42:36,426 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:38,427 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:40,427 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:42,428 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:44,429 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:46,430 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:48,431 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:50,432 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:51,117 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:42:51,117 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:42:52,433 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:54,434 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:56,435 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:42:58,133 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:42:58,435 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:00,436 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:02,437 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:04,438 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:06,248 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:43:06,249 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:43:06,439 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:08,440 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:10,441 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:12,442 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:14,443 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:16,444 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:18,445 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:20,446 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:21,382 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:43:21,383 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:43:22,446 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:24,447 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:26,448 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:28,203 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:43:28,449 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:30,450 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:32,451 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:34,452 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:36,453 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:36,515 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:43:36,516 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:43:38,453 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:40,454 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:42,455 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:44,456 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:46,457 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:48,458 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:50,459 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:51,649 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:43:51,649 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:43:52,460 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:54,461 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:56,462 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:43:58,275 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:43:58,463 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:00,464 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:02,465 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:04,466 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:06,467 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:06,780 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:44:06,781 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:44:08,468 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:10,469 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:12,469 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:14,470 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:16,471 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:18,472 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:20,473 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:21,910 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:44:21,911 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:44:22,474 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:24,474 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:26,475 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:28,347 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:44:28,476 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:30,477 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:32,477 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:34,478 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:36,479 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:37,042 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:44:37,042 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:44:38,480 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:40,481 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:42,482 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:44,482 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:46,483 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:48,484 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:50,485 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:52,173 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:44:52,174 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:44:52,486 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:54,486 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:56,487 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:44:58,423 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:44:58,488 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:00,489 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:02,489 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:04,490 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:06,491 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:07,305 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:45:07,305 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:45:08,492 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:10,493 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:12,493 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:14,494 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:16,495 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:18,496 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:20,496 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:22,436 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:45:22,436 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:45:22,497 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:24,498 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:27,500 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:28,499 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:45:29,502 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:31,503 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:33,504 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:35,504 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:37,505 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:37,572 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:45:37,572 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:45:39,506 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:41,507 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:43,508 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:45,509 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:47,510 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:49,511 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:51,512 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:52,704 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:45:52,705 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:45:53,512 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:55,513 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:57,514 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:45:58,571 DEBUG SenderThread:690242 [sender.py:send():179] send: stats | |
| 2021-07-15 13:45:59,515 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:01,516 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:03,516 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:05,517 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:07,518 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:07,841 DEBUG HandlerThread:690242 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-15 13:46:07,841 DEBUG SenderThread:690242 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-15 13:46:09,519 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:11,520 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:13,520 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:15,521 INFO Thread-8 :690242 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/files/output.log | |
| 2021-07-15 13:46:15,809 WARNING MainThread:690242 [internal.py:wandb_internal():147] Internal process interrupt: 1 | |
| 2021-07-15 13:46:15,991 WARNING MainThread:690242 [internal.py:wandb_internal():147] Internal process interrupt: 2 | |
| 2021-07-15 13:46:15,991 ERROR MainThread:690242 [internal.py:wandb_internal():150] Internal process interrupted. | |
| 2021-07-15 13:46:16,587 INFO SenderThread:690242 [sender.py:finish():945] shutting down sender | |
| 2021-07-15 13:46:16,587 INFO SenderThread:690242 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-15 13:46:16,588 INFO WriterThread:690242 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210715_132956-37l0zsn1/run-37l0zsn1.wandb | |
| 2021-07-15 13:46:16,589 INFO HandlerThread:690242 [handler.py:finish():638] shutting down handler | |
| 2021-07-15 13:46:16,860 INFO MainThread:690242 [internal.py:handle_exit():78] Internal process exited | |