| 2021-07-13 00:20:32,185 INFO MainThread:320742 [internal.py:wandb_internal():88] W&B internal server running at pid: 320742, started at: 2021-07-13 00:20:32.185297 | |
| 2021-07-13 00:20:32,187 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-13 00:20:32,187 INFO WriterThread:320742 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/run-11bfig2u.wandb | |
| 2021-07-13 00:20:32,188 DEBUG SenderThread:320742 [sender.py:send():179] send: header | |
| 2021-07-13 00:20:32,189 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-13 00:20:32,228 DEBUG SenderThread:320742 [sender.py:send():179] send: run | |
| 2021-07-13 00:20:32,408 INFO SenderThread:320742 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files | |
| 2021-07-13 00:20:32,409 INFO SenderThread:320742 [sender.py:_start_run_threads():716] run started: 11bfig2u with start time 1626135631 | |
| 2021-07-13 00:20:32,409 DEBUG SenderThread:320742 [sender.py:send():179] send: summary | |
| 2021-07-13 00:20:32,409 INFO SenderThread:320742 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-13 00:20:32,409 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-13 00:20:33,411 INFO Thread-8 :320742 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-summary.json | |
| 2021-07-13 00:20:33,520 DEBUG HandlerThread:320742 [meta.py:__init__():39] meta init | |
| 2021-07-13 00:20:33,520 DEBUG HandlerThread:320742 [meta.py:__init__():53] meta init done | |
| 2021-07-13 00:20:33,520 DEBUG HandlerThread:320742 [meta.py:probe():210] probe | |
| 2021-07-13 00:20:33,522 DEBUG HandlerThread:320742 [meta.py:_setup_git():200] setup git | |
| 2021-07-13 00:20:33,551 DEBUG HandlerThread:320742 [meta.py:_setup_git():207] setup git done | |
| 2021-07-13 00:20:33,551 DEBUG HandlerThread:320742 [meta.py:_save_pip():57] save pip | |
| 2021-07-13 00:20:33,551 DEBUG HandlerThread:320742 [meta.py:_save_pip():71] save pip done | |
| 2021-07-13 00:20:33,551 DEBUG HandlerThread:320742 [meta.py:probe():252] probe done | |
| 2021-07-13 00:20:33,554 DEBUG SenderThread:320742 [sender.py:send():179] send: files | |
| 2021-07-13 00:20:33,554 INFO SenderThread:320742 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-13 00:20:33,561 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:20:33,561 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:20:33,690 DEBUG SenderThread:320742 [sender.py:send():179] send: config | |
| 2021-07-13 00:20:33,691 DEBUG SenderThread:320742 [sender.py:send():179] send: config | |
| 2021-07-13 00:20:33,691 DEBUG SenderThread:320742 [sender.py:send():179] send: config | |
| 2021-07-13 00:20:34,001 INFO Thread-11 :320742 [upload_job.py:push():137] Uploaded file /tmp/tmpp780arizwandb/1gon5458-wandb-metadata.json | |
| 2021-07-13 00:20:34,410 INFO Thread-8 :320742 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:20:34,410 INFO Thread-8 :320742 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-metadata.json | |
| 2021-07-13 00:20:34,410 INFO Thread-8 :320742 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/requirements.txt | |
| 2021-07-13 00:20:48,692 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:20:48,692 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:20:52,417 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:21:01,606 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:21:03,421 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/config.yaml | |
| 2021-07-13 00:21:03,825 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:21:03,825 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:21:18,959 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:21:18,959 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:21:31,685 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:21:34,089 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:21:34,090 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:21:49,221 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:21:49,221 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:22:01,760 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:22:04,352 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:22:04,353 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:22:19,487 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:22:19,487 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:22:24,448 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:22:31,839 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:22:34,633 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:22:34,633 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:22:49,777 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:22:49,778 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:23:01,918 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:23:04,908 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:23:04,908 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:23:20,043 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:23:20,043 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:23:31,998 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:23:35,174 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:23:35,174 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:23:50,319 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:23:50,319 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:24:02,076 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:24:04,486 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:05,464 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:24:05,464 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:24:06,487 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:08,488 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:10,489 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:12,489 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:14,491 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:16,491 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:18,492 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:20,493 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:20,641 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:24:20,641 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:24:22,494 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:24,495 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:26,496 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:28,496 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:30,497 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:32,154 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:24:32,498 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:34,499 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:35,771 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:24:35,771 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:24:36,500 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:38,500 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:40,501 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:42,502 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:44,502 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:46,503 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:48,504 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:50,505 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:50,909 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:24:50,910 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:24:52,506 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:54,506 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:56,507 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:24:58,508 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:00,509 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:02,232 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:25:02,509 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:04,510 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:06,066 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:25:06,067 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:25:06,511 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:08,512 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:10,513 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:12,513 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:14,514 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:16,515 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:18,516 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:20,517 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:21,197 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:25:21,198 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:25:22,518 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:24,518 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:26,519 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:28,520 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:30,521 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:32,322 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:25:32,521 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:34,522 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:36,329 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:25:36,329 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:25:36,523 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:39,524 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:41,525 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:43,526 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:45,526 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:47,527 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:49,528 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:51,529 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:51,530 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:25:51,530 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:25:53,530 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:55,531 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:57,531 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:25:59,532 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:01,533 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:02,385 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:26:03,534 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:05,535 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:06,658 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:26:06,659 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:26:07,535 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:09,536 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:11,537 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:13,538 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:15,538 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:17,539 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:19,540 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:21,541 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:21,788 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:26:21,788 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:26:23,542 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:25,542 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:27,543 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:29,544 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:31,545 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:32,454 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:26:33,545 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:35,546 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:36,954 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:26:36,954 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:26:37,547 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:39,548 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:41,548 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:43,549 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:45,550 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:47,551 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:49,552 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:51,553 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:52,093 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:26:52,094 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:26:53,553 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:55,554 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:57,555 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:26:59,556 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:01,556 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:02,524 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:27:03,557 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:05,558 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:07,225 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:27:07,225 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:27:07,559 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:09,559 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:11,560 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:13,561 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:15,562 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:17,563 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:19,563 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:21,564 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:22,412 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:27:22,412 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:27:23,565 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:25,566 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:27,566 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:29,567 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:31,568 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:32,597 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:27:33,569 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:35,569 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:37,542 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:27:37,542 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:27:37,570 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:39,571 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:41,572 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:43,572 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:45,573 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:47,574 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:49,575 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:51,575 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:52,672 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:27:52,673 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:27:53,576 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:55,577 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:57,578 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:27:59,578 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:01,579 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:02,671 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:28:03,580 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:05,581 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:07,582 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:07,868 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:28:07,869 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:28:10,583 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:12,583 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:14,584 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:16,585 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:18,586 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:20,587 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:22,588 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:22,998 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:28:22,998 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:28:24,588 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:26,589 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:28,590 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:30,591 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:32,592 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:32,746 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:28:34,593 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:38,128 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:28:38,128 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:28:52,048 DEBUG SenderThread:320742 [sender.py:send():179] send: history | |
| 2021-07-13 00:28:52,049 DEBUG SenderThread:320742 [sender.py:send():179] send: summary | |
| 2021-07-13 00:28:52,049 INFO SenderThread:320742 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-13 00:28:52,600 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-summary.json | |
| 2021-07-13 00:28:53,447 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:28:53,447 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:28:54,601 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:56,601 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:28:58,602 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:00,603 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:02,604 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:02,822 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:29:04,605 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:06,606 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:08,607 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:08,643 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:29:08,643 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:29:10,607 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:12,608 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:14,609 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:16,610 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:18,611 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:20,612 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:22,612 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:23,774 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:29:23,774 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:29:24,613 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:26,614 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:28,615 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:30,616 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:32,617 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:32,899 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:29:34,618 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:36,618 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:38,619 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:38,915 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:29:38,916 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:29:40,620 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:42,621 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:44,621 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:46,622 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:48,623 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:50,624 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:52,625 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:54,045 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:29:54,046 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:29:54,626 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:56,627 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:29:58,628 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:00,629 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:02,630 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:02,986 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:30:04,631 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:06,632 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:08,632 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:09,182 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:30:09,183 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:30:10,633 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:12,634 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:14,635 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:16,636 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:18,637 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:20,638 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:22,638 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:24,346 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:30:24,347 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:30:24,639 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:26,640 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:28,641 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:30,642 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:32,642 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:33,062 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:30:34,643 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:36,644 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:38,645 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:39,476 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:30:39,477 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:30:40,646 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:42,646 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:44,647 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:46,648 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:48,649 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:50,650 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:52,651 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:54,608 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:30:54,608 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:30:54,652 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:56,652 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:30:58,653 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:00,654 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:02,655 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:03,139 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:31:04,656 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:06,656 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:08,657 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:09,738 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:31:09,738 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:31:11,658 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:13,659 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:15,660 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:17,661 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:19,662 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:21,663 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:23,663 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:24,869 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:31:24,869 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:31:25,664 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:27,665 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:29,666 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:31,667 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:33,216 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:31:33,668 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:35,669 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:37,670 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:39,671 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:40,001 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:31:40,001 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:31:41,672 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:43,673 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:45,674 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:47,674 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:49,675 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:51,676 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:53,677 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:55,133 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:31:55,134 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:31:55,678 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:57,679 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:31:59,680 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:01,681 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:03,294 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:32:03,682 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:05,683 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:07,684 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:09,685 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:10,266 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:32:10,266 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:32:11,686 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:13,687 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:15,688 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:17,689 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:19,690 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:21,691 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:23,691 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:25,396 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:32:25,396 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:32:25,692 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:27,693 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:29,694 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:31,695 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:33,373 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:32:33,696 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:35,697 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:37,698 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:39,699 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:40,527 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:32:40,527 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:32:41,700 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:43,701 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:45,702 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:47,703 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:49,704 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:51,704 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:53,705 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:55,686 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:32:55,686 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:32:55,706 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:57,707 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:32:59,708 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:01,709 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:03,450 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:33:03,710 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:05,711 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:07,712 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:09,713 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:10,817 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:33:10,817 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:33:11,714 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:13,715 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:15,716 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:17,717 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:19,718 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:21,719 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:25,951 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:33:25,951 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:33:33,528 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:33:40,391 DEBUG SenderThread:320742 [sender.py:send():179] send: history | |
| 2021-07-13 00:33:40,392 DEBUG SenderThread:320742 [sender.py:send():179] send: summary | |
| 2021-07-13 00:33:40,393 INFO SenderThread:320742 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-13 00:33:40,727 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-summary.json | |
| 2021-07-13 00:33:41,082 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:33:41,083 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:33:41,727 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:43,728 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:45,729 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:47,730 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:49,731 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:51,732 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:53,732 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:55,733 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:56,269 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:33:56,269 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:33:57,734 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:33:59,735 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:01,736 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:03,606 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:34:03,737 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:05,738 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:07,739 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:09,740 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:11,403 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:34:11,403 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:34:12,741 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:14,742 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:16,743 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:18,744 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:20,745 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:22,746 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:24,747 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:26,534 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:34:26,535 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:34:26,747 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:28,748 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:30,749 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:32,750 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:33,683 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:34:34,751 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:36,752 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:38,753 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:40,754 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:41,673 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:34:41,674 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:34:42,755 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:44,756 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:46,757 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:48,758 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:50,759 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:52,759 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:54,760 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:56,761 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:34:56,805 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:34:56,805 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:34:58,762 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:00,763 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:02,764 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:03,761 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:35:04,765 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:06,766 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:08,767 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:10,768 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:11,934 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:35:11,935 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:35:12,769 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:14,770 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:16,771 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:18,771 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:20,772 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:22,773 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:24,774 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:26,775 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:27,118 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:35:27,118 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:35:28,776 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:30,777 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:32,778 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:33,840 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:35:34,779 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:36,780 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:38,781 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:40,781 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:42,250 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:35:42,250 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:35:42,782 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:44,783 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:46,784 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:48,785 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:50,786 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:52,787 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:54,788 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:56,789 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:35:57,380 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:35:57,380 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:35:58,790 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:00,791 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:02,792 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:03,916 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:36:04,792 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:06,793 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:08,794 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:11,795 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:12,574 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:36:12,574 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:36:13,796 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:15,797 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:17,798 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:19,799 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:21,799 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:23,800 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:25,801 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:27,703 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:36:27,703 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:36:27,802 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:29,803 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:31,804 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:33,805 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:33,992 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:36:35,806 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:37,806 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:39,807 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:41,808 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:42,837 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:36:42,837 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:36:43,809 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:45,809 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:47,810 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:49,811 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:51,812 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:53,813 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:55,813 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:57,814 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:36:58,033 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:36:58,034 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:36:59,815 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:01,816 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:03,817 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:04,067 DEBUG SenderThread:320742 [sender.py:send():179] send: stats | |
| 2021-07-13 00:37:05,818 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:07,819 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:09,819 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:11,820 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:13,164 DEBUG HandlerThread:320742 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-13 00:37:13,164 DEBUG SenderThread:320742 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-13 00:37:13,821 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:15,822 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:17,823 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:19,824 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:21,824 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:23,825 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:25,826 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:27,492 WARNING MainThread:320742 [internal.py:wandb_internal():147] Internal process interrupt: 1 | |
| 2021-07-13 00:37:27,730 WARNING MainThread:320742 [internal.py:wandb_internal():147] Internal process interrupt: 2 | |
| 2021-07-13 00:37:27,731 ERROR MainThread:320742 [internal.py:wandb_internal():150] Internal process interrupted. | |
| 2021-07-13 00:37:27,827 INFO Thread-8 :320742 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log | |
| 2021-07-13 00:37:28,330 INFO SenderThread:320742 [sender.py:finish():945] shutting down sender | |
| 2021-07-13 00:37:28,330 INFO SenderThread:320742 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-13 00:37:28,330 INFO WriterThread:320742 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/run-11bfig2u.wandb | |
| 2021-07-13 00:37:28,331 INFO HandlerThread:320742 [handler.py:finish():638] shutting down handler | |
| 2021-07-13 00:37:28,828 INFO SenderThread:320742 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files | |
| 2021-07-13 00:37:28,829 INFO SenderThread:320742 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/requirements.txt requirements.txt | |
| 2021-07-13 00:37:28,829 INFO SenderThread:320742 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/output.log output.log | |
| 2021-07-13 00:37:28,829 INFO SenderThread:320742 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-13 00:37:28,829 INFO SenderThread:320742 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/config.yaml config.yaml | |
| 2021-07-13 00:37:28,829 INFO SenderThread:320742 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210713_002031-11bfig2u/files/wandb-summary.json wandb-summary.json | |
| 2021-07-13 00:37:28,830 INFO SenderThread:320742 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-13 00:37:28,830 INFO SenderThread:320742 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-13 00:37:29,011 INFO MainThread:320742 [internal.py:handle_exit():78] Internal process exited | |