| 2021-07-14 20:07:31,715 INFO MainThread:566515 [internal.py:wandb_internal():88] W&B internal server running at pid: 566515, started at: 2021-07-14 20:07:31.714999 | |
| 2021-07-14 20:07:31,717 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-14 20:07:31,717 INFO WriterThread:566515 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/run-1nr232tb.wandb | |
| 2021-07-14 20:07:31,718 DEBUG SenderThread:566515 [sender.py:send():179] send: header | |
| 2021-07-14 20:07:31,718 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-14 20:07:31,754 DEBUG SenderThread:566515 [sender.py:send():179] send: run | |
| 2021-07-14 20:07:31,935 INFO SenderThread:566515 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files | |
| 2021-07-14 20:07:31,935 INFO SenderThread:566515 [sender.py:_start_run_threads():716] run started: 1nr232tb with start time 1626293251 | |
| 2021-07-14 20:07:31,935 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:07:31,936 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:07:31,936 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-14 20:07:32,938 INFO Thread-8 :566515 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:07:33,069 DEBUG HandlerThread:566515 [meta.py:__init__():39] meta init | |
| 2021-07-14 20:07:33,069 DEBUG HandlerThread:566515 [meta.py:__init__():53] meta init done | |
| 2021-07-14 20:07:33,069 DEBUG HandlerThread:566515 [meta.py:probe():210] probe | |
| 2021-07-14 20:07:33,071 DEBUG HandlerThread:566515 [meta.py:_setup_git():200] setup git | |
| 2021-07-14 20:07:33,100 DEBUG HandlerThread:566515 [meta.py:_setup_git():207] setup git done | |
| 2021-07-14 20:07:33,100 DEBUG HandlerThread:566515 [meta.py:_save_pip():57] save pip | |
| 2021-07-14 20:07:33,101 DEBUG HandlerThread:566515 [meta.py:_save_pip():71] save pip done | |
| 2021-07-14 20:07:33,101 DEBUG HandlerThread:566515 [meta.py:probe():252] probe done | |
| 2021-07-14 20:07:33,104 DEBUG SenderThread:566515 [sender.py:send():179] send: files | |
| 2021-07-14 20:07:33,104 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-14 20:07:33,110 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:07:33,110 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:07:33,237 DEBUG SenderThread:566515 [sender.py:send():179] send: config | |
| 2021-07-14 20:07:33,237 DEBUG SenderThread:566515 [sender.py:send():179] send: config | |
| 2021-07-14 20:07:33,237 DEBUG SenderThread:566515 [sender.py:send():179] send: config | |
| 2021-07-14 20:07:33,539 INFO Thread-11 :566515 [upload_job.py:push():137] Uploaded file /tmp/tmpx7_gpcyxwandb/2fjtpy4h-wandb-metadata.json | |
| 2021-07-14 20:07:33,936 INFO Thread-8 :566515 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:07:33,936 INFO Thread-8 :566515 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/requirements.txt | |
| 2021-07-14 20:07:33,937 INFO Thread-8 :566515 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-metadata.json | |
| 2021-07-14 20:07:47,942 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:07:48,239 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:07:48,239 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:07:53,944 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:08:01,150 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:08:02,948 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/config.yaml | |
| 2021-07-14 20:08:03,386 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:08:03,386 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:08:18,538 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:08:18,538 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:08:31,226 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:08:33,669 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:08:33,670 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:08:48,802 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:08:48,802 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:09:01,300 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:09:03,932 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:09:03,932 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:09:17,979 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:19,060 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:09:19,061 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:09:19,980 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:21,980 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:23,981 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:25,982 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:27,983 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:29,984 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:31,376 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:09:31,984 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:33,985 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:34,206 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:09:34,206 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:09:35,986 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:37,987 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:39,988 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:41,988 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:43,989 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:45,990 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:47,991 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:49,337 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:09:49,338 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:09:49,992 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:51,993 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:53,993 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:55,994 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:57,995 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:09:59,996 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:01,448 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:10:01,997 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:03,998 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:04,466 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:10:04,466 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:10:05,999 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:07,999 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:10,000 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:12,001 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:14,002 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:16,003 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:18,004 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:19,597 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:10:19,597 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:10:20,004 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:22,005 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:24,006 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:26,007 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:28,007 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:30,008 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:31,520 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:10:32,009 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:34,010 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:34,732 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:10:34,732 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:10:36,011 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:38,012 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:40,012 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:42,013 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:44,014 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:46,015 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:48,016 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:49,870 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:10:49,871 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:10:50,017 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:52,018 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:54,018 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:56,019 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:10:58,020 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:00,021 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:01,597 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:11:02,022 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:04,023 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:05,128 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:11:05,129 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:11:06,024 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:08,025 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:10,025 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:12,026 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:14,027 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:16,028 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:18,029 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:20,029 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:20,264 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:11:20,264 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:11:22,030 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:24,031 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:26,032 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:28,033 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:30,034 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:31,674 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:11:32,035 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:34,036 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:35,397 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:11:35,397 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:11:36,036 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:38,037 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:40,038 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:42,039 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:44,040 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:46,041 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:48,041 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:50,042 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:50,529 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:11:50,530 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:11:52,043 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:54,044 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:11:56,045 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:01,751 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:12:05,662 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:12:05,662 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:12:18,086 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:12:18,087 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:12:18,089 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:12:19,054 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:12:20,055 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:20,796 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:12:20,797 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:12:22,055 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:24,056 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:26,057 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:28,058 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:30,059 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:31,829 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:12:32,060 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:34,061 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:35,940 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:12:35,941 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:12:36,061 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:38,062 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:40,063 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:42,064 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:44,065 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:46,065 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:48,066 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:50,067 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:51,073 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:12:51,073 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:12:52,068 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:54,069 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:56,070 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:12:58,070 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:00,071 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:01,907 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:13:02,072 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:04,073 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:06,073 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:06,206 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:13:06,206 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:13:08,074 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:10,075 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:12,076 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:14,077 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:16,078 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:18,079 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:20,079 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:21,336 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:13:21,336 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:13:22,080 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:24,081 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:26,082 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:28,083 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:30,083 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:31,983 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:13:32,084 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:34,085 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:36,086 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:36,469 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:13:36,469 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:13:38,086 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:40,087 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:43,089 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:45,089 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:47,090 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:49,091 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:51,092 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:51,600 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:13:51,601 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:13:53,093 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:55,094 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:57,095 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:13:59,096 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:01,096 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:02,063 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:14:03,097 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:05,098 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:06,737 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:14:06,738 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:14:07,099 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:09,099 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:11,100 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:13,101 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:15,102 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:17,103 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:19,104 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:21,105 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:21,868 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:14:21,868 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:14:23,106 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:25,106 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:27,107 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:29,108 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:31,109 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:32,140 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:14:33,110 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:35,111 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:36,998 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:14:36,998 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:14:37,112 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:39,113 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:41,114 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:43,115 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:45,116 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:47,116 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:49,117 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:51,118 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:52,130 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:14:52,130 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:14:53,119 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:55,120 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:14:57,120 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:02,218 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:15:07,259 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:15:07,259 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:15:18,883 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:15:18,884 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:15:18,884 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:15:19,129 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:15:20,129 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:21,130 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:22,387 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:15:22,388 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:15:23,130 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:25,131 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:27,132 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:29,133 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:31,134 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:32,295 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:15:33,135 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:35,136 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:37,137 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:37,548 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:15:37,549 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:15:39,137 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:41,138 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:43,139 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:45,140 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:47,141 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:49,141 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:51,142 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:52,677 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:15:52,677 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:15:53,144 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:55,144 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:57,145 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:15:59,146 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:01,147 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:02,377 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:16:03,148 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:05,149 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:07,150 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:07,812 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:16:07,812 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:16:09,151 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:11,152 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:13,152 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:15,153 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:17,154 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:19,155 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:21,156 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:22,944 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:16:22,944 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:16:23,157 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:25,158 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:27,158 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:29,159 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:31,160 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:32,451 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:16:33,161 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:35,162 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:37,163 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:38,076 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:16:38,076 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:16:39,164 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:41,165 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:43,165 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:45,166 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:47,167 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:49,168 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:51,169 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:53,169 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:53,207 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:16:53,207 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:16:55,170 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:57,171 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:16:59,172 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:01,173 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:02,529 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:17:03,173 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:05,174 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:07,175 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:08,337 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:17:08,337 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:17:09,176 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:11,177 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:13,178 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:15,179 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:17,180 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:19,180 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:21,181 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:23,182 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:23,470 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:17:23,470 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:17:25,183 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:27,184 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:29,184 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:31,185 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:32,603 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:17:33,186 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:35,187 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:37,189 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:38,604 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:17:38,604 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:17:39,190 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:41,190 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:43,191 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:45,192 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:47,193 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:49,194 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:51,195 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:53,196 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:53,735 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:17:53,735 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:17:55,198 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:17:57,198 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:02,679 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:18:08,867 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:18:08,867 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:18:19,684 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:18:19,684 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:18:19,685 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:18:20,207 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:18:21,207 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:23,208 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:24,001 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:18:24,002 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:18:25,209 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:27,210 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:29,211 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:31,212 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:32,757 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:18:33,212 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:35,213 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:37,214 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:39,132 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:18:39,133 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:18:39,215 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:41,216 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:43,217 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:45,218 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:47,219 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:49,219 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:51,220 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:53,221 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:54,264 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:18:54,265 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:18:55,222 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:57,223 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:18:59,224 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:01,225 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:02,833 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:19:03,226 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:05,227 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:07,228 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:09,228 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:09,395 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:19:09,395 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:19:11,229 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:13,230 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:15,231 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:17,232 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:19,233 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:21,234 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:23,235 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:24,524 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:19:24,524 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:19:25,235 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:27,236 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:29,237 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:31,238 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:32,913 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:19:33,239 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:35,240 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:37,241 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:39,242 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:39,653 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:19:39,653 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:19:41,242 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:43,243 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:45,244 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:47,245 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:49,246 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:51,247 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:53,247 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:54,783 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:19:54,783 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:19:55,248 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:57,249 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:19:59,250 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:01,251 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:02,990 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:20:03,252 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:05,252 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:07,253 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:09,254 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:09,917 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:20:09,918 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:20:11,255 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:13,256 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:15,256 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:17,257 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:19,258 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:21,259 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:23,260 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:25,051 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:20:25,052 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:20:25,261 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:27,261 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:29,262 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:31,263 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:33,069 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:20:33,264 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:35,264 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:37,265 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:39,266 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:40,196 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:20:40,197 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:20:41,267 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:43,267 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:45,268 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:47,269 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:50,270 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:52,271 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:54,272 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:55,327 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:20:55,328 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:20:56,273 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:20:58,274 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:03,146 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:21:10,456 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:21:10,456 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:21:20,479 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:21:20,479 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:21:20,479 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:21:21,283 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:21,283 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:21:22,284 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:24,284 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:25,588 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:21:25,589 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:21:26,285 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:28,286 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:30,287 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:32,288 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:33,224 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:21:34,289 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:36,290 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:38,290 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:40,291 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:40,733 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:21:40,733 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:21:42,292 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:44,293 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:46,294 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:48,294 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:50,295 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:52,296 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:54,297 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:55,896 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:21:55,896 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:21:56,297 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:21:58,298 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:00,299 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:02,300 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:03,301 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:22:04,301 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:06,302 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:08,303 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:10,304 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:11,047 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:22:11,047 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:22:12,305 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:14,305 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:16,306 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:18,307 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:20,308 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:22,309 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:24,310 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:26,203 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:22:26,204 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:22:26,310 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:28,311 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:30,312 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:32,313 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:33,378 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:22:34,314 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:36,315 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:38,316 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:40,316 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:41,363 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:22:41,363 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:22:42,317 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:44,318 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:46,319 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:48,319 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:50,320 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:52,321 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:54,322 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:56,323 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:22:56,674 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:22:56,674 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:22:58,324 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:00,324 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:02,325 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:03,454 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:23:04,326 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:06,327 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:08,328 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:10,329 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:11,826 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:23:11,826 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:23:12,330 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:14,330 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:16,331 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:18,332 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:20,333 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:22,333 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:24,334 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:26,335 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:26,984 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:23:26,985 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:23:28,336 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:30,337 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:32,338 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:33,531 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:23:34,339 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:36,339 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:38,340 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:40,341 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:42,135 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:23:42,135 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:23:42,342 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:44,343 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:46,344 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:48,344 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:50,345 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:52,346 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:54,347 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:56,348 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:23:57,279 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:23:57,280 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:23:58,348 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:03,607 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:24:12,410 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:24:12,411 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:24:22,445 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:24:22,445 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:24:22,446 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:24:23,359 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:24:24,359 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:26,360 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:27,539 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:24:27,540 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:24:28,361 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:30,362 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:32,363 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:33,686 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:24:34,364 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:36,365 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:38,365 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:40,366 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:42,367 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:42,685 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:24:42,685 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:24:44,368 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:46,369 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:48,370 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:50,370 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:52,371 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:54,372 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:56,373 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:24:57,841 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:24:57,841 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:24:58,374 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:00,375 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:02,376 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:03,764 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:25:04,377 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:06,378 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:08,378 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:10,379 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:12,380 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:12,999 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:25:12,999 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:25:14,381 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:16,382 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:18,382 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:20,383 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:22,384 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:24,385 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:26,386 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:28,149 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:25:28,149 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:25:28,387 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:30,388 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:32,388 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:33,841 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:25:34,389 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:36,390 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:38,391 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:40,392 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:42,393 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:43,294 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:25:43,295 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:25:44,394 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:46,395 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:48,395 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:50,396 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:52,397 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:54,398 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:56,399 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:58,399 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:25:58,450 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:25:58,451 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:26:00,400 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:02,401 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:03,919 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:26:04,402 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:06,403 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:08,404 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:10,405 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:12,406 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:13,618 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:26:13,618 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:26:14,406 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:16,407 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:18,408 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:20,409 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:22,409 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:24,410 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:26,411 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:28,412 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:28,768 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:26:28,769 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:26:30,413 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:32,414 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:33,994 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:26:34,415 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:36,415 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:38,416 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:40,417 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:42,418 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:43,915 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:26:43,915 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:26:44,419 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:46,420 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:48,420 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:50,421 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:52,422 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:54,423 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:56,423 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:58,424 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:26:59,082 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:26:59,083 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:27:00,425 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:04,074 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:27:14,232 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:27:14,233 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:27:23,233 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:27:23,233 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:27:23,234 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:27:23,434 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:27:24,434 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:27,436 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:29,366 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:27:29,366 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:27:29,436 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:31,437 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:33,438 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:34,152 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:27:35,439 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:37,440 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:39,441 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:41,442 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:43,443 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:44,501 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:27:44,501 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:27:45,444 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:47,445 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:49,445 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:51,446 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:53,447 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:55,448 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:57,448 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:59,449 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:27:59,635 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:27:59,635 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:28:01,450 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:03,451 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:04,228 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:28:05,451 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:07,452 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:09,453 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:11,454 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:13,455 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:14,769 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:28:14,769 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:28:15,456 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:17,457 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:19,458 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:21,458 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:23,459 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:25,460 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:27,461 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:29,462 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:29,907 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:28:29,907 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:28:31,462 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:33,463 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:34,306 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:28:35,464 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:37,465 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:39,466 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:41,466 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:43,467 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:45,053 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:28:45,053 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:28:45,468 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:47,469 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:49,470 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:51,471 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:53,472 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:55,472 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:57,473 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:28:59,474 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:00,186 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:29:00,186 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:29:01,475 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:03,476 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:04,385 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:29:05,477 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:07,477 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:09,478 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:11,479 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:13,480 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:15,423 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:29:15,424 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:29:15,481 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:17,481 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:19,482 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:21,483 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:23,484 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:25,485 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:27,485 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:29,486 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:30,554 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:29:30,554 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:29:31,487 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:33,488 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:34,462 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:29:35,489 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:37,490 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:39,490 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:41,491 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:43,492 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:45,493 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:45,688 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:29:45,688 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:29:47,494 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:49,495 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:51,495 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:53,496 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:55,497 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:57,498 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:29:59,499 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:30:00,818 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:30:00,818 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:30:01,500 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:30:04,537 DEBUG SenderThread:566515 [sender.py:send():179] send: stats | |
| 2021-07-14 20:30:15,952 DEBUG HandlerThread:566515 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:30:15,952 DEBUG SenderThread:566515 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:30:24,030 DEBUG SenderThread:566515 [sender.py:send():179] send: history | |
| 2021-07-14 20:30:24,031 DEBUG SenderThread:566515 [sender.py:send():179] send: summary | |
| 2021-07-14 20:30:24,031 INFO SenderThread:566515 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:30:24,509 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:30:25,509 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:30:26,440 WARNING MainThread:566515 [internal.py:wandb_internal():147] Internal process interrupt: 1 | |
| 2021-07-14 20:30:27,510 INFO Thread-8 :566515 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:30:27,654 WARNING MainThread:566515 [internal.py:wandb_internal():147] Internal process interrupt: 2 | |
| 2021-07-14 20:30:27,654 ERROR MainThread:566515 [internal.py:wandb_internal():150] Internal process interrupted. | |
| 2021-07-14 20:30:27,903 INFO SenderThread:566515 [sender.py:finish():945] shutting down sender | |
| 2021-07-14 20:30:27,903 INFO WriterThread:566515 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/run-1nr232tb.wandb | |
| 2021-07-14 20:30:27,903 INFO SenderThread:566515 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-14 20:30:27,904 INFO HandlerThread:566515 [handler.py:finish():638] shutting down handler | |
| 2021-07-14 20:30:28,511 INFO SenderThread:566515 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files | |
| 2021-07-14 20:30:28,511 INFO SenderThread:566515 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/requirements.txt requirements.txt | |
| 2021-07-14 20:30:28,511 INFO SenderThread:566515 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log output.log | |
| 2021-07-14 20:30:28,511 INFO SenderThread:566515 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-14 20:30:28,511 INFO SenderThread:566515 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/config.yaml config.yaml | |
| 2021-07-14 20:30:28,512 INFO SenderThread:566515 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json wandb-summary.json | |
| 2021-07-14 20:30:28,515 INFO SenderThread:566515 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 20:30:28,515 INFO SenderThread:566515 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 20:30:29,047 INFO Thread-15 :566515 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/wandb-summary.json | |
| 2021-07-14 20:30:29,220 INFO Thread-12 :566515 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/requirements.txt | |
| 2021-07-14 20:30:29,221 INFO Thread-14 :566515 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/config.yaml | |
| 2021-07-14 20:30:29,222 INFO Thread-13 :566515 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_200731-1nr232tb/files/output.log | |
| 2021-07-14 20:30:29,224 INFO MainThread:566515 [internal.py:handle_exit():78] Internal process exited | |