diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,50 +1,50 @@ -[2023-02-25 13:36:50,795][00699] Saving configuration to /content/train_dir/default_experiment/config.json... -[2023-02-25 13:36:50,798][00699] Rollout worker 0 uses device cpu -[2023-02-25 13:36:50,799][00699] Rollout worker 1 uses device cpu -[2023-02-25 13:36:50,803][00699] Rollout worker 2 uses device cpu -[2023-02-25 13:36:50,804][00699] Rollout worker 3 uses device cpu -[2023-02-25 13:36:50,806][00699] Rollout worker 4 uses device cpu -[2023-02-25 13:36:50,807][00699] Rollout worker 5 uses device cpu -[2023-02-25 13:36:50,809][00699] Rollout worker 6 uses device cpu -[2023-02-25 13:36:50,810][00699] Rollout worker 7 uses device cpu -[2023-02-25 13:36:51,020][00699] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:36:51,025][00699] InferenceWorker_p0-w0: min num requests: 2 -[2023-02-25 13:36:51,055][00699] Starting all processes... -[2023-02-25 13:36:51,057][00699] Starting process learner_proc0 -[2023-02-25 13:36:51,111][00699] Starting all processes... -[2023-02-25 13:36:51,129][00699] Starting process inference_proc0-0 -[2023-02-25 13:36:51,133][00699] Starting process rollout_proc0 -[2023-02-25 13:36:51,133][00699] Starting process rollout_proc1 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc3 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc4 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc5 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc6 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc7 -[2023-02-25 13:36:51,141][00699] Starting process rollout_proc2 -[2023-02-25 13:37:01,955][10893] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:37:01,963][10893] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-02-25 13:37:02,647][10911] Worker 5 uses CPU cores [1] -[2023-02-25 13:37:03,151][10907] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:37:03,160][10907] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-02-25 13:37:03,356][10909] Worker 1 uses CPU cores [1] -[2023-02-25 13:37:03,421][10908] Worker 0 uses CPU cores [0] -[2023-02-25 13:37:03,684][10912] Worker 4 uses CPU cores [0] -[2023-02-25 13:37:03,765][10910] Worker 3 uses CPU cores [1] -[2023-02-25 13:37:03,771][10914] Worker 7 uses CPU cores [1] -[2023-02-25 13:37:03,932][10915] Worker 2 uses CPU cores [0] -[2023-02-25 13:37:03,933][10913] Worker 6 uses CPU cores [0] -[2023-02-25 13:37:04,054][10907] Num visible devices: 1 -[2023-02-25 13:37:04,057][10893] Num visible devices: 1 -[2023-02-25 13:37:04,078][10893] Starting seed is not provided -[2023-02-25 13:37:04,079][10893] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:37:04,079][10893] Initializing actor-critic model on device cuda:0 -[2023-02-25 13:37:04,079][10893] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:37:04,081][10893] RunningMeanStd input shape: (1,) -[2023-02-25 13:37:04,135][10893] ConvEncoder: input_channels=3 -[2023-02-25 13:37:04,617][10893] Conv encoder output size: 512 -[2023-02-25 13:37:04,618][10893] Policy head output size: 512 -[2023-02-25 13:37:04,697][10893] Created Actor Critic model with architecture: -[2023-02-25 13:37:04,697][10893] ActorCriticSharedWeights( +[2023-02-26 10:10:18,458][00304] Saving configuration to /content/train_dir/default_experiment/config.json... +[2023-02-26 10:10:18,462][00304] Rollout worker 0 uses device cpu +[2023-02-26 10:10:18,464][00304] Rollout worker 1 uses device cpu +[2023-02-26 10:10:18,467][00304] Rollout worker 2 uses device cpu +[2023-02-26 10:10:18,468][00304] Rollout worker 3 uses device cpu +[2023-02-26 10:10:18,469][00304] Rollout worker 4 uses device cpu +[2023-02-26 10:10:18,472][00304] Rollout worker 5 uses device cpu +[2023-02-26 10:10:18,473][00304] Rollout worker 6 uses device cpu +[2023-02-26 10:10:18,475][00304] Rollout worker 7 uses device cpu +[2023-02-26 10:10:18,704][00304] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-26 10:10:18,707][00304] InferenceWorker_p0-w0: min num requests: 2 +[2023-02-26 10:10:18,751][00304] Starting all processes... +[2023-02-26 10:10:18,754][00304] Starting process learner_proc0 +[2023-02-26 10:10:18,840][00304] Starting all processes... +[2023-02-26 10:10:18,854][00304] Starting process inference_proc0-0 +[2023-02-26 10:10:18,870][00304] Starting process rollout_proc0 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc1 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc2 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc3 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc4 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc5 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc6 +[2023-02-26 10:10:18,871][00304] Starting process rollout_proc7 +[2023-02-26 10:10:32,790][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-26 10:10:32,792][10798] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2023-02-26 10:10:33,007][10818] Worker 5 uses CPU cores [1] +[2023-02-26 10:10:33,023][10813] Worker 1 uses CPU cores [1] +[2023-02-26 10:10:33,128][10820] Worker 7 uses CPU cores [1] +[2023-02-26 10:10:33,180][10814] Worker 2 uses CPU cores [0] +[2023-02-26 10:10:33,339][10817] Worker 4 uses CPU cores [0] +[2023-02-26 10:10:33,362][10816] Worker 3 uses CPU cores [1] +[2023-02-26 10:10:33,506][10815] Worker 0 uses CPU cores [0] +[2023-02-26 10:10:33,647][10811] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-26 10:10:33,652][10811] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2023-02-26 10:10:33,721][10819] Worker 6 uses CPU cores [0] +[2023-02-26 10:10:33,860][10811] Num visible devices: 1 +[2023-02-26 10:10:33,865][10798] Num visible devices: 1 +[2023-02-26 10:10:33,866][10798] Starting seed is not provided +[2023-02-26 10:10:33,866][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-26 10:10:33,867][10798] Initializing actor-critic model on device cuda:0 +[2023-02-26 10:10:33,867][10798] RunningMeanStd input shape: (3, 72, 128) +[2023-02-26 10:10:33,869][10798] RunningMeanStd input shape: (1,) +[2023-02-26 10:10:33,896][10798] ConvEncoder: input_channels=3 +[2023-02-26 10:10:34,468][10798] Conv encoder output size: 512 +[2023-02-26 10:10:34,469][10798] Policy head output size: 512 +[2023-02-26 10:10:34,548][10798] Created Actor Critic model with architecture: +[2023-02-26 10:10:34,549][10798] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -85,1569 +85,1061 @@ (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) -[2023-02-25 13:37:11,013][00699] Heartbeat connected on Batcher_0 -[2023-02-25 13:37:11,021][00699] Heartbeat connected on InferenceWorker_p0-w0 -[2023-02-25 13:37:11,031][00699] Heartbeat connected on RolloutWorker_w0 -[2023-02-25 13:37:11,035][00699] Heartbeat connected on RolloutWorker_w1 -[2023-02-25 13:37:11,038][00699] Heartbeat connected on RolloutWorker_w2 -[2023-02-25 13:37:11,041][00699] Heartbeat connected on RolloutWorker_w3 -[2023-02-25 13:37:11,045][00699] Heartbeat connected on RolloutWorker_w4 -[2023-02-25 13:37:11,050][00699] Heartbeat connected on RolloutWorker_w6 -[2023-02-25 13:37:11,051][00699] Heartbeat connected on RolloutWorker_w5 -[2023-02-25 13:37:11,054][00699] Heartbeat connected on RolloutWorker_w7 -[2023-02-25 13:37:13,611][10893] Using optimizer -[2023-02-25 13:37:13,612][10893] No checkpoints found -[2023-02-25 13:37:13,612][10893] Did not load from checkpoint, starting from scratch! -[2023-02-25 13:37:13,613][10893] Initialized policy 0 weights for model version 0 -[2023-02-25 13:37:13,617][10893] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:37:13,624][10893] LearnerWorker_p0 finished initialization! -[2023-02-25 13:37:13,625][00699] Heartbeat connected on LearnerWorker_p0 -[2023-02-25 13:37:13,719][10907] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:37:13,721][10907] RunningMeanStd input shape: (1,) -[2023-02-25 13:37:13,739][10907] ConvEncoder: input_channels=3 -[2023-02-25 13:37:13,836][10907] Conv encoder output size: 512 -[2023-02-25 13:37:13,836][10907] Policy head output size: 512 -[2023-02-25 13:37:15,425][00699] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-25 13:37:16,389][00699] Inference worker 0-0 is ready! -[2023-02-25 13:37:16,391][00699] All inference workers are ready! Signal rollout workers to start! -[2023-02-25 13:37:16,499][10913] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,511][10915] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,515][10912] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,556][00699] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 699], exiting... -[2023-02-25 13:37:16,562][10893] Stopping Batcher_0... -[2023-02-25 13:37:16,563][10893] Loop batcher_evt_loop terminating... -[2023-02-25 13:37:16,561][00699] Runner profile tree view: -main_loop: 25.5057 -[2023-02-25 13:37:16,568][10893] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... -[2023-02-25 13:37:16,566][00699] Collected {0: 0}, FPS: 0.0 -[2023-02-25 13:37:16,555][10908] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,627][10907] Weights refcount: 2 0 -[2023-02-25 13:37:16,639][10907] Stopping InferenceWorker_p0-w0... -[2023-02-25 13:37:16,643][10907] Loop inference_proc0-0_evt_loop terminating... -[2023-02-25 13:37:16,653][00699] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-25 13:37:16,666][10893] Stopping LearnerWorker_p0... -[2023-02-25 13:37:16,666][10893] Loop learner_proc0_evt_loop terminating... -[2023-02-25 13:37:16,661][00699] Overriding arg 'num_workers' with value 1 passed from command line -[2023-02-25 13:37:16,667][00699] Adding new argument 'no_render'=True that is not in the saved config file! -[2023-02-25 13:37:16,672][00699] Adding new argument 'save_video'=True that is not in the saved config file! -[2023-02-25 13:37:16,678][00699] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2023-02-25 13:37:16,683][00699] Adding new argument 'video_name'=None that is not in the saved config file! -[2023-02-25 13:37:16,689][00699] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! -[2023-02-25 13:37:16,690][00699] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2023-02-25 13:37:16,693][00699] Adding new argument 'push_to_hub'=False that is not in the saved config file! -[2023-02-25 13:37:16,695][00699] Adding new argument 'hf_repository'=None that is not in the saved config file! -[2023-02-25 13:37:16,700][00699] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2023-02-25 13:37:16,706][00699] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2023-02-25 13:37:16,711][00699] Adding new argument 'train_script'=None that is not in the saved config file! -[2023-02-25 13:37:16,713][00699] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2023-02-25 13:37:16,719][00699] Using frameskip 1 and render_action_repeat=4 for evaluation -[2023-02-25 13:37:16,786][00699] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,795][00699] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:37:16,803][00699] RunningMeanStd input shape: (1,) -[2023-02-25 13:37:16,866][10911] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,862][00699] ConvEncoder: input_channels=3 -[2023-02-25 13:37:16,906][10914] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:16,918][10910] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:17,007][10909] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:37:17,199][00699] Conv encoder output size: 512 -[2023-02-25 13:37:17,214][00699] Policy head output size: 512 -[2023-02-25 13:37:20,333][10915] Decorrelating experience for 0 frames... -[2023-02-25 13:37:20,335][10913] Decorrelating experience for 0 frames... -[2023-02-25 13:37:20,337][10908] Decorrelating experience for 0 frames... -[2023-02-25 13:37:20,515][10911] Decorrelating experience for 0 frames... -[2023-02-25 13:37:22,092][10914] Decorrelating experience for 0 frames... -[2023-02-25 13:37:22,349][00699] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... -[2023-02-25 13:37:22,486][10912] Decorrelating experience for 0 frames... -[2023-02-25 13:37:22,488][10913] Decorrelating experience for 32 frames... -[2023-02-25 13:37:22,490][10915] Decorrelating experience for 32 frames... -[2023-02-25 13:37:22,495][10908] Decorrelating experience for 32 frames... -[2023-02-25 13:37:22,798][10914] Decorrelating experience for 32 frames... -[2023-02-25 13:37:22,874][00699] VizDoom game.init() threw an exception SignalException('Signal SIGINT received. ViZDoom instance has been closed.'). Terminate process... -[2023-02-25 13:37:22,881][10915] VizDoom game.init() threw an exception SignalException('Signal SIGINT received. ViZDoom instance has been closed.'). Terminate process... -[2023-02-25 13:37:22,889][10908] VizDoom game.init() threw an exception SignalException('Signal SIGINT received. ViZDoom instance has been closed.'). Terminate process... -[2023-02-25 13:37:22,892][10912] VizDoom game.init() threw an exception SignalException('Signal SIGINT received. ViZDoom instance has been closed.'). Terminate process... -[2023-02-25 13:37:22,897][10909] VizDoom game.init() threw an exception SignalException('Signal SIGINT received. ViZDoom instance has been closed.'). Terminate process... -[2023-02-25 13:37:22,889][10914] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 439, in _reset - observations, rew, terminated, truncated, info = e.step(actions) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 319, in step - return self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 129, in step - obs, rew, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step - obs, rew, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step - observation, reward, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 384, in step - observation, reward, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 88, in step - obs, reward, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 319, in step - return self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step - obs, reward, terminated, truncated, info = self.env.step(action) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step - reward = self.game.make_action(actions_flattened, self.skip_frames) -vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. -[2023-02-25 13:37:22,914][10914] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc7_evt_loop -[2023-02-25 13:37:22,902][10912] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init - self.game.init() -vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset - observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset - self._ensure_initialized() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized - self.initialize() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize - self._game_init() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init - raise EnvCriticalError() -sample_factory.envs.env_utils.EnvCriticalError -[2023-02-25 13:37:22,915][10912] Unhandled exception in evt loop rollout_proc4_evt_loop -[2023-02-25 13:37:22,902][10909] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init - self.game.init() -vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset - observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset - self._ensure_initialized() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized - self.initialize() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize - self._game_init() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init - raise EnvCriticalError() -sample_factory.envs.env_utils.EnvCriticalError -[2023-02-25 13:37:22,930][10909] Unhandled exception in evt loop rollout_proc1_evt_loop -[2023-02-25 13:37:22,882][10915] EvtLoop [rollout_proc2_evt_loop, process=rollout_proc2] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init - self.game.init() -vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset - observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset - self._ensure_initialized() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized - self.initialize() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize - self._game_init() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init - raise EnvCriticalError() -sample_factory.envs.env_utils.EnvCriticalError -[2023-02-25 13:37:22,931][10915] Unhandled exception in evt loop rollout_proc2_evt_loop -[2023-02-25 13:37:22,893][10908] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init - self.game.init() -vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset - observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset - self._ensure_initialized() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized - self.initialize() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize - self._game_init() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init - raise EnvCriticalError() -sample_factory.envs.env_utils.EnvCriticalError -[2023-02-25 13:37:22,935][10908] Unhandled exception in evt loop rollout_proc0_evt_loop -[2023-02-25 13:37:24,225][10910] Decorrelating experience for 0 frames... -[2023-02-25 13:37:24,579][10913] Decorrelating experience for 64 frames... -[2023-02-25 13:37:24,610][10910] Decorrelating experience for 32 frames... -[2023-02-25 13:37:25,165][10911] Decorrelating experience for 32 frames... -[2023-02-25 13:37:25,263][10910] Decorrelating experience for 64 frames... -[2023-02-25 13:37:25,515][10913] Decorrelating experience for 96 frames... -[2023-02-25 13:37:25,597][10913] Stopping RolloutWorker_w6... -[2023-02-25 13:37:25,598][10913] Loop rollout_proc6_evt_loop terminating... -[2023-02-25 13:37:25,929][10911] Decorrelating experience for 64 frames... -[2023-02-25 13:37:25,966][10910] Decorrelating experience for 96 frames... -[2023-02-25 13:37:26,051][10910] Stopping RolloutWorker_w3... -[2023-02-25 13:37:26,052][10910] Loop rollout_proc3_evt_loop terminating... -[2023-02-25 13:37:26,458][10911] Decorrelating experience for 96 frames... -[2023-02-25 13:37:26,524][10911] Stopping RolloutWorker_w5... -[2023-02-25 13:37:26,524][10911] Loop rollout_proc5_evt_loop terminating... -[2023-02-25 13:37:49,127][00699] Environment doom_basic already registered, overwriting... -[2023-02-25 13:37:49,129][00699] Environment doom_two_colors_easy already registered, overwriting... -[2023-02-25 13:37:49,131][00699] Environment doom_two_colors_hard already registered, overwriting... -[2023-02-25 13:37:49,135][00699] Environment doom_dm already registered, overwriting... -[2023-02-25 13:37:49,138][00699] Environment doom_dwango5 already registered, overwriting... -[2023-02-25 13:37:49,139][00699] Environment doom_my_way_home_flat_actions already registered, overwriting... -[2023-02-25 13:37:49,140][00699] Environment doom_defend_the_center_flat_actions already registered, overwriting... -[2023-02-25 13:37:49,141][00699] Environment doom_my_way_home already registered, overwriting... -[2023-02-25 13:37:49,142][00699] Environment doom_deadly_corridor already registered, overwriting... -[2023-02-25 13:37:49,143][00699] Environment doom_defend_the_center already registered, overwriting... -[2023-02-25 13:37:49,145][00699] Environment doom_defend_the_line already registered, overwriting... -[2023-02-25 13:37:49,146][00699] Environment doom_health_gathering already registered, overwriting... -[2023-02-25 13:37:49,147][00699] Environment doom_health_gathering_supreme already registered, overwriting... -[2023-02-25 13:37:49,148][00699] Environment doom_battle already registered, overwriting... -[2023-02-25 13:37:49,149][00699] Environment doom_battle2 already registered, overwriting... -[2023-02-25 13:37:49,150][00699] Environment doom_duel_bots already registered, overwriting... -[2023-02-25 13:37:49,151][00699] Environment doom_deathmatch_bots already registered, overwriting... -[2023-02-25 13:37:49,153][00699] Environment doom_duel already registered, overwriting... -[2023-02-25 13:37:49,154][00699] Environment doom_deathmatch_full already registered, overwriting... -[2023-02-25 13:37:49,155][00699] Environment doom_benchmark already registered, overwriting... -[2023-02-25 13:37:49,156][00699] register_encoder_factory: -[2023-02-25 13:37:49,184][00699] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-25 13:37:49,190][00699] Experiment dir /content/train_dir/default_experiment already exists! -[2023-02-25 13:37:49,192][00699] Resuming existing experiment from /content/train_dir/default_experiment... -[2023-02-25 13:37:49,193][00699] Weights and Biases integration disabled -[2023-02-25 13:37:49,196][00699] Environment var CUDA_VISIBLE_DEVICES is 0 - -[2023-02-25 13:37:50,634][00699] Starting experiment with the following configuration: -help=False -algo=APPO -env=doom_health_gathering_supreme -experiment=default_experiment -train_dir=/content/train_dir -restart_behavior=resume -device=gpu -seed=None -num_policies=1 -async_rl=True -serial_mode=False -batched_sampling=False -num_batches_to_accumulate=2 -worker_num_splits=2 -policy_workers_per_policy=1 -max_policy_lag=1000 -num_workers=8 -num_envs_per_worker=4 -batch_size=1024 -num_batches_per_epoch=1 -num_epochs=1 -rollout=32 -recurrence=32 -shuffle_minibatches=False -gamma=0.99 -reward_scale=1.0 -reward_clip=1000.0 -value_bootstrap=False -normalize_returns=True -exploration_loss_coeff=0.001 -value_loss_coeff=0.5 -kl_loss_coeff=0.0 -exploration_loss=symmetric_kl -gae_lambda=0.95 -ppo_clip_ratio=0.1 -ppo_clip_value=0.2 -with_vtrace=False -vtrace_rho=1.0 -vtrace_c=1.0 -optimizer=adam -adam_eps=1e-06 -adam_beta1=0.9 -adam_beta2=0.999 -max_grad_norm=4.0 -learning_rate=0.0001 -lr_schedule=constant -lr_schedule_kl_threshold=0.008 -lr_adaptive_min=1e-06 -lr_adaptive_max=0.01 -obs_subtract_mean=0.0 -obs_scale=255.0 -normalize_input=True -normalize_input_keys=None -decorrelate_experience_max_seconds=0 -decorrelate_envs_on_one_worker=True -actor_worker_gpus=[] -set_workers_cpu_affinity=True -force_envs_single_thread=False -default_niceness=0 -log_to_file=True -experiment_summaries_interval=10 -flush_summaries_interval=30 -stats_avg=100 -summaries_use_frameskip=True -heartbeat_interval=20 -heartbeat_reporting_interval=600 -train_for_env_steps=4000000 -train_for_seconds=10000000000 -save_every_sec=120 -keep_checkpoints=2 -load_checkpoint_kind=latest -save_milestones_sec=-1 -save_best_every_sec=5 -save_best_metric=reward -save_best_after=100000 -benchmark=False -encoder_mlp_layers=[512, 512] -encoder_conv_architecture=convnet_simple -encoder_conv_mlp_layers=[512] -use_rnn=True -rnn_size=512 -rnn_type=gru -rnn_num_layers=1 -decoder_mlp_layers=[] -nonlinearity=elu -policy_initialization=orthogonal -policy_init_gain=1.0 -actor_critic_share_weights=True -adaptive_stddev=True -continuous_tanh_scale=0.0 -initial_stddev=1.0 -use_env_info_cache=False -env_gpu_actions=False -env_gpu_observations=True -env_frameskip=4 -env_framestack=1 -pixel_format=CHW -use_record_episode_statistics=False -with_wandb=False -wandb_user=None -wandb_project=sample_factory -wandb_group=None -wandb_job_type=SF -wandb_tags=[] -with_pbt=False -pbt_mix_policies_in_one_env=True -pbt_period_env_steps=5000000 -pbt_start_mutation=20000000 -pbt_replace_fraction=0.3 -pbt_mutation_rate=0.15 -pbt_replace_reward_gap=0.1 -pbt_replace_reward_gap_absolute=1e-06 -pbt_optimize_gamma=False -pbt_target_objective=true_objective -pbt_perturb_min=1.1 -pbt_perturb_max=1.5 -num_agents=-1 -num_humans=0 -num_bots=-1 -start_bot_difficulty=None -timelimit=None -res_w=128 -res_h=72 -wide_aspect_ratio=False -eval_env_frameskip=1 -fps=35 -command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 -cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} -git_hash=unknown -git_repo_name=not a git repository -[2023-02-25 13:37:50,636][00699] Saving configuration to /content/train_dir/default_experiment/config.json... -[2023-02-25 13:37:50,643][00699] Rollout worker 0 uses device cpu -[2023-02-25 13:37:50,644][00699] Rollout worker 1 uses device cpu -[2023-02-25 13:37:50,647][00699] Rollout worker 2 uses device cpu -[2023-02-25 13:37:50,649][00699] Rollout worker 3 uses device cpu -[2023-02-25 13:37:50,653][00699] Rollout worker 4 uses device cpu -[2023-02-25 13:37:50,658][00699] Rollout worker 5 uses device cpu -[2023-02-25 13:37:50,659][00699] Rollout worker 6 uses device cpu -[2023-02-25 13:37:50,663][00699] Rollout worker 7 uses device cpu -[2023-02-25 13:37:50,791][00699] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:37:50,792][00699] InferenceWorker_p0-w0: min num requests: 2 -[2023-02-25 13:37:50,826][00699] Starting all processes... -[2023-02-25 13:37:50,827][00699] Starting process learner_proc0 -[2023-02-25 13:37:50,924][00699] Starting all processes... -[2023-02-25 13:37:50,934][00699] Starting process inference_proc0-0 -[2023-02-25 13:37:50,934][00699] Starting process rollout_proc0 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc1 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc2 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc3 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc4 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc5 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc6 -[2023-02-25 13:37:50,939][00699] Starting process rollout_proc7 -[2023-02-25 13:38:02,761][12789] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:38:02,761][12789] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-02-25 13:38:02,816][12789] Num visible devices: 1 -[2023-02-25 13:38:02,863][12789] Starting seed is not provided -[2023-02-25 13:38:02,864][12789] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:38:02,864][12789] Initializing actor-critic model on device cuda:0 -[2023-02-25 13:38:02,865][12789] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:38:02,866][12789] RunningMeanStd input shape: (1,) -[2023-02-25 13:38:02,945][12789] ConvEncoder: input_channels=3 -[2023-02-25 13:38:03,118][12808] Worker 2 uses CPU cores [0] -[2023-02-25 13:38:03,401][12803] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:38:03,402][12803] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-02-25 13:38:03,459][12803] Num visible devices: 1 -[2023-02-25 13:38:03,661][12805] Worker 0 uses CPU cores [0] -[2023-02-25 13:38:03,678][12789] Conv encoder output size: 512 -[2023-02-25 13:38:03,678][12789] Policy head output size: 512 -[2023-02-25 13:38:03,680][12804] Worker 1 uses CPU cores [1] -[2023-02-25 13:38:03,782][12789] Created Actor Critic model with architecture: -[2023-02-25 13:38:03,788][12789] ActorCriticSharedWeights( - (obs_normalizer): ObservationNormalizer( - (running_mean_std): RunningMeanStdDictInPlace( - (running_mean_std): ModuleDict( - (obs): RunningMeanStdInPlace() - ) - ) - ) - (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) - (encoder): VizdoomEncoder( - (basic_encoder): ConvEncoder( - (enc): RecursiveScriptModule( - original_name=ConvEncoderImpl - (conv_head): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Conv2d) - (1): RecursiveScriptModule(original_name=ELU) - (2): RecursiveScriptModule(original_name=Conv2d) - (3): RecursiveScriptModule(original_name=ELU) - (4): RecursiveScriptModule(original_name=Conv2d) - (5): RecursiveScriptModule(original_name=ELU) - ) - (mlp_layers): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Linear) - (1): RecursiveScriptModule(original_name=ELU) - ) - ) - ) - ) - (core): ModelCoreRNN( - (core): GRU(512, 512) - ) - (decoder): MlpDecoder( - (mlp): Identity() - ) - (critic_linear): Linear(in_features=512, out_features=1, bias=True) - (action_parameterization): ActionParameterizationDefault( - (distribution_linear): Linear(in_features=512, out_features=5, bias=True) - ) -) -[2023-02-25 13:38:04,128][12809] Worker 4 uses CPU cores [0] -[2023-02-25 13:38:04,180][12813] Worker 3 uses CPU cores [1] -[2023-02-25 13:38:04,182][12822] Worker 7 uses CPU cores [1] -[2023-02-25 13:38:04,211][12819] Worker 5 uses CPU cores [1] -[2023-02-25 13:38:04,243][12814] Worker 6 uses CPU cores [0] -[2023-02-25 13:38:06,384][12789] Using optimizer -[2023-02-25 13:38:06,385][12789] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... -[2023-02-25 13:38:06,399][12789] Loading model from checkpoint -[2023-02-25 13:38:06,400][12789] Loaded experiment state at self.train_step=0, self.env_steps=0 -[2023-02-25 13:38:06,401][12789] Initialized policy 0 weights for model version 0 -[2023-02-25 13:38:06,403][12789] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-25 13:38:06,414][12789] LearnerWorker_p0 finished initialization! -[2023-02-25 13:38:06,528][12803] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:38:06,530][12803] RunningMeanStd input shape: (1,) -[2023-02-25 13:38:06,549][12803] ConvEncoder: input_channels=3 -[2023-02-25 13:38:06,657][12803] Conv encoder output size: 512 -[2023-02-25 13:38:06,657][12803] Policy head output size: 512 -[2023-02-25 13:38:09,041][00699] Inference worker 0-0 is ready! -[2023-02-25 13:38:09,043][00699] All inference workers are ready! Signal rollout workers to start! -[2023-02-25 13:38:09,155][12819] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,169][12822] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,168][12804] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,166][12813] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,196][00699] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-25 13:38:09,211][12808] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,220][12814] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,223][12809] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,231][12805] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-25 13:38:09,665][12808] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,341][12808] Decorrelating experience for 32 frames... -[2023-02-25 13:38:10,368][12814] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,631][12819] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,641][12813] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,639][12804] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,646][12822] Decorrelating experience for 0 frames... -[2023-02-25 13:38:10,783][00699] Heartbeat connected on Batcher_0 -[2023-02-25 13:38:10,789][00699] Heartbeat connected on LearnerWorker_p0 -[2023-02-25 13:38:10,827][00699] Heartbeat connected on InferenceWorker_p0-w0 -[2023-02-25 13:38:10,966][12808] Decorrelating experience for 64 frames... -[2023-02-25 13:38:11,731][12819] Decorrelating experience for 32 frames... -[2023-02-25 13:38:11,747][12813] Decorrelating experience for 32 frames... -[2023-02-25 13:38:11,745][12822] Decorrelating experience for 32 frames... -[2023-02-25 13:38:11,774][12805] Decorrelating experience for 0 frames... -[2023-02-25 13:38:11,778][12809] Decorrelating experience for 0 frames... -[2023-02-25 13:38:12,181][12808] Decorrelating experience for 96 frames... -[2023-02-25 13:38:12,375][00699] Heartbeat connected on RolloutWorker_w2 -[2023-02-25 13:38:12,915][12814] Decorrelating experience for 32 frames... -[2023-02-25 13:38:12,938][12805] Decorrelating experience for 32 frames... -[2023-02-25 13:38:13,102][12804] Decorrelating experience for 32 frames... -[2023-02-25 13:38:13,276][12809] Decorrelating experience for 32 frames... -[2023-02-25 13:38:13,696][12805] Decorrelating experience for 64 frames... -[2023-02-25 13:38:13,771][12813] Decorrelating experience for 64 frames... -[2023-02-25 13:38:14,196][00699] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-25 13:38:14,266][12809] Decorrelating experience for 64 frames... -[2023-02-25 13:38:14,973][12814] Decorrelating experience for 64 frames... -[2023-02-25 13:38:15,029][12809] Decorrelating experience for 96 frames... -[2023-02-25 13:38:15,172][00699] Heartbeat connected on RolloutWorker_w4 -[2023-02-25 13:38:15,531][12822] Decorrelating experience for 64 frames... -[2023-02-25 13:38:16,136][12804] Decorrelating experience for 64 frames... -[2023-02-25 13:38:16,653][12814] Decorrelating experience for 96 frames... -[2023-02-25 13:38:16,733][00699] Heartbeat connected on RolloutWorker_w6 -[2023-02-25 13:38:17,407][12819] Decorrelating experience for 64 frames... -[2023-02-25 13:38:17,940][12805] Decorrelating experience for 96 frames... -[2023-02-25 13:38:18,032][00699] Heartbeat connected on RolloutWorker_w0 -[2023-02-25 13:38:18,531][12822] Decorrelating experience for 96 frames... -[2023-02-25 13:38:18,964][12813] Decorrelating experience for 96 frames... -[2023-02-25 13:38:19,024][00699] Heartbeat connected on RolloutWorker_w7 -[2023-02-25 13:38:19,142][12804] Decorrelating experience for 96 frames... -[2023-02-25 13:38:19,196][00699] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 2.6. Samples: 26. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-25 13:38:19,318][00699] Heartbeat connected on RolloutWorker_w3 -[2023-02-25 13:38:19,633][00699] Heartbeat connected on RolloutWorker_w1 -[2023-02-25 13:38:20,921][12819] Decorrelating experience for 96 frames... -[2023-02-25 13:38:21,595][00699] Heartbeat connected on RolloutWorker_w5 -[2023-02-25 13:38:23,189][12789] Signal inference workers to stop experience collection... -[2023-02-25 13:38:23,196][12803] InferenceWorker_p0-w0: stopping experience collection -[2023-02-25 13:38:24,196][00699] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 177.5. Samples: 2662. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-25 13:38:24,202][00699] Avg episode reward: [(0, '2.199')] -[2023-02-25 13:38:25,924][12789] Signal inference workers to resume experience collection... -[2023-02-25 13:38:25,925][12803] InferenceWorker_p0-w0: resuming experience collection -[2023-02-25 13:38:29,199][00699] Fps is (10 sec: 1638.0, 60 sec: 819.1, 300 sec: 819.1). Total num frames: 16384. Throughput: 0: 179.2. Samples: 3584. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) -[2023-02-25 13:38:29,203][00699] Avg episode reward: [(0, '3.435')] -[2023-02-25 13:38:34,197][00699] Fps is (10 sec: 3276.7, 60 sec: 1310.7, 300 sec: 1310.7). Total num frames: 32768. Throughput: 0: 362.8. Samples: 9070. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:38:34,204][00699] Avg episode reward: [(0, '3.939')] -[2023-02-25 13:38:35,652][12803] Updated weights for policy 0, policy_version 10 (0.0018) -[2023-02-25 13:38:39,200][00699] Fps is (10 sec: 3276.2, 60 sec: 1638.2, 300 sec: 1638.2). Total num frames: 49152. Throughput: 0: 445.7. Samples: 13374. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:38:39,211][00699] Avg episode reward: [(0, '4.401')] -[2023-02-25 13:38:44,196][00699] Fps is (10 sec: 3276.9, 60 sec: 1872.5, 300 sec: 1872.5). Total num frames: 65536. Throughput: 0: 443.0. Samples: 15504. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:38:44,200][00699] Avg episode reward: [(0, '4.297')] -[2023-02-25 13:38:47,260][12803] Updated weights for policy 0, policy_version 20 (0.0013) -[2023-02-25 13:38:49,196][00699] Fps is (10 sec: 4097.6, 60 sec: 2252.8, 300 sec: 2252.8). Total num frames: 90112. Throughput: 0: 551.7. Samples: 22066. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:38:49,204][00699] Avg episode reward: [(0, '4.240')] -[2023-02-25 13:38:54,196][00699] Fps is (10 sec: 4505.6, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 110592. Throughput: 0: 624.9. Samples: 28122. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:38:54,201][00699] Avg episode reward: [(0, '4.268')] -[2023-02-25 13:38:54,212][12789] Saving new best policy, reward=4.268! -[2023-02-25 13:38:59,197][00699] Fps is (10 sec: 2867.2, 60 sec: 2375.7, 300 sec: 2375.7). Total num frames: 118784. Throughput: 0: 668.4. Samples: 30078. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:38:59,206][00699] Avg episode reward: [(0, '4.382')] -[2023-02-25 13:38:59,265][12789] Saving new best policy, reward=4.382! -[2023-02-25 13:38:59,287][12803] Updated weights for policy 0, policy_version 30 (0.0026) -[2023-02-25 13:39:04,196][00699] Fps is (10 sec: 2048.0, 60 sec: 2383.1, 300 sec: 2383.1). Total num frames: 131072. Throughput: 0: 739.6. Samples: 33310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:39:04,204][00699] Avg episode reward: [(0, '4.457')] -[2023-02-25 13:39:04,206][12789] Saving new best policy, reward=4.457! -[2023-02-25 13:39:09,197][00699] Fps is (10 sec: 2457.6, 60 sec: 2389.3, 300 sec: 2389.3). Total num frames: 143360. Throughput: 0: 761.1. Samples: 36912. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:39:09,203][00699] Avg episode reward: [(0, '4.486')] -[2023-02-25 13:39:09,216][12789] Saving new best policy, reward=4.486! -[2023-02-25 13:39:13,707][12803] Updated weights for policy 0, policy_version 40 (0.0038) -[2023-02-25 13:39:14,196][00699] Fps is (10 sec: 3276.8, 60 sec: 2730.7, 300 sec: 2520.6). Total num frames: 163840. Throughput: 0: 801.7. Samples: 39660. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:39:14,202][00699] Avg episode reward: [(0, '4.470')] -[2023-02-25 13:39:19,197][00699] Fps is (10 sec: 4096.1, 60 sec: 3072.0, 300 sec: 2633.1). Total num frames: 184320. Throughput: 0: 812.4. Samples: 45626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:39:19,202][00699] Avg episode reward: [(0, '4.309')] -[2023-02-25 13:39:24,199][00699] Fps is (10 sec: 3276.1, 60 sec: 3276.7, 300 sec: 2621.4). Total num frames: 196608. Throughput: 0: 812.0. Samples: 49914. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:39:24,202][00699] Avg episode reward: [(0, '4.356')] -[2023-02-25 13:39:26,654][12803] Updated weights for policy 0, policy_version 50 (0.0022) -[2023-02-25 13:39:29,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3276.9, 300 sec: 2662.4). Total num frames: 212992. Throughput: 0: 810.8. Samples: 51992. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:39:29,199][00699] Avg episode reward: [(0, '4.447')] -[2023-02-25 13:39:34,196][00699] Fps is (10 sec: 4096.9, 60 sec: 3413.3, 300 sec: 2794.9). Total num frames: 237568. Throughput: 0: 813.2. Samples: 58658. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:39:34,204][00699] Avg episode reward: [(0, '4.415')] -[2023-02-25 13:39:36,075][12803] Updated weights for policy 0, policy_version 60 (0.0018) -[2023-02-25 13:39:39,198][00699] Fps is (10 sec: 4095.5, 60 sec: 3413.5, 300 sec: 2821.6). Total num frames: 253952. Throughput: 0: 807.1. Samples: 64442. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:39:39,206][00699] Avg episode reward: [(0, '4.347')] -[2023-02-25 13:39:44,199][00699] Fps is (10 sec: 2866.5, 60 sec: 3344.9, 300 sec: 2802.5). Total num frames: 266240. Throughput: 0: 808.7. Samples: 66470. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:39:44,206][00699] Avg episode reward: [(0, '4.475')] -[2023-02-25 13:39:49,198][00699] Fps is (10 sec: 2867.2, 60 sec: 3208.5, 300 sec: 2826.2). Total num frames: 282624. Throughput: 0: 825.4. Samples: 70452. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:39:49,205][00699] Avg episode reward: [(0, '4.382')] -[2023-02-25 13:39:49,218][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth... -[2023-02-25 13:39:49,889][12803] Updated weights for policy 0, policy_version 70 (0.0017) -[2023-02-25 13:39:54,196][00699] Fps is (10 sec: 3687.2, 60 sec: 3208.5, 300 sec: 2886.7). Total num frames: 303104. Throughput: 0: 882.8. Samples: 76640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:39:54,203][00699] Avg episode reward: [(0, '4.416')] -[2023-02-25 13:39:59,196][00699] Fps is (10 sec: 4096.5, 60 sec: 3413.3, 300 sec: 2941.7). Total num frames: 323584. Throughput: 0: 890.4. Samples: 79726. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:39:59,199][00699] Avg episode reward: [(0, '4.471')] -[2023-02-25 13:40:00,583][12803] Updated weights for policy 0, policy_version 80 (0.0013) -[2023-02-25 13:40:04,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 2920.6). Total num frames: 335872. Throughput: 0: 854.7. Samples: 84088. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:40:04,208][00699] Avg episode reward: [(0, '4.442')] -[2023-02-25 13:40:09,197][00699] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 2901.3). Total num frames: 348160. Throughput: 0: 852.1. Samples: 88258. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:40:09,203][00699] Avg episode reward: [(0, '4.489')] -[2023-02-25 13:40:09,218][12789] Saving new best policy, reward=4.489! -[2023-02-25 13:40:13,388][12803] Updated weights for policy 0, policy_version 90 (0.0026) -[2023-02-25 13:40:14,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 2949.1). Total num frames: 368640. Throughput: 0: 872.2. Samples: 91242. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:40:14,199][00699] Avg episode reward: [(0, '4.461')] -[2023-02-25 13:40:19,199][00699] Fps is (10 sec: 4095.0, 60 sec: 3413.2, 300 sec: 2993.2). Total num frames: 389120. Throughput: 0: 863.8. Samples: 97530. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:40:19,207][00699] Avg episode reward: [(0, '4.482')] -[2023-02-25 13:40:24,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.5, 300 sec: 2973.4). Total num frames: 401408. Throughput: 0: 831.8. Samples: 101874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:40:24,203][00699] Avg episode reward: [(0, '4.534')] -[2023-02-25 13:40:24,267][12789] Saving new best policy, reward=4.534! -[2023-02-25 13:40:25,830][12803] Updated weights for policy 0, policy_version 100 (0.0014) -[2023-02-25 13:40:29,196][00699] Fps is (10 sec: 2868.0, 60 sec: 3413.3, 300 sec: 2984.2). Total num frames: 417792. Throughput: 0: 828.8. Samples: 103766. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:40:29,203][00699] Avg episode reward: [(0, '4.524')] -[2023-02-25 13:40:34,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3022.6). Total num frames: 438272. Throughput: 0: 865.7. Samples: 109408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:40:34,203][00699] Avg episode reward: [(0, '4.426')] -[2023-02-25 13:40:36,606][12803] Updated weights for policy 0, policy_version 110 (0.0028) -[2023-02-25 13:40:39,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3413.4, 300 sec: 3058.3). Total num frames: 458752. Throughput: 0: 873.7. Samples: 115956. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:40:39,199][00699] Avg episode reward: [(0, '4.391')] -[2023-02-25 13:40:44,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3481.7, 300 sec: 3065.4). Total num frames: 475136. Throughput: 0: 852.6. Samples: 118092. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:40:44,199][00699] Avg episode reward: [(0, '4.386')] -[2023-02-25 13:40:49,197][00699] Fps is (10 sec: 2867.1, 60 sec: 3413.4, 300 sec: 3046.4). Total num frames: 487424. Throughput: 0: 847.5. Samples: 122224. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:40:49,201][00699] Avg episode reward: [(0, '4.395')] -[2023-02-25 13:40:49,737][12803] Updated weights for policy 0, policy_version 120 (0.0034) -[2023-02-25 13:40:54,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3078.2). Total num frames: 507904. Throughput: 0: 887.2. Samples: 128182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:40:54,204][00699] Avg episode reward: [(0, '4.368')] -[2023-02-25 13:40:59,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3413.3, 300 sec: 3108.1). Total num frames: 528384. Throughput: 0: 893.0. Samples: 131428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:40:59,198][00699] Avg episode reward: [(0, '4.583')] -[2023-02-25 13:40:59,214][12789] Saving new best policy, reward=4.583! -[2023-02-25 13:40:59,218][12803] Updated weights for policy 0, policy_version 130 (0.0026) -[2023-02-25 13:41:04,198][00699] Fps is (10 sec: 3685.8, 60 sec: 3481.5, 300 sec: 3112.9). Total num frames: 544768. Throughput: 0: 864.6. Samples: 136438. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:41:04,206][00699] Avg episode reward: [(0, '4.595')] -[2023-02-25 13:41:04,208][12789] Saving new best policy, reward=4.595! -[2023-02-25 13:41:09,197][00699] Fps is (10 sec: 2867.0, 60 sec: 3481.6, 300 sec: 3094.7). Total num frames: 557056. Throughput: 0: 857.6. Samples: 140466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:41:09,205][00699] Avg episode reward: [(0, '4.544')] -[2023-02-25 13:41:12,959][12803] Updated weights for policy 0, policy_version 140 (0.0013) -[2023-02-25 13:41:14,196][00699] Fps is (10 sec: 3277.3, 60 sec: 3481.6, 300 sec: 3121.8). Total num frames: 577536. Throughput: 0: 871.3. Samples: 142976. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:41:14,202][00699] Avg episode reward: [(0, '4.354')] -[2023-02-25 13:41:19,197][00699] Fps is (10 sec: 4096.2, 60 sec: 3481.7, 300 sec: 3147.4). Total num frames: 598016. Throughput: 0: 892.4. Samples: 149564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:41:19,202][00699] Avg episode reward: [(0, '4.352')] -[2023-02-25 13:41:23,498][12803] Updated weights for policy 0, policy_version 150 (0.0012) -[2023-02-25 13:41:24,214][00699] Fps is (10 sec: 3679.9, 60 sec: 3548.8, 300 sec: 3150.5). Total num frames: 614400. Throughput: 0: 858.0. Samples: 154580. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-25 13:41:24,217][00699] Avg episode reward: [(0, '4.365')] -[2023-02-25 13:41:29,197][00699] Fps is (10 sec: 2867.0, 60 sec: 3481.5, 300 sec: 3133.4). Total num frames: 626688. Throughput: 0: 855.9. Samples: 156610. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-25 13:41:29,204][00699] Avg episode reward: [(0, '4.388')] -[2023-02-25 13:41:34,196][00699] Fps is (10 sec: 3282.5, 60 sec: 3481.6, 300 sec: 3156.9). Total num frames: 647168. Throughput: 0: 872.9. Samples: 161504. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:41:34,199][00699] Avg episode reward: [(0, '4.522')] -[2023-02-25 13:41:35,799][12803] Updated weights for policy 0, policy_version 160 (0.0020) -[2023-02-25 13:41:39,196][00699] Fps is (10 sec: 4096.4, 60 sec: 3481.6, 300 sec: 3179.3). Total num frames: 667648. Throughput: 0: 886.0. Samples: 168054. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:41:39,199][00699] Avg episode reward: [(0, '4.532')] -[2023-02-25 13:41:44,197][00699] Fps is (10 sec: 3686.1, 60 sec: 3481.6, 300 sec: 3181.5). Total num frames: 684032. Throughput: 0: 873.9. Samples: 170756. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:41:44,200][00699] Avg episode reward: [(0, '4.544')] -[2023-02-25 13:41:47,745][12803] Updated weights for policy 0, policy_version 170 (0.0024) -[2023-02-25 13:41:49,197][00699] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3165.1). Total num frames: 696320. Throughput: 0: 852.1. Samples: 174782. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:41:49,203][00699] Avg episode reward: [(0, '4.440')] -[2023-02-25 13:41:49,219][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000170_696320.pth... -[2023-02-25 13:41:49,435][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth -[2023-02-25 13:41:54,196][00699] Fps is (10 sec: 3277.1, 60 sec: 3481.6, 300 sec: 3185.8). Total num frames: 716800. Throughput: 0: 872.1. Samples: 179708. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:41:54,200][00699] Avg episode reward: [(0, '4.546')] -[2023-02-25 13:41:59,104][12803] Updated weights for policy 0, policy_version 180 (0.0024) -[2023-02-25 13:41:59,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3481.6, 300 sec: 3205.6). Total num frames: 737280. Throughput: 0: 883.7. Samples: 182742. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:41:59,199][00699] Avg episode reward: [(0, '4.559')] -[2023-02-25 13:42:04,197][00699] Fps is (10 sec: 3686.3, 60 sec: 3481.7, 300 sec: 3207.1). Total num frames: 753664. Throughput: 0: 862.5. Samples: 188378. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:42:04,201][00699] Avg episode reward: [(0, '4.434')] -[2023-02-25 13:42:09,201][00699] Fps is (10 sec: 2866.0, 60 sec: 3481.4, 300 sec: 3191.4). Total num frames: 765952. Throughput: 0: 834.1. Samples: 192102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:42:09,210][00699] Avg episode reward: [(0, '4.536')] -[2023-02-25 13:42:14,202][00699] Fps is (10 sec: 2047.0, 60 sec: 3276.5, 300 sec: 3159.7). Total num frames: 774144. Throughput: 0: 822.5. Samples: 193626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:42:14,207][00699] Avg episode reward: [(0, '4.519')] -[2023-02-25 13:42:14,734][12803] Updated weights for policy 0, policy_version 190 (0.0020) -[2023-02-25 13:42:19,196][00699] Fps is (10 sec: 2048.8, 60 sec: 3140.3, 300 sec: 3145.7). Total num frames: 786432. Throughput: 0: 793.6. Samples: 197218. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:42:19,205][00699] Avg episode reward: [(0, '4.542')] -[2023-02-25 13:42:24,196][00699] Fps is (10 sec: 3278.5, 60 sec: 3209.5, 300 sec: 3164.4). Total num frames: 806912. Throughput: 0: 771.0. Samples: 202748. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:42:24,203][00699] Avg episode reward: [(0, '4.444')] -[2023-02-25 13:42:27,963][12803] Updated weights for policy 0, policy_version 200 (0.0015) -[2023-02-25 13:42:29,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3208.6, 300 sec: 3150.8). Total num frames: 819200. Throughput: 0: 757.3. Samples: 204834. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:42:29,201][00699] Avg episode reward: [(0, '4.356')] -[2023-02-25 13:42:34,197][00699] Fps is (10 sec: 2867.2, 60 sec: 3140.3, 300 sec: 3153.1). Total num frames: 835584. Throughput: 0: 757.6. Samples: 208872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:42:34,206][00699] Avg episode reward: [(0, '4.367')] -[2023-02-25 13:42:39,197][00699] Fps is (10 sec: 3686.4, 60 sec: 3140.3, 300 sec: 3170.6). Total num frames: 856064. Throughput: 0: 779.8. Samples: 214798. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:42:39,199][00699] Avg episode reward: [(0, '4.418')] -[2023-02-25 13:42:39,787][12803] Updated weights for policy 0, policy_version 210 (0.0012) -[2023-02-25 13:42:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3208.6, 300 sec: 3187.4). Total num frames: 876544. Throughput: 0: 784.2. Samples: 218032. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:42:44,199][00699] Avg episode reward: [(0, '4.571')] -[2023-02-25 13:42:49,198][00699] Fps is (10 sec: 3685.7, 60 sec: 3276.7, 300 sec: 3189.0). Total num frames: 892928. Throughput: 0: 770.9. Samples: 223068. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:42:49,203][00699] Avg episode reward: [(0, '4.552')] -[2023-02-25 13:42:52,323][12803] Updated weights for policy 0, policy_version 220 (0.0023) -[2023-02-25 13:42:54,197][00699] Fps is (10 sec: 2867.1, 60 sec: 3140.3, 300 sec: 3176.2). Total num frames: 905216. Throughput: 0: 776.3. Samples: 227032. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:42:54,205][00699] Avg episode reward: [(0, '4.517')] -[2023-02-25 13:42:59,197][00699] Fps is (10 sec: 3277.4, 60 sec: 3140.3, 300 sec: 3192.1). Total num frames: 925696. Throughput: 0: 806.2. Samples: 229900. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:42:59,205][00699] Avg episode reward: [(0, '4.448')] -[2023-02-25 13:43:02,756][12803] Updated weights for policy 0, policy_version 230 (0.0021) -[2023-02-25 13:43:04,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3208.6, 300 sec: 3207.4). Total num frames: 946176. Throughput: 0: 871.6. Samples: 236440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:43:04,207][00699] Avg episode reward: [(0, '4.605')] -[2023-02-25 13:43:04,209][12789] Saving new best policy, reward=4.605! -[2023-02-25 13:43:09,203][00699] Fps is (10 sec: 3684.1, 60 sec: 3276.7, 300 sec: 3262.8). Total num frames: 962560. Throughput: 0: 856.7. Samples: 241306. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:43:09,206][00699] Avg episode reward: [(0, '4.533')] -[2023-02-25 13:43:14,199][00699] Fps is (10 sec: 2866.5, 60 sec: 3345.2, 300 sec: 3304.5). Total num frames: 974848. Throughput: 0: 854.8. Samples: 243302. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:43:14,202][00699] Avg episode reward: [(0, '4.522')] -[2023-02-25 13:43:16,165][12803] Updated weights for policy 0, policy_version 240 (0.0012) -[2023-02-25 13:43:19,196][00699] Fps is (10 sec: 3278.9, 60 sec: 3481.6, 300 sec: 3374.0). Total num frames: 995328. Throughput: 0: 879.1. Samples: 248430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:43:19,204][00699] Avg episode reward: [(0, '4.599')] -[2023-02-25 13:43:24,196][00699] Fps is (10 sec: 4096.9, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 1015808. Throughput: 0: 900.3. Samples: 255310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:43:24,204][00699] Avg episode reward: [(0, '4.764')] -[2023-02-25 13:43:24,290][12789] Saving new best policy, reward=4.764! -[2023-02-25 13:43:25,257][12803] Updated weights for policy 0, policy_version 250 (0.0029) -[2023-02-25 13:43:29,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3387.9). Total num frames: 1032192. Throughput: 0: 888.2. Samples: 258002. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:43:29,202][00699] Avg episode reward: [(0, '4.763')] -[2023-02-25 13:43:34,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3387.9). Total num frames: 1048576. Throughput: 0: 871.7. Samples: 262294. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-25 13:43:34,207][00699] Avg episode reward: [(0, '4.736')] -[2023-02-25 13:43:37,992][12803] Updated weights for policy 0, policy_version 260 (0.0028) -[2023-02-25 13:43:39,197][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 1069056. Throughput: 0: 906.2. Samples: 267812. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:43:39,204][00699] Avg episode reward: [(0, '4.703')] -[2023-02-25 13:43:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3387.9). Total num frames: 1089536. Throughput: 0: 915.5. Samples: 271096. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:43:44,205][00699] Avg episode reward: [(0, '4.586')] -[2023-02-25 13:43:47,983][12803] Updated weights for policy 0, policy_version 270 (0.0018) -[2023-02-25 13:43:49,197][00699] Fps is (10 sec: 3686.3, 60 sec: 3550.0, 300 sec: 3374.0). Total num frames: 1105920. Throughput: 0: 901.1. Samples: 276990. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:43:49,205][00699] Avg episode reward: [(0, '4.837')] -[2023-02-25 13:43:49,220][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000270_1105920.pth... -[2023-02-25 13:43:49,401][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth -[2023-02-25 13:43:49,434][12789] Saving new best policy, reward=4.837! -[2023-02-25 13:43:54,197][00699] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3401.8). Total num frames: 1122304. Throughput: 0: 883.9. Samples: 281074. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-25 13:43:54,199][00699] Avg episode reward: [(0, '4.964')] -[2023-02-25 13:43:54,210][12789] Saving new best policy, reward=4.964! -[2023-02-25 13:43:59,197][00699] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1138688. Throughput: 0: 890.2. Samples: 283360. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-25 13:43:59,199][00699] Avg episode reward: [(0, '4.907')] -[2023-02-25 13:44:00,560][12803] Updated weights for policy 0, policy_version 280 (0.0019) -[2023-02-25 13:44:04,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 1159168. Throughput: 0: 922.9. Samples: 289962. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-25 13:44:04,202][00699] Avg episode reward: [(0, '4.968')] -[2023-02-25 13:44:04,207][12789] Saving new best policy, reward=4.968! -[2023-02-25 13:44:09,200][00699] Fps is (10 sec: 4094.4, 60 sec: 3618.3, 300 sec: 3443.4). Total num frames: 1179648. Throughput: 0: 891.6. Samples: 295436. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:44:09,212][00699] Avg episode reward: [(0, '5.087')] -[2023-02-25 13:44:09,228][12789] Saving new best policy, reward=5.087! -[2023-02-25 13:44:11,973][12803] Updated weights for policy 0, policy_version 290 (0.0014) -[2023-02-25 13:44:14,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3415.6). Total num frames: 1191936. Throughput: 0: 876.0. Samples: 297422. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:44:14,201][00699] Avg episode reward: [(0, '5.121')] -[2023-02-25 13:44:14,204][12789] Saving new best policy, reward=5.121! -[2023-02-25 13:44:19,196][00699] Fps is (10 sec: 2868.4, 60 sec: 3549.9, 300 sec: 3429.6). Total num frames: 1208320. Throughput: 0: 878.6. Samples: 301832. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:44:19,199][00699] Avg episode reward: [(0, '5.006')] -[2023-02-25 13:44:23,475][12803] Updated weights for policy 0, policy_version 300 (0.0022) -[2023-02-25 13:44:24,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 1228800. Throughput: 0: 902.5. Samples: 308424. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:44:24,204][00699] Avg episode reward: [(0, '5.221')] -[2023-02-25 13:44:24,207][12789] Saving new best policy, reward=5.221! -[2023-02-25 13:44:29,197][00699] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3429.5). Total num frames: 1249280. Throughput: 0: 902.2. Samples: 311696. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:44:29,202][00699] Avg episode reward: [(0, '5.346')] -[2023-02-25 13:44:29,214][12789] Saving new best policy, reward=5.346! -[2023-02-25 13:44:34,197][00699] Fps is (10 sec: 3276.6, 60 sec: 3549.8, 300 sec: 3415.7). Total num frames: 1261568. Throughput: 0: 861.8. Samples: 315770. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:44:34,204][00699] Avg episode reward: [(0, '5.321')] -[2023-02-25 13:44:36,311][12803] Updated weights for policy 0, policy_version 310 (0.0012) -[2023-02-25 13:44:39,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3429.6). Total num frames: 1277952. Throughput: 0: 875.7. Samples: 320480. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:44:39,205][00699] Avg episode reward: [(0, '5.228')] -[2023-02-25 13:44:44,201][00699] Fps is (10 sec: 3685.1, 60 sec: 3481.4, 300 sec: 3443.4). Total num frames: 1298432. Throughput: 0: 898.0. Samples: 323774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:44:44,204][00699] Avg episode reward: [(0, '4.965')] -[2023-02-25 13:44:46,138][12803] Updated weights for policy 0, policy_version 320 (0.0014) -[2023-02-25 13:44:49,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 1318912. Throughput: 0: 894.5. Samples: 330214. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:44:49,204][00699] Avg episode reward: [(0, '4.892')] -[2023-02-25 13:44:54,196][00699] Fps is (10 sec: 3688.0, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 1335296. Throughput: 0: 865.2. Samples: 334366. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:44:54,199][00699] Avg episode reward: [(0, '5.219')] -[2023-02-25 13:44:59,197][00699] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 1347584. Throughput: 0: 866.7. Samples: 336422. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:44:59,200][00699] Avg episode reward: [(0, '5.157')] -[2023-02-25 13:44:59,565][12803] Updated weights for policy 0, policy_version 330 (0.0027) -[2023-02-25 13:45:04,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 1368064. Throughput: 0: 900.4. Samples: 342350. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:45:04,204][00699] Avg episode reward: [(0, '5.232')] -[2023-02-25 13:45:09,148][12803] Updated weights for policy 0, policy_version 340 (0.0020) -[2023-02-25 13:45:09,196][00699] Fps is (10 sec: 4505.7, 60 sec: 3550.1, 300 sec: 3471.2). Total num frames: 1392640. Throughput: 0: 894.3. Samples: 348668. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:45:09,201][00699] Avg episode reward: [(0, '5.322')] -[2023-02-25 13:45:14,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 1404928. Throughput: 0: 867.0. Samples: 350712. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:45:14,206][00699] Avg episode reward: [(0, '5.523')] -[2023-02-25 13:45:14,211][12789] Saving new best policy, reward=5.523! -[2023-02-25 13:45:19,196][00699] Fps is (10 sec: 2457.6, 60 sec: 3481.6, 300 sec: 3443.4). Total num frames: 1417216. Throughput: 0: 865.8. Samples: 354732. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:45:19,201][00699] Avg episode reward: [(0, '5.400')] -[2023-02-25 13:45:22,991][12803] Updated weights for policy 0, policy_version 350 (0.0028) -[2023-02-25 13:45:24,198][00699] Fps is (10 sec: 2866.7, 60 sec: 3413.2, 300 sec: 3443.4). Total num frames: 1433600. Throughput: 0: 872.1. Samples: 359726. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:45:24,201][00699] Avg episode reward: [(0, '5.494')] -[2023-02-25 13:45:29,197][00699] Fps is (10 sec: 3276.7, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 1449984. Throughput: 0: 845.0. Samples: 361798. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-25 13:45:29,201][00699] Avg episode reward: [(0, '5.438')] -[2023-02-25 13:45:34,197][00699] Fps is (10 sec: 2867.7, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 1462272. Throughput: 0: 786.4. Samples: 365600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:45:34,201][00699] Avg episode reward: [(0, '5.156')] -[2023-02-25 13:45:38,162][12803] Updated weights for policy 0, policy_version 360 (0.0027) -[2023-02-25 13:45:39,197][00699] Fps is (10 sec: 2457.6, 60 sec: 3276.8, 300 sec: 3387.9). Total num frames: 1474560. Throughput: 0: 789.2. Samples: 369882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:45:39,201][00699] Avg episode reward: [(0, '5.260')] -[2023-02-25 13:45:44,196][00699] Fps is (10 sec: 3276.9, 60 sec: 3277.0, 300 sec: 3415.7). Total num frames: 1495040. Throughput: 0: 805.2. Samples: 372654. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:45:44,198][00699] Avg episode reward: [(0, '5.495')] -[2023-02-25 13:45:48,350][12803] Updated weights for policy 0, policy_version 370 (0.0024) -[2023-02-25 13:45:49,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 1515520. Throughput: 0: 821.2. Samples: 379306. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-25 13:45:49,204][00699] Avg episode reward: [(0, '5.517')] -[2023-02-25 13:45:49,217][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000370_1515520.pth... -[2023-02-25 13:45:49,385][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000170_696320.pth -[2023-02-25 13:45:54,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3401.8). Total num frames: 1531904. Throughput: 0: 790.0. Samples: 384220. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:45:54,202][00699] Avg episode reward: [(0, '5.389')] -[2023-02-25 13:45:59,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3387.9). Total num frames: 1544192. Throughput: 0: 790.9. Samples: 386304. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:45:59,204][00699] Avg episode reward: [(0, '5.361')] -[2023-02-25 13:46:01,652][12803] Updated weights for policy 0, policy_version 380 (0.0019) -[2023-02-25 13:46:04,197][00699] Fps is (10 sec: 3276.7, 60 sec: 3276.8, 300 sec: 3415.7). Total num frames: 1564672. Throughput: 0: 811.3. Samples: 391240. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:46:04,206][00699] Avg episode reward: [(0, '5.372')] -[2023-02-25 13:46:09,199][00699] Fps is (10 sec: 4095.1, 60 sec: 3208.4, 300 sec: 3415.6). Total num frames: 1585152. Throughput: 0: 845.1. Samples: 397758. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:46:09,205][00699] Avg episode reward: [(0, '5.069')] -[2023-02-25 13:46:11,502][12803] Updated weights for policy 0, policy_version 390 (0.0012) -[2023-02-25 13:46:14,196][00699] Fps is (10 sec: 3686.5, 60 sec: 3276.8, 300 sec: 3401.8). Total num frames: 1601536. Throughput: 0: 860.2. Samples: 400506. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:46:14,199][00699] Avg episode reward: [(0, '5.066')] -[2023-02-25 13:46:19,199][00699] Fps is (10 sec: 3276.7, 60 sec: 3344.9, 300 sec: 3401.9). Total num frames: 1617920. Throughput: 0: 867.3. Samples: 404632. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:46:19,202][00699] Avg episode reward: [(0, '5.093')] -[2023-02-25 13:46:24,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3345.2, 300 sec: 3415.7). Total num frames: 1634304. Throughput: 0: 885.7. Samples: 409738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:46:24,199][00699] Avg episode reward: [(0, '5.133')] -[2023-02-25 13:46:24,714][12803] Updated weights for policy 0, policy_version 400 (0.0026) -[2023-02-25 13:46:29,196][00699] Fps is (10 sec: 3687.4, 60 sec: 3413.4, 300 sec: 3415.6). Total num frames: 1654784. Throughput: 0: 894.7. Samples: 412916. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:46:29,199][00699] Avg episode reward: [(0, '5.210')] -[2023-02-25 13:46:34,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1675264. Throughput: 0: 879.4. Samples: 418880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:46:34,199][00699] Avg episode reward: [(0, '5.206')] -[2023-02-25 13:46:35,208][12803] Updated weights for policy 0, policy_version 410 (0.0018) -[2023-02-25 13:46:39,203][00699] Fps is (10 sec: 3274.8, 60 sec: 3549.5, 300 sec: 3401.7). Total num frames: 1687552. Throughput: 0: 863.1. Samples: 423066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:46:39,220][00699] Avg episode reward: [(0, '5.233')] -[2023-02-25 13:46:44,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 1703936. Throughput: 0: 863.1. Samples: 425142. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:46:44,204][00699] Avg episode reward: [(0, '4.942')] -[2023-02-25 13:46:47,222][12803] Updated weights for policy 0, policy_version 420 (0.0018) -[2023-02-25 13:46:49,196][00699] Fps is (10 sec: 3688.7, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 1724416. Throughput: 0: 893.7. Samples: 431456. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:46:49,204][00699] Avg episode reward: [(0, '5.188')] -[2023-02-25 13:46:54,197][00699] Fps is (10 sec: 4095.9, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1744896. Throughput: 0: 872.4. Samples: 437014. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:46:54,200][00699] Avg episode reward: [(0, '5.318')] -[2023-02-25 13:46:59,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 1757184. Throughput: 0: 857.0. Samples: 439070. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:46:59,200][00699] Avg episode reward: [(0, '5.614')] -[2023-02-25 13:46:59,215][12789] Saving new best policy, reward=5.614! -[2023-02-25 13:46:59,725][12803] Updated weights for policy 0, policy_version 430 (0.0038) -[2023-02-25 13:47:04,196][00699] Fps is (10 sec: 2867.3, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 1773568. Throughput: 0: 858.9. Samples: 443282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:47:04,201][00699] Avg episode reward: [(0, '5.488')] -[2023-02-25 13:47:09,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3481.7, 300 sec: 3457.4). Total num frames: 1794048. Throughput: 0: 891.8. Samples: 449868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:47:09,199][00699] Avg episode reward: [(0, '5.686')] -[2023-02-25 13:47:09,208][12789] Saving new best policy, reward=5.686! -[2023-02-25 13:47:10,549][12803] Updated weights for policy 0, policy_version 440 (0.0022) -[2023-02-25 13:47:14,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 1814528. Throughput: 0: 886.3. Samples: 452800. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:47:14,200][00699] Avg episode reward: [(0, '5.653')] -[2023-02-25 13:47:19,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3481.8, 300 sec: 3457.3). Total num frames: 1826816. Throughput: 0: 849.6. Samples: 457114. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:47:19,202][00699] Avg episode reward: [(0, '5.435')] -[2023-02-25 13:47:23,981][12803] Updated weights for policy 0, policy_version 450 (0.0036) -[2023-02-25 13:47:24,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 1843200. Throughput: 0: 852.7. Samples: 461430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:47:24,205][00699] Avg episode reward: [(0, '5.554')] -[2023-02-25 13:47:29,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 1863680. Throughput: 0: 877.7. Samples: 464638. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:47:29,203][00699] Avg episode reward: [(0, '5.407')] -[2023-02-25 13:47:33,456][12803] Updated weights for policy 0, policy_version 460 (0.0023) -[2023-02-25 13:47:34,198][00699] Fps is (10 sec: 4095.5, 60 sec: 3481.5, 300 sec: 3485.1). Total num frames: 1884160. Throughput: 0: 882.4. Samples: 471164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:47:34,206][00699] Avg episode reward: [(0, '5.426')] -[2023-02-25 13:47:39,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3482.0, 300 sec: 3457.3). Total num frames: 1896448. Throughput: 0: 852.9. Samples: 475396. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:47:39,201][00699] Avg episode reward: [(0, '5.364')] -[2023-02-25 13:47:44,196][00699] Fps is (10 sec: 2867.6, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 1912832. Throughput: 0: 852.0. Samples: 477412. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:47:44,204][00699] Avg episode reward: [(0, '5.431')] -[2023-02-25 13:47:46,683][12803] Updated weights for policy 0, policy_version 470 (0.0019) -[2023-02-25 13:47:49,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 1933312. Throughput: 0: 890.3. Samples: 483344. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:47:49,205][00699] Avg episode reward: [(0, '5.595')] -[2023-02-25 13:47:49,219][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000472_1933312.pth... -[2023-02-25 13:47:49,371][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000270_1105920.pth -[2023-02-25 13:47:54,201][00699] Fps is (10 sec: 4094.2, 60 sec: 3481.4, 300 sec: 3485.0). Total num frames: 1953792. Throughput: 0: 884.1. Samples: 489654. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:47:54,207][00699] Avg episode reward: [(0, '5.797')] -[2023-02-25 13:47:54,212][12789] Saving new best policy, reward=5.797! -[2023-02-25 13:47:57,676][12803] Updated weights for policy 0, policy_version 480 (0.0015) -[2023-02-25 13:47:59,202][00699] Fps is (10 sec: 3684.5, 60 sec: 3549.6, 300 sec: 3471.1). Total num frames: 1970176. Throughput: 0: 864.4. Samples: 491702. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:47:59,205][00699] Avg episode reward: [(0, '5.744')] -[2023-02-25 13:48:04,201][00699] Fps is (10 sec: 2867.0, 60 sec: 3481.3, 300 sec: 3457.3). Total num frames: 1982464. Throughput: 0: 863.2. Samples: 495960. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:04,204][00699] Avg episode reward: [(0, '5.671')] -[2023-02-25 13:48:09,196][00699] Fps is (10 sec: 3278.5, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 2002944. Throughput: 0: 905.2. Samples: 502166. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:48:09,205][00699] Avg episode reward: [(0, '5.494')] -[2023-02-25 13:48:09,258][12803] Updated weights for policy 0, policy_version 490 (0.0016) -[2023-02-25 13:48:14,199][00699] Fps is (10 sec: 4506.7, 60 sec: 3549.7, 300 sec: 3498.9). Total num frames: 2027520. Throughput: 0: 906.2. Samples: 505418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:14,207][00699] Avg episode reward: [(0, '5.479')] -[2023-02-25 13:48:19,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 2039808. Throughput: 0: 873.8. Samples: 510486. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:48:19,201][00699] Avg episode reward: [(0, '5.613')] -[2023-02-25 13:48:21,142][12803] Updated weights for policy 0, policy_version 500 (0.0012) -[2023-02-25 13:48:24,198][00699] Fps is (10 sec: 2867.5, 60 sec: 3549.8, 300 sec: 3471.2). Total num frames: 2056192. Throughput: 0: 872.7. Samples: 514668. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:24,203][00699] Avg episode reward: [(0, '5.812')] -[2023-02-25 13:48:24,207][12789] Saving new best policy, reward=5.812! -[2023-02-25 13:48:29,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2076672. Throughput: 0: 894.7. Samples: 517672. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:29,205][00699] Avg episode reward: [(0, '5.924')] -[2023-02-25 13:48:29,217][12789] Saving new best policy, reward=5.924! -[2023-02-25 13:48:31,972][12803] Updated weights for policy 0, policy_version 510 (0.0021) -[2023-02-25 13:48:34,196][00699] Fps is (10 sec: 4096.5, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2097152. Throughput: 0: 903.6. Samples: 524006. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:48:34,199][00699] Avg episode reward: [(0, '6.142')] -[2023-02-25 13:48:34,202][12789] Saving new best policy, reward=6.142! -[2023-02-25 13:48:39,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 2109440. Throughput: 0: 850.1. Samples: 527906. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:39,199][00699] Avg episode reward: [(0, '5.880')] -[2023-02-25 13:48:44,199][00699] Fps is (10 sec: 2047.5, 60 sec: 3413.2, 300 sec: 3429.5). Total num frames: 2117632. Throughput: 0: 841.5. Samples: 529566. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:48:44,201][00699] Avg episode reward: [(0, '6.047')] -[2023-02-25 13:48:48,142][12803] Updated weights for policy 0, policy_version 520 (0.0018) -[2023-02-25 13:48:49,197][00699] Fps is (10 sec: 2047.9, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 2129920. Throughput: 0: 822.7. Samples: 532980. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:48:49,200][00699] Avg episode reward: [(0, '6.281')] -[2023-02-25 13:48:49,217][12789] Saving new best policy, reward=6.281! -[2023-02-25 13:48:54,201][00699] Fps is (10 sec: 3276.2, 60 sec: 3276.8, 300 sec: 3429.5). Total num frames: 2150400. Throughput: 0: 804.8. Samples: 538386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:48:54,209][00699] Avg episode reward: [(0, '6.131')] -[2023-02-25 13:48:58,539][12803] Updated weights for policy 0, policy_version 530 (0.0026) -[2023-02-25 13:48:59,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3345.4, 300 sec: 3429.5). Total num frames: 2170880. Throughput: 0: 805.4. Samples: 541658. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:48:59,206][00699] Avg episode reward: [(0, '6.068')] -[2023-02-25 13:49:04,202][00699] Fps is (10 sec: 3686.0, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 2187264. Throughput: 0: 814.7. Samples: 547152. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-25 13:49:04,205][00699] Avg episode reward: [(0, '6.140')] -[2023-02-25 13:49:09,197][00699] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 2199552. Throughput: 0: 813.0. Samples: 551254. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:49:09,199][00699] Avg episode reward: [(0, '6.317')] -[2023-02-25 13:49:09,253][12789] Saving new best policy, reward=6.317! -[2023-02-25 13:49:11,967][12803] Updated weights for policy 0, policy_version 540 (0.0031) -[2023-02-25 13:49:14,199][00699] Fps is (10 sec: 3277.8, 60 sec: 3208.5, 300 sec: 3429.5). Total num frames: 2220032. Throughput: 0: 798.5. Samples: 553608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:49:14,207][00699] Avg episode reward: [(0, '6.686')] -[2023-02-25 13:49:14,210][12789] Saving new best policy, reward=6.686! -[2023-02-25 13:49:19,203][00699] Fps is (10 sec: 4093.3, 60 sec: 3344.7, 300 sec: 3429.5). Total num frames: 2240512. Throughput: 0: 800.2. Samples: 560022. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:49:19,206][00699] Avg episode reward: [(0, '6.619')] -[2023-02-25 13:49:21,678][12803] Updated weights for policy 0, policy_version 550 (0.0019) -[2023-02-25 13:49:24,196][00699] Fps is (10 sec: 3687.2, 60 sec: 3345.1, 300 sec: 3415.6). Total num frames: 2256896. Throughput: 0: 832.6. Samples: 565372. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:49:24,199][00699] Avg episode reward: [(0, '7.067')] -[2023-02-25 13:49:24,207][12789] Saving new best policy, reward=7.067! -[2023-02-25 13:49:29,202][00699] Fps is (10 sec: 3277.2, 60 sec: 3276.5, 300 sec: 3429.5). Total num frames: 2273280. Throughput: 0: 841.9. Samples: 567454. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:49:29,209][00699] Avg episode reward: [(0, '6.617')] -[2023-02-25 13:49:34,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3208.5, 300 sec: 3429.5). Total num frames: 2289664. Throughput: 0: 871.7. Samples: 572208. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:49:34,204][00699] Avg episode reward: [(0, '6.969')] -[2023-02-25 13:49:34,714][12803] Updated weights for policy 0, policy_version 560 (0.0026) -[2023-02-25 13:49:39,196][00699] Fps is (10 sec: 3688.3, 60 sec: 3345.1, 300 sec: 3429.6). Total num frames: 2310144. Throughput: 0: 900.7. Samples: 578912. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:49:39,208][00699] Avg episode reward: [(0, '6.914')] -[2023-02-25 13:49:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3550.0, 300 sec: 3429.5). Total num frames: 2330624. Throughput: 0: 895.4. Samples: 581950. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:49:44,200][00699] Avg episode reward: [(0, '7.068')] -[2023-02-25 13:49:45,320][12803] Updated weights for policy 0, policy_version 570 (0.0024) -[2023-02-25 13:49:49,197][00699] Fps is (10 sec: 3276.6, 60 sec: 3549.8, 300 sec: 3415.6). Total num frames: 2342912. Throughput: 0: 866.2. Samples: 586128. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:49:49,199][00699] Avg episode reward: [(0, '6.805')] -[2023-02-25 13:49:49,218][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000572_2342912.pth... -[2023-02-25 13:49:49,443][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000370_1515520.pth -[2023-02-25 13:49:54,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.8, 300 sec: 3429.5). Total num frames: 2359296. Throughput: 0: 884.8. Samples: 591070. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:49:54,202][00699] Avg episode reward: [(0, '6.399')] -[2023-02-25 13:49:57,255][12803] Updated weights for policy 0, policy_version 580 (0.0024) -[2023-02-25 13:49:59,196][00699] Fps is (10 sec: 4096.3, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 2383872. Throughput: 0: 905.0. Samples: 594332. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:49:59,203][00699] Avg episode reward: [(0, '6.943')] -[2023-02-25 13:50:04,197][00699] Fps is (10 sec: 4095.9, 60 sec: 3550.2, 300 sec: 3415.6). Total num frames: 2400256. Throughput: 0: 894.7. Samples: 600278. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:50:04,204][00699] Avg episode reward: [(0, '7.075')] -[2023-02-25 13:50:04,208][12789] Saving new best policy, reward=7.075! -[2023-02-25 13:50:09,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 2412544. Throughput: 0: 865.2. Samples: 604306. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:50:09,204][00699] Avg episode reward: [(0, '7.393')] -[2023-02-25 13:50:09,216][12789] Saving new best policy, reward=7.393! -[2023-02-25 13:50:09,742][12803] Updated weights for policy 0, policy_version 590 (0.0022) -[2023-02-25 13:50:14,197][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.7, 300 sec: 3429.5). Total num frames: 2428928. Throughput: 0: 866.6. Samples: 606448. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:50:14,199][00699] Avg episode reward: [(0, '7.025')] -[2023-02-25 13:50:19,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3550.3, 300 sec: 3457.3). Total num frames: 2453504. Throughput: 0: 907.4. Samples: 613042. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:50:19,205][00699] Avg episode reward: [(0, '7.622')] -[2023-02-25 13:50:19,215][12789] Saving new best policy, reward=7.622! -[2023-02-25 13:50:19,867][12803] Updated weights for policy 0, policy_version 600 (0.0030) -[2023-02-25 13:50:24,196][00699] Fps is (10 sec: 4505.7, 60 sec: 3618.1, 300 sec: 3471.2). Total num frames: 2473984. Throughput: 0: 890.0. Samples: 618964. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:50:24,199][00699] Avg episode reward: [(0, '8.359')] -[2023-02-25 13:50:24,202][12789] Saving new best policy, reward=8.359! -[2023-02-25 13:50:29,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3550.2, 300 sec: 3471.2). Total num frames: 2486272. Throughput: 0: 868.7. Samples: 621042. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:50:29,203][00699] Avg episode reward: [(0, '8.629')] -[2023-02-25 13:50:29,215][12789] Saving new best policy, reward=8.629! -[2023-02-25 13:50:33,115][12803] Updated weights for policy 0, policy_version 610 (0.0020) -[2023-02-25 13:50:34,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2502656. Throughput: 0: 868.2. Samples: 625196. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:50:34,204][00699] Avg episode reward: [(0, '8.166')] -[2023-02-25 13:50:39,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2523136. Throughput: 0: 907.3. Samples: 631900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:50:39,206][00699] Avg episode reward: [(0, '7.996')] -[2023-02-25 13:50:42,278][12803] Updated weights for policy 0, policy_version 620 (0.0014) -[2023-02-25 13:50:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2543616. Throughput: 0: 909.4. Samples: 635256. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:50:44,203][00699] Avg episode reward: [(0, '7.996')] -[2023-02-25 13:50:49,197][00699] Fps is (10 sec: 3686.3, 60 sec: 3618.2, 300 sec: 3485.1). Total num frames: 2560000. Throughput: 0: 880.7. Samples: 639908. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:50:49,204][00699] Avg episode reward: [(0, '8.128')] -[2023-02-25 13:50:54,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 2572288. Throughput: 0: 889.6. Samples: 644340. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:50:54,199][00699] Avg episode reward: [(0, '8.162')] -[2023-02-25 13:50:55,295][12803] Updated weights for policy 0, policy_version 630 (0.0021) -[2023-02-25 13:50:59,196][00699] Fps is (10 sec: 3686.5, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 2596864. Throughput: 0: 915.8. Samples: 647660. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:50:59,199][00699] Avg episode reward: [(0, '7.966')] -[2023-02-25 13:51:04,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 2617344. Throughput: 0: 917.9. Samples: 654348. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:51:04,201][00699] Avg episode reward: [(0, '7.839')] -[2023-02-25 13:51:04,934][12803] Updated weights for policy 0, policy_version 640 (0.0026) -[2023-02-25 13:51:09,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3485.1). Total num frames: 2629632. Throughput: 0: 885.6. Samples: 658818. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:51:09,202][00699] Avg episode reward: [(0, '7.454')] -[2023-02-25 13:51:14,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3485.1). Total num frames: 2646016. Throughput: 0: 885.6. Samples: 660896. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:51:14,205][00699] Avg episode reward: [(0, '7.534')] -[2023-02-25 13:51:17,509][12803] Updated weights for policy 0, policy_version 650 (0.0018) -[2023-02-25 13:51:19,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 2666496. Throughput: 0: 926.8. Samples: 666902. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:51:19,201][00699] Avg episode reward: [(0, '7.714')] -[2023-02-25 13:51:24,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3512.8). Total num frames: 2691072. Throughput: 0: 924.7. Samples: 673510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:51:24,201][00699] Avg episode reward: [(0, '8.175')] -[2023-02-25 13:51:27,976][12803] Updated weights for policy 0, policy_version 660 (0.0020) -[2023-02-25 13:51:29,199][00699] Fps is (10 sec: 3685.6, 60 sec: 3618.0, 300 sec: 3485.0). Total num frames: 2703360. Throughput: 0: 899.2. Samples: 675724. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:51:29,201][00699] Avg episode reward: [(0, '7.850')] -[2023-02-25 13:51:34,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 2719744. Throughput: 0: 894.9. Samples: 680180. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:51:34,199][00699] Avg episode reward: [(0, '7.599')] -[2023-02-25 13:51:39,196][00699] Fps is (10 sec: 3687.2, 60 sec: 3618.1, 300 sec: 3512.8). Total num frames: 2740224. Throughput: 0: 934.8. Samples: 686408. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:51:39,204][00699] Avg episode reward: [(0, '7.200')] -[2023-02-25 13:51:39,323][12803] Updated weights for policy 0, policy_version 670 (0.0028) -[2023-02-25 13:51:44,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3526.7). Total num frames: 2764800. Throughput: 0: 937.0. Samples: 689826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:51:44,203][00699] Avg episode reward: [(0, '7.977')] -[2023-02-25 13:51:49,197][00699] Fps is (10 sec: 3686.3, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 2777088. Throughput: 0: 901.8. Samples: 694930. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:51:49,201][00699] Avg episode reward: [(0, '8.604')] -[2023-02-25 13:51:49,213][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000678_2777088.pth... -[2023-02-25 13:51:49,363][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000472_1933312.pth -[2023-02-25 13:51:51,049][12803] Updated weights for policy 0, policy_version 680 (0.0012) -[2023-02-25 13:51:54,196][00699] Fps is (10 sec: 2457.6, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 2789376. Throughput: 0: 886.7. Samples: 698718. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-25 13:51:54,203][00699] Avg episode reward: [(0, '8.738')] -[2023-02-25 13:51:54,209][12789] Saving new best policy, reward=8.738! -[2023-02-25 13:51:59,196][00699] Fps is (10 sec: 2457.7, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 2801664. Throughput: 0: 881.5. Samples: 700562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:51:59,203][00699] Avg episode reward: [(0, '9.336')] -[2023-02-25 13:51:59,215][12789] Saving new best policy, reward=9.336! -[2023-02-25 13:52:04,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3471.2). Total num frames: 2818048. Throughput: 0: 842.8. Samples: 704830. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:52:04,204][00699] Avg episode reward: [(0, '9.087')] -[2023-02-25 13:52:05,183][12803] Updated weights for policy 0, policy_version 690 (0.0017) -[2023-02-25 13:52:09,197][00699] Fps is (10 sec: 3686.3, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 2838528. Throughput: 0: 817.7. Samples: 710308. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:52:09,199][00699] Avg episode reward: [(0, '8.521')] -[2023-02-25 13:52:14,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 2850816. Throughput: 0: 815.0. Samples: 712396. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:52:14,201][00699] Avg episode reward: [(0, '8.555')] -[2023-02-25 13:52:18,156][12803] Updated weights for policy 0, policy_version 700 (0.0013) -[2023-02-25 13:52:19,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 2871296. Throughput: 0: 826.8. Samples: 717384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:52:19,202][00699] Avg episode reward: [(0, '8.433')] -[2023-02-25 13:52:24,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3485.1). Total num frames: 2891776. Throughput: 0: 839.5. Samples: 724184. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:52:24,201][00699] Avg episode reward: [(0, '7.871')] -[2023-02-25 13:52:27,496][12803] Updated weights for policy 0, policy_version 710 (0.0022) -[2023-02-25 13:52:29,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3481.7, 300 sec: 3485.1). Total num frames: 2912256. Throughput: 0: 833.7. Samples: 727342. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:52:29,200][00699] Avg episode reward: [(0, '7.437')] -[2023-02-25 13:52:34,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 2924544. Throughput: 0: 816.9. Samples: 731692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:52:34,198][00699] Avg episode reward: [(0, '8.225')] -[2023-02-25 13:52:39,197][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 2945024. Throughput: 0: 850.5. Samples: 736992. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:52:39,199][00699] Avg episode reward: [(0, '8.567')] -[2023-02-25 13:52:39,830][12803] Updated weights for policy 0, policy_version 720 (0.0036) -[2023-02-25 13:52:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3499.0). Total num frames: 2965504. Throughput: 0: 883.9. Samples: 740338. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:52:44,198][00699] Avg episode reward: [(0, '9.372')] -[2023-02-25 13:52:44,210][12789] Saving new best policy, reward=9.372! -[2023-02-25 13:52:49,197][00699] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3499.0). Total num frames: 2985984. Throughput: 0: 926.0. Samples: 746498. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:52:49,199][00699] Avg episode reward: [(0, '9.216')] -[2023-02-25 13:52:50,089][12803] Updated weights for policy 0, policy_version 730 (0.0012) -[2023-02-25 13:52:54,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 2998272. Throughput: 0: 899.1. Samples: 750768. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:52:54,200][00699] Avg episode reward: [(0, '9.030')] -[2023-02-25 13:52:59,197][00699] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3512.9). Total num frames: 3018752. Throughput: 0: 901.1. Samples: 752944. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:52:59,200][00699] Avg episode reward: [(0, '8.767')] -[2023-02-25 13:53:01,859][12803] Updated weights for policy 0, policy_version 740 (0.0013) -[2023-02-25 13:53:04,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3512.8). Total num frames: 3039232. Throughput: 0: 940.8. Samples: 759718. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:53:04,202][00699] Avg episode reward: [(0, '9.064')] -[2023-02-25 13:53:09,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3499.0). Total num frames: 3059712. Throughput: 0: 917.3. Samples: 765462. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:53:09,199][00699] Avg episode reward: [(0, '9.870')] -[2023-02-25 13:53:09,212][12789] Saving new best policy, reward=9.870! -[2023-02-25 13:53:13,528][12803] Updated weights for policy 0, policy_version 750 (0.0024) -[2023-02-25 13:53:14,199][00699] Fps is (10 sec: 3276.1, 60 sec: 3686.3, 300 sec: 3498.9). Total num frames: 3072000. Throughput: 0: 892.8. Samples: 767520. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:53:14,209][00699] Avg episode reward: [(0, '10.192')] -[2023-02-25 13:53:14,212][12789] Saving new best policy, reward=10.192! -[2023-02-25 13:53:19,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 3088384. Throughput: 0: 896.3. Samples: 772026. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:53:19,199][00699] Avg episode reward: [(0, '9.140')] -[2023-02-25 13:53:24,024][12803] Updated weights for policy 0, policy_version 760 (0.0020) -[2023-02-25 13:53:24,196][00699] Fps is (10 sec: 4096.9, 60 sec: 3686.4, 300 sec: 3512.8). Total num frames: 3112960. Throughput: 0: 928.3. Samples: 778766. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:53:24,199][00699] Avg episode reward: [(0, '8.645')] -[2023-02-25 13:53:29,198][00699] Fps is (10 sec: 4095.5, 60 sec: 3618.1, 300 sec: 3498.9). Total num frames: 3129344. Throughput: 0: 929.6. Samples: 782170. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:53:29,202][00699] Avg episode reward: [(0, '8.455')] -[2023-02-25 13:53:34,199][00699] Fps is (10 sec: 3276.0, 60 sec: 3686.3, 300 sec: 3512.8). Total num frames: 3145728. Throughput: 0: 890.1. Samples: 786556. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:53:34,202][00699] Avg episode reward: [(0, '8.672')] -[2023-02-25 13:53:36,470][12803] Updated weights for policy 0, policy_version 770 (0.0016) -[2023-02-25 13:53:39,196][00699] Fps is (10 sec: 3277.2, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 3162112. Throughput: 0: 911.2. Samples: 791774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:53:39,205][00699] Avg episode reward: [(0, '9.079')] -[2023-02-25 13:53:44,196][00699] Fps is (10 sec: 4097.0, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 3186688. Throughput: 0: 939.8. Samples: 795236. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:53:44,200][00699] Avg episode reward: [(0, '9.456')] -[2023-02-25 13:53:45,527][12803] Updated weights for policy 0, policy_version 780 (0.0012) -[2023-02-25 13:53:49,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 3207168. Throughput: 0: 936.8. Samples: 801874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:53:49,201][00699] Avg episode reward: [(0, '10.128')] -[2023-02-25 13:53:49,211][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000783_3207168.pth... -[2023-02-25 13:53:49,383][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000572_2342912.pth -[2023-02-25 13:53:54,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3219456. Throughput: 0: 903.7. Samples: 806128. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:53:54,199][00699] Avg episode reward: [(0, '9.263')] -[2023-02-25 13:53:58,387][12803] Updated weights for policy 0, policy_version 790 (0.0012) -[2023-02-25 13:53:59,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3554.6). Total num frames: 3235840. Throughput: 0: 905.2. Samples: 808250. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:53:59,200][00699] Avg episode reward: [(0, '9.499')] -[2023-02-25 13:54:04,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 3260416. Throughput: 0: 954.9. Samples: 814996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:54:04,205][00699] Avg episode reward: [(0, '8.998')] -[2023-02-25 13:54:07,258][12803] Updated weights for policy 0, policy_version 800 (0.0019) -[2023-02-25 13:54:09,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3596.2). Total num frames: 3280896. Throughput: 0: 943.8. Samples: 821238. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:54:09,201][00699] Avg episode reward: [(0, '9.859')] -[2023-02-25 13:54:14,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3754.8, 300 sec: 3582.3). Total num frames: 3297280. Throughput: 0: 916.1. Samples: 823392. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:54:14,204][00699] Avg episode reward: [(0, '10.271')] -[2023-02-25 13:54:14,211][12789] Saving new best policy, reward=10.271! -[2023-02-25 13:54:19,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3582.3). Total num frames: 3313664. Throughput: 0: 918.4. Samples: 827882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:54:19,204][00699] Avg episode reward: [(0, '11.178')] -[2023-02-25 13:54:19,213][12789] Saving new best policy, reward=11.178! -[2023-02-25 13:54:19,831][12803] Updated weights for policy 0, policy_version 810 (0.0015) -[2023-02-25 13:54:24,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3596.2). Total num frames: 3334144. Throughput: 0: 954.4. Samples: 834720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:54:24,203][00699] Avg episode reward: [(0, '12.009')] -[2023-02-25 13:54:24,207][12789] Saving new best policy, reward=12.009! -[2023-02-25 13:54:29,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 3354624. Throughput: 0: 949.5. Samples: 837964. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:54:29,207][00699] Avg episode reward: [(0, '11.789')] -[2023-02-25 13:54:29,807][12803] Updated weights for policy 0, policy_version 820 (0.0018) -[2023-02-25 13:54:34,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3754.8, 300 sec: 3596.2). Total num frames: 3371008. Throughput: 0: 905.8. Samples: 842636. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:54:34,201][00699] Avg episode reward: [(0, '11.614')] -[2023-02-25 13:54:39,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3582.3). Total num frames: 3387392. Throughput: 0: 921.5. Samples: 847596. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:54:39,199][00699] Avg episode reward: [(0, '11.846')] -[2023-02-25 13:54:41,674][12803] Updated weights for policy 0, policy_version 830 (0.0037) -[2023-02-25 13:54:44,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3623.9). Total num frames: 3411968. Throughput: 0: 950.7. Samples: 851030. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:54:44,199][00699] Avg episode reward: [(0, '12.023')] -[2023-02-25 13:54:44,204][12789] Saving new best policy, reward=12.023! -[2023-02-25 13:54:49,199][00699] Fps is (10 sec: 4504.6, 60 sec: 3754.5, 300 sec: 3637.8). Total num frames: 3432448. Throughput: 0: 951.0. Samples: 857792. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:54:49,201][00699] Avg episode reward: [(0, '13.010')] -[2023-02-25 13:54:49,223][12789] Saving new best policy, reward=13.010! -[2023-02-25 13:54:52,100][12803] Updated weights for policy 0, policy_version 840 (0.0013) -[2023-02-25 13:54:54,197][00699] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3596.1). Total num frames: 3444736. Throughput: 0: 907.1. Samples: 862056. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:54:54,201][00699] Avg episode reward: [(0, '13.990')] -[2023-02-25 13:54:54,205][12789] Saving new best policy, reward=13.990! -[2023-02-25 13:54:59,196][00699] Fps is (10 sec: 2867.8, 60 sec: 3754.7, 300 sec: 3596.1). Total num frames: 3461120. Throughput: 0: 906.9. Samples: 864204. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:54:59,199][00699] Avg episode reward: [(0, '13.417')] -[2023-02-25 13:55:03,378][12803] Updated weights for policy 0, policy_version 850 (0.0028) -[2023-02-25 13:55:04,196][00699] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 3481600. Throughput: 0: 952.8. Samples: 870758. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:55:04,203][00699] Avg episode reward: [(0, '13.524')] -[2023-02-25 13:55:09,197][00699] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 3506176. Throughput: 0: 948.9. Samples: 877420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:55:09,201][00699] Avg episode reward: [(0, '13.040')] -[2023-02-25 13:55:14,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 3518464. Throughput: 0: 917.6. Samples: 879254. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:55:14,206][00699] Avg episode reward: [(0, '13.379')] -[2023-02-25 13:55:15,292][12803] Updated weights for policy 0, policy_version 860 (0.0027) -[2023-02-25 13:55:19,196][00699] Fps is (10 sec: 2457.6, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 3530752. Throughput: 0: 889.9. Samples: 882682. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-25 13:55:19,199][00699] Avg episode reward: [(0, '13.830')] -[2023-02-25 13:55:24,197][00699] Fps is (10 sec: 2457.5, 60 sec: 3481.6, 300 sec: 3582.3). Total num frames: 3543040. Throughput: 0: 862.4. Samples: 886402. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:55:24,201][00699] Avg episode reward: [(0, '15.119')] -[2023-02-25 13:55:24,205][12789] Saving new best policy, reward=15.119! -[2023-02-25 13:55:28,773][12803] Updated weights for policy 0, policy_version 870 (0.0022) -[2023-02-25 13:55:29,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3596.1). Total num frames: 3563520. Throughput: 0: 858.4. Samples: 889656. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:55:29,203][00699] Avg episode reward: [(0, '16.336')] -[2023-02-25 13:55:29,214][12789] Saving new best policy, reward=16.336! -[2023-02-25 13:55:34,196][00699] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 3584000. Throughput: 0: 862.5. Samples: 896602. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2023-02-25 13:55:34,202][00699] Avg episode reward: [(0, '15.369')] -[2023-02-25 13:55:39,197][00699] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 3600384. Throughput: 0: 866.6. Samples: 901054. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:55:39,202][00699] Avg episode reward: [(0, '14.383')] -[2023-02-25 13:55:40,209][12803] Updated weights for policy 0, policy_version 880 (0.0022) -[2023-02-25 13:55:44,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3582.3). Total num frames: 3616768. Throughput: 0: 867.6. Samples: 903244. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:55:44,204][00699] Avg episode reward: [(0, '13.175')] -[2023-02-25 13:55:49,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3481.7, 300 sec: 3623.9). Total num frames: 3641344. Throughput: 0: 868.5. Samples: 909840. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:55:49,203][00699] Avg episode reward: [(0, '13.199')] -[2023-02-25 13:55:49,215][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000889_3641344.pth... -[2023-02-25 13:55:49,350][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000678_2777088.pth -[2023-02-25 13:55:50,082][12803] Updated weights for policy 0, policy_version 890 (0.0023) -[2023-02-25 13:55:54,196][00699] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 3661824. Throughput: 0: 863.9. Samples: 916294. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:55:54,208][00699] Avg episode reward: [(0, '13.931')] -[2023-02-25 13:55:59,196][00699] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 3674112. Throughput: 0: 869.7. Samples: 918390. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:55:59,203][00699] Avg episode reward: [(0, '14.026')] -[2023-02-25 13:56:02,241][12803] Updated weights for policy 0, policy_version 900 (0.0014) -[2023-02-25 13:56:04,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3596.1). Total num frames: 3690496. Throughput: 0: 891.6. Samples: 922806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:56:04,203][00699] Avg episode reward: [(0, '14.478')] -[2023-02-25 13:56:09,196][00699] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3623.9). Total num frames: 3715072. Throughput: 0: 959.4. Samples: 929576. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:56:09,203][00699] Avg episode reward: [(0, '15.262')] -[2023-02-25 13:56:11,623][12803] Updated weights for policy 0, policy_version 910 (0.0030) -[2023-02-25 13:56:14,200][00699] Fps is (10 sec: 4504.2, 60 sec: 3617.9, 300 sec: 3623.9). Total num frames: 3735552. Throughput: 0: 962.2. Samples: 932956. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:56:14,211][00699] Avg episode reward: [(0, '17.171')] -[2023-02-25 13:56:14,213][12789] Saving new best policy, reward=17.171! -[2023-02-25 13:56:19,202][00699] Fps is (10 sec: 3275.1, 60 sec: 3617.8, 300 sec: 3582.2). Total num frames: 3747840. Throughput: 0: 915.0. Samples: 937784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:56:19,210][00699] Avg episode reward: [(0, '18.018')] -[2023-02-25 13:56:19,219][12789] Saving new best policy, reward=18.018! -[2023-02-25 13:56:24,196][00699] Fps is (10 sec: 2868.1, 60 sec: 3686.4, 300 sec: 3596.2). Total num frames: 3764224. Throughput: 0: 913.3. Samples: 942154. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:56:24,203][00699] Avg episode reward: [(0, '19.681')] -[2023-02-25 13:56:24,206][12789] Saving new best policy, reward=19.681! -[2023-02-25 13:56:24,681][12803] Updated weights for policy 0, policy_version 920 (0.0013) -[2023-02-25 13:56:29,196][00699] Fps is (10 sec: 4098.2, 60 sec: 3754.7, 300 sec: 3623.9). Total num frames: 3788800. Throughput: 0: 941.6. Samples: 945614. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:56:29,198][00699] Avg episode reward: [(0, '18.967')] -[2023-02-25 13:56:33,342][12803] Updated weights for policy 0, policy_version 930 (0.0012) -[2023-02-25 13:56:34,196][00699] Fps is (10 sec: 4915.2, 60 sec: 3822.9, 300 sec: 3637.8). Total num frames: 3813376. Throughput: 0: 948.5. Samples: 952522. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-25 13:56:34,203][00699] Avg episode reward: [(0, '18.027')] -[2023-02-25 13:56:39,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3596.2). Total num frames: 3825664. Throughput: 0: 913.2. Samples: 957390. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-25 13:56:39,202][00699] Avg episode reward: [(0, '17.612')] -[2023-02-25 13:56:44,196][00699] Fps is (10 sec: 2457.6, 60 sec: 3686.4, 300 sec: 3596.2). Total num frames: 3837952. Throughput: 0: 913.7. Samples: 959508. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:56:44,199][00699] Avg episode reward: [(0, '18.667')] -[2023-02-25 13:56:46,003][12803] Updated weights for policy 0, policy_version 940 (0.0025) -[2023-02-25 13:56:49,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 3862528. Throughput: 0: 952.2. Samples: 965656. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2023-02-25 13:56:49,204][00699] Avg episode reward: [(0, '18.743')] -[2023-02-25 13:56:54,196][00699] Fps is (10 sec: 4915.2, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 3887104. Throughput: 0: 956.5. Samples: 972618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:56:54,204][00699] Avg episode reward: [(0, '20.027')] -[2023-02-25 13:56:54,210][12789] Saving new best policy, reward=20.027! -[2023-02-25 13:56:55,173][12803] Updated weights for policy 0, policy_version 950 (0.0019) -[2023-02-25 13:56:59,196][00699] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 3899392. Throughput: 0: 928.8. Samples: 974748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:56:59,204][00699] Avg episode reward: [(0, '20.815')] -[2023-02-25 13:56:59,215][12789] Saving new best policy, reward=20.815! -[2023-02-25 13:57:04,196][00699] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 3915776. Throughput: 0: 918.1. Samples: 979092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:57:04,200][00699] Avg episode reward: [(0, '20.811')] -[2023-02-25 13:57:07,565][12803] Updated weights for policy 0, policy_version 960 (0.0015) -[2023-02-25 13:57:09,197][00699] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 3936256. Throughput: 0: 960.0. Samples: 985352. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:57:09,204][00699] Avg episode reward: [(0, '20.494')] -[2023-02-25 13:57:14,200][00699] Fps is (10 sec: 4504.2, 60 sec: 3754.7, 300 sec: 3693.3). Total num frames: 3960832. Throughput: 0: 959.1. Samples: 988776. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-25 13:57:14,203][00699] Avg episode reward: [(0, '18.956')] -[2023-02-25 13:57:17,451][12803] Updated weights for policy 0, policy_version 970 (0.0011) -[2023-02-25 13:57:19,197][00699] Fps is (10 sec: 4096.0, 60 sec: 3823.3, 300 sec: 3679.5). Total num frames: 3977216. Throughput: 0: 929.5. Samples: 994348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-25 13:57:19,203][00699] Avg episode reward: [(0, '19.121')] -[2023-02-25 13:57:24,196][00699] Fps is (10 sec: 2868.1, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 3989504. Throughput: 0: 919.7. Samples: 998778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-25 13:57:24,201][00699] Avg episode reward: [(0, '18.999')] -[2023-02-25 13:57:27,520][12789] Stopping Batcher_0... -[2023-02-25 13:57:27,521][12789] Loop batcher_evt_loop terminating... -[2023-02-25 13:57:27,521][00699] Component Batcher_0 stopped! -[2023-02-25 13:57:27,532][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-25 13:57:27,583][12809] Stopping RolloutWorker_w4... -[2023-02-25 13:57:27,586][12808] Stopping RolloutWorker_w2... -[2023-02-25 13:57:27,586][12808] Loop rollout_proc2_evt_loop terminating... -[2023-02-25 13:57:27,583][00699] Component RolloutWorker_w4 stopped! -[2023-02-25 13:57:27,589][00699] Component RolloutWorker_w2 stopped! -[2023-02-25 13:57:27,597][12814] Stopping RolloutWorker_w6... -[2023-02-25 13:57:27,597][00699] Component RolloutWorker_w6 stopped! -[2023-02-25 13:57:27,583][12809] Loop rollout_proc4_evt_loop terminating... -[2023-02-25 13:57:27,603][12803] Weights refcount: 2 0 -[2023-02-25 13:57:27,600][12814] Loop rollout_proc6_evt_loop terminating... -[2023-02-25 13:57:27,609][00699] Component InferenceWorker_p0-w0 stopped! -[2023-02-25 13:57:27,609][12803] Stopping InferenceWorker_p0-w0... -[2023-02-25 13:57:27,612][12803] Loop inference_proc0-0_evt_loop terminating... -[2023-02-25 13:57:27,627][12805] Stopping RolloutWorker_w0... -[2023-02-25 13:57:27,631][12805] Loop rollout_proc0_evt_loop terminating... -[2023-02-25 13:57:27,626][00699] Component RolloutWorker_w1 stopped! -[2023-02-25 13:57:27,634][00699] Component RolloutWorker_w0 stopped! -[2023-02-25 13:57:27,626][12804] Stopping RolloutWorker_w1... -[2023-02-25 13:57:27,640][12804] Loop rollout_proc1_evt_loop terminating... -[2023-02-25 13:57:27,642][00699] Component RolloutWorker_w3 stopped! -[2023-02-25 13:57:27,645][12813] Stopping RolloutWorker_w3... -[2023-02-25 13:57:27,652][00699] Component RolloutWorker_w7 stopped! -[2023-02-25 13:57:27,657][12822] Stopping RolloutWorker_w7... -[2023-02-25 13:57:27,649][12813] Loop rollout_proc3_evt_loop terminating... -[2023-02-25 13:57:27,660][12822] Loop rollout_proc7_evt_loop terminating... -[2023-02-25 13:57:27,666][00699] Component RolloutWorker_w5 stopped! -[2023-02-25 13:57:27,671][12819] Stopping RolloutWorker_w5... -[2023-02-25 13:57:27,672][12819] Loop rollout_proc5_evt_loop terminating... -[2023-02-25 13:57:27,728][12789] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000783_3207168.pth -[2023-02-25 13:57:27,742][12789] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-25 13:57:27,930][00699] Component LearnerWorker_p0 stopped! -[2023-02-25 13:57:27,930][12789] Stopping LearnerWorker_p0... -[2023-02-25 13:57:27,937][12789] Loop learner_proc0_evt_loop terminating... -[2023-02-25 13:57:27,937][00699] Waiting for process learner_proc0 to stop... -[2023-02-25 13:57:29,680][00699] Waiting for process inference_proc0-0 to join... -[2023-02-25 13:57:29,835][00699] Waiting for process rollout_proc0 to join... -[2023-02-25 13:57:30,434][00699] Waiting for process rollout_proc1 to join... -[2023-02-25 13:57:30,436][00699] Waiting for process rollout_proc2 to join... -[2023-02-25 13:57:30,440][00699] Waiting for process rollout_proc3 to join... -[2023-02-25 13:57:30,444][00699] Waiting for process rollout_proc4 to join... -[2023-02-25 13:57:30,446][00699] Waiting for process rollout_proc5 to join... -[2023-02-25 13:57:30,448][00699] Waiting for process rollout_proc6 to join... -[2023-02-25 13:57:30,450][00699] Waiting for process rollout_proc7 to join... -[2023-02-25 13:57:30,451][00699] Batcher 0 profile tree view: -batching: 26.9716, releasing_batches: 0.0232 -[2023-02-25 13:57:30,452][00699] InferenceWorker_p0-w0 profile tree view: +[2023-02-26 10:10:38,695][00304] Heartbeat connected on Batcher_0 +[2023-02-26 10:10:38,705][00304] Heartbeat connected on InferenceWorker_p0-w0 +[2023-02-26 10:10:38,718][00304] Heartbeat connected on RolloutWorker_w0 +[2023-02-26 10:10:38,722][00304] Heartbeat connected on RolloutWorker_w1 +[2023-02-26 10:10:38,728][00304] Heartbeat connected on RolloutWorker_w2 +[2023-02-26 10:10:38,733][00304] Heartbeat connected on RolloutWorker_w3 +[2023-02-26 10:10:38,737][00304] Heartbeat connected on RolloutWorker_w4 +[2023-02-26 10:10:38,743][00304] Heartbeat connected on RolloutWorker_w5 +[2023-02-26 10:10:38,748][00304] Heartbeat connected on RolloutWorker_w6 +[2023-02-26 10:10:38,751][00304] Heartbeat connected on RolloutWorker_w7 +[2023-02-26 10:10:42,390][10798] Using optimizer +[2023-02-26 10:10:42,391][10798] No checkpoints found +[2023-02-26 10:10:42,391][10798] Did not load from checkpoint, starting from scratch! +[2023-02-26 10:10:42,391][10798] Initialized policy 0 weights for model version 0 +[2023-02-26 10:10:42,395][10798] LearnerWorker_p0 finished initialization! +[2023-02-26 10:10:42,396][00304] Heartbeat connected on LearnerWorker_p0 +[2023-02-26 10:10:42,403][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-26 10:10:42,597][10811] RunningMeanStd input shape: (3, 72, 128) +[2023-02-26 10:10:42,598][10811] RunningMeanStd input shape: (1,) +[2023-02-26 10:10:42,611][10811] ConvEncoder: input_channels=3 +[2023-02-26 10:10:42,716][10811] Conv encoder output size: 512 +[2023-02-26 10:10:42,716][10811] Policy head output size: 512 +[2023-02-26 10:10:43,705][00304] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-26 10:10:45,011][00304] Inference worker 0-0 is ready! +[2023-02-26 10:10:45,012][00304] All inference workers are ready! Signal rollout workers to start! +[2023-02-26 10:10:45,105][10814] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,108][10819] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,141][10815] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,147][10817] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,168][10816] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,166][10820] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,182][10818] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:45,203][10813] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:10:46,013][10820] Decorrelating experience for 0 frames... +[2023-02-26 10:10:46,011][10813] Decorrelating experience for 0 frames... +[2023-02-26 10:10:46,256][10819] Decorrelating experience for 0 frames... +[2023-02-26 10:10:46,260][10817] Decorrelating experience for 0 frames... +[2023-02-26 10:10:46,266][10814] Decorrelating experience for 0 frames... +[2023-02-26 10:10:46,651][10819] Decorrelating experience for 32 frames... +[2023-02-26 10:10:47,006][10818] Decorrelating experience for 0 frames... +[2023-02-26 10:10:47,077][10813] Decorrelating experience for 32 frames... +[2023-02-26 10:10:47,165][10819] Decorrelating experience for 64 frames... +[2023-02-26 10:10:47,643][10820] Decorrelating experience for 32 frames... +[2023-02-26 10:10:47,674][10816] Decorrelating experience for 0 frames... +[2023-02-26 10:10:48,367][10817] Decorrelating experience for 32 frames... +[2023-02-26 10:10:48,553][10814] Decorrelating experience for 32 frames... +[2023-02-26 10:10:48,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-26 10:10:48,822][10819] Decorrelating experience for 96 frames... +[2023-02-26 10:10:48,915][10818] Decorrelating experience for 32 frames... +[2023-02-26 10:10:49,361][10815] Decorrelating experience for 0 frames... +[2023-02-26 10:10:49,415][10813] Decorrelating experience for 64 frames... +[2023-02-26 10:10:50,037][10817] Decorrelating experience for 64 frames... +[2023-02-26 10:10:50,202][10814] Decorrelating experience for 64 frames... +[2023-02-26 10:10:50,546][10815] Decorrelating experience for 32 frames... +[2023-02-26 10:10:50,921][10820] Decorrelating experience for 64 frames... +[2023-02-26 10:10:51,695][10817] Decorrelating experience for 96 frames... +[2023-02-26 10:10:51,989][10818] Decorrelating experience for 64 frames... +[2023-02-26 10:10:52,139][10813] Decorrelating experience for 96 frames... +[2023-02-26 10:10:52,315][10816] Decorrelating experience for 32 frames... +[2023-02-26 10:10:52,529][10820] Decorrelating experience for 96 frames... +[2023-02-26 10:10:53,036][10816] Decorrelating experience for 64 frames... +[2023-02-26 10:10:53,426][10814] Decorrelating experience for 96 frames... +[2023-02-26 10:10:53,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-26 10:10:53,836][10816] Decorrelating experience for 96 frames... +[2023-02-26 10:10:54,048][10815] Decorrelating experience for 64 frames... +[2023-02-26 10:10:54,340][10818] Decorrelating experience for 96 frames... +[2023-02-26 10:10:54,884][10815] Decorrelating experience for 96 frames... +[2023-02-26 10:10:58,063][10798] Signal inference workers to stop experience collection... +[2023-02-26 10:10:58,076][10811] InferenceWorker_p0-w0: stopping experience collection +[2023-02-26 10:10:58,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 101.6. Samples: 1524. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-26 10:10:58,706][00304] Avg episode reward: [(0, '1.840')] +[2023-02-26 10:11:00,697][10798] Signal inference workers to resume experience collection... +[2023-02-26 10:11:00,697][10811] InferenceWorker_p0-w0: resuming experience collection +[2023-02-26 10:11:03,705][00304] Fps is (10 sec: 1638.4, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 16384. Throughput: 0: 200.5. Samples: 4010. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:11:03,712][00304] Avg episode reward: [(0, '3.125')] +[2023-02-26 10:11:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 1474.6, 300 sec: 1474.6). Total num frames: 36864. Throughput: 0: 298.8. Samples: 7470. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-26 10:11:08,714][00304] Avg episode reward: [(0, '3.767')] +[2023-02-26 10:11:09,903][10811] Updated weights for policy 0, policy_version 10 (0.0019) +[2023-02-26 10:11:13,705][00304] Fps is (10 sec: 3276.8, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 49152. Throughput: 0: 404.4. Samples: 12132. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-26 10:11:13,710][00304] Avg episode reward: [(0, '4.354')] +[2023-02-26 10:11:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 1989.5, 300 sec: 1989.5). Total num frames: 69632. Throughput: 0: 496.4. Samples: 17374. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:11:18,708][00304] Avg episode reward: [(0, '4.419')] +[2023-02-26 10:11:21,006][10811] Updated weights for policy 0, policy_version 20 (0.0014) +[2023-02-26 10:11:23,705][00304] Fps is (10 sec: 4505.5, 60 sec: 2355.2, 300 sec: 2355.2). Total num frames: 94208. Throughput: 0: 522.3. Samples: 20892. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:11:23,707][00304] Avg episode reward: [(0, '4.410')] +[2023-02-26 10:11:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 110592. Throughput: 0: 617.9. Samples: 27804. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:11:28,710][00304] Avg episode reward: [(0, '4.198')] +[2023-02-26 10:11:28,714][10798] Saving new best policy, reward=4.198! +[2023-02-26 10:11:31,700][10811] Updated weights for policy 0, policy_version 30 (0.0016) +[2023-02-26 10:11:33,705][00304] Fps is (10 sec: 3276.8, 60 sec: 2539.5, 300 sec: 2539.5). Total num frames: 126976. Throughput: 0: 716.5. Samples: 32244. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:11:33,713][00304] Avg episode reward: [(0, '4.307')] +[2023-02-26 10:11:33,722][10798] Saving new best policy, reward=4.307! +[2023-02-26 10:11:38,705][00304] Fps is (10 sec: 3276.8, 60 sec: 2606.5, 300 sec: 2606.5). Total num frames: 143360. Throughput: 0: 762.7. Samples: 34320. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:11:38,708][00304] Avg episode reward: [(0, '4.424')] +[2023-02-26 10:11:38,710][10798] Saving new best policy, reward=4.424! +[2023-02-26 10:11:42,765][10811] Updated weights for policy 0, policy_version 40 (0.0021) +[2023-02-26 10:11:43,705][00304] Fps is (10 sec: 4095.9, 60 sec: 2798.9, 300 sec: 2798.9). Total num frames: 167936. Throughput: 0: 871.0. Samples: 40720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:11:43,711][00304] Avg episode reward: [(0, '4.387')] +[2023-02-26 10:11:48,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3140.3, 300 sec: 2898.7). Total num frames: 188416. Throughput: 0: 959.5. Samples: 47188. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:11:48,708][00304] Avg episode reward: [(0, '4.307')] +[2023-02-26 10:11:53,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 2867.2). Total num frames: 200704. Throughput: 0: 932.0. Samples: 49408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:11:53,712][00304] Avg episode reward: [(0, '4.442')] +[2023-02-26 10:11:53,727][10798] Saving new best policy, reward=4.442! +[2023-02-26 10:11:54,028][10811] Updated weights for policy 0, policy_version 50 (0.0023) +[2023-02-26 10:11:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 2949.1). Total num frames: 221184. Throughput: 0: 931.2. Samples: 54038. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:11:58,712][00304] Avg episode reward: [(0, '4.491')] +[2023-02-26 10:11:58,718][10798] Saving new best policy, reward=4.491! +[2023-02-26 10:12:03,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3020.8). Total num frames: 241664. Throughput: 0: 970.9. Samples: 61064. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:12:03,713][00304] Avg episode reward: [(0, '4.602')] +[2023-02-26 10:12:03,730][10798] Saving new best policy, reward=4.602! +[2023-02-26 10:12:03,964][10811] Updated weights for policy 0, policy_version 60 (0.0029) +[2023-02-26 10:12:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3084.1). Total num frames: 262144. Throughput: 0: 969.3. Samples: 64510. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:12:08,709][00304] Avg episode reward: [(0, '4.708')] +[2023-02-26 10:12:08,714][10798] Saving new best policy, reward=4.708! +[2023-02-26 10:12:13,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3094.8). Total num frames: 278528. Throughput: 0: 917.1. Samples: 69072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:12:13,711][00304] Avg episode reward: [(0, '4.701')] +[2023-02-26 10:12:13,722][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth... +[2023-02-26 10:12:16,684][10811] Updated weights for policy 0, policy_version 70 (0.0042) +[2023-02-26 10:12:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3104.3). Total num frames: 294912. Throughput: 0: 927.2. Samples: 73966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:12:18,712][00304] Avg episode reward: [(0, '4.519')] +[2023-02-26 10:12:23,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3194.9). Total num frames: 319488. Throughput: 0: 959.2. Samples: 77482. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:12:23,711][00304] Avg episode reward: [(0, '4.522')] +[2023-02-26 10:12:25,092][10811] Updated weights for policy 0, policy_version 80 (0.0024) +[2023-02-26 10:12:28,710][00304] Fps is (10 sec: 4503.1, 60 sec: 3822.6, 300 sec: 3237.6). Total num frames: 339968. Throughput: 0: 970.9. Samples: 84416. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:12:28,713][00304] Avg episode reward: [(0, '4.482')] +[2023-02-26 10:12:33,712][00304] Fps is (10 sec: 3274.4, 60 sec: 3754.2, 300 sec: 3202.1). Total num frames: 352256. Throughput: 0: 923.7. Samples: 88762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:12:33,715][00304] Avg episode reward: [(0, '4.444')] +[2023-02-26 10:12:37,829][10811] Updated weights for policy 0, policy_version 90 (0.0014) +[2023-02-26 10:12:38,705][00304] Fps is (10 sec: 3278.6, 60 sec: 3822.9, 300 sec: 3241.2). Total num frames: 372736. Throughput: 0: 921.5. Samples: 90874. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:12:38,707][00304] Avg episode reward: [(0, '4.521')] +[2023-02-26 10:12:43,705][00304] Fps is (10 sec: 4099.1, 60 sec: 3754.7, 300 sec: 3276.8). Total num frames: 393216. Throughput: 0: 971.2. Samples: 97742. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:12:43,711][00304] Avg episode reward: [(0, '4.738')] +[2023-02-26 10:12:43,718][10798] Saving new best policy, reward=4.738! +[2023-02-26 10:12:46,389][10811] Updated weights for policy 0, policy_version 100 (0.0017) +[2023-02-26 10:12:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3309.6). Total num frames: 413696. Throughput: 0: 954.7. Samples: 104024. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:12:48,708][00304] Avg episode reward: [(0, '4.659')] +[2023-02-26 10:12:53,706][00304] Fps is (10 sec: 3685.8, 60 sec: 3822.8, 300 sec: 3308.3). Total num frames: 430080. Throughput: 0: 925.9. Samples: 106176. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:12:53,711][00304] Avg episode reward: [(0, '4.789')] +[2023-02-26 10:12:53,721][10798] Saving new best policy, reward=4.789! +[2023-02-26 10:12:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3307.1). Total num frames: 446464. Throughput: 0: 927.1. Samples: 110792. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:12:58,707][00304] Avg episode reward: [(0, '4.871')] +[2023-02-26 10:12:58,712][10798] Saving new best policy, reward=4.871! +[2023-02-26 10:12:59,215][10811] Updated weights for policy 0, policy_version 110 (0.0014) +[2023-02-26 10:13:03,705][00304] Fps is (10 sec: 4096.6, 60 sec: 3822.9, 300 sec: 3364.6). Total num frames: 471040. Throughput: 0: 971.8. Samples: 117698. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:13:03,711][00304] Avg episode reward: [(0, '4.540')] +[2023-02-26 10:13:08,711][00304] Fps is (10 sec: 4093.3, 60 sec: 3754.3, 300 sec: 3361.4). Total num frames: 487424. Throughput: 0: 972.0. Samples: 121230. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:13:08,714][00304] Avg episode reward: [(0, '4.564')] +[2023-02-26 10:13:08,776][10811] Updated weights for policy 0, policy_version 120 (0.0020) +[2023-02-26 10:13:13,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3358.7). Total num frames: 503808. Throughput: 0: 921.0. Samples: 125854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:13:13,710][00304] Avg episode reward: [(0, '4.649')] +[2023-02-26 10:13:18,705][00304] Fps is (10 sec: 3688.8, 60 sec: 3822.9, 300 sec: 3382.5). Total num frames: 524288. Throughput: 0: 936.7. Samples: 130908. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:13:18,712][00304] Avg episode reward: [(0, '4.832')] +[2023-02-26 10:13:20,254][10811] Updated weights for policy 0, policy_version 130 (0.0012) +[2023-02-26 10:13:23,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3404.8). Total num frames: 544768. Throughput: 0: 969.2. Samples: 134488. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:13:23,717][00304] Avg episode reward: [(0, '4.935')] +[2023-02-26 10:13:23,725][10798] Saving new best policy, reward=4.935! +[2023-02-26 10:13:28,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3686.7, 300 sec: 3400.9). Total num frames: 561152. Throughput: 0: 953.0. Samples: 140626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:13:28,711][00304] Avg episode reward: [(0, '4.875')] +[2023-02-26 10:13:32,596][10811] Updated weights for policy 0, policy_version 140 (0.0012) +[2023-02-26 10:13:33,709][00304] Fps is (10 sec: 2865.9, 60 sec: 3686.6, 300 sec: 3373.1). Total num frames: 573440. Throughput: 0: 892.1. Samples: 144174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:13:33,714][00304] Avg episode reward: [(0, '4.991')] +[2023-02-26 10:13:33,726][10798] Saving new best policy, reward=4.991! +[2023-02-26 10:13:38,705][00304] Fps is (10 sec: 2457.7, 60 sec: 3549.9, 300 sec: 3347.0). Total num frames: 585728. Throughput: 0: 882.2. Samples: 145872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:13:38,710][00304] Avg episode reward: [(0, '4.768')] +[2023-02-26 10:13:43,705][00304] Fps is (10 sec: 3278.3, 60 sec: 3549.9, 300 sec: 3367.8). Total num frames: 606208. Throughput: 0: 880.3. Samples: 150404. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:13:43,707][00304] Avg episode reward: [(0, '4.839')] +[2023-02-26 10:13:45,280][10811] Updated weights for policy 0, policy_version 150 (0.0016) +[2023-02-26 10:13:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3387.5). Total num frames: 626688. Throughput: 0: 884.7. Samples: 157508. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:13:48,707][00304] Avg episode reward: [(0, '5.079')] +[2023-02-26 10:13:48,733][10798] Saving new best policy, reward=5.079! +[2023-02-26 10:13:53,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3618.2, 300 sec: 3406.1). Total num frames: 647168. Throughput: 0: 884.2. Samples: 161014. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:13:53,708][00304] Avg episode reward: [(0, '5.043')] +[2023-02-26 10:13:55,776][10811] Updated weights for policy 0, policy_version 160 (0.0038) +[2023-02-26 10:13:58,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3402.8). Total num frames: 663552. Throughput: 0: 881.5. Samples: 165524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:13:58,710][00304] Avg episode reward: [(0, '5.149')] +[2023-02-26 10:13:58,715][10798] Saving new best policy, reward=5.149! +[2023-02-26 10:14:03,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3420.2). Total num frames: 684032. Throughput: 0: 889.1. Samples: 170916. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:14:03,710][00304] Avg episode reward: [(0, '5.004')] +[2023-02-26 10:14:06,285][10811] Updated weights for policy 0, policy_version 170 (0.0021) +[2023-02-26 10:14:08,705][00304] Fps is (10 sec: 4096.2, 60 sec: 3618.5, 300 sec: 3436.6). Total num frames: 704512. Throughput: 0: 886.8. Samples: 174392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:14:08,708][00304] Avg episode reward: [(0, '5.318')] +[2023-02-26 10:14:08,712][10798] Saving new best policy, reward=5.318! +[2023-02-26 10:14:13,706][00304] Fps is (10 sec: 4095.8, 60 sec: 3686.3, 300 sec: 3452.3). Total num frames: 724992. Throughput: 0: 894.9. Samples: 180898. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:14:13,712][00304] Avg episode reward: [(0, '5.346')] +[2023-02-26 10:14:13,723][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth... +[2023-02-26 10:14:13,854][10798] Saving new best policy, reward=5.346! +[2023-02-26 10:14:17,388][10811] Updated weights for policy 0, policy_version 180 (0.0017) +[2023-02-26 10:14:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3429.2). Total num frames: 737280. Throughput: 0: 912.6. Samples: 185238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:14:18,707][00304] Avg episode reward: [(0, '5.452')] +[2023-02-26 10:14:18,787][10798] Saving new best policy, reward=5.452! +[2023-02-26 10:14:23,705][00304] Fps is (10 sec: 3277.1, 60 sec: 3549.9, 300 sec: 3444.4). Total num frames: 757760. Throughput: 0: 929.3. Samples: 187690. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:14:23,708][00304] Avg episode reward: [(0, '5.361')] +[2023-02-26 10:14:27,353][10811] Updated weights for policy 0, policy_version 190 (0.0026) +[2023-02-26 10:14:28,705][00304] Fps is (10 sec: 4505.5, 60 sec: 3686.4, 300 sec: 3477.0). Total num frames: 782336. Throughput: 0: 987.4. Samples: 194838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:14:28,713][00304] Avg episode reward: [(0, '5.738')] +[2023-02-26 10:14:28,718][10798] Saving new best policy, reward=5.738! +[2023-02-26 10:14:33,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3754.9, 300 sec: 3472.7). Total num frames: 798720. Throughput: 0: 960.5. Samples: 200730. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:14:33,709][00304] Avg episode reward: [(0, '5.963')] +[2023-02-26 10:14:33,720][10798] Saving new best policy, reward=5.963! +[2023-02-26 10:14:38,705][00304] Fps is (10 sec: 3276.6, 60 sec: 3822.9, 300 sec: 3468.5). Total num frames: 815104. Throughput: 0: 929.7. Samples: 202850. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:14:38,711][00304] Avg episode reward: [(0, '5.805')] +[2023-02-26 10:14:39,664][10811] Updated weights for policy 0, policy_version 200 (0.0012) +[2023-02-26 10:14:43,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3481.6). Total num frames: 835584. Throughput: 0: 943.1. Samples: 207962. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:14:43,710][00304] Avg episode reward: [(0, '5.583')] +[2023-02-26 10:14:48,560][10811] Updated weights for policy 0, policy_version 210 (0.0028) +[2023-02-26 10:14:48,705][00304] Fps is (10 sec: 4505.9, 60 sec: 3891.2, 300 sec: 3510.9). Total num frames: 860160. Throughput: 0: 982.2. Samples: 215114. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:14:48,711][00304] Avg episode reward: [(0, '5.683')] +[2023-02-26 10:14:53,709][00304] Fps is (10 sec: 4094.3, 60 sec: 3822.7, 300 sec: 3506.1). Total num frames: 876544. Throughput: 0: 982.2. Samples: 218594. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:14:53,711][00304] Avg episode reward: [(0, '6.138')] +[2023-02-26 10:14:53,724][10798] Saving new best policy, reward=6.138! +[2023-02-26 10:14:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3501.7). Total num frames: 892928. Throughput: 0: 934.4. Samples: 222946. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:14:58,708][00304] Avg episode reward: [(0, '6.480')] +[2023-02-26 10:14:58,710][10798] Saving new best policy, reward=6.480! +[2023-02-26 10:15:00,950][10811] Updated weights for policy 0, policy_version 220 (0.0021) +[2023-02-26 10:15:03,705][00304] Fps is (10 sec: 3687.9, 60 sec: 3822.9, 300 sec: 3513.1). Total num frames: 913408. Throughput: 0: 961.2. Samples: 228490. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:15:03,708][00304] Avg episode reward: [(0, '6.730')] +[2023-02-26 10:15:03,716][10798] Saving new best policy, reward=6.730! +[2023-02-26 10:15:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3524.1). Total num frames: 933888. Throughput: 0: 985.1. Samples: 232018. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:15:08,707][00304] Avg episode reward: [(0, '6.977')] +[2023-02-26 10:15:08,714][10798] Saving new best policy, reward=6.977! +[2023-02-26 10:15:09,745][10811] Updated weights for policy 0, policy_version 230 (0.0014) +[2023-02-26 10:15:13,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3823.0, 300 sec: 3534.7). Total num frames: 954368. Throughput: 0: 965.6. Samples: 238292. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:15:13,707][00304] Avg episode reward: [(0, '7.558')] +[2023-02-26 10:15:13,719][10798] Saving new best policy, reward=7.558! +[2023-02-26 10:15:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3515.1). Total num frames: 966656. Throughput: 0: 932.7. Samples: 242700. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:15:18,707][00304] Avg episode reward: [(0, '7.857')] +[2023-02-26 10:15:18,714][10798] Saving new best policy, reward=7.873! +[2023-02-26 10:15:22,041][10811] Updated weights for policy 0, policy_version 240 (0.0016) +[2023-02-26 10:15:23,707][00304] Fps is (10 sec: 3275.9, 60 sec: 3822.8, 300 sec: 3525.5). Total num frames: 987136. Throughput: 0: 941.9. Samples: 245236. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:15:23,713][00304] Avg episode reward: [(0, '7.205')] +[2023-02-26 10:15:28,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3549.9). Total num frames: 1011712. Throughput: 0: 986.4. Samples: 252348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:15:28,707][00304] Avg episode reward: [(0, '6.740')] +[2023-02-26 10:15:30,544][10811] Updated weights for policy 0, policy_version 250 (0.0025) +[2023-02-26 10:15:33,705][00304] Fps is (10 sec: 4506.6, 60 sec: 3891.2, 300 sec: 3559.3). Total num frames: 1032192. Throughput: 0: 960.8. Samples: 258350. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:15:33,708][00304] Avg episode reward: [(0, '7.093')] +[2023-02-26 10:15:38,708][00304] Fps is (10 sec: 3275.7, 60 sec: 3822.8, 300 sec: 3540.6). Total num frames: 1044480. Throughput: 0: 931.9. Samples: 260530. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:15:38,713][00304] Avg episode reward: [(0, '7.351')] +[2023-02-26 10:15:42,805][10811] Updated weights for policy 0, policy_version 260 (0.0045) +[2023-02-26 10:15:43,705][00304] Fps is (10 sec: 3277.0, 60 sec: 3822.9, 300 sec: 3610.0). Total num frames: 1064960. Throughput: 0: 951.2. Samples: 265748. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:15:43,708][00304] Avg episode reward: [(0, '7.397')] +[2023-02-26 10:15:48,705][00304] Fps is (10 sec: 4507.1, 60 sec: 3822.9, 300 sec: 3693.3). Total num frames: 1089536. Throughput: 0: 985.1. Samples: 272820. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:15:48,711][00304] Avg episode reward: [(0, '7.952')] +[2023-02-26 10:15:48,715][10798] Saving new best policy, reward=7.952! +[2023-02-26 10:15:52,180][10811] Updated weights for policy 0, policy_version 270 (0.0019) +[2023-02-26 10:15:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.5, 300 sec: 3762.8). Total num frames: 1110016. Throughput: 0: 978.0. Samples: 276028. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:15:53,710][00304] Avg episode reward: [(0, '7.779')] +[2023-02-26 10:15:58,705][00304] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1122304. Throughput: 0: 939.8. Samples: 280582. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:15:58,709][00304] Avg episode reward: [(0, '8.659')] +[2023-02-26 10:15:58,714][10798] Saving new best policy, reward=8.659! +[2023-02-26 10:16:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1142784. Throughput: 0: 965.9. Samples: 286166. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:16:03,710][00304] Avg episode reward: [(0, '8.576')] +[2023-02-26 10:16:04,077][10811] Updated weights for policy 0, policy_version 280 (0.0019) +[2023-02-26 10:16:08,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1167360. Throughput: 0: 986.7. Samples: 289634. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:16:08,708][00304] Avg episode reward: [(0, '9.058')] +[2023-02-26 10:16:08,713][10798] Saving new best policy, reward=9.058! +[2023-02-26 10:16:13,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1183744. Throughput: 0: 968.1. Samples: 295914. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:16:13,710][00304] Avg episode reward: [(0, '8.894')] +[2023-02-26 10:16:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000289_1183744.pth... +[2023-02-26 10:16:13,862][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth +[2023-02-26 10:16:14,128][10811] Updated weights for policy 0, policy_version 290 (0.0022) +[2023-02-26 10:16:18,705][00304] Fps is (10 sec: 3276.9, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 1200128. Throughput: 0: 934.2. Samples: 300388. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:16:18,716][00304] Avg episode reward: [(0, '9.432')] +[2023-02-26 10:16:18,720][10798] Saving new best policy, reward=9.432! +[2023-02-26 10:16:23,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.4, 300 sec: 3762.8). Total num frames: 1220608. Throughput: 0: 940.0. Samples: 302826. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:16:23,709][00304] Avg episode reward: [(0, '9.769')] +[2023-02-26 10:16:23,723][10798] Saving new best policy, reward=9.769! +[2023-02-26 10:16:25,514][10811] Updated weights for policy 0, policy_version 300 (0.0040) +[2023-02-26 10:16:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3762.8). Total num frames: 1236992. Throughput: 0: 954.4. Samples: 308696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:16:28,708][00304] Avg episode reward: [(0, '10.407')] +[2023-02-26 10:16:28,715][10798] Saving new best policy, reward=10.407! +[2023-02-26 10:16:33,706][00304] Fps is (10 sec: 2866.9, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1249280. Throughput: 0: 888.8. Samples: 312818. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:16:33,708][00304] Avg episode reward: [(0, '10.981')] +[2023-02-26 10:16:33,720][10798] Saving new best policy, reward=10.981! +[2023-02-26 10:16:38,705][00304] Fps is (10 sec: 2457.6, 60 sec: 3618.3, 300 sec: 3707.2). Total num frames: 1261568. Throughput: 0: 858.6. Samples: 314666. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:16:38,716][00304] Avg episode reward: [(0, '10.511')] +[2023-02-26 10:16:40,799][10811] Updated weights for policy 0, policy_version 310 (0.0033) +[2023-02-26 10:16:43,705][00304] Fps is (10 sec: 2867.5, 60 sec: 3549.9, 300 sec: 3693.3). Total num frames: 1277952. Throughput: 0: 854.9. Samples: 319054. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:16:43,707][00304] Avg episode reward: [(0, '10.395')] +[2023-02-26 10:16:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 1302528. Throughput: 0: 883.5. Samples: 325924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:16:48,711][00304] Avg episode reward: [(0, '10.319')] +[2023-02-26 10:16:49,971][10811] Updated weights for policy 0, policy_version 320 (0.0021) +[2023-02-26 10:16:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1327104. Throughput: 0: 883.3. Samples: 329384. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-26 10:16:53,710][00304] Avg episode reward: [(0, '10.794')] +[2023-02-26 10:16:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3721.1). Total num frames: 1339392. Throughput: 0: 862.8. Samples: 334740. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:16:58,709][00304] Avg episode reward: [(0, '10.550')] +[2023-02-26 10:17:01,924][10811] Updated weights for policy 0, policy_version 330 (0.0018) +[2023-02-26 10:17:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3707.2). Total num frames: 1355776. Throughput: 0: 866.1. Samples: 339362. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:17:03,714][00304] Avg episode reward: [(0, '11.330')] +[2023-02-26 10:17:03,725][10798] Saving new best policy, reward=11.330! +[2023-02-26 10:17:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 1380352. Throughput: 0: 888.9. Samples: 342826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:17:08,716][00304] Avg episode reward: [(0, '12.349')] +[2023-02-26 10:17:08,721][10798] Saving new best policy, reward=12.349! +[2023-02-26 10:17:11,257][10811] Updated weights for policy 0, policy_version 340 (0.0021) +[2023-02-26 10:17:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1400832. Throughput: 0: 913.6. Samples: 349808. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:17:13,709][00304] Avg episode reward: [(0, '13.176')] +[2023-02-26 10:17:13,729][10798] Saving new best policy, reward=13.176! +[2023-02-26 10:17:18,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 1417216. Throughput: 0: 927.3. Samples: 354546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:17:18,707][00304] Avg episode reward: [(0, '13.347')] +[2023-02-26 10:17:18,710][10798] Saving new best policy, reward=13.347! +[2023-02-26 10:17:23,570][10811] Updated weights for policy 0, policy_version 350 (0.0030) +[2023-02-26 10:17:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3707.3). Total num frames: 1433600. Throughput: 0: 935.0. Samples: 356740. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:17:23,707][00304] Avg episode reward: [(0, '13.270')] +[2023-02-26 10:17:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3749.0). Total num frames: 1458176. Throughput: 0: 979.4. Samples: 363128. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:17:28,707][00304] Avg episode reward: [(0, '13.893')] +[2023-02-26 10:17:28,713][10798] Saving new best policy, reward=13.893! +[2023-02-26 10:17:32,226][10811] Updated weights for policy 0, policy_version 360 (0.0026) +[2023-02-26 10:17:33,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3748.9). Total num frames: 1478656. Throughput: 0: 982.0. Samples: 370112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:17:33,709][00304] Avg episode reward: [(0, '14.632')] +[2023-02-26 10:17:33,722][10798] Saving new best policy, reward=14.632! +[2023-02-26 10:17:38,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 1490944. Throughput: 0: 952.1. Samples: 372230. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:17:38,707][00304] Avg episode reward: [(0, '15.106')] +[2023-02-26 10:17:38,786][10798] Saving new best policy, reward=15.106! +[2023-02-26 10:17:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 1507328. Throughput: 0: 931.5. Samples: 376656. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:17:43,710][00304] Avg episode reward: [(0, '15.887')] +[2023-02-26 10:17:43,726][10798] Saving new best policy, reward=15.887! +[2023-02-26 10:17:44,764][10811] Updated weights for policy 0, policy_version 370 (0.0016) +[2023-02-26 10:17:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 1531904. Throughput: 0: 978.3. Samples: 383386. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:17:48,712][00304] Avg episode reward: [(0, '16.659')] +[2023-02-26 10:17:48,719][10798] Saving new best policy, reward=16.659! +[2023-02-26 10:17:53,669][10811] Updated weights for policy 0, policy_version 380 (0.0012) +[2023-02-26 10:17:53,705][00304] Fps is (10 sec: 4915.0, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1556480. Throughput: 0: 979.0. Samples: 386882. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:17:53,708][00304] Avg episode reward: [(0, '15.999')] +[2023-02-26 10:17:58,706][00304] Fps is (10 sec: 3685.9, 60 sec: 3822.8, 300 sec: 3721.1). Total num frames: 1568768. Throughput: 0: 940.0. Samples: 392108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:17:58,710][00304] Avg episode reward: [(0, '16.391')] +[2023-02-26 10:18:03,705][00304] Fps is (10 sec: 2867.3, 60 sec: 3822.9, 300 sec: 3721.2). Total num frames: 1585152. Throughput: 0: 935.5. Samples: 396642. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:18:03,708][00304] Avg episode reward: [(0, '17.330')] +[2023-02-26 10:18:03,718][10798] Saving new best policy, reward=17.330! +[2023-02-26 10:18:05,841][10811] Updated weights for policy 0, policy_version 390 (0.0024) +[2023-02-26 10:18:08,705][00304] Fps is (10 sec: 4096.6, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1609728. Throughput: 0: 963.2. Samples: 400086. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:18:08,711][00304] Avg episode reward: [(0, '17.604')] +[2023-02-26 10:18:08,716][10798] Saving new best policy, reward=17.604! +[2023-02-26 10:18:13,707][00304] Fps is (10 sec: 4504.5, 60 sec: 3822.8, 300 sec: 3748.9). Total num frames: 1630208. Throughput: 0: 976.9. Samples: 407090. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:18:13,711][00304] Avg episode reward: [(0, '17.843')] +[2023-02-26 10:18:13,723][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth... +[2023-02-26 10:18:13,917][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth +[2023-02-26 10:18:13,934][10798] Saving new best policy, reward=17.843! +[2023-02-26 10:18:15,806][10811] Updated weights for policy 0, policy_version 400 (0.0026) +[2023-02-26 10:18:18,705][00304] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 1646592. Throughput: 0: 926.6. Samples: 411810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:18:18,708][00304] Avg episode reward: [(0, '18.108')] +[2023-02-26 10:18:18,710][10798] Saving new best policy, reward=18.108! +[2023-02-26 10:18:23,705][00304] Fps is (10 sec: 2867.9, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 1658880. Throughput: 0: 928.9. Samples: 414030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:18:23,708][00304] Avg episode reward: [(0, '17.318')] +[2023-02-26 10:18:27,147][10811] Updated weights for policy 0, policy_version 410 (0.0014) +[2023-02-26 10:18:28,707][00304] Fps is (10 sec: 3685.8, 60 sec: 3754.5, 300 sec: 3762.8). Total num frames: 1683456. Throughput: 0: 972.9. Samples: 420438. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:18:28,709][00304] Avg episode reward: [(0, '17.299')] +[2023-02-26 10:18:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 1708032. Throughput: 0: 980.8. Samples: 427524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:18:33,711][00304] Avg episode reward: [(0, '15.535')] +[2023-02-26 10:18:36,963][10811] Updated weights for policy 0, policy_version 420 (0.0014) +[2023-02-26 10:18:38,709][00304] Fps is (10 sec: 4095.0, 60 sec: 3890.9, 300 sec: 3790.5). Total num frames: 1724416. Throughput: 0: 955.2. Samples: 429870. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:18:38,714][00304] Avg episode reward: [(0, '15.266')] +[2023-02-26 10:18:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 1740800. Throughput: 0: 939.3. Samples: 434376. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:18:43,707][00304] Avg episode reward: [(0, '14.433')] +[2023-02-26 10:18:48,075][10811] Updated weights for policy 0, policy_version 430 (0.0033) +[2023-02-26 10:18:48,705][00304] Fps is (10 sec: 3688.1, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1761280. Throughput: 0: 984.4. Samples: 440938. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:18:48,708][00304] Avg episode reward: [(0, '14.989')] +[2023-02-26 10:18:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3804.4). Total num frames: 1785856. Throughput: 0: 987.5. Samples: 444524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:18:53,708][00304] Avg episode reward: [(0, '14.899')] +[2023-02-26 10:18:58,512][10811] Updated weights for policy 0, policy_version 440 (0.0015) +[2023-02-26 10:18:58,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.3, 300 sec: 3790.5). Total num frames: 1802240. Throughput: 0: 955.7. Samples: 450096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:18:58,709][00304] Avg episode reward: [(0, '15.640')] +[2023-02-26 10:19:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1814528. Throughput: 0: 954.2. Samples: 454748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:19:03,712][00304] Avg episode reward: [(0, '15.480')] +[2023-02-26 10:19:08,705][00304] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1839104. Throughput: 0: 980.1. Samples: 458136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:19:08,706][00304] Avg episode reward: [(0, '15.769')] +[2023-02-26 10:19:08,859][10811] Updated weights for policy 0, policy_version 450 (0.0030) +[2023-02-26 10:19:13,705][00304] Fps is (10 sec: 4915.0, 60 sec: 3891.3, 300 sec: 3818.3). Total num frames: 1863680. Throughput: 0: 995.9. Samples: 465250. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:19:13,713][00304] Avg episode reward: [(0, '16.614')] +[2023-02-26 10:19:18,705][00304] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 1880064. Throughput: 0: 949.9. Samples: 470272. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:19:18,710][00304] Avg episode reward: [(0, '16.401')] +[2023-02-26 10:19:19,698][10811] Updated weights for policy 0, policy_version 460 (0.0046) +[2023-02-26 10:19:23,705][00304] Fps is (10 sec: 2867.3, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1892352. Throughput: 0: 948.0. Samples: 472524. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:19:23,712][00304] Avg episode reward: [(0, '16.627')] +[2023-02-26 10:19:28,706][00304] Fps is (10 sec: 2866.9, 60 sec: 3754.7, 300 sec: 3762.7). Total num frames: 1908736. Throughput: 0: 949.5. Samples: 477104. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:19:28,709][00304] Avg episode reward: [(0, '18.535')] +[2023-02-26 10:19:28,714][10798] Saving new best policy, reward=18.535! +[2023-02-26 10:19:33,275][10811] Updated weights for policy 0, policy_version 470 (0.0050) +[2023-02-26 10:19:33,708][00304] Fps is (10 sec: 3275.7, 60 sec: 3617.9, 300 sec: 3762.7). Total num frames: 1925120. Throughput: 0: 905.6. Samples: 481692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:19:33,711][00304] Avg episode reward: [(0, '18.495')] +[2023-02-26 10:19:38,705][00304] Fps is (10 sec: 3277.3, 60 sec: 3618.4, 300 sec: 3748.9). Total num frames: 1941504. Throughput: 0: 874.1. Samples: 483858. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:19:38,707][00304] Avg episode reward: [(0, '19.608')] +[2023-02-26 10:19:38,715][10798] Saving new best policy, reward=19.608! +[2023-02-26 10:19:43,706][00304] Fps is (10 sec: 2867.9, 60 sec: 3549.8, 300 sec: 3707.2). Total num frames: 1953792. Throughput: 0: 850.9. Samples: 488386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:19:43,711][00304] Avg episode reward: [(0, '19.577')] +[2023-02-26 10:19:45,806][10811] Updated weights for policy 0, policy_version 480 (0.0012) +[2023-02-26 10:19:48,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 1978368. Throughput: 0: 890.6. Samples: 494824. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:19:48,710][00304] Avg episode reward: [(0, '18.656')] +[2023-02-26 10:19:53,705][00304] Fps is (10 sec: 4915.6, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 2002944. Throughput: 0: 894.9. Samples: 498406. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:19:53,712][00304] Avg episode reward: [(0, '17.933')] +[2023-02-26 10:19:54,674][10811] Updated weights for policy 0, policy_version 490 (0.0039) +[2023-02-26 10:19:58,705][00304] Fps is (10 sec: 4096.2, 60 sec: 3618.2, 300 sec: 3748.9). Total num frames: 2019328. Throughput: 0: 863.2. Samples: 504092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:19:58,709][00304] Avg episode reward: [(0, '16.509')] +[2023-02-26 10:20:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 2031616. Throughput: 0: 849.9. Samples: 508516. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:20:03,709][00304] Avg episode reward: [(0, '14.949')] +[2023-02-26 10:20:06,682][10811] Updated weights for policy 0, policy_version 500 (0.0051) +[2023-02-26 10:20:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 2056192. Throughput: 0: 872.4. Samples: 511782. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:20:08,709][00304] Avg episode reward: [(0, '14.981')] +[2023-02-26 10:20:13,705][00304] Fps is (10 sec: 4915.1, 60 sec: 3618.2, 300 sec: 3776.6). Total num frames: 2080768. Throughput: 0: 931.0. Samples: 518996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:13,713][00304] Avg episode reward: [(0, '14.595')] +[2023-02-26 10:20:13,728][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000508_2080768.pth... +[2023-02-26 10:20:13,910][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000289_1183744.pth +[2023-02-26 10:20:16,193][10811] Updated weights for policy 0, policy_version 510 (0.0028) +[2023-02-26 10:20:18,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2093056. Throughput: 0: 943.2. Samples: 524132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:20:18,713][00304] Avg episode reward: [(0, '15.244')] +[2023-02-26 10:20:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 2109440. Throughput: 0: 946.7. Samples: 526458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:20:23,715][00304] Avg episode reward: [(0, '15.341')] +[2023-02-26 10:20:27,442][10811] Updated weights for policy 0, policy_version 520 (0.0022) +[2023-02-26 10:20:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3735.0). Total num frames: 2134016. Throughput: 0: 980.8. Samples: 532522. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:28,706][00304] Avg episode reward: [(0, '15.672')] +[2023-02-26 10:20:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.4, 300 sec: 3776.7). Total num frames: 2158592. Throughput: 0: 998.3. Samples: 539748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:33,707][00304] Avg episode reward: [(0, '15.373')] +[2023-02-26 10:20:37,396][10811] Updated weights for policy 0, policy_version 530 (0.0017) +[2023-02-26 10:20:38,706][00304] Fps is (10 sec: 3686.0, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2170880. Throughput: 0: 974.7. Samples: 542270. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:38,710][00304] Avg episode reward: [(0, '15.410')] +[2023-02-26 10:20:43,705][00304] Fps is (10 sec: 2867.1, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 2187264. Throughput: 0: 949.3. Samples: 546810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:20:43,714][00304] Avg episode reward: [(0, '15.977')] +[2023-02-26 10:20:48,436][10811] Updated weights for policy 0, policy_version 540 (0.0019) +[2023-02-26 10:20:48,705][00304] Fps is (10 sec: 4096.4, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 2211840. Throughput: 0: 990.6. Samples: 553094. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:20:48,707][00304] Avg episode reward: [(0, '17.205')] +[2023-02-26 10:20:53,706][00304] Fps is (10 sec: 4914.6, 60 sec: 3891.1, 300 sec: 3776.6). Total num frames: 2236416. Throughput: 0: 996.8. Samples: 556640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:53,709][00304] Avg episode reward: [(0, '17.741')] +[2023-02-26 10:20:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2248704. Throughput: 0: 964.1. Samples: 562380. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:20:58,711][00304] Avg episode reward: [(0, '18.728')] +[2023-02-26 10:20:58,892][10811] Updated weights for policy 0, policy_version 550 (0.0023) +[2023-02-26 10:21:03,705][00304] Fps is (10 sec: 2867.6, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 2265088. Throughput: 0: 950.0. Samples: 566880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:21:03,709][00304] Avg episode reward: [(0, '19.662')] +[2023-02-26 10:21:03,729][10798] Saving new best policy, reward=19.662! +[2023-02-26 10:21:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2289664. Throughput: 0: 969.8. Samples: 570098. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:21:08,707][00304] Avg episode reward: [(0, '18.747')] +[2023-02-26 10:21:09,425][10811] Updated weights for policy 0, policy_version 560 (0.0016) +[2023-02-26 10:21:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 2310144. Throughput: 0: 992.3. Samples: 577174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:21:13,711][00304] Avg episode reward: [(0, '19.773')] +[2023-02-26 10:21:13,775][10798] Saving new best policy, reward=19.773! +[2023-02-26 10:21:18,706][00304] Fps is (10 sec: 3685.8, 60 sec: 3891.1, 300 sec: 3748.9). Total num frames: 2326528. Throughput: 0: 945.5. Samples: 582296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:21:18,711][00304] Avg episode reward: [(0, '20.030')] +[2023-02-26 10:21:18,717][10798] Saving new best policy, reward=20.030! +[2023-02-26 10:21:20,685][10811] Updated weights for policy 0, policy_version 570 (0.0029) +[2023-02-26 10:21:23,705][00304] Fps is (10 sec: 3276.6, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2342912. Throughput: 0: 937.5. Samples: 584456. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:21:23,708][00304] Avg episode reward: [(0, '20.282')] +[2023-02-26 10:21:23,724][10798] Saving new best policy, reward=20.282! +[2023-02-26 10:21:28,705][00304] Fps is (10 sec: 3686.9, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2363392. Throughput: 0: 969.5. Samples: 590438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:21:28,714][00304] Avg episode reward: [(0, '20.098')] +[2023-02-26 10:21:30,506][10811] Updated weights for policy 0, policy_version 580 (0.0027) +[2023-02-26 10:21:33,705][00304] Fps is (10 sec: 4505.9, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2387968. Throughput: 0: 989.3. Samples: 597614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:21:33,714][00304] Avg episode reward: [(0, '21.631')] +[2023-02-26 10:21:33,831][10798] Saving new best policy, reward=21.631! +[2023-02-26 10:21:38,708][00304] Fps is (10 sec: 4094.5, 60 sec: 3891.0, 300 sec: 3818.3). Total num frames: 2404352. Throughput: 0: 968.0. Samples: 600202. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:21:38,715][00304] Avg episode reward: [(0, '20.796')] +[2023-02-26 10:21:42,157][10811] Updated weights for policy 0, policy_version 590 (0.0027) +[2023-02-26 10:21:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2420736. Throughput: 0: 941.8. Samples: 604760. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:21:43,713][00304] Avg episode reward: [(0, '20.107')] +[2023-02-26 10:21:48,705][00304] Fps is (10 sec: 3687.7, 60 sec: 3822.9, 300 sec: 3776.6). Total num frames: 2441216. Throughput: 0: 979.5. Samples: 610956. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:21:48,711][00304] Avg episode reward: [(0, '21.372')] +[2023-02-26 10:21:51,392][10811] Updated weights for policy 0, policy_version 600 (0.0013) +[2023-02-26 10:21:53,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3823.0, 300 sec: 3818.3). Total num frames: 2465792. Throughput: 0: 988.4. Samples: 614578. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:21:53,707][00304] Avg episode reward: [(0, '20.301')] +[2023-02-26 10:21:58,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 2482176. Throughput: 0: 963.0. Samples: 620508. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:21:58,709][00304] Avg episode reward: [(0, '19.867')] +[2023-02-26 10:22:03,061][10811] Updated weights for policy 0, policy_version 610 (0.0016) +[2023-02-26 10:22:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2498560. Throughput: 0: 951.3. Samples: 625104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:22:03,715][00304] Avg episode reward: [(0, '19.721')] +[2023-02-26 10:22:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 2523136. Throughput: 0: 971.8. Samples: 628186. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:22:08,707][00304] Avg episode reward: [(0, '18.784')] +[2023-02-26 10:22:11,923][10811] Updated weights for policy 0, policy_version 620 (0.0030) +[2023-02-26 10:22:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 2547712. Throughput: 0: 1000.0. Samples: 635440. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:22:13,712][00304] Avg episode reward: [(0, '16.969')] +[2023-02-26 10:22:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000622_2547712.pth... +[2023-02-26 10:22:13,844][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth +[2023-02-26 10:22:18,711][00304] Fps is (10 sec: 3684.1, 60 sec: 3890.9, 300 sec: 3818.2). Total num frames: 2560000. Throughput: 0: 959.8. Samples: 640812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:22:18,713][00304] Avg episode reward: [(0, '17.219')] +[2023-02-26 10:22:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2576384. Throughput: 0: 952.2. Samples: 643046. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:22:23,708][00304] Avg episode reward: [(0, '17.718')] +[2023-02-26 10:22:24,253][10811] Updated weights for policy 0, policy_version 630 (0.0025) +[2023-02-26 10:22:28,705][00304] Fps is (10 sec: 3278.8, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2592768. Throughput: 0: 953.6. Samples: 647670. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:22:28,710][00304] Avg episode reward: [(0, '19.042')] +[2023-02-26 10:22:33,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3776.6). Total num frames: 2605056. Throughput: 0: 915.6. Samples: 652158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:22:33,713][00304] Avg episode reward: [(0, '20.517')] +[2023-02-26 10:22:38,015][10811] Updated weights for policy 0, policy_version 640 (0.0019) +[2023-02-26 10:22:38,709][00304] Fps is (10 sec: 2865.9, 60 sec: 3618.1, 300 sec: 3776.6). Total num frames: 2621440. Throughput: 0: 885.5. Samples: 654430. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:22:38,717][00304] Avg episode reward: [(0, '19.093')] +[2023-02-26 10:22:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 2637824. Throughput: 0: 852.2. Samples: 658858. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:22:43,707][00304] Avg episode reward: [(0, '19.732')] +[2023-02-26 10:22:48,705][00304] Fps is (10 sec: 3688.0, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 2658304. Throughput: 0: 884.1. Samples: 664888. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:22:48,713][00304] Avg episode reward: [(0, '20.346')] +[2023-02-26 10:22:49,146][10811] Updated weights for policy 0, policy_version 650 (0.0014) +[2023-02-26 10:22:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 2682880. Throughput: 0: 894.4. Samples: 668436. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:22:53,714][00304] Avg episode reward: [(0, '21.043')] +[2023-02-26 10:22:58,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 2699264. Throughput: 0: 866.4. Samples: 674428. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:22:58,708][00304] Avg episode reward: [(0, '20.892')] +[2023-02-26 10:22:59,741][10811] Updated weights for policy 0, policy_version 660 (0.0018) +[2023-02-26 10:23:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 2711552. Throughput: 0: 848.5. Samples: 678988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:23:03,712][00304] Avg episode reward: [(0, '21.428')] +[2023-02-26 10:23:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2736128. Throughput: 0: 865.1. Samples: 681976. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:23:08,707][00304] Avg episode reward: [(0, '22.448')] +[2023-02-26 10:23:08,714][10798] Saving new best policy, reward=22.448! +[2023-02-26 10:23:10,105][10811] Updated weights for policy 0, policy_version 670 (0.0031) +[2023-02-26 10:23:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3549.9, 300 sec: 3776.7). Total num frames: 2760704. Throughput: 0: 919.6. Samples: 689054. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:23:13,714][00304] Avg episode reward: [(0, '21.231')] +[2023-02-26 10:23:18,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3618.5, 300 sec: 3790.5). Total num frames: 2777088. Throughput: 0: 941.0. Samples: 694504. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:23:18,712][00304] Avg episode reward: [(0, '20.246')] +[2023-02-26 10:23:21,404][10811] Updated weights for policy 0, policy_version 680 (0.0019) +[2023-02-26 10:23:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2789376. Throughput: 0: 941.3. Samples: 696786. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:23:23,710][00304] Avg episode reward: [(0, '22.131')] +[2023-02-26 10:23:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 2813952. Throughput: 0: 973.8. Samples: 702680. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:23:28,713][00304] Avg episode reward: [(0, '21.294')] +[2023-02-26 10:23:30,854][10811] Updated weights for policy 0, policy_version 690 (0.0029) +[2023-02-26 10:23:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2838528. Throughput: 0: 999.7. Samples: 709874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:23:33,710][00304] Avg episode reward: [(0, '20.813')] +[2023-02-26 10:23:38,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.5, 300 sec: 3776.7). Total num frames: 2854912. Throughput: 0: 982.5. Samples: 712650. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:23:38,711][00304] Avg episode reward: [(0, '21.234')] +[2023-02-26 10:23:42,581][10811] Updated weights for policy 0, policy_version 700 (0.0013) +[2023-02-26 10:23:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2867200. Throughput: 0: 951.3. Samples: 717236. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:23:43,708][00304] Avg episode reward: [(0, '20.695')] +[2023-02-26 10:23:48,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2891776. Throughput: 0: 990.4. Samples: 723556. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) +[2023-02-26 10:23:48,707][00304] Avg episode reward: [(0, '18.929')] +[2023-02-26 10:23:51,616][10811] Updated weights for policy 0, policy_version 710 (0.0020) +[2023-02-26 10:23:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2916352. Throughput: 0: 1001.4. Samples: 727040. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:23:53,708][00304] Avg episode reward: [(0, '19.262')] +[2023-02-26 10:23:58,706][00304] Fps is (10 sec: 4095.6, 60 sec: 3891.1, 300 sec: 3790.5). Total num frames: 2932736. Throughput: 0: 973.2. Samples: 732850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:23:58,712][00304] Avg episode reward: [(0, '19.302')] +[2023-02-26 10:24:03,523][10811] Updated weights for policy 0, policy_version 720 (0.0030) +[2023-02-26 10:24:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3762.8). Total num frames: 2949120. Throughput: 0: 951.6. Samples: 737328. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:24:03,707][00304] Avg episode reward: [(0, '19.912')] +[2023-02-26 10:24:08,705][00304] Fps is (10 sec: 3686.7, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2969600. Throughput: 0: 970.8. Samples: 740474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:24:08,708][00304] Avg episode reward: [(0, '20.725')] +[2023-02-26 10:24:12,491][10811] Updated weights for policy 0, policy_version 730 (0.0034) +[2023-02-26 10:24:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2994176. Throughput: 0: 999.5. Samples: 747656. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:24:13,708][00304] Avg episode reward: [(0, '21.823')] +[2023-02-26 10:24:13,720][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth... +[2023-02-26 10:24:13,841][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000508_2080768.pth +[2023-02-26 10:24:18,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3010560. Throughput: 0: 959.7. Samples: 753062. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:24:18,715][00304] Avg episode reward: [(0, '21.658')] +[2023-02-26 10:24:23,707][00304] Fps is (10 sec: 3276.2, 60 sec: 3959.3, 300 sec: 3790.5). Total num frames: 3026944. Throughput: 0: 947.6. Samples: 755296. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:24:23,720][00304] Avg episode reward: [(0, '20.520')] +[2023-02-26 10:24:24,666][10811] Updated weights for policy 0, policy_version 740 (0.0028) +[2023-02-26 10:24:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3804.5). Total num frames: 3047424. Throughput: 0: 977.6. Samples: 761226. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:24:28,712][00304] Avg episode reward: [(0, '20.635')] +[2023-02-26 10:24:33,127][10811] Updated weights for policy 0, policy_version 750 (0.0035) +[2023-02-26 10:24:33,705][00304] Fps is (10 sec: 4506.5, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3072000. Throughput: 0: 996.0. Samples: 768374. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:24:33,708][00304] Avg episode reward: [(0, '19.186')] +[2023-02-26 10:24:38,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3088384. Throughput: 0: 980.3. Samples: 771152. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:24:38,711][00304] Avg episode reward: [(0, '17.652')] +[2023-02-26 10:24:43,708][00304] Fps is (10 sec: 3275.8, 60 sec: 3959.3, 300 sec: 3818.3). Total num frames: 3104768. Throughput: 0: 952.4. Samples: 775708. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:24:43,711][00304] Avg episode reward: [(0, '19.014')] +[2023-02-26 10:24:45,407][10811] Updated weights for policy 0, policy_version 760 (0.0012) +[2023-02-26 10:24:48,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3125248. Throughput: 0: 992.1. Samples: 781972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:24:48,708][00304] Avg episode reward: [(0, '20.495')] +[2023-02-26 10:24:53,705][00304] Fps is (10 sec: 4507.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3149824. Throughput: 0: 1002.0. Samples: 785562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:24:53,708][00304] Avg episode reward: [(0, '21.666')] +[2023-02-26 10:24:54,016][10811] Updated weights for policy 0, policy_version 770 (0.0017) +[2023-02-26 10:24:58,706][00304] Fps is (10 sec: 4095.5, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3166208. Throughput: 0: 972.7. Samples: 791428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:24:58,709][00304] Avg episode reward: [(0, '21.770')] +[2023-02-26 10:25:03,705][00304] Fps is (10 sec: 3276.7, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 3182592. Throughput: 0: 952.6. Samples: 795930. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:25:03,713][00304] Avg episode reward: [(0, '23.762')] +[2023-02-26 10:25:03,729][10798] Saving new best policy, reward=23.762! +[2023-02-26 10:25:06,383][10811] Updated weights for policy 0, policy_version 780 (0.0018) +[2023-02-26 10:25:08,705][00304] Fps is (10 sec: 3686.8, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3203072. Throughput: 0: 971.5. Samples: 799012. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-26 10:25:08,710][00304] Avg episode reward: [(0, '23.497')] +[2023-02-26 10:25:13,705][00304] Fps is (10 sec: 4505.8, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3227648. Throughput: 0: 999.9. Samples: 806222. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:25:13,706][00304] Avg episode reward: [(0, '22.452')] +[2023-02-26 10:25:15,237][10811] Updated weights for policy 0, policy_version 790 (0.0014) +[2023-02-26 10:25:18,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3244032. Throughput: 0: 958.5. Samples: 811508. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:25:18,708][00304] Avg episode reward: [(0, '21.802')] +[2023-02-26 10:25:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.3, 300 sec: 3818.3). Total num frames: 3260416. Throughput: 0: 946.7. Samples: 813754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:25:23,711][00304] Avg episode reward: [(0, '22.312')] +[2023-02-26 10:25:28,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 3272704. Throughput: 0: 948.0. Samples: 818366. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:25:28,708][00304] Avg episode reward: [(0, '21.747')] +[2023-02-26 10:25:28,784][10811] Updated weights for policy 0, policy_version 800 (0.0023) +[2023-02-26 10:25:33,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3289088. Throughput: 0: 909.2. Samples: 822884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:25:33,709][00304] Avg episode reward: [(0, '22.344')] +[2023-02-26 10:25:38,706][00304] Fps is (10 sec: 3276.3, 60 sec: 3618.0, 300 sec: 3790.5). Total num frames: 3305472. Throughput: 0: 881.3. Samples: 825222. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:25:38,709][00304] Avg episode reward: [(0, '22.870')] +[2023-02-26 10:25:42,287][10811] Updated weights for policy 0, policy_version 810 (0.0059) +[2023-02-26 10:25:43,708][00304] Fps is (10 sec: 3275.8, 60 sec: 3618.1, 300 sec: 3762.7). Total num frames: 3321856. Throughput: 0: 852.8. Samples: 829806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:25:43,710][00304] Avg episode reward: [(0, '23.196')] +[2023-02-26 10:25:48,705][00304] Fps is (10 sec: 3686.9, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 3342336. Throughput: 0: 893.4. Samples: 836132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:25:48,708][00304] Avg episode reward: [(0, '24.225')] +[2023-02-26 10:25:48,711][10798] Saving new best policy, reward=24.225! +[2023-02-26 10:25:51,535][10811] Updated weights for policy 0, policy_version 820 (0.0017) +[2023-02-26 10:25:53,705][00304] Fps is (10 sec: 4506.9, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3366912. Throughput: 0: 903.2. Samples: 839654. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:25:53,711][00304] Avg episode reward: [(0, '23.863')] +[2023-02-26 10:25:58,707][00304] Fps is (10 sec: 4095.0, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3383296. Throughput: 0: 875.0. Samples: 845598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:25:58,710][00304] Avg episode reward: [(0, '24.273')] +[2023-02-26 10:25:58,722][10798] Saving new best policy, reward=24.273! +[2023-02-26 10:26:03,584][10811] Updated weights for policy 0, policy_version 830 (0.0015) +[2023-02-26 10:26:03,706][00304] Fps is (10 sec: 3276.5, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3399680. Throughput: 0: 858.5. Samples: 850142. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:26:03,712][00304] Avg episode reward: [(0, '23.528')] +[2023-02-26 10:26:08,705][00304] Fps is (10 sec: 3687.3, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3420160. Throughput: 0: 878.6. Samples: 853292. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:26:08,707][00304] Avg episode reward: [(0, '21.981')] +[2023-02-26 10:26:12,079][10811] Updated weights for policy 0, policy_version 840 (0.0021) +[2023-02-26 10:26:13,705][00304] Fps is (10 sec: 4506.0, 60 sec: 3618.1, 300 sec: 3790.6). Total num frames: 3444736. Throughput: 0: 938.5. Samples: 860600. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:26:13,707][00304] Avg episode reward: [(0, '21.487')] +[2023-02-26 10:26:13,785][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000842_3448832.pth... +[2023-02-26 10:26:13,904][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000622_2547712.pth +[2023-02-26 10:26:18,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3461120. Throughput: 0: 956.2. Samples: 865914. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:26:18,712][00304] Avg episode reward: [(0, '22.784')] +[2023-02-26 10:26:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 3477504. Throughput: 0: 953.6. Samples: 868132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:26:23,709][00304] Avg episode reward: [(0, '22.340')] +[2023-02-26 10:26:24,486][10811] Updated weights for policy 0, policy_version 850 (0.0030) +[2023-02-26 10:26:28,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 3502080. Throughput: 0: 984.4. Samples: 874100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:26:28,708][00304] Avg episode reward: [(0, '22.856')] +[2023-02-26 10:26:32,906][10811] Updated weights for policy 0, policy_version 860 (0.0026) +[2023-02-26 10:26:33,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3790.6). Total num frames: 3522560. Throughput: 0: 1003.4. Samples: 881286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:26:33,708][00304] Avg episode reward: [(0, '23.450')] +[2023-02-26 10:26:38,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 3790.5). Total num frames: 3538944. Throughput: 0: 984.8. Samples: 883972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:26:38,712][00304] Avg episode reward: [(0, '21.838')] +[2023-02-26 10:26:43,707][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.4, 300 sec: 3776.7). Total num frames: 3555328. Throughput: 0: 952.4. Samples: 888452. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:26:43,710][00304] Avg episode reward: [(0, '20.925')] +[2023-02-26 10:26:45,277][10811] Updated weights for policy 0, policy_version 870 (0.0029) +[2023-02-26 10:26:48,707][00304] Fps is (10 sec: 4095.0, 60 sec: 3959.3, 300 sec: 3776.6). Total num frames: 3579904. Throughput: 0: 994.5. Samples: 894898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:26:48,709][00304] Avg episode reward: [(0, '21.275')] +[2023-02-26 10:26:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3600384. Throughput: 0: 1004.6. Samples: 898498. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:26:53,710][00304] Avg episode reward: [(0, '20.815')] +[2023-02-26 10:26:53,844][10811] Updated weights for policy 0, policy_version 880 (0.0014) +[2023-02-26 10:26:58,705][00304] Fps is (10 sec: 3687.3, 60 sec: 3891.4, 300 sec: 3790.5). Total num frames: 3616768. Throughput: 0: 968.7. Samples: 904192. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:26:58,714][00304] Avg episode reward: [(0, '20.567')] +[2023-02-26 10:27:03,706][00304] Fps is (10 sec: 3276.6, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 3633152. Throughput: 0: 952.8. Samples: 908790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:27:03,711][00304] Avg episode reward: [(0, '21.615')] +[2023-02-26 10:27:06,140][10811] Updated weights for policy 0, policy_version 890 (0.0025) +[2023-02-26 10:27:08,709][00304] Fps is (10 sec: 4094.0, 60 sec: 3959.2, 300 sec: 3762.7). Total num frames: 3657728. Throughput: 0: 974.7. Samples: 911996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:27:08,714][00304] Avg episode reward: [(0, '22.592')] +[2023-02-26 10:27:13,705][00304] Fps is (10 sec: 4506.0, 60 sec: 3891.2, 300 sec: 3790.6). Total num frames: 3678208. Throughput: 0: 1001.4. Samples: 919164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:27:13,708][00304] Avg episode reward: [(0, '23.654')] +[2023-02-26 10:27:15,148][10811] Updated weights for policy 0, policy_version 900 (0.0013) +[2023-02-26 10:27:18,707][00304] Fps is (10 sec: 3687.2, 60 sec: 3891.1, 300 sec: 3790.5). Total num frames: 3694592. Throughput: 0: 961.6. Samples: 924560. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:27:18,713][00304] Avg episode reward: [(0, '23.070')] +[2023-02-26 10:27:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3710976. Throughput: 0: 950.7. Samples: 926752. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:27:23,710][00304] Avg episode reward: [(0, '23.794')] +[2023-02-26 10:27:26,893][10811] Updated weights for policy 0, policy_version 910 (0.0016) +[2023-02-26 10:27:28,705][00304] Fps is (10 sec: 4097.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3735552. Throughput: 0: 982.7. Samples: 932674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:27:28,713][00304] Avg episode reward: [(0, '24.272')] +[2023-02-26 10:27:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3760128. Throughput: 0: 999.6. Samples: 939878. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:27:33,707][00304] Avg episode reward: [(0, '24.563')] +[2023-02-26 10:27:33,716][10798] Saving new best policy, reward=24.563! +[2023-02-26 10:27:36,126][10811] Updated weights for policy 0, policy_version 920 (0.0013) +[2023-02-26 10:27:38,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3772416. Throughput: 0: 977.3. Samples: 942478. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-26 10:27:38,707][00304] Avg episode reward: [(0, '23.800')] +[2023-02-26 10:27:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3788800. Throughput: 0: 952.3. Samples: 947046. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:27:43,713][00304] Avg episode reward: [(0, '24.264')] +[2023-02-26 10:27:47,684][10811] Updated weights for policy 0, policy_version 930 (0.0014) +[2023-02-26 10:27:48,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.4, 300 sec: 3832.2). Total num frames: 3813376. Throughput: 0: 990.2. Samples: 953348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:27:48,708][00304] Avg episode reward: [(0, '23.326')] +[2023-02-26 10:27:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3837952. Throughput: 0: 999.2. Samples: 956954. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-26 10:27:53,714][00304] Avg episode reward: [(0, '23.495')] +[2023-02-26 10:27:57,652][10811] Updated weights for policy 0, policy_version 940 (0.0017) +[2023-02-26 10:27:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 3850240. Throughput: 0: 965.6. Samples: 962616. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:27:58,708][00304] Avg episode reward: [(0, '24.289')] +[2023-02-26 10:28:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3866624. Throughput: 0: 946.7. Samples: 967158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:28:03,712][00304] Avg episode reward: [(0, '25.362')] +[2023-02-26 10:28:03,723][10798] Saving new best policy, reward=25.362! +[2023-02-26 10:28:08,620][10811] Updated weights for policy 0, policy_version 950 (0.0019) +[2023-02-26 10:28:08,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.5, 300 sec: 3832.2). Total num frames: 3891200. Throughput: 0: 965.6. Samples: 970206. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:28:08,713][00304] Avg episode reward: [(0, '24.897')] +[2023-02-26 10:28:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3915776. Throughput: 0: 995.4. Samples: 977466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-26 10:28:13,707][00304] Avg episode reward: [(0, '23.066')] +[2023-02-26 10:28:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000956_3915776.pth... +[2023-02-26 10:28:13,843][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth +[2023-02-26 10:28:18,705][00304] Fps is (10 sec: 3686.5, 60 sec: 3891.4, 300 sec: 3860.0). Total num frames: 3928064. Throughput: 0: 957.2. Samples: 982950. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-26 10:28:18,708][00304] Avg episode reward: [(0, '21.745')] +[2023-02-26 10:28:18,815][10811] Updated weights for policy 0, policy_version 960 (0.0022) +[2023-02-26 10:28:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3944448. Throughput: 0: 949.4. Samples: 985200. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:28:23,708][00304] Avg episode reward: [(0, '20.301')] +[2023-02-26 10:28:28,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 3960832. Throughput: 0: 948.6. Samples: 989732. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-26 10:28:28,711][00304] Avg episode reward: [(0, '19.722')] +[2023-02-26 10:28:32,462][10811] Updated weights for policy 0, policy_version 970 (0.0045) +[2023-02-26 10:28:33,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 3977216. Throughput: 0: 910.0. Samples: 994300. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-26 10:28:33,709][00304] Avg episode reward: [(0, '19.654')] +[2023-02-26 10:28:38,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 3989504. Throughput: 0: 879.4. Samples: 996528. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-26 10:28:38,707][00304] Avg episode reward: [(0, '19.283')] +[2023-02-26 10:28:43,262][10798] Stopping Batcher_0... +[2023-02-26 10:28:43,265][10798] Loop batcher_evt_loop terminating... +[2023-02-26 10:28:43,263][00304] Component Batcher_0 stopped! +[2023-02-26 10:28:43,272][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-26 10:28:43,345][00304] Component RolloutWorker_w5 stopped! +[2023-02-26 10:28:43,352][10818] Stopping RolloutWorker_w5... +[2023-02-26 10:28:43,353][10818] Loop rollout_proc5_evt_loop terminating... +[2023-02-26 10:28:43,375][10817] Stopping RolloutWorker_w4... +[2023-02-26 10:28:43,379][10817] Loop rollout_proc4_evt_loop terminating... +[2023-02-26 10:28:43,373][00304] Component RolloutWorker_w1 stopped! +[2023-02-26 10:28:43,380][00304] Component RolloutWorker_w4 stopped! +[2023-02-26 10:28:43,383][10813] Stopping RolloutWorker_w1... +[2023-02-26 10:28:43,385][10813] Loop rollout_proc1_evt_loop terminating... +[2023-02-26 10:28:43,393][10811] Weights refcount: 2 0 +[2023-02-26 10:28:43,411][10815] Stopping RolloutWorker_w0... +[2023-02-26 10:28:43,411][10815] Loop rollout_proc0_evt_loop terminating... +[2023-02-26 10:28:43,412][00304] Component RolloutWorker_w0 stopped! +[2023-02-26 10:28:43,417][10811] Stopping InferenceWorker_p0-w0... +[2023-02-26 10:28:43,418][10811] Loop inference_proc0-0_evt_loop terminating... +[2023-02-26 10:28:43,415][00304] Component RolloutWorker_w3 stopped! +[2023-02-26 10:28:43,411][10816] Stopping RolloutWorker_w3... +[2023-02-26 10:28:43,421][10816] Loop rollout_proc3_evt_loop terminating... +[2023-02-26 10:28:43,420][00304] Component InferenceWorker_p0-w0 stopped! +[2023-02-26 10:28:43,439][00304] Component RolloutWorker_w7 stopped! +[2023-02-26 10:28:43,444][10820] Stopping RolloutWorker_w7... +[2023-02-26 10:28:43,445][10820] Loop rollout_proc7_evt_loop terminating... +[2023-02-26 10:28:43,450][10814] Stopping RolloutWorker_w2... +[2023-02-26 10:28:43,450][10814] Loop rollout_proc2_evt_loop terminating... +[2023-02-26 10:28:43,455][10819] Stopping RolloutWorker_w6... +[2023-02-26 10:28:43,456][10819] Loop rollout_proc6_evt_loop terminating... +[2023-02-26 10:28:43,450][00304] Component RolloutWorker_w2 stopped! +[2023-02-26 10:28:43,456][00304] Component RolloutWorker_w6 stopped! +[2023-02-26 10:28:43,504][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000842_3448832.pth +[2023-02-26 10:28:43,525][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-26 10:28:43,815][00304] Component LearnerWorker_p0 stopped! +[2023-02-26 10:28:43,822][00304] Waiting for process learner_proc0 to stop... +[2023-02-26 10:28:43,826][10798] Stopping LearnerWorker_p0... +[2023-02-26 10:28:43,827][10798] Loop learner_proc0_evt_loop terminating... +[2023-02-26 10:28:45,715][00304] Waiting for process inference_proc0-0 to join... +[2023-02-26 10:28:46,135][00304] Waiting for process rollout_proc0 to join... +[2023-02-26 10:28:46,462][00304] Waiting for process rollout_proc1 to join... +[2023-02-26 10:28:46,464][00304] Waiting for process rollout_proc2 to join... +[2023-02-26 10:28:46,465][00304] Waiting for process rollout_proc3 to join... +[2023-02-26 10:28:46,466][00304] Waiting for process rollout_proc4 to join... +[2023-02-26 10:28:46,467][00304] Waiting for process rollout_proc5 to join... +[2023-02-26 10:28:46,469][00304] Waiting for process rollout_proc6 to join... +[2023-02-26 10:28:46,470][00304] Waiting for process rollout_proc7 to join... +[2023-02-26 10:28:46,471][00304] Batcher 0 profile tree view: +batching: 25.6426, releasing_batches: 0.0233 +[2023-02-26 10:28:46,473][00304] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 - wait_policy_total: 551.9726 -update_model: 7.9287 - weight_update: 0.0039 -one_step: 0.0123 - handle_policy_step: 553.2146 - deserialize: 15.3083, stack: 3.0778, obs_to_device_normalize: 118.8941, forward: 270.8024, send_messages: 27.3986 - prepare_outputs: 89.8224 - to_cpu: 56.1061 -[2023-02-25 13:57:30,454][00699] Learner 0 profile tree view: -misc: 0.0065, prepare_batch: 16.5891 -train: 77.8750 - epoch_init: 0.0121, minibatch_init: 0.0073, losses_postprocess: 0.5452, kl_divergence: 0.6278, after_optimizer: 32.9051 - calculate_losses: 28.1156 - losses_init: 0.0036, forward_head: 1.8022, bptt_initial: 18.4407, tail: 1.2613, advantages_returns: 0.3129, losses: 3.5919 - bptt: 2.2877 - bptt_forward_core: 2.2119 - update: 14.9934 - clip: 1.4682 -[2023-02-25 13:57:30,455][00699] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 0.3872, enqueue_policy_requests: 151.8462, env_step: 868.9891, overhead: 23.0898, complete_rollouts: 6.7247 -save_policy_outputs: 21.5644 - split_output_tensors: 10.6949 -[2023-02-25 13:57:30,457][00699] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.3382, enqueue_policy_requests: 157.7623, env_step: 863.9580, overhead: 23.0078, complete_rollouts: 8.1152 -save_policy_outputs: 21.0963 - split_output_tensors: 10.0864 -[2023-02-25 13:57:30,458][00699] Loop Runner_EvtLoop terminating... -[2023-02-25 13:57:30,460][00699] Runner profile tree view: -main_loop: 1179.6343 -[2023-02-25 13:57:30,461][00699] Collected {0: 4005888}, FPS: 3395.9 -[2023-02-25 13:57:30,516][00699] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-25 13:57:30,518][00699] Overriding arg 'num_workers' with value 1 passed from command line -[2023-02-25 13:57:30,520][00699] Adding new argument 'no_render'=True that is not in the saved config file! -[2023-02-25 13:57:30,521][00699] Adding new argument 'save_video'=True that is not in the saved config file! -[2023-02-25 13:57:30,523][00699] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2023-02-25 13:57:30,527][00699] Adding new argument 'video_name'=None that is not in the saved config file! -[2023-02-25 13:57:30,530][00699] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! -[2023-02-25 13:57:30,531][00699] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2023-02-25 13:57:30,533][00699] Adding new argument 'push_to_hub'=False that is not in the saved config file! -[2023-02-25 13:57:30,536][00699] Adding new argument 'hf_repository'=None that is not in the saved config file! -[2023-02-25 13:57:30,538][00699] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2023-02-25 13:57:30,540][00699] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2023-02-25 13:57:30,542][00699] Adding new argument 'train_script'=None that is not in the saved config file! -[2023-02-25 13:57:30,544][00699] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2023-02-25 13:57:30,546][00699] Using frameskip 1 and render_action_repeat=4 for evaluation -[2023-02-25 13:57:30,566][00699] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:57:30,567][00699] RunningMeanStd input shape: (1,) -[2023-02-25 13:57:30,584][00699] ConvEncoder: input_channels=3 -[2023-02-25 13:57:30,623][00699] Conv encoder output size: 512 -[2023-02-25 13:57:30,625][00699] Policy head output size: 512 -[2023-02-25 13:57:30,649][00699] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-25 13:57:32,046][00699] Num frames 100... -[2023-02-25 13:57:32,155][00699] Num frames 200... -[2023-02-25 13:57:32,265][00699] Num frames 300... -[2023-02-25 13:57:32,354][00699] Avg episode rewards: #0: 6.300, true rewards: #0: 3.300 -[2023-02-25 13:57:32,358][00699] Avg episode reward: 6.300, avg true_objective: 3.300 -[2023-02-25 13:57:32,439][00699] Num frames 400... -[2023-02-25 13:57:32,555][00699] Num frames 500... -[2023-02-25 13:57:32,669][00699] Num frames 600... -[2023-02-25 13:57:32,782][00699] Num frames 700... -[2023-02-25 13:57:32,898][00699] Num frames 800... -[2023-02-25 13:57:33,027][00699] Num frames 900... -[2023-02-25 13:57:33,141][00699] Num frames 1000... -[2023-02-25 13:57:33,308][00699] Avg episode rewards: #0: 9.490, true rewards: #0: 5.490 -[2023-02-25 13:57:33,310][00699] Avg episode reward: 9.490, avg true_objective: 5.490 -[2023-02-25 13:57:33,316][00699] Num frames 1100... -[2023-02-25 13:57:33,433][00699] Num frames 1200... -[2023-02-25 13:57:33,545][00699] Num frames 1300... -[2023-02-25 13:57:33,662][00699] Num frames 1400... -[2023-02-25 13:57:33,784][00699] Num frames 1500... -[2023-02-25 13:57:33,857][00699] Avg episode rewards: #0: 8.380, true rewards: #0: 5.047 -[2023-02-25 13:57:33,858][00699] Avg episode reward: 8.380, avg true_objective: 5.047 -[2023-02-25 13:57:33,957][00699] Num frames 1600... -[2023-02-25 13:57:34,076][00699] Num frames 1700... -[2023-02-25 13:57:34,194][00699] Num frames 1800... -[2023-02-25 13:57:34,308][00699] Num frames 1900... -[2023-02-25 13:57:34,427][00699] Num frames 2000... -[2023-02-25 13:57:34,539][00699] Num frames 2100... -[2023-02-25 13:57:34,652][00699] Num frames 2200... -[2023-02-25 13:57:34,765][00699] Avg episode rewards: #0: 10.375, true rewards: #0: 5.625 -[2023-02-25 13:57:34,767][00699] Avg episode reward: 10.375, avg true_objective: 5.625 -[2023-02-25 13:57:34,828][00699] Num frames 2300... -[2023-02-25 13:57:34,948][00699] Num frames 2400... -[2023-02-25 13:57:35,076][00699] Num frames 2500... -[2023-02-25 13:57:35,190][00699] Num frames 2600... -[2023-02-25 13:57:35,347][00699] Num frames 2700... -[2023-02-25 13:57:35,511][00699] Num frames 2800... -[2023-02-25 13:57:35,668][00699] Num frames 2900... -[2023-02-25 13:57:35,820][00699] Num frames 3000... -[2023-02-25 13:57:35,972][00699] Num frames 3100... -[2023-02-25 13:57:36,136][00699] Num frames 3200... -[2023-02-25 13:57:36,295][00699] Num frames 3300... -[2023-02-25 13:57:36,453][00699] Num frames 3400... -[2023-02-25 13:57:36,625][00699] Avg episode rewards: #0: 14.732, true rewards: #0: 6.932 -[2023-02-25 13:57:36,631][00699] Avg episode reward: 14.732, avg true_objective: 6.932 -[2023-02-25 13:57:36,698][00699] Num frames 3500... -[2023-02-25 13:57:36,866][00699] Num frames 3600... -[2023-02-25 13:57:37,029][00699] Num frames 3700... -[2023-02-25 13:57:37,204][00699] Num frames 3800... -[2023-02-25 13:57:37,361][00699] Num frames 3900... -[2023-02-25 13:57:37,532][00699] Num frames 4000... -[2023-02-25 13:57:37,691][00699] Num frames 4100... -[2023-02-25 13:57:37,850][00699] Num frames 4200... -[2023-02-25 13:57:38,007][00699] Num frames 4300... -[2023-02-25 13:57:38,175][00699] Num frames 4400... -[2023-02-25 13:57:38,333][00699] Num frames 4500... -[2023-02-25 13:57:38,495][00699] Num frames 4600... -[2023-02-25 13:57:38,654][00699] Num frames 4700... -[2023-02-25 13:57:38,784][00699] Num frames 4800... -[2023-02-25 13:57:38,896][00699] Num frames 4900... -[2023-02-25 13:57:39,014][00699] Num frames 5000... -[2023-02-25 13:57:39,126][00699] Num frames 5100... -[2023-02-25 13:57:39,247][00699] Num frames 5200... -[2023-02-25 13:57:39,368][00699] Num frames 5300... -[2023-02-25 13:57:39,448][00699] Avg episode rewards: #0: 19.703, true rewards: #0: 8.870 -[2023-02-25 13:57:39,451][00699] Avg episode reward: 19.703, avg true_objective: 8.870 -[2023-02-25 13:57:39,549][00699] Num frames 5400... -[2023-02-25 13:57:39,661][00699] Num frames 5500... -[2023-02-25 13:57:39,774][00699] Num frames 5600... -[2023-02-25 13:57:39,888][00699] Num frames 5700... -[2023-02-25 13:57:39,999][00699] Num frames 5800... -[2023-02-25 13:57:40,111][00699] Num frames 5900... -[2023-02-25 13:57:40,230][00699] Num frames 6000... -[2023-02-25 13:57:40,343][00699] Num frames 6100... -[2023-02-25 13:57:40,488][00699] Avg episode rewards: #0: 19.257, true rewards: #0: 8.829 -[2023-02-25 13:57:40,489][00699] Avg episode reward: 19.257, avg true_objective: 8.829 -[2023-02-25 13:57:40,515][00699] Num frames 6200... -[2023-02-25 13:57:40,630][00699] Num frames 6300... -[2023-02-25 13:57:40,746][00699] Num frames 6400... -[2023-02-25 13:57:40,860][00699] Num frames 6500... -[2023-02-25 13:57:40,983][00699] Num frames 6600... -[2023-02-25 13:57:41,075][00699] Avg episode rewards: #0: 17.660, true rewards: #0: 8.285 -[2023-02-25 13:57:41,077][00699] Avg episode reward: 17.660, avg true_objective: 8.285 -[2023-02-25 13:57:41,173][00699] Num frames 6700... -[2023-02-25 13:57:41,290][00699] Num frames 6800... -[2023-02-25 13:57:41,408][00699] Num frames 6900... -[2023-02-25 13:57:41,516][00699] Num frames 7000... -[2023-02-25 13:57:41,627][00699] Num frames 7100... -[2023-02-25 13:57:41,734][00699] Num frames 7200... -[2023-02-25 13:57:41,842][00699] Num frames 7300... -[2023-02-25 13:57:41,953][00699] Num frames 7400... -[2023-02-25 13:57:42,062][00699] Num frames 7500... -[2023-02-25 13:57:42,187][00699] Avg episode rewards: #0: 17.951, true rewards: #0: 8.396 -[2023-02-25 13:57:42,189][00699] Avg episode reward: 17.951, avg true_objective: 8.396 -[2023-02-25 13:57:42,251][00699] Num frames 7600... -[2023-02-25 13:57:42,364][00699] Num frames 7700... -[2023-02-25 13:57:42,479][00699] Num frames 7800... -[2023-02-25 13:57:42,590][00699] Num frames 7900... -[2023-02-25 13:57:42,701][00699] Num frames 8000... -[2023-02-25 13:57:42,813][00699] Num frames 8100... -[2023-02-25 13:57:42,905][00699] Avg episode rewards: #0: 17.032, true rewards: #0: 8.132 -[2023-02-25 13:57:42,907][00699] Avg episode reward: 17.032, avg true_objective: 8.132 -[2023-02-25 13:58:33,047][00699] Replay video saved to /content/train_dir/default_experiment/replay.mp4! -[2023-02-25 13:58:33,400][00699] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-25 13:58:33,403][00699] Overriding arg 'num_workers' with value 1 passed from command line -[2023-02-25 13:58:33,407][00699] Adding new argument 'no_render'=True that is not in the saved config file! -[2023-02-25 13:58:33,410][00699] Adding new argument 'save_video'=True that is not in the saved config file! -[2023-02-25 13:58:33,413][00699] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2023-02-25 13:58:33,415][00699] Adding new argument 'video_name'=None that is not in the saved config file! -[2023-02-25 13:58:33,418][00699] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! -[2023-02-25 13:58:33,420][00699] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2023-02-25 13:58:33,421][00699] Adding new argument 'push_to_hub'=True that is not in the saved config file! -[2023-02-25 13:58:33,426][00699] Adding new argument 'hf_repository'='RegisGraptin/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! -[2023-02-25 13:58:33,429][00699] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2023-02-25 13:58:33,431][00699] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2023-02-25 13:58:33,434][00699] Adding new argument 'train_script'=None that is not in the saved config file! -[2023-02-25 13:58:33,442][00699] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2023-02-25 13:58:33,446][00699] Using frameskip 1 and render_action_repeat=4 for evaluation -[2023-02-25 13:58:33,475][00699] RunningMeanStd input shape: (3, 72, 128) -[2023-02-25 13:58:33,478][00699] RunningMeanStd input shape: (1,) -[2023-02-25 13:58:33,505][00699] ConvEncoder: input_channels=3 -[2023-02-25 13:58:33,570][00699] Conv encoder output size: 512 -[2023-02-25 13:58:33,572][00699] Policy head output size: 512 -[2023-02-25 13:58:33,614][00699] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-25 13:58:34,388][00699] Num frames 100... -[2023-02-25 13:58:34,563][00699] Num frames 200... -[2023-02-25 13:58:34,750][00699] Num frames 300... -[2023-02-25 13:58:34,934][00699] Num frames 400... -[2023-02-25 13:58:35,091][00699] Avg episode rewards: #0: 7.480, true rewards: #0: 4.480 -[2023-02-25 13:58:35,093][00699] Avg episode reward: 7.480, avg true_objective: 4.480 -[2023-02-25 13:58:35,202][00699] Num frames 500... -[2023-02-25 13:58:35,390][00699] Num frames 600... -[2023-02-25 13:58:35,569][00699] Num frames 700... -[2023-02-25 13:58:35,752][00699] Num frames 800... -[2023-02-25 13:58:35,934][00699] Num frames 900... -[2023-02-25 13:58:36,112][00699] Num frames 1000... -[2023-02-25 13:58:36,215][00699] Avg episode rewards: #0: 10.120, true rewards: #0: 5.120 -[2023-02-25 13:58:36,218][00699] Avg episode reward: 10.120, avg true_objective: 5.120 -[2023-02-25 13:58:36,367][00699] Num frames 1100... -[2023-02-25 13:58:36,554][00699] Num frames 1200... -[2023-02-25 13:58:36,737][00699] Num frames 1300... -[2023-02-25 13:58:36,921][00699] Num frames 1400... -[2023-02-25 13:58:37,104][00699] Num frames 1500... -[2023-02-25 13:58:37,281][00699] Num frames 1600... -[2023-02-25 13:58:37,467][00699] Num frames 1700... -[2023-02-25 13:58:37,645][00699] Num frames 1800... -[2023-02-25 13:58:37,822][00699] Num frames 1900... -[2023-02-25 13:58:38,001][00699] Num frames 2000... -[2023-02-25 13:58:38,169][00699] Num frames 2100... -[2023-02-25 13:58:38,360][00699] Avg episode rewards: #0: 15.920, true rewards: #0: 7.253 -[2023-02-25 13:58:38,362][00699] Avg episode reward: 15.920, avg true_objective: 7.253 -[2023-02-25 13:58:38,415][00699] Num frames 2200... -[2023-02-25 13:58:38,599][00699] Num frames 2300... -[2023-02-25 13:58:38,788][00699] Num frames 2400... -[2023-02-25 13:58:38,977][00699] Num frames 2500... -[2023-02-25 13:58:39,168][00699] Num frames 2600... -[2023-02-25 13:58:39,348][00699] Num frames 2700... -[2023-02-25 13:58:39,532][00699] Num frames 2800... -[2023-02-25 13:58:39,725][00699] Num frames 2900... -[2023-02-25 13:58:39,922][00699] Num frames 3000... -[2023-02-25 13:58:40,124][00699] Num frames 3100... -[2023-02-25 13:58:40,302][00699] Num frames 3200... -[2023-02-25 13:58:40,476][00699] Num frames 3300... -[2023-02-25 13:58:40,664][00699] Num frames 3400... -[2023-02-25 13:58:40,883][00699] Avg episode rewards: #0: 20.220, true rewards: #0: 8.720 -[2023-02-25 13:58:40,884][00699] Avg episode reward: 20.220, avg true_objective: 8.720 -[2023-02-25 13:58:40,912][00699] Num frames 3500... -[2023-02-25 13:58:41,104][00699] Num frames 3600... -[2023-02-25 13:58:41,299][00699] Num frames 3700... -[2023-02-25 13:58:41,501][00699] Num frames 3800... -[2023-02-25 13:58:41,675][00699] Num frames 3900... -[2023-02-25 13:58:41,833][00699] Num frames 4000... -[2023-02-25 13:58:41,985][00699] Num frames 4100... -[2023-02-25 13:58:42,125][00699] Num frames 4200... -[2023-02-25 13:58:42,238][00699] Num frames 4300... -[2023-02-25 13:58:42,355][00699] Num frames 4400... -[2023-02-25 13:58:42,414][00699] Avg episode rewards: #0: 20.002, true rewards: #0: 8.802 -[2023-02-25 13:58:42,416][00699] Avg episode reward: 20.002, avg true_objective: 8.802 -[2023-02-25 13:58:42,531][00699] Num frames 4500... -[2023-02-25 13:58:42,658][00699] Num frames 4600... -[2023-02-25 13:58:42,772][00699] Num frames 4700... -[2023-02-25 13:58:42,883][00699] Num frames 4800... -[2023-02-25 13:58:42,959][00699] Avg episode rewards: #0: 17.528, true rewards: #0: 8.028 -[2023-02-25 13:58:42,960][00699] Avg episode reward: 17.528, avg true_objective: 8.028 -[2023-02-25 13:58:43,059][00699] Num frames 4900... -[2023-02-25 13:58:43,174][00699] Num frames 5000... -[2023-02-25 13:58:43,284][00699] Num frames 5100... -[2023-02-25 13:58:43,396][00699] Num frames 5200... -[2023-02-25 13:58:43,506][00699] Num frames 5300... -[2023-02-25 13:58:43,616][00699] Num frames 5400... -[2023-02-25 13:58:43,726][00699] Num frames 5500... -[2023-02-25 13:58:43,834][00699] Num frames 5600... -[2023-02-25 13:58:43,944][00699] Num frames 5700... -[2023-02-25 13:58:44,015][00699] Avg episode rewards: #0: 17.590, true rewards: #0: 8.161 -[2023-02-25 13:58:44,017][00699] Avg episode reward: 17.590, avg true_objective: 8.161 -[2023-02-25 13:58:44,128][00699] Num frames 5800... -[2023-02-25 13:58:44,241][00699] Num frames 5900... -[2023-02-25 13:58:44,351][00699] Num frames 6000... -[2023-02-25 13:58:44,463][00699] Num frames 6100... -[2023-02-25 13:58:44,574][00699] Num frames 6200... -[2023-02-25 13:58:44,659][00699] Avg episode rewards: #0: 16.406, true rewards: #0: 7.781 -[2023-02-25 13:58:44,661][00699] Avg episode reward: 16.406, avg true_objective: 7.781 -[2023-02-25 13:58:44,750][00699] Num frames 6300... -[2023-02-25 13:58:44,871][00699] Num frames 6400... -[2023-02-25 13:58:44,986][00699] Num frames 6500... -[2023-02-25 13:58:45,106][00699] Num frames 6600... -[2023-02-25 13:58:45,222][00699] Num frames 6700... -[2023-02-25 13:58:45,332][00699] Num frames 6800... -[2023-02-25 13:58:45,442][00699] Num frames 6900... -[2023-02-25 13:58:45,554][00699] Num frames 7000... -[2023-02-25 13:58:45,667][00699] Num frames 7100... -[2023-02-25 13:58:45,777][00699] Num frames 7200... -[2023-02-25 13:58:45,889][00699] Num frames 7300... -[2023-02-25 13:58:46,000][00699] Num frames 7400... -[2023-02-25 13:58:46,070][00699] Avg episode rewards: #0: 17.232, true rewards: #0: 8.232 -[2023-02-25 13:58:46,072][00699] Avg episode reward: 17.232, avg true_objective: 8.232 -[2023-02-25 13:58:46,185][00699] Num frames 7500... -[2023-02-25 13:58:46,300][00699] Num frames 7600... -[2023-02-25 13:58:46,407][00699] Num frames 7700... -[2023-02-25 13:58:46,517][00699] Num frames 7800... -[2023-02-25 13:58:46,628][00699] Num frames 7900... -[2023-02-25 13:58:46,742][00699] Num frames 8000... -[2023-02-25 13:58:46,854][00699] Num frames 8100... -[2023-02-25 13:58:46,956][00699] Avg episode rewards: #0: 17.043, true rewards: #0: 8.143 -[2023-02-25 13:58:46,957][00699] Avg episode reward: 17.043, avg true_objective: 8.143 -[2023-02-25 13:59:37,297][00699] Replay video saved to /content/train_dir/default_experiment/replay.mp4! + wait_policy_total: 515.4273 +update_model: 7.7167 + weight_update: 0.0020 +one_step: 0.0160 + handle_policy_step: 512.5966 + deserialize: 14.5257, stack: 3.1535, obs_to_device_normalize: 113.7392, forward: 246.9027, send_messages: 26.2634 + prepare_outputs: 82.6558 + to_cpu: 51.8260 +[2023-02-26 10:28:46,474][00304] Learner 0 profile tree view: +misc: 0.0054, prepare_batch: 15.6405 +train: 75.6909 + epoch_init: 0.0126, minibatch_init: 0.0087, losses_postprocess: 0.6636, kl_divergence: 0.5817, after_optimizer: 33.0682 + calculate_losses: 26.6977 + losses_init: 0.0033, forward_head: 1.7413, bptt_initial: 17.7023, tail: 1.0487, advantages_returns: 0.3586, losses: 3.4147 + bptt: 2.1564 + bptt_forward_core: 2.0685 + update: 14.0975 + clip: 1.3532 +[2023-02-26 10:28:46,475][00304] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.2947, enqueue_policy_requests: 138.2589, env_step: 812.3938, overhead: 20.3042, complete_rollouts: 6.7712 +save_policy_outputs: 19.4772 + split_output_tensors: 9.6457 +[2023-02-26 10:28:46,479][00304] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.3469, enqueue_policy_requests: 137.8682, env_step: 812.1064, overhead: 19.4119, complete_rollouts: 6.7750 +save_policy_outputs: 19.7513 + split_output_tensors: 9.5300 +[2023-02-26 10:28:46,480][00304] Loop Runner_EvtLoop terminating... +[2023-02-26 10:28:46,482][00304] Runner profile tree view: +main_loop: 1107.7311 +[2023-02-26 10:28:46,485][00304] Collected {0: 4005888}, FPS: 3616.3 +[2023-02-26 10:28:46,639][00304] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2023-02-26 10:28:46,645][00304] Overriding arg 'num_workers' with value 1 passed from command line +[2023-02-26 10:28:46,647][00304] Adding new argument 'no_render'=True that is not in the saved config file! +[2023-02-26 10:28:46,648][00304] Adding new argument 'save_video'=True that is not in the saved config file! +[2023-02-26 10:28:46,650][00304] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2023-02-26 10:28:46,651][00304] Adding new argument 'video_name'=None that is not in the saved config file! +[2023-02-26 10:28:46,652][00304] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2023-02-26 10:28:46,653][00304] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2023-02-26 10:28:46,656][00304] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2023-02-26 10:28:46,658][00304] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2023-02-26 10:28:46,659][00304] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2023-02-26 10:28:46,661][00304] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2023-02-26 10:28:46,662][00304] Adding new argument 'train_script'=None that is not in the saved config file! +[2023-02-26 10:28:46,664][00304] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2023-02-26 10:28:46,666][00304] Using frameskip 1 and render_action_repeat=4 for evaluation +[2023-02-26 10:28:46,698][00304] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-26 10:28:46,701][00304] RunningMeanStd input shape: (3, 72, 128) +[2023-02-26 10:28:46,703][00304] RunningMeanStd input shape: (1,) +[2023-02-26 10:28:46,720][00304] ConvEncoder: input_channels=3 +[2023-02-26 10:28:47,377][00304] Conv encoder output size: 512 +[2023-02-26 10:28:47,379][00304] Policy head output size: 512 +[2023-02-26 10:28:49,785][00304] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-26 10:28:51,033][00304] Num frames 100... +[2023-02-26 10:28:51,153][00304] Num frames 200... +[2023-02-26 10:28:51,276][00304] Num frames 300... +[2023-02-26 10:28:51,388][00304] Num frames 400... +[2023-02-26 10:28:51,502][00304] Num frames 500... +[2023-02-26 10:28:51,621][00304] Num frames 600... +[2023-02-26 10:28:51,734][00304] Num frames 700... +[2023-02-26 10:28:51,853][00304] Num frames 800... +[2023-02-26 10:28:51,967][00304] Num frames 900... +[2023-02-26 10:28:52,128][00304] Avg episode rewards: #0: 24.920, true rewards: #0: 9.920 +[2023-02-26 10:28:52,130][00304] Avg episode reward: 24.920, avg true_objective: 9.920 +[2023-02-26 10:28:52,143][00304] Num frames 1000... +[2023-02-26 10:28:52,261][00304] Num frames 1100... +[2023-02-26 10:28:52,385][00304] Num frames 1200... +[2023-02-26 10:28:52,496][00304] Num frames 1300... +[2023-02-26 10:28:52,621][00304] Num frames 1400... +[2023-02-26 10:28:52,736][00304] Num frames 1500... +[2023-02-26 10:28:52,798][00304] Avg episode rewards: #0: 17.520, true rewards: #0: 7.520 +[2023-02-26 10:28:52,800][00304] Avg episode reward: 17.520, avg true_objective: 7.520 +[2023-02-26 10:28:52,908][00304] Num frames 1600... +[2023-02-26 10:28:53,016][00304] Num frames 1700... +[2023-02-26 10:28:53,135][00304] Num frames 1800... +[2023-02-26 10:28:53,256][00304] Num frames 1900... +[2023-02-26 10:28:53,373][00304] Num frames 2000... +[2023-02-26 10:28:53,488][00304] Num frames 2100... +[2023-02-26 10:28:53,608][00304] Num frames 2200... +[2023-02-26 10:28:53,723][00304] Num frames 2300... +[2023-02-26 10:28:53,846][00304] Num frames 2400... +[2023-02-26 10:28:53,959][00304] Num frames 2500... +[2023-02-26 10:28:54,082][00304] Num frames 2600... +[2023-02-26 10:28:54,247][00304] Num frames 2700... +[2023-02-26 10:28:54,446][00304] Avg episode rewards: #0: 23.280, true rewards: #0: 9.280 +[2023-02-26 10:28:54,448][00304] Avg episode reward: 23.280, avg true_objective: 9.280 +[2023-02-26 10:28:54,475][00304] Num frames 2800... +[2023-02-26 10:28:54,637][00304] Num frames 2900... +[2023-02-26 10:28:54,797][00304] Num frames 3000... +[2023-02-26 10:28:54,954][00304] Num frames 3100... +[2023-02-26 10:28:55,115][00304] Num frames 3200... +[2023-02-26 10:28:55,279][00304] Num frames 3300... +[2023-02-26 10:28:55,455][00304] Num frames 3400... +[2023-02-26 10:28:55,620][00304] Num frames 3500... +[2023-02-26 10:28:55,821][00304] Avg episode rewards: #0: 21.960, true rewards: #0: 8.960 +[2023-02-26 10:28:55,825][00304] Avg episode reward: 21.960, avg true_objective: 8.960 +[2023-02-26 10:28:55,864][00304] Num frames 3600... +[2023-02-26 10:28:56,036][00304] Num frames 3700... +[2023-02-26 10:28:56,206][00304] Num frames 3800... +[2023-02-26 10:28:56,366][00304] Num frames 3900... +[2023-02-26 10:28:56,529][00304] Num frames 4000... +[2023-02-26 10:28:56,691][00304] Num frames 4100... +[2023-02-26 10:28:56,851][00304] Num frames 4200... +[2023-02-26 10:28:57,006][00304] Num frames 4300... +[2023-02-26 10:28:57,174][00304] Num frames 4400... +[2023-02-26 10:28:57,335][00304] Num frames 4500... +[2023-02-26 10:28:57,499][00304] Num frames 4600... +[2023-02-26 10:28:57,667][00304] Num frames 4700... +[2023-02-26 10:28:57,783][00304] Num frames 4800... +[2023-02-26 10:28:57,892][00304] Avg episode rewards: #0: 23.076, true rewards: #0: 9.676 +[2023-02-26 10:28:57,895][00304] Avg episode reward: 23.076, avg true_objective: 9.676 +[2023-02-26 10:28:57,964][00304] Num frames 4900... +[2023-02-26 10:28:58,082][00304] Num frames 5000... +[2023-02-26 10:28:58,210][00304] Num frames 5100... +[2023-02-26 10:28:58,333][00304] Num frames 5200... +[2023-02-26 10:28:58,450][00304] Avg episode rewards: #0: 20.257, true rewards: #0: 8.757 +[2023-02-26 10:28:58,452][00304] Avg episode reward: 20.257, avg true_objective: 8.757 +[2023-02-26 10:28:58,508][00304] Num frames 5300... +[2023-02-26 10:28:58,629][00304] Num frames 5400... +[2023-02-26 10:28:58,745][00304] Num frames 5500... +[2023-02-26 10:28:58,864][00304] Num frames 5600... +[2023-02-26 10:28:58,977][00304] Num frames 5700... +[2023-02-26 10:28:59,071][00304] Avg episode rewards: #0: 18.334, true rewards: #0: 8.191 +[2023-02-26 10:28:59,072][00304] Avg episode reward: 18.334, avg true_objective: 8.191 +[2023-02-26 10:28:59,149][00304] Num frames 5800... +[2023-02-26 10:28:59,269][00304] Num frames 5900... +[2023-02-26 10:28:59,427][00304] Avg episode rewards: #0: 16.363, true rewards: #0: 7.487 +[2023-02-26 10:28:59,429][00304] Avg episode reward: 16.363, avg true_objective: 7.487 +[2023-02-26 10:28:59,449][00304] Num frames 6000... +[2023-02-26 10:28:59,563][00304] Num frames 6100... +[2023-02-26 10:28:59,684][00304] Num frames 6200... +[2023-02-26 10:28:59,800][00304] Num frames 6300... +[2023-02-26 10:28:59,919][00304] Num frames 6400... +[2023-02-26 10:29:00,033][00304] Num frames 6500... +[2023-02-26 10:29:00,152][00304] Num frames 6600... +[2023-02-26 10:29:00,267][00304] Num frames 6700... +[2023-02-26 10:29:00,384][00304] Num frames 6800... +[2023-02-26 10:29:00,504][00304] Num frames 6900... +[2023-02-26 10:29:00,622][00304] Num frames 7000... +[2023-02-26 10:29:00,738][00304] Num frames 7100... +[2023-02-26 10:29:00,851][00304] Num frames 7200... +[2023-02-26 10:29:00,969][00304] Num frames 7300... +[2023-02-26 10:29:01,082][00304] Num frames 7400... +[2023-02-26 10:29:01,203][00304] Num frames 7500... +[2023-02-26 10:29:01,319][00304] Num frames 7600... +[2023-02-26 10:29:01,436][00304] Num frames 7700... +[2023-02-26 10:29:01,555][00304] Num frames 7800... +[2023-02-26 10:29:01,679][00304] Num frames 7900... +[2023-02-26 10:29:01,769][00304] Avg episode rewards: #0: 19.477, true rewards: #0: 8.810 +[2023-02-26 10:29:01,771][00304] Avg episode reward: 19.477, avg true_objective: 8.810 +[2023-02-26 10:29:01,854][00304] Num frames 8000... +[2023-02-26 10:29:01,973][00304] Num frames 8100... +[2023-02-26 10:29:02,086][00304] Num frames 8200... +[2023-02-26 10:29:02,208][00304] Num frames 8300... +[2023-02-26 10:29:02,325][00304] Num frames 8400... +[2023-02-26 10:29:02,441][00304] Num frames 8500... +[2023-02-26 10:29:02,563][00304] Num frames 8600... +[2023-02-26 10:29:02,684][00304] Num frames 8700... +[2023-02-26 10:29:02,802][00304] Num frames 8800... +[2023-02-26 10:29:02,924][00304] Num frames 8900... +[2023-02-26 10:29:03,042][00304] Num frames 9000... +[2023-02-26 10:29:03,120][00304] Avg episode rewards: #0: 19.717, true rewards: #0: 9.017 +[2023-02-26 10:29:03,122][00304] Avg episode reward: 19.717, avg true_objective: 9.017 +[2023-02-26 10:29:56,992][00304] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2023-02-26 10:29:57,292][00304] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2023-02-26 10:29:57,299][00304] Overriding arg 'num_workers' with value 1 passed from command line +[2023-02-26 10:29:57,300][00304] Adding new argument 'no_render'=True that is not in the saved config file! +[2023-02-26 10:29:57,301][00304] Adding new argument 'save_video'=True that is not in the saved config file! +[2023-02-26 10:29:57,302][00304] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2023-02-26 10:29:57,304][00304] Adding new argument 'video_name'=None that is not in the saved config file! +[2023-02-26 10:29:57,305][00304] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2023-02-26 10:29:57,306][00304] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2023-02-26 10:29:57,307][00304] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2023-02-26 10:29:57,308][00304] Adding new argument 'hf_repository'='RegisGraptin/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2023-02-26 10:29:57,310][00304] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2023-02-26 10:29:57,311][00304] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2023-02-26 10:29:57,312][00304] Adding new argument 'train_script'=None that is not in the saved config file! +[2023-02-26 10:29:57,313][00304] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2023-02-26 10:29:57,314][00304] Using frameskip 1 and render_action_repeat=4 for evaluation +[2023-02-26 10:29:57,341][00304] RunningMeanStd input shape: (3, 72, 128) +[2023-02-26 10:29:57,350][00304] RunningMeanStd input shape: (1,) +[2023-02-26 10:29:57,368][00304] ConvEncoder: input_channels=3 +[2023-02-26 10:29:57,425][00304] Conv encoder output size: 512 +[2023-02-26 10:29:57,427][00304] Policy head output size: 512 +[2023-02-26 10:29:57,457][00304] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-26 10:29:58,143][00304] Num frames 100... +[2023-02-26 10:29:58,302][00304] Num frames 200... +[2023-02-26 10:29:58,453][00304] Num frames 300... +[2023-02-26 10:29:58,601][00304] Num frames 400... +[2023-02-26 10:29:58,750][00304] Num frames 500... +[2023-02-26 10:29:58,898][00304] Num frames 600... +[2023-02-26 10:29:59,061][00304] Num frames 700... +[2023-02-26 10:29:59,232][00304] Num frames 800... +[2023-02-26 10:29:59,382][00304] Num frames 900... +[2023-02-26 10:29:59,531][00304] Num frames 1000... +[2023-02-26 10:29:59,689][00304] Num frames 1100... +[2023-02-26 10:29:59,832][00304] Avg episode rewards: #0: 26.520, true rewards: #0: 11.520 +[2023-02-26 10:29:59,834][00304] Avg episode reward: 26.520, avg true_objective: 11.520 +[2023-02-26 10:29:59,907][00304] Num frames 1200... +[2023-02-26 10:30:00,079][00304] Num frames 1300... +[2023-02-26 10:30:00,242][00304] Num frames 1400... +[2023-02-26 10:30:00,405][00304] Num frames 1500... +[2023-02-26 10:30:00,579][00304] Num frames 1600... +[2023-02-26 10:30:00,772][00304] Num frames 1700... +[2023-02-26 10:30:00,954][00304] Num frames 1800... +[2023-02-26 10:30:01,134][00304] Num frames 1900... +[2023-02-26 10:30:01,303][00304] Num frames 2000... +[2023-02-26 10:30:01,463][00304] Num frames 2100... +[2023-02-26 10:30:01,630][00304] Num frames 2200... +[2023-02-26 10:30:01,802][00304] Num frames 2300... +[2023-02-26 10:30:01,957][00304] Num frames 2400... +[2023-02-26 10:30:02,116][00304] Num frames 2500... +[2023-02-26 10:30:02,272][00304] Num frames 2600... +[2023-02-26 10:30:02,410][00304] Avg episode rewards: #0: 30.365, true rewards: #0: 13.365 +[2023-02-26 10:30:02,412][00304] Avg episode reward: 30.365, avg true_objective: 13.365 +[2023-02-26 10:30:02,445][00304] Num frames 2700... +[2023-02-26 10:30:02,560][00304] Num frames 2800... +[2023-02-26 10:30:02,672][00304] Num frames 2900... +[2023-02-26 10:30:02,783][00304] Num frames 3000... +[2023-02-26 10:30:02,904][00304] Num frames 3100... +[2023-02-26 10:30:03,015][00304] Num frames 3200... +[2023-02-26 10:30:03,134][00304] Num frames 3300... +[2023-02-26 10:30:03,256][00304] Num frames 3400... +[2023-02-26 10:30:03,371][00304] Num frames 3500... +[2023-02-26 10:30:03,483][00304] Num frames 3600... +[2023-02-26 10:30:03,603][00304] Num frames 3700... +[2023-02-26 10:30:03,721][00304] Num frames 3800... +[2023-02-26 10:30:03,831][00304] Num frames 3900... +[2023-02-26 10:30:03,992][00304] Num frames 4000... +[2023-02-26 10:30:04,156][00304] Num frames 4100... +[2023-02-26 10:30:04,324][00304] Num frames 4200... +[2023-02-26 10:30:04,493][00304] Num frames 4300... +[2023-02-26 10:30:04,659][00304] Num frames 4400... +[2023-02-26 10:30:04,819][00304] Num frames 4500... +[2023-02-26 10:30:04,992][00304] Num frames 4600... +[2023-02-26 10:30:05,169][00304] Num frames 4700... +[2023-02-26 10:30:05,282][00304] Avg episode rewards: #0: 37.436, true rewards: #0: 15.770 +[2023-02-26 10:30:05,284][00304] Avg episode reward: 37.436, avg true_objective: 15.770 +[2023-02-26 10:30:05,392][00304] Num frames 4800... +[2023-02-26 10:30:05,556][00304] Num frames 4900... +[2023-02-26 10:30:05,718][00304] Num frames 5000... +[2023-02-26 10:30:05,874][00304] Num frames 5100... +[2023-02-26 10:30:06,034][00304] Num frames 5200... +[2023-02-26 10:30:06,202][00304] Num frames 5300... +[2023-02-26 10:30:06,380][00304] Num frames 5400... +[2023-02-26 10:30:06,550][00304] Num frames 5500... +[2023-02-26 10:30:06,714][00304] Num frames 5600... +[2023-02-26 10:30:06,888][00304] Num frames 5700... +[2023-02-26 10:30:07,052][00304] Num frames 5800... +[2023-02-26 10:30:07,216][00304] Num frames 5900... +[2023-02-26 10:30:07,390][00304] Num frames 6000... +[2023-02-26 10:30:07,506][00304] Num frames 6100... +[2023-02-26 10:30:07,629][00304] Num frames 6200... +[2023-02-26 10:30:07,744][00304] Num frames 6300... +[2023-02-26 10:30:07,858][00304] Num frames 6400... +[2023-02-26 10:30:07,972][00304] Num frames 6500... +[2023-02-26 10:30:08,089][00304] Num frames 6600... +[2023-02-26 10:30:08,210][00304] Num frames 6700... +[2023-02-26 10:30:08,329][00304] Num frames 6800... +[2023-02-26 10:30:08,421][00304] Avg episode rewards: #0: 41.577, true rewards: #0: 17.078 +[2023-02-26 10:30:08,423][00304] Avg episode reward: 41.577, avg true_objective: 17.078 +[2023-02-26 10:30:08,504][00304] Num frames 6900... +[2023-02-26 10:30:08,619][00304] Num frames 7000... +[2023-02-26 10:30:08,732][00304] Num frames 7100... +[2023-02-26 10:30:08,844][00304] Num frames 7200... +[2023-02-26 10:30:08,957][00304] Num frames 7300... +[2023-02-26 10:30:09,070][00304] Num frames 7400... +[2023-02-26 10:30:09,185][00304] Num frames 7500... +[2023-02-26 10:30:09,297][00304] Num frames 7600... +[2023-02-26 10:30:09,414][00304] Num frames 7700... +[2023-02-26 10:30:09,525][00304] Num frames 7800... +[2023-02-26 10:30:09,679][00304] Avg episode rewards: #0: 37.974, true rewards: #0: 15.774 +[2023-02-26 10:30:09,681][00304] Avg episode reward: 37.974, avg true_objective: 15.774 +[2023-02-26 10:30:09,699][00304] Num frames 7900... +[2023-02-26 10:30:09,808][00304] Num frames 8000... +[2023-02-26 10:30:09,918][00304] Num frames 8100... +[2023-02-26 10:30:10,030][00304] Num frames 8200... +[2023-02-26 10:30:10,143][00304] Num frames 8300... +[2023-02-26 10:30:10,256][00304] Num frames 8400... +[2023-02-26 10:30:10,376][00304] Num frames 8500... +[2023-02-26 10:30:10,488][00304] Num frames 8600... +[2023-02-26 10:30:10,601][00304] Num frames 8700... +[2023-02-26 10:30:10,727][00304] Avg episode rewards: #0: 34.936, true rewards: #0: 14.603 +[2023-02-26 10:30:10,729][00304] Avg episode reward: 34.936, avg true_objective: 14.603 +[2023-02-26 10:30:10,772][00304] Num frames 8800... +[2023-02-26 10:30:10,886][00304] Num frames 8900... +[2023-02-26 10:30:10,998][00304] Num frames 9000... +[2023-02-26 10:30:11,111][00304] Num frames 9100... +[2023-02-26 10:30:11,229][00304] Num frames 9200... +[2023-02-26 10:30:11,340][00304] Num frames 9300... +[2023-02-26 10:30:11,457][00304] Num frames 9400... +[2023-02-26 10:30:11,569][00304] Num frames 9500... +[2023-02-26 10:30:11,685][00304] Num frames 9600... +[2023-02-26 10:30:11,798][00304] Num frames 9700... +[2023-02-26 10:30:11,910][00304] Num frames 9800... +[2023-02-26 10:30:12,020][00304] Num frames 9900... +[2023-02-26 10:30:12,135][00304] Num frames 10000... +[2023-02-26 10:30:12,250][00304] Num frames 10100... +[2023-02-26 10:30:12,364][00304] Num frames 10200... +[2023-02-26 10:30:12,485][00304] Num frames 10300... +[2023-02-26 10:30:12,607][00304] Avg episode rewards: #0: 35.510, true rewards: #0: 14.796 +[2023-02-26 10:30:12,609][00304] Avg episode reward: 35.510, avg true_objective: 14.796 +[2023-02-26 10:30:12,659][00304] Num frames 10400... +[2023-02-26 10:30:12,770][00304] Num frames 10500... +[2023-02-26 10:30:12,882][00304] Num frames 10600... +[2023-02-26 10:30:12,993][00304] Num frames 10700... +[2023-02-26 10:30:13,104][00304] Num frames 10800... +[2023-02-26 10:30:13,220][00304] Num frames 10900... +[2023-02-26 10:30:13,337][00304] Num frames 11000... +[2023-02-26 10:30:13,456][00304] Num frames 11100... +[2023-02-26 10:30:13,567][00304] Num frames 11200... +[2023-02-26 10:30:13,680][00304] Num frames 11300... +[2023-02-26 10:30:13,791][00304] Num frames 11400... +[2023-02-26 10:30:13,912][00304] Num frames 11500... +[2023-02-26 10:30:14,023][00304] Num frames 11600... +[2023-02-26 10:30:14,135][00304] Num frames 11700... +[2023-02-26 10:30:14,194][00304] Avg episode rewards: #0: 34.501, true rewards: #0: 14.626 +[2023-02-26 10:30:14,196][00304] Avg episode reward: 34.501, avg true_objective: 14.626 +[2023-02-26 10:30:14,313][00304] Num frames 11800... +[2023-02-26 10:30:14,434][00304] Num frames 11900... +[2023-02-26 10:30:14,547][00304] Num frames 12000... +[2023-02-26 10:30:14,663][00304] Num frames 12100... +[2023-02-26 10:30:14,775][00304] Num frames 12200... +[2023-02-26 10:30:14,888][00304] Num frames 12300... +[2023-02-26 10:30:14,999][00304] Num frames 12400... +[2023-02-26 10:30:15,062][00304] Avg episode rewards: #0: 32.561, true rewards: #0: 13.783 +[2023-02-26 10:30:15,063][00304] Avg episode reward: 32.561, avg true_objective: 13.783 +[2023-02-26 10:30:15,180][00304] Num frames 12500... +[2023-02-26 10:30:15,296][00304] Num frames 12600... +[2023-02-26 10:30:15,408][00304] Num frames 12700... +[2023-02-26 10:30:15,528][00304] Num frames 12800... +[2023-02-26 10:30:15,643][00304] Num frames 12900... +[2023-02-26 10:30:15,754][00304] Num frames 13000... +[2023-02-26 10:30:15,864][00304] Num frames 13100... +[2023-02-26 10:30:15,975][00304] Num frames 13200... +[2023-02-26 10:30:16,087][00304] Num frames 13300... +[2023-02-26 10:30:16,206][00304] Num frames 13400... +[2023-02-26 10:30:16,300][00304] Avg episode rewards: #0: 31.229, true rewards: #0: 13.429 +[2023-02-26 10:30:16,301][00304] Avg episode reward: 31.229, avg true_objective: 13.429 +[2023-02-26 10:31:38,407][00304] Replay video saved to /content/train_dir/default_experiment/replay.mp4!