| [2024-05-27 23:16:03,598][1934158] Saving configuration to /media/fast/code/learning/train_dir/default_experiment/config.json... | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 0 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 1 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 2 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 3 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 4 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 5 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 6 uses device cpu | |
| [2024-05-27 23:16:03,599][1934158] Rollout worker 7 uses device cpu | |
| [2024-05-27 23:16:03,623][1934158] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:16:03,623][1934158] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-05-27 23:16:03,635][1934158] Starting all processes... | |
| [2024-05-27 23:16:03,635][1934158] Starting process learner_proc0 | |
| [2024-05-27 23:16:05,222][1934158] Starting all processes... | |
| [2024-05-27 23:16:05,225][1934158] Starting process inference_proc0-0 | |
| [2024-05-27 23:16:05,225][1934158] Starting process rollout_proc0 | |
| [2024-05-27 23:16:05,225][1934158] Starting process rollout_proc1 | |
| [2024-05-27 23:16:05,225][1934270] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:16:05,225][1934270] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-05-27 23:16:05,225][1934158] Starting process rollout_proc2 | |
| [2024-05-27 23:16:05,225][1934158] Starting process rollout_proc3 | |
| [2024-05-27 23:16:05,227][1934158] Starting process rollout_proc4 | |
| [2024-05-27 23:16:05,228][1934158] Starting process rollout_proc5 | |
| [2024-05-27 23:16:05,235][1934270] Num visible devices: 1 | |
| [2024-05-27 23:16:05,228][1934158] Starting process rollout_proc6 | |
| [2024-05-27 23:16:05,231][1934158] Starting process rollout_proc7 | |
| [2024-05-27 23:16:05,252][1934270] Starting seed is not provided | |
| [2024-05-27 23:16:05,252][1934270] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:16:05,252][1934270] Initializing actor-critic model on device cuda:0 | |
| [2024-05-27 23:16:05,252][1934270] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:16:05,253][1934270] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:16:05,265][1934270] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:16:05,383][1934270] Conv encoder output size: 512 | |
| [2024-05-27 23:16:05,383][1934270] Policy head output size: 512 | |
| [2024-05-27 23:16:05,396][1934270] Created Actor Critic model with architecture: | |
| [2024-05-27 23:16:05,396][1934270] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-05-27 23:16:05,572][1934270] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-05-27 23:16:06,494][1934270] No checkpoints found | |
| [2024-05-27 23:16:06,494][1934270] Did not load from checkpoint, starting from scratch! | |
| [2024-05-27 23:16:06,497][1934270] Initialized policy 0 weights for model version 0 | |
| [2024-05-27 23:16:06,525][1934270] LearnerWorker_p0 finished initialization! | |
| [2024-05-27 23:16:06,525][1934270] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:16:09,677][1934319] Worker 3 uses CPU cores [6, 7] | |
| [2024-05-27 23:16:09,679][1934318] Worker 2 uses CPU cores [4, 5] | |
| [2024-05-27 23:16:09,679][1934337] Worker 6 uses CPU cores [12, 13] | |
| [2024-05-27 23:16:09,691][1934315] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:16:09,691][1934315] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-05-27 23:16:09,696][1934316] Worker 0 uses CPU cores [0, 1] | |
| [2024-05-27 23:16:09,715][1934315] Num visible devices: 1 | |
| [2024-05-27 23:16:09,755][1934320] Worker 4 uses CPU cores [8, 9] | |
| [2024-05-27 23:16:09,759][1934317] Worker 1 uses CPU cores [2, 3] | |
| [2024-05-27 23:16:09,760][1934336] Worker 5 uses CPU cores [10, 11] | |
| [2024-05-27 23:16:09,766][1934158] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-05-27 23:16:09,769][1934342] Worker 7 uses CPU cores [14, 15] | |
| [2024-05-27 23:16:09,810][1934315] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:16:09,811][1934315] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:16:09,820][1934315] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:16:09,897][1934315] Conv encoder output size: 512 | |
| [2024-05-27 23:16:09,898][1934315] Policy head output size: 512 | |
| [2024-05-27 23:16:09,948][1934158] Inference worker 0-0 is ready! | |
| [2024-05-27 23:16:09,948][1934158] All inference workers are ready! Signal rollout workers to start! | |
| [2024-05-27 23:16:09,966][1934316] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934319] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934320] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934318] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934342] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934336] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934337] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:09,966][1934317] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:16:10,026][1934319] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... | |
| [2024-05-27 23:16:10,027][1934319] EvtLoop [rollout_proc3_evt_loop, process=rollout_proc3] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() | |
| Traceback (most recent call last): | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init | |
| self.game.init() | |
| vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. | |
| During handling of the above exception, another exception occurred: | |
| Traceback (most recent call last): | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init | |
| env_runner.init(self.timing) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init | |
| self._reset() | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset | |
| observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/gymnasium/core.py", line 414, in reset | |
| return self.env.reset(seed=seed, options=options) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset | |
| obs, info = self.env.reset(**kwargs) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset | |
| obs, info = self.env.reset(**kwargs) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset | |
| return self.env.reset(**kwargs) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/gymnasium/core.py", line 462, in reset | |
| obs, info = self.env.reset(seed=seed, options=options) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 82, in reset | |
| obs, info = self.env.reset(**kwargs) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/gymnasium/core.py", line 414, in reset | |
| return self.env.reset(seed=seed, options=options) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset | |
| return self.env.reset(**kwargs) | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset | |
| self._ensure_initialized() | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized | |
| self.initialize() | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize | |
| self._game_init() | |
| File "/media/fast/code/learning/venv_learning/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init | |
| raise EnvCriticalError() | |
| sample_factory.envs.env_utils.EnvCriticalError | |
| [2024-05-27 23:16:10,029][1934319] Unhandled exception in evt loop rollout_proc3_evt_loop | |
| [2024-05-27 23:16:10,583][1934337] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:10,583][1934342] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:10,657][1934318] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:10,916][1934342] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:11,000][1934316] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:11,225][1934158] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-05-27 23:16:11,427][1934337] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:11,428][1934316] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:11,606][1934320] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:11,882][1934316] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:12,052][1934342] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:12,155][1934320] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:12,672][1934316] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:12,755][1934337] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:12,756][1934318] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:12,831][1934317] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:13,085][1934342] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:13,412][1934320] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:13,493][1934337] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:13,592][1934318] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:13,763][1934336] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:16:14,101][1934317] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:14,182][1934318] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:14,425][1934320] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:14,429][1934336] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:16:14,657][1934317] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:14,666][1934270] Signal inference workers to stop experience collection... | |
| [2024-05-27 23:16:14,670][1934315] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-05-27 23:16:14,811][1934336] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:16:14,967][1934317] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:15,277][1934336] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:16:15,790][1934270] Signal inference workers to resume experience collection... | |
| [2024-05-27 23:16:15,790][1934315] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-05-27 23:16:16,225][1934158] Fps is (10 sec: 1902.6, 60 sec: 1902.6, 300 sec: 1902.6). Total num frames: 12288. Throughput: 0: 379.3. Samples: 2450. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2024-05-27 23:16:16,225][1934158] Avg episode reward: [(0, '3.187')] | |
| [2024-05-27 23:16:17,981][1934315] Updated weights for policy 0, policy_version 10 (0.0101) | |
| [2024-05-27 23:16:19,839][1934315] Updated weights for policy 0, policy_version 20 (0.0007) | |
| [2024-05-27 23:16:21,225][1934158] Fps is (10 sec: 11059.1, 60 sec: 9651.5, 300 sec: 9651.5). Total num frames: 110592. Throughput: 0: 1415.4. Samples: 16218. Policy #0 lag: (min: 0.0, avg: 0.5, max: 3.0) | |
| [2024-05-27 23:16:21,225][1934158] Avg episode reward: [(0, '4.379')] | |
| [2024-05-27 23:16:21,228][1934270] Saving new best policy, reward=4.379! | |
| [2024-05-27 23:16:21,659][1934315] Updated weights for policy 0, policy_version 30 (0.0007) | |
| [2024-05-27 23:16:23,614][1934315] Updated weights for policy 0, policy_version 40 (0.0008) | |
| [2024-05-27 23:16:23,619][1934158] Heartbeat connected on Batcher_0 | |
| [2024-05-27 23:16:23,621][1934158] Heartbeat connected on LearnerWorker_p0 | |
| [2024-05-27 23:16:23,625][1934158] Heartbeat connected on RolloutWorker_w0 | |
| [2024-05-27 23:16:23,626][1934158] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-05-27 23:16:23,627][1934158] Heartbeat connected on RolloutWorker_w1 | |
| [2024-05-27 23:16:23,632][1934158] Heartbeat connected on RolloutWorker_w4 | |
| [2024-05-27 23:16:23,632][1934158] Heartbeat connected on RolloutWorker_w5 | |
| [2024-05-27 23:16:23,634][1934158] Heartbeat connected on RolloutWorker_w6 | |
| [2024-05-27 23:16:23,635][1934158] Heartbeat connected on RolloutWorker_w2 | |
| [2024-05-27 23:16:23,636][1934158] Heartbeat connected on RolloutWorker_w7 | |
| [2024-05-27 23:16:25,556][1934315] Updated weights for policy 0, policy_version 50 (0.0012) | |
| [2024-05-27 23:16:26,225][1934158] Fps is (10 sec: 20480.0, 60 sec: 13190.0, 300 sec: 13190.0). Total num frames: 217088. Throughput: 0: 2945.8. Samples: 48484. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-05-27 23:16:26,225][1934158] Avg episode reward: [(0, '4.620')] | |
| [2024-05-27 23:16:26,225][1934270] Saving new best policy, reward=4.620! | |
| [2024-05-27 23:16:27,524][1934315] Updated weights for policy 0, policy_version 60 (0.0011) | |
| [2024-05-27 23:16:29,497][1934315] Updated weights for policy 0, policy_version 70 (0.0007) | |
| [2024-05-27 23:16:31,225][1934158] Fps is (10 sec: 20889.7, 60 sec: 14888.7, 300 sec: 14888.7). Total num frames: 319488. Throughput: 0: 3703.7. Samples: 79476. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:16:31,225][1934158] Avg episode reward: [(0, '4.594')] | |
| [2024-05-27 23:16:31,523][1934315] Updated weights for policy 0, policy_version 80 (0.0007) | |
| [2024-05-27 23:16:33,629][1934315] Updated weights for policy 0, policy_version 90 (0.0007) | |
| [2024-05-27 23:16:35,621][1934315] Updated weights for policy 0, policy_version 100 (0.0007) | |
| [2024-05-27 23:16:36,225][1934158] Fps is (10 sec: 20480.0, 60 sec: 15945.3, 300 sec: 15945.3). Total num frames: 421888. Throughput: 0: 3572.4. Samples: 94520. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-05-27 23:16:36,225][1934158] Avg episode reward: [(0, '4.672')] | |
| [2024-05-27 23:16:36,225][1934270] Saving new best policy, reward=4.672! | |
| [2024-05-27 23:16:37,650][1934315] Updated weights for policy 0, policy_version 110 (0.0007) | |
| [2024-05-27 23:16:39,626][1934315] Updated weights for policy 0, policy_version 120 (0.0008) | |
| [2024-05-27 23:16:41,225][1934158] Fps is (10 sec: 20070.3, 60 sec: 16535.8, 300 sec: 16535.8). Total num frames: 520192. Throughput: 0: 3978.1. Samples: 125144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:16:41,225][1934158] Avg episode reward: [(0, '4.368')] | |
| [2024-05-27 23:16:41,688][1934315] Updated weights for policy 0, policy_version 130 (0.0007) | |
| [2024-05-27 23:16:43,620][1934315] Updated weights for policy 0, policy_version 140 (0.0007) | |
| [2024-05-27 23:16:45,415][1934315] Updated weights for policy 0, policy_version 150 (0.0007) | |
| [2024-05-27 23:16:46,225][1934158] Fps is (10 sec: 20889.6, 60 sec: 17301.4, 300 sec: 17301.4). Total num frames: 630784. Throughput: 0: 4313.8. Samples: 157276. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:16:46,225][1934158] Avg episode reward: [(0, '4.638')] | |
| [2024-05-27 23:16:47,205][1934315] Updated weights for policy 0, policy_version 160 (0.0007) | |
| [2024-05-27 23:16:48,998][1934315] Updated weights for policy 0, policy_version 170 (0.0010) | |
| [2024-05-27 23:16:50,924][1934315] Updated weights for policy 0, policy_version 180 (0.0007) | |
| [2024-05-27 23:16:51,225][1934158] Fps is (10 sec: 22118.5, 60 sec: 17882.4, 300 sec: 17882.4). Total num frames: 741376. Throughput: 0: 4207.1. Samples: 174418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:16:51,225][1934158] Avg episode reward: [(0, '4.486')] | |
| [2024-05-27 23:16:52,935][1934315] Updated weights for policy 0, policy_version 190 (0.0007) | |
| [2024-05-27 23:16:54,845][1934315] Updated weights for policy 0, policy_version 200 (0.0008) | |
| [2024-05-27 23:16:56,225][1934158] Fps is (10 sec: 21708.7, 60 sec: 18250.1, 300 sec: 18250.1). Total num frames: 847872. Throughput: 0: 4577.8. Samples: 206002. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-05-27 23:16:56,225][1934158] Avg episode reward: [(0, '4.466')] | |
| [2024-05-27 23:16:56,781][1934315] Updated weights for policy 0, policy_version 210 (0.0007) | |
| [2024-05-27 23:16:58,717][1934315] Updated weights for policy 0, policy_version 220 (0.0008) | |
| [2024-05-27 23:17:00,636][1934315] Updated weights for policy 0, policy_version 230 (0.0007) | |
| [2024-05-27 23:17:01,225][1934158] Fps is (10 sec: 20889.6, 60 sec: 18466.8, 300 sec: 18466.8). Total num frames: 950272. Throughput: 0: 5229.4. Samples: 237774. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:17:01,225][1934158] Avg episode reward: [(0, '4.450')] | |
| [2024-05-27 23:17:02,613][1934315] Updated weights for policy 0, policy_version 240 (0.0008) | |
| [2024-05-27 23:17:04,662][1934315] Updated weights for policy 0, policy_version 250 (0.0007) | |
| [2024-05-27 23:17:06,225][1934158] Fps is (10 sec: 20480.0, 60 sec: 18645.1, 300 sec: 18645.1). Total num frames: 1052672. Throughput: 0: 5261.9. Samples: 253004. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:17:06,225][1934158] Avg episode reward: [(0, '4.710')] | |
| [2024-05-27 23:17:06,225][1934270] Saving new best policy, reward=4.710! | |
| [2024-05-27 23:17:06,803][1934315] Updated weights for policy 0, policy_version 260 (0.0011) | |
| [2024-05-27 23:17:08,893][1934315] Updated weights for policy 0, policy_version 270 (0.0007) | |
| [2024-05-27 23:17:10,844][1934315] Updated weights for policy 0, policy_version 280 (0.0012) | |
| [2024-05-27 23:17:11,225][1934158] Fps is (10 sec: 20070.2, 60 sec: 19182.9, 300 sec: 18727.7). Total num frames: 1150976. Throughput: 0: 5203.4. Samples: 282636. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:17:11,225][1934158] Avg episode reward: [(0, '4.512')] | |
| [2024-05-27 23:17:12,691][1934315] Updated weights for policy 0, policy_version 290 (0.0007) | |
| [2024-05-27 23:17:14,604][1934315] Updated weights for policy 0, policy_version 300 (0.0007) | |
| [2024-05-27 23:17:16,225][1934158] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 19044.4). Total num frames: 1265664. Throughput: 0: 5248.3. Samples: 315650. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:17:16,225][1934158] Avg episode reward: [(0, '4.587')] | |
| [2024-05-27 23:17:16,410][1934315] Updated weights for policy 0, policy_version 310 (0.0007) | |
| [2024-05-27 23:17:18,299][1934315] Updated weights for policy 0, policy_version 320 (0.0008) | |
| [2024-05-27 23:17:20,246][1934315] Updated weights for policy 0, policy_version 330 (0.0007) | |
| [2024-05-27 23:17:21,225][1934158] Fps is (10 sec: 22118.6, 60 sec: 21026.2, 300 sec: 19202.2). Total num frames: 1372160. Throughput: 0: 5274.4. Samples: 331866. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:17:21,225][1934158] Avg episode reward: [(0, '4.428')] | |
| [2024-05-27 23:17:22,228][1934315] Updated weights for policy 0, policy_version 340 (0.0007) | |
| [2024-05-27 23:17:24,250][1934315] Updated weights for policy 0, policy_version 350 (0.0007) | |
| [2024-05-27 23:17:26,216][1934315] Updated weights for policy 0, policy_version 360 (0.0008) | |
| [2024-05-27 23:17:26,225][1934158] Fps is (10 sec: 20889.8, 60 sec: 20957.9, 300 sec: 19285.8). Total num frames: 1474560. Throughput: 0: 5278.6. Samples: 362680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:17:26,225][1934158] Avg episode reward: [(0, '4.760')] | |
| [2024-05-27 23:17:26,225][1934270] Saving new best policy, reward=4.760! | |
| [2024-05-27 23:17:28,176][1934315] Updated weights for policy 0, policy_version 370 (0.0007) | |
| [2024-05-27 23:17:30,122][1934315] Updated weights for policy 0, policy_version 380 (0.0007) | |
| [2024-05-27 23:17:31,225][1934158] Fps is (10 sec: 20480.0, 60 sec: 20957.9, 300 sec: 19359.1). Total num frames: 1576960. Throughput: 0: 5261.3. Samples: 394036. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:17:31,225][1934158] Avg episode reward: [(0, '5.042')] | |
| [2024-05-27 23:17:31,227][1934270] Saving new best policy, reward=5.042! | |
| [2024-05-27 23:17:32,083][1934315] Updated weights for policy 0, policy_version 390 (0.0007) | |
| [2024-05-27 23:17:33,997][1934315] Updated weights for policy 0, policy_version 400 (0.0008) | |
| [2024-05-27 23:17:35,937][1934315] Updated weights for policy 0, policy_version 410 (0.0008) | |
| [2024-05-27 23:17:36,225][1934158] Fps is (10 sec: 20889.5, 60 sec: 21026.1, 300 sec: 19471.3). Total num frames: 1683456. Throughput: 0: 5235.8. Samples: 410028. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-05-27 23:17:36,225][1934158] Avg episode reward: [(0, '4.829')] | |
| [2024-05-27 23:17:37,913][1934315] Updated weights for policy 0, policy_version 420 (0.0007) | |
| [2024-05-27 23:17:39,821][1934315] Updated weights for policy 0, policy_version 430 (0.0007) | |
| [2024-05-27 23:17:41,225][1934158] Fps is (10 sec: 21299.2, 60 sec: 21162.7, 300 sec: 19571.2). Total num frames: 1789952. Throughput: 0: 5239.0. Samples: 441758. Policy #0 lag: (min: 0.0, avg: 0.6, max: 3.0) | |
| [2024-05-27 23:17:41,225][1934158] Avg episode reward: [(0, '5.811')] | |
| [2024-05-27 23:17:41,227][1934270] Saving new best policy, reward=5.811! | |
| [2024-05-27 23:17:41,621][1934315] Updated weights for policy 0, policy_version 440 (0.0007) | |
| [2024-05-27 23:17:43,431][1934315] Updated weights for policy 0, policy_version 450 (0.0008) | |
| [2024-05-27 23:17:45,265][1934315] Updated weights for policy 0, policy_version 460 (0.0007) | |
| [2024-05-27 23:17:46,225][1934158] Fps is (10 sec: 22118.4, 60 sec: 21230.9, 300 sec: 19745.7). Total num frames: 1904640. Throughput: 0: 5286.2. Samples: 475652. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-05-27 23:17:46,225][1934158] Avg episode reward: [(0, '6.064')] | |
| [2024-05-27 23:17:46,225][1934270] Saving new best policy, reward=6.064! | |
| [2024-05-27 23:17:47,258][1934315] Updated weights for policy 0, policy_version 470 (0.0007) | |
| [2024-05-27 23:17:49,311][1934315] Updated weights for policy 0, policy_version 480 (0.0007) | |
| [2024-05-27 23:17:51,225][1934158] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 19741.5). Total num frames: 2002944. Throughput: 0: 5272.8. Samples: 490282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-05-27 23:17:51,225][1934158] Avg episode reward: [(0, '8.117')] | |
| [2024-05-27 23:17:51,228][1934270] Saving new best policy, reward=8.117! | |
| [2024-05-27 23:17:51,309][1934315] Updated weights for policy 0, policy_version 490 (0.0007) | |
| [2024-05-27 23:17:53,208][1934315] Updated weights for policy 0, policy_version 500 (0.0007) | |
| [2024-05-27 23:17:55,270][1934315] Updated weights for policy 0, policy_version 510 (0.0007) | |
| [2024-05-27 23:17:56,225][1934158] Fps is (10 sec: 20070.4, 60 sec: 20957.9, 300 sec: 19776.2). Total num frames: 2105344. Throughput: 0: 5304.0. Samples: 521316. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:17:56,225][1934158] Avg episode reward: [(0, '9.056')] | |
| [2024-05-27 23:17:56,225][1934270] Saving new best policy, reward=9.056! | |
| [2024-05-27 23:17:57,265][1934315] Updated weights for policy 0, policy_version 520 (0.0007) | |
| [2024-05-27 23:17:59,319][1934315] Updated weights for policy 0, policy_version 530 (0.0011) | |
| [2024-05-27 23:18:01,225][1934158] Fps is (10 sec: 20479.9, 60 sec: 20957.8, 300 sec: 19807.7). Total num frames: 2207744. Throughput: 0: 5240.7. Samples: 551482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:18:01,225][1934158] Avg episode reward: [(0, '11.238')] | |
| [2024-05-27 23:18:01,228][1934270] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000539_2207744.pth... | |
| [2024-05-27 23:18:01,267][1934270] Saving new best policy, reward=11.238! | |
| [2024-05-27 23:18:01,335][1934315] Updated weights for policy 0, policy_version 540 (0.0007) | |
| [2024-05-27 23:18:03,448][1934315] Updated weights for policy 0, policy_version 550 (0.0012) | |
| [2024-05-27 23:18:05,446][1934315] Updated weights for policy 0, policy_version 560 (0.0007) | |
| [2024-05-27 23:18:06,225][1934158] Fps is (10 sec: 20070.5, 60 sec: 20889.6, 300 sec: 19801.5). Total num frames: 2306048. Throughput: 0: 5211.2. Samples: 566368. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:06,225][1934158] Avg episode reward: [(0, '11.336')] | |
| [2024-05-27 23:18:06,225][1934270] Saving new best policy, reward=11.336! | |
| [2024-05-27 23:18:07,421][1934315] Updated weights for policy 0, policy_version 570 (0.0007) | |
| [2024-05-27 23:18:09,239][1934315] Updated weights for policy 0, policy_version 580 (0.0009) | |
| [2024-05-27 23:18:11,047][1934315] Updated weights for policy 0, policy_version 590 (0.0007) | |
| [2024-05-27 23:18:11,224][1934158] Fps is (10 sec: 21299.6, 60 sec: 21162.7, 300 sec: 19930.6). Total num frames: 2420736. Throughput: 0: 5245.1. Samples: 598710. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:11,225][1934158] Avg episode reward: [(0, '14.662')] | |
| [2024-05-27 23:18:11,227][1934270] Saving new best policy, reward=14.662! | |
| [2024-05-27 23:18:12,854][1934315] Updated weights for policy 0, policy_version 600 (0.0009) | |
| [2024-05-27 23:18:14,740][1934315] Updated weights for policy 0, policy_version 610 (0.0007) | |
| [2024-05-27 23:18:16,225][1934158] Fps is (10 sec: 22118.2, 60 sec: 21026.1, 300 sec: 19984.7). Total num frames: 2527232. Throughput: 0: 5272.6. Samples: 631302. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:18:16,225][1934158] Avg episode reward: [(0, '15.552')] | |
| [2024-05-27 23:18:16,225][1934270] Saving new best policy, reward=15.552! | |
| [2024-05-27 23:18:16,763][1934315] Updated weights for policy 0, policy_version 620 (0.0011) | |
| [2024-05-27 23:18:18,712][1934315] Updated weights for policy 0, policy_version 630 (0.0012) | |
| [2024-05-27 23:18:20,722][1934315] Updated weights for policy 0, policy_version 640 (0.0012) | |
| [2024-05-27 23:18:21,225][1934158] Fps is (10 sec: 20889.2, 60 sec: 20957.8, 300 sec: 20003.5). Total num frames: 2629632. Throughput: 0: 5265.6. Samples: 646982. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:21,225][1934158] Avg episode reward: [(0, '15.319')] | |
| [2024-05-27 23:18:22,704][1934315] Updated weights for policy 0, policy_version 650 (0.0012) | |
| [2024-05-27 23:18:24,737][1934315] Updated weights for policy 0, policy_version 660 (0.0009) | |
| [2024-05-27 23:18:26,225][1934158] Fps is (10 sec: 20480.1, 60 sec: 20957.9, 300 sec: 20021.0). Total num frames: 2732032. Throughput: 0: 5239.9. Samples: 677552. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:26,225][1934158] Avg episode reward: [(0, '18.705')] | |
| [2024-05-27 23:18:26,225][1934270] Saving new best policy, reward=18.705! | |
| [2024-05-27 23:18:26,687][1934315] Updated weights for policy 0, policy_version 670 (0.0007) | |
| [2024-05-27 23:18:28,690][1934315] Updated weights for policy 0, policy_version 680 (0.0007) | |
| [2024-05-27 23:18:30,666][1934315] Updated weights for policy 0, policy_version 690 (0.0007) | |
| [2024-05-27 23:18:31,225][1934158] Fps is (10 sec: 20480.2, 60 sec: 20957.9, 300 sec: 20037.2). Total num frames: 2834432. Throughput: 0: 5177.5. Samples: 708638. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:18:31,225][1934158] Avg episode reward: [(0, '19.747')] | |
| [2024-05-27 23:18:31,228][1934270] Saving new best policy, reward=19.747! | |
| [2024-05-27 23:18:32,744][1934315] Updated weights for policy 0, policy_version 700 (0.0007) | |
| [2024-05-27 23:18:34,708][1934315] Updated weights for policy 0, policy_version 710 (0.0008) | |
| [2024-05-27 23:18:36,225][1934158] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20052.3). Total num frames: 2936832. Throughput: 0: 5187.1. Samples: 723702. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-05-27 23:18:36,225][1934158] Avg episode reward: [(0, '20.573')] | |
| [2024-05-27 23:18:36,226][1934270] Saving new best policy, reward=20.573! | |
| [2024-05-27 23:18:36,748][1934315] Updated weights for policy 0, policy_version 720 (0.0008) | |
| [2024-05-27 23:18:38,561][1934315] Updated weights for policy 0, policy_version 730 (0.0008) | |
| [2024-05-27 23:18:40,387][1934315] Updated weights for policy 0, policy_version 740 (0.0008) | |
| [2024-05-27 23:18:41,225][1934158] Fps is (10 sec: 21299.2, 60 sec: 20957.9, 300 sec: 20120.5). Total num frames: 3047424. Throughput: 0: 5210.7. Samples: 755796. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:41,225][1934158] Avg episode reward: [(0, '21.032')] | |
| [2024-05-27 23:18:41,228][1934270] Saving new best policy, reward=21.032! | |
| [2024-05-27 23:18:42,185][1934315] Updated weights for policy 0, policy_version 750 (0.0007) | |
| [2024-05-27 23:18:44,027][1934315] Updated weights for policy 0, policy_version 760 (0.0007) | |
| [2024-05-27 23:18:45,915][1934315] Updated weights for policy 0, policy_version 770 (0.0007) | |
| [2024-05-27 23:18:46,225][1934158] Fps is (10 sec: 22118.7, 60 sec: 20889.6, 300 sec: 20184.4). Total num frames: 3158016. Throughput: 0: 5280.4. Samples: 789100. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:18:46,225][1934158] Avg episode reward: [(0, '18.887')] | |
| [2024-05-27 23:18:47,935][1934315] Updated weights for policy 0, policy_version 780 (0.0007) | |
| [2024-05-27 23:18:49,831][1934315] Updated weights for policy 0, policy_version 790 (0.0007) | |
| [2024-05-27 23:18:51,225][1934158] Fps is (10 sec: 21708.7, 60 sec: 21026.2, 300 sec: 20218.9). Total num frames: 3264512. Throughput: 0: 5299.6. Samples: 804852. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:18:51,225][1934158] Avg episode reward: [(0, '17.324')] | |
| [2024-05-27 23:18:51,873][1934315] Updated weights for policy 0, policy_version 800 (0.0012) | |
| [2024-05-27 23:18:53,869][1934315] Updated weights for policy 0, policy_version 810 (0.0008) | |
| [2024-05-27 23:18:55,945][1934315] Updated weights for policy 0, policy_version 820 (0.0011) | |
| [2024-05-27 23:18:56,225][1934158] Fps is (10 sec: 20480.0, 60 sec: 20957.9, 300 sec: 20202.1). Total num frames: 3362816. Throughput: 0: 5260.6. Samples: 835438. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:18:56,225][1934158] Avg episode reward: [(0, '17.131')] | |
| [2024-05-27 23:18:57,993][1934315] Updated weights for policy 0, policy_version 830 (0.0016) | |
| [2024-05-27 23:18:59,990][1934315] Updated weights for policy 0, policy_version 840 (0.0007) | |
| [2024-05-27 23:19:01,225][1934158] Fps is (10 sec: 20070.4, 60 sec: 20957.9, 300 sec: 20210.2). Total num frames: 3465216. Throughput: 0: 5207.4. Samples: 865636. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-05-27 23:19:01,225][1934158] Avg episode reward: [(0, '20.196')] | |
| [2024-05-27 23:19:02,004][1934315] Updated weights for policy 0, policy_version 850 (0.0008) | |
| [2024-05-27 23:19:04,080][1934315] Updated weights for policy 0, policy_version 860 (0.0007) | |
| [2024-05-27 23:19:05,800][1934315] Updated weights for policy 0, policy_version 870 (0.0007) | |
| [2024-05-27 23:19:06,225][1934158] Fps is (10 sec: 20889.5, 60 sec: 21094.4, 300 sec: 20241.1). Total num frames: 3571712. Throughput: 0: 5194.5. Samples: 880732. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:19:06,225][1934158] Avg episode reward: [(0, '21.470')] | |
| [2024-05-27 23:19:06,225][1934270] Saving new best policy, reward=21.470! | |
| [2024-05-27 23:19:07,672][1934315] Updated weights for policy 0, policy_version 880 (0.0008) | |
| [2024-05-27 23:19:09,539][1934315] Updated weights for policy 0, policy_version 890 (0.0008) | |
| [2024-05-27 23:19:11,225][1934158] Fps is (10 sec: 21708.9, 60 sec: 21026.1, 300 sec: 20292.8). Total num frames: 3682304. Throughput: 0: 5259.7. Samples: 914240. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:19:11,225][1934158] Avg episode reward: [(0, '21.748')] | |
| [2024-05-27 23:19:11,228][1934270] Saving new best policy, reward=21.748! | |
| [2024-05-27 23:19:11,365][1934315] Updated weights for policy 0, policy_version 900 (0.0007) | |
| [2024-05-27 23:19:13,481][1934315] Updated weights for policy 0, policy_version 910 (0.0009) | |
| [2024-05-27 23:19:15,430][1934315] Updated weights for policy 0, policy_version 920 (0.0008) | |
| [2024-05-27 23:19:16,225][1934158] Fps is (10 sec: 20889.5, 60 sec: 20889.6, 300 sec: 20275.9). Total num frames: 3780608. Throughput: 0: 5263.5. Samples: 945494. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-05-27 23:19:16,225][1934158] Avg episode reward: [(0, '20.029')] | |
| [2024-05-27 23:19:17,514][1934315] Updated weights for policy 0, policy_version 930 (0.0011) | |
| [2024-05-27 23:19:19,529][1934315] Updated weights for policy 0, policy_version 940 (0.0008) | |
| [2024-05-27 23:19:21,225][1934158] Fps is (10 sec: 20070.4, 60 sec: 20889.6, 300 sec: 20281.2). Total num frames: 3883008. Throughput: 0: 5255.3. Samples: 960192. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:19:21,225][1934158] Avg episode reward: [(0, '22.181')] | |
| [2024-05-27 23:19:21,227][1934270] Saving new best policy, reward=22.181! | |
| [2024-05-27 23:19:21,559][1934315] Updated weights for policy 0, policy_version 950 (0.0007) | |
| [2024-05-27 23:19:23,512][1934315] Updated weights for policy 0, policy_version 960 (0.0007) | |
| [2024-05-27 23:19:25,534][1934315] Updated weights for policy 0, policy_version 970 (0.0011) | |
| [2024-05-27 23:19:26,225][1934158] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20286.3). Total num frames: 3985408. Throughput: 0: 5222.8. Samples: 990822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-05-27 23:19:26,225][1934158] Avg episode reward: [(0, '19.642')] | |
| [2024-05-27 23:19:27,196][1934158] Component Batcher_0 stopped! | |
| [2024-05-27 23:19:27,196][1934270] Stopping Batcher_0... | |
| [2024-05-27 23:19:27,196][1934158] Component RolloutWorker_w3 process died already! Don't wait for it. | |
| [2024-05-27 23:19:27,196][1934270] Loop batcher_evt_loop terminating... | |
| [2024-05-27 23:19:27,196][1934270] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-05-27 23:19:27,202][1934317] Stopping RolloutWorker_w1... | |
| [2024-05-27 23:19:27,202][1934158] Component RolloutWorker_w1 stopped! | |
| [2024-05-27 23:19:27,202][1934317] Loop rollout_proc1_evt_loop terminating... | |
| [2024-05-27 23:19:27,202][1934320] Stopping RolloutWorker_w4... | |
| [2024-05-27 23:19:27,202][1934158] Component RolloutWorker_w4 stopped! | |
| [2024-05-27 23:19:27,202][1934316] Stopping RolloutWorker_w0... | |
| [2024-05-27 23:19:27,202][1934336] Stopping RolloutWorker_w5... | |
| [2024-05-27 23:19:27,203][1934320] Loop rollout_proc4_evt_loop terminating... | |
| [2024-05-27 23:19:27,203][1934158] Component RolloutWorker_w0 stopped! | |
| [2024-05-27 23:19:27,203][1934316] Loop rollout_proc0_evt_loop terminating... | |
| [2024-05-27 23:19:27,203][1934318] Stopping RolloutWorker_w2... | |
| [2024-05-27 23:19:27,203][1934158] Component RolloutWorker_w5 stopped! | |
| [2024-05-27 23:19:27,203][1934336] Loop rollout_proc5_evt_loop terminating... | |
| [2024-05-27 23:19:27,203][1934337] Stopping RolloutWorker_w6... | |
| [2024-05-27 23:19:27,203][1934158] Component RolloutWorker_w2 stopped! | |
| [2024-05-27 23:19:27,203][1934158] Component RolloutWorker_w6 stopped! | |
| [2024-05-27 23:19:27,203][1934318] Loop rollout_proc2_evt_loop terminating... | |
| [2024-05-27 23:19:27,203][1934337] Loop rollout_proc6_evt_loop terminating... | |
| [2024-05-27 23:19:27,208][1934342] Stopping RolloutWorker_w7... | |
| [2024-05-27 23:19:27,208][1934158] Component RolloutWorker_w7 stopped! | |
| [2024-05-27 23:19:27,208][1934342] Loop rollout_proc7_evt_loop terminating... | |
| [2024-05-27 23:19:27,210][1934315] Weights refcount: 2 0 | |
| [2024-05-27 23:19:27,250][1934270] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-05-27 23:19:27,261][1934315] Stopping InferenceWorker_p0-w0... | |
| [2024-05-27 23:19:27,261][1934315] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-05-27 23:19:27,261][1934158] Component InferenceWorker_p0-w0 stopped! | |
| [2024-05-27 23:19:27,311][1934270] Stopping LearnerWorker_p0... | |
| [2024-05-27 23:19:27,311][1934158] Component LearnerWorker_p0 stopped! | |
| [2024-05-27 23:19:27,311][1934270] Loop learner_proc0_evt_loop terminating... | |
| [2024-05-27 23:19:27,312][1934158] Waiting for process learner_proc0 to stop... | |
| [2024-05-27 23:19:27,998][1934158] Waiting for process inference_proc0-0 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc0 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc1 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc2 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc3 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc4 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc5 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc6 to join... | |
| [2024-05-27 23:19:27,999][1934158] Waiting for process rollout_proc7 to join... | |
| [2024-05-27 23:19:27,999][1934158] Batcher 0 profile tree view: | |
| batching: 17.1563, releasing_batches: 0.0218 | |
| [2024-05-27 23:19:27,999][1934158] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0000 | |
| wait_policy_total: 4.5927 | |
| update_model: 2.6467 | |
| weight_update: 0.0007 | |
| one_step: 0.0025 | |
| handle_policy_step: 180.1540 | |
| deserialize: 5.4368, stack: 0.9178, obs_to_device_normalize: 40.7850, forward: 100.2096, send_messages: 7.8527 | |
| prepare_outputs: 18.8993 | |
| to_cpu: 11.8170 | |
| [2024-05-27 23:19:27,999][1934158] Learner 0 profile tree view: | |
| misc: 0.0033, prepare_batch: 6.9298 | |
| train: 19.9916 | |
| epoch_init: 0.0036, minibatch_init: 0.0045, losses_postprocess: 0.3051, kl_divergence: 0.3106, after_optimizer: 6.7727 | |
| calculate_losses: 8.3137 | |
| losses_init: 0.0021, forward_head: 0.5755, bptt_initial: 5.5591, tail: 0.4520, advantages_returns: 0.1178, losses: 0.7330 | |
| bptt: 0.7343 | |
| bptt_forward_core: 0.6989 | |
| update: 3.9895 | |
| clip: 0.4408 | |
| [2024-05-27 23:19:27,999][1934158] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.1094, enqueue_policy_requests: 6.7447, env_step: 74.1145, overhead: 8.3155, complete_rollouts: 0.2030 | |
| save_policy_outputs: 6.8527 | |
| split_output_tensors: 3.2861 | |
| [2024-05-27 23:19:27,999][1934158] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.1229, enqueue_policy_requests: 6.7854, env_step: 69.9361, overhead: 8.1086, complete_rollouts: 0.2114 | |
| save_policy_outputs: 6.5377 | |
| split_output_tensors: 3.1628 | |
| [2024-05-27 23:19:28,000][1934158] Loop Runner_EvtLoop terminating... | |
| [2024-05-27 23:19:28,000][1934158] Runner profile tree view: | |
| main_loop: 204.3645 | |
| [2024-05-27 23:19:28,000][1934158] Collected {0: 4005888}, FPS: 19601.7 | |
| [2024-05-27 23:19:28,286][1934158] Loading existing experiment configuration from /media/fast/code/learning/train_dir/default_experiment/config.json | |
| [2024-05-27 23:19:28,286][1934158] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-05-27 23:19:28,286][1934158] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-05-27 23:19:28,286][1934158] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-05-27 23:19:28,286][1934158] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:19:28,286][1934158] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-05-27 23:19:28,287][1934158] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-05-27 23:19:28,294][1934158] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:19:28,294][1934158] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:19:28,295][1934158] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:19:28,301][1934158] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:19:28,358][1934158] Conv encoder output size: 512 | |
| [2024-05-27 23:19:28,358][1934158] Policy head output size: 512 | |
| [2024-05-27 23:19:28,450][1934158] Loading state from checkpoint /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-05-27 23:19:29,259][1934158] Num frames 100... | |
| [2024-05-27 23:19:29,371][1934158] Num frames 200... | |
| [2024-05-27 23:19:29,454][1934158] Num frames 300... | |
| [2024-05-27 23:19:29,532][1934158] Num frames 400... | |
| [2024-05-27 23:19:29,631][1934158] Num frames 500... | |
| [2024-05-27 23:19:29,698][1934158] Num frames 600... | |
| [2024-05-27 23:19:29,791][1934158] Num frames 700... | |
| [2024-05-27 23:19:29,862][1934158] Num frames 800... | |
| [2024-05-27 23:19:29,927][1934158] Num frames 900... | |
| [2024-05-27 23:19:29,998][1934158] Num frames 1000... | |
| [2024-05-27 23:19:30,066][1934158] Num frames 1100... | |
| [2024-05-27 23:19:30,177][1934158] Num frames 1200... | |
| [2024-05-27 23:19:30,260][1934158] Num frames 1300... | |
| [2024-05-27 23:19:30,323][1934158] Num frames 1400... | |
| [2024-05-27 23:19:30,380][1934158] Avg episode rewards: #0: 36.080, true rewards: #0: 14.080 | |
| [2024-05-27 23:19:30,380][1934158] Avg episode reward: 36.080, avg true_objective: 14.080 | |
| [2024-05-27 23:19:30,437][1934158] Num frames 1500... | |
| [2024-05-27 23:19:30,499][1934158] Num frames 1600... | |
| [2024-05-27 23:19:30,558][1934158] Num frames 1700... | |
| [2024-05-27 23:19:30,615][1934158] Num frames 1800... | |
| [2024-05-27 23:19:30,674][1934158] Num frames 1900... | |
| [2024-05-27 23:19:30,737][1934158] Num frames 2000... | |
| [2024-05-27 23:19:30,840][1934158] Avg episode rewards: #0: 24.400, true rewards: #0: 10.400 | |
| [2024-05-27 23:19:30,840][1934158] Avg episode reward: 24.400, avg true_objective: 10.400 | |
| [2024-05-27 23:19:30,853][1934158] Num frames 2100... | |
| [2024-05-27 23:19:30,916][1934158] Num frames 2200... | |
| [2024-05-27 23:19:30,975][1934158] Num frames 2300... | |
| [2024-05-27 23:19:31,035][1934158] Num frames 2400... | |
| [2024-05-27 23:19:31,097][1934158] Num frames 2500... | |
| [2024-05-27 23:19:31,155][1934158] Num frames 2600... | |
| [2024-05-27 23:19:31,213][1934158] Num frames 2700... | |
| [2024-05-27 23:19:31,271][1934158] Num frames 2800... | |
| [2024-05-27 23:19:31,330][1934158] Num frames 2900... | |
| [2024-05-27 23:19:31,392][1934158] Num frames 3000... | |
| [2024-05-27 23:19:31,450][1934158] Num frames 3100... | |
| [2024-05-27 23:19:31,505][1934158] Num frames 3200... | |
| [2024-05-27 23:19:31,561][1934158] Num frames 3300... | |
| [2024-05-27 23:19:31,668][1934158] Avg episode rewards: #0: 26.640, true rewards: #0: 11.307 | |
| [2024-05-27 23:19:31,668][1934158] Avg episode reward: 26.640, avg true_objective: 11.307 | |
| [2024-05-27 23:19:31,674][1934158] Num frames 3400... | |
| [2024-05-27 23:19:31,733][1934158] Num frames 3500... | |
| [2024-05-27 23:19:31,791][1934158] Num frames 3600... | |
| [2024-05-27 23:19:31,849][1934158] Num frames 3700... | |
| [2024-05-27 23:19:31,908][1934158] Num frames 3800... | |
| [2024-05-27 23:19:31,961][1934158] Num frames 3900... | |
| [2024-05-27 23:19:32,019][1934158] Num frames 4000... | |
| [2024-05-27 23:19:32,075][1934158] Num frames 4100... | |
| [2024-05-27 23:19:32,133][1934158] Num frames 4200... | |
| [2024-05-27 23:19:32,237][1934158] Avg episode rewards: #0: 24.720, true rewards: #0: 10.720 | |
| [2024-05-27 23:19:32,237][1934158] Avg episode reward: 24.720, avg true_objective: 10.720 | |
| [2024-05-27 23:19:32,245][1934158] Num frames 4300... | |
| [2024-05-27 23:19:32,301][1934158] Num frames 4400... | |
| [2024-05-27 23:19:32,360][1934158] Num frames 4500... | |
| [2024-05-27 23:19:32,418][1934158] Num frames 4600... | |
| [2024-05-27 23:19:32,476][1934158] Num frames 4700... | |
| [2024-05-27 23:19:32,545][1934158] Num frames 4800... | |
| [2024-05-27 23:19:32,607][1934158] Num frames 4900... | |
| [2024-05-27 23:19:32,667][1934158] Num frames 5000... | |
| [2024-05-27 23:19:32,732][1934158] Num frames 5100... | |
| [2024-05-27 23:19:32,794][1934158] Num frames 5200... | |
| [2024-05-27 23:19:32,852][1934158] Num frames 5300... | |
| [2024-05-27 23:19:32,906][1934158] Avg episode rewards: #0: 24.204, true rewards: #0: 10.604 | |
| [2024-05-27 23:19:32,906][1934158] Avg episode reward: 24.204, avg true_objective: 10.604 | |
| [2024-05-27 23:19:32,970][1934158] Num frames 5400... | |
| [2024-05-27 23:19:33,037][1934158] Num frames 5500... | |
| [2024-05-27 23:19:33,102][1934158] Num frames 5600... | |
| [2024-05-27 23:19:33,165][1934158] Num frames 5700... | |
| [2024-05-27 23:19:33,222][1934158] Num frames 5800... | |
| [2024-05-27 23:19:33,310][1934158] Avg episode rewards: #0: 21.607, true rewards: #0: 9.773 | |
| [2024-05-27 23:19:33,310][1934158] Avg episode reward: 21.607, avg true_objective: 9.773 | |
| [2024-05-27 23:19:33,333][1934158] Num frames 5900... | |
| [2024-05-27 23:19:33,482][1934158] Num frames 6000... | |
| [2024-05-27 23:19:33,548][1934158] Num frames 6100... | |
| [2024-05-27 23:19:33,609][1934158] Num frames 6200... | |
| [2024-05-27 23:19:33,671][1934158] Num frames 6300... | |
| [2024-05-27 23:19:33,734][1934158] Num frames 6400... | |
| [2024-05-27 23:19:33,794][1934158] Num frames 6500... | |
| [2024-05-27 23:19:33,856][1934158] Num frames 6600... | |
| [2024-05-27 23:19:33,921][1934158] Num frames 6700... | |
| [2024-05-27 23:19:33,978][1934158] Num frames 6800... | |
| [2024-05-27 23:19:34,035][1934158] Num frames 6900... | |
| [2024-05-27 23:19:34,144][1934158] Avg episode rewards: #0: 21.702, true rewards: #0: 9.987 | |
| [2024-05-27 23:19:34,144][1934158] Avg episode reward: 21.702, avg true_objective: 9.987 | |
| [2024-05-27 23:19:34,151][1934158] Num frames 7000... | |
| [2024-05-27 23:19:34,217][1934158] Num frames 7100... | |
| [2024-05-27 23:19:34,281][1934158] Num frames 7200... | |
| [2024-05-27 23:19:34,341][1934158] Num frames 7300... | |
| [2024-05-27 23:19:34,451][1934158] Num frames 7400... | |
| [2024-05-27 23:19:34,510][1934158] Num frames 7500... | |
| [2024-05-27 23:19:34,570][1934158] Num frames 7600... | |
| [2024-05-27 23:19:34,634][1934158] Num frames 7700... | |
| [2024-05-27 23:19:34,694][1934158] Num frames 7800... | |
| [2024-05-27 23:19:34,781][1934158] Num frames 7900... | |
| [2024-05-27 23:19:34,851][1934158] Num frames 8000... | |
| [2024-05-27 23:19:34,933][1934158] Num frames 8100... | |
| [2024-05-27 23:19:35,030][1934158] Num frames 8200... | |
| [2024-05-27 23:19:35,099][1934158] Num frames 8300... | |
| [2024-05-27 23:19:35,251][1934158] Num frames 8400... | |
| [2024-05-27 23:19:35,382][1934158] Num frames 8500... | |
| [2024-05-27 23:19:35,466][1934158] Num frames 8600... | |
| [2024-05-27 23:19:35,533][1934158] Num frames 8700... | |
| [2024-05-27 23:19:35,603][1934158] Num frames 8800... | |
| [2024-05-27 23:19:35,672][1934158] Num frames 8900... | |
| [2024-05-27 23:19:35,772][1934158] Num frames 9000... | |
| [2024-05-27 23:19:35,903][1934158] Avg episode rewards: #0: 25.989, true rewards: #0: 11.364 | |
| [2024-05-27 23:19:35,903][1934158] Avg episode reward: 25.989, avg true_objective: 11.364 | |
| [2024-05-27 23:19:35,910][1934158] Num frames 9100... | |
| [2024-05-27 23:19:35,989][1934158] Num frames 9200... | |
| [2024-05-27 23:19:36,090][1934158] Num frames 9300... | |
| [2024-05-27 23:19:36,194][1934158] Num frames 9400... | |
| [2024-05-27 23:19:36,262][1934158] Num frames 9500... | |
| [2024-05-27 23:19:36,364][1934158] Num frames 9600... | |
| [2024-05-27 23:19:36,441][1934158] Avg episode rewards: #0: 24.039, true rewards: #0: 10.706 | |
| [2024-05-27 23:19:36,441][1934158] Avg episode reward: 24.039, avg true_objective: 10.706 | |
| [2024-05-27 23:19:36,491][1934158] Num frames 9700... | |
| [2024-05-27 23:19:36,600][1934158] Num frames 9800... | |
| [2024-05-27 23:19:36,669][1934158] Num frames 9900... | |
| [2024-05-27 23:19:36,743][1934158] Num frames 10000... | |
| [2024-05-27 23:19:36,810][1934158] Num frames 10100... | |
| [2024-05-27 23:19:36,895][1934158] Avg episode rewards: #0: 22.550, true rewards: #0: 10.150 | |
| [2024-05-27 23:19:36,895][1934158] Avg episode reward: 22.550, avg true_objective: 10.150 | |
| [2024-05-27 23:19:49,448][1934158] Replay video saved to /media/fast/code/learning/train_dir/default_experiment/replay.mp4! | |
| [2024-05-27 23:19:49,685][1934158] Loading existing experiment configuration from /media/fast/code/learning/train_dir/default_experiment/config.json | |
| [2024-05-27 23:19:49,685][1934158] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'hf_repository'='DavidPL1/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-05-27 23:19:49,685][1934158] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-05-27 23:19:49,688][1934158] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:19:49,689][1934158] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:19:49,694][1934158] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:19:49,712][1934158] Conv encoder output size: 512 | |
| [2024-05-27 23:19:49,712][1934158] Policy head output size: 512 | |
| [2024-05-27 23:19:49,721][1934158] Loading state from checkpoint /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-05-27 23:19:50,123][1934158] Num frames 100... | |
| [2024-05-27 23:19:50,179][1934158] Num frames 200... | |
| [2024-05-27 23:19:50,237][1934158] Num frames 300... | |
| [2024-05-27 23:19:50,298][1934158] Num frames 400... | |
| [2024-05-27 23:19:50,364][1934158] Num frames 500... | |
| [2024-05-27 23:19:50,424][1934158] Num frames 600... | |
| [2024-05-27 23:19:50,483][1934158] Num frames 700... | |
| [2024-05-27 23:19:50,544][1934158] Num frames 800... | |
| [2024-05-27 23:19:50,615][1934158] Avg episode rewards: #0: 15.320, true rewards: #0: 8.320 | |
| [2024-05-27 23:19:50,616][1934158] Avg episode reward: 15.320, avg true_objective: 8.320 | |
| [2024-05-27 23:19:50,657][1934158] Num frames 900... | |
| [2024-05-27 23:19:50,717][1934158] Num frames 1000... | |
| [2024-05-27 23:19:50,777][1934158] Num frames 1100... | |
| [2024-05-27 23:19:50,834][1934158] Num frames 1200... | |
| [2024-05-27 23:19:50,898][1934158] Num frames 1300... | |
| [2024-05-27 23:19:51,089][1934158] Num frames 1400... | |
| [2024-05-27 23:19:51,192][1934158] Num frames 1500... | |
| [2024-05-27 23:19:51,258][1934158] Num frames 1600... | |
| [2024-05-27 23:19:51,324][1934158] Num frames 1700... | |
| [2024-05-27 23:19:51,544][1934158] Num frames 1800... | |
| [2024-05-27 23:19:51,627][1934158] Num frames 1900... | |
| [2024-05-27 23:19:51,718][1934158] Num frames 2000... | |
| [2024-05-27 23:19:51,785][1934158] Num frames 2100... | |
| [2024-05-27 23:19:51,866][1934158] Avg episode rewards: #0: 23.700, true rewards: #0: 10.700 | |
| [2024-05-27 23:19:51,866][1934158] Avg episode reward: 23.700, avg true_objective: 10.700 | |
| [2024-05-27 23:19:51,915][1934158] Num frames 2200... | |
| [2024-05-27 23:19:51,983][1934158] Num frames 2300... | |
| [2024-05-27 23:19:52,098][1934158] Num frames 2400... | |
| [2024-05-27 23:19:52,189][1934158] Num frames 2500... | |
| [2024-05-27 23:19:52,259][1934158] Num frames 2600... | |
| [2024-05-27 23:19:52,432][1934158] Num frames 2700... | |
| [2024-05-27 23:19:52,507][1934158] Num frames 2800... | |
| [2024-05-27 23:19:52,571][1934158] Num frames 2900... | |
| [2024-05-27 23:19:52,754][1934158] Num frames 3000... | |
| [2024-05-27 23:19:52,874][1934158] Num frames 3100... | |
| [2024-05-27 23:19:52,940][1934158] Num frames 3200... | |
| [2024-05-27 23:19:53,007][1934158] Num frames 3300... | |
| [2024-05-27 23:19:53,074][1934158] Num frames 3400... | |
| [2024-05-27 23:19:53,180][1934158] Avg episode rewards: #0: 26.280, true rewards: #0: 11.613 | |
| [2024-05-27 23:19:53,180][1934158] Avg episode reward: 26.280, avg true_objective: 11.613 | |
| [2024-05-27 23:19:53,191][1934158] Num frames 3500... | |
| [2024-05-27 23:19:53,254][1934158] Num frames 3600... | |
| [2024-05-27 23:19:53,331][1934158] Num frames 3700... | |
| [2024-05-27 23:19:53,423][1934158] Num frames 3800... | |
| [2024-05-27 23:19:53,511][1934158] Num frames 3900... | |
| [2024-05-27 23:19:53,578][1934158] Num frames 4000... | |
| [2024-05-27 23:19:53,649][1934158] Num frames 4100... | |
| [2024-05-27 23:19:53,715][1934158] Num frames 4200... | |
| [2024-05-27 23:19:53,811][1934158] Num frames 4300... | |
| [2024-05-27 23:19:53,883][1934158] Num frames 4400... | |
| [2024-05-27 23:19:53,953][1934158] Num frames 4500... | |
| [2024-05-27 23:19:54,033][1934158] Num frames 4600... | |
| [2024-05-27 23:19:54,114][1934158] Num frames 4700... | |
| [2024-05-27 23:19:54,212][1934158] Avg episode rewards: #0: 26.830, true rewards: #0: 11.830 | |
| [2024-05-27 23:19:54,213][1934158] Avg episode reward: 26.830, avg true_objective: 11.830 | |
| [2024-05-27 23:19:54,258][1934158] Num frames 4800... | |
| [2024-05-27 23:19:54,415][1934158] Num frames 4900... | |
| [2024-05-27 23:19:54,590][1934158] Num frames 5000... | |
| [2024-05-27 23:19:54,654][1934158] Num frames 5100... | |
| [2024-05-27 23:19:54,722][1934158] Num frames 5200... | |
| [2024-05-27 23:19:54,785][1934158] Num frames 5300... | |
| [2024-05-27 23:19:54,876][1934158] Num frames 5400... | |
| [2024-05-27 23:19:55,026][1934158] Num frames 5500... | |
| [2024-05-27 23:19:55,145][1934158] Num frames 5600... | |
| [2024-05-27 23:19:55,269][1934158] Num frames 5700... | |
| [2024-05-27 23:19:55,339][1934158] Num frames 5800... | |
| [2024-05-27 23:19:55,535][1934158] Num frames 5900... | |
| [2024-05-27 23:19:55,627][1934158] Num frames 6000... | |
| [2024-05-27 23:19:55,699][1934158] Num frames 6100... | |
| [2024-05-27 23:19:55,766][1934158] Num frames 6200... | |
| [2024-05-27 23:19:55,834][1934158] Num frames 6300... | |
| [2024-05-27 23:19:55,900][1934158] Num frames 6400... | |
| [2024-05-27 23:19:56,032][1934158] Num frames 6500... | |
| [2024-05-27 23:19:56,229][1934158] Num frames 6600... | |
| [2024-05-27 23:19:56,296][1934158] Num frames 6700... | |
| [2024-05-27 23:19:56,406][1934158] Num frames 6800... | |
| [2024-05-27 23:19:56,480][1934158] Avg episode rewards: #0: 33.064, true rewards: #0: 13.664 | |
| [2024-05-27 23:19:56,481][1934158] Avg episode reward: 33.064, avg true_objective: 13.664 | |
| [2024-05-27 23:19:56,526][1934158] Num frames 6900... | |
| [2024-05-27 23:19:56,763][1934158] Num frames 7000... | |
| [2024-05-27 23:19:56,865][1934158] Num frames 7100... | |
| [2024-05-27 23:19:56,932][1934158] Num frames 7200... | |
| [2024-05-27 23:19:57,014][1934158] Num frames 7300... | |
| [2024-05-27 23:19:57,145][1934158] Avg episode rewards: #0: 29.163, true rewards: #0: 12.330 | |
| [2024-05-27 23:19:57,145][1934158] Avg episode reward: 29.163, avg true_objective: 12.330 | |
| [2024-05-27 23:19:57,147][1934158] Num frames 7400... | |
| [2024-05-27 23:19:57,215][1934158] Num frames 7500... | |
| [2024-05-27 23:19:57,399][1934158] Num frames 7600... | |
| [2024-05-27 23:19:57,518][1934158] Num frames 7700... | |
| [2024-05-27 23:19:57,586][1934158] Num frames 7800... | |
| [2024-05-27 23:19:57,695][1934158] Num frames 7900... | |
| [2024-05-27 23:19:57,823][1934158] Num frames 8000... | |
| [2024-05-27 23:19:57,964][1934158] Num frames 8100... | |
| [2024-05-27 23:19:58,032][1934158] Num frames 8200... | |
| [2024-05-27 23:19:58,101][1934158] Num frames 8300... | |
| [2024-05-27 23:19:58,168][1934158] Num frames 8400... | |
| [2024-05-27 23:19:58,262][1934158] Num frames 8500... | |
| [2024-05-27 23:19:58,340][1934158] Num frames 8600... | |
| [2024-05-27 23:19:58,411][1934158] Num frames 8700... | |
| [2024-05-27 23:19:58,523][1934158] Num frames 8800... | |
| [2024-05-27 23:19:58,660][1934158] Num frames 8900... | |
| [2024-05-27 23:19:58,727][1934158] Num frames 9000... | |
| [2024-05-27 23:19:58,802][1934158] Num frames 9100... | |
| [2024-05-27 23:19:59,012][1934158] Num frames 9200... | |
| [2024-05-27 23:19:59,116][1934158] Num frames 9300... | |
| [2024-05-27 23:19:59,185][1934158] Num frames 9400... | |
| [2024-05-27 23:19:59,306][1934158] Avg episode rewards: #0: 33.711, true rewards: #0: 13.569 | |
| [2024-05-27 23:19:59,306][1934158] Avg episode reward: 33.711, avg true_objective: 13.569 | |
| [2024-05-27 23:19:59,307][1934158] Num frames 9500... | |
| [2024-05-27 23:19:59,421][1934158] Num frames 9600... | |
| [2024-05-27 23:19:59,512][1934158] Num frames 9700... | |
| [2024-05-27 23:19:59,576][1934158] Num frames 9800... | |
| [2024-05-27 23:19:59,647][1934158] Num frames 9900... | |
| [2024-05-27 23:19:59,729][1934158] Avg episode rewards: #0: 30.182, true rewards: #0: 12.433 | |
| [2024-05-27 23:19:59,729][1934158] Avg episode reward: 30.182, avg true_objective: 12.433 | |
| [2024-05-27 23:19:59,775][1934158] Num frames 10000... | |
| [2024-05-27 23:19:59,844][1934158] Num frames 10100... | |
| [2024-05-27 23:19:59,909][1934158] Num frames 10200... | |
| [2024-05-27 23:20:00,028][1934158] Num frames 10300... | |
| [2024-05-27 23:20:00,151][1934158] Num frames 10400... | |
| [2024-05-27 23:20:00,252][1934158] Num frames 10500... | |
| [2024-05-27 23:20:00,320][1934158] Num frames 10600... | |
| [2024-05-27 23:20:00,496][1934158] Num frames 10700... | |
| [2024-05-27 23:20:00,583][1934158] Num frames 10800... | |
| [2024-05-27 23:20:00,669][1934158] Num frames 10900... | |
| [2024-05-27 23:20:00,763][1934158] Num frames 11000... | |
| [2024-05-27 23:20:00,883][1934158] Num frames 11100... | |
| [2024-05-27 23:20:00,972][1934158] Num frames 11200... | |
| [2024-05-27 23:20:01,045][1934158] Num frames 11300... | |
| [2024-05-27 23:20:01,140][1934158] Num frames 11400... | |
| [2024-05-27 23:20:01,322][1934158] Num frames 11500... | |
| [2024-05-27 23:20:01,404][1934158] Avg episode rewards: #0: 31.269, true rewards: #0: 12.824 | |
| [2024-05-27 23:20:01,404][1934158] Avg episode reward: 31.269, avg true_objective: 12.824 | |
| [2024-05-27 23:20:01,495][1934158] Num frames 11600... | |
| [2024-05-27 23:20:01,560][1934158] Num frames 11700... | |
| [2024-05-27 23:20:01,631][1934158] Num frames 11800... | |
| [2024-05-27 23:20:01,747][1934158] Num frames 11900... | |
| [2024-05-27 23:20:01,829][1934158] Num frames 12000... | |
| [2024-05-27 23:20:01,911][1934158] Num frames 12100... | |
| [2024-05-27 23:20:02,025][1934158] Num frames 12200... | |
| [2024-05-27 23:20:02,094][1934158] Num frames 12300... | |
| [2024-05-27 23:20:02,161][1934158] Num frames 12400... | |
| [2024-05-27 23:20:02,230][1934158] Num frames 12500... | |
| [2024-05-27 23:20:02,301][1934158] Num frames 12600... | |
| [2024-05-27 23:20:02,394][1934158] Avg episode rewards: #0: 30.430, true rewards: #0: 12.630 | |
| [2024-05-27 23:20:02,394][1934158] Avg episode reward: 30.430, avg true_objective: 12.630 | |
| [2024-05-27 23:20:17,697][1934158] Replay video saved to /media/fast/code/learning/train_dir/default_experiment/replay.mp4! | |
| [2024-05-27 23:20:23,037][1934158] The model has been pushed to https://huggingface.co/DavidPL1/rl_course_vizdoom_health_gathering_supreme | |
| [2024-05-27 23:24:06,850][1944769] Saving configuration to /media/fast/code/learning/train_dir/default_experiment/config.json... | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 0 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 1 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 2 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 3 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 4 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 5 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 6 uses device cpu | |
| [2024-05-27 23:24:06,851][1944769] Rollout worker 7 uses device cpu | |
| [2024-05-27 23:24:06,885][1944769] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:24:06,885][1944769] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-05-27 23:24:06,898][1944769] Starting all processes... | |
| [2024-05-27 23:24:06,898][1944769] Starting process learner_proc0 | |
| [2024-05-27 23:24:08,668][1944769] Starting all processes... | |
| [2024-05-27 23:24:08,671][1944769] Starting process inference_proc0-0 | |
| [2024-05-27 23:24:08,671][1944769] Starting process rollout_proc0 | |
| [2024-05-27 23:24:08,672][1944769] Starting process rollout_proc4 | |
| [2024-05-27 23:24:08,672][1944894] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:24:08,672][1944894] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-05-27 23:24:08,671][1944769] Starting process rollout_proc2 | |
| [2024-05-27 23:24:08,672][1944769] Starting process rollout_proc3 | |
| [2024-05-27 23:24:08,671][1944769] Starting process rollout_proc1 | |
| [2024-05-27 23:24:08,672][1944769] Starting process rollout_proc5 | |
| [2024-05-27 23:24:08,672][1944769] Starting process rollout_proc6 | |
| [2024-05-27 23:24:08,682][1944894] Num visible devices: 1 | |
| [2024-05-27 23:24:08,672][1944769] Starting process rollout_proc7 | |
| [2024-05-27 23:24:08,707][1944894] Starting seed is not provided | |
| [2024-05-27 23:24:08,707][1944894] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:24:08,707][1944894] Initializing actor-critic model on device cuda:0 | |
| [2024-05-27 23:24:08,708][1944894] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:24:08,708][1944894] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:24:08,721][1944894] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:24:08,826][1944894] Conv encoder output size: 512 | |
| [2024-05-27 23:24:08,827][1944894] Policy head output size: 512 | |
| [2024-05-27 23:24:08,840][1944894] Created Actor Critic model with architecture: | |
| [2024-05-27 23:24:08,840][1944894] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-05-27 23:24:08,935][1944894] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-05-27 23:24:09,730][1944894] Loading state from checkpoint /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-05-27 23:24:09,767][1944894] Loading model from checkpoint | |
| [2024-05-27 23:24:09,769][1944894] Loaded experiment state at self.train_step=978, self.env_steps=4005888 | |
| [2024-05-27 23:24:09,769][1944894] Initialized policy 0 weights for model version 978 | |
| [2024-05-27 23:24:09,771][1944894] LearnerWorker_p0 finished initialization! | |
| [2024-05-27 23:24:09,771][1944894] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:24:10,329][1944942] Worker 3 uses CPU cores [6, 7] | |
| [2024-05-27 23:24:10,513][1944941] Worker 2 uses CPU cores [4, 5] | |
| [2024-05-27 23:24:10,610][1944938] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-05-27 23:24:10,610][1944938] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-05-27 23:24:10,620][1944938] Num visible devices: 1 | |
| [2024-05-27 23:24:10,718][1944940] Worker 4 uses CPU cores [8, 9] | |
| [2024-05-27 23:24:10,777][1944959] Worker 5 uses CPU cores [10, 11] | |
| [2024-05-27 23:24:10,791][1944960] Worker 6 uses CPU cores [12, 13] | |
| [2024-05-27 23:24:10,810][1944939] Worker 0 uses CPU cores [0, 1] | |
| [2024-05-27 23:24:10,825][1944769] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-05-27 23:24:10,826][1944943] Worker 1 uses CPU cores [2, 3] | |
| [2024-05-27 23:24:10,829][1944961] Worker 7 uses CPU cores [14, 15] | |
| [2024-05-27 23:24:10,925][1944938] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:24:10,926][1944938] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:24:10,932][1944938] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:24:10,989][1944938] Conv encoder output size: 512 | |
| [2024-05-27 23:24:10,989][1944938] Policy head output size: 512 | |
| [2024-05-27 23:24:11,020][1944769] Inference worker 0-0 is ready! | |
| [2024-05-27 23:24:11,020][1944769] All inference workers are ready! Signal rollout workers to start! | |
| [2024-05-27 23:24:11,031][1944941] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,033][1944960] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,035][1944959] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,036][1944940] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,036][1944943] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,036][1944939] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,036][1944961] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,036][1944942] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:24:11,635][1944940] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:11,716][1944959] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:11,795][1944943] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:11,796][1944941] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:11,798][1944960] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:12,598][1944940] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:12,600][1944959] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:12,679][1944942] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:12,679][1944961] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:12,681][1944960] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:12,997][1944941] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:13,512][1944940] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:13,592][1944942] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:13,594][1944943] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:13,594][1944959] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:13,997][1944939] Decorrelating experience for 0 frames... | |
| [2024-05-27 23:24:14,001][1944960] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:14,080][1944941] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:14,331][1944769] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-05-27 23:24:14,411][1944942] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:14,490][1944959] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:14,729][1944940] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:14,811][1944939] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:14,889][1944943] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:15,357][1944942] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:15,358][1944960] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:15,359][1944959] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:15,681][1944961] Decorrelating experience for 32 frames... | |
| [2024-05-27 23:24:15,765][1944940] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:15,843][1944943] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:16,252][1944939] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:16,419][1944959] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:16,419][1944960] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:16,674][1944942] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:16,759][1944941] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:16,760][1944961] Decorrelating experience for 64 frames... | |
| [2024-05-27 23:24:17,276][1944939] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:17,359][1944940] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:17,592][1944960] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:17,676][1944959] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:17,764][1944961] Decorrelating experience for 96 frames... | |
| [2024-05-27 23:24:17,764][1944943] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:18,096][1944939] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:18,410][1944940] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:18,572][1944960] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:18,648][1944959] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:18,649][1944941] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:18,649][1944961] Decorrelating experience for 128 frames... | |
| [2024-05-27 23:24:19,132][1944942] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:19,288][1944939] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:19,291][1944960] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:19,331][1944769] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-05-27 23:24:19,332][1944769] Avg episode reward: [(0, '0.640')] | |
| [2024-05-27 23:24:19,460][1944940] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:19,795][1944943] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:19,872][1944961] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:20,033][1944942] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:20,147][1944894] Signal inference workers to stop experience collection... | |
| [2024-05-27 23:24:20,150][1944938] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-05-27 23:24:20,353][1944943] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:20,357][1944939] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:20,437][1944961] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:20,795][1944942] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:20,961][1944894] Signal inference workers to resume experience collection... | |
| [2024-05-27 23:24:20,962][1944938] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-05-27 23:24:20,962][1944939] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:21,051][1944961] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:21,251][1944941] Decorrelating experience for 160 frames... | |
| [2024-05-27 23:24:21,685][1944943] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:21,768][1944941] Decorrelating experience for 192 frames... | |
| [2024-05-27 23:24:22,199][1944941] Decorrelating experience for 224 frames... | |
| [2024-05-27 23:24:22,633][1944938] Updated weights for policy 0, policy_version 988 (0.0142) | |
| [2024-05-27 23:24:23,973][1944938] Updated weights for policy 0, policy_version 998 (0.0009) | |
| [2024-05-27 23:24:24,332][1944769] Fps is (10 sec: 9011.1, 60 sec: 6671.7, 300 sec: 6671.7). Total num frames: 4096000. Throughput: 0: 1034.8. Samples: 13976. Policy #0 lag: (min: 0.0, avg: 1.3, max: 4.0) | |
| [2024-05-27 23:24:24,332][1944769] Avg episode reward: [(0, '14.106')] | |
| [2024-05-27 23:24:25,343][1944938] Updated weights for policy 0, policy_version 1008 (0.0008) | |
| [2024-05-27 23:24:26,541][1944938] Updated weights for policy 0, policy_version 1018 (0.0008) | |
| [2024-05-27 23:24:26,882][1944769] Heartbeat connected on Batcher_0 | |
| [2024-05-27 23:24:26,883][1944769] Heartbeat connected on LearnerWorker_p0 | |
| [2024-05-27 23:24:26,889][1944769] Heartbeat connected on RolloutWorker_w0 | |
| [2024-05-27 23:24:26,890][1944769] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-05-27 23:24:26,892][1944769] Heartbeat connected on RolloutWorker_w3 | |
| [2024-05-27 23:24:26,893][1944769] Heartbeat connected on RolloutWorker_w1 | |
| [2024-05-27 23:24:26,895][1944769] Heartbeat connected on RolloutWorker_w5 | |
| [2024-05-27 23:24:26,896][1944769] Heartbeat connected on RolloutWorker_w2 | |
| [2024-05-27 23:24:26,901][1944769] Heartbeat connected on RolloutWorker_w4 | |
| [2024-05-27 23:24:26,902][1944769] Heartbeat connected on RolloutWorker_w7 | |
| [2024-05-27 23:24:26,902][1944769] Heartbeat connected on RolloutWorker_w6 | |
| [2024-05-27 23:24:27,857][1944938] Updated weights for policy 0, policy_version 1028 (0.0009) | |
| [2024-05-27 23:24:29,131][1944938] Updated weights for policy 0, policy_version 1038 (0.0008) | |
| [2024-05-27 23:24:29,331][1944769] Fps is (10 sec: 24985.6, 60 sec: 13501.0, 300 sec: 13501.0). Total num frames: 4255744. Throughput: 0: 3344.3. Samples: 61892. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2024-05-27 23:24:29,332][1944769] Avg episode reward: [(0, '19.936')] | |
| [2024-05-27 23:24:30,405][1944938] Updated weights for policy 0, policy_version 1048 (0.0008) | |
| [2024-05-27 23:24:31,658][1944938] Updated weights for policy 0, policy_version 1058 (0.0008) | |
| [2024-05-27 23:24:32,904][1944938] Updated weights for policy 0, policy_version 1068 (0.0007) | |
| [2024-05-27 23:24:34,244][1944938] Updated weights for policy 0, policy_version 1078 (0.0007) | |
| [2024-05-27 23:24:34,331][1944769] Fps is (10 sec: 31949.2, 60 sec: 17425.0, 300 sec: 17425.0). Total num frames: 4415488. Throughput: 0: 3679.0. Samples: 86480. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2024-05-27 23:24:34,332][1944769] Avg episode reward: [(0, '24.717')] | |
| [2024-05-27 23:24:34,334][1944894] Saving new best policy, reward=24.717! | |
| [2024-05-27 23:24:35,485][1944938] Updated weights for policy 0, policy_version 1088 (0.0008) | |
| [2024-05-27 23:24:37,044][1944938] Updated weights for policy 0, policy_version 1098 (0.0014) | |
| [2024-05-27 23:24:38,236][1944938] Updated weights for policy 0, policy_version 1108 (0.0008) | |
| [2024-05-27 23:24:39,331][1944769] Fps is (10 sec: 31539.0, 60 sec: 19828.7, 300 sec: 19828.7). Total num frames: 4571136. Throughput: 0: 4656.3. Samples: 132736. Policy #0 lag: (min: 0.0, avg: 1.9, max: 3.0) | |
| [2024-05-27 23:24:39,332][1944769] Avg episode reward: [(0, '24.188')] | |
| [2024-05-27 23:24:39,508][1944938] Updated weights for policy 0, policy_version 1118 (0.0012) | |
| [2024-05-27 23:24:40,818][1944938] Updated weights for policy 0, policy_version 1128 (0.0009) | |
| [2024-05-27 23:24:42,144][1944938] Updated weights for policy 0, policy_version 1138 (0.0008) | |
| [2024-05-27 23:24:43,414][1944938] Updated weights for policy 0, policy_version 1148 (0.0008) | |
| [2024-05-27 23:24:44,332][1944769] Fps is (10 sec: 31538.7, 60 sec: 21637.2, 300 sec: 21637.2). Total num frames: 4730880. Throughput: 0: 5386.4. Samples: 180480. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2024-05-27 23:24:44,332][1944769] Avg episode reward: [(0, '23.123')] | |
| [2024-05-27 23:24:44,683][1944938] Updated weights for policy 0, policy_version 1158 (0.0010) | |
| [2024-05-27 23:24:45,942][1944938] Updated weights for policy 0, policy_version 1168 (0.0008) | |
| [2024-05-27 23:24:47,177][1944938] Updated weights for policy 0, policy_version 1178 (0.0007) | |
| [2024-05-27 23:24:48,496][1944938] Updated weights for policy 0, policy_version 1188 (0.0007) | |
| [2024-05-27 23:24:49,332][1944769] Fps is (10 sec: 31948.8, 60 sec: 22976.2, 300 sec: 22976.2). Total num frames: 4890624. Throughput: 0: 5315.0. Samples: 204664. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2024-05-27 23:24:49,332][1944769] Avg episode reward: [(0, '21.083')] | |
| [2024-05-27 23:24:49,810][1944938] Updated weights for policy 0, policy_version 1198 (0.0008) | |
| [2024-05-27 23:24:51,091][1944938] Updated weights for policy 0, policy_version 1208 (0.0016) | |
| [2024-05-27 23:24:52,308][1944938] Updated weights for policy 0, policy_version 1218 (0.0009) | |
| [2024-05-27 23:24:53,558][1944938] Updated weights for policy 0, policy_version 1228 (0.0008) | |
| [2024-05-27 23:24:54,331][1944769] Fps is (10 sec: 32358.9, 60 sec: 24101.6, 300 sec: 24101.6). Total num frames: 5054464. Throughput: 0: 5811.0. Samples: 252816. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2024-05-27 23:24:54,332][1944769] Avg episode reward: [(0, '21.497')] | |
| [2024-05-27 23:24:54,819][1944938] Updated weights for policy 0, policy_version 1238 (0.0008) | |
| [2024-05-27 23:24:56,059][1944938] Updated weights for policy 0, policy_version 1248 (0.0009) | |
| [2024-05-27 23:24:57,290][1944938] Updated weights for policy 0, policy_version 1258 (0.0008) | |
| [2024-05-27 23:24:58,543][1944938] Updated weights for policy 0, policy_version 1268 (0.0008) | |
| [2024-05-27 23:24:59,331][1944769] Fps is (10 sec: 32768.1, 60 sec: 24994.9, 300 sec: 24994.9). Total num frames: 5218304. Throughput: 0: 6715.3. Samples: 302188. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2024-05-27 23:24:59,332][1944769] Avg episode reward: [(0, '23.600')] | |
| [2024-05-27 23:24:59,798][1944938] Updated weights for policy 0, policy_version 1278 (0.0010) | |
| [2024-05-27 23:25:01,050][1944938] Updated weights for policy 0, policy_version 1288 (0.0008) | |
| [2024-05-27 23:25:02,329][1944938] Updated weights for policy 0, policy_version 1298 (0.0008) | |
| [2024-05-27 23:25:03,696][1944938] Updated weights for policy 0, policy_version 1308 (0.0014) | |
| [2024-05-27 23:25:04,332][1944769] Fps is (10 sec: 32357.9, 60 sec: 25644.7, 300 sec: 25644.7). Total num frames: 5378048. Throughput: 0: 7255.6. Samples: 326504. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-05-27 23:25:04,332][1944769] Avg episode reward: [(0, '22.440')] | |
| [2024-05-27 23:25:04,968][1944938] Updated weights for policy 0, policy_version 1318 (0.0008) | |
| [2024-05-27 23:25:06,188][1944938] Updated weights for policy 0, policy_version 1328 (0.0008) | |
| [2024-05-27 23:25:07,478][1944938] Updated weights for policy 0, policy_version 1338 (0.0008) | |
| [2024-05-27 23:25:08,699][1944938] Updated weights for policy 0, policy_version 1348 (0.0008) | |
| [2024-05-27 23:25:09,333][1944769] Fps is (10 sec: 32354.4, 60 sec: 26252.9, 300 sec: 26252.9). Total num frames: 5541888. Throughput: 0: 8013.2. Samples: 374580. Policy #0 lag: (min: 1.0, avg: 1.3, max: 3.0) | |
| [2024-05-27 23:25:09,333][1944769] Avg episode reward: [(0, '22.809')] | |
| [2024-05-27 23:25:09,969][1944938] Updated weights for policy 0, policy_version 1358 (0.0008) | |
| [2024-05-27 23:25:11,255][1944938] Updated weights for policy 0, policy_version 1368 (0.0018) | |
| [2024-05-27 23:25:12,552][1944938] Updated weights for policy 0, policy_version 1378 (0.0007) | |
| [2024-05-27 23:25:13,813][1944938] Updated weights for policy 0, policy_version 1388 (0.0008) | |
| [2024-05-27 23:25:14,331][1944769] Fps is (10 sec: 32358.9, 60 sec: 28262.4, 300 sec: 26701.9). Total num frames: 5701632. Throughput: 0: 8021.2. Samples: 422848. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2024-05-27 23:25:14,332][1944769] Avg episode reward: [(0, '24.407')] | |
| [2024-05-27 23:25:15,063][1944938] Updated weights for policy 0, policy_version 1398 (0.0008) | |
| [2024-05-27 23:25:16,369][1944938] Updated weights for policy 0, policy_version 1408 (0.0013) | |
| [2024-05-27 23:25:17,782][1944938] Updated weights for policy 0, policy_version 1418 (0.0011) | |
| [2024-05-27 23:25:19,073][1944938] Updated weights for policy 0, policy_version 1428 (0.0014) | |
| [2024-05-27 23:25:19,331][1944769] Fps is (10 sec: 31543.1, 60 sec: 30856.5, 300 sec: 27025.0). Total num frames: 5857280. Throughput: 0: 8008.2. Samples: 446848. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2024-05-27 23:25:19,332][1944769] Avg episode reward: [(0, '23.988')] | |
| [2024-05-27 23:25:20,331][1944938] Updated weights for policy 0, policy_version 1438 (0.0010) | |
| [2024-05-27 23:25:21,582][1944938] Updated weights for policy 0, policy_version 1448 (0.0007) | |
| [2024-05-27 23:25:22,872][1944938] Updated weights for policy 0, policy_version 1458 (0.0009) | |
| [2024-05-27 23:25:24,142][1944938] Updated weights for policy 0, policy_version 1468 (0.0013) | |
| [2024-05-27 23:25:24,331][1944769] Fps is (10 sec: 31539.1, 60 sec: 32017.1, 300 sec: 27360.0). Total num frames: 6017024. Throughput: 0: 8028.4. Samples: 494012. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2024-05-27 23:25:24,332][1944769] Avg episode reward: [(0, '24.169')] | |
| [2024-05-27 23:25:25,527][1944938] Updated weights for policy 0, policy_version 1478 (0.0008) | |
| [2024-05-27 23:25:26,732][1944938] Updated weights for policy 0, policy_version 1488 (0.0009) | |
| [2024-05-27 23:25:28,017][1944938] Updated weights for policy 0, policy_version 1498 (0.0008) | |
| [2024-05-27 23:25:29,185][1944938] Updated weights for policy 0, policy_version 1508 (0.0008) | |
| [2024-05-27 23:25:29,332][1944769] Fps is (10 sec: 32358.2, 60 sec: 32085.3, 300 sec: 27704.4). Total num frames: 6180864. Throughput: 0: 8043.0. Samples: 542416. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2024-05-27 23:25:29,332][1944769] Avg episode reward: [(0, '21.199')] | |
| [2024-05-27 23:25:30,420][1944938] Updated weights for policy 0, policy_version 1518 (0.0008) | |
| [2024-05-27 23:25:31,670][1944938] Updated weights for policy 0, policy_version 1528 (0.0012) | |
| [2024-05-27 23:25:32,874][1944938] Updated weights for policy 0, policy_version 1538 (0.0008) | |
| [2024-05-27 23:25:34,228][1944938] Updated weights for policy 0, policy_version 1548 (0.0013) | |
| [2024-05-27 23:25:34,332][1944769] Fps is (10 sec: 32358.2, 60 sec: 32085.3, 300 sec: 27958.5). Total num frames: 6340608. Throughput: 0: 8061.0. Samples: 567408. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2024-05-27 23:25:34,332][1944769] Avg episode reward: [(0, '25.027')] | |
| [2024-05-27 23:25:34,335][1944894] Saving new best policy, reward=25.027! | |
| [2024-05-27 23:25:35,469][1944938] Updated weights for policy 0, policy_version 1558 (0.0008) | |
| [2024-05-27 23:25:36,758][1944938] Updated weights for policy 0, policy_version 1568 (0.0008) | |
| [2024-05-27 23:25:38,071][1944938] Updated weights for policy 0, policy_version 1578 (0.0009) | |
| [2024-05-27 23:25:39,331][1944769] Fps is (10 sec: 31949.0, 60 sec: 32153.6, 300 sec: 28183.9). Total num frames: 6500352. Throughput: 0: 8060.3. Samples: 615528. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) | |
| [2024-05-27 23:25:39,332][1944769] Avg episode reward: [(0, '21.823')] | |
| [2024-05-27 23:25:39,412][1944938] Updated weights for policy 0, policy_version 1588 (0.0008) | |
| [2024-05-27 23:25:40,793][1944938] Updated weights for policy 0, policy_version 1598 (0.0014) | |
| [2024-05-27 23:25:42,055][1944938] Updated weights for policy 0, policy_version 1608 (0.0008) | |
| [2024-05-27 23:25:43,282][1944938] Updated weights for policy 0, policy_version 1618 (0.0010) | |
| [2024-05-27 23:25:44,331][1944769] Fps is (10 sec: 31949.1, 60 sec: 32153.7, 300 sec: 28385.3). Total num frames: 6660096. Throughput: 0: 8004.3. Samples: 662380. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2024-05-27 23:25:44,332][1944769] Avg episode reward: [(0, '22.736')] | |
| [2024-05-27 23:25:44,643][1944938] Updated weights for policy 0, policy_version 1628 (0.0014) | |
| [2024-05-27 23:25:46,073][1944938] Updated weights for policy 0, policy_version 1638 (0.0007) | |
| [2024-05-27 23:25:47,425][1944938] Updated weights for policy 0, policy_version 1648 (0.0008) | |
| [2024-05-27 23:25:48,626][1944938] Updated weights for policy 0, policy_version 1658 (0.0008) | |
| [2024-05-27 23:25:49,331][1944769] Fps is (10 sec: 31129.8, 60 sec: 32017.1, 300 sec: 28483.0). Total num frames: 6811648. Throughput: 0: 7951.6. Samples: 684324. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2024-05-27 23:25:49,332][1944769] Avg episode reward: [(0, '22.791')] | |
| [2024-05-27 23:25:49,975][1944938] Updated weights for policy 0, policy_version 1668 (0.0008) | |
| [2024-05-27 23:25:51,221][1944938] Updated weights for policy 0, policy_version 1678 (0.0009) | |
| [2024-05-27 23:25:52,458][1944938] Updated weights for policy 0, policy_version 1688 (0.0007) | |
| [2024-05-27 23:25:53,876][1944938] Updated weights for policy 0, policy_version 1698 (0.0008) | |
| [2024-05-27 23:25:54,331][1944769] Fps is (10 sec: 30720.0, 60 sec: 31880.5, 300 sec: 28610.8). Total num frames: 6967296. Throughput: 0: 7956.2. Samples: 732600. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2024-05-27 23:25:54,332][1944769] Avg episode reward: [(0, '23.574')] | |
| [2024-05-27 23:25:55,258][1944938] Updated weights for policy 0, policy_version 1708 (0.0008) | |
| [2024-05-27 23:25:56,539][1944938] Updated weights for policy 0, policy_version 1718 (0.0012) | |
| [2024-05-27 23:25:57,767][1944938] Updated weights for policy 0, policy_version 1728 (0.0013) | |
| [2024-05-27 23:25:59,041][1944938] Updated weights for policy 0, policy_version 1738 (0.0009) | |
| [2024-05-27 23:25:59,331][1944769] Fps is (10 sec: 31539.1, 60 sec: 31812.3, 300 sec: 28764.6). Total num frames: 7127040. Throughput: 0: 7920.0. Samples: 779248. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2024-05-27 23:25:59,332][1944769] Avg episode reward: [(0, '23.735')] | |
| [2024-05-27 23:26:00,279][1944938] Updated weights for policy 0, policy_version 1748 (0.0008) | |
| [2024-05-27 23:26:01,601][1944938] Updated weights for policy 0, policy_version 1758 (0.0010) | |
| [2024-05-27 23:26:02,916][1944938] Updated weights for policy 0, policy_version 1768 (0.0010) | |
| [2024-05-27 23:26:04,190][1944938] Updated weights for policy 0, policy_version 1778 (0.0009) | |
| [2024-05-27 23:26:04,332][1944769] Fps is (10 sec: 31948.3, 60 sec: 31812.3, 300 sec: 28904.9). Total num frames: 7286784. Throughput: 0: 7919.5. Samples: 803224. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2024-05-27 23:26:04,332][1944769] Avg episode reward: [(0, '25.925')] | |
| [2024-05-27 23:26:04,336][1944894] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000001779_7286784.pth... | |
| [2024-05-27 23:26:04,377][1944894] Removing /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000539_2207744.pth | |
| [2024-05-27 23:26:04,385][1944894] Saving new best policy, reward=25.925! | |
| [2024-05-27 23:26:05,437][1944938] Updated weights for policy 0, policy_version 1788 (0.0010) | |
| [2024-05-27 23:26:06,807][1944938] Updated weights for policy 0, policy_version 1798 (0.0015) | |
| [2024-05-27 23:26:08,054][1944938] Updated weights for policy 0, policy_version 1808 (0.0016) | |
| [2024-05-27 23:26:09,297][1944938] Updated weights for policy 0, policy_version 1818 (0.0013) | |
| [2024-05-27 23:26:09,332][1944769] Fps is (10 sec: 31948.4, 60 sec: 31744.6, 300 sec: 29033.3). Total num frames: 7446528. Throughput: 0: 7928.8. Samples: 850808. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2024-05-27 23:26:09,332][1944769] Avg episode reward: [(0, '25.408')] | |
| [2024-05-27 23:26:10,740][1944938] Updated weights for policy 0, policy_version 1828 (0.0008) | |
| [2024-05-27 23:26:12,001][1944938] Updated weights for policy 0, policy_version 1838 (0.0008) | |
| [2024-05-27 23:26:13,281][1944938] Updated weights for policy 0, policy_version 1848 (0.0007) | |
| [2024-05-27 23:26:14,334][1944769] Fps is (10 sec: 31530.6, 60 sec: 31674.2, 300 sec: 29117.5). Total num frames: 7602176. Throughput: 0: 7900.8. Samples: 897972. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) | |
| [2024-05-27 23:26:14,335][1944769] Avg episode reward: [(0, '23.550')] | |
| [2024-05-27 23:26:14,592][1944938] Updated weights for policy 0, policy_version 1858 (0.0008) | |
| [2024-05-27 23:26:15,901][1944938] Updated weights for policy 0, policy_version 1868 (0.0012) | |
| [2024-05-27 23:26:17,177][1944938] Updated weights for policy 0, policy_version 1878 (0.0013) | |
| [2024-05-27 23:26:18,678][1944938] Updated weights for policy 0, policy_version 1888 (0.0014) | |
| [2024-05-27 23:26:19,332][1944769] Fps is (10 sec: 30720.1, 60 sec: 31607.4, 300 sec: 29164.6). Total num frames: 7753728. Throughput: 0: 7870.1. Samples: 921564. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2024-05-27 23:26:19,332][1944769] Avg episode reward: [(0, '24.534')] | |
| [2024-05-27 23:26:19,871][1944938] Updated weights for policy 0, policy_version 1898 (0.0008) | |
| [2024-05-27 23:26:21,197][1944938] Updated weights for policy 0, policy_version 1908 (0.0008) | |
| [2024-05-27 23:26:22,533][1944938] Updated weights for policy 0, policy_version 1918 (0.0009) | |
| [2024-05-27 23:26:23,811][1944938] Updated weights for policy 0, policy_version 1928 (0.0008) | |
| [2024-05-27 23:26:24,331][1944769] Fps is (10 sec: 30728.8, 60 sec: 31539.2, 300 sec: 29238.2). Total num frames: 7909376. Throughput: 0: 7826.9. Samples: 967736. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0) | |
| [2024-05-27 23:26:24,332][1944769] Avg episode reward: [(0, '23.025')] | |
| [2024-05-27 23:26:25,114][1944938] Updated weights for policy 0, policy_version 1938 (0.0008) | |
| [2024-05-27 23:26:26,494][1944938] Updated weights for policy 0, policy_version 1948 (0.0015) | |
| [2024-05-27 23:26:27,436][1944769] Component Batcher_0 stopped! | |
| [2024-05-27 23:26:27,436][1944894] Stopping Batcher_0... | |
| [2024-05-27 23:26:27,436][1944894] Loop batcher_evt_loop terminating... | |
| [2024-05-27 23:26:27,437][1944894] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... | |
| [2024-05-27 23:26:27,450][1944960] Stopping RolloutWorker_w6... | |
| [2024-05-27 23:26:27,450][1944960] Loop rollout_proc6_evt_loop terminating... | |
| [2024-05-27 23:26:27,450][1944769] Component RolloutWorker_w6 stopped! | |
| [2024-05-27 23:26:27,451][1944942] Stopping RolloutWorker_w3... | |
| [2024-05-27 23:26:27,451][1944769] Component RolloutWorker_w3 stopped! | |
| [2024-05-27 23:26:27,451][1944939] Stopping RolloutWorker_w0... | |
| [2024-05-27 23:26:27,452][1944942] Loop rollout_proc3_evt_loop terminating... | |
| [2024-05-27 23:26:27,452][1944769] Component RolloutWorker_w0 stopped! | |
| [2024-05-27 23:26:27,452][1944939] Loop rollout_proc0_evt_loop terminating... | |
| [2024-05-27 23:26:27,452][1944940] Stopping RolloutWorker_w4... | |
| [2024-05-27 23:26:27,452][1944769] Component RolloutWorker_w4 stopped! | |
| [2024-05-27 23:26:27,452][1944769] Component RolloutWorker_w7 stopped! | |
| [2024-05-27 23:26:27,452][1944961] Stopping RolloutWorker_w7... | |
| [2024-05-27 23:26:27,452][1944940] Loop rollout_proc4_evt_loop terminating... | |
| [2024-05-27 23:26:27,452][1944961] Loop rollout_proc7_evt_loop terminating... | |
| [2024-05-27 23:26:27,453][1944769] Component RolloutWorker_w5 stopped! | |
| [2024-05-27 23:26:27,453][1944959] Stopping RolloutWorker_w5... | |
| [2024-05-27 23:26:27,452][1944938] Weights refcount: 2 0 | |
| [2024-05-27 23:26:27,453][1944959] Loop rollout_proc5_evt_loop terminating... | |
| [2024-05-27 23:26:27,470][1944941] Stopping RolloutWorker_w2... | |
| [2024-05-27 23:26:27,470][1944769] Component RolloutWorker_w2 stopped! | |
| [2024-05-27 23:26:27,470][1944941] Loop rollout_proc2_evt_loop terminating... | |
| [2024-05-27 23:26:27,482][1944894] Removing /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth | |
| [2024-05-27 23:26:27,489][1944894] Saving /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... | |
| [2024-05-27 23:26:27,490][1944943] Stopping RolloutWorker_w1... | |
| [2024-05-27 23:26:27,490][1944769] Component RolloutWorker_w1 stopped! | |
| [2024-05-27 23:26:27,491][1944943] Loop rollout_proc1_evt_loop terminating... | |
| [2024-05-27 23:26:27,536][1944938] Stopping InferenceWorker_p0-w0... | |
| [2024-05-27 23:26:27,537][1944769] Component InferenceWorker_p0-w0 stopped! | |
| [2024-05-27 23:26:27,537][1944938] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-05-27 23:26:27,566][1944894] Stopping LearnerWorker_p0... | |
| [2024-05-27 23:26:27,567][1944769] Component LearnerWorker_p0 stopped! | |
| [2024-05-27 23:26:27,567][1944894] Loop learner_proc0_evt_loop terminating... | |
| [2024-05-27 23:26:27,567][1944769] Waiting for process learner_proc0 to stop... | |
| [2024-05-27 23:26:28,282][1944769] Waiting for process inference_proc0-0 to join... | |
| [2024-05-27 23:26:28,282][1944769] Waiting for process rollout_proc0 to join... | |
| [2024-05-27 23:26:28,282][1944769] Waiting for process rollout_proc1 to join... | |
| [2024-05-27 23:26:28,282][1944769] Waiting for process rollout_proc2 to join... | |
| [2024-05-27 23:26:28,282][1944769] Waiting for process rollout_proc3 to join... | |
| [2024-05-27 23:26:28,283][1944769] Waiting for process rollout_proc4 to join... | |
| [2024-05-27 23:26:28,283][1944769] Waiting for process rollout_proc5 to join... | |
| [2024-05-27 23:26:28,283][1944769] Waiting for process rollout_proc6 to join... | |
| [2024-05-27 23:26:28,283][1944769] Waiting for process rollout_proc7 to join... | |
| [2024-05-27 23:26:28,283][1944769] Batcher 0 profile tree view: | |
| batching: 15.3245, releasing_batches: 0.0233 | |
| [2024-05-27 23:26:28,283][1944769] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0000 | |
| wait_policy_total: 9.0043 | |
| update_model: 2.2328 | |
| weight_update: 0.0020 | |
| one_step: 0.0026 | |
| handle_policy_step: 119.2006 | |
| deserialize: 5.5536, stack: 0.5907, obs_to_device_normalize: 28.7723, forward: 62.5003, send_messages: 5.0913 | |
| prepare_outputs: 12.6494 | |
| to_cpu: 7.0284 | |
| [2024-05-27 23:26:28,283][1944769] Learner 0 profile tree view: | |
| misc: 0.0041, prepare_batch: 7.3997 | |
| train: 21.3461 | |
| epoch_init: 0.0033, minibatch_init: 0.0037, losses_postprocess: 0.3643, kl_divergence: 0.4054, after_optimizer: 0.6822 | |
| calculate_losses: 8.8365 | |
| losses_init: 0.0019, forward_head: 0.5491, bptt_initial: 5.4800, tail: 0.5743, advantages_returns: 0.1749, losses: 1.0358 | |
| bptt: 0.8461 | |
| bptt_forward_core: 0.8056 | |
| update: 10.6762 | |
| clip: 0.5238 | |
| [2024-05-27 23:26:28,283][1944769] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.0804, enqueue_policy_requests: 4.6237, env_step: 65.5304, overhead: 7.3248, complete_rollouts: 0.1038 | |
| save_policy_outputs: 5.4512 | |
| split_output_tensors: 2.5760 | |
| [2024-05-27 23:26:28,283][1944769] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.0774, enqueue_policy_requests: 4.4785, env_step: 68.5710, overhead: 7.0470, complete_rollouts: 0.1004 | |
| save_policy_outputs: 5.3293 | |
| split_output_tensors: 2.4943 | |
| [2024-05-27 23:26:28,283][1944769] Loop Runner_EvtLoop terminating... | |
| [2024-05-27 23:26:28,283][1944769] Runner profile tree view: | |
| main_loop: 141.3852 | |
| [2024-05-27 23:26:28,284][1944769] Collected {0: 8007680}, FPS: 28304.2 | |
| [2024-05-27 23:26:28,493][1944769] Loading existing experiment configuration from /media/fast/code/learning/train_dir/default_experiment/config.json | |
| [2024-05-27 23:26:28,493][1944769] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-05-27 23:26:28,493][1944769] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-05-27 23:26:28,493][1944769] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-05-27 23:26:28,493][1944769] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:26:28,493][1944769] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-05-27 23:26:28,494][1944769] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-05-27 23:26:28,500][1944769] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-05-27 23:26:28,501][1944769] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:26:28,502][1944769] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:26:28,508][1944769] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:26:28,562][1944769] Conv encoder output size: 512 | |
| [2024-05-27 23:26:28,563][1944769] Policy head output size: 512 | |
| [2024-05-27 23:26:28,832][1944769] Loading state from checkpoint /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... | |
| [2024-05-27 23:26:29,547][1944769] Num frames 100... | |
| [2024-05-27 23:26:29,608][1944769] Num frames 200... | |
| [2024-05-27 23:26:29,666][1944769] Num frames 300... | |
| [2024-05-27 23:26:29,736][1944769] Num frames 400... | |
| [2024-05-27 23:26:29,810][1944769] Num frames 500... | |
| [2024-05-27 23:26:29,918][1944769] Avg episode rewards: #0: 12.920, true rewards: #0: 5.920 | |
| [2024-05-27 23:26:29,918][1944769] Avg episode reward: 12.920, avg true_objective: 5.920 | |
| [2024-05-27 23:26:29,923][1944769] Num frames 600... | |
| [2024-05-27 23:26:29,980][1944769] Num frames 700... | |
| [2024-05-27 23:26:30,040][1944769] Num frames 800... | |
| [2024-05-27 23:26:30,094][1944769] Num frames 900... | |
| [2024-05-27 23:26:30,152][1944769] Num frames 1000... | |
| [2024-05-27 23:26:30,206][1944769] Num frames 1100... | |
| [2024-05-27 23:26:30,266][1944769] Num frames 1200... | |
| [2024-05-27 23:26:30,321][1944769] Num frames 1300... | |
| [2024-05-27 23:26:30,497][1944769] Avg episode rewards: #0: 14.300, true rewards: #0: 6.800 | |
| [2024-05-27 23:26:30,497][1944769] Avg episode reward: 14.300, avg true_objective: 6.800 | |
| [2024-05-27 23:26:30,522][1944769] Num frames 1400... | |
| [2024-05-27 23:26:30,580][1944769] Num frames 1500... | |
| [2024-05-27 23:26:30,637][1944769] Num frames 1600... | |
| [2024-05-27 23:26:30,696][1944769] Num frames 1700... | |
| [2024-05-27 23:26:30,760][1944769] Num frames 1800... | |
| [2024-05-27 23:26:30,825][1944769] Num frames 1900... | |
| [2024-05-27 23:26:30,884][1944769] Num frames 2000... | |
| [2024-05-27 23:26:30,980][1944769] Avg episode rewards: #0: 13.897, true rewards: #0: 6.897 | |
| [2024-05-27 23:26:30,980][1944769] Avg episode reward: 13.897, avg true_objective: 6.897 | |
| [2024-05-27 23:26:30,999][1944769] Num frames 2100... | |
| [2024-05-27 23:26:31,061][1944769] Num frames 2200... | |
| [2024-05-27 23:26:31,125][1944769] Num frames 2300... | |
| [2024-05-27 23:26:31,190][1944769] Num frames 2400... | |
| [2024-05-27 23:26:31,259][1944769] Num frames 2500... | |
| [2024-05-27 23:26:31,326][1944769] Num frames 2600... | |
| [2024-05-27 23:26:31,393][1944769] Num frames 2700... | |
| [2024-05-27 23:26:31,458][1944769] Num frames 2800... | |
| [2024-05-27 23:26:31,520][1944769] Num frames 2900... | |
| [2024-05-27 23:26:31,582][1944769] Num frames 3000... | |
| [2024-05-27 23:26:31,646][1944769] Num frames 3100... | |
| [2024-05-27 23:26:31,712][1944769] Num frames 3200... | |
| [2024-05-27 23:26:31,777][1944769] Num frames 3300... | |
| [2024-05-27 23:26:31,841][1944769] Num frames 3400... | |
| [2024-05-27 23:26:31,932][1944769] Avg episode rewards: #0: 18.405, true rewards: #0: 8.655 | |
| [2024-05-27 23:26:31,932][1944769] Avg episode reward: 18.405, avg true_objective: 8.655 | |
| [2024-05-27 23:26:31,956][1944769] Num frames 3500... | |
| [2024-05-27 23:26:32,011][1944769] Num frames 3600... | |
| [2024-05-27 23:26:32,073][1944769] Num frames 3700... | |
| [2024-05-27 23:26:32,129][1944769] Num frames 3800... | |
| [2024-05-27 23:26:32,191][1944769] Num frames 3900... | |
| [2024-05-27 23:26:32,256][1944769] Num frames 4000... | |
| [2024-05-27 23:26:32,319][1944769] Num frames 4100... | |
| [2024-05-27 23:26:32,383][1944769] Num frames 4200... | |
| [2024-05-27 23:26:32,445][1944769] Num frames 4300... | |
| [2024-05-27 23:26:32,508][1944769] Num frames 4400... | |
| [2024-05-27 23:26:32,572][1944769] Num frames 4500... | |
| [2024-05-27 23:26:32,633][1944769] Num frames 4600... | |
| [2024-05-27 23:26:32,692][1944769] Num frames 4700... | |
| [2024-05-27 23:26:32,755][1944769] Num frames 4800... | |
| [2024-05-27 23:26:32,821][1944769] Num frames 4900... | |
| [2024-05-27 23:26:32,880][1944769] Num frames 5000... | |
| [2024-05-27 23:26:32,942][1944769] Num frames 5100... | |
| [2024-05-27 23:26:33,004][1944769] Num frames 5200... | |
| [2024-05-27 23:26:33,091][1944769] Avg episode rewards: #0: 24.908, true rewards: #0: 10.508 | |
| [2024-05-27 23:26:33,092][1944769] Avg episode reward: 24.908, avg true_objective: 10.508 | |
| [2024-05-27 23:26:33,119][1944769] Num frames 5300... | |
| [2024-05-27 23:26:33,183][1944769] Num frames 5400... | |
| [2024-05-27 23:26:33,243][1944769] Num frames 5500... | |
| [2024-05-27 23:26:33,300][1944769] Num frames 5600... | |
| [2024-05-27 23:26:33,363][1944769] Num frames 5700... | |
| [2024-05-27 23:26:33,424][1944769] Num frames 5800... | |
| [2024-05-27 23:26:33,486][1944769] Num frames 5900... | |
| [2024-05-27 23:26:33,553][1944769] Num frames 6000... | |
| [2024-05-27 23:26:33,617][1944769] Num frames 6100... | |
| [2024-05-27 23:26:33,681][1944769] Num frames 6200... | |
| [2024-05-27 23:26:33,755][1944769] Num frames 6300... | |
| [2024-05-27 23:26:33,827][1944769] Num frames 6400... | |
| [2024-05-27 23:26:33,939][1944769] Avg episode rewards: #0: 25.983, true rewards: #0: 10.817 | |
| [2024-05-27 23:26:33,939][1944769] Avg episode reward: 25.983, avg true_objective: 10.817 | |
| [2024-05-27 23:26:33,947][1944769] Num frames 6500... | |
| [2024-05-27 23:26:34,086][1944769] Num frames 6600... | |
| [2024-05-27 23:26:34,206][1944769] Num frames 6700... | |
| [2024-05-27 23:26:34,274][1944769] Num frames 6800... | |
| [2024-05-27 23:26:34,346][1944769] Num frames 6900... | |
| [2024-05-27 23:26:34,502][1944769] Num frames 7000... | |
| [2024-05-27 23:26:34,609][1944769] Num frames 7100... | |
| [2024-05-27 23:26:34,698][1944769] Num frames 7200... | |
| [2024-05-27 23:26:34,814][1944769] Num frames 7300... | |
| [2024-05-27 23:26:34,891][1944769] Avg episode rewards: #0: 24.753, true rewards: #0: 10.467 | |
| [2024-05-27 23:26:34,891][1944769] Avg episode reward: 24.753, avg true_objective: 10.467 | |
| [2024-05-27 23:26:34,940][1944769] Num frames 7400... | |
| [2024-05-27 23:26:35,044][1944769] Num frames 7500... | |
| [2024-05-27 23:26:35,110][1944769] Num frames 7600... | |
| [2024-05-27 23:26:35,191][1944769] Num frames 7700... | |
| [2024-05-27 23:26:35,260][1944769] Num frames 7800... | |
| [2024-05-27 23:26:35,348][1944769] Num frames 7900... | |
| [2024-05-27 23:26:35,498][1944769] Avg episode rewards: #0: 22.929, true rewards: #0: 9.929 | |
| [2024-05-27 23:26:35,499][1944769] Avg episode reward: 22.929, avg true_objective: 9.929 | |
| [2024-05-27 23:26:35,538][1944769] Num frames 8000... | |
| [2024-05-27 23:26:35,630][1944769] Num frames 8100... | |
| [2024-05-27 23:26:35,731][1944769] Num frames 8200... | |
| [2024-05-27 23:26:35,882][1944769] Num frames 8300... | |
| [2024-05-27 23:26:35,971][1944769] Num frames 8400... | |
| [2024-05-27 23:26:36,041][1944769] Num frames 8500... | |
| [2024-05-27 23:26:36,140][1944769] Num frames 8600... | |
| [2024-05-27 23:26:36,295][1944769] Num frames 8700... | |
| [2024-05-27 23:26:36,377][1944769] Num frames 8800... | |
| [2024-05-27 23:26:36,446][1944769] Num frames 8900... | |
| [2024-05-27 23:26:36,514][1944769] Num frames 9000... | |
| [2024-05-27 23:26:36,602][1944769] Num frames 9100... | |
| [2024-05-27 23:26:36,670][1944769] Num frames 9200... | |
| [2024-05-27 23:26:36,737][1944769] Num frames 9300... | |
| [2024-05-27 23:26:36,826][1944769] Num frames 9400... | |
| [2024-05-27 23:26:36,926][1944769] Num frames 9500... | |
| [2024-05-27 23:26:37,023][1944769] Num frames 9600... | |
| [2024-05-27 23:26:37,092][1944769] Num frames 9700... | |
| [2024-05-27 23:26:37,160][1944769] Num frames 9800... | |
| [2024-05-27 23:26:37,295][1944769] Num frames 9900... | |
| [2024-05-27 23:26:37,435][1944769] Num frames 10000... | |
| [2024-05-27 23:26:37,516][1944769] Avg episode rewards: #0: 26.492, true rewards: #0: 11.159 | |
| [2024-05-27 23:26:37,516][1944769] Avg episode reward: 26.492, avg true_objective: 11.159 | |
| [2024-05-27 23:26:37,556][1944769] Num frames 10100... | |
| [2024-05-27 23:26:37,626][1944769] Num frames 10200... | |
| [2024-05-27 23:26:37,711][1944769] Num frames 10300... | |
| [2024-05-27 23:26:37,781][1944769] Num frames 10400... | |
| [2024-05-27 23:26:37,909][1944769] Num frames 10500... | |
| [2024-05-27 23:26:37,988][1944769] Num frames 10600... | |
| [2024-05-27 23:26:38,158][1944769] Num frames 10700... | |
| [2024-05-27 23:26:38,228][1944769] Num frames 10800... | |
| [2024-05-27 23:26:38,291][1944769] Num frames 10900... | |
| [2024-05-27 23:26:38,395][1944769] Num frames 11000... | |
| [2024-05-27 23:26:38,461][1944769] Num frames 11100... | |
| [2024-05-27 23:26:38,528][1944769] Num frames 11200... | |
| [2024-05-27 23:26:38,608][1944769] Num frames 11300... | |
| [2024-05-27 23:26:38,713][1944769] Num frames 11400... | |
| [2024-05-27 23:26:38,783][1944769] Num frames 11500... | |
| [2024-05-27 23:26:38,880][1944769] Num frames 11600... | |
| [2024-05-27 23:26:38,948][1944769] Num frames 11700... | |
| [2024-05-27 23:26:39,015][1944769] Num frames 11800... | |
| [2024-05-27 23:26:39,143][1944769] Num frames 11900... | |
| [2024-05-27 23:26:39,303][1944769] Num frames 12000... | |
| [2024-05-27 23:26:39,377][1944769] Num frames 12100... | |
| [2024-05-27 23:26:39,460][1944769] Avg episode rewards: #0: 28.943, true rewards: #0: 12.143 | |
| [2024-05-27 23:26:39,460][1944769] Avg episode reward: 28.943, avg true_objective: 12.143 | |
| [2024-05-27 23:26:54,252][1944769] Replay video saved to /media/fast/code/learning/train_dir/default_experiment/replay.mp4! | |
| [2024-05-27 23:26:54,444][1944769] Loading existing experiment configuration from /media/fast/code/learning/train_dir/default_experiment/config.json | |
| [2024-05-27 23:26:54,444][1944769] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-05-27 23:26:54,444][1944769] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-05-27 23:26:54,444][1944769] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'hf_repository'='DavidPL1/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-05-27 23:26:54,445][1944769] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-05-27 23:26:54,450][1944769] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-05-27 23:26:54,450][1944769] RunningMeanStd input shape: (1,) | |
| [2024-05-27 23:26:54,459][1944769] ConvEncoder: input_channels=3 | |
| [2024-05-27 23:26:54,489][1944769] Conv encoder output size: 512 | |
| [2024-05-27 23:26:54,489][1944769] Policy head output size: 512 | |
| [2024-05-27 23:26:54,503][1944769] Loading state from checkpoint /media/fast/code/learning/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... | |
| [2024-05-27 23:26:55,013][1944769] Num frames 100... | |
| [2024-05-27 23:26:55,114][1944769] Num frames 200... | |
| [2024-05-27 23:26:55,180][1944769] Num frames 300... | |
| [2024-05-27 23:26:55,321][1944769] Num frames 400... | |
| [2024-05-27 23:26:55,436][1944769] Num frames 500... | |
| [2024-05-27 23:26:55,524][1944769] Num frames 600... | |
| [2024-05-27 23:26:55,592][1944769] Num frames 700... | |
| [2024-05-27 23:26:55,655][1944769] Num frames 800... | |
| [2024-05-27 23:26:55,722][1944769] Num frames 900... | |
| [2024-05-27 23:26:55,787][1944769] Num frames 1000... | |
| [2024-05-27 23:26:55,855][1944769] Num frames 1100... | |
| [2024-05-27 23:26:55,999][1944769] Num frames 1200... | |
| [2024-05-27 23:26:56,088][1944769] Num frames 1300... | |
| [2024-05-27 23:26:56,195][1944769] Num frames 1400... | |
| [2024-05-27 23:26:56,273][1944769] Num frames 1500... | |
| [2024-05-27 23:26:56,339][1944769] Num frames 1600... | |
| [2024-05-27 23:26:56,449][1944769] Avg episode rewards: #0: 37.320, true rewards: #0: 16.320 | |
| [2024-05-27 23:26:56,449][1944769] Avg episode reward: 37.320, avg true_objective: 16.320 | |
| [2024-05-27 23:26:56,528][1944769] Num frames 1700... | |
| [2024-05-27 23:26:56,595][1944769] Num frames 1800... | |
| [2024-05-27 23:26:56,654][1944769] Num frames 1900... | |
| [2024-05-27 23:26:56,719][1944769] Num frames 2000... | |
| [2024-05-27 23:26:56,784][1944769] Num frames 2100... | |
| [2024-05-27 23:26:56,861][1944769] Num frames 2200... | |
| [2024-05-27 23:26:56,939][1944769] Avg episode rewards: #0: 24.700, true rewards: #0: 11.200 | |
| [2024-05-27 23:26:56,939][1944769] Avg episode reward: 24.700, avg true_objective: 11.200 | |
| [2024-05-27 23:26:56,981][1944769] Num frames 2300... | |
| [2024-05-27 23:26:57,056][1944769] Num frames 2400... | |
| [2024-05-27 23:26:57,182][1944769] Num frames 2500... | |
| [2024-05-27 23:26:57,381][1944769] Num frames 2600... | |
| [2024-05-27 23:26:57,449][1944769] Num frames 2700... | |
| [2024-05-27 23:26:57,614][1944769] Num frames 2800... | |
| [2024-05-27 23:26:57,734][1944769] Num frames 2900... | |
| [2024-05-27 23:26:57,849][1944769] Avg episode rewards: #0: 21.920, true rewards: #0: 9.920 | |
| [2024-05-27 23:26:57,849][1944769] Avg episode reward: 21.920, avg true_objective: 9.920 | |
| [2024-05-27 23:26:57,865][1944769] Num frames 3000... | |
| [2024-05-27 23:26:57,971][1944769] Num frames 3100... | |
| [2024-05-27 23:26:58,045][1944769] Num frames 3200... | |
| [2024-05-27 23:26:58,122][1944769] Num frames 3300... | |
| [2024-05-27 23:26:58,234][1944769] Num frames 3400... | |
| [2024-05-27 23:26:58,368][1944769] Num frames 3500... | |
| [2024-05-27 23:26:58,458][1944769] Num frames 3600... | |
| [2024-05-27 23:26:58,529][1944769] Num frames 3700... | |
| [2024-05-27 23:26:58,579][1944769] Avg episode rewards: #0: 20.500, true rewards: #0: 9.250 | |
| [2024-05-27 23:26:58,580][1944769] Avg episode reward: 20.500, avg true_objective: 9.250 | |
| [2024-05-27 23:26:58,647][1944769] Num frames 3800... | |
| [2024-05-27 23:26:58,709][1944769] Num frames 3900... | |
| [2024-05-27 23:26:58,773][1944769] Num frames 4000... | |
| [2024-05-27 23:26:58,913][1944769] Num frames 4100... | |
| [2024-05-27 23:26:58,993][1944769] Num frames 4200... | |
| [2024-05-27 23:26:59,062][1944769] Num frames 4300... | |
| [2024-05-27 23:26:59,129][1944769] Num frames 4400... | |
| [2024-05-27 23:26:59,197][1944769] Num frames 4500... | |
| [2024-05-27 23:26:59,309][1944769] Num frames 4600... | |
| [2024-05-27 23:26:59,379][1944769] Avg episode rewards: #0: 20.856, true rewards: #0: 9.256 | |
| [2024-05-27 23:26:59,379][1944769] Avg episode reward: 20.856, avg true_objective: 9.256 | |
| [2024-05-27 23:26:59,455][1944769] Num frames 4700... | |
| [2024-05-27 23:26:59,522][1944769] Num frames 4800... | |
| [2024-05-27 23:26:59,591][1944769] Num frames 4900... | |
| [2024-05-27 23:26:59,656][1944769] Num frames 5000... | |
| [2024-05-27 23:26:59,721][1944769] Num frames 5100... | |
| [2024-05-27 23:26:59,825][1944769] Num frames 5200... | |
| [2024-05-27 23:26:59,916][1944769] Num frames 5300... | |
| [2024-05-27 23:27:00,074][1944769] Num frames 5400... | |
| [2024-05-27 23:27:00,136][1944769] Num frames 5500... | |
| [2024-05-27 23:27:00,209][1944769] Num frames 5600... | |
| [2024-05-27 23:27:00,278][1944769] Num frames 5700... | |
| [2024-05-27 23:27:00,346][1944769] Num frames 5800... | |
| [2024-05-27 23:27:00,487][1944769] Num frames 5900... | |
| [2024-05-27 23:27:00,552][1944769] Num frames 6000... | |
| [2024-05-27 23:27:00,627][1944769] Num frames 6100... | |
| [2024-05-27 23:27:00,694][1944769] Num frames 6200... | |
| [2024-05-27 23:27:00,806][1944769] Avg episode rewards: #0: 23.820, true rewards: #0: 10.487 | |
| [2024-05-27 23:27:00,806][1944769] Avg episode reward: 23.820, avg true_objective: 10.487 | |
| [2024-05-27 23:27:00,812][1944769] Num frames 6300... | |
| [2024-05-27 23:27:00,902][1944769] Num frames 6400... | |
| [2024-05-27 23:27:01,011][1944769] Num frames 6500... | |
| [2024-05-27 23:27:01,075][1944769] Num frames 6600... | |
| [2024-05-27 23:27:01,198][1944769] Num frames 6700... | |
| [2024-05-27 23:27:01,263][1944769] Num frames 6800... | |
| [2024-05-27 23:27:01,353][1944769] Num frames 6900... | |
| [2024-05-27 23:27:01,615][1944769] Num frames 7000... | |
| [2024-05-27 23:27:01,684][1944769] Num frames 7100... | |
| [2024-05-27 23:27:01,753][1944769] Num frames 7200... | |
| [2024-05-27 23:27:01,840][1944769] Avg episode rewards: #0: 23.074, true rewards: #0: 10.360 | |
| [2024-05-27 23:27:01,840][1944769] Avg episode reward: 23.074, avg true_objective: 10.360 | |
| [2024-05-27 23:27:01,873][1944769] Num frames 7300... | |
| [2024-05-27 23:27:01,932][1944769] Num frames 7400... | |
| [2024-05-27 23:27:01,993][1944769] Num frames 7500... | |
| [2024-05-27 23:27:02,054][1944769] Num frames 7600... | |
| [2024-05-27 23:27:02,114][1944769] Num frames 7700... | |
| [2024-05-27 23:27:02,175][1944769] Num frames 7800... | |
| [2024-05-27 23:27:02,239][1944769] Num frames 7900... | |
| [2024-05-27 23:27:02,353][1944769] Avg episode rewards: #0: 22.110, true rewards: #0: 9.985 | |
| [2024-05-27 23:27:02,353][1944769] Avg episode reward: 22.110, avg true_objective: 9.985 | |
| [2024-05-27 23:27:02,426][1944769] Num frames 8000... | |
| [2024-05-27 23:27:02,489][1944769] Num frames 8100... | |
| [2024-05-27 23:27:02,544][1944769] Num frames 8200... | |
| [2024-05-27 23:27:02,602][1944769] Num frames 8300... | |
| [2024-05-27 23:27:02,663][1944769] Num frames 8400... | |
| [2024-05-27 23:27:02,721][1944769] Num frames 8500... | |
| [2024-05-27 23:27:02,784][1944769] Num frames 8600... | |
| [2024-05-27 23:27:02,843][1944769] Num frames 8700... | |
| [2024-05-27 23:27:02,904][1944769] Num frames 8800... | |
| [2024-05-27 23:27:02,965][1944769] Num frames 8900... | |
| [2024-05-27 23:27:03,028][1944769] Num frames 9000... | |
| [2024-05-27 23:27:03,102][1944769] Avg episode rewards: #0: 22.486, true rewards: #0: 10.041 | |
| [2024-05-27 23:27:03,102][1944769] Avg episode reward: 22.486, avg true_objective: 10.041 | |
| [2024-05-27 23:27:03,141][1944769] Num frames 9100... | |
| [2024-05-27 23:27:03,204][1944769] Num frames 9200... | |
| [2024-05-27 23:27:03,267][1944769] Num frames 9300... | |
| [2024-05-27 23:27:03,328][1944769] Num frames 9400... | |
| [2024-05-27 23:27:03,387][1944769] Num frames 9500... | |
| [2024-05-27 23:27:03,447][1944769] Num frames 9600... | |
| [2024-05-27 23:27:03,506][1944769] Num frames 9700... | |
| [2024-05-27 23:27:03,565][1944769] Num frames 9800... | |
| [2024-05-27 23:27:03,626][1944769] Num frames 9900... | |
| [2024-05-27 23:27:03,682][1944769] Num frames 10000... | |
| [2024-05-27 23:27:03,739][1944769] Num frames 10100... | |
| [2024-05-27 23:27:03,799][1944769] Num frames 10200... | |
| [2024-05-27 23:27:03,864][1944769] Num frames 10300... | |
| [2024-05-27 23:27:03,928][1944769] Num frames 10400... | |
| [2024-05-27 23:27:03,990][1944769] Num frames 10500... | |
| [2024-05-27 23:27:04,054][1944769] Num frames 10600... | |
| [2024-05-27 23:27:04,116][1944769] Num frames 10700... | |
| [2024-05-27 23:27:04,179][1944769] Num frames 10800... | |
| [2024-05-27 23:27:04,238][1944769] Num frames 10900... | |
| [2024-05-27 23:27:04,305][1944769] Num frames 11000... | |
| [2024-05-27 23:27:04,369][1944769] Num frames 11100... | |
| [2024-05-27 23:27:04,446][1944769] Avg episode rewards: #0: 25.637, true rewards: #0: 11.137 | |
| [2024-05-27 23:27:04,446][1944769] Avg episode reward: 25.637, avg true_objective: 11.137 | |
| [2024-05-27 23:27:18,008][1944769] Replay video saved to /media/fast/code/learning/train_dir/default_experiment/replay.mp4! | |