| [2023-05-24 20:15:24,322][2722668] Saving configuration to /home/mark/rl_course/unit8/train_dir/default_experiment/config.json... | |
| [2023-05-24 20:15:24,325][2722668] Rollout worker 0 uses device cpu | |
| [2023-05-24 20:15:24,326][2722668] Rollout worker 1 uses device cpu | |
| [2023-05-24 20:15:24,327][2722668] Rollout worker 2 uses device cpu | |
| [2023-05-24 20:15:24,328][2722668] Rollout worker 3 uses device cpu | |
| [2023-05-24 20:15:24,329][2722668] Rollout worker 4 uses device cpu | |
| [2023-05-24 20:15:24,331][2722668] Rollout worker 5 uses device cpu | |
| [2023-05-24 20:15:24,332][2722668] Rollout worker 6 uses device cpu | |
| [2023-05-24 20:15:24,333][2722668] Rollout worker 7 uses device cpu | |
| [2023-05-24 20:15:24,397][2722668] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:15:24,398][2722668] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-05-24 20:15:24,427][2722668] Starting all processes... | |
| [2023-05-24 20:15:24,428][2722668] Starting process learner_proc0 | |
| [2023-05-24 20:15:24,476][2722668] Starting all processes... | |
| [2023-05-24 20:15:24,485][2722668] Starting process inference_proc0-0 | |
| [2023-05-24 20:15:24,485][2722668] Starting process rollout_proc0 | |
| [2023-05-24 20:15:24,485][2722668] Starting process rollout_proc1 | |
| [2023-05-24 20:15:24,486][2722668] Starting process rollout_proc2 | |
| [2023-05-24 20:15:24,486][2722668] Starting process rollout_proc3 | |
| [2023-05-24 20:15:24,487][2722668] Starting process rollout_proc4 | |
| [2023-05-24 20:15:24,487][2722668] Starting process rollout_proc5 | |
| [2023-05-24 20:15:24,488][2722668] Starting process rollout_proc6 | |
| [2023-05-24 20:15:24,488][2722668] Starting process rollout_proc7 | |
| [2023-05-24 20:15:26,022][2737021] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:15:26,022][2737021] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-05-24 20:15:26,039][2737021] Num visible devices: 1 | |
| [2023-05-24 20:15:26,061][2737021] Starting seed is not provided | |
| [2023-05-24 20:15:26,062][2737021] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:15:26,062][2737021] Initializing actor-critic model on device cuda:0 | |
| [2023-05-24 20:15:26,062][2737021] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:15:26,063][2737021] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:15:26,077][2737021] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:15:26,147][2737054] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,181][2737046] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:15:26,181][2737046] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-05-24 20:15:26,189][2737053] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,191][2737049] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,193][2737047] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,200][2737046] Num visible devices: 1 | |
| [2023-05-24 20:15:26,200][2737052] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,201][2737048] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,203][2737051] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,203][2737021] Conv encoder output size: 512 | |
| [2023-05-24 20:15:26,204][2737021] Policy head output size: 512 | |
| [2023-05-24 20:15:26,217][2737021] Created Actor Critic model with architecture: | |
| [2023-05-24 20:15:26,217][2737050] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:15:26,217][2737021] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-05-24 20:15:28,728][2737021] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-05-24 20:15:28,729][2737021] No checkpoints found | |
| [2023-05-24 20:15:28,729][2737021] Did not load from checkpoint, starting from scratch! | |
| [2023-05-24 20:15:28,729][2737021] Initialized policy 0 weights for model version 0 | |
| [2023-05-24 20:15:28,731][2737021] LearnerWorker_p0 finished initialization! | |
| [2023-05-24 20:15:28,731][2737021] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:15:28,856][2737046] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:15:28,857][2737046] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:15:28,872][2737046] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:15:29,004][2737046] Conv encoder output size: 512 | |
| [2023-05-24 20:15:29,004][2737046] Policy head output size: 512 | |
| [2023-05-24 20:15:30,976][2722668] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-05-24 20:15:31,437][2722668] Inference worker 0-0 is ready! | |
| [2023-05-24 20:15:31,438][2722668] All inference workers are ready! Signal rollout workers to start! | |
| [2023-05-24 20:15:31,473][2737047] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,496][2737051] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,499][2737050] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,502][2737053] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,502][2737054] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,502][2737049] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,508][2737052] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:31,510][2737048] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:15:32,115][2737047] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,121][2737049] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,121][2737052] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,122][2737048] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,123][2737054] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,123][2737050] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,430][2737052] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:32,433][2737050] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:32,438][2737049] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:32,439][2737054] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:32,482][2737047] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:32,772][2737051] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,789][2737052] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:32,792][2737053] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:15:32,801][2737050] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:32,808][2737048] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:33,085][2737051] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:33,105][2737053] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:15:33,148][2737047] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,151][2737049] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,163][2737054] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,173][2737048] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,445][2737051] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,456][2737053] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:15:33,471][2737050] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:33,511][2737047] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:33,788][2737054] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:33,826][2737053] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:33,836][2737051] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:34,104][2737048] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:34,418][2737052] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:34,783][2737049] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:15:34,984][2737021] Signal inference workers to stop experience collection... | |
| [2023-05-24 20:15:34,992][2737046] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-05-24 20:15:35,976][2722668] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 62.8. Samples: 314. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-05-24 20:15:35,978][2722668] Avg episode reward: [(0, '2.719')] | |
| [2023-05-24 20:15:36,386][2737021] Signal inference workers to resume experience collection... | |
| [2023-05-24 20:15:36,387][2737046] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-05-24 20:15:38,875][2737046] Updated weights for policy 0, policy_version 10 (0.0469) | |
| [2023-05-24 20:15:40,695][2737046] Updated weights for policy 0, policy_version 20 (0.0009) | |
| [2023-05-24 20:15:40,976][2722668] Fps is (10 sec: 8601.6, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 86016. Throughput: 0: 1969.0. Samples: 19690. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:15:40,977][2722668] Avg episode reward: [(0, '4.404')] | |
| [2023-05-24 20:15:42,521][2737046] Updated weights for policy 0, policy_version 30 (0.0008) | |
| [2023-05-24 20:15:44,344][2737046] Updated weights for policy 0, policy_version 40 (0.0009) | |
| [2023-05-24 20:15:44,390][2722668] Heartbeat connected on Batcher_0 | |
| [2023-05-24 20:15:44,393][2722668] Heartbeat connected on LearnerWorker_p0 | |
| [2023-05-24 20:15:44,399][2722668] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-05-24 20:15:44,404][2722668] Heartbeat connected on RolloutWorker_w0 | |
| [2023-05-24 20:15:44,405][2722668] Heartbeat connected on RolloutWorker_w1 | |
| [2023-05-24 20:15:44,409][2722668] Heartbeat connected on RolloutWorker_w2 | |
| [2023-05-24 20:15:44,415][2722668] Heartbeat connected on RolloutWorker_w3 | |
| [2023-05-24 20:15:44,422][2722668] Heartbeat connected on RolloutWorker_w4 | |
| [2023-05-24 20:15:44,423][2722668] Heartbeat connected on RolloutWorker_w5 | |
| [2023-05-24 20:15:44,425][2722668] Heartbeat connected on RolloutWorker_w6 | |
| [2023-05-24 20:15:44,427][2722668] Heartbeat connected on RolloutWorker_w7 | |
| [2023-05-24 20:15:45,976][2722668] Fps is (10 sec: 19661.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 196608. Throughput: 0: 2436.5. Samples: 36548. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:15:45,977][2722668] Avg episode reward: [(0, '4.379')] | |
| [2023-05-24 20:15:45,999][2737021] Saving new best policy, reward=4.379! | |
| [2023-05-24 20:15:46,177][2737046] Updated weights for policy 0, policy_version 50 (0.0008) | |
| [2023-05-24 20:15:48,003][2737046] Updated weights for policy 0, policy_version 60 (0.0009) | |
| [2023-05-24 20:15:49,817][2737046] Updated weights for policy 0, policy_version 70 (0.0008) | |
| [2023-05-24 20:15:50,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 15564.8, 300 sec: 15564.8). Total num frames: 311296. Throughput: 0: 3517.5. Samples: 70350. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:15:50,977][2722668] Avg episode reward: [(0, '4.638')] | |
| [2023-05-24 20:15:50,982][2737021] Saving new best policy, reward=4.638! | |
| [2023-05-24 20:15:51,666][2737046] Updated weights for policy 0, policy_version 80 (0.0008) | |
| [2023-05-24 20:15:53,506][2737046] Updated weights for policy 0, policy_version 90 (0.0009) | |
| [2023-05-24 20:15:55,344][2737046] Updated weights for policy 0, policy_version 100 (0.0009) | |
| [2023-05-24 20:15:55,976][2722668] Fps is (10 sec: 22528.1, 60 sec: 16875.5, 300 sec: 16875.5). Total num frames: 421888. Throughput: 0: 4150.5. Samples: 103762. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:15:55,977][2722668] Avg episode reward: [(0, '4.824')] | |
| [2023-05-24 20:15:55,978][2737021] Saving new best policy, reward=4.824! | |
| [2023-05-24 20:15:57,161][2737046] Updated weights for policy 0, policy_version 110 (0.0009) | |
| [2023-05-24 20:15:58,982][2737046] Updated weights for policy 0, policy_version 120 (0.0008) | |
| [2023-05-24 20:16:00,811][2737046] Updated weights for policy 0, policy_version 130 (0.0008) | |
| [2023-05-24 20:16:00,976][2722668] Fps is (10 sec: 22118.3, 60 sec: 17749.3, 300 sec: 17749.3). Total num frames: 532480. Throughput: 0: 4018.2. Samples: 120546. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:16:00,977][2722668] Avg episode reward: [(0, '4.588')] | |
| [2023-05-24 20:16:02,629][2737046] Updated weights for policy 0, policy_version 140 (0.0010) | |
| [2023-05-24 20:16:04,461][2737046] Updated weights for policy 0, policy_version 150 (0.0009) | |
| [2023-05-24 20:16:05,976][2722668] Fps is (10 sec: 22527.9, 60 sec: 18490.5, 300 sec: 18490.5). Total num frames: 647168. Throughput: 0: 4408.3. Samples: 154290. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:16:05,977][2722668] Avg episode reward: [(0, '4.587')] | |
| [2023-05-24 20:16:06,238][2737046] Updated weights for policy 0, policy_version 160 (0.0008) | |
| [2023-05-24 20:16:08,048][2737046] Updated weights for policy 0, policy_version 170 (0.0008) | |
| [2023-05-24 20:16:09,867][2737046] Updated weights for policy 0, policy_version 180 (0.0009) | |
| [2023-05-24 20:16:10,976][2722668] Fps is (10 sec: 22937.7, 60 sec: 19046.4, 300 sec: 19046.4). Total num frames: 761856. Throughput: 0: 4703.9. Samples: 188156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-05-24 20:16:10,977][2722668] Avg episode reward: [(0, '4.869')] | |
| [2023-05-24 20:16:10,985][2737021] Saving new best policy, reward=4.869! | |
| [2023-05-24 20:16:11,691][2737046] Updated weights for policy 0, policy_version 190 (0.0009) | |
| [2023-05-24 20:16:13,543][2737046] Updated weights for policy 0, policy_version 200 (0.0008) | |
| [2023-05-24 20:16:15,376][2737046] Updated weights for policy 0, policy_version 210 (0.0009) | |
| [2023-05-24 20:16:15,976][2722668] Fps is (10 sec: 22528.1, 60 sec: 19387.7, 300 sec: 19387.7). Total num frames: 872448. Throughput: 0: 4553.1. Samples: 204890. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:16:15,977][2722668] Avg episode reward: [(0, '5.324')] | |
| [2023-05-24 20:16:15,978][2737021] Saving new best policy, reward=5.324! | |
| [2023-05-24 20:16:17,217][2737046] Updated weights for policy 0, policy_version 220 (0.0008) | |
| [2023-05-24 20:16:19,046][2737046] Updated weights for policy 0, policy_version 230 (0.0010) | |
| [2023-05-24 20:16:20,891][2737046] Updated weights for policy 0, policy_version 240 (0.0008) | |
| [2023-05-24 20:16:20,976][2722668] Fps is (10 sec: 22118.4, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 983040. Throughput: 0: 5292.6. Samples: 238478. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:16:20,977][2722668] Avg episode reward: [(0, '5.011')] | |
| [2023-05-24 20:16:22,750][2737046] Updated weights for policy 0, policy_version 250 (0.0009) | |
| [2023-05-24 20:16:24,611][2737046] Updated weights for policy 0, policy_version 260 (0.0009) | |
| [2023-05-24 20:16:25,976][2722668] Fps is (10 sec: 22118.2, 60 sec: 19884.2, 300 sec: 19884.2). Total num frames: 1093632. Throughput: 0: 5603.0. Samples: 271824. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:16:25,977][2722668] Avg episode reward: [(0, '5.167')] | |
| [2023-05-24 20:16:26,448][2737046] Updated weights for policy 0, policy_version 270 (0.0009) | |
| [2023-05-24 20:16:28,301][2737046] Updated weights for policy 0, policy_version 280 (0.0009) | |
| [2023-05-24 20:16:30,111][2737046] Updated weights for policy 0, policy_version 290 (0.0009) | |
| [2023-05-24 20:16:30,976][2722668] Fps is (10 sec: 22118.4, 60 sec: 20070.4, 300 sec: 20070.4). Total num frames: 1204224. Throughput: 0: 5598.6. Samples: 288486. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:16:30,977][2722668] Avg episode reward: [(0, '5.892')] | |
| [2023-05-24 20:16:31,012][2737021] Saving new best policy, reward=5.892! | |
| [2023-05-24 20:16:31,920][2737046] Updated weights for policy 0, policy_version 300 (0.0008) | |
| [2023-05-24 20:16:33,732][2737046] Updated weights for policy 0, policy_version 310 (0.0008) | |
| [2023-05-24 20:16:35,541][2737046] Updated weights for policy 0, policy_version 320 (0.0009) | |
| [2023-05-24 20:16:35,976][2722668] Fps is (10 sec: 22528.3, 60 sec: 21981.9, 300 sec: 20291.0). Total num frames: 1318912. Throughput: 0: 5597.6. Samples: 322244. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:16:35,977][2722668] Avg episode reward: [(0, '6.006')] | |
| [2023-05-24 20:16:35,978][2737021] Saving new best policy, reward=6.006! | |
| [2023-05-24 20:16:37,335][2737046] Updated weights for policy 0, policy_version 330 (0.0008) | |
| [2023-05-24 20:16:39,153][2737046] Updated weights for policy 0, policy_version 340 (0.0008) | |
| [2023-05-24 20:16:40,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 20421.5). Total num frames: 1429504. Throughput: 0: 5605.9. Samples: 356028. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:16:40,977][2722668] Avg episode reward: [(0, '8.106')] | |
| [2023-05-24 20:16:40,983][2737021] Saving new best policy, reward=8.106! | |
| [2023-05-24 20:16:40,984][2737046] Updated weights for policy 0, policy_version 350 (0.0009) | |
| [2023-05-24 20:16:42,811][2737046] Updated weights for policy 0, policy_version 360 (0.0008) | |
| [2023-05-24 20:16:44,631][2737046] Updated weights for policy 0, policy_version 370 (0.0008) | |
| [2023-05-24 20:16:45,976][2722668] Fps is (10 sec: 22527.8, 60 sec: 22459.7, 300 sec: 20589.2). Total num frames: 1544192. Throughput: 0: 5609.3. Samples: 372964. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-05-24 20:16:45,977][2722668] Avg episode reward: [(0, '7.390')] | |
| [2023-05-24 20:16:46,458][2737046] Updated weights for policy 0, policy_version 380 (0.0008) | |
| [2023-05-24 20:16:48,289][2737046] Updated weights for policy 0, policy_version 390 (0.0008) | |
| [2023-05-24 20:16:50,129][2737046] Updated weights for policy 0, policy_version 400 (0.0009) | |
| [2023-05-24 20:16:50,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 20684.8). Total num frames: 1654784. Throughput: 0: 5607.5. Samples: 406626. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:16:50,977][2722668] Avg episode reward: [(0, '7.569')] | |
| [2023-05-24 20:16:51,933][2737046] Updated weights for policy 0, policy_version 410 (0.0009) | |
| [2023-05-24 20:16:53,746][2737046] Updated weights for policy 0, policy_version 420 (0.0009) | |
| [2023-05-24 20:16:55,581][2737046] Updated weights for policy 0, policy_version 430 (0.0008) | |
| [2023-05-24 20:16:55,976][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.7, 300 sec: 20817.3). Total num frames: 1769472. Throughput: 0: 5605.7. Samples: 440412. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:16:55,977][2722668] Avg episode reward: [(0, '8.576')] | |
| [2023-05-24 20:16:55,978][2737021] Saving new best policy, reward=8.576! | |
| [2023-05-24 20:16:57,388][2737046] Updated weights for policy 0, policy_version 440 (0.0009) | |
| [2023-05-24 20:16:59,202][2737046] Updated weights for policy 0, policy_version 450 (0.0008) | |
| [2023-05-24 20:17:00,974][2737046] Updated weights for policy 0, policy_version 460 (0.0009) | |
| [2023-05-24 20:17:00,976][2722668] Fps is (10 sec: 22937.6, 60 sec: 22528.0, 300 sec: 20935.1). Total num frames: 1884160. Throughput: 0: 5609.6. Samples: 457320. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:17:00,977][2722668] Avg episode reward: [(0, '11.579')] | |
| [2023-05-24 20:17:00,982][2737021] Saving new best policy, reward=11.579! | |
| [2023-05-24 20:17:02,777][2737046] Updated weights for policy 0, policy_version 470 (0.0010) | |
| [2023-05-24 20:17:04,583][2737046] Updated weights for policy 0, policy_version 480 (0.0008) | |
| [2023-05-24 20:17:05,976][2722668] Fps is (10 sec: 22527.9, 60 sec: 22459.7, 300 sec: 20997.4). Total num frames: 1994752. Throughput: 0: 5617.1. Samples: 491246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-05-24 20:17:05,977][2722668] Avg episode reward: [(0, '11.487')] | |
| [2023-05-24 20:17:06,386][2737046] Updated weights for policy 0, policy_version 490 (0.0009) | |
| [2023-05-24 20:17:08,197][2737046] Updated weights for policy 0, policy_version 500 (0.0008) | |
| [2023-05-24 20:17:10,043][2737046] Updated weights for policy 0, policy_version 510 (0.0010) | |
| [2023-05-24 20:17:10,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22459.7, 300 sec: 21094.4). Total num frames: 2109440. Throughput: 0: 5627.3. Samples: 525054. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:17:10,977][2722668] Avg episode reward: [(0, '13.819')] | |
| [2023-05-24 20:17:10,982][2737021] Saving new best policy, reward=13.819! | |
| [2023-05-24 20:17:11,882][2737046] Updated weights for policy 0, policy_version 520 (0.0009) | |
| [2023-05-24 20:17:13,742][2737046] Updated weights for policy 0, policy_version 530 (0.0009) | |
| [2023-05-24 20:17:15,605][2737046] Updated weights for policy 0, policy_version 540 (0.0009) | |
| [2023-05-24 20:17:15,976][2722668] Fps is (10 sec: 22527.9, 60 sec: 22459.7, 300 sec: 21143.2). Total num frames: 2220032. Throughput: 0: 5625.3. Samples: 541624. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:17:15,977][2722668] Avg episode reward: [(0, '14.199')] | |
| [2023-05-24 20:17:15,979][2737021] Saving new best policy, reward=14.199! | |
| [2023-05-24 20:17:17,499][2737046] Updated weights for policy 0, policy_version 550 (0.0009) | |
| [2023-05-24 20:17:19,360][2737046] Updated weights for policy 0, policy_version 560 (0.0009) | |
| [2023-05-24 20:17:20,976][2722668] Fps is (10 sec: 21708.6, 60 sec: 22391.4, 300 sec: 21150.2). Total num frames: 2326528. Throughput: 0: 5604.8. Samples: 574462. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-05-24 20:17:20,977][2722668] Avg episode reward: [(0, '18.293')] | |
| [2023-05-24 20:17:20,997][2737021] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000569_2330624.pth... | |
| [2023-05-24 20:17:21,040][2737021] Saving new best policy, reward=18.293! | |
| [2023-05-24 20:17:21,192][2737046] Updated weights for policy 0, policy_version 570 (0.0008) | |
| [2023-05-24 20:17:23,003][2737046] Updated weights for policy 0, policy_version 580 (0.0009) | |
| [2023-05-24 20:17:24,820][2737046] Updated weights for policy 0, policy_version 590 (0.0009) | |
| [2023-05-24 20:17:25,976][2722668] Fps is (10 sec: 22118.5, 60 sec: 22459.8, 300 sec: 21228.0). Total num frames: 2441216. Throughput: 0: 5604.1. Samples: 608214. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-05-24 20:17:25,977][2722668] Avg episode reward: [(0, '17.900')] | |
| [2023-05-24 20:17:26,623][2737046] Updated weights for policy 0, policy_version 600 (0.0008) | |
| [2023-05-24 20:17:28,477][2737046] Updated weights for policy 0, policy_version 610 (0.0009) | |
| [2023-05-24 20:17:30,310][2737046] Updated weights for policy 0, policy_version 620 (0.0009) | |
| [2023-05-24 20:17:30,976][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.7, 300 sec: 21265.1). Total num frames: 2551808. Throughput: 0: 5600.6. Samples: 624990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:17:30,977][2722668] Avg episode reward: [(0, '19.475')] | |
| [2023-05-24 20:17:30,983][2737021] Saving new best policy, reward=19.475! | |
| [2023-05-24 20:17:32,166][2737046] Updated weights for policy 0, policy_version 630 (0.0009) | |
| [2023-05-24 20:17:34,004][2737046] Updated weights for policy 0, policy_version 640 (0.0008) | |
| [2023-05-24 20:17:35,838][2737046] Updated weights for policy 0, policy_version 650 (0.0009) | |
| [2023-05-24 20:17:35,976][2722668] Fps is (10 sec: 22118.5, 60 sec: 22391.5, 300 sec: 21299.2). Total num frames: 2662400. Throughput: 0: 5596.0. Samples: 658444. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:17:35,977][2722668] Avg episode reward: [(0, '20.309')] | |
| [2023-05-24 20:17:35,978][2737021] Saving new best policy, reward=20.309! | |
| [2023-05-24 20:17:37,680][2737046] Updated weights for policy 0, policy_version 660 (0.0009) | |
| [2023-05-24 20:17:39,524][2737046] Updated weights for policy 0, policy_version 670 (0.0009) | |
| [2023-05-24 20:17:40,976][2722668] Fps is (10 sec: 22118.6, 60 sec: 22391.5, 300 sec: 21330.7). Total num frames: 2772992. Throughput: 0: 5591.3. Samples: 692022. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-05-24 20:17:40,977][2722668] Avg episode reward: [(0, '20.988')] | |
| [2023-05-24 20:17:40,984][2737021] Saving new best policy, reward=20.988! | |
| [2023-05-24 20:17:41,339][2737046] Updated weights for policy 0, policy_version 680 (0.0008) | |
| [2023-05-24 20:17:43,169][2737046] Updated weights for policy 0, policy_version 690 (0.0008) | |
| [2023-05-24 20:17:44,986][2737046] Updated weights for policy 0, policy_version 700 (0.0009) | |
| [2023-05-24 20:17:45,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21390.2). Total num frames: 2887680. Throughput: 0: 5590.0. Samples: 708872. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-05-24 20:17:45,977][2722668] Avg episode reward: [(0, '23.068')] | |
| [2023-05-24 20:17:45,978][2737021] Saving new best policy, reward=23.068! | |
| [2023-05-24 20:17:46,819][2737046] Updated weights for policy 0, policy_version 710 (0.0008) | |
| [2023-05-24 20:17:48,686][2737046] Updated weights for policy 0, policy_version 720 (0.0008) | |
| [2023-05-24 20:17:50,505][2737046] Updated weights for policy 0, policy_version 730 (0.0008) | |
| [2023-05-24 20:17:50,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21416.2). Total num frames: 2998272. Throughput: 0: 5580.3. Samples: 742360. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-05-24 20:17:50,977][2722668] Avg episode reward: [(0, '21.517')] | |
| [2023-05-24 20:17:52,347][2737046] Updated weights for policy 0, policy_version 740 (0.0009) | |
| [2023-05-24 20:17:54,179][2737046] Updated weights for policy 0, policy_version 750 (0.0009) | |
| [2023-05-24 20:17:55,976][2722668] Fps is (10 sec: 22118.3, 60 sec: 22323.2, 300 sec: 21440.4). Total num frames: 3108864. Throughput: 0: 5577.1. Samples: 776024. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:17:55,977][2722668] Avg episode reward: [(0, '23.550')] | |
| [2023-05-24 20:17:55,992][2737021] Saving new best policy, reward=23.550! | |
| [2023-05-24 20:17:55,992][2737046] Updated weights for policy 0, policy_version 760 (0.0009) | |
| [2023-05-24 20:17:57,814][2737046] Updated weights for policy 0, policy_version 770 (0.0009) | |
| [2023-05-24 20:17:59,632][2737046] Updated weights for policy 0, policy_version 780 (0.0009) | |
| [2023-05-24 20:18:00,976][2722668] Fps is (10 sec: 22527.9, 60 sec: 22323.2, 300 sec: 21490.4). Total num frames: 3223552. Throughput: 0: 5583.7. Samples: 792890. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:18:00,977][2722668] Avg episode reward: [(0, '26.290')] | |
| [2023-05-24 20:18:00,982][2737021] Saving new best policy, reward=26.290! | |
| [2023-05-24 20:18:01,438][2737046] Updated weights for policy 0, policy_version 790 (0.0009) | |
| [2023-05-24 20:18:03,272][2737046] Updated weights for policy 0, policy_version 800 (0.0008) | |
| [2023-05-24 20:18:05,079][2737046] Updated weights for policy 0, policy_version 810 (0.0009) | |
| [2023-05-24 20:18:05,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21510.6). Total num frames: 3334144. Throughput: 0: 5604.7. Samples: 826672. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:18:05,977][2722668] Avg episode reward: [(0, '22.770')] | |
| [2023-05-24 20:18:06,915][2737046] Updated weights for policy 0, policy_version 820 (0.0009) | |
| [2023-05-24 20:18:08,737][2737046] Updated weights for policy 0, policy_version 830 (0.0009) | |
| [2023-05-24 20:18:10,541][2737046] Updated weights for policy 0, policy_version 840 (0.0009) | |
| [2023-05-24 20:18:10,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21555.2). Total num frames: 3448832. Throughput: 0: 5604.0. Samples: 860396. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:18:10,977][2722668] Avg episode reward: [(0, '22.683')] | |
| [2023-05-24 20:18:12,338][2737046] Updated weights for policy 0, policy_version 850 (0.0008) | |
| [2023-05-24 20:18:14,145][2737046] Updated weights for policy 0, policy_version 860 (0.0008) | |
| [2023-05-24 20:18:15,953][2737046] Updated weights for policy 0, policy_version 870 (0.0009) | |
| [2023-05-24 20:18:15,976][2722668] Fps is (10 sec: 22937.7, 60 sec: 22391.5, 300 sec: 21597.1). Total num frames: 3563520. Throughput: 0: 5608.9. Samples: 877388. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:18:15,977][2722668] Avg episode reward: [(0, '22.152')] | |
| [2023-05-24 20:18:17,733][2737046] Updated weights for policy 0, policy_version 880 (0.0008) | |
| [2023-05-24 20:18:19,538][2737046] Updated weights for policy 0, policy_version 890 (0.0008) | |
| [2023-05-24 20:18:20,976][2722668] Fps is (10 sec: 22528.0, 60 sec: 22459.8, 300 sec: 21612.4). Total num frames: 3674112. Throughput: 0: 5625.5. Samples: 911592. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:18:20,977][2722668] Avg episode reward: [(0, '22.559')] | |
| [2023-05-24 20:18:21,328][2737046] Updated weights for policy 0, policy_version 900 (0.0008) | |
| [2023-05-24 20:18:23,130][2737046] Updated weights for policy 0, policy_version 910 (0.0009) | |
| [2023-05-24 20:18:24,913][2737046] Updated weights for policy 0, policy_version 920 (0.0009) | |
| [2023-05-24 20:18:25,976][2722668] Fps is (10 sec: 22527.9, 60 sec: 22459.7, 300 sec: 21650.3). Total num frames: 3788800. Throughput: 0: 5641.0. Samples: 945866. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:18:25,977][2722668] Avg episode reward: [(0, '24.137')] | |
| [2023-05-24 20:18:26,715][2737046] Updated weights for policy 0, policy_version 930 (0.0008) | |
| [2023-05-24 20:18:28,514][2737046] Updated weights for policy 0, policy_version 940 (0.0009) | |
| [2023-05-24 20:18:30,317][2737046] Updated weights for policy 0, policy_version 950 (0.0008) | |
| [2023-05-24 20:18:30,976][2722668] Fps is (10 sec: 22937.5, 60 sec: 22528.0, 300 sec: 21686.0). Total num frames: 3903488. Throughput: 0: 5646.4. Samples: 962962. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:18:30,977][2722668] Avg episode reward: [(0, '25.558')] | |
| [2023-05-24 20:18:32,129][2737046] Updated weights for policy 0, policy_version 960 (0.0009) | |
| [2023-05-24 20:18:33,905][2737046] Updated weights for policy 0, policy_version 970 (0.0008) | |
| [2023-05-24 20:18:35,357][2737021] Stopping Batcher_0... | |
| [2023-05-24 20:18:35,357][2737021] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-05-24 20:18:35,358][2737021] Loop batcher_evt_loop terminating... | |
| [2023-05-24 20:18:35,365][2722668] Component Batcher_0 stopped! | |
| [2023-05-24 20:18:35,370][2737054] Stopping RolloutWorker_w7... | |
| [2023-05-24 20:18:35,370][2737054] Loop rollout_proc7_evt_loop terminating... | |
| [2023-05-24 20:18:35,370][2737050] Stopping RolloutWorker_w3... | |
| [2023-05-24 20:18:35,371][2737049] Stopping RolloutWorker_w0... | |
| [2023-05-24 20:18:35,371][2737050] Loop rollout_proc3_evt_loop terminating... | |
| [2023-05-24 20:18:35,371][2737049] Loop rollout_proc0_evt_loop terminating... | |
| [2023-05-24 20:18:35,370][2722668] Component RolloutWorker_w7 stopped! | |
| [2023-05-24 20:18:35,371][2737052] Stopping RolloutWorker_w4... | |
| [2023-05-24 20:18:35,371][2737047] Stopping RolloutWorker_w2... | |
| [2023-05-24 20:18:35,371][2737052] Loop rollout_proc4_evt_loop terminating... | |
| [2023-05-24 20:18:35,372][2737047] Loop rollout_proc2_evt_loop terminating... | |
| [2023-05-24 20:18:35,372][2737046] Weights refcount: 2 0 | |
| [2023-05-24 20:18:35,372][2737051] Stopping RolloutWorker_w5... | |
| [2023-05-24 20:18:35,372][2737048] Stopping RolloutWorker_w1... | |
| [2023-05-24 20:18:35,372][2722668] Component RolloutWorker_w3 stopped! | |
| [2023-05-24 20:18:35,372][2737051] Loop rollout_proc5_evt_loop terminating... | |
| [2023-05-24 20:18:35,372][2737048] Loop rollout_proc1_evt_loop terminating... | |
| [2023-05-24 20:18:35,372][2722668] Component RolloutWorker_w0 stopped! | |
| [2023-05-24 20:18:35,373][2737046] Stopping InferenceWorker_p0-w0... | |
| [2023-05-24 20:18:35,373][2737046] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-05-24 20:18:35,373][2737053] Stopping RolloutWorker_w6... | |
| [2023-05-24 20:18:35,374][2737053] Loop rollout_proc6_evt_loop terminating... | |
| [2023-05-24 20:18:35,373][2722668] Component RolloutWorker_w4 stopped! | |
| [2023-05-24 20:18:35,375][2722668] Component RolloutWorker_w2 stopped! | |
| [2023-05-24 20:18:35,376][2722668] Component RolloutWorker_w5 stopped! | |
| [2023-05-24 20:18:35,377][2722668] Component RolloutWorker_w1 stopped! | |
| [2023-05-24 20:18:35,377][2722668] Component InferenceWorker_p0-w0 stopped! | |
| [2023-05-24 20:18:35,378][2722668] Component RolloutWorker_w6 stopped! | |
| [2023-05-24 20:18:35,414][2737021] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-05-24 20:18:35,481][2737021] Stopping LearnerWorker_p0... | |
| [2023-05-24 20:18:35,481][2737021] Loop learner_proc0_evt_loop terminating... | |
| [2023-05-24 20:18:35,481][2722668] Component LearnerWorker_p0 stopped! | |
| [2023-05-24 20:18:35,483][2722668] Waiting for process learner_proc0 to stop... | |
| [2023-05-24 20:18:36,292][2722668] Waiting for process inference_proc0-0 to join... | |
| [2023-05-24 20:18:36,294][2722668] Waiting for process rollout_proc0 to join... | |
| [2023-05-24 20:18:36,296][2722668] Waiting for process rollout_proc1 to join... | |
| [2023-05-24 20:18:36,297][2722668] Waiting for process rollout_proc2 to join... | |
| [2023-05-24 20:18:36,298][2722668] Waiting for process rollout_proc3 to join... | |
| [2023-05-24 20:18:36,300][2722668] Waiting for process rollout_proc4 to join... | |
| [2023-05-24 20:18:36,301][2722668] Waiting for process rollout_proc5 to join... | |
| [2023-05-24 20:18:36,302][2722668] Waiting for process rollout_proc6 to join... | |
| [2023-05-24 20:18:36,303][2722668] Waiting for process rollout_proc7 to join... | |
| [2023-05-24 20:18:36,304][2722668] Batcher 0 profile tree view: | |
| batching: 12.0222, releasing_batches: 0.0254 | |
| [2023-05-24 20:18:36,305][2722668] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0000 | |
| wait_policy_total: 5.1293 | |
| update_model: 2.8909 | |
| weight_update: 0.0008 | |
| one_step: 0.0017 | |
| handle_policy_step: 163.8550 | |
| deserialize: 6.4795, stack: 0.9489, obs_to_device_normalize: 40.6929, forward: 70.6622, send_messages: 10.9683 | |
| prepare_outputs: 26.5535 | |
| to_cpu: 17.5240 | |
| [2023-05-24 20:18:36,306][2722668] Learner 0 profile tree view: | |
| misc: 0.0045, prepare_batch: 8.7858 | |
| train: 24.3247 | |
| epoch_init: 0.0055, minibatch_init: 0.0054, losses_postprocess: 0.2306, kl_divergence: 0.2157, after_optimizer: 7.1346 | |
| calculate_losses: 8.0696 | |
| losses_init: 0.0036, forward_head: 0.7724, bptt_initial: 4.9783, tail: 0.4018, advantages_returns: 0.1117, losses: 0.8255 | |
| bptt: 0.8351 | |
| bptt_forward_core: 0.8008 | |
| update: 8.3318 | |
| clip: 1.1532 | |
| [2023-05-24 20:18:36,306][2722668] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.1621, enqueue_policy_requests: 7.3629, env_step: 118.9770, overhead: 8.9700, complete_rollouts: 0.2239 | |
| save_policy_outputs: 8.9400 | |
| split_output_tensors: 4.3843 | |
| [2023-05-24 20:18:36,307][2722668] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.1557, enqueue_policy_requests: 7.3900, env_step: 118.9123, overhead: 9.0051, complete_rollouts: 0.2221 | |
| save_policy_outputs: 9.1419 | |
| split_output_tensors: 4.4803 | |
| [2023-05-24 20:18:36,308][2722668] Loop Runner_EvtLoop terminating... | |
| [2023-05-24 20:18:36,309][2722668] Runner profile tree view: | |
| main_loop: 191.8832 | |
| [2023-05-24 20:18:36,310][2722668] Collected {0: 4005888}, FPS: 20876.7 | |
| [2023-05-24 20:25:41,995][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json | |
| [2023-05-24 20:25:41,996][2722668] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-05-24 20:25:41,997][2722668] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-05-24 20:25:41,997][2722668] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-05-24 20:25:41,998][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:25:41,999][2722668] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-05-24 20:25:41,999][2722668] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:25:42,000][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-05-24 20:25:42,001][2722668] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2023-05-24 20:25:42,001][2722668] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2023-05-24 20:25:42,002][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-05-24 20:25:42,003][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-05-24 20:25:42,004][2722668] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-05-24 20:25:42,004][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-05-24 20:25:42,007][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-05-24 20:25:42,019][2722668] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:25:42,021][2722668] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:25:42,023][2722668] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:25:42,047][2722668] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:25:42,207][2722668] Conv encoder output size: 512 | |
| [2023-05-24 20:25:42,208][2722668] Policy head output size: 512 | |
| [2023-05-24 20:25:44,701][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-05-24 20:25:46,226][2722668] Num frames 100... | |
| [2023-05-24 20:25:46,388][2722668] Num frames 200... | |
| [2023-05-24 20:25:46,546][2722668] Num frames 300... | |
| [2023-05-24 20:25:46,704][2722668] Num frames 400... | |
| [2023-05-24 20:25:46,870][2722668] Num frames 500... | |
| [2023-05-24 20:25:47,030][2722668] Num frames 600... | |
| [2023-05-24 20:25:47,200][2722668] Num frames 700... | |
| [2023-05-24 20:25:47,369][2722668] Num frames 800... | |
| [2023-05-24 20:25:47,528][2722668] Num frames 900... | |
| [2023-05-24 20:25:47,694][2722668] Num frames 1000... | |
| [2023-05-24 20:25:47,853][2722668] Num frames 1100... | |
| [2023-05-24 20:25:48,018][2722668] Num frames 1200... | |
| [2023-05-24 20:25:48,181][2722668] Num frames 1300... | |
| [2023-05-24 20:25:48,342][2722668] Num frames 1400... | |
| [2023-05-24 20:25:48,512][2722668] Num frames 1500... | |
| [2023-05-24 20:25:48,684][2722668] Num frames 1600... | |
| [2023-05-24 20:25:48,870][2722668] Avg episode rewards: #0: 39.780, true rewards: #0: 16.780 | |
| [2023-05-24 20:25:48,872][2722668] Avg episode reward: 39.780, avg true_objective: 16.780 | |
| [2023-05-24 20:25:48,913][2722668] Num frames 1700... | |
| [2023-05-24 20:25:49,081][2722668] Num frames 1800... | |
| [2023-05-24 20:25:49,236][2722668] Num frames 1900... | |
| [2023-05-24 20:25:49,394][2722668] Num frames 2000... | |
| [2023-05-24 20:25:49,553][2722668] Num frames 2100... | |
| [2023-05-24 20:25:49,720][2722668] Num frames 2200... | |
| [2023-05-24 20:25:49,885][2722668] Num frames 2300... | |
| [2023-05-24 20:25:50,050][2722668] Num frames 2400... | |
| [2023-05-24 20:25:50,215][2722668] Num frames 2500... | |
| [2023-05-24 20:25:50,370][2722668] Num frames 2600... | |
| [2023-05-24 20:25:50,528][2722668] Num frames 2700... | |
| [2023-05-24 20:25:50,629][2722668] Avg episode rewards: #0: 30.640, true rewards: #0: 13.640 | |
| [2023-05-24 20:25:50,631][2722668] Avg episode reward: 30.640, avg true_objective: 13.640 | |
| [2023-05-24 20:25:50,749][2722668] Num frames 2800... | |
| [2023-05-24 20:25:50,906][2722668] Num frames 2900... | |
| [2023-05-24 20:25:51,071][2722668] Num frames 3000... | |
| [2023-05-24 20:25:51,220][2722668] Num frames 3100... | |
| [2023-05-24 20:25:51,283][2722668] Avg episode rewards: #0: 22.010, true rewards: #0: 10.343 | |
| [2023-05-24 20:25:51,284][2722668] Avg episode reward: 22.010, avg true_objective: 10.343 | |
| [2023-05-24 20:25:51,452][2722668] Num frames 3200... | |
| [2023-05-24 20:25:51,616][2722668] Num frames 3300... | |
| [2023-05-24 20:25:51,774][2722668] Num frames 3400... | |
| [2023-05-24 20:25:51,938][2722668] Num frames 3500... | |
| [2023-05-24 20:25:52,091][2722668] Num frames 3600... | |
| [2023-05-24 20:25:52,251][2722668] Num frames 3700... | |
| [2023-05-24 20:25:52,422][2722668] Num frames 3800... | |
| [2023-05-24 20:25:52,596][2722668] Num frames 3900... | |
| [2023-05-24 20:25:52,765][2722668] Num frames 4000... | |
| [2023-05-24 20:25:52,945][2722668] Num frames 4100... | |
| [2023-05-24 20:25:53,115][2722668] Num frames 4200... | |
| [2023-05-24 20:25:53,280][2722668] Num frames 4300... | |
| [2023-05-24 20:25:53,439][2722668] Num frames 4400... | |
| [2023-05-24 20:25:53,575][2722668] Avg episode rewards: #0: 23.368, true rewards: #0: 11.117 | |
| [2023-05-24 20:25:53,577][2722668] Avg episode reward: 23.368, avg true_objective: 11.117 | |
| [2023-05-24 20:25:53,670][2722668] Num frames 4500... | |
| [2023-05-24 20:25:53,833][2722668] Num frames 4600... | |
| [2023-05-24 20:25:53,990][2722668] Num frames 4700... | |
| [2023-05-24 20:25:54,159][2722668] Num frames 4800... | |
| [2023-05-24 20:25:54,317][2722668] Num frames 4900... | |
| [2023-05-24 20:25:54,483][2722668] Num frames 5000... | |
| [2023-05-24 20:25:54,656][2722668] Num frames 5100... | |
| [2023-05-24 20:25:54,819][2722668] Num frames 5200... | |
| [2023-05-24 20:25:54,978][2722668] Num frames 5300... | |
| [2023-05-24 20:25:55,055][2722668] Avg episode rewards: #0: 22.222, true rewards: #0: 10.622 | |
| [2023-05-24 20:25:55,057][2722668] Avg episode reward: 22.222, avg true_objective: 10.622 | |
| [2023-05-24 20:25:55,204][2722668] Num frames 5400... | |
| [2023-05-24 20:25:55,368][2722668] Num frames 5500... | |
| [2023-05-24 20:25:55,527][2722668] Num frames 5600... | |
| [2023-05-24 20:25:55,686][2722668] Num frames 5700... | |
| [2023-05-24 20:25:55,853][2722668] Num frames 5800... | |
| [2023-05-24 20:25:56,015][2722668] Num frames 5900... | |
| [2023-05-24 20:25:56,177][2722668] Num frames 6000... | |
| [2023-05-24 20:25:56,325][2722668] Num frames 6100... | |
| [2023-05-24 20:25:56,461][2722668] Num frames 6200... | |
| [2023-05-24 20:25:56,624][2722668] Num frames 6300... | |
| [2023-05-24 20:25:56,686][2722668] Avg episode rewards: #0: 22.172, true rewards: #0: 10.505 | |
| [2023-05-24 20:25:56,688][2722668] Avg episode reward: 22.172, avg true_objective: 10.505 | |
| [2023-05-24 20:25:56,845][2722668] Num frames 6400... | |
| [2023-05-24 20:25:57,007][2722668] Num frames 6500... | |
| [2023-05-24 20:25:57,166][2722668] Num frames 6600... | |
| [2023-05-24 20:25:57,325][2722668] Num frames 6700... | |
| [2023-05-24 20:25:57,485][2722668] Num frames 6800... | |
| [2023-05-24 20:25:57,645][2722668] Num frames 6900... | |
| [2023-05-24 20:25:57,773][2722668] Num frames 7000... | |
| [2023-05-24 20:25:57,919][2722668] Num frames 7100... | |
| [2023-05-24 20:25:58,091][2722668] Num frames 7200... | |
| [2023-05-24 20:25:58,254][2722668] Num frames 7300... | |
| [2023-05-24 20:25:58,356][2722668] Avg episode rewards: #0: 22.469, true rewards: #0: 10.469 | |
| [2023-05-24 20:25:58,358][2722668] Avg episode reward: 22.469, avg true_objective: 10.469 | |
| [2023-05-24 20:25:58,478][2722668] Num frames 7400... | |
| [2023-05-24 20:25:58,648][2722668] Num frames 7500... | |
| [2023-05-24 20:25:58,810][2722668] Num frames 7600... | |
| [2023-05-24 20:25:58,966][2722668] Num frames 7700... | |
| [2023-05-24 20:25:59,124][2722668] Num frames 7800... | |
| [2023-05-24 20:25:59,297][2722668] Num frames 7900... | |
| [2023-05-24 20:25:59,413][2722668] Avg episode rewards: #0: 21.039, true rewards: #0: 9.914 | |
| [2023-05-24 20:25:59,415][2722668] Avg episode reward: 21.039, avg true_objective: 9.914 | |
| [2023-05-24 20:25:59,534][2722668] Num frames 8000... | |
| [2023-05-24 20:25:59,679][2722668] Num frames 8100... | |
| [2023-05-24 20:25:59,814][2722668] Num frames 8200... | |
| [2023-05-24 20:25:59,951][2722668] Num frames 8300... | |
| [2023-05-24 20:26:00,065][2722668] Avg episode rewards: #0: 19.491, true rewards: #0: 9.269 | |
| [2023-05-24 20:26:00,067][2722668] Avg episode reward: 19.491, avg true_objective: 9.269 | |
| [2023-05-24 20:26:00,151][2722668] Num frames 8400... | |
| [2023-05-24 20:26:00,305][2722668] Num frames 8500... | |
| [2023-05-24 20:26:00,467][2722668] Num frames 8600... | |
| [2023-05-24 20:26:00,632][2722668] Num frames 8700... | |
| [2023-05-24 20:26:00,796][2722668] Num frames 8800... | |
| [2023-05-24 20:26:00,958][2722668] Num frames 8900... | |
| [2023-05-24 20:26:01,115][2722668] Num frames 9000... | |
| [2023-05-24 20:26:01,289][2722668] Num frames 9100... | |
| [2023-05-24 20:26:01,458][2722668] Num frames 9200... | |
| [2023-05-24 20:26:01,606][2722668] Num frames 9300... | |
| [2023-05-24 20:26:01,770][2722668] Num frames 9400... | |
| [2023-05-24 20:26:01,931][2722668] Num frames 9500... | |
| [2023-05-24 20:26:02,093][2722668] Num frames 9600... | |
| [2023-05-24 20:26:02,250][2722668] Num frames 9700... | |
| [2023-05-24 20:26:02,416][2722668] Num frames 9800... | |
| [2023-05-24 20:26:02,573][2722668] Num frames 9900... | |
| [2023-05-24 20:26:02,736][2722668] Num frames 10000... | |
| [2023-05-24 20:26:02,903][2722668] Num frames 10100... | |
| [2023-05-24 20:26:03,073][2722668] Num frames 10200... | |
| [2023-05-24 20:26:03,243][2722668] Num frames 10300... | |
| [2023-05-24 20:26:03,442][2722668] Avg episode rewards: #0: 23.177, true rewards: #0: 10.377 | |
| [2023-05-24 20:26:03,443][2722668] Avg episode reward: 23.177, avg true_objective: 10.377 | |
| [2023-05-24 20:26:28,662][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-05-24 20:36:55,928][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json | |
| [2023-05-24 20:36:55,929][2722668] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-05-24 20:36:55,930][2722668] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-05-24 20:36:55,931][2722668] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-05-24 20:36:55,931][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:36:55,932][2722668] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-05-24 20:36:55,933][2722668] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-05-24 20:36:55,935][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-05-24 20:36:55,935][2722668] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-05-24 20:36:55,936][2722668] Adding new argument 'hf_repository'='markeidsaune/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-05-24 20:36:55,937][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-05-24 20:36:55,938][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-05-24 20:36:55,939][2722668] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-05-24 20:36:55,940][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-05-24 20:36:55,942][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-05-24 20:36:55,956][2722668] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:36:55,958][2722668] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:36:55,973][2722668] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:36:56,024][2722668] Conv encoder output size: 512 | |
| [2023-05-24 20:36:56,025][2722668] Policy head output size: 512 | |
| [2023-05-24 20:36:56,073][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-05-24 20:36:56,899][2722668] Num frames 100... | |
| [2023-05-24 20:36:57,069][2722668] Num frames 200... | |
| [2023-05-24 20:36:57,233][2722668] Num frames 300... | |
| [2023-05-24 20:36:57,385][2722668] Num frames 400... | |
| [2023-05-24 20:36:57,538][2722668] Num frames 500... | |
| [2023-05-24 20:36:57,632][2722668] Avg episode rewards: #0: 9.230, true rewards: #0: 5.230 | |
| [2023-05-24 20:36:57,634][2722668] Avg episode reward: 9.230, avg true_objective: 5.230 | |
| [2023-05-24 20:36:57,756][2722668] Num frames 600... | |
| [2023-05-24 20:36:57,913][2722668] Num frames 700... | |
| [2023-05-24 20:36:58,075][2722668] Num frames 800... | |
| [2023-05-24 20:36:58,229][2722668] Num frames 900... | |
| [2023-05-24 20:36:58,384][2722668] Num frames 1000... | |
| [2023-05-24 20:36:58,539][2722668] Num frames 1100... | |
| [2023-05-24 20:36:58,697][2722668] Num frames 1200... | |
| [2023-05-24 20:36:58,858][2722668] Num frames 1300... | |
| [2023-05-24 20:36:59,059][2722668] Avg episode rewards: #0: 14.435, true rewards: #0: 6.935 | |
| [2023-05-24 20:36:59,061][2722668] Avg episode reward: 14.435, avg true_objective: 6.935 | |
| [2023-05-24 20:36:59,088][2722668] Num frames 1400... | |
| [2023-05-24 20:36:59,254][2722668] Num frames 1500... | |
| [2023-05-24 20:36:59,418][2722668] Num frames 1600... | |
| [2023-05-24 20:36:59,577][2722668] Num frames 1700... | |
| [2023-05-24 20:36:59,736][2722668] Num frames 1800... | |
| [2023-05-24 20:36:59,799][2722668] Avg episode rewards: #0: 11.677, true rewards: #0: 6.010 | |
| [2023-05-24 20:36:59,800][2722668] Avg episode reward: 11.677, avg true_objective: 6.010 | |
| [2023-05-24 20:36:59,961][2722668] Num frames 1900... | |
| [2023-05-24 20:37:00,133][2722668] Num frames 2000... | |
| [2023-05-24 20:37:00,299][2722668] Num frames 2100... | |
| [2023-05-24 20:37:00,452][2722668] Num frames 2200... | |
| [2023-05-24 20:37:00,589][2722668] Num frames 2300... | |
| [2023-05-24 20:37:00,726][2722668] Num frames 2400... | |
| [2023-05-24 20:37:00,861][2722668] Num frames 2500... | |
| [2023-05-24 20:37:01,014][2722668] Num frames 2600... | |
| [2023-05-24 20:37:01,177][2722668] Num frames 2700... | |
| [2023-05-24 20:37:01,338][2722668] Num frames 2800... | |
| [2023-05-24 20:37:01,494][2722668] Num frames 2900... | |
| [2023-05-24 20:37:01,651][2722668] Num frames 3000... | |
| [2023-05-24 20:37:01,757][2722668] Avg episode rewards: #0: 15.830, true rewards: #0: 7.580 | |
| [2023-05-24 20:37:01,759][2722668] Avg episode reward: 15.830, avg true_objective: 7.580 | |
| [2023-05-24 20:37:01,869][2722668] Num frames 3100... | |
| [2023-05-24 20:37:02,015][2722668] Num frames 3200... | |
| [2023-05-24 20:37:02,173][2722668] Num frames 3300... | |
| [2023-05-24 20:37:02,344][2722668] Num frames 3400... | |
| [2023-05-24 20:37:02,511][2722668] Num frames 3500... | |
| [2023-05-24 20:37:02,677][2722668] Num frames 3600... | |
| [2023-05-24 20:37:02,838][2722668] Num frames 3700... | |
| [2023-05-24 20:37:03,002][2722668] Num frames 3800... | |
| [2023-05-24 20:37:03,163][2722668] Num frames 3900... | |
| [2023-05-24 20:37:03,330][2722668] Num frames 4000... | |
| [2023-05-24 20:37:03,480][2722668] Num frames 4100... | |
| [2023-05-24 20:37:03,643][2722668] Num frames 4200... | |
| [2023-05-24 20:37:03,803][2722668] Num frames 4300... | |
| [2023-05-24 20:37:03,980][2722668] Num frames 4400... | |
| [2023-05-24 20:37:04,057][2722668] Avg episode rewards: #0: 18.616, true rewards: #0: 8.816 | |
| [2023-05-24 20:37:04,058][2722668] Avg episode reward: 18.616, avg true_objective: 8.816 | |
| [2023-05-24 20:37:04,219][2722668] Num frames 4500... | |
| [2023-05-24 20:37:04,399][2722668] Num frames 4600... | |
| [2023-05-24 20:37:04,557][2722668] Num frames 4700... | |
| [2023-05-24 20:37:04,721][2722668] Num frames 4800... | |
| [2023-05-24 20:37:04,884][2722668] Num frames 4900... | |
| [2023-05-24 20:37:05,038][2722668] Num frames 5000... | |
| [2023-05-24 20:37:05,198][2722668] Num frames 5100... | |
| [2023-05-24 20:37:05,274][2722668] Avg episode rewards: #0: 18.187, true rewards: #0: 8.520 | |
| [2023-05-24 20:37:05,276][2722668] Avg episode reward: 18.187, avg true_objective: 8.520 | |
| [2023-05-24 20:37:05,417][2722668] Num frames 5200... | |
| [2023-05-24 20:37:05,574][2722668] Num frames 5300... | |
| [2023-05-24 20:37:05,746][2722668] Num frames 5400... | |
| [2023-05-24 20:37:05,910][2722668] Num frames 5500... | |
| [2023-05-24 20:37:06,071][2722668] Num frames 5600... | |
| [2023-05-24 20:37:06,250][2722668] Num frames 5700... | |
| [2023-05-24 20:37:06,405][2722668] Num frames 5800... | |
| [2023-05-24 20:37:06,564][2722668] Num frames 5900... | |
| [2023-05-24 20:37:06,728][2722668] Num frames 6000... | |
| [2023-05-24 20:37:06,887][2722668] Num frames 6100... | |
| [2023-05-24 20:37:07,042][2722668] Num frames 6200... | |
| [2023-05-24 20:37:07,201][2722668] Num frames 6300... | |
| [2023-05-24 20:37:07,363][2722668] Num frames 6400... | |
| [2023-05-24 20:37:07,520][2722668] Num frames 6500... | |
| [2023-05-24 20:37:07,686][2722668] Num frames 6600... | |
| [2023-05-24 20:37:07,852][2722668] Num frames 6700... | |
| [2023-05-24 20:37:08,013][2722668] Num frames 6800... | |
| [2023-05-24 20:37:08,180][2722668] Num frames 6900... | |
| [2023-05-24 20:37:08,296][2722668] Avg episode rewards: #0: 22.194, true rewards: #0: 9.909 | |
| [2023-05-24 20:37:08,298][2722668] Avg episode reward: 22.194, avg true_objective: 9.909 | |
| [2023-05-24 20:37:08,399][2722668] Num frames 7000... | |
| [2023-05-24 20:37:08,566][2722668] Num frames 7100... | |
| [2023-05-24 20:37:08,728][2722668] Num frames 7200... | |
| [2023-05-24 20:37:08,936][2722668] Avg episode rewards: #0: 20.373, true rewards: #0: 9.122 | |
| [2023-05-24 20:37:08,938][2722668] Avg episode reward: 20.373, avg true_objective: 9.122 | |
| [2023-05-24 20:37:08,947][2722668] Num frames 7300... | |
| [2023-05-24 20:37:09,111][2722668] Num frames 7400... | |
| [2023-05-24 20:37:09,270][2722668] Num frames 7500... | |
| [2023-05-24 20:37:09,436][2722668] Num frames 7600... | |
| [2023-05-24 20:37:09,599][2722668] Num frames 7700... | |
| [2023-05-24 20:37:09,755][2722668] Num frames 7800... | |
| [2023-05-24 20:37:09,916][2722668] Num frames 7900... | |
| [2023-05-24 20:37:10,079][2722668] Num frames 8000... | |
| [2023-05-24 20:37:10,238][2722668] Num frames 8100... | |
| [2023-05-24 20:37:10,397][2722668] Num frames 8200... | |
| [2023-05-24 20:37:10,557][2722668] Num frames 8300... | |
| [2023-05-24 20:37:10,713][2722668] Num frames 8400... | |
| [2023-05-24 20:37:10,873][2722668] Num frames 8500... | |
| [2023-05-24 20:37:11,027][2722668] Num frames 8600... | |
| [2023-05-24 20:37:11,193][2722668] Num frames 8700... | |
| [2023-05-24 20:37:11,356][2722668] Avg episode rewards: #0: 22.078, true rewards: #0: 9.744 | |
| [2023-05-24 20:37:11,358][2722668] Avg episode reward: 22.078, avg true_objective: 9.744 | |
| [2023-05-24 20:37:11,408][2722668] Num frames 8800... | |
| [2023-05-24 20:37:11,565][2722668] Num frames 8900... | |
| [2023-05-24 20:37:11,726][2722668] Num frames 9000... | |
| [2023-05-24 20:37:11,891][2722668] Num frames 9100... | |
| [2023-05-24 20:37:12,051][2722668] Num frames 9200... | |
| [2023-05-24 20:37:12,203][2722668] Num frames 9300... | |
| [2023-05-24 20:37:12,284][2722668] Avg episode rewards: #0: 20.614, true rewards: #0: 9.314 | |
| [2023-05-24 20:37:12,285][2722668] Avg episode reward: 20.614, avg true_objective: 9.314 | |
| [2023-05-24 20:37:34,803][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-05-24 20:39:03,801][2722668] The model has been pushed to https://huggingface.co/markeidsaune/rl_course_vizdoom_health_gathering_supreme | |
| [2023-05-24 20:40:18,815][2722668] Environment doom_basic already registered, overwriting... | |
| [2023-05-24 20:40:18,816][2722668] Environment doom_two_colors_easy already registered, overwriting... | |
| [2023-05-24 20:40:18,817][2722668] Environment doom_two_colors_hard already registered, overwriting... | |
| [2023-05-24 20:40:18,817][2722668] Environment doom_dm already registered, overwriting... | |
| [2023-05-24 20:40:18,818][2722668] Environment doom_dwango5 already registered, overwriting... | |
| [2023-05-24 20:40:18,819][2722668] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2023-05-24 20:40:18,820][2722668] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2023-05-24 20:40:18,821][2722668] Environment doom_my_way_home already registered, overwriting... | |
| [2023-05-24 20:40:18,821][2722668] Environment doom_deadly_corridor already registered, overwriting... | |
| [2023-05-24 20:40:18,822][2722668] Environment doom_defend_the_center already registered, overwriting... | |
| [2023-05-24 20:40:18,823][2722668] Environment doom_defend_the_line already registered, overwriting... | |
| [2023-05-24 20:40:18,823][2722668] Environment doom_health_gathering already registered, overwriting... | |
| [2023-05-24 20:40:18,824][2722668] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2023-05-24 20:40:18,825][2722668] Environment doom_battle already registered, overwriting... | |
| [2023-05-24 20:40:18,825][2722668] Environment doom_battle2 already registered, overwriting... | |
| [2023-05-24 20:40:18,826][2722668] Environment doom_duel_bots already registered, overwriting... | |
| [2023-05-24 20:40:18,826][2722668] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2023-05-24 20:40:18,827][2722668] Environment doom_duel already registered, overwriting... | |
| [2023-05-24 20:40:18,828][2722668] Environment doom_deathmatch_full already registered, overwriting... | |
| [2023-05-24 20:40:18,828][2722668] Environment doom_benchmark already registered, overwriting... | |
| [2023-05-24 20:40:18,829][2722668] register_encoder_factory: <function make_vizdoom_encoder at 0x7f9e60e3bc70> | |
| [2023-05-24 20:40:18,843][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json | |
| [2023-05-24 20:40:18,844][2722668] Overriding arg 'train_for_env_steps' with value 10000000 passed from command line | |
| [2023-05-24 20:40:18,849][2722668] Experiment dir /home/mark/rl_course/unit8/train_dir/default_experiment already exists! | |
| [2023-05-24 20:40:18,850][2722668] Resuming existing experiment from /home/mark/rl_course/unit8/train_dir/default_experiment... | |
| [2023-05-24 20:40:18,850][2722668] Weights and Biases integration disabled | |
| [2023-05-24 20:40:18,854][2722668] Environment var CUDA_VISIBLE_DEVICES is 0,1 | |
| [2023-05-24 20:40:20,634][2722668] Starting experiment with the following configuration: | |
| help=False | |
| algo=APPO | |
| env=doom_health_gathering_supreme | |
| experiment=default_experiment | |
| train_dir=/home/mark/rl_course/unit8/train_dir | |
| restart_behavior=resume | |
| device=gpu | |
| seed=None | |
| num_policies=1 | |
| async_rl=True | |
| serial_mode=False | |
| batched_sampling=False | |
| num_batches_to_accumulate=2 | |
| worker_num_splits=2 | |
| policy_workers_per_policy=1 | |
| max_policy_lag=1000 | |
| num_workers=8 | |
| num_envs_per_worker=4 | |
| batch_size=1024 | |
| num_batches_per_epoch=1 | |
| num_epochs=1 | |
| rollout=32 | |
| recurrence=32 | |
| shuffle_minibatches=False | |
| gamma=0.99 | |
| reward_scale=1.0 | |
| reward_clip=1000.0 | |
| value_bootstrap=False | |
| normalize_returns=True | |
| exploration_loss_coeff=0.001 | |
| value_loss_coeff=0.5 | |
| kl_loss_coeff=0.0 | |
| exploration_loss=symmetric_kl | |
| gae_lambda=0.95 | |
| ppo_clip_ratio=0.1 | |
| ppo_clip_value=0.2 | |
| with_vtrace=False | |
| vtrace_rho=1.0 | |
| vtrace_c=1.0 | |
| optimizer=adam | |
| adam_eps=1e-06 | |
| adam_beta1=0.9 | |
| adam_beta2=0.999 | |
| max_grad_norm=4.0 | |
| learning_rate=0.0001 | |
| lr_schedule=constant | |
| lr_schedule_kl_threshold=0.008 | |
| lr_adaptive_min=1e-06 | |
| lr_adaptive_max=0.01 | |
| obs_subtract_mean=0.0 | |
| obs_scale=255.0 | |
| normalize_input=True | |
| normalize_input_keys=None | |
| decorrelate_experience_max_seconds=0 | |
| decorrelate_envs_on_one_worker=True | |
| actor_worker_gpus=[] | |
| set_workers_cpu_affinity=True | |
| force_envs_single_thread=False | |
| default_niceness=0 | |
| log_to_file=True | |
| experiment_summaries_interval=10 | |
| flush_summaries_interval=30 | |
| stats_avg=100 | |
| summaries_use_frameskip=True | |
| heartbeat_interval=20 | |
| heartbeat_reporting_interval=600 | |
| train_for_env_steps=10000000 | |
| train_for_seconds=10000000000 | |
| save_every_sec=120 | |
| keep_checkpoints=2 | |
| load_checkpoint_kind=latest | |
| save_milestones_sec=-1 | |
| save_best_every_sec=5 | |
| save_best_metric=reward | |
| save_best_after=100000 | |
| benchmark=False | |
| encoder_mlp_layers=[512, 512] | |
| encoder_conv_architecture=convnet_simple | |
| encoder_conv_mlp_layers=[512] | |
| use_rnn=True | |
| rnn_size=512 | |
| rnn_type=gru | |
| rnn_num_layers=1 | |
| decoder_mlp_layers=[] | |
| nonlinearity=elu | |
| policy_initialization=orthogonal | |
| policy_init_gain=1.0 | |
| actor_critic_share_weights=True | |
| adaptive_stddev=True | |
| continuous_tanh_scale=0.0 | |
| initial_stddev=1.0 | |
| use_env_info_cache=False | |
| env_gpu_actions=False | |
| env_gpu_observations=True | |
| env_frameskip=4 | |
| env_framestack=1 | |
| pixel_format=CHW | |
| use_record_episode_statistics=False | |
| with_wandb=False | |
| wandb_user=None | |
| wandb_project=sample_factory | |
| wandb_group=None | |
| wandb_job_type=SF | |
| wandb_tags=[] | |
| with_pbt=False | |
| pbt_mix_policies_in_one_env=True | |
| pbt_period_env_steps=5000000 | |
| pbt_start_mutation=20000000 | |
| pbt_replace_fraction=0.3 | |
| pbt_mutation_rate=0.15 | |
| pbt_replace_reward_gap=0.1 | |
| pbt_replace_reward_gap_absolute=1e-06 | |
| pbt_optimize_gamma=False | |
| pbt_target_objective=true_objective | |
| pbt_perturb_min=1.1 | |
| pbt_perturb_max=1.5 | |
| num_agents=-1 | |
| num_humans=0 | |
| num_bots=-1 | |
| start_bot_difficulty=None | |
| timelimit=None | |
| res_w=128 | |
| res_h=72 | |
| wide_aspect_ratio=False | |
| eval_env_frameskip=1 | |
| fps=35 | |
| command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 | |
| cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} | |
| git_hash=unknown | |
| git_repo_name=not a git repository | |
| [2023-05-24 20:40:20,636][2722668] Saving configuration to /home/mark/rl_course/unit8/train_dir/default_experiment/config.json... | |
| [2023-05-24 20:40:20,638][2722668] Rollout worker 0 uses device cpu | |
| [2023-05-24 20:40:20,638][2722668] Rollout worker 1 uses device cpu | |
| [2023-05-24 20:40:20,640][2722668] Rollout worker 2 uses device cpu | |
| [2023-05-24 20:40:20,641][2722668] Rollout worker 3 uses device cpu | |
| [2023-05-24 20:40:20,642][2722668] Rollout worker 4 uses device cpu | |
| [2023-05-24 20:40:20,643][2722668] Rollout worker 5 uses device cpu | |
| [2023-05-24 20:40:20,644][2722668] Rollout worker 6 uses device cpu | |
| [2023-05-24 20:40:20,645][2722668] Rollout worker 7 uses device cpu | |
| [2023-05-24 20:40:20,676][2722668] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:40:20,677][2722668] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-05-24 20:40:20,706][2722668] Starting all processes... | |
| [2023-05-24 20:40:20,707][2722668] Starting process learner_proc0 | |
| [2023-05-24 20:40:20,756][2722668] Starting all processes... | |
| [2023-05-24 20:40:20,762][2722668] Starting process inference_proc0-0 | |
| [2023-05-24 20:40:20,762][2722668] Starting process rollout_proc0 | |
| [2023-05-24 20:40:20,762][2722668] Starting process rollout_proc1 | |
| [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc2 | |
| [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc3 | |
| [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc4 | |
| [2023-05-24 20:40:20,764][2722668] Starting process rollout_proc5 | |
| [2023-05-24 20:40:20,764][2722668] Starting process rollout_proc6 | |
| [2023-05-24 20:40:20,765][2722668] Starting process rollout_proc7 | |
| [2023-05-24 20:40:22,441][2740685] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,497][2740681] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:40:22,497][2740681] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-05-24 20:40:22,515][2740681] Num visible devices: 1 | |
| [2023-05-24 20:40:22,523][2740682] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,526][2740692] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,527][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:40:22,527][2740668] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-05-24 20:40:22,529][2740690] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,529][2740687] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,544][2740668] Num visible devices: 1 | |
| [2023-05-24 20:40:22,545][2740686] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,586][2740691] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,589][2740668] Starting seed is not provided | |
| [2023-05-24 20:40:22,589][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:40:22,589][2740668] Initializing actor-critic model on device cuda:0 | |
| [2023-05-24 20:40:22,589][2740668] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:40:22,590][2740668] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:40:22,600][2740668] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:40:22,627][2740684] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] | |
| [2023-05-24 20:40:22,722][2740668] Conv encoder output size: 512 | |
| [2023-05-24 20:40:22,722][2740668] Policy head output size: 512 | |
| [2023-05-24 20:40:22,747][2740668] Created Actor Critic model with architecture: | |
| [2023-05-24 20:40:22,747][2740668] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-05-24 20:40:25,214][2740668] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-05-24 20:40:25,215][2740668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-05-24 20:40:25,237][2740668] Loading model from checkpoint | |
| [2023-05-24 20:40:25,241][2740668] Loaded experiment state at self.train_step=978, self.env_steps=4005888 | |
| [2023-05-24 20:40:25,242][2740668] Initialized policy 0 weights for model version 978 | |
| [2023-05-24 20:40:25,244][2740668] LearnerWorker_p0 finished initialization! | |
| [2023-05-24 20:40:25,244][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-05-24 20:40:25,355][2740681] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:40:25,356][2740681] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:40:25,371][2740681] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:40:25,509][2740681] Conv encoder output size: 512 | |
| [2023-05-24 20:40:25,509][2740681] Policy head output size: 512 | |
| [2023-05-24 20:40:27,915][2722668] Inference worker 0-0 is ready! | |
| [2023-05-24 20:40:27,917][2722668] All inference workers are ready! Signal rollout workers to start! | |
| [2023-05-24 20:40:27,960][2740685] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:27,965][2740690] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:27,967][2740682] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:27,968][2740686] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:27,968][2740687] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:27,970][2740692] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:28,012][2740691] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:28,016][2740684] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-05-24 20:40:28,553][2740685] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,556][2740686] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,557][2740682] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,559][2740692] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,560][2740687] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,563][2740690] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,854][2722668] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-05-24 20:40:28,871][2740686] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:28,876][2740692] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:28,882][2740690] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:28,884][2740684] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:28,910][2740691] Decorrelating experience for 0 frames... | |
| [2023-05-24 20:40:29,200][2740684] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:29,227][2740682] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:29,228][2740686] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,229][2740691] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:29,521][2740685] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:29,543][2740692] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,556][2740684] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,585][2740687] Decorrelating experience for 32 frames... | |
| [2023-05-24 20:40:29,600][2740691] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,841][2740682] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,897][2740686] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:29,918][2740684] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:29,952][2740687] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:29,966][2740691] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:30,191][2740685] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:30,233][2740682] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:30,248][2740690] Decorrelating experience for 64 frames... | |
| [2023-05-24 20:40:30,312][2740687] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:30,541][2740692] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:30,615][2740690] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:30,870][2740685] Decorrelating experience for 96 frames... | |
| [2023-05-24 20:40:31,319][2740668] Signal inference workers to stop experience collection... | |
| [2023-05-24 20:40:31,322][2740681] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-05-24 20:40:32,820][2740668] Signal inference workers to resume experience collection... | |
| [2023-05-24 20:40:32,821][2740681] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-05-24 20:40:33,854][2722668] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4009984. Throughput: 0: 97.6. Samples: 488. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-05-24 20:40:33,856][2722668] Avg episode reward: [(0, '4.377')] | |
| [2023-05-24 20:40:35,239][2740681] Updated weights for policy 0, policy_version 988 (0.0452) | |
| [2023-05-24 20:40:37,059][2740681] Updated weights for policy 0, policy_version 998 (0.0008) | |
| [2023-05-24 20:40:38,854][2722668] Fps is (10 sec: 11878.3, 60 sec: 11878.3, 300 sec: 11878.3). Total num frames: 4124672. Throughput: 0: 2054.2. Samples: 20542. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-05-24 20:40:38,855][2722668] Avg episode reward: [(0, '24.933')] | |
| [2023-05-24 20:40:38,872][2740681] Updated weights for policy 0, policy_version 1008 (0.0009) | |
| [2023-05-24 20:40:40,669][2722668] Heartbeat connected on Batcher_0 | |
| [2023-05-24 20:40:40,678][2740681] Updated weights for policy 0, policy_version 1018 (0.0009) | |
| [2023-05-24 20:40:40,678][2722668] Heartbeat connected on LearnerWorker_p0 | |
| [2023-05-24 20:40:40,680][2722668] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-05-24 20:40:40,681][2722668] Heartbeat connected on RolloutWorker_w0 | |
| [2023-05-24 20:40:40,687][2722668] Heartbeat connected on RolloutWorker_w1 | |
| [2023-05-24 20:40:40,690][2722668] Heartbeat connected on RolloutWorker_w2 | |
| [2023-05-24 20:40:40,693][2722668] Heartbeat connected on RolloutWorker_w3 | |
| [2023-05-24 20:40:40,698][2722668] Heartbeat connected on RolloutWorker_w4 | |
| [2023-05-24 20:40:40,699][2722668] Heartbeat connected on RolloutWorker_w5 | |
| [2023-05-24 20:40:40,701][2722668] Heartbeat connected on RolloutWorker_w6 | |
| [2023-05-24 20:40:40,706][2722668] Heartbeat connected on RolloutWorker_w7 | |
| [2023-05-24 20:40:42,493][2740681] Updated weights for policy 0, policy_version 1028 (0.0009) | |
| [2023-05-24 20:40:43,854][2722668] Fps is (10 sec: 22937.9, 60 sec: 15564.8, 300 sec: 15564.8). Total num frames: 4239360. Throughput: 0: 3632.0. Samples: 54480. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:40:43,855][2722668] Avg episode reward: [(0, '23.081')] | |
| [2023-05-24 20:40:44,347][2740681] Updated weights for policy 0, policy_version 1038 (0.0009) | |
| [2023-05-24 20:40:46,219][2740681] Updated weights for policy 0, policy_version 1048 (0.0008) | |
| [2023-05-24 20:40:48,027][2740681] Updated weights for policy 0, policy_version 1058 (0.0009) | |
| [2023-05-24 20:40:48,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 17203.2, 300 sec: 17203.2). Total num frames: 4349952. Throughput: 0: 3548.5. Samples: 70970. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:40:48,855][2722668] Avg episode reward: [(0, '22.824')] | |
| [2023-05-24 20:40:49,860][2740681] Updated weights for policy 0, policy_version 1068 (0.0009) | |
| [2023-05-24 20:40:51,644][2740681] Updated weights for policy 0, policy_version 1078 (0.0009) | |
| [2023-05-24 20:40:53,471][2740681] Updated weights for policy 0, policy_version 1088 (0.0008) | |
| [2023-05-24 20:40:53,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 18350.0, 300 sec: 18350.0). Total num frames: 4464640. Throughput: 0: 4191.5. Samples: 104788. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:40:53,855][2722668] Avg episode reward: [(0, '23.583')] | |
| [2023-05-24 20:40:55,305][2740681] Updated weights for policy 0, policy_version 1098 (0.0008) | |
| [2023-05-24 20:40:57,111][2740681] Updated weights for policy 0, policy_version 1108 (0.0008) | |
| [2023-05-24 20:40:58,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 18978.2, 300 sec: 18978.2). Total num frames: 4575232. Throughput: 0: 4619.2. Samples: 138576. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:40:58,855][2722668] Avg episode reward: [(0, '23.463')] | |
| [2023-05-24 20:40:58,940][2740681] Updated weights for policy 0, policy_version 1118 (0.0009) | |
| [2023-05-24 20:41:00,782][2740681] Updated weights for policy 0, policy_version 1128 (0.0009) | |
| [2023-05-24 20:41:02,592][2740681] Updated weights for policy 0, policy_version 1138 (0.0008) | |
| [2023-05-24 20:41:03,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 19426.7, 300 sec: 19426.7). Total num frames: 4685824. Throughput: 0: 4439.7. Samples: 155390. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:41:03,855][2722668] Avg episode reward: [(0, '23.344')] | |
| [2023-05-24 20:41:04,437][2740681] Updated weights for policy 0, policy_version 1148 (0.0008) | |
| [2023-05-24 20:41:06,265][2740681] Updated weights for policy 0, policy_version 1158 (0.0009) | |
| [2023-05-24 20:41:08,070][2740681] Updated weights for policy 0, policy_version 1168 (0.0009) | |
| [2023-05-24 20:41:08,854][2722668] Fps is (10 sec: 22527.7, 60 sec: 19865.6, 300 sec: 19865.6). Total num frames: 4800512. Throughput: 0: 4725.7. Samples: 189030. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:41:08,855][2722668] Avg episode reward: [(0, '23.984')] | |
| [2023-05-24 20:41:09,863][2740681] Updated weights for policy 0, policy_version 1178 (0.0009) | |
| [2023-05-24 20:41:11,658][2740681] Updated weights for policy 0, policy_version 1188 (0.0008) | |
| [2023-05-24 20:41:13,513][2740681] Updated weights for policy 0, policy_version 1198 (0.0008) | |
| [2023-05-24 20:41:13,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 20115.9, 300 sec: 20115.9). Total num frames: 4911104. Throughput: 0: 4952.5. Samples: 222862. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:41:13,855][2722668] Avg episode reward: [(0, '28.266')] | |
| [2023-05-24 20:41:13,866][2740668] Saving new best policy, reward=28.266! | |
| [2023-05-24 20:41:15,358][2740681] Updated weights for policy 0, policy_version 1208 (0.0009) | |
| [2023-05-24 20:41:17,206][2740681] Updated weights for policy 0, policy_version 1218 (0.0009) | |
| [2023-05-24 20:41:18,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 20398.1, 300 sec: 20398.1). Total num frames: 5025792. Throughput: 0: 5310.3. Samples: 239450. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:41:18,855][2722668] Avg episode reward: [(0, '24.734')] | |
| [2023-05-24 20:41:19,040][2740681] Updated weights for policy 0, policy_version 1228 (0.0009) | |
| [2023-05-24 20:41:20,866][2740681] Updated weights for policy 0, policy_version 1238 (0.0009) | |
| [2023-05-24 20:41:22,720][2740681] Updated weights for policy 0, policy_version 1248 (0.0009) | |
| [2023-05-24 20:41:23,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 20554.4, 300 sec: 20554.4). Total num frames: 5136384. Throughput: 0: 5610.5. Samples: 273016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-05-24 20:41:23,855][2722668] Avg episode reward: [(0, '28.659')] | |
| [2023-05-24 20:41:23,857][2740668] Saving new best policy, reward=28.659! | |
| [2023-05-24 20:41:24,558][2740681] Updated weights for policy 0, policy_version 1258 (0.0008) | |
| [2023-05-24 20:41:26,381][2740681] Updated weights for policy 0, policy_version 1268 (0.0009) | |
| [2023-05-24 20:41:28,188][2740681] Updated weights for policy 0, policy_version 1278 (0.0009) | |
| [2023-05-24 20:41:28,854][2722668] Fps is (10 sec: 22118.1, 60 sec: 20684.8, 300 sec: 20684.8). Total num frames: 5246976. Throughput: 0: 5600.2. Samples: 306490. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:41:28,855][2722668] Avg episode reward: [(0, '24.789')] | |
| [2023-05-24 20:41:30,008][2740681] Updated weights for policy 0, policy_version 1288 (0.0009) | |
| [2023-05-24 20:41:31,855][2740681] Updated weights for policy 0, policy_version 1298 (0.0008) | |
| [2023-05-24 20:41:33,670][2740681] Updated weights for policy 0, policy_version 1308 (0.0008) | |
| [2023-05-24 20:41:33,859][2722668] Fps is (10 sec: 22517.7, 60 sec: 22526.3, 300 sec: 20856.6). Total num frames: 5361664. Throughput: 0: 5606.0. Samples: 323266. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:41:33,861][2722668] Avg episode reward: [(0, '27.411')] | |
| [2023-05-24 20:41:35,483][2740681] Updated weights for policy 0, policy_version 1318 (0.0009) | |
| [2023-05-24 20:41:37,314][2740681] Updated weights for policy 0, policy_version 1328 (0.0009) | |
| [2023-05-24 20:41:38,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.7, 300 sec: 20948.1). Total num frames: 5472256. Throughput: 0: 5602.2. Samples: 356886. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:41:38,855][2722668] Avg episode reward: [(0, '27.920')] | |
| [2023-05-24 20:41:39,173][2740681] Updated weights for policy 0, policy_version 1338 (0.0009) | |
| [2023-05-24 20:41:40,976][2740681] Updated weights for policy 0, policy_version 1348 (0.0008) | |
| [2023-05-24 20:41:42,799][2740681] Updated weights for policy 0, policy_version 1358 (0.0008) | |
| [2023-05-24 20:41:43,854][2722668] Fps is (10 sec: 22128.9, 60 sec: 22391.5, 300 sec: 21026.2). Total num frames: 5582848. Throughput: 0: 5599.6. Samples: 390556. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:41:43,855][2722668] Avg episode reward: [(0, '26.898')] | |
| [2023-05-24 20:41:44,619][2740681] Updated weights for policy 0, policy_version 1368 (0.0009) | |
| [2023-05-24 20:41:46,452][2740681] Updated weights for policy 0, policy_version 1378 (0.0009) | |
| [2023-05-24 20:41:48,289][2740681] Updated weights for policy 0, policy_version 1388 (0.0008) | |
| [2023-05-24 20:41:48,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.8, 300 sec: 21145.6). Total num frames: 5697536. Throughput: 0: 5600.0. Samples: 407388. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:41:48,855][2722668] Avg episode reward: [(0, '27.218')] | |
| [2023-05-24 20:41:50,124][2740681] Updated weights for policy 0, policy_version 1398 (0.0008) | |
| [2023-05-24 20:41:51,976][2740681] Updated weights for policy 0, policy_version 1408 (0.0009) | |
| [2023-05-24 20:41:53,854][2722668] Fps is (10 sec: 22118.1, 60 sec: 22323.2, 300 sec: 21154.6). Total num frames: 5804032. Throughput: 0: 5593.0. Samples: 440716. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:41:53,855][2722668] Avg episode reward: [(0, '27.321')] | |
| [2023-05-24 20:41:53,860][2740681] Updated weights for policy 0, policy_version 1418 (0.0008) | |
| [2023-05-24 20:41:55,705][2740681] Updated weights for policy 0, policy_version 1428 (0.0008) | |
| [2023-05-24 20:41:57,556][2740681] Updated weights for policy 0, policy_version 1438 (0.0009) | |
| [2023-05-24 20:41:58,854][2722668] Fps is (10 sec: 21708.7, 60 sec: 22323.2, 300 sec: 21208.2). Total num frames: 5914624. Throughput: 0: 5573.1. Samples: 473652. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:41:58,855][2722668] Avg episode reward: [(0, '21.784')] | |
| [2023-05-24 20:41:59,430][2740681] Updated weights for policy 0, policy_version 1448 (0.0009) | |
| [2023-05-24 20:42:01,259][2740681] Updated weights for policy 0, policy_version 1458 (0.0008) | |
| [2023-05-24 20:42:03,088][2740681] Updated weights for policy 0, policy_version 1468 (0.0008) | |
| [2023-05-24 20:42:03,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21299.2). Total num frames: 6029312. Throughput: 0: 5574.8. Samples: 490318. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:03,855][2722668] Avg episode reward: [(0, '27.011')] | |
| [2023-05-24 20:42:04,905][2740681] Updated weights for policy 0, policy_version 1478 (0.0009) | |
| [2023-05-24 20:42:06,738][2740681] Updated weights for policy 0, policy_version 1488 (0.0009) | |
| [2023-05-24 20:42:08,562][2740681] Updated weights for policy 0, policy_version 1498 (0.0009) | |
| [2023-05-24 20:42:08,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21340.2). Total num frames: 6139904. Throughput: 0: 5577.9. Samples: 524020. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:08,855][2722668] Avg episode reward: [(0, '26.691')] | |
| [2023-05-24 20:42:10,377][2740681] Updated weights for policy 0, policy_version 1508 (0.0009) | |
| [2023-05-24 20:42:12,210][2740681] Updated weights for policy 0, policy_version 1518 (0.0008) | |
| [2023-05-24 20:42:13,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21377.2). Total num frames: 6250496. Throughput: 0: 5582.2. Samples: 557688. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:13,856][2722668] Avg episode reward: [(0, '28.372')] | |
| [2023-05-24 20:42:14,046][2740681] Updated weights for policy 0, policy_version 1528 (0.0010) | |
| [2023-05-24 20:42:15,895][2740681] Updated weights for policy 0, policy_version 1538 (0.0009) | |
| [2023-05-24 20:42:17,697][2740681] Updated weights for policy 0, policy_version 1548 (0.0008) | |
| [2023-05-24 20:42:18,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22323.1, 300 sec: 21448.1). Total num frames: 6365184. Throughput: 0: 5580.9. Samples: 574382. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:42:18,855][2722668] Avg episode reward: [(0, '28.683')] | |
| [2023-05-24 20:42:18,861][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001554_6365184.pth... | |
| [2023-05-24 20:42:18,904][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000569_2330624.pth | |
| [2023-05-24 20:42:18,910][2740668] Saving new best policy, reward=28.683! | |
| [2023-05-24 20:42:19,545][2740681] Updated weights for policy 0, policy_version 1558 (0.0009) | |
| [2023-05-24 20:42:21,365][2740681] Updated weights for policy 0, policy_version 1568 (0.0008) | |
| [2023-05-24 20:42:23,174][2740681] Updated weights for policy 0, policy_version 1578 (0.0009) | |
| [2023-05-24 20:42:23,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21477.3). Total num frames: 6475776. Throughput: 0: 5583.6. Samples: 608146. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:42:23,855][2722668] Avg episode reward: [(0, '28.246')] | |
| [2023-05-24 20:42:24,992][2740681] Updated weights for policy 0, policy_version 1588 (0.0009) | |
| [2023-05-24 20:42:26,845][2740681] Updated weights for policy 0, policy_version 1598 (0.0009) | |
| [2023-05-24 20:42:28,717][2740681] Updated weights for policy 0, policy_version 1608 (0.0009) | |
| [2023-05-24 20:42:28,854][2722668] Fps is (10 sec: 22118.3, 60 sec: 22323.2, 300 sec: 21504.0). Total num frames: 6586368. Throughput: 0: 5578.0. Samples: 641568. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:42:28,855][2722668] Avg episode reward: [(0, '30.804')] | |
| [2023-05-24 20:42:28,860][2740668] Saving new best policy, reward=30.804! | |
| [2023-05-24 20:42:30,573][2740681] Updated weights for policy 0, policy_version 1618 (0.0008) | |
| [2023-05-24 20:42:32,405][2740681] Updated weights for policy 0, policy_version 1628 (0.0008) | |
| [2023-05-24 20:42:33,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22256.7, 300 sec: 21528.6). Total num frames: 6696960. Throughput: 0: 5570.7. Samples: 658070. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:33,855][2722668] Avg episode reward: [(0, '24.207')] | |
| [2023-05-24 20:42:34,261][2740681] Updated weights for policy 0, policy_version 1638 (0.0008) | |
| [2023-05-24 20:42:36,166][2740681] Updated weights for policy 0, policy_version 1648 (0.0009) | |
| [2023-05-24 20:42:38,015][2740681] Updated weights for policy 0, policy_version 1658 (0.0009) | |
| [2023-05-24 20:42:38,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22254.9, 300 sec: 21551.3). Total num frames: 6807552. Throughput: 0: 5559.1. Samples: 690874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:42:38,855][2722668] Avg episode reward: [(0, '28.196')] | |
| [2023-05-24 20:42:39,844][2740681] Updated weights for policy 0, policy_version 1668 (0.0008) | |
| [2023-05-24 20:42:41,662][2740681] Updated weights for policy 0, policy_version 1678 (0.0008) | |
| [2023-05-24 20:42:43,455][2740681] Updated weights for policy 0, policy_version 1688 (0.0008) | |
| [2023-05-24 20:42:43,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21602.6). Total num frames: 6922240. Throughput: 0: 5578.1. Samples: 724666. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:43,855][2722668] Avg episode reward: [(0, '31.920')] | |
| [2023-05-24 20:42:43,856][2740668] Saving new best policy, reward=31.920! | |
| [2023-05-24 20:42:45,312][2740681] Updated weights for policy 0, policy_version 1698 (0.0009) | |
| [2023-05-24 20:42:47,157][2740681] Updated weights for policy 0, policy_version 1708 (0.0008) | |
| [2023-05-24 20:42:48,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22254.9, 300 sec: 21621.0). Total num frames: 7032832. Throughput: 0: 5577.6. Samples: 741310. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:48,855][2722668] Avg episode reward: [(0, '27.695')] | |
| [2023-05-24 20:42:48,995][2740681] Updated weights for policy 0, policy_version 1718 (0.0008) | |
| [2023-05-24 20:42:50,839][2740681] Updated weights for policy 0, policy_version 1728 (0.0008) | |
| [2023-05-24 20:42:52,669][2740681] Updated weights for policy 0, policy_version 1738 (0.0008) | |
| [2023-05-24 20:42:53,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21638.2). Total num frames: 7143424. Throughput: 0: 5573.2. Samples: 774812. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:42:53,855][2722668] Avg episode reward: [(0, '27.728')] | |
| [2023-05-24 20:42:54,522][2740681] Updated weights for policy 0, policy_version 1748 (0.0009) | |
| [2023-05-24 20:42:56,333][2740681] Updated weights for policy 0, policy_version 1758 (0.0008) | |
| [2023-05-24 20:42:58,208][2740681] Updated weights for policy 0, policy_version 1768 (0.0009) | |
| [2023-05-24 20:42:58,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21654.2). Total num frames: 7254016. Throughput: 0: 5569.2. Samples: 808304. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:42:58,855][2722668] Avg episode reward: [(0, '29.743')] | |
| [2023-05-24 20:43:00,033][2740681] Updated weights for policy 0, policy_version 1778 (0.0009) | |
| [2023-05-24 20:43:01,872][2740681] Updated weights for policy 0, policy_version 1788 (0.0008) | |
| [2023-05-24 20:43:03,729][2740681] Updated weights for policy 0, policy_version 1798 (0.0008) | |
| [2023-05-24 20:43:03,854][2722668] Fps is (10 sec: 22118.3, 60 sec: 22254.9, 300 sec: 21669.2). Total num frames: 7364608. Throughput: 0: 5567.1. Samples: 824900. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:43:03,855][2722668] Avg episode reward: [(0, '28.198')] | |
| [2023-05-24 20:43:05,562][2740681] Updated weights for policy 0, policy_version 1808 (0.0008) | |
| [2023-05-24 20:43:07,388][2740681] Updated weights for policy 0, policy_version 1818 (0.0009) | |
| [2023-05-24 20:43:08,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21708.8). Total num frames: 7479296. Throughput: 0: 5561.0. Samples: 858390. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:43:08,855][2722668] Avg episode reward: [(0, '29.986')] | |
| [2023-05-24 20:43:09,179][2740681] Updated weights for policy 0, policy_version 1828 (0.0009) | |
| [2023-05-24 20:43:10,969][2740681] Updated weights for policy 0, policy_version 1838 (0.0008) | |
| [2023-05-24 20:43:12,795][2740681] Updated weights for policy 0, policy_version 1848 (0.0008) | |
| [2023-05-24 20:43:13,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21721.2). Total num frames: 7589888. Throughput: 0: 5572.1. Samples: 892314. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:43:13,855][2722668] Avg episode reward: [(0, '26.336')] | |
| [2023-05-24 20:43:14,619][2740681] Updated weights for policy 0, policy_version 1858 (0.0008) | |
| [2023-05-24 20:43:16,431][2740681] Updated weights for policy 0, policy_version 1868 (0.0008) | |
| [2023-05-24 20:43:18,272][2740681] Updated weights for policy 0, policy_version 1878 (0.0009) | |
| [2023-05-24 20:43:18,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22323.2, 300 sec: 21757.0). Total num frames: 7704576. Throughput: 0: 5580.9. Samples: 909212. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:43:18,855][2722668] Avg episode reward: [(0, '27.271')] | |
| [2023-05-24 20:43:20,078][2740681] Updated weights for policy 0, policy_version 1888 (0.0008) | |
| [2023-05-24 20:43:21,909][2740681] Updated weights for policy 0, policy_version 1898 (0.0008) | |
| [2023-05-24 20:43:23,753][2740681] Updated weights for policy 0, policy_version 1908 (0.0009) | |
| [2023-05-24 20:43:23,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 22323.3, 300 sec: 21767.3). Total num frames: 7815168. Throughput: 0: 5600.1. Samples: 942878. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:43:23,855][2722668] Avg episode reward: [(0, '29.434')] | |
| [2023-05-24 20:43:25,586][2740681] Updated weights for policy 0, policy_version 1918 (0.0009) | |
| [2023-05-24 20:43:27,395][2740681] Updated weights for policy 0, policy_version 1928 (0.0009) | |
| [2023-05-24 20:43:28,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 22391.5, 300 sec: 21799.8). Total num frames: 7929856. Throughput: 0: 5597.6. Samples: 976560. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:43:28,855][2722668] Avg episode reward: [(0, '29.442')] | |
| [2023-05-24 20:43:29,195][2740681] Updated weights for policy 0, policy_version 1938 (0.0008) | |
| [2023-05-24 20:43:31,023][2740681] Updated weights for policy 0, policy_version 1948 (0.0009) | |
| [2023-05-24 20:43:32,827][2740681] Updated weights for policy 0, policy_version 1958 (0.0009) | |
| [2023-05-24 20:43:33,854][2722668] Fps is (10 sec: 22527.6, 60 sec: 22391.5, 300 sec: 21808.4). Total num frames: 8040448. Throughput: 0: 5604.0. Samples: 993490. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:43:33,855][2722668] Avg episode reward: [(0, '27.952')] | |
| [2023-05-24 20:43:34,670][2740681] Updated weights for policy 0, policy_version 1968 (0.0008) | |
| [2023-05-24 20:43:36,467][2740681] Updated weights for policy 0, policy_version 1978 (0.0008) | |
| [2023-05-24 20:43:38,244][2740681] Updated weights for policy 0, policy_version 1988 (0.0008) | |
| [2023-05-24 20:43:38,854][2722668] Fps is (10 sec: 22527.8, 60 sec: 22459.7, 300 sec: 21838.1). Total num frames: 8155136. Throughput: 0: 5611.8. Samples: 1027342. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:43:38,855][2722668] Avg episode reward: [(0, '28.687')] | |
| [2023-05-24 20:43:40,098][2740681] Updated weights for policy 0, policy_version 1998 (0.0010) | |
| [2023-05-24 20:43:41,961][2740681] Updated weights for policy 0, policy_version 2008 (0.0009) | |
| [2023-05-24 20:43:43,806][2740681] Updated weights for policy 0, policy_version 2018 (0.0009) | |
| [2023-05-24 20:43:43,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 22391.5, 300 sec: 21845.3). Total num frames: 8265728. Throughput: 0: 5605.6. Samples: 1060556. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-05-24 20:43:43,855][2722668] Avg episode reward: [(0, '26.381')] | |
| [2023-05-24 20:43:45,628][2740681] Updated weights for policy 0, policy_version 2028 (0.0008) | |
| [2023-05-24 20:43:47,479][2740681] Updated weights for policy 0, policy_version 2038 (0.0010) | |
| [2023-05-24 20:43:48,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22391.5, 300 sec: 21852.2). Total num frames: 8376320. Throughput: 0: 5609.4. Samples: 1077324. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:43:48,855][2722668] Avg episode reward: [(0, '28.460')] | |
| [2023-05-24 20:43:49,334][2740681] Updated weights for policy 0, policy_version 2048 (0.0008) | |
| [2023-05-24 20:43:51,158][2740681] Updated weights for policy 0, policy_version 2058 (0.0009) | |
| [2023-05-24 20:43:52,989][2740681] Updated weights for policy 0, policy_version 2068 (0.0008) | |
| [2023-05-24 20:43:53,854][2722668] Fps is (10 sec: 22118.2, 60 sec: 22391.5, 300 sec: 21858.7). Total num frames: 8486912. Throughput: 0: 5608.4. Samples: 1110768. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:43:53,855][2722668] Avg episode reward: [(0, '26.161')] | |
| [2023-05-24 20:43:54,818][2740681] Updated weights for policy 0, policy_version 2078 (0.0008) | |
| [2023-05-24 20:43:56,656][2740681] Updated weights for policy 0, policy_version 2088 (0.0009) | |
| [2023-05-24 20:43:58,457][2740681] Updated weights for policy 0, policy_version 2098 (0.0009) | |
| [2023-05-24 20:43:58,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22459.7, 300 sec: 21884.3). Total num frames: 8601600. Throughput: 0: 5601.3. Samples: 1144374. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:43:58,855][2722668] Avg episode reward: [(0, '26.754')] | |
| [2023-05-24 20:44:00,317][2740681] Updated weights for policy 0, policy_version 2108 (0.0008) | |
| [2023-05-24 20:44:02,200][2740681] Updated weights for policy 0, policy_version 2118 (0.0008) | |
| [2023-05-24 20:44:03,854][2722668] Fps is (10 sec: 22118.2, 60 sec: 22391.5, 300 sec: 21870.7). Total num frames: 8708096. Throughput: 0: 5596.8. Samples: 1161066. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:44:03,855][2722668] Avg episode reward: [(0, '28.003')] | |
| [2023-05-24 20:44:04,042][2740681] Updated weights for policy 0, policy_version 2128 (0.0009) | |
| [2023-05-24 20:44:05,869][2740681] Updated weights for policy 0, policy_version 2138 (0.0008) | |
| [2023-05-24 20:44:07,696][2740681] Updated weights for policy 0, policy_version 2148 (0.0008) | |
| [2023-05-24 20:44:08,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22391.5, 300 sec: 21895.0). Total num frames: 8822784. Throughput: 0: 5596.4. Samples: 1194718. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-05-24 20:44:08,855][2722668] Avg episode reward: [(0, '28.981')] | |
| [2023-05-24 20:44:09,544][2740681] Updated weights for policy 0, policy_version 2158 (0.0008) | |
| [2023-05-24 20:44:11,378][2740681] Updated weights for policy 0, policy_version 2168 (0.0008) | |
| [2023-05-24 20:44:13,204][2740681] Updated weights for policy 0, policy_version 2178 (0.0009) | |
| [2023-05-24 20:44:13,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 22391.5, 300 sec: 21899.9). Total num frames: 8933376. Throughput: 0: 5586.7. Samples: 1227960. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:44:13,855][2722668] Avg episode reward: [(0, '31.399')] | |
| [2023-05-24 20:44:15,008][2740681] Updated weights for policy 0, policy_version 2188 (0.0008) | |
| [2023-05-24 20:44:16,831][2740681] Updated weights for policy 0, policy_version 2198 (0.0008) | |
| [2023-05-24 20:44:18,630][2740681] Updated weights for policy 0, policy_version 2208 (0.0008) | |
| [2023-05-24 20:44:18,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21922.5). Total num frames: 9048064. Throughput: 0: 5588.3. Samples: 1244962. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-05-24 20:44:18,855][2722668] Avg episode reward: [(0, '29.507')] | |
| [2023-05-24 20:44:18,860][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002209_9048064.pth... | |
| [2023-05-24 20:44:18,903][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth | |
| [2023-05-24 20:44:20,450][2740681] Updated weights for policy 0, policy_version 2218 (0.0008) | |
| [2023-05-24 20:44:22,295][2740681] Updated weights for policy 0, policy_version 2228 (0.0009) | |
| [2023-05-24 20:44:23,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22391.4, 300 sec: 21926.7). Total num frames: 9158656. Throughput: 0: 5588.7. Samples: 1278834. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-05-24 20:44:23,855][2722668] Avg episode reward: [(0, '32.078')] | |
| [2023-05-24 20:44:23,857][2740668] Saving new best policy, reward=32.078! | |
| [2023-05-24 20:44:24,099][2740681] Updated weights for policy 0, policy_version 2238 (0.0009) | |
| [2023-05-24 20:44:25,919][2740681] Updated weights for policy 0, policy_version 2248 (0.0009) | |
| [2023-05-24 20:44:27,741][2740681] Updated weights for policy 0, policy_version 2258 (0.0008) | |
| [2023-05-24 20:44:28,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22391.5, 300 sec: 21947.7). Total num frames: 9273344. Throughput: 0: 5597.3. Samples: 1312434. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-05-24 20:44:28,855][2722668] Avg episode reward: [(0, '27.981')] | |
| [2023-05-24 20:44:29,555][2740681] Updated weights for policy 0, policy_version 2268 (0.0009) | |
| [2023-05-24 20:44:31,380][2740681] Updated weights for policy 0, policy_version 2278 (0.0009) | |
| [2023-05-24 20:44:33,224][2740681] Updated weights for policy 0, policy_version 2288 (0.0010) | |
| [2023-05-24 20:44:33,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22391.5, 300 sec: 21951.2). Total num frames: 9383936. Throughput: 0: 5598.3. Samples: 1329246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-05-24 20:44:33,855][2722668] Avg episode reward: [(0, '29.901')] | |
| [2023-05-24 20:44:35,056][2740681] Updated weights for policy 0, policy_version 2298 (0.0010) | |
| [2023-05-24 20:44:36,882][2740681] Updated weights for policy 0, policy_version 2308 (0.0009) | |
| [2023-05-24 20:44:38,786][2740681] Updated weights for policy 0, policy_version 2318 (0.0009) | |
| [2023-05-24 20:44:38,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22323.2, 300 sec: 21954.6). Total num frames: 9494528. Throughput: 0: 5599.2. Samples: 1362734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:44:38,855][2722668] Avg episode reward: [(0, '30.109')] | |
| [2023-05-24 20:44:40,619][2740681] Updated weights for policy 0, policy_version 2328 (0.0009) | |
| [2023-05-24 20:44:42,459][2740681] Updated weights for policy 0, policy_version 2338 (0.0009) | |
| [2023-05-24 20:44:43,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.1, 300 sec: 21957.8). Total num frames: 9605120. Throughput: 0: 5586.2. Samples: 1395754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:44:43,855][2722668] Avg episode reward: [(0, '30.028')] | |
| [2023-05-24 20:44:44,395][2740681] Updated weights for policy 0, policy_version 2348 (0.0008) | |
| [2023-05-24 20:44:46,339][2740681] Updated weights for policy 0, policy_version 2358 (0.0009) | |
| [2023-05-24 20:44:48,184][2740681] Updated weights for policy 0, policy_version 2368 (0.0009) | |
| [2023-05-24 20:44:48,854][2722668] Fps is (10 sec: 21708.7, 60 sec: 22254.9, 300 sec: 21945.1). Total num frames: 9711616. Throughput: 0: 5571.5. Samples: 1411784. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-05-24 20:44:48,856][2722668] Avg episode reward: [(0, '29.331')] | |
| [2023-05-24 20:44:50,046][2740681] Updated weights for policy 0, policy_version 2378 (0.0009) | |
| [2023-05-24 20:44:51,883][2740681] Updated weights for policy 0, policy_version 2388 (0.0009) | |
| [2023-05-24 20:44:53,680][2740681] Updated weights for policy 0, policy_version 2398 (0.0009) | |
| [2023-05-24 20:44:53,854][2722668] Fps is (10 sec: 21708.9, 60 sec: 22254.9, 300 sec: 21948.4). Total num frames: 9822208. Throughput: 0: 5557.7. Samples: 1444814. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-05-24 20:44:53,855][2722668] Avg episode reward: [(0, '27.804')] | |
| [2023-05-24 20:44:55,523][2740681] Updated weights for policy 0, policy_version 2408 (0.0008) | |
| [2023-05-24 20:44:57,342][2740681] Updated weights for policy 0, policy_version 2418 (0.0008) | |
| [2023-05-24 20:44:58,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22186.7, 300 sec: 21951.5). Total num frames: 9932800. Throughput: 0: 5563.8. Samples: 1478332. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-05-24 20:44:58,855][2722668] Avg episode reward: [(0, '29.302')] | |
| [2023-05-24 20:44:59,233][2740681] Updated weights for policy 0, policy_version 2428 (0.0010) | |
| [2023-05-24 20:45:01,067][2740681] Updated weights for policy 0, policy_version 2438 (0.0008) | |
| [2023-05-24 20:45:01,991][2740668] Stopping Batcher_0... | |
| [2023-05-24 20:45:01,991][2740668] Loop batcher_evt_loop terminating... | |
| [2023-05-24 20:45:01,991][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth... | |
| [2023-05-24 20:45:01,998][2722668] Component Batcher_0 stopped! | |
| [2023-05-24 20:45:02,003][2740684] Stopping RolloutWorker_w3... | |
| [2023-05-24 20:45:02,004][2740684] Loop rollout_proc3_evt_loop terminating... | |
| [2023-05-24 20:45:02,003][2722668] Component RolloutWorker_w3 stopped! | |
| [2023-05-24 20:45:02,004][2740691] Stopping RolloutWorker_w7... | |
| [2023-05-24 20:45:02,004][2740686] Stopping RolloutWorker_w2... | |
| [2023-05-24 20:45:02,005][2740691] Loop rollout_proc7_evt_loop terminating... | |
| [2023-05-24 20:45:02,004][2740685] Stopping RolloutWorker_w1... | |
| [2023-05-24 20:45:02,004][2740692] Stopping RolloutWorker_w6... | |
| [2023-05-24 20:45:02,005][2740682] Stopping RolloutWorker_w0... | |
| [2023-05-24 20:45:02,005][2740687] Stopping RolloutWorker_w4... | |
| [2023-05-24 20:45:02,005][2740686] Loop rollout_proc2_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2740685] Loop rollout_proc1_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2740692] Loop rollout_proc6_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2740690] Stopping RolloutWorker_w5... | |
| [2023-05-24 20:45:02,005][2740687] Loop rollout_proc4_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2740682] Loop rollout_proc0_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2740690] Loop rollout_proc5_evt_loop terminating... | |
| [2023-05-24 20:45:02,005][2722668] Component RolloutWorker_w7 stopped! | |
| [2023-05-24 20:45:02,006][2740681] Weights refcount: 2 0 | |
| [2023-05-24 20:45:02,006][2722668] Component RolloutWorker_w2 stopped! | |
| [2023-05-24 20:45:02,007][2740681] Stopping InferenceWorker_p0-w0... | |
| [2023-05-24 20:45:02,008][2740681] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-05-24 20:45:02,007][2722668] Component RolloutWorker_w6 stopped! | |
| [2023-05-24 20:45:02,008][2722668] Component RolloutWorker_w1 stopped! | |
| [2023-05-24 20:45:02,009][2722668] Component RolloutWorker_w0 stopped! | |
| [2023-05-24 20:45:02,010][2722668] Component RolloutWorker_w4 stopped! | |
| [2023-05-24 20:45:02,011][2722668] Component RolloutWorker_w5 stopped! | |
| [2023-05-24 20:45:02,012][2722668] Component InferenceWorker_p0-w0 stopped! | |
| [2023-05-24 20:45:02,036][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001554_6365184.pth | |
| [2023-05-24 20:45:02,042][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth... | |
| [2023-05-24 20:45:02,095][2740668] Stopping LearnerWorker_p0... | |
| [2023-05-24 20:45:02,096][2740668] Loop learner_proc0_evt_loop terminating... | |
| [2023-05-24 20:45:02,095][2722668] Component LearnerWorker_p0 stopped! | |
| [2023-05-24 20:45:02,097][2722668] Waiting for process learner_proc0 to stop... | |
| [2023-05-24 20:45:02,894][2722668] Waiting for process inference_proc0-0 to join... | |
| [2023-05-24 20:45:02,896][2722668] Waiting for process rollout_proc0 to join... | |
| [2023-05-24 20:45:02,898][2722668] Waiting for process rollout_proc1 to join... | |
| [2023-05-24 20:45:02,899][2722668] Waiting for process rollout_proc2 to join... | |
| [2023-05-24 20:45:02,901][2722668] Waiting for process rollout_proc3 to join... | |
| [2023-05-24 20:45:02,902][2722668] Waiting for process rollout_proc4 to join... | |
| [2023-05-24 20:45:02,903][2722668] Waiting for process rollout_proc5 to join... | |
| [2023-05-24 20:45:02,904][2722668] Waiting for process rollout_proc6 to join... | |
| [2023-05-24 20:45:02,904][2722668] Waiting for process rollout_proc7 to join... | |
| [2023-05-24 20:45:02,905][2722668] Batcher 0 profile tree view: | |
| batching: 13.3147, releasing_batches: 0.0342 | |
| [2023-05-24 20:45:02,906][2722668] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0001 | |
| wait_policy_total: 6.5332 | |
| update_model: 4.1118 | |
| weight_update: 0.0009 | |
| one_step: 0.0019 | |
| handle_policy_step: 246.0819 | |
| deserialize: 9.9033, stack: 1.4958, obs_to_device_normalize: 61.1206, forward: 106.2121, send_messages: 16.6256 | |
| prepare_outputs: 39.1913 | |
| to_cpu: 25.6796 | |
| [2023-05-24 20:45:02,907][2722668] Learner 0 profile tree view: | |
| misc: 0.0066, prepare_batch: 11.8106 | |
| train: 39.8102 | |
| epoch_init: 0.0084, minibatch_init: 0.0091, losses_postprocess: 0.3513, kl_divergence: 0.3257, after_optimizer: 0.5826 | |
| calculate_losses: 11.6125 | |
| losses_init: 0.0055, forward_head: 1.1278, bptt_initial: 6.9832, tail: 0.6039, advantages_returns: 0.1700, losses: 1.2730 | |
| bptt: 1.2362 | |
| bptt_forward_core: 1.1854 | |
| update: 26.4486 | |
| clip: 1.7273 | |
| [2023-05-24 20:45:02,908][2722668] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.2403, enqueue_policy_requests: 11.0413, env_step: 176.9094, overhead: 13.4279, complete_rollouts: 0.3495 | |
| save_policy_outputs: 13.2120 | |
| split_output_tensors: 6.4741 | |
| [2023-05-24 20:45:02,909][2722668] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.2375, enqueue_policy_requests: 11.0937, env_step: 177.0302, overhead: 13.4463, complete_rollouts: 0.3416 | |
| save_policy_outputs: 13.1739 | |
| split_output_tensors: 6.4360 | |
| [2023-05-24 20:45:02,910][2722668] Loop Runner_EvtLoop terminating... | |
| [2023-05-24 20:45:02,911][2722668] Runner profile tree view: | |
| main_loop: 282.2049 | |
| [2023-05-24 20:45:02,912][2722668] Collected {0: 10006528}, FPS: 21263.4 | |
| [2023-05-24 20:45:02,990][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json | |
| [2023-05-24 20:45:02,991][2722668] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-05-24 20:45:02,992][2722668] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-05-24 20:45:02,993][2722668] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-05-24 20:45:02,994][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:45:02,995][2722668] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-05-24 20:45:02,996][2722668] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:45:02,997][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-05-24 20:45:02,997][2722668] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2023-05-24 20:45:02,998][2722668] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2023-05-24 20:45:02,999][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-05-24 20:45:03,000][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-05-24 20:45:03,002][2722668] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-05-24 20:45:03,003][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-05-24 20:45:03,004][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-05-24 20:45:03,009][2722668] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:45:03,011][2722668] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:45:03,027][2722668] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:45:03,078][2722668] Conv encoder output size: 512 | |
| [2023-05-24 20:45:03,079][2722668] Policy head output size: 512 | |
| [2023-05-24 20:45:03,110][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth... | |
| [2023-05-24 20:45:03,949][2722668] Num frames 100... | |
| [2023-05-24 20:45:04,111][2722668] Num frames 200... | |
| [2023-05-24 20:45:04,278][2722668] Num frames 300... | |
| [2023-05-24 20:45:04,443][2722668] Num frames 400... | |
| [2023-05-24 20:45:04,602][2722668] Num frames 500... | |
| [2023-05-24 20:45:04,761][2722668] Num frames 600... | |
| [2023-05-24 20:45:04,831][2722668] Avg episode rewards: #0: 11.080, true rewards: #0: 6.080 | |
| [2023-05-24 20:45:04,833][2722668] Avg episode reward: 11.080, avg true_objective: 6.080 | |
| [2023-05-24 20:45:04,986][2722668] Num frames 700... | |
| [2023-05-24 20:45:05,145][2722668] Num frames 800... | |
| [2023-05-24 20:45:05,309][2722668] Num frames 900... | |
| [2023-05-24 20:45:05,472][2722668] Num frames 1000... | |
| [2023-05-24 20:45:05,642][2722668] Num frames 1100... | |
| [2023-05-24 20:45:05,781][2722668] Avg episode rewards: #0: 9.260, true rewards: #0: 5.760 | |
| [2023-05-24 20:45:05,783][2722668] Avg episode reward: 9.260, avg true_objective: 5.760 | |
| [2023-05-24 20:45:05,862][2722668] Num frames 1200... | |
| [2023-05-24 20:45:06,020][2722668] Num frames 1300... | |
| [2023-05-24 20:45:06,184][2722668] Num frames 1400... | |
| [2023-05-24 20:45:06,346][2722668] Num frames 1500... | |
| [2023-05-24 20:45:06,502][2722668] Num frames 1600... | |
| [2023-05-24 20:45:06,661][2722668] Num frames 1700... | |
| [2023-05-24 20:45:06,820][2722668] Num frames 1800... | |
| [2023-05-24 20:45:06,982][2722668] Num frames 1900... | |
| [2023-05-24 20:45:07,143][2722668] Num frames 2000... | |
| [2023-05-24 20:45:07,302][2722668] Num frames 2100... | |
| [2023-05-24 20:45:07,464][2722668] Num frames 2200... | |
| [2023-05-24 20:45:07,626][2722668] Num frames 2300... | |
| [2023-05-24 20:45:07,780][2722668] Num frames 2400... | |
| [2023-05-24 20:45:07,948][2722668] Num frames 2500... | |
| [2023-05-24 20:45:08,091][2722668] Num frames 2600... | |
| [2023-05-24 20:45:08,249][2722668] Num frames 2700... | |
| [2023-05-24 20:45:08,407][2722668] Num frames 2800... | |
| [2023-05-24 20:45:08,566][2722668] Num frames 2900... | |
| [2023-05-24 20:45:08,723][2722668] Num frames 3000... | |
| [2023-05-24 20:45:08,876][2722668] Num frames 3100... | |
| [2023-05-24 20:45:09,008][2722668] Num frames 3200... | |
| [2023-05-24 20:45:09,146][2722668] Avg episode rewards: #0: 25.173, true rewards: #0: 10.840 | |
| [2023-05-24 20:45:09,148][2722668] Avg episode reward: 25.173, avg true_objective: 10.840 | |
| [2023-05-24 20:45:09,227][2722668] Num frames 3300... | |
| [2023-05-24 20:45:09,385][2722668] Num frames 3400... | |
| [2023-05-24 20:45:09,556][2722668] Num frames 3500... | |
| [2023-05-24 20:45:09,716][2722668] Num frames 3600... | |
| [2023-05-24 20:45:09,874][2722668] Num frames 3700... | |
| [2023-05-24 20:45:10,029][2722668] Num frames 3800... | |
| [2023-05-24 20:45:10,193][2722668] Num frames 3900... | |
| [2023-05-24 20:45:10,343][2722668] Num frames 4000... | |
| [2023-05-24 20:45:10,425][2722668] Avg episode rewards: #0: 23.547, true rewards: #0: 10.048 | |
| [2023-05-24 20:45:10,426][2722668] Avg episode reward: 23.547, avg true_objective: 10.048 | |
| [2023-05-24 20:45:10,540][2722668] Num frames 4100... | |
| [2023-05-24 20:45:10,691][2722668] Num frames 4200... | |
| [2023-05-24 20:45:10,847][2722668] Num frames 4300... | |
| [2023-05-24 20:45:10,985][2722668] Num frames 4400... | |
| [2023-05-24 20:45:11,128][2722668] Num frames 4500... | |
| [2023-05-24 20:45:11,287][2722668] Num frames 4600... | |
| [2023-05-24 20:45:11,425][2722668] Num frames 4700... | |
| [2023-05-24 20:45:11,584][2722668] Num frames 4800... | |
| [2023-05-24 20:45:11,731][2722668] Num frames 4900... | |
| [2023-05-24 20:45:11,885][2722668] Num frames 5000... | |
| [2023-05-24 20:45:12,052][2722668] Num frames 5100... | |
| [2023-05-24 20:45:12,222][2722668] Num frames 5200... | |
| [2023-05-24 20:45:12,382][2722668] Num frames 5300... | |
| [2023-05-24 20:45:12,540][2722668] Num frames 5400... | |
| [2023-05-24 20:45:12,701][2722668] Num frames 5500... | |
| [2023-05-24 20:45:12,866][2722668] Num frames 5600... | |
| [2023-05-24 20:45:13,045][2722668] Num frames 5700... | |
| [2023-05-24 20:45:13,208][2722668] Num frames 5800... | |
| [2023-05-24 20:45:13,370][2722668] Num frames 5900... | |
| [2023-05-24 20:45:13,528][2722668] Num frames 6000... | |
| [2023-05-24 20:45:13,697][2722668] Num frames 6100... | |
| [2023-05-24 20:45:13,787][2722668] Avg episode rewards: #0: 30.038, true rewards: #0: 12.238 | |
| [2023-05-24 20:45:13,788][2722668] Avg episode reward: 30.038, avg true_objective: 12.238 | |
| [2023-05-24 20:45:13,924][2722668] Num frames 6200... | |
| [2023-05-24 20:45:14,079][2722668] Num frames 6300... | |
| [2023-05-24 20:45:14,241][2722668] Num frames 6400... | |
| [2023-05-24 20:45:14,398][2722668] Num frames 6500... | |
| [2023-05-24 20:45:14,558][2722668] Num frames 6600... | |
| [2023-05-24 20:45:14,737][2722668] Num frames 6700... | |
| [2023-05-24 20:45:14,882][2722668] Num frames 6800... | |
| [2023-05-24 20:45:15,043][2722668] Num frames 6900... | |
| [2023-05-24 20:45:15,204][2722668] Num frames 7000... | |
| [2023-05-24 20:45:15,364][2722668] Num frames 7100... | |
| [2023-05-24 20:45:15,522][2722668] Num frames 7200... | |
| [2023-05-24 20:45:15,688][2722668] Num frames 7300... | |
| [2023-05-24 20:45:15,846][2722668] Num frames 7400... | |
| [2023-05-24 20:45:15,909][2722668] Avg episode rewards: #0: 30.005, true rewards: #0: 12.338 | |
| [2023-05-24 20:45:15,911][2722668] Avg episode reward: 30.005, avg true_objective: 12.338 | |
| [2023-05-24 20:45:16,075][2722668] Num frames 7500... | |
| [2023-05-24 20:45:16,224][2722668] Num frames 7600... | |
| [2023-05-24 20:45:16,381][2722668] Num frames 7700... | |
| [2023-05-24 20:45:16,534][2722668] Num frames 7800... | |
| [2023-05-24 20:45:16,695][2722668] Num frames 7900... | |
| [2023-05-24 20:45:16,853][2722668] Num frames 8000... | |
| [2023-05-24 20:45:17,011][2722668] Num frames 8100... | |
| [2023-05-24 20:45:17,174][2722668] Num frames 8200... | |
| [2023-05-24 20:45:17,343][2722668] Num frames 8300... | |
| [2023-05-24 20:45:17,509][2722668] Num frames 8400... | |
| [2023-05-24 20:45:17,683][2722668] Num frames 8500... | |
| [2023-05-24 20:45:17,851][2722668] Num frames 8600... | |
| [2023-05-24 20:45:18,023][2722668] Num frames 8700... | |
| [2023-05-24 20:45:18,188][2722668] Num frames 8800... | |
| [2023-05-24 20:45:18,359][2722668] Num frames 8900... | |
| [2023-05-24 20:45:18,550][2722668] Avg episode rewards: #0: 31.680, true rewards: #0: 12.823 | |
| [2023-05-24 20:45:18,552][2722668] Avg episode reward: 31.680, avg true_objective: 12.823 | |
| [2023-05-24 20:45:18,597][2722668] Num frames 9000... | |
| [2023-05-24 20:45:18,755][2722668] Num frames 9100... | |
| [2023-05-24 20:45:18,914][2722668] Num frames 9200... | |
| [2023-05-24 20:45:19,078][2722668] Num frames 9300... | |
| [2023-05-24 20:45:19,182][2722668] Avg episode rewards: #0: 28.535, true rewards: #0: 11.660 | |
| [2023-05-24 20:45:19,184][2722668] Avg episode reward: 28.535, avg true_objective: 11.660 | |
| [2023-05-24 20:45:19,306][2722668] Num frames 9400... | |
| [2023-05-24 20:45:19,469][2722668] Num frames 9500... | |
| [2023-05-24 20:45:19,667][2722668] Avg episode rewards: #0: 25.649, true rewards: #0: 10.649 | |
| [2023-05-24 20:45:19,669][2722668] Avg episode reward: 25.649, avg true_objective: 10.649 | |
| [2023-05-24 20:45:19,701][2722668] Num frames 9600... | |
| [2023-05-24 20:45:19,863][2722668] Num frames 9700... | |
| [2023-05-24 20:45:20,026][2722668] Num frames 9800... | |
| [2023-05-24 20:45:20,185][2722668] Num frames 9900... | |
| [2023-05-24 20:45:20,338][2722668] Num frames 10000... | |
| [2023-05-24 20:45:20,502][2722668] Num frames 10100... | |
| [2023-05-24 20:45:20,661][2722668] Num frames 10200... | |
| [2023-05-24 20:45:20,831][2722668] Num frames 10300... | |
| [2023-05-24 20:45:21,032][2722668] Avg episode rewards: #0: 24.484, true rewards: #0: 10.384 | |
| [2023-05-24 20:45:21,034][2722668] Avg episode reward: 24.484, avg true_objective: 10.384 | |
| [2023-05-24 20:45:46,594][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-05-24 20:45:49,050][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json | |
| [2023-05-24 20:45:49,051][2722668] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-05-24 20:45:49,052][2722668] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-05-24 20:45:49,053][2722668] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-05-24 20:45:49,055][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-05-24 20:45:49,055][2722668] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-05-24 20:45:49,056][2722668] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-05-24 20:45:49,057][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-05-24 20:45:49,057][2722668] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-05-24 20:45:49,058][2722668] Adding new argument 'hf_repository'='markeidsaune/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-05-24 20:45:49,059][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-05-24 20:45:49,059][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-05-24 20:45:49,060][2722668] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-05-24 20:45:49,060][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-05-24 20:45:49,061][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-05-24 20:45:49,074][2722668] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-05-24 20:45:49,075][2722668] RunningMeanStd input shape: (1,) | |
| [2023-05-24 20:45:49,091][2722668] ConvEncoder: input_channels=3 | |
| [2023-05-24 20:45:49,146][2722668] Conv encoder output size: 512 | |
| [2023-05-24 20:45:49,147][2722668] Policy head output size: 512 | |
| [2023-05-24 20:45:49,189][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth... | |
| [2023-05-24 20:45:50,034][2722668] Num frames 100... | |
| [2023-05-24 20:45:50,198][2722668] Num frames 200... | |
| [2023-05-24 20:45:50,358][2722668] Num frames 300... | |
| [2023-05-24 20:45:50,518][2722668] Num frames 400... | |
| [2023-05-24 20:45:50,677][2722668] Num frames 500... | |
| [2023-05-24 20:45:50,848][2722668] Num frames 600... | |
| [2023-05-24 20:45:51,020][2722668] Num frames 700... | |
| [2023-05-24 20:45:51,193][2722668] Num frames 800... | |
| [2023-05-24 20:45:51,355][2722668] Num frames 900... | |
| [2023-05-24 20:45:51,436][2722668] Avg episode rewards: #0: 18.150, true rewards: #0: 9.150 | |
| [2023-05-24 20:45:51,437][2722668] Avg episode reward: 18.150, avg true_objective: 9.150 | |
| [2023-05-24 20:45:51,578][2722668] Num frames 1000... | |
| [2023-05-24 20:45:51,736][2722668] Num frames 1100... | |
| [2023-05-24 20:45:51,896][2722668] Num frames 1200... | |
| [2023-05-24 20:45:52,055][2722668] Num frames 1300... | |
| [2023-05-24 20:45:52,213][2722668] Num frames 1400... | |
| [2023-05-24 20:45:52,362][2722668] Avg episode rewards: #0: 12.795, true rewards: #0: 7.295 | |
| [2023-05-24 20:45:52,364][2722668] Avg episode reward: 12.795, avg true_objective: 7.295 | |
| [2023-05-24 20:45:52,433][2722668] Num frames 1500... | |
| [2023-05-24 20:45:52,589][2722668] Num frames 1600... | |
| [2023-05-24 20:45:52,746][2722668] Num frames 1700... | |
| [2023-05-24 20:45:52,901][2722668] Num frames 1800... | |
| [2023-05-24 20:45:53,064][2722668] Num frames 1900... | |
| [2023-05-24 20:45:53,223][2722668] Num frames 2000... | |
| [2023-05-24 20:45:53,382][2722668] Num frames 2100... | |
| [2023-05-24 20:45:53,542][2722668] Num frames 2200... | |
| [2023-05-24 20:45:53,704][2722668] Num frames 2300... | |
| [2023-05-24 20:45:53,868][2722668] Num frames 2400... | |
| [2023-05-24 20:45:54,030][2722668] Num frames 2500... | |
| [2023-05-24 20:45:54,193][2722668] Num frames 2600... | |
| [2023-05-24 20:45:54,356][2722668] Num frames 2700... | |
| [2023-05-24 20:45:54,523][2722668] Num frames 2800... | |
| [2023-05-24 20:45:54,687][2722668] Num frames 2900... | |
| [2023-05-24 20:45:54,851][2722668] Num frames 3000... | |
| [2023-05-24 20:45:55,017][2722668] Num frames 3100... | |
| [2023-05-24 20:45:55,187][2722668] Num frames 3200... | |
| [2023-05-24 20:45:55,357][2722668] Num frames 3300... | |
| [2023-05-24 20:45:55,527][2722668] Num frames 3400... | |
| [2023-05-24 20:45:55,705][2722668] Num frames 3500... | |
| [2023-05-24 20:45:55,857][2722668] Avg episode rewards: #0: 27.196, true rewards: #0: 11.863 | |
| [2023-05-24 20:45:55,859][2722668] Avg episode reward: 27.196, avg true_objective: 11.863 | |
| [2023-05-24 20:45:55,930][2722668] Num frames 3600... | |
| [2023-05-24 20:45:56,085][2722668] Num frames 3700... | |
| [2023-05-24 20:45:56,248][2722668] Num frames 3800... | |
| [2023-05-24 20:45:56,408][2722668] Num frames 3900... | |
| [2023-05-24 20:45:56,569][2722668] Num frames 4000... | |
| [2023-05-24 20:45:56,731][2722668] Num frames 4100... | |
| [2023-05-24 20:45:56,918][2722668] Avg episode rewards: #0: 23.455, true rewards: #0: 10.455 | |
| [2023-05-24 20:45:56,920][2722668] Avg episode reward: 23.455, avg true_objective: 10.455 | |
| [2023-05-24 20:45:56,953][2722668] Num frames 4200... | |
| [2023-05-24 20:45:57,115][2722668] Num frames 4300... | |
| [2023-05-24 20:45:57,275][2722668] Num frames 4400... | |
| [2023-05-24 20:45:57,434][2722668] Num frames 4500... | |
| [2023-05-24 20:45:57,587][2722668] Num frames 4600... | |
| [2023-05-24 20:45:57,742][2722668] Num frames 4700... | |
| [2023-05-24 20:45:57,901][2722668] Num frames 4800... | |
| [2023-05-24 20:45:58,082][2722668] Avg episode rewards: #0: 21.556, true rewards: #0: 9.756 | |
| [2023-05-24 20:45:58,084][2722668] Avg episode reward: 21.556, avg true_objective: 9.756 | |
| [2023-05-24 20:45:58,124][2722668] Num frames 4900... | |
| [2023-05-24 20:45:58,280][2722668] Num frames 5000... | |
| [2023-05-24 20:45:58,441][2722668] Num frames 5100... | |
| [2023-05-24 20:45:58,593][2722668] Num frames 5200... | |
| [2023-05-24 20:45:58,763][2722668] Num frames 5300... | |
| [2023-05-24 20:45:58,924][2722668] Num frames 5400... | |
| [2023-05-24 20:45:59,080][2722668] Num frames 5500... | |
| [2023-05-24 20:45:59,247][2722668] Num frames 5600... | |
| [2023-05-24 20:45:59,412][2722668] Num frames 5700... | |
| [2023-05-24 20:45:59,570][2722668] Num frames 5800... | |
| [2023-05-24 20:45:59,734][2722668] Num frames 5900... | |
| [2023-05-24 20:45:59,794][2722668] Avg episode rewards: #0: 21.503, true rewards: #0: 9.837 | |
| [2023-05-24 20:45:59,796][2722668] Avg episode reward: 21.503, avg true_objective: 9.837 | |
| [2023-05-24 20:45:59,956][2722668] Num frames 6000... | |
| [2023-05-24 20:46:00,109][2722668] Num frames 6100... | |
| [2023-05-24 20:46:00,269][2722668] Num frames 6200... | |
| [2023-05-24 20:46:00,431][2722668] Num frames 6300... | |
| [2023-05-24 20:46:00,600][2722668] Num frames 6400... | |
| [2023-05-24 20:46:00,775][2722668] Num frames 6500... | |
| [2023-05-24 20:46:00,943][2722668] Num frames 6600... | |
| [2023-05-24 20:46:01,108][2722668] Num frames 6700... | |
| [2023-05-24 20:46:01,269][2722668] Num frames 6800... | |
| [2023-05-24 20:46:01,435][2722668] Num frames 6900... | |
| [2023-05-24 20:46:01,604][2722668] Num frames 7000... | |
| [2023-05-24 20:46:01,760][2722668] Num frames 7100... | |
| [2023-05-24 20:46:01,918][2722668] Num frames 7200... | |
| [2023-05-24 20:46:02,075][2722668] Num frames 7300... | |
| [2023-05-24 20:46:02,238][2722668] Num frames 7400... | |
| [2023-05-24 20:46:02,404][2722668] Num frames 7500... | |
| [2023-05-24 20:46:02,562][2722668] Num frames 7600... | |
| [2023-05-24 20:46:02,701][2722668] Num frames 7700... | |
| [2023-05-24 20:46:02,838][2722668] Num frames 7800... | |
| [2023-05-24 20:46:02,978][2722668] Num frames 7900... | |
| [2023-05-24 20:46:03,113][2722668] Num frames 8000... | |
| [2023-05-24 20:46:03,171][2722668] Avg episode rewards: #0: 27.145, true rewards: #0: 11.431 | |
| [2023-05-24 20:46:03,172][2722668] Avg episode reward: 27.145, avg true_objective: 11.431 | |
| [2023-05-24 20:46:03,308][2722668] Num frames 8100... | |
| [2023-05-24 20:46:03,454][2722668] Num frames 8200... | |
| [2023-05-24 20:46:03,607][2722668] Num frames 8300... | |
| [2023-05-24 20:46:03,767][2722668] Num frames 8400... | |
| [2023-05-24 20:46:03,932][2722668] Num frames 8500... | |
| [2023-05-24 20:46:04,090][2722668] Num frames 8600... | |
| [2023-05-24 20:46:04,229][2722668] Num frames 8700... | |
| [2023-05-24 20:46:04,385][2722668] Num frames 8800... | |
| [2023-05-24 20:46:04,523][2722668] Num frames 8900... | |
| [2023-05-24 20:46:04,693][2722668] Num frames 9000... | |
| [2023-05-24 20:46:04,862][2722668] Num frames 9100... | |
| [2023-05-24 20:46:05,034][2722668] Num frames 9200... | |
| [2023-05-24 20:46:05,195][2722668] Num frames 9300... | |
| [2023-05-24 20:46:05,359][2722668] Num frames 9400... | |
| [2023-05-24 20:46:05,520][2722668] Num frames 9500... | |
| [2023-05-24 20:46:05,679][2722668] Num frames 9600... | |
| [2023-05-24 20:46:05,828][2722668] Num frames 9700... | |
| [2023-05-24 20:46:05,988][2722668] Num frames 9800... | |
| [2023-05-24 20:46:06,151][2722668] Num frames 9900... | |
| [2023-05-24 20:46:06,303][2722668] Num frames 10000... | |
| [2023-05-24 20:46:06,463][2722668] Num frames 10100... | |
| [2023-05-24 20:46:06,525][2722668] Avg episode rewards: #0: 30.752, true rewards: #0: 12.628 | |
| [2023-05-24 20:46:06,526][2722668] Avg episode reward: 30.752, avg true_objective: 12.628 | |
| [2023-05-24 20:46:06,676][2722668] Num frames 10200... | |
| [2023-05-24 20:46:06,836][2722668] Num frames 10300... | |
| [2023-05-24 20:46:07,005][2722668] Num frames 10400... | |
| [2023-05-24 20:46:07,182][2722668] Num frames 10500... | |
| [2023-05-24 20:46:07,348][2722668] Num frames 10600... | |
| [2023-05-24 20:46:07,512][2722668] Num frames 10700... | |
| [2023-05-24 20:46:07,676][2722668] Num frames 10800... | |
| [2023-05-24 20:46:07,833][2722668] Num frames 10900... | |
| [2023-05-24 20:46:08,004][2722668] Num frames 11000... | |
| [2023-05-24 20:46:08,175][2722668] Num frames 11100... | |
| [2023-05-24 20:46:08,340][2722668] Num frames 11200... | |
| [2023-05-24 20:46:08,506][2722668] Num frames 11300... | |
| [2023-05-24 20:46:08,673][2722668] Num frames 11400... | |
| [2023-05-24 20:46:08,838][2722668] Num frames 11500... | |
| [2023-05-24 20:46:09,009][2722668] Num frames 11600... | |
| [2023-05-24 20:46:09,172][2722668] Num frames 11700... | |
| [2023-05-24 20:46:09,341][2722668] Num frames 11800... | |
| [2023-05-24 20:46:09,499][2722668] Num frames 11900... | |
| [2023-05-24 20:46:09,657][2722668] Num frames 12000... | |
| [2023-05-24 20:46:09,822][2722668] Num frames 12100... | |
| [2023-05-24 20:46:09,991][2722668] Num frames 12200... | |
| [2023-05-24 20:46:10,053][2722668] Avg episode rewards: #0: 33.780, true rewards: #0: 13.558 | |
| [2023-05-24 20:46:10,054][2722668] Avg episode reward: 33.780, avg true_objective: 13.558 | |
| [2023-05-24 20:46:10,212][2722668] Num frames 12300... | |
| [2023-05-24 20:46:10,366][2722668] Num frames 12400... | |
| [2023-05-24 20:46:10,526][2722668] Num frames 12500... | |
| [2023-05-24 20:46:10,698][2722668] Num frames 12600... | |
| [2023-05-24 20:46:10,868][2722668] Num frames 12700... | |
| [2023-05-24 20:46:11,033][2722668] Num frames 12800... | |
| [2023-05-24 20:46:11,165][2722668] Avg episode rewards: #0: 31.947, true rewards: #0: 12.847 | |
| [2023-05-24 20:46:11,167][2722668] Avg episode reward: 31.947, avg true_objective: 12.847 | |
| [2023-05-24 20:46:42,742][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4! | |