[2024-12-31 06:24:32,564][00788] Saving configuration to /content/train_dir/default_experiment/config.json... [2024-12-31 06:24:32,566][00788] Rollout worker 0 uses device cpu [2024-12-31 06:24:32,568][00788] Rollout worker 1 uses device cpu [2024-12-31 06:24:32,570][00788] Rollout worker 2 uses device cpu [2024-12-31 06:24:32,571][00788] Rollout worker 3 uses device cpu [2024-12-31 06:24:32,572][00788] Rollout worker 4 uses device cpu [2024-12-31 06:24:32,573][00788] Rollout worker 5 uses device cpu [2024-12-31 06:24:32,574][00788] Rollout worker 6 uses device cpu [2024-12-31 06:24:32,575][00788] Rollout worker 7 uses device cpu [2024-12-31 06:24:32,732][00788] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:24:32,734][00788] InferenceWorker_p0-w0: min num requests: 2 [2024-12-31 06:24:32,766][00788] Starting all processes... [2024-12-31 06:24:32,767][00788] Starting process learner_proc0 [2024-12-31 06:24:32,814][00788] Starting all processes... [2024-12-31 06:24:32,821][00788] Starting process inference_proc0-0 [2024-12-31 06:24:32,821][00788] Starting process rollout_proc0 [2024-12-31 06:24:32,822][00788] Starting process rollout_proc1 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc2 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc3 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc4 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc5 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc6 [2024-12-31 06:24:32,823][00788] Starting process rollout_proc7 [2024-12-31 06:24:48,949][03021] Worker 7 uses CPU cores [1] [2024-12-31 06:24:48,955][03019] Worker 5 uses CPU cores [1] [2024-12-31 06:24:49,098][03017] Worker 2 uses CPU cores [0] [2024-12-31 06:24:49,156][03015] Worker 1 uses CPU cores [1] [2024-12-31 06:24:49,182][03020] Worker 6 uses CPU cores [0] [2024-12-31 06:24:49,187][03016] Worker 3 uses CPU cores [1] [2024-12-31 06:24:49,252][03014] Worker 0 uses CPU cores [0] [2024-12-31 06:24:49,257][03000] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:24:49,258][03000] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-12-31 06:24:49,275][03000] Num visible devices: 1 [2024-12-31 06:24:49,297][03000] Starting seed is not provided [2024-12-31 06:24:49,298][03000] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:24:49,299][03000] Initializing actor-critic model on device cuda:0 [2024-12-31 06:24:49,299][03000] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:24:49,303][03000] RunningMeanStd input shape: (1,) [2024-12-31 06:24:49,335][03018] Worker 4 uses CPU cores [0] [2024-12-31 06:24:49,330][03000] ConvEncoder: input_channels=3 [2024-12-31 06:24:49,351][03013] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:24:49,352][03013] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-12-31 06:24:49,368][03013] Num visible devices: 1 [2024-12-31 06:24:49,596][03000] Conv encoder output size: 512 [2024-12-31 06:24:49,596][03000] Policy head output size: 512 [2024-12-31 06:24:49,644][03000] Created Actor Critic model with architecture: [2024-12-31 06:24:49,645][03000] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-12-31 06:24:50,009][03000] Using optimizer [2024-12-31 06:24:52,730][00788] Heartbeat connected on Batcher_0 [2024-12-31 06:24:52,733][00788] Heartbeat connected on InferenceWorker_p0-w0 [2024-12-31 06:24:52,742][00788] Heartbeat connected on RolloutWorker_w0 [2024-12-31 06:24:52,745][00788] Heartbeat connected on RolloutWorker_w1 [2024-12-31 06:24:52,749][00788] Heartbeat connected on RolloutWorker_w2 [2024-12-31 06:24:52,752][00788] Heartbeat connected on RolloutWorker_w3 [2024-12-31 06:24:52,755][00788] Heartbeat connected on RolloutWorker_w4 [2024-12-31 06:24:52,759][00788] Heartbeat connected on RolloutWorker_w5 [2024-12-31 06:24:52,764][00788] Heartbeat connected on RolloutWorker_w6 [2024-12-31 06:24:52,767][00788] Heartbeat connected on RolloutWorker_w7 [2024-12-31 06:24:53,290][03000] No checkpoints found [2024-12-31 06:24:53,290][03000] Did not load from checkpoint, starting from scratch! [2024-12-31 06:24:53,290][03000] Initialized policy 0 weights for model version 0 [2024-12-31 06:24:53,294][03000] LearnerWorker_p0 finished initialization! [2024-12-31 06:24:53,295][03000] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:24:53,304][00788] Heartbeat connected on LearnerWorker_p0 [2024-12-31 06:24:53,493][03013] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:24:53,494][03013] RunningMeanStd input shape: (1,) [2024-12-31 06:24:53,507][03013] ConvEncoder: input_channels=3 [2024-12-31 06:24:53,612][03013] Conv encoder output size: 512 [2024-12-31 06:24:53,613][03013] Policy head output size: 512 [2024-12-31 06:24:53,663][00788] Inference worker 0-0 is ready! [2024-12-31 06:24:53,664][00788] All inference workers are ready! Signal rollout workers to start! [2024-12-31 06:24:53,844][03021] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,846][03015] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,848][03016] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,849][03019] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,886][03020] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,889][03014] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,893][03017] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:53,891][03018] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:24:54,522][03014] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,145][03016] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,152][03021] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,156][03015] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,151][03019] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,896][03020] Decorrelating experience for 0 frames... [2024-12-31 06:24:55,943][03014] Decorrelating experience for 32 frames... [2024-12-31 06:24:56,725][00788] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:24:56,870][03021] Decorrelating experience for 32 frames... [2024-12-31 06:24:56,867][03015] Decorrelating experience for 32 frames... [2024-12-31 06:24:56,876][03016] Decorrelating experience for 32 frames... [2024-12-31 06:24:56,890][03019] Decorrelating experience for 32 frames... [2024-12-31 06:24:57,707][03018] Decorrelating experience for 0 frames... [2024-12-31 06:24:58,122][03014] Decorrelating experience for 64 frames... [2024-12-31 06:24:58,478][03017] Decorrelating experience for 0 frames... [2024-12-31 06:24:58,491][03020] Decorrelating experience for 32 frames... [2024-12-31 06:24:59,567][03021] Decorrelating experience for 64 frames... [2024-12-31 06:24:59,586][03019] Decorrelating experience for 64 frames... [2024-12-31 06:25:00,521][03016] Decorrelating experience for 64 frames... [2024-12-31 06:25:01,345][03017] Decorrelating experience for 32 frames... [2024-12-31 06:25:01,445][03014] Decorrelating experience for 96 frames... [2024-12-31 06:25:01,728][00788] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:25:02,151][03020] Decorrelating experience for 64 frames... [2024-12-31 06:25:02,297][03015] Decorrelating experience for 64 frames... [2024-12-31 06:25:02,336][03018] Decorrelating experience for 32 frames... [2024-12-31 06:25:02,434][03021] Decorrelating experience for 96 frames... [2024-12-31 06:25:03,273][03016] Decorrelating experience for 96 frames... [2024-12-31 06:25:03,802][03015] Decorrelating experience for 96 frames... [2024-12-31 06:25:04,022][03020] Decorrelating experience for 96 frames... [2024-12-31 06:25:04,151][03017] Decorrelating experience for 64 frames... [2024-12-31 06:25:04,627][03018] Decorrelating experience for 64 frames... [2024-12-31 06:25:06,491][03017] Decorrelating experience for 96 frames... [2024-12-31 06:25:06,725][00788] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 128.6. Samples: 1286. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:25:06,727][00788] Avg episode reward: [(0, '2.320')] [2024-12-31 06:25:07,242][03000] Signal inference workers to stop experience collection... [2024-12-31 06:25:07,256][03013] InferenceWorker_p0-w0: stopping experience collection [2024-12-31 06:25:07,300][03018] Decorrelating experience for 96 frames... [2024-12-31 06:25:07,371][03019] Decorrelating experience for 96 frames... [2024-12-31 06:25:10,320][03000] Signal inference workers to resume experience collection... [2024-12-31 06:25:10,321][03013] InferenceWorker_p0-w0: resuming experience collection [2024-12-31 06:25:11,725][00788] Fps is (10 sec: 1229.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 12288. Throughput: 0: 231.7. Samples: 3476. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2024-12-31 06:25:11,726][00788] Avg episode reward: [(0, '3.094')] [2024-12-31 06:25:16,725][00788] Fps is (10 sec: 2867.1, 60 sec: 1433.6, 300 sec: 1433.6). Total num frames: 28672. Throughput: 0: 328.8. Samples: 6576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:25:16,732][00788] Avg episode reward: [(0, '3.800')] [2024-12-31 06:25:20,472][03013] Updated weights for policy 0, policy_version 10 (0.0158) [2024-12-31 06:25:21,726][00788] Fps is (10 sec: 3276.2, 60 sec: 1802.1, 300 sec: 1802.1). Total num frames: 45056. Throughput: 0: 448.7. Samples: 11218. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:25:21,728][00788] Avg episode reward: [(0, '4.292')] [2024-12-31 06:25:26,725][00788] Fps is (10 sec: 4096.1, 60 sec: 2321.1, 300 sec: 2321.1). Total num frames: 69632. Throughput: 0: 623.1. Samples: 18692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:25:26,731][00788] Avg episode reward: [(0, '4.451')] [2024-12-31 06:25:28,563][03013] Updated weights for policy 0, policy_version 20 (0.0020) [2024-12-31 06:25:31,725][00788] Fps is (10 sec: 4915.7, 60 sec: 2691.6, 300 sec: 2691.6). Total num frames: 94208. Throughput: 0: 637.3. Samples: 22306. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:25:31,728][00788] Avg episode reward: [(0, '4.349')] [2024-12-31 06:25:36,728][00788] Fps is (10 sec: 3685.0, 60 sec: 2662.1, 300 sec: 2662.1). Total num frames: 106496. Throughput: 0: 675.5. Samples: 27024. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:25:36,736][00788] Avg episode reward: [(0, '4.230')] [2024-12-31 06:25:36,741][03000] Saving new best policy, reward=4.230! [2024-12-31 06:25:39,944][03013] Updated weights for policy 0, policy_version 30 (0.0025) [2024-12-31 06:25:41,725][00788] Fps is (10 sec: 3686.7, 60 sec: 2912.7, 300 sec: 2912.7). Total num frames: 131072. Throughput: 0: 742.7. Samples: 33422. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:25:41,731][00788] Avg episode reward: [(0, '4.298')] [2024-12-31 06:25:41,734][03000] Saving new best policy, reward=4.298! [2024-12-31 06:25:46,725][00788] Fps is (10 sec: 4917.0, 60 sec: 3113.0, 300 sec: 3113.0). Total num frames: 155648. Throughput: 0: 823.7. Samples: 37062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:25:46,727][00788] Avg episode reward: [(0, '4.453')] [2024-12-31 06:25:46,733][03000] Saving new best policy, reward=4.453! [2024-12-31 06:25:48,836][03013] Updated weights for policy 0, policy_version 40 (0.0017) [2024-12-31 06:25:51,731][00788] Fps is (10 sec: 4096.0, 60 sec: 3127.9, 300 sec: 3127.9). Total num frames: 172032. Throughput: 0: 925.2. Samples: 42918. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:25:51,734][00788] Avg episode reward: [(0, '4.539')] [2024-12-31 06:25:51,736][03000] Saving new best policy, reward=4.539! [2024-12-31 06:25:56,725][00788] Fps is (10 sec: 3276.8, 60 sec: 3140.3, 300 sec: 3140.3). Total num frames: 188416. Throughput: 0: 1002.2. Samples: 48574. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:25:56,730][00788] Avg episode reward: [(0, '4.472')] [2024-12-31 06:25:59,353][03013] Updated weights for policy 0, policy_version 50 (0.0023) [2024-12-31 06:26:01,725][00788] Fps is (10 sec: 4096.0, 60 sec: 3550.1, 300 sec: 3276.8). Total num frames: 212992. Throughput: 0: 1014.5. Samples: 52228. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:01,730][00788] Avg episode reward: [(0, '4.312')] [2024-12-31 06:26:06,725][00788] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3335.3). Total num frames: 233472. Throughput: 0: 1056.0. Samples: 58736. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:06,727][00788] Avg episode reward: [(0, '4.342')] [2024-12-31 06:26:10,214][03013] Updated weights for policy 0, policy_version 60 (0.0019) [2024-12-31 06:26:11,725][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3331.4). Total num frames: 249856. Throughput: 0: 997.1. Samples: 63560. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:11,731][00788] Avg episode reward: [(0, '4.571')] [2024-12-31 06:26:11,734][03000] Saving new best policy, reward=4.571! [2024-12-31 06:26:16,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3430.4). Total num frames: 274432. Throughput: 0: 996.2. Samples: 67136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:16,729][00788] Avg episode reward: [(0, '4.663')] [2024-12-31 06:26:16,736][03000] Saving new best policy, reward=4.663! [2024-12-31 06:26:18,818][03013] Updated weights for policy 0, policy_version 70 (0.0034) [2024-12-31 06:26:21,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.7, 300 sec: 3517.7). Total num frames: 299008. Throughput: 0: 1054.6. Samples: 74478. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:21,730][00788] Avg episode reward: [(0, '4.453')] [2024-12-31 06:26:26,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3458.8). Total num frames: 311296. Throughput: 0: 1013.3. Samples: 79022. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:26,732][00788] Avg episode reward: [(0, '4.416')] [2024-12-31 06:26:26,738][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000076_311296.pth... [2024-12-31 06:26:30,006][03013] Updated weights for policy 0, policy_version 80 (0.0036) [2024-12-31 06:26:31,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.8, 300 sec: 3535.5). Total num frames: 335872. Throughput: 0: 1002.5. Samples: 82176. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:31,729][00788] Avg episode reward: [(0, '4.533')] [2024-12-31 06:26:36,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.5, 300 sec: 3563.5). Total num frames: 356352. Throughput: 0: 1033.6. Samples: 89432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:36,730][00788] Avg episode reward: [(0, '4.306')] [2024-12-31 06:26:38,680][03013] Updated weights for policy 0, policy_version 90 (0.0030) [2024-12-31 06:26:41,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3588.9). Total num frames: 376832. Throughput: 0: 1025.9. Samples: 94738. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2024-12-31 06:26:41,731][00788] Avg episode reward: [(0, '4.245')] [2024-12-31 06:26:46,725][00788] Fps is (10 sec: 3686.3, 60 sec: 3959.4, 300 sec: 3574.7). Total num frames: 393216. Throughput: 0: 999.0. Samples: 97184. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:26:46,732][00788] Avg episode reward: [(0, '4.601')] [2024-12-31 06:26:49,441][03013] Updated weights for policy 0, policy_version 100 (0.0019) [2024-12-31 06:26:51,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4096.0, 300 sec: 3633.0). Total num frames: 417792. Throughput: 0: 1020.0. Samples: 104634. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:26:51,733][00788] Avg episode reward: [(0, '4.627')] [2024-12-31 06:26:56,725][00788] Fps is (10 sec: 4505.7, 60 sec: 4164.3, 300 sec: 3652.3). Total num frames: 438272. Throughput: 0: 1051.6. Samples: 110884. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:26:56,730][00788] Avg episode reward: [(0, '4.379')] [2024-12-31 06:26:59,894][03013] Updated weights for policy 0, policy_version 110 (0.0039) [2024-12-31 06:27:01,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3637.2). Total num frames: 454656. Throughput: 0: 1021.7. Samples: 113112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:27:01,730][00788] Avg episode reward: [(0, '4.297')] [2024-12-31 06:27:06,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3686.4). Total num frames: 479232. Throughput: 0: 1004.2. Samples: 119668. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:27:06,727][00788] Avg episode reward: [(0, '4.533')] [2024-12-31 06:27:08,626][03013] Updated weights for policy 0, policy_version 120 (0.0019) [2024-12-31 06:27:11,725][00788] Fps is (10 sec: 4915.4, 60 sec: 4232.5, 300 sec: 3731.9). Total num frames: 503808. Throughput: 0: 1062.2. Samples: 126822. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:27:11,731][00788] Avg episode reward: [(0, '4.599')] [2024-12-31 06:27:16,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3686.4). Total num frames: 516096. Throughput: 0: 1040.8. Samples: 129010. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:27:16,729][00788] Avg episode reward: [(0, '4.451')] [2024-12-31 06:27:19,701][03013] Updated weights for policy 0, policy_version 130 (0.0030) [2024-12-31 06:27:21,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3728.8). Total num frames: 540672. Throughput: 0: 1009.0. Samples: 134836. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:21,732][00788] Avg episode reward: [(0, '4.504')] [2024-12-31 06:27:26,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 3768.3). Total num frames: 565248. Throughput: 0: 1056.6. Samples: 142284. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:26,732][00788] Avg episode reward: [(0, '4.543')] [2024-12-31 06:27:28,009][03013] Updated weights for policy 0, policy_version 140 (0.0016) [2024-12-31 06:27:31,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3752.5). Total num frames: 581632. Throughput: 0: 1065.5. Samples: 145130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:31,727][00788] Avg episode reward: [(0, '4.376')] [2024-12-31 06:27:36,725][00788] Fps is (10 sec: 3686.3, 60 sec: 4096.0, 300 sec: 3763.2). Total num frames: 602112. Throughput: 0: 1006.5. Samples: 149928. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:36,732][00788] Avg episode reward: [(0, '4.459')] [2024-12-31 06:27:39,125][03013] Updated weights for policy 0, policy_version 150 (0.0022) [2024-12-31 06:27:41,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3798.1). Total num frames: 626688. Throughput: 0: 1030.8. Samples: 157272. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:41,726][00788] Avg episode reward: [(0, '4.424')] [2024-12-31 06:27:46,725][00788] Fps is (10 sec: 4505.4, 60 sec: 4232.5, 300 sec: 3806.9). Total num frames: 647168. Throughput: 0: 1065.1. Samples: 161042. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:27:46,728][00788] Avg episode reward: [(0, '4.360')] [2024-12-31 06:27:48,987][03013] Updated weights for policy 0, policy_version 160 (0.0026) [2024-12-31 06:27:51,725][00788] Fps is (10 sec: 3276.8, 60 sec: 4027.7, 300 sec: 3768.3). Total num frames: 659456. Throughput: 0: 1020.9. Samples: 165608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:27:51,730][00788] Avg episode reward: [(0, '4.394')] [2024-12-31 06:27:56,725][00788] Fps is (10 sec: 4096.3, 60 sec: 4164.3, 300 sec: 3822.9). Total num frames: 688128. Throughput: 0: 1014.6. Samples: 172478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:27:56,727][00788] Avg episode reward: [(0, '4.341')] [2024-12-31 06:27:58,323][03013] Updated weights for policy 0, policy_version 170 (0.0018) [2024-12-31 06:28:01,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.6, 300 sec: 3830.3). Total num frames: 708608. Throughput: 0: 1047.5. Samples: 176146. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:28:01,727][00788] Avg episode reward: [(0, '4.455')] [2024-12-31 06:28:06,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 3815.7). Total num frames: 724992. Throughput: 0: 1040.5. Samples: 181660. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:28:06,731][00788] Avg episode reward: [(0, '4.434')] [2024-12-31 06:28:09,546][03013] Updated weights for policy 0, policy_version 180 (0.0027) [2024-12-31 06:28:11,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3822.9). Total num frames: 745472. Throughput: 0: 1006.0. Samples: 187554. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:28:11,727][00788] Avg episode reward: [(0, '4.379')] [2024-12-31 06:28:16,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 3850.2). Total num frames: 770048. Throughput: 0: 1024.0. Samples: 191208. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:28:16,727][00788] Avg episode reward: [(0, '4.661')] [2024-12-31 06:28:17,690][03013] Updated weights for policy 0, policy_version 190 (0.0026) [2024-12-31 06:28:21,727][00788] Fps is (10 sec: 4504.7, 60 sec: 4164.1, 300 sec: 3856.2). Total num frames: 790528. Throughput: 0: 1063.4. Samples: 197784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:28:21,729][00788] Avg episode reward: [(0, '4.908')] [2024-12-31 06:28:21,733][03000] Saving new best policy, reward=4.908! [2024-12-31 06:28:26,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3842.4). Total num frames: 806912. Throughput: 0: 1009.5. Samples: 202700. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:28:26,728][00788] Avg episode reward: [(0, '4.734')] [2024-12-31 06:28:26,746][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000197_806912.pth... [2024-12-31 06:28:28,793][03013] Updated weights for policy 0, policy_version 200 (0.0036) [2024-12-31 06:28:31,725][00788] Fps is (10 sec: 4096.8, 60 sec: 4164.3, 300 sec: 3867.4). Total num frames: 831488. Throughput: 0: 1005.3. Samples: 206278. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:28:31,729][00788] Avg episode reward: [(0, '4.457')] [2024-12-31 06:28:36,729][00788] Fps is (10 sec: 4503.5, 60 sec: 4164.0, 300 sec: 3872.5). Total num frames: 851968. Throughput: 0: 1064.5. Samples: 213516. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:28:36,736][00788] Avg episode reward: [(0, '4.538')] [2024-12-31 06:28:38,277][03013] Updated weights for policy 0, policy_version 210 (0.0021) [2024-12-31 06:28:41,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3859.3). Total num frames: 868352. Throughput: 0: 1012.8. Samples: 218052. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:28:41,730][00788] Avg episode reward: [(0, '4.596')] [2024-12-31 06:28:46,725][00788] Fps is (10 sec: 4097.9, 60 sec: 4096.0, 300 sec: 3882.3). Total num frames: 892928. Throughput: 0: 1005.0. Samples: 221372. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:28:46,727][00788] Avg episode reward: [(0, '4.484')] [2024-12-31 06:28:48,283][03013] Updated weights for policy 0, policy_version 220 (0.0015) [2024-12-31 06:28:51,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 3904.3). Total num frames: 917504. Throughput: 0: 1047.0. Samples: 228774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:28:51,727][00788] Avg episode reward: [(0, '4.475')] [2024-12-31 06:28:56,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3891.2). Total num frames: 933888. Throughput: 0: 1036.1. Samples: 234178. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:28:56,730][00788] Avg episode reward: [(0, '4.386')] [2024-12-31 06:28:59,381][03013] Updated weights for policy 0, policy_version 230 (0.0020) [2024-12-31 06:29:01,725][00788] Fps is (10 sec: 3276.8, 60 sec: 4027.7, 300 sec: 3878.7). Total num frames: 950272. Throughput: 0: 1008.6. Samples: 236594. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:29:01,734][00788] Avg episode reward: [(0, '4.448')] [2024-12-31 06:29:06,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 3899.4). Total num frames: 974848. Throughput: 0: 1025.9. Samples: 243946. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:29:06,726][00788] Avg episode reward: [(0, '4.642')] [2024-12-31 06:29:07,632][03013] Updated weights for policy 0, policy_version 240 (0.0025) [2024-12-31 06:29:11,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3903.2). Total num frames: 995328. Throughput: 0: 1056.6. Samples: 250248. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:29:11,727][00788] Avg episode reward: [(0, '4.508')] [2024-12-31 06:29:16,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3891.2). Total num frames: 1011712. Throughput: 0: 1025.8. Samples: 252440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:16,727][00788] Avg episode reward: [(0, '4.402')] [2024-12-31 06:29:18,690][03013] Updated weights for policy 0, policy_version 250 (0.0028) [2024-12-31 06:29:21,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.1, 300 sec: 3910.5). Total num frames: 1036288. Throughput: 0: 1013.1. Samples: 259100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:21,732][00788] Avg episode reward: [(0, '4.560')] [2024-12-31 06:29:26,726][00788] Fps is (10 sec: 4914.6, 60 sec: 4232.4, 300 sec: 3929.1). Total num frames: 1060864. Throughput: 0: 1072.0. Samples: 266294. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:26,733][00788] Avg episode reward: [(0, '4.428')] [2024-12-31 06:29:27,302][03013] Updated weights for policy 0, policy_version 260 (0.0024) [2024-12-31 06:29:31,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3917.3). Total num frames: 1077248. Throughput: 0: 1047.1. Samples: 268490. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:29:31,732][00788] Avg episode reward: [(0, '4.437')] [2024-12-31 06:29:36,725][00788] Fps is (10 sec: 3686.6, 60 sec: 4096.3, 300 sec: 3920.4). Total num frames: 1097728. Throughput: 0: 1006.8. Samples: 274082. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:29:36,728][00788] Avg episode reward: [(0, '4.462')] [2024-12-31 06:29:38,043][03013] Updated weights for policy 0, policy_version 270 (0.0033) [2024-12-31 06:29:41,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 3937.9). Total num frames: 1122304. Throughput: 0: 1052.4. Samples: 281536. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:41,726][00788] Avg episode reward: [(0, '4.569')] [2024-12-31 06:29:46,725][00788] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 3926.5). Total num frames: 1138688. Throughput: 0: 1062.9. Samples: 284424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:46,731][00788] Avg episode reward: [(0, '4.546')] [2024-12-31 06:29:48,418][03013] Updated weights for policy 0, policy_version 280 (0.0016) [2024-12-31 06:29:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3929.4). Total num frames: 1159168. Throughput: 0: 1008.6. Samples: 289332. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:29:51,727][00788] Avg episode reward: [(0, '4.500')] [2024-12-31 06:29:56,725][00788] Fps is (10 sec: 4505.9, 60 sec: 4164.3, 300 sec: 4012.7). Total num frames: 1183744. Throughput: 0: 1031.2. Samples: 296654. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:29:56,729][00788] Avg episode reward: [(0, '4.599')] [2024-12-31 06:29:57,512][03013] Updated weights for policy 0, policy_version 290 (0.0025) [2024-12-31 06:30:01,727][00788] Fps is (10 sec: 4504.4, 60 sec: 4232.3, 300 sec: 4082.1). Total num frames: 1204224. Throughput: 0: 1064.3. Samples: 300338. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:30:01,729][00788] Avg episode reward: [(0, '4.781')] [2024-12-31 06:30:06,725][00788] Fps is (10 sec: 3276.8, 60 sec: 4027.7, 300 sec: 4082.1). Total num frames: 1216512. Throughput: 0: 1018.0. Samples: 304912. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:30:06,727][00788] Avg episode reward: [(0, '4.650')] [2024-12-31 06:30:08,571][03013] Updated weights for policy 0, policy_version 300 (0.0035) [2024-12-31 06:30:11,725][00788] Fps is (10 sec: 3687.4, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 1241088. Throughput: 0: 1010.9. Samples: 311782. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:30:11,727][00788] Avg episode reward: [(0, '4.635')] [2024-12-31 06:30:16,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 1265664. Throughput: 0: 1043.7. Samples: 315456. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:30:16,727][00788] Avg episode reward: [(0, '4.611')] [2024-12-31 06:30:16,811][03013] Updated weights for policy 0, policy_version 310 (0.0018) [2024-12-31 06:30:21,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 1282048. Throughput: 0: 1043.1. Samples: 321022. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2024-12-31 06:30:21,727][00788] Avg episode reward: [(0, '4.727')] [2024-12-31 06:30:26,726][00788] Fps is (10 sec: 3685.7, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 1302528. Throughput: 0: 1011.1. Samples: 327038. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:30:26,729][00788] Avg episode reward: [(0, '4.914')] [2024-12-31 06:30:26,819][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000319_1306624.pth... [2024-12-31 06:30:26,956][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000076_311296.pth [2024-12-31 06:30:26,976][03000] Saving new best policy, reward=4.914! [2024-12-31 06:30:27,848][03013] Updated weights for policy 0, policy_version 320 (0.0035) [2024-12-31 06:30:31,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4137.7). Total num frames: 1327104. Throughput: 0: 1023.6. Samples: 330484. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:30:31,727][00788] Avg episode reward: [(0, '4.796')] [2024-12-31 06:30:36,725][00788] Fps is (10 sec: 4506.4, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 1347584. Throughput: 0: 1056.7. Samples: 336882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:30:36,726][00788] Avg episode reward: [(0, '4.590')] [2024-12-31 06:30:37,932][03013] Updated weights for policy 0, policy_version 330 (0.0023) [2024-12-31 06:30:41,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 1363968. Throughput: 0: 1003.5. Samples: 341810. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:30:41,729][00788] Avg episode reward: [(0, '4.565')] [2024-12-31 06:30:46,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 1388544. Throughput: 0: 1004.1. Samples: 345520. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:30:46,732][00788] Avg episode reward: [(0, '4.640')] [2024-12-31 06:30:47,276][03013] Updated weights for policy 0, policy_version 340 (0.0022) [2024-12-31 06:30:51,727][00788] Fps is (10 sec: 4913.8, 60 sec: 4232.3, 300 sec: 4151.5). Total num frames: 1413120. Throughput: 0: 1067.5. Samples: 352954. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:30:51,730][00788] Avg episode reward: [(0, '4.689')] [2024-12-31 06:30:56,725][00788] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 4109.9). Total num frames: 1425408. Throughput: 0: 1015.6. Samples: 357482. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2024-12-31 06:30:56,728][00788] Avg episode reward: [(0, '4.706')] [2024-12-31 06:30:58,113][03013] Updated weights for policy 0, policy_version 350 (0.0026) [2024-12-31 06:31:01,725][00788] Fps is (10 sec: 3687.4, 60 sec: 4096.2, 300 sec: 4123.8). Total num frames: 1449984. Throughput: 0: 1007.4. Samples: 360790. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:31:01,731][00788] Avg episode reward: [(0, '4.591')] [2024-12-31 06:31:06,545][03013] Updated weights for policy 0, policy_version 360 (0.0017) [2024-12-31 06:31:06,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4151.5). Total num frames: 1474560. Throughput: 0: 1045.8. Samples: 368084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:31:06,731][00788] Avg episode reward: [(0, '4.505')] [2024-12-31 06:31:11,734][00788] Fps is (10 sec: 4092.2, 60 sec: 4163.6, 300 sec: 4123.6). Total num frames: 1490944. Throughput: 0: 1027.9. Samples: 373302. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:31:11,737][00788] Avg episode reward: [(0, '4.468')] [2024-12-31 06:31:16,725][00788] Fps is (10 sec: 3276.8, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 1507328. Throughput: 0: 1006.8. Samples: 375790. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:31:16,728][00788] Avg episode reward: [(0, '4.486')] [2024-12-31 06:31:17,604][03013] Updated weights for policy 0, policy_version 370 (0.0018) [2024-12-31 06:31:21,725][00788] Fps is (10 sec: 4099.8, 60 sec: 4164.3, 300 sec: 4137.7). Total num frames: 1531904. Throughput: 0: 1028.4. Samples: 383160. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:31:21,726][00788] Avg episode reward: [(0, '4.662')] [2024-12-31 06:31:26,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.4, 300 sec: 4123.8). Total num frames: 1552384. Throughput: 0: 1058.4. Samples: 389438. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:31:26,732][00788] Avg episode reward: [(0, '4.754')] [2024-12-31 06:31:27,152][03013] Updated weights for policy 0, policy_version 380 (0.0018) [2024-12-31 06:31:31,725][00788] Fps is (10 sec: 3686.3, 60 sec: 4027.7, 300 sec: 4109.9). Total num frames: 1568768. Throughput: 0: 1024.4. Samples: 391620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:31:31,731][00788] Avg episode reward: [(0, '4.636')] [2024-12-31 06:31:36,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 1593344. Throughput: 0: 1006.6. Samples: 398246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:31:36,729][00788] Avg episode reward: [(0, '4.646')] [2024-12-31 06:31:37,062][03013] Updated weights for policy 0, policy_version 390 (0.0016) [2024-12-31 06:31:41,725][00788] Fps is (10 sec: 4915.3, 60 sec: 4232.5, 300 sec: 4151.5). Total num frames: 1617920. Throughput: 0: 1061.6. Samples: 405254. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:31:41,729][00788] Avg episode reward: [(0, '4.846')] [2024-12-31 06:31:46,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.8, 300 sec: 4109.9). Total num frames: 1630208. Throughput: 0: 1036.4. Samples: 407426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:31:46,727][00788] Avg episode reward: [(0, '4.713')] [2024-12-31 06:31:48,266][03013] Updated weights for policy 0, policy_version 400 (0.0020) [2024-12-31 06:31:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.9, 300 sec: 4123.8). Total num frames: 1654784. Throughput: 0: 1003.8. Samples: 413256. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:31:51,732][00788] Avg episode reward: [(0, '4.591')] [2024-12-31 06:31:56,513][03013] Updated weights for policy 0, policy_version 410 (0.0021) [2024-12-31 06:31:56,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4151.5). Total num frames: 1679360. Throughput: 0: 1050.1. Samples: 420548. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:31:56,731][00788] Avg episode reward: [(0, '4.621')] [2024-12-31 06:32:01,727][00788] Fps is (10 sec: 4094.9, 60 sec: 4095.8, 300 sec: 4123.7). Total num frames: 1695744. Throughput: 0: 1059.6. Samples: 423474. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:32:01,732][00788] Avg episode reward: [(0, '4.783')] [2024-12-31 06:32:06,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4109.9). Total num frames: 1716224. Throughput: 0: 1005.5. Samples: 428406. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:06,727][00788] Avg episode reward: [(0, '4.762')] [2024-12-31 06:32:07,661][03013] Updated weights for policy 0, policy_version 420 (0.0039) [2024-12-31 06:32:11,725][00788] Fps is (10 sec: 4097.1, 60 sec: 4096.6, 300 sec: 4137.7). Total num frames: 1736704. Throughput: 0: 1026.2. Samples: 435618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:11,728][00788] Avg episode reward: [(0, '4.849')] [2024-12-31 06:32:16,581][03013] Updated weights for policy 0, policy_version 430 (0.0019) [2024-12-31 06:32:16,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 1761280. Throughput: 0: 1058.8. Samples: 439266. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:16,727][00788] Avg episode reward: [(0, '4.901')] [2024-12-31 06:32:21,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 1773568. Throughput: 0: 1013.4. Samples: 443848. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:32:21,732][00788] Avg episode reward: [(0, '5.011')] [2024-12-31 06:32:21,734][03000] Saving new best policy, reward=5.011! [2024-12-31 06:32:26,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 1798144. Throughput: 0: 1008.2. Samples: 450622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:26,732][00788] Avg episode reward: [(0, '5.006')] [2024-12-31 06:32:26,740][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000439_1798144.pth... [2024-12-31 06:32:26,861][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000197_806912.pth [2024-12-31 06:32:27,188][03013] Updated weights for policy 0, policy_version 440 (0.0025) [2024-12-31 06:32:31,725][00788] Fps is (10 sec: 4915.0, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 1822720. Throughput: 0: 1037.9. Samples: 454132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:31,731][00788] Avg episode reward: [(0, '4.958')] [2024-12-31 06:32:36,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 1839104. Throughput: 0: 1028.7. Samples: 459548. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:36,731][00788] Avg episode reward: [(0, '5.068')] [2024-12-31 06:32:36,739][03000] Saving new best policy, reward=5.068! [2024-12-31 06:32:38,275][03013] Updated weights for policy 0, policy_version 450 (0.0029) [2024-12-31 06:32:41,725][00788] Fps is (10 sec: 3276.9, 60 sec: 3959.5, 300 sec: 4096.0). Total num frames: 1855488. Throughput: 0: 994.2. Samples: 465286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:32:41,731][00788] Avg episode reward: [(0, '4.949')] [2024-12-31 06:32:46,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4137.7). Total num frames: 1880064. Throughput: 0: 1009.8. Samples: 468914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:32:46,731][00788] Avg episode reward: [(0, '4.886')] [2024-12-31 06:32:46,900][03013] Updated weights for policy 0, policy_version 460 (0.0026) [2024-12-31 06:32:51,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 1900544. Throughput: 0: 1042.8. Samples: 475330. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:32:51,730][00788] Avg episode reward: [(0, '4.834')] [2024-12-31 06:32:56,725][00788] Fps is (10 sec: 3686.3, 60 sec: 3959.5, 300 sec: 4096.0). Total num frames: 1916928. Throughput: 0: 991.8. Samples: 480248. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:32:56,727][00788] Avg episode reward: [(0, '4.701')] [2024-12-31 06:32:58,054][03013] Updated weights for policy 0, policy_version 470 (0.0018) [2024-12-31 06:33:01,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4096.2, 300 sec: 4123.8). Total num frames: 1941504. Throughput: 0: 991.6. Samples: 483890. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:01,727][00788] Avg episode reward: [(0, '4.660')] [2024-12-31 06:33:06,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 1961984. Throughput: 0: 1049.0. Samples: 491052. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:06,727][00788] Avg episode reward: [(0, '4.563')] [2024-12-31 06:33:07,164][03013] Updated weights for policy 0, policy_version 480 (0.0018) [2024-12-31 06:33:11,725][00788] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 1978368. Throughput: 0: 995.3. Samples: 495412. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:11,727][00788] Avg episode reward: [(0, '4.830')] [2024-12-31 06:33:16,725][00788] Fps is (10 sec: 3686.5, 60 sec: 3959.5, 300 sec: 4096.0). Total num frames: 1998848. Throughput: 0: 987.2. Samples: 498554. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:16,730][00788] Avg episode reward: [(0, '4.740')] [2024-12-31 06:33:17,855][03013] Updated weights for policy 0, policy_version 490 (0.0022) [2024-12-31 06:33:21,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2023424. Throughput: 0: 1029.3. Samples: 505868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:21,729][00788] Avg episode reward: [(0, '4.469')] [2024-12-31 06:33:26,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 2039808. Throughput: 0: 1023.0. Samples: 511322. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:33:26,727][00788] Avg episode reward: [(0, '4.724')] [2024-12-31 06:33:28,422][03013] Updated weights for policy 0, policy_version 500 (0.0025) [2024-12-31 06:33:31,725][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4096.1). Total num frames: 2060288. Throughput: 0: 992.2. Samples: 513562. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:33:31,727][00788] Avg episode reward: [(0, '4.799')] [2024-12-31 06:33:36,725][00788] Fps is (10 sec: 4505.8, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 2084864. Throughput: 0: 1010.8. Samples: 520816. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:33:36,727][00788] Avg episode reward: [(0, '4.584')] [2024-12-31 06:33:37,451][03013] Updated weights for policy 0, policy_version 510 (0.0018) [2024-12-31 06:33:41,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4109.9). Total num frames: 2105344. Throughput: 0: 1038.9. Samples: 526998. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:41,727][00788] Avg episode reward: [(0, '4.674')] [2024-12-31 06:33:46,725][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 2117632. Throughput: 0: 1006.5. Samples: 529182. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:33:46,727][00788] Avg episode reward: [(0, '4.645')] [2024-12-31 06:33:48,482][03013] Updated weights for policy 0, policy_version 520 (0.0027) [2024-12-31 06:33:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 2142208. Throughput: 0: 993.5. Samples: 535758. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:33:51,731][00788] Avg episode reward: [(0, '4.654')] [2024-12-31 06:33:56,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2166784. Throughput: 0: 1057.9. Samples: 543018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:33:56,727][00788] Avg episode reward: [(0, '4.778')] [2024-12-31 06:33:57,217][03013] Updated weights for policy 0, policy_version 530 (0.0042) [2024-12-31 06:34:01,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 2183168. Throughput: 0: 1037.1. Samples: 545222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:01,730][00788] Avg episode reward: [(0, '4.677')] [2024-12-31 06:34:06,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 2203648. Throughput: 0: 1000.4. Samples: 550888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:34:06,730][00788] Avg episode reward: [(0, '4.556')] [2024-12-31 06:34:07,966][03013] Updated weights for policy 0, policy_version 540 (0.0016) [2024-12-31 06:34:11,725][00788] Fps is (10 sec: 4505.7, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2228224. Throughput: 0: 1039.6. Samples: 558102. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:11,730][00788] Avg episode reward: [(0, '4.526')] [2024-12-31 06:34:16,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2244608. Throughput: 0: 1056.0. Samples: 561080. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:16,731][00788] Avg episode reward: [(0, '4.593')] [2024-12-31 06:34:18,617][03013] Updated weights for policy 0, policy_version 550 (0.0028) [2024-12-31 06:34:21,725][00788] Fps is (10 sec: 3686.3, 60 sec: 4027.7, 300 sec: 4082.1). Total num frames: 2265088. Throughput: 0: 997.7. Samples: 565714. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:21,727][00788] Avg episode reward: [(0, '5.112')] [2024-12-31 06:34:21,731][03000] Saving new best policy, reward=5.112! [2024-12-31 06:34:26,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2285568. Throughput: 0: 1023.3. Samples: 573048. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:34:26,729][00788] Avg episode reward: [(0, '5.052')] [2024-12-31 06:34:26,750][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000559_2289664.pth... [2024-12-31 06:34:26,879][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000319_1306624.pth [2024-12-31 06:34:27,638][03013] Updated weights for policy 0, policy_version 560 (0.0036) [2024-12-31 06:34:31,728][00788] Fps is (10 sec: 4503.9, 60 sec: 4164.0, 300 sec: 4109.8). Total num frames: 2310144. Throughput: 0: 1052.9. Samples: 576566. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:34:31,736][00788] Avg episode reward: [(0, '4.617')] [2024-12-31 06:34:36,725][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 2322432. Throughput: 0: 1010.4. Samples: 581224. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:36,727][00788] Avg episode reward: [(0, '4.755')] [2024-12-31 06:34:38,725][03013] Updated weights for policy 0, policy_version 570 (0.0019) [2024-12-31 06:34:41,725][00788] Fps is (10 sec: 3687.8, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 2347008. Throughput: 0: 995.8. Samples: 587830. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:41,732][00788] Avg episode reward: [(0, '4.932')] [2024-12-31 06:34:46,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4109.9). Total num frames: 2371584. Throughput: 0: 1027.3. Samples: 591452. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:34:46,730][00788] Avg episode reward: [(0, '4.857')] [2024-12-31 06:34:47,015][03013] Updated weights for policy 0, policy_version 580 (0.0029) [2024-12-31 06:34:51,729][00788] Fps is (10 sec: 4094.1, 60 sec: 4095.7, 300 sec: 4082.1). Total num frames: 2387968. Throughput: 0: 1025.1. Samples: 597024. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:34:51,732][00788] Avg episode reward: [(0, '4.577')] [2024-12-31 06:34:56,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4082.2). Total num frames: 2408448. Throughput: 0: 993.6. Samples: 602814. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:34:56,727][00788] Avg episode reward: [(0, '4.594')] [2024-12-31 06:34:58,252][03013] Updated weights for policy 0, policy_version 590 (0.0022) [2024-12-31 06:35:01,725][00788] Fps is (10 sec: 4507.7, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2433024. Throughput: 0: 1008.6. Samples: 606468. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:35:01,731][00788] Avg episode reward: [(0, '4.565')] [2024-12-31 06:35:06,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2449408. Throughput: 0: 1046.7. Samples: 612814. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:35:06,731][00788] Avg episode reward: [(0, '4.532')] [2024-12-31 06:35:08,377][03013] Updated weights for policy 0, policy_version 600 (0.0026) [2024-12-31 06:35:11,725][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 4068.2). Total num frames: 2465792. Throughput: 0: 992.1. Samples: 617692. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:35:11,736][00788] Avg episode reward: [(0, '4.475')] [2024-12-31 06:35:16,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2490368. Throughput: 0: 994.4. Samples: 621308. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:35:16,731][00788] Avg episode reward: [(0, '4.573')] [2024-12-31 06:35:17,786][03013] Updated weights for policy 0, policy_version 610 (0.0035) [2024-12-31 06:35:21,725][00788] Fps is (10 sec: 4915.1, 60 sec: 4164.3, 300 sec: 4109.9). Total num frames: 2514944. Throughput: 0: 1054.4. Samples: 628670. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:35:21,729][00788] Avg episode reward: [(0, '4.640')] [2024-12-31 06:35:26,730][00788] Fps is (10 sec: 3684.3, 60 sec: 4027.4, 300 sec: 4068.2). Total num frames: 2527232. Throughput: 0: 1006.7. Samples: 633136. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:35:26,732][00788] Avg episode reward: [(0, '4.693')] [2024-12-31 06:35:28,965][03013] Updated weights for policy 0, policy_version 620 (0.0033) [2024-12-31 06:35:31,725][00788] Fps is (10 sec: 3686.5, 60 sec: 4028.0, 300 sec: 4082.1). Total num frames: 2551808. Throughput: 0: 1000.1. Samples: 636458. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:35:31,730][00788] Avg episode reward: [(0, '5.191')] [2024-12-31 06:35:31,734][03000] Saving new best policy, reward=5.191! [2024-12-31 06:35:36,725][00788] Fps is (10 sec: 4918.0, 60 sec: 4232.5, 300 sec: 4109.9). Total num frames: 2576384. Throughput: 0: 1037.0. Samples: 643682. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:35:36,731][00788] Avg episode reward: [(0, '5.190')] [2024-12-31 06:35:37,376][03013] Updated weights for policy 0, policy_version 630 (0.0031) [2024-12-31 06:35:41,726][00788] Fps is (10 sec: 4095.5, 60 sec: 4095.9, 300 sec: 4082.1). Total num frames: 2592768. Throughput: 0: 1023.0. Samples: 648850. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:35:41,729][00788] Avg episode reward: [(0, '5.067')] [2024-12-31 06:35:46,725][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 4054.4). Total num frames: 2609152. Throughput: 0: 991.9. Samples: 651102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:35:46,730][00788] Avg episode reward: [(0, '4.876')] [2024-12-31 06:35:48,646][03013] Updated weights for policy 0, policy_version 640 (0.0030) [2024-12-31 06:35:51,725][00788] Fps is (10 sec: 4096.5, 60 sec: 4096.3, 300 sec: 4096.0). Total num frames: 2633728. Throughput: 0: 1015.4. Samples: 658506. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:35:51,729][00788] Avg episode reward: [(0, '4.652')] [2024-12-31 06:35:56,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 2654208. Throughput: 0: 1047.6. Samples: 664832. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:35:56,728][00788] Avg episode reward: [(0, '4.710')] [2024-12-31 06:35:58,400][03013] Updated weights for policy 0, policy_version 650 (0.0029) [2024-12-31 06:36:01,725][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4054.3). Total num frames: 2670592. Throughput: 0: 1015.8. Samples: 667020. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:36:01,731][00788] Avg episode reward: [(0, '4.812')] [2024-12-31 06:36:06,725][00788] Fps is (10 sec: 4095.9, 60 sec: 4096.0, 300 sec: 4082.2). Total num frames: 2695168. Throughput: 0: 998.3. Samples: 673594. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:36:06,732][00788] Avg episode reward: [(0, '4.758')] [2024-12-31 06:36:08,067][03013] Updated weights for policy 0, policy_version 660 (0.0019) [2024-12-31 06:36:11,725][00788] Fps is (10 sec: 4915.1, 60 sec: 4232.5, 300 sec: 4109.9). Total num frames: 2719744. Throughput: 0: 1056.5. Samples: 680672. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:36:11,728][00788] Avg episode reward: [(0, '4.755')] [2024-12-31 06:36:16,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4068.2). Total num frames: 2732032. Throughput: 0: 1030.7. Samples: 682842. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:36:16,733][00788] Avg episode reward: [(0, '4.900')] [2024-12-31 06:36:19,133][03013] Updated weights for policy 0, policy_version 670 (0.0024) [2024-12-31 06:36:21,725][00788] Fps is (10 sec: 3686.5, 60 sec: 4027.8, 300 sec: 4082.1). Total num frames: 2756608. Throughput: 0: 997.1. Samples: 688550. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:36:21,726][00788] Avg episode reward: [(0, '5.248')] [2024-12-31 06:36:21,735][03000] Saving new best policy, reward=5.248! [2024-12-31 06:36:26,725][00788] Fps is (10 sec: 4505.7, 60 sec: 4164.7, 300 sec: 4096.0). Total num frames: 2777088. Throughput: 0: 1044.4. Samples: 695846. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:36:26,731][00788] Avg episode reward: [(0, '5.210')] [2024-12-31 06:36:26,742][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000679_2781184.pth... [2024-12-31 06:36:26,876][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000439_1798144.pth [2024-12-31 06:36:27,708][03013] Updated weights for policy 0, policy_version 680 (0.0034) [2024-12-31 06:36:31,727][00788] Fps is (10 sec: 4095.2, 60 sec: 4095.9, 300 sec: 4082.1). Total num frames: 2797568. Throughput: 0: 1056.2. Samples: 698634. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:36:31,731][00788] Avg episode reward: [(0, '4.850')] [2024-12-31 06:36:36,725][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4054.3). Total num frames: 2813952. Throughput: 0: 999.2. Samples: 703470. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:36:36,728][00788] Avg episode reward: [(0, '4.681')] [2024-12-31 06:36:38,681][03013] Updated weights for policy 0, policy_version 690 (0.0021) [2024-12-31 06:36:41,725][00788] Fps is (10 sec: 4096.8, 60 sec: 4096.1, 300 sec: 4096.0). Total num frames: 2838528. Throughput: 0: 1019.7. Samples: 710720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:36:41,727][00788] Avg episode reward: [(0, '4.471')] [2024-12-31 06:36:46,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4082.1). Total num frames: 2859008. Throughput: 0: 1053.9. Samples: 714446. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:36:46,731][00788] Avg episode reward: [(0, '4.451')] [2024-12-31 06:36:48,454][03013] Updated weights for policy 0, policy_version 700 (0.0023) [2024-12-31 06:36:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4054.3). Total num frames: 2875392. Throughput: 0: 1009.3. Samples: 719014. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:36:51,731][00788] Avg episode reward: [(0, '4.515')] [2024-12-31 06:36:56,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4082.2). Total num frames: 2899968. Throughput: 0: 1009.7. Samples: 726110. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:36:56,734][00788] Avg episode reward: [(0, '4.693')] [2024-12-31 06:36:57,849][03013] Updated weights for policy 0, policy_version 710 (0.0023) [2024-12-31 06:37:01,727][00788] Fps is (10 sec: 4914.1, 60 sec: 4232.4, 300 sec: 4096.0). Total num frames: 2924544. Throughput: 0: 1041.2. Samples: 729700. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:37:01,733][00788] Avg episode reward: [(0, '4.819')] [2024-12-31 06:37:06,729][00788] Fps is (10 sec: 4094.0, 60 sec: 4095.7, 300 sec: 4082.0). Total num frames: 2940928. Throughput: 0: 1034.4. Samples: 735104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:37:06,732][00788] Avg episode reward: [(0, '4.836')] [2024-12-31 06:37:08,831][03013] Updated weights for policy 0, policy_version 720 (0.0015) [2024-12-31 06:37:11,725][00788] Fps is (10 sec: 3687.2, 60 sec: 4027.8, 300 sec: 4068.2). Total num frames: 2961408. Throughput: 0: 1008.4. Samples: 741222. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:37:11,727][00788] Avg episode reward: [(0, '4.679')] [2024-12-31 06:37:16,725][00788] Fps is (10 sec: 4507.7, 60 sec: 4232.6, 300 sec: 4109.9). Total num frames: 2985984. Throughput: 0: 1030.0. Samples: 744984. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:37:16,727][00788] Avg episode reward: [(0, '4.755')] [2024-12-31 06:37:17,140][03013] Updated weights for policy 0, policy_version 730 (0.0026) [2024-12-31 06:37:21,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3002368. Throughput: 0: 1064.4. Samples: 751370. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:37:21,729][00788] Avg episode reward: [(0, '4.694')] [2024-12-31 06:37:26,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4068.2). Total num frames: 3022848. Throughput: 0: 1024.0. Samples: 756798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:37:26,728][00788] Avg episode reward: [(0, '4.793')] [2024-12-31 06:37:27,977][03013] Updated weights for policy 0, policy_version 740 (0.0015) [2024-12-31 06:37:31,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.4, 300 sec: 4096.0). Total num frames: 3047424. Throughput: 0: 1024.1. Samples: 760532. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:37:31,728][00788] Avg episode reward: [(0, '5.149')] [2024-12-31 06:37:36,726][00788] Fps is (10 sec: 4504.8, 60 sec: 4232.4, 300 sec: 4109.9). Total num frames: 3067904. Throughput: 0: 1082.3. Samples: 767720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:37:36,731][00788] Avg episode reward: [(0, '5.430')] [2024-12-31 06:37:36,779][03000] Saving new best policy, reward=5.430! [2024-12-31 06:37:36,782][03013] Updated weights for policy 0, policy_version 750 (0.0036) [2024-12-31 06:37:41,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3084288. Throughput: 0: 1024.7. Samples: 772222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:37:41,727][00788] Avg episode reward: [(0, '5.348')] [2024-12-31 06:37:46,725][00788] Fps is (10 sec: 4096.7, 60 sec: 4164.3, 300 sec: 4096.0). Total num frames: 3108864. Throughput: 0: 1026.3. Samples: 775882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:37:46,729][00788] Avg episode reward: [(0, '5.058')] [2024-12-31 06:37:46,959][03013] Updated weights for policy 0, policy_version 760 (0.0021) [2024-12-31 06:37:51,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4123.8). Total num frames: 3133440. Throughput: 0: 1073.2. Samples: 783394. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:37:51,729][00788] Avg episode reward: [(0, '5.273')] [2024-12-31 06:37:56,726][00788] Fps is (10 sec: 4095.5, 60 sec: 4164.2, 300 sec: 4096.0). Total num frames: 3149824. Throughput: 0: 1051.3. Samples: 788530. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:37:56,735][00788] Avg episode reward: [(0, '5.479')] [2024-12-31 06:37:56,751][03000] Saving new best policy, reward=5.479! [2024-12-31 06:37:57,379][03013] Updated weights for policy 0, policy_version 770 (0.0014) [2024-12-31 06:38:01,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.2, 300 sec: 4096.0). Total num frames: 3170304. Throughput: 0: 1029.3. Samples: 791304. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:38:01,731][00788] Avg episode reward: [(0, '5.492')] [2024-12-31 06:38:01,741][03000] Saving new best policy, reward=5.492! [2024-12-31 06:38:06,096][03013] Updated weights for policy 0, policy_version 780 (0.0027) [2024-12-31 06:38:06,725][00788] Fps is (10 sec: 4506.1, 60 sec: 4232.9, 300 sec: 4123.8). Total num frames: 3194880. Throughput: 0: 1053.8. Samples: 798792. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:38:06,729][00788] Avg episode reward: [(0, '5.542')] [2024-12-31 06:38:06,738][03000] Saving new best policy, reward=5.542! [2024-12-31 06:38:11,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4123.8). Total num frames: 3215360. Throughput: 0: 1061.0. Samples: 804544. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:38:11,727][00788] Avg episode reward: [(0, '5.708')] [2024-12-31 06:38:11,734][03000] Saving new best policy, reward=5.708! [2024-12-31 06:38:16,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 3231744. Throughput: 0: 1027.4. Samples: 806764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:38:16,731][00788] Avg episode reward: [(0, '6.143')] [2024-12-31 06:38:16,739][03000] Saving new best policy, reward=6.143! [2024-12-31 06:38:17,290][03013] Updated weights for policy 0, policy_version 790 (0.0019) [2024-12-31 06:38:21,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 4123.8). Total num frames: 3256320. Throughput: 0: 1023.9. Samples: 813794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:38:21,731][00788] Avg episode reward: [(0, '6.453')] [2024-12-31 06:38:21,734][03000] Saving new best policy, reward=6.453! [2024-12-31 06:38:25,532][03013] Updated weights for policy 0, policy_version 800 (0.0019) [2024-12-31 06:38:26,729][00788] Fps is (10 sec: 4503.5, 60 sec: 4232.2, 300 sec: 4123.7). Total num frames: 3276800. Throughput: 0: 1076.6. Samples: 820676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:38:26,732][00788] Avg episode reward: [(0, '6.212')] [2024-12-31 06:38:26,821][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000801_3280896.pth... [2024-12-31 06:38:26,995][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000559_2289664.pth [2024-12-31 06:38:31,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 3293184. Throughput: 0: 1041.7. Samples: 822760. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:38:31,727][00788] Avg episode reward: [(0, '6.360')] [2024-12-31 06:38:36,471][03013] Updated weights for policy 0, policy_version 810 (0.0021) [2024-12-31 06:38:36,725][00788] Fps is (10 sec: 4097.9, 60 sec: 4164.4, 300 sec: 4109.9). Total num frames: 3317760. Throughput: 0: 1009.7. Samples: 828832. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:38:36,727][00788] Avg episode reward: [(0, '6.445')] [2024-12-31 06:38:41,725][00788] Fps is (10 sec: 4505.5, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 3338240. Throughput: 0: 1057.9. Samples: 836136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:38:41,727][00788] Avg episode reward: [(0, '6.525')] [2024-12-31 06:38:41,793][03000] Saving new best policy, reward=6.525! [2024-12-31 06:38:46,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 3354624. Throughput: 0: 1049.0. Samples: 838510. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:38:46,731][00788] Avg episode reward: [(0, '6.823')] [2024-12-31 06:38:46,750][03000] Saving new best policy, reward=6.823! [2024-12-31 06:38:47,297][03013] Updated weights for policy 0, policy_version 820 (0.0021) [2024-12-31 06:38:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 3375104. Throughput: 0: 999.6. Samples: 843776. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:38:51,727][00788] Avg episode reward: [(0, '6.252')] [2024-12-31 06:38:56,006][03013] Updated weights for policy 0, policy_version 830 (0.0049) [2024-12-31 06:38:56,725][00788] Fps is (10 sec: 4915.3, 60 sec: 4232.6, 300 sec: 4137.7). Total num frames: 3403776. Throughput: 0: 1040.0. Samples: 851346. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:38:56,731][00788] Avg episode reward: [(0, '5.721')] [2024-12-31 06:39:01,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.2, 300 sec: 4123.8). Total num frames: 3420160. Throughput: 0: 1064.0. Samples: 854644. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:39:01,729][00788] Avg episode reward: [(0, '6.046')] [2024-12-31 06:39:06,725][00788] Fps is (10 sec: 3276.8, 60 sec: 4027.7, 300 sec: 4096.0). Total num frames: 3436544. Throughput: 0: 1010.9. Samples: 859284. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:39:06,727][00788] Avg episode reward: [(0, '6.603')] [2024-12-31 06:39:06,858][03013] Updated weights for policy 0, policy_version 840 (0.0016) [2024-12-31 06:39:11,725][00788] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 3461120. Throughput: 0: 1022.2. Samples: 866668. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:11,732][00788] Avg episode reward: [(0, '6.764')] [2024-12-31 06:39:15,197][03013] Updated weights for policy 0, policy_version 850 (0.0034) [2024-12-31 06:39:16,727][00788] Fps is (10 sec: 4913.8, 60 sec: 4232.3, 300 sec: 4137.6). Total num frames: 3485696. Throughput: 0: 1059.9. Samples: 870458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:16,730][00788] Avg episode reward: [(0, '6.579')] [2024-12-31 06:39:21,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 3502080. Throughput: 0: 1035.4. Samples: 875426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:21,727][00788] Avg episode reward: [(0, '6.595')] [2024-12-31 06:39:25,952][03013] Updated weights for policy 0, policy_version 860 (0.0035) [2024-12-31 06:39:26,725][00788] Fps is (10 sec: 4097.1, 60 sec: 4164.6, 300 sec: 4123.8). Total num frames: 3526656. Throughput: 0: 1022.2. Samples: 882136. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:26,728][00788] Avg episode reward: [(0, '7.343')] [2024-12-31 06:39:26,737][03000] Saving new best policy, reward=7.343! [2024-12-31 06:39:31,728][00788] Fps is (10 sec: 4503.9, 60 sec: 4232.3, 300 sec: 4151.5). Total num frames: 3547136. Throughput: 0: 1050.6. Samples: 885792. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:31,734][00788] Avg episode reward: [(0, '8.127')] [2024-12-31 06:39:31,752][03000] Saving new best policy, reward=8.127! [2024-12-31 06:39:35,561][03013] Updated weights for policy 0, policy_version 870 (0.0021) [2024-12-31 06:39:36,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 3563520. Throughput: 0: 1061.1. Samples: 891524. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:36,730][00788] Avg episode reward: [(0, '8.015')] [2024-12-31 06:39:41,725][00788] Fps is (10 sec: 3687.7, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 3584000. Throughput: 0: 1017.8. Samples: 897148. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:39:41,727][00788] Avg episode reward: [(0, '7.574')] [2024-12-31 06:39:45,322][03013] Updated weights for policy 0, policy_version 880 (0.0013) [2024-12-31 06:39:46,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 3608576. Throughput: 0: 1026.3. Samples: 900826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:39:46,727][00788] Avg episode reward: [(0, '8.188')] [2024-12-31 06:39:46,738][03000] Saving new best policy, reward=8.188! [2024-12-31 06:39:51,725][00788] Fps is (10 sec: 4505.7, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 3629056. Throughput: 0: 1072.8. Samples: 907562. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:39:51,727][00788] Avg episode reward: [(0, '8.649')] [2024-12-31 06:39:51,732][03000] Saving new best policy, reward=8.649! [2024-12-31 06:39:56,539][03013] Updated weights for policy 0, policy_version 890 (0.0019) [2024-12-31 06:39:56,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4109.9). Total num frames: 3645440. Throughput: 0: 1010.6. Samples: 912144. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:39:56,728][00788] Avg episode reward: [(0, '8.509')] [2024-12-31 06:40:01,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4137.7). Total num frames: 3670016. Throughput: 0: 1010.1. Samples: 915908. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:40:01,729][00788] Avg episode reward: [(0, '7.351')] [2024-12-31 06:40:04,754][03013] Updated weights for policy 0, policy_version 900 (0.0026) [2024-12-31 06:40:06,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4165.4). Total num frames: 3694592. Throughput: 0: 1064.5. Samples: 923328. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:06,727][00788] Avg episode reward: [(0, '7.777')] [2024-12-31 06:40:11,726][00788] Fps is (10 sec: 3685.9, 60 sec: 4095.9, 300 sec: 4123.8). Total num frames: 3706880. Throughput: 0: 1021.3. Samples: 928096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:11,734][00788] Avg episode reward: [(0, '8.393')] [2024-12-31 06:40:15,761][03013] Updated weights for policy 0, policy_version 910 (0.0022) [2024-12-31 06:40:16,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.2, 300 sec: 4123.8). Total num frames: 3731456. Throughput: 0: 1004.7. Samples: 931000. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:40:16,726][00788] Avg episode reward: [(0, '8.184')] [2024-12-31 06:40:21,725][00788] Fps is (10 sec: 4915.9, 60 sec: 4232.5, 300 sec: 4165.5). Total num frames: 3756032. Throughput: 0: 1042.9. Samples: 938454. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:21,732][00788] Avg episode reward: [(0, '7.912')] [2024-12-31 06:40:24,531][03013] Updated weights for policy 0, policy_version 920 (0.0025) [2024-12-31 06:40:26,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4137.7). Total num frames: 3772416. Throughput: 0: 1044.9. Samples: 944168. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:26,729][00788] Avg episode reward: [(0, '8.519')] [2024-12-31 06:40:26,736][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000921_3772416.pth... [2024-12-31 06:40:26,922][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000679_2781184.pth [2024-12-31 06:40:31,725][00788] Fps is (10 sec: 3276.7, 60 sec: 4028.0, 300 sec: 4109.9). Total num frames: 3788800. Throughput: 0: 1013.3. Samples: 946426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:31,727][00788] Avg episode reward: [(0, '9.684')] [2024-12-31 06:40:31,737][03000] Saving new best policy, reward=9.684! [2024-12-31 06:40:35,243][03013] Updated weights for policy 0, policy_version 930 (0.0029) [2024-12-31 06:40:36,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4137.7). Total num frames: 3813376. Throughput: 0: 1019.3. Samples: 953432. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:40:36,729][00788] Avg episode reward: [(0, '8.932')] [2024-12-31 06:40:41,725][00788] Fps is (10 sec: 4915.3, 60 sec: 4232.5, 300 sec: 4165.4). Total num frames: 3837952. Throughput: 0: 1067.9. Samples: 960198. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:40:41,730][00788] Avg episode reward: [(0, '9.035')] [2024-12-31 06:40:45,588][03013] Updated weights for policy 0, policy_version 940 (0.0019) [2024-12-31 06:40:46,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4123.8). Total num frames: 3850240. Throughput: 0: 1031.6. Samples: 962332. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:40:46,727][00788] Avg episode reward: [(0, '9.110')] [2024-12-31 06:40:51,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4137.7). Total num frames: 3874816. Throughput: 0: 1004.5. Samples: 968532. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:40:51,727][00788] Avg episode reward: [(0, '10.204')] [2024-12-31 06:40:51,734][03000] Saving new best policy, reward=10.204! [2024-12-31 06:40:54,750][03013] Updated weights for policy 0, policy_version 950 (0.0025) [2024-12-31 06:40:56,725][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4165.4). Total num frames: 3899392. Throughput: 0: 1060.3. Samples: 975810. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:40:56,731][00788] Avg episode reward: [(0, '9.868')] [2024-12-31 06:41:01,725][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4137.7). Total num frames: 3915776. Throughput: 0: 1053.5. Samples: 978408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:41:01,729][00788] Avg episode reward: [(0, '9.479')] [2024-12-31 06:41:05,710][03013] Updated weights for policy 0, policy_version 960 (0.0029) [2024-12-31 06:41:06,725][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4123.8). Total num frames: 3936256. Throughput: 0: 1004.9. Samples: 983674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:41:06,732][00788] Avg episode reward: [(0, '9.567')] [2024-12-31 06:41:11,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4232.6, 300 sec: 4165.4). Total num frames: 3960832. Throughput: 0: 1043.4. Samples: 991120. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:41:11,727][00788] Avg episode reward: [(0, '9.947')] [2024-12-31 06:41:13,926][03013] Updated weights for policy 0, policy_version 970 (0.0021) [2024-12-31 06:41:16,725][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4151.5). Total num frames: 3981312. Throughput: 0: 1069.1. Samples: 994536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:41:16,732][00788] Avg episode reward: [(0, '10.466')] [2024-12-31 06:41:16,743][03000] Saving new best policy, reward=10.466! [2024-12-31 06:41:21,725][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 4123.8). Total num frames: 3993600. Throughput: 0: 1011.9. Samples: 998966. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:41:21,732][00788] Avg episode reward: [(0, '10.210')] [2024-12-31 06:41:23,462][03000] Stopping Batcher_0... [2024-12-31 06:41:23,463][03000] Loop batcher_evt_loop terminating... [2024-12-31 06:41:23,464][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-12-31 06:41:23,463][00788] Component Batcher_0 stopped! [2024-12-31 06:41:23,524][03013] Weights refcount: 2 0 [2024-12-31 06:41:23,530][00788] Component InferenceWorker_p0-w0 stopped! [2024-12-31 06:41:23,537][03013] Stopping InferenceWorker_p0-w0... [2024-12-31 06:41:23,537][03013] Loop inference_proc0-0_evt_loop terminating... [2024-12-31 06:41:23,601][03000] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000801_3280896.pth [2024-12-31 06:41:23,620][03000] Saving new best policy, reward=10.496! [2024-12-31 06:41:23,765][03000] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-12-31 06:41:23,877][03020] Stopping RolloutWorker_w6... [2024-12-31 06:41:23,884][00788] Component RolloutWorker_w6 stopped! [2024-12-31 06:41:23,879][03020] Loop rollout_proc6_evt_loop terminating... [2024-12-31 06:41:23,934][00788] Component RolloutWorker_w4 stopped! [2024-12-31 06:41:23,939][03018] Stopping RolloutWorker_w4... [2024-12-31 06:41:23,950][00788] Component RolloutWorker_w0 stopped! [2024-12-31 06:41:23,967][00788] Component RolloutWorker_w2 stopped! [2024-12-31 06:41:23,972][03017] Stopping RolloutWorker_w2... [2024-12-31 06:41:23,943][03018] Loop rollout_proc4_evt_loop terminating... [2024-12-31 06:41:23,976][03015] Stopping RolloutWorker_w1... [2024-12-31 06:41:23,976][00788] Component RolloutWorker_w1 stopped! [2024-12-31 06:41:23,976][03015] Loop rollout_proc1_evt_loop terminating... [2024-12-31 06:41:23,955][03014] Stopping RolloutWorker_w0... [2024-12-31 06:41:23,973][03017] Loop rollout_proc2_evt_loop terminating... [2024-12-31 06:41:23,981][03014] Loop rollout_proc0_evt_loop terminating... [2024-12-31 06:41:24,001][03016] Stopping RolloutWorker_w3... [2024-12-31 06:41:24,003][03019] Stopping RolloutWorker_w5... [2024-12-31 06:41:24,004][03019] Loop rollout_proc5_evt_loop terminating... [2024-12-31 06:41:23,998][00788] Component RolloutWorker_w3 stopped! [2024-12-31 06:41:24,004][03016] Loop rollout_proc3_evt_loop terminating... [2024-12-31 06:41:24,007][00788] Component RolloutWorker_w5 stopped! [2024-12-31 06:41:24,023][03021] Stopping RolloutWorker_w7... [2024-12-31 06:41:24,023][00788] Component RolloutWorker_w7 stopped! [2024-12-31 06:41:24,028][03021] Loop rollout_proc7_evt_loop terminating... [2024-12-31 06:41:24,044][00788] Component LearnerWorker_p0 stopped! [2024-12-31 06:41:24,043][03000] Stopping LearnerWorker_p0... [2024-12-31 06:41:24,045][00788] Waiting for process learner_proc0 to stop... [2024-12-31 06:41:24,045][03000] Loop learner_proc0_evt_loop terminating... [2024-12-31 06:41:25,596][00788] Waiting for process inference_proc0-0 to join... [2024-12-31 06:41:25,599][00788] Waiting for process rollout_proc0 to join... [2024-12-31 06:41:27,455][00788] Waiting for process rollout_proc1 to join... [2024-12-31 06:41:27,457][00788] Waiting for process rollout_proc2 to join... [2024-12-31 06:41:27,458][00788] Waiting for process rollout_proc3 to join... [2024-12-31 06:41:27,460][00788] Waiting for process rollout_proc4 to join... [2024-12-31 06:41:27,462][00788] Waiting for process rollout_proc5 to join... [2024-12-31 06:41:27,464][00788] Waiting for process rollout_proc6 to join... [2024-12-31 06:41:27,466][00788] Waiting for process rollout_proc7 to join... [2024-12-31 06:41:27,468][00788] Batcher 0 profile tree view: batching: 26.1969, releasing_batches: 0.0258 [2024-12-31 06:41:27,470][00788] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 381.0816 update_model: 8.5363 weight_update: 0.0028 one_step: 0.0024 handle_policy_step: 557.1223 deserialize: 14.3121, stack: 3.0853, obs_to_device_normalize: 119.5419, forward: 278.4132, send_messages: 26.9837 prepare_outputs: 86.8674 to_cpu: 52.9164 [2024-12-31 06:41:27,471][00788] Learner 0 profile tree view: misc: 0.0051, prepare_batch: 13.2330 train: 73.3492 epoch_init: 0.0128, minibatch_init: 0.0125, losses_postprocess: 0.7573, kl_divergence: 0.5518, after_optimizer: 33.6334 calculate_losses: 26.2378 losses_init: 0.0045, forward_head: 1.2232, bptt_initial: 17.7144, tail: 1.0115, advantages_returns: 0.2426, losses: 3.7787 bptt: 1.9598 bptt_forward_core: 1.8419 update: 11.4862 clip: 0.8621 [2024-12-31 06:41:27,472][00788] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.3549, enqueue_policy_requests: 87.8263, env_step: 774.6256, overhead: 11.7799, complete_rollouts: 6.3283 save_policy_outputs: 20.1779 split_output_tensors: 8.1182 [2024-12-31 06:41:27,474][00788] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.3151, enqueue_policy_requests: 85.8950, env_step: 774.6481, overhead: 12.4036, complete_rollouts: 7.0383 save_policy_outputs: 20.6207 split_output_tensors: 8.2105 [2024-12-31 06:41:27,475][00788] Loop Runner_EvtLoop terminating... [2024-12-31 06:41:27,477][00788] Runner profile tree view: main_loop: 1014.7117 [2024-12-31 06:41:27,478][00788] Collected {0: 4005888}, FPS: 3947.8 [2024-12-31 06:44:38,578][00788] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-12-31 06:44:38,580][00788] Overriding arg 'num_workers' with value 1 passed from command line [2024-12-31 06:44:38,581][00788] Adding new argument 'no_render'=True that is not in the saved config file! [2024-12-31 06:44:38,583][00788] Adding new argument 'save_video'=True that is not in the saved config file! [2024-12-31 06:44:38,585][00788] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-12-31 06:44:38,587][00788] Adding new argument 'video_name'=None that is not in the saved config file! [2024-12-31 06:44:38,588][00788] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-12-31 06:44:38,590][00788] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-12-31 06:44:38,592][00788] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-12-31 06:44:38,593][00788] Adding new argument 'hf_repository'='LunaMeme/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-12-31 06:44:38,594][00788] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-12-31 06:44:38,595][00788] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-12-31 06:44:38,596][00788] Adding new argument 'train_script'=None that is not in the saved config file! [2024-12-31 06:44:38,597][00788] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-12-31 06:44:38,598][00788] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-12-31 06:44:38,630][00788] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:44:38,634][00788] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:44:38,635][00788] RunningMeanStd input shape: (1,) [2024-12-31 06:44:38,652][00788] ConvEncoder: input_channels=3 [2024-12-31 06:44:38,752][00788] Conv encoder output size: 512 [2024-12-31 06:44:38,754][00788] Policy head output size: 512 [2024-12-31 06:44:39,013][00788] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-12-31 06:44:39,825][00788] Num frames 100... [2024-12-31 06:44:39,943][00788] Num frames 200... [2024-12-31 06:44:40,071][00788] Num frames 300... [2024-12-31 06:44:40,191][00788] Num frames 400... [2024-12-31 06:44:40,311][00788] Num frames 500... [2024-12-31 06:44:40,433][00788] Num frames 600... [2024-12-31 06:44:40,550][00788] Num frames 700... [2024-12-31 06:44:40,675][00788] Num frames 800... [2024-12-31 06:44:40,797][00788] Num frames 900... [2024-12-31 06:44:40,915][00788] Num frames 1000... [2024-12-31 06:44:41,038][00788] Avg episode rewards: #0: 20.560, true rewards: #0: 10.560 [2024-12-31 06:44:41,039][00788] Avg episode reward: 20.560, avg true_objective: 10.560 [2024-12-31 06:44:41,103][00788] Num frames 1100... [2024-12-31 06:44:41,222][00788] Num frames 1200... [2024-12-31 06:44:41,348][00788] Num frames 1300... [2024-12-31 06:44:41,474][00788] Num frames 1400... [2024-12-31 06:44:41,594][00788] Num frames 1500... [2024-12-31 06:44:41,663][00788] Avg episode rewards: #0: 13.550, true rewards: #0: 7.550 [2024-12-31 06:44:41,665][00788] Avg episode reward: 13.550, avg true_objective: 7.550 [2024-12-31 06:44:41,776][00788] Num frames 1600... [2024-12-31 06:44:41,897][00788] Num frames 1700... [2024-12-31 06:44:42,014][00788] Num frames 1800... [2024-12-31 06:44:42,142][00788] Num frames 1900... [2024-12-31 06:44:42,260][00788] Num frames 2000... [2024-12-31 06:44:42,379][00788] Avg episode rewards: #0: 11.513, true rewards: #0: 6.847 [2024-12-31 06:44:42,381][00788] Avg episode reward: 11.513, avg true_objective: 6.847 [2024-12-31 06:44:42,443][00788] Num frames 2100... [2024-12-31 06:44:42,565][00788] Num frames 2200... [2024-12-31 06:44:42,689][00788] Num frames 2300... [2024-12-31 06:44:42,810][00788] Num frames 2400... [2024-12-31 06:44:42,933][00788] Num frames 2500... [2024-12-31 06:44:43,069][00788] Num frames 2600... [2024-12-31 06:44:43,228][00788] Avg episode rewards: #0: 11.155, true rewards: #0: 6.655 [2024-12-31 06:44:43,230][00788] Avg episode reward: 11.155, avg true_objective: 6.655 [2024-12-31 06:44:43,278][00788] Num frames 2700... [2024-12-31 06:44:43,406][00788] Num frames 2800... [2024-12-31 06:44:43,524][00788] Num frames 2900... [2024-12-31 06:44:43,643][00788] Num frames 3000... [2024-12-31 06:44:43,761][00788] Num frames 3100... [2024-12-31 06:44:43,885][00788] Num frames 3200... [2024-12-31 06:44:44,007][00788] Num frames 3300... [2024-12-31 06:44:44,125][00788] Num frames 3400... [2024-12-31 06:44:44,256][00788] Num frames 3500... [2024-12-31 06:44:44,382][00788] Num frames 3600... [2024-12-31 06:44:44,505][00788] Num frames 3700... [2024-12-31 06:44:44,584][00788] Avg episode rewards: #0: 13.036, true rewards: #0: 7.436 [2024-12-31 06:44:44,585][00788] Avg episode reward: 13.036, avg true_objective: 7.436 [2024-12-31 06:44:44,686][00788] Num frames 3800... [2024-12-31 06:44:44,803][00788] Num frames 3900... [2024-12-31 06:44:44,926][00788] Num frames 4000... [2024-12-31 06:44:45,045][00788] Num frames 4100... [2024-12-31 06:44:45,162][00788] Num frames 4200... [2024-12-31 06:44:45,293][00788] Num frames 4300... [2024-12-31 06:44:45,417][00788] Num frames 4400... [2024-12-31 06:44:45,585][00788] Avg episode rewards: #0: 12.977, true rewards: #0: 7.477 [2024-12-31 06:44:45,587][00788] Avg episode reward: 12.977, avg true_objective: 7.477 [2024-12-31 06:44:45,613][00788] Num frames 4500... [2024-12-31 06:44:45,778][00788] Num frames 4600... [2024-12-31 06:44:45,945][00788] Num frames 4700... [2024-12-31 06:44:46,111][00788] Num frames 4800... [2024-12-31 06:44:46,278][00788] Num frames 4900... [2024-12-31 06:44:46,446][00788] Num frames 5000... [2024-12-31 06:44:46,616][00788] Num frames 5100... [2024-12-31 06:44:46,781][00788] Num frames 5200... [2024-12-31 06:44:46,944][00788] Num frames 5300... [2024-12-31 06:44:47,121][00788] Num frames 5400... [2024-12-31 06:44:47,285][00788] Num frames 5500... [2024-12-31 06:44:47,463][00788] Num frames 5600... [2024-12-31 06:44:47,633][00788] Num frames 5700... [2024-12-31 06:44:47,752][00788] Avg episode rewards: #0: 14.763, true rewards: #0: 8.191 [2024-12-31 06:44:47,754][00788] Avg episode reward: 14.763, avg true_objective: 8.191 [2024-12-31 06:44:47,866][00788] Num frames 5800... [2024-12-31 06:44:48,008][00788] Num frames 5900... [2024-12-31 06:44:48,131][00788] Num frames 6000... [2024-12-31 06:44:48,248][00788] Num frames 6100... [2024-12-31 06:44:48,385][00788] Num frames 6200... [2024-12-31 06:44:48,513][00788] Num frames 6300... [2024-12-31 06:44:48,682][00788] Avg episode rewards: #0: 14.370, true rewards: #0: 7.995 [2024-12-31 06:44:48,683][00788] Avg episode reward: 14.370, avg true_objective: 7.995 [2024-12-31 06:44:48,692][00788] Num frames 6400... [2024-12-31 06:44:48,811][00788] Num frames 6500... [2024-12-31 06:44:48,929][00788] Num frames 6600... [2024-12-31 06:44:49,048][00788] Num frames 6700... [2024-12-31 06:44:49,198][00788] Avg episode rewards: #0: 13.200, true rewards: #0: 7.533 [2024-12-31 06:44:49,200][00788] Avg episode reward: 13.200, avg true_objective: 7.533 [2024-12-31 06:44:49,226][00788] Num frames 6800... [2024-12-31 06:44:49,358][00788] Num frames 6900... [2024-12-31 06:44:49,486][00788] Num frames 7000... [2024-12-31 06:44:49,605][00788] Num frames 7100... [2024-12-31 06:44:49,728][00788] Num frames 7200... [2024-12-31 06:44:49,849][00788] Num frames 7300... [2024-12-31 06:44:49,968][00788] Num frames 7400... [2024-12-31 06:44:50,089][00788] Num frames 7500... [2024-12-31 06:44:50,251][00788] Avg episode rewards: #0: 13.489, true rewards: #0: 7.589 [2024-12-31 06:44:50,252][00788] Avg episode reward: 13.489, avg true_objective: 7.589 [2024-12-31 06:45:31,854][00788] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2024-12-31 06:45:43,298][00788] The model has been pushed to https://huggingface.co/LunaMeme/rl_course_vizdoom_health_gathering_supreme [2024-12-31 06:46:56,032][00788] Environment doom_basic already registered, overwriting... [2024-12-31 06:46:56,035][00788] Environment doom_two_colors_easy already registered, overwriting... [2024-12-31 06:46:56,037][00788] Environment doom_two_colors_hard already registered, overwriting... [2024-12-31 06:46:56,040][00788] Environment doom_dm already registered, overwriting... [2024-12-31 06:46:56,042][00788] Environment doom_dwango5 already registered, overwriting... [2024-12-31 06:46:56,044][00788] Environment doom_my_way_home_flat_actions already registered, overwriting... [2024-12-31 06:46:56,045][00788] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2024-12-31 06:46:56,047][00788] Environment doom_my_way_home already registered, overwriting... [2024-12-31 06:46:56,048][00788] Environment doom_deadly_corridor already registered, overwriting... [2024-12-31 06:46:56,049][00788] Environment doom_defend_the_center already registered, overwriting... [2024-12-31 06:46:56,050][00788] Environment doom_defend_the_line already registered, overwriting... [2024-12-31 06:46:56,051][00788] Environment doom_health_gathering already registered, overwriting... [2024-12-31 06:46:56,052][00788] Environment doom_health_gathering_supreme already registered, overwriting... [2024-12-31 06:46:56,053][00788] Environment doom_battle already registered, overwriting... [2024-12-31 06:46:56,055][00788] Environment doom_battle2 already registered, overwriting... [2024-12-31 06:46:56,056][00788] Environment doom_duel_bots already registered, overwriting... [2024-12-31 06:46:56,058][00788] Environment doom_deathmatch_bots already registered, overwriting... [2024-12-31 06:46:56,059][00788] Environment doom_duel already registered, overwriting... [2024-12-31 06:46:56,061][00788] Environment doom_deathmatch_full already registered, overwriting... [2024-12-31 06:46:56,062][00788] Environment doom_benchmark already registered, overwriting... [2024-12-31 06:46:56,063][00788] register_encoder_factory: [2024-12-31 06:46:56,084][00788] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-12-31 06:46:56,085][00788] Overriding arg 'train_for_env_steps' with value 5000000 passed from command line [2024-12-31 06:46:56,092][00788] Experiment dir /content/train_dir/default_experiment already exists! [2024-12-31 06:46:56,094][00788] Resuming existing experiment from /content/train_dir/default_experiment... [2024-12-31 06:46:56,097][00788] Weights and Biases integration disabled [2024-12-31 06:46:56,101][00788] Environment var CUDA_VISIBLE_DEVICES is 0 [2024-12-31 06:46:58,626][00788] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=5000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2024-12-31 06:46:58,629][00788] Saving configuration to /content/train_dir/default_experiment/config.json... [2024-12-31 06:46:58,631][00788] Rollout worker 0 uses device cpu [2024-12-31 06:46:58,634][00788] Rollout worker 1 uses device cpu [2024-12-31 06:46:58,635][00788] Rollout worker 2 uses device cpu [2024-12-31 06:46:58,636][00788] Rollout worker 3 uses device cpu [2024-12-31 06:46:58,637][00788] Rollout worker 4 uses device cpu [2024-12-31 06:46:58,638][00788] Rollout worker 5 uses device cpu [2024-12-31 06:46:58,639][00788] Rollout worker 6 uses device cpu [2024-12-31 06:46:58,643][00788] Rollout worker 7 uses device cpu [2024-12-31 06:46:58,754][00788] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:46:58,756][00788] InferenceWorker_p0-w0: min num requests: 2 [2024-12-31 06:46:58,798][00788] Starting all processes... [2024-12-31 06:46:58,799][00788] Starting process learner_proc0 [2024-12-31 06:46:58,852][00788] Starting all processes... [2024-12-31 06:46:58,862][00788] Starting process inference_proc0-0 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc0 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc1 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc2 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc3 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc4 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc5 [2024-12-31 06:46:58,863][00788] Starting process rollout_proc6 [2024-12-31 06:46:58,865][00788] Starting process rollout_proc7 [2024-12-31 06:47:15,796][12529] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:47:15,798][12529] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-12-31 06:47:15,868][12529] Num visible devices: 1 [2024-12-31 06:47:15,880][12549] Worker 6 uses CPU cores [0] [2024-12-31 06:47:15,891][12529] Starting seed is not provided [2024-12-31 06:47:15,892][12529] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:47:15,893][12529] Initializing actor-critic model on device cuda:0 [2024-12-31 06:47:15,894][12529] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:47:15,895][12529] RunningMeanStd input shape: (1,) [2024-12-31 06:47:15,917][12547] Worker 5 uses CPU cores [1] [2024-12-31 06:47:15,971][12529] ConvEncoder: input_channels=3 [2024-12-31 06:47:16,224][12550] Worker 7 uses CPU cores [1] [2024-12-31 06:47:16,275][12544] Worker 1 uses CPU cores [1] [2024-12-31 06:47:16,288][12546] Worker 3 uses CPU cores [1] [2024-12-31 06:47:16,332][12542] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:47:16,333][12545] Worker 2 uses CPU cores [0] [2024-12-31 06:47:16,333][12542] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-12-31 06:47:16,353][12542] Num visible devices: 1 [2024-12-31 06:47:16,409][12543] Worker 0 uses CPU cores [0] [2024-12-31 06:47:16,426][12548] Worker 4 uses CPU cores [0] [2024-12-31 06:47:16,457][12529] Conv encoder output size: 512 [2024-12-31 06:47:16,457][12529] Policy head output size: 512 [2024-12-31 06:47:16,472][12529] Created Actor Critic model with architecture: [2024-12-31 06:47:16,472][12529] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-12-31 06:47:16,673][12529] Using optimizer [2024-12-31 06:47:17,461][12529] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-12-31 06:47:17,498][12529] Loading model from checkpoint [2024-12-31 06:47:17,499][12529] Loaded experiment state at self.train_step=978, self.env_steps=4005888 [2024-12-31 06:47:17,500][12529] Initialized policy 0 weights for model version 978 [2024-12-31 06:47:17,503][12529] LearnerWorker_p0 finished initialization! [2024-12-31 06:47:17,503][12529] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-12-31 06:47:17,686][12542] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:47:17,688][12542] RunningMeanStd input shape: (1,) [2024-12-31 06:47:17,700][12542] ConvEncoder: input_channels=3 [2024-12-31 06:47:17,801][12542] Conv encoder output size: 512 [2024-12-31 06:47:17,802][12542] Policy head output size: 512 [2024-12-31 06:47:17,851][00788] Inference worker 0-0 is ready! [2024-12-31 06:47:17,853][00788] All inference workers are ready! Signal rollout workers to start! [2024-12-31 06:47:18,047][12548] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,049][12545] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,050][12549] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,050][12543] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,045][12544] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,052][12550] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,061][12546] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,053][12547] Doom resolution: 160x120, resize resolution: (128, 72) [2024-12-31 06:47:18,743][00788] Heartbeat connected on Batcher_0 [2024-12-31 06:47:18,748][00788] Heartbeat connected on LearnerWorker_p0 [2024-12-31 06:47:18,787][00788] Heartbeat connected on InferenceWorker_p0-w0 [2024-12-31 06:47:19,054][12544] Decorrelating experience for 0 frames... [2024-12-31 06:47:19,053][12546] Decorrelating experience for 0 frames... [2024-12-31 06:47:19,472][12546] Decorrelating experience for 32 frames... [2024-12-31 06:47:19,718][12545] Decorrelating experience for 0 frames... [2024-12-31 06:47:19,714][12548] Decorrelating experience for 0 frames... [2024-12-31 06:47:19,719][12549] Decorrelating experience for 0 frames... [2024-12-31 06:47:19,725][12543] Decorrelating experience for 0 frames... [2024-12-31 06:47:20,421][12546] Decorrelating experience for 64 frames... [2024-12-31 06:47:20,485][12548] Decorrelating experience for 32 frames... [2024-12-31 06:47:20,507][12545] Decorrelating experience for 32 frames... [2024-12-31 06:47:20,856][12547] Decorrelating experience for 0 frames... [2024-12-31 06:47:20,859][12550] Decorrelating experience for 0 frames... [2024-12-31 06:47:21,102][00788] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:47:21,388][12546] Decorrelating experience for 96 frames... [2024-12-31 06:47:21,560][00788] Heartbeat connected on RolloutWorker_w3 [2024-12-31 06:47:21,809][12550] Decorrelating experience for 32 frames... [2024-12-31 06:47:21,825][12549] Decorrelating experience for 32 frames... [2024-12-31 06:47:21,922][12543] Decorrelating experience for 32 frames... [2024-12-31 06:47:22,436][12545] Decorrelating experience for 64 frames... [2024-12-31 06:47:22,960][12547] Decorrelating experience for 32 frames... [2024-12-31 06:47:23,479][12549] Decorrelating experience for 64 frames... [2024-12-31 06:47:23,488][12544] Decorrelating experience for 32 frames... [2024-12-31 06:47:23,559][12545] Decorrelating experience for 96 frames... [2024-12-31 06:47:23,759][00788] Heartbeat connected on RolloutWorker_w2 [2024-12-31 06:47:24,152][12550] Decorrelating experience for 64 frames... [2024-12-31 06:47:25,275][12548] Decorrelating experience for 64 frames... [2024-12-31 06:47:25,465][12547] Decorrelating experience for 64 frames... [2024-12-31 06:47:25,567][12549] Decorrelating experience for 96 frames... [2024-12-31 06:47:26,102][00788] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 66.0. Samples: 330. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:47:26,105][00788] Avg episode reward: [(0, '3.806')] [2024-12-31 06:47:26,365][00788] Heartbeat connected on RolloutWorker_w6 [2024-12-31 06:47:27,195][12544] Decorrelating experience for 64 frames... [2024-12-31 06:47:28,667][12543] Decorrelating experience for 64 frames... [2024-12-31 06:47:30,710][12550] Decorrelating experience for 96 frames... [2024-12-31 06:47:30,740][12529] Signal inference workers to stop experience collection... [2024-12-31 06:47:30,777][12542] InferenceWorker_p0-w0: stopping experience collection [2024-12-31 06:47:31,102][00788] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 196.8. Samples: 1968. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-12-31 06:47:31,108][00788] Avg episode reward: [(0, '7.503')] [2024-12-31 06:47:31,495][00788] Heartbeat connected on RolloutWorker_w7 [2024-12-31 06:47:31,572][12547] Decorrelating experience for 96 frames... [2024-12-31 06:47:31,684][12548] Decorrelating experience for 96 frames... [2024-12-31 06:47:31,898][00788] Heartbeat connected on RolloutWorker_w5 [2024-12-31 06:47:31,927][12529] Signal inference workers to resume experience collection... [2024-12-31 06:47:31,928][12542] InferenceWorker_p0-w0: resuming experience collection [2024-12-31 06:47:31,980][12543] Decorrelating experience for 96 frames... [2024-12-31 06:47:32,051][00788] Heartbeat connected on RolloutWorker_w4 [2024-12-31 06:47:32,397][00788] Heartbeat connected on RolloutWorker_w0 [2024-12-31 06:47:32,591][12544] Decorrelating experience for 96 frames... [2024-12-31 06:47:32,965][00788] Heartbeat connected on RolloutWorker_w1 [2024-12-31 06:47:36,103][00788] Fps is (10 sec: 2047.8, 60 sec: 1365.3, 300 sec: 1365.3). Total num frames: 4026368. Throughput: 0: 418.6. Samples: 6280. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:47:36,106][00788] Avg episode reward: [(0, '5.910')] [2024-12-31 06:47:39,529][12542] Updated weights for policy 0, policy_version 988 (0.0024) [2024-12-31 06:47:41,102][00788] Fps is (10 sec: 4505.3, 60 sec: 2252.7, 300 sec: 2252.7). Total num frames: 4050944. Throughput: 0: 499.0. Samples: 9980. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:47:41,112][00788] Avg episode reward: [(0, '8.473')] [2024-12-31 06:47:46,102][00788] Fps is (10 sec: 4096.3, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 4067328. Throughput: 0: 635.9. Samples: 15898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:47:46,108][00788] Avg episode reward: [(0, '9.356')] [2024-12-31 06:47:51,102][00788] Fps is (10 sec: 3277.0, 60 sec: 2594.1, 300 sec: 2594.1). Total num frames: 4083712. Throughput: 0: 683.1. Samples: 20494. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:47:51,104][00788] Avg episode reward: [(0, '10.161')] [2024-12-31 06:47:51,330][12542] Updated weights for policy 0, policy_version 998 (0.0032) [2024-12-31 06:47:56,102][00788] Fps is (10 sec: 4096.0, 60 sec: 2925.7, 300 sec: 2925.7). Total num frames: 4108288. Throughput: 0: 690.2. Samples: 24158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:47:56,104][00788] Avg episode reward: [(0, '11.551')] [2024-12-31 06:47:56,175][12529] Saving new best policy, reward=11.551! [2024-12-31 06:47:59,698][12542] Updated weights for policy 0, policy_version 1008 (0.0028) [2024-12-31 06:48:01,102][00788] Fps is (10 sec: 4915.2, 60 sec: 3174.4, 300 sec: 3174.4). Total num frames: 4132864. Throughput: 0: 785.7. Samples: 31428. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2024-12-31 06:48:01,104][00788] Avg episode reward: [(0, '11.666')] [2024-12-31 06:48:01,109][12529] Saving new best policy, reward=11.666! [2024-12-31 06:48:06,102][00788] Fps is (10 sec: 3686.3, 60 sec: 3094.7, 300 sec: 3094.7). Total num frames: 4145152. Throughput: 0: 798.6. Samples: 35936. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:48:06,112][00788] Avg episode reward: [(0, '12.461')] [2024-12-31 06:48:06,126][12529] Saving new best policy, reward=12.461! [2024-12-31 06:48:11,055][12542] Updated weights for policy 0, policy_version 1018 (0.0021) [2024-12-31 06:48:11,102][00788] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3276.8). Total num frames: 4169728. Throughput: 0: 855.5. Samples: 38826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:48:11,108][00788] Avg episode reward: [(0, '12.970')] [2024-12-31 06:48:11,110][12529] Saving new best policy, reward=12.970! [2024-12-31 06:48:16,104][00788] Fps is (10 sec: 4914.2, 60 sec: 3425.6, 300 sec: 3425.6). Total num frames: 4194304. Throughput: 0: 983.7. Samples: 46236. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:48:16,108][00788] Avg episode reward: [(0, '12.575')] [2024-12-31 06:48:21,022][12542] Updated weights for policy 0, policy_version 1028 (0.0038) [2024-12-31 06:48:21,102][00788] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3413.3). Total num frames: 4210688. Throughput: 0: 1011.7. Samples: 51806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:48:21,105][00788] Avg episode reward: [(0, '11.798')] [2024-12-31 06:48:26,102][00788] Fps is (10 sec: 3277.5, 60 sec: 3686.4, 300 sec: 3402.8). Total num frames: 4227072. Throughput: 0: 979.8. Samples: 54070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:48:26,108][00788] Avg episode reward: [(0, '12.757')] [2024-12-31 06:48:30,532][12542] Updated weights for policy 0, policy_version 1038 (0.0025) [2024-12-31 06:48:31,102][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3510.9). Total num frames: 4251648. Throughput: 0: 1009.3. Samples: 61318. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:48:31,109][00788] Avg episode reward: [(0, '14.018')] [2024-12-31 06:48:31,114][12529] Saving new best policy, reward=14.018! [2024-12-31 06:48:36,107][00788] Fps is (10 sec: 4503.2, 60 sec: 4095.7, 300 sec: 3549.6). Total num frames: 4272128. Throughput: 0: 1054.1. Samples: 67934. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:48:36,110][00788] Avg episode reward: [(0, '14.480')] [2024-12-31 06:48:36,121][12529] Saving new best policy, reward=14.480! [2024-12-31 06:48:41,102][00788] Fps is (10 sec: 3686.3, 60 sec: 3959.5, 300 sec: 3532.8). Total num frames: 4288512. Throughput: 0: 1021.3. Samples: 70118. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:48:41,108][00788] Avg episode reward: [(0, '14.393')] [2024-12-31 06:48:41,507][12542] Updated weights for policy 0, policy_version 1048 (0.0020) [2024-12-31 06:48:46,102][00788] Fps is (10 sec: 4098.3, 60 sec: 4096.0, 300 sec: 3614.1). Total num frames: 4313088. Throughput: 0: 997.3. Samples: 76308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:48:46,107][00788] Avg episode reward: [(0, '15.015')] [2024-12-31 06:48:46,112][12529] Saving new best policy, reward=15.015! [2024-12-31 06:48:50,069][12542] Updated weights for policy 0, policy_version 1058 (0.0022) [2024-12-31 06:48:51,102][00788] Fps is (10 sec: 4915.3, 60 sec: 4232.5, 300 sec: 3686.4). Total num frames: 4337664. Throughput: 0: 1058.6. Samples: 83574. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:48:51,104][00788] Avg episode reward: [(0, '15.390')] [2024-12-31 06:48:51,107][12529] Saving new best policy, reward=15.390! [2024-12-31 06:48:56,102][00788] Fps is (10 sec: 4095.7, 60 sec: 4096.0, 300 sec: 3664.8). Total num frames: 4354048. Throughput: 0: 1046.8. Samples: 85934. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:48:56,113][00788] Avg episode reward: [(0, '15.675')] [2024-12-31 06:48:56,123][12529] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001063_4354048.pth... [2024-12-31 06:48:56,255][12529] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000921_3772416.pth [2024-12-31 06:48:56,270][12529] Saving new best policy, reward=15.675! [2024-12-31 06:49:01,102][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3645.4). Total num frames: 4370432. Throughput: 0: 994.0. Samples: 90962. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:49:01,104][00788] Avg episode reward: [(0, '16.098')] [2024-12-31 06:49:01,109][12529] Saving new best policy, reward=16.098! [2024-12-31 06:49:01,352][12542] Updated weights for policy 0, policy_version 1068 (0.0024) [2024-12-31 06:49:06,102][00788] Fps is (10 sec: 4096.3, 60 sec: 4164.3, 300 sec: 3705.9). Total num frames: 4395008. Throughput: 0: 1032.7. Samples: 98278. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:49:06,104][00788] Avg episode reward: [(0, '16.325')] [2024-12-31 06:49:06,111][12529] Saving new best policy, reward=16.325! [2024-12-31 06:49:10,766][12542] Updated weights for policy 0, policy_version 1078 (0.0017) [2024-12-31 06:49:11,104][00788] Fps is (10 sec: 4504.5, 60 sec: 4095.8, 300 sec: 3723.6). Total num frames: 4415488. Throughput: 0: 1055.8. Samples: 101584. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:49:11,107][00788] Avg episode reward: [(0, '15.673')] [2024-12-31 06:49:16,102][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.6, 300 sec: 3704.2). Total num frames: 4431872. Throughput: 0: 996.4. Samples: 106158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:49:16,104][00788] Avg episode reward: [(0, '15.740')] [2024-12-31 06:49:20,870][12542] Updated weights for policy 0, policy_version 1088 (0.0021) [2024-12-31 06:49:21,102][00788] Fps is (10 sec: 4097.0, 60 sec: 4096.0, 300 sec: 3754.7). Total num frames: 4456448. Throughput: 0: 1007.3. Samples: 113256. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:49:21,104][00788] Avg episode reward: [(0, '16.929')] [2024-12-31 06:49:21,109][12529] Saving new best policy, reward=16.929! [2024-12-31 06:49:26,102][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3768.3). Total num frames: 4476928. Throughput: 0: 1040.1. Samples: 116922. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:49:26,110][00788] Avg episode reward: [(0, '15.702')] [2024-12-31 06:49:31,105][00788] Fps is (10 sec: 3685.0, 60 sec: 4027.5, 300 sec: 3749.3). Total num frames: 4493312. Throughput: 0: 1017.5. Samples: 122098. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2024-12-31 06:49:31,109][00788] Avg episode reward: [(0, '15.155')] [2024-12-31 06:49:31,697][12542] Updated weights for policy 0, policy_version 1098 (0.0013) [2024-12-31 06:49:36,102][00788] Fps is (10 sec: 4096.0, 60 sec: 4096.4, 300 sec: 3792.6). Total num frames: 4517888. Throughput: 0: 997.9. Samples: 128480. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:49:36,107][00788] Avg episode reward: [(0, '15.183')] [2024-12-31 06:49:40,164][12542] Updated weights for policy 0, policy_version 1108 (0.0022) [2024-12-31 06:49:41,102][00788] Fps is (10 sec: 4917.0, 60 sec: 4232.5, 300 sec: 3832.7). Total num frames: 4542464. Throughput: 0: 1029.1. Samples: 132244. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:49:41,104][00788] Avg episode reward: [(0, '15.470')] [2024-12-31 06:49:46,102][00788] Fps is (10 sec: 4095.7, 60 sec: 4096.0, 300 sec: 3813.5). Total num frames: 4558848. Throughput: 0: 1053.4. Samples: 138364. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:49:46,107][00788] Avg episode reward: [(0, '15.776')] [2024-12-31 06:49:51,102][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3795.6). Total num frames: 4575232. Throughput: 0: 1009.2. Samples: 143692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:49:51,106][00788] Avg episode reward: [(0, '15.905')] [2024-12-31 06:49:51,155][12542] Updated weights for policy 0, policy_version 1118 (0.0024) [2024-12-31 06:49:56,102][00788] Fps is (10 sec: 4505.9, 60 sec: 4164.3, 300 sec: 3858.2). Total num frames: 4603904. Throughput: 0: 1018.1. Samples: 147396. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:49:56,104][00788] Avg episode reward: [(0, '15.299')] [2024-12-31 06:49:59,384][12542] Updated weights for policy 0, policy_version 1128 (0.0024) [2024-12-31 06:50:01,102][00788] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 3865.6). Total num frames: 4624384. Throughput: 0: 1074.6. Samples: 154516. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:50:01,107][00788] Avg episode reward: [(0, '16.276')] [2024-12-31 06:50:06,102][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 3847.8). Total num frames: 4640768. Throughput: 0: 1017.1. Samples: 159026. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:50:06,110][00788] Avg episode reward: [(0, '16.406')] [2024-12-31 06:50:10,422][12542] Updated weights for policy 0, policy_version 1138 (0.0026) [2024-12-31 06:50:11,102][00788] Fps is (10 sec: 3686.4, 60 sec: 4096.2, 300 sec: 3855.1). Total num frames: 4661248. Throughput: 0: 1014.1. Samples: 162556. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2024-12-31 06:50:11,109][00788] Avg episode reward: [(0, '17.782')] [2024-12-31 06:50:11,112][12529] Saving new best policy, reward=17.782! [2024-12-31 06:50:16,102][00788] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 3908.8). Total num frames: 4689920. Throughput: 0: 1064.3. Samples: 169986. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2024-12-31 06:50:16,104][00788] Avg episode reward: [(0, '17.874')] [2024-12-31 06:50:16,123][12529] Saving new best policy, reward=17.874! [2024-12-31 06:50:20,595][12542] Updated weights for policy 0, policy_version 1148 (0.0014) [2024-12-31 06:50:21,104][00788] Fps is (10 sec: 4094.9, 60 sec: 4095.8, 300 sec: 3868.4). Total num frames: 4702208. Throughput: 0: 1030.8. Samples: 174870. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:50:21,106][00788] Avg episode reward: [(0, '18.689')] [2024-12-31 06:50:21,117][12529] Saving new best policy, reward=18.689! [2024-12-31 06:50:26,102][00788] Fps is (10 sec: 3276.8, 60 sec: 4096.0, 300 sec: 3874.6). Total num frames: 4722688. Throughput: 0: 1004.1. Samples: 177430. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:50:26,104][00788] Avg episode reward: [(0, '18.892')] [2024-12-31 06:50:26,113][12529] Saving new best policy, reward=18.892! [2024-12-31 06:50:30,092][12542] Updated weights for policy 0, policy_version 1158 (0.0014) [2024-12-31 06:50:31,102][00788] Fps is (10 sec: 4506.8, 60 sec: 4232.8, 300 sec: 3902.0). Total num frames: 4747264. Throughput: 0: 1030.5. Samples: 184734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:50:31,109][00788] Avg episode reward: [(0, '19.508')] [2024-12-31 06:50:31,112][12529] Saving new best policy, reward=19.508! [2024-12-31 06:50:36,102][00788] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3907.0). Total num frames: 4767744. Throughput: 0: 1046.4. Samples: 190778. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) [2024-12-31 06:50:36,107][00788] Avg episode reward: [(0, '20.437')] [2024-12-31 06:50:36,117][12529] Saving new best policy, reward=20.437! [2024-12-31 06:50:41,104][00788] Fps is (10 sec: 3275.9, 60 sec: 3959.3, 300 sec: 3870.7). Total num frames: 4780032. Throughput: 0: 1011.9. Samples: 192934. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2024-12-31 06:50:41,107][00788] Avg episode reward: [(0, '20.822')] [2024-12-31 06:50:41,151][12529] Saving new best policy, reward=20.822! [2024-12-31 06:50:41,159][12542] Updated weights for policy 0, policy_version 1168 (0.0051) [2024-12-31 06:50:46,102][00788] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 3916.2). Total num frames: 4808704. Throughput: 0: 1002.7. Samples: 199638. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:50:46,108][00788] Avg episode reward: [(0, '20.004')] [2024-12-31 06:50:49,549][12542] Updated weights for policy 0, policy_version 1178 (0.0025) [2024-12-31 06:50:51,102][00788] Fps is (10 sec: 4916.5, 60 sec: 4232.5, 300 sec: 3920.5). Total num frames: 4829184. Throughput: 0: 1057.2. Samples: 206598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2024-12-31 06:50:51,106][00788] Avg episode reward: [(0, '20.522')] [2024-12-31 06:50:56,104][00788] Fps is (10 sec: 3685.4, 60 sec: 4027.6, 300 sec: 3905.4). Total num frames: 4845568. Throughput: 0: 1028.8. Samples: 208856. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:50:56,106][00788] Avg episode reward: [(0, '19.478')] [2024-12-31 06:50:56,119][12529] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001183_4845568.pth... [2024-12-31 06:50:56,285][12529] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth [2024-12-31 06:51:00,697][12542] Updated weights for policy 0, policy_version 1188 (0.0046) [2024-12-31 06:51:01,102][00788] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3909.8). Total num frames: 4866048. Throughput: 0: 992.3. Samples: 214640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:51:01,109][00788] Avg episode reward: [(0, '18.701')] [2024-12-31 06:51:06,102][00788] Fps is (10 sec: 4506.8, 60 sec: 4164.3, 300 sec: 3932.2). Total num frames: 4890624. Throughput: 0: 1047.3. Samples: 221996. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:51:06,109][00788] Avg episode reward: [(0, '19.041')] [2024-12-31 06:51:10,199][12542] Updated weights for policy 0, policy_version 1198 (0.0028) [2024-12-31 06:51:11,105][00788] Fps is (10 sec: 4094.9, 60 sec: 4095.8, 300 sec: 3917.9). Total num frames: 4907008. Throughput: 0: 1056.3. Samples: 224966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:51:11,107][00788] Avg episode reward: [(0, '19.196')] [2024-12-31 06:51:16,102][00788] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3921.7). Total num frames: 4927488. Throughput: 0: 1002.6. Samples: 229852. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:51:16,108][00788] Avg episode reward: [(0, '19.040')] [2024-12-31 06:51:20,151][12542] Updated weights for policy 0, policy_version 1208 (0.0022) [2024-12-31 06:51:21,102][00788] Fps is (10 sec: 4506.8, 60 sec: 4164.5, 300 sec: 3942.4). Total num frames: 4952064. Throughput: 0: 1029.4. Samples: 237102. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:51:21,108][00788] Avg episode reward: [(0, '18.821')] [2024-12-31 06:51:26,108][00788] Fps is (10 sec: 4502.6, 60 sec: 4163.8, 300 sec: 3945.4). Total num frames: 4972544. Throughput: 0: 1065.5. Samples: 240886. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2024-12-31 06:51:26,114][00788] Avg episode reward: [(0, '17.562')] [2024-12-31 06:51:31,102][00788] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3915.8). Total num frames: 4984832. Throughput: 0: 1019.4. Samples: 245512. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2024-12-31 06:51:31,107][00788] Avg episode reward: [(0, '17.742')] [2024-12-31 06:51:31,130][12542] Updated weights for policy 0, policy_version 1218 (0.0015) [2024-12-31 06:51:34,258][12529] Stopping Batcher_0... [2024-12-31 06:51:34,259][12529] Loop batcher_evt_loop terminating... [2024-12-31 06:51:34,260][00788] Component Batcher_0 stopped! [2024-12-31 06:51:34,265][12529] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... [2024-12-31 06:51:34,314][12542] Weights refcount: 2 0 [2024-12-31 06:51:34,319][12542] Stopping InferenceWorker_p0-w0... [2024-12-31 06:51:34,320][12542] Loop inference_proc0-0_evt_loop terminating... [2024-12-31 06:51:34,319][00788] Component InferenceWorker_p0-w0 stopped! [2024-12-31 06:51:34,388][12529] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001063_4354048.pth [2024-12-31 06:51:34,404][12529] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... [2024-12-31 06:51:34,589][00788] Component LearnerWorker_p0 stopped! [2024-12-31 06:51:34,593][12529] Stopping LearnerWorker_p0... [2024-12-31 06:51:34,594][12529] Loop learner_proc0_evt_loop terminating... [2024-12-31 06:51:34,691][00788] Component RolloutWorker_w0 stopped! [2024-12-31 06:51:34,697][00788] Component RolloutWorker_w2 stopped! [2024-12-31 06:51:34,702][12545] Stopping RolloutWorker_w2... [2024-12-31 06:51:34,705][00788] Component RolloutWorker_w4 stopped! [2024-12-31 06:51:34,706][12548] Stopping RolloutWorker_w4... [2024-12-31 06:51:34,696][12543] Stopping RolloutWorker_w0... [2024-12-31 06:51:34,703][12545] Loop rollout_proc2_evt_loop terminating... [2024-12-31 06:51:34,710][12548] Loop rollout_proc4_evt_loop terminating... [2024-12-31 06:51:34,711][12543] Loop rollout_proc0_evt_loop terminating... [2024-12-31 06:51:34,722][12547] Stopping RolloutWorker_w5... [2024-12-31 06:51:34,724][12547] Loop rollout_proc5_evt_loop terminating... [2024-12-31 06:51:34,721][00788] Component RolloutWorker_w6 stopped! [2024-12-31 06:51:34,726][00788] Component RolloutWorker_w5 stopped! [2024-12-31 06:51:34,729][12549] Stopping RolloutWorker_w6... [2024-12-31 06:51:34,730][12549] Loop rollout_proc6_evt_loop terminating... [2024-12-31 06:51:34,756][12550] Stopping RolloutWorker_w7... [2024-12-31 06:51:34,757][12550] Loop rollout_proc7_evt_loop terminating... [2024-12-31 06:51:34,756][00788] Component RolloutWorker_w7 stopped! [2024-12-31 06:51:34,790][12546] Stopping RolloutWorker_w3... [2024-12-31 06:51:34,790][12546] Loop rollout_proc3_evt_loop terminating... [2024-12-31 06:51:34,790][00788] Component RolloutWorker_w3 stopped! [2024-12-31 06:51:34,823][12544] Stopping RolloutWorker_w1... [2024-12-31 06:51:34,823][00788] Component RolloutWorker_w1 stopped! [2024-12-31 06:51:34,828][00788] Waiting for process learner_proc0 to stop... [2024-12-31 06:51:34,825][12544] Loop rollout_proc1_evt_loop terminating... [2024-12-31 06:51:36,099][00788] Waiting for process inference_proc0-0 to join... [2024-12-31 06:51:36,107][00788] Waiting for process rollout_proc0 to join... [2024-12-31 06:51:38,084][00788] Waiting for process rollout_proc1 to join... [2024-12-31 06:51:38,139][00788] Waiting for process rollout_proc2 to join... [2024-12-31 06:51:38,144][00788] Waiting for process rollout_proc3 to join... [2024-12-31 06:51:38,147][00788] Waiting for process rollout_proc4 to join... [2024-12-31 06:51:38,152][00788] Waiting for process rollout_proc5 to join... [2024-12-31 06:51:38,155][00788] Waiting for process rollout_proc6 to join... [2024-12-31 06:51:38,159][00788] Waiting for process rollout_proc7 to join... [2024-12-31 06:51:38,162][00788] Batcher 0 profile tree view: batching: 6.4824, releasing_batches: 0.0070 [2024-12-31 06:51:38,163][00788] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 101.7051 update_model: 2.0691 weight_update: 0.0016 one_step: 0.0024 handle_policy_step: 141.0899 deserialize: 3.4112, stack: 0.7982, obs_to_device_normalize: 30.3189, forward: 71.0267, send_messages: 6.9623 prepare_outputs: 21.5683 to_cpu: 13.2306 [2024-12-31 06:51:38,164][00788] Learner 0 profile tree view: misc: 0.0012, prepare_batch: 5.3333 train: 22.4108 epoch_init: 0.0022, minibatch_init: 0.0015, losses_postprocess: 0.1515, kl_divergence: 0.1923, after_optimizer: 0.8858 calculate_losses: 8.3276 losses_init: 0.0032, forward_head: 0.6429, bptt_initial: 5.5336, tail: 0.3489, advantages_returns: 0.0542, losses: 1.1814 bptt: 0.4784 bptt_forward_core: 0.4428 update: 12.6843 clip: 0.2821 [2024-12-31 06:51:38,166][00788] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.0970, enqueue_policy_requests: 21.7528, env_step: 192.3859, overhead: 3.0991, complete_rollouts: 1.4554 save_policy_outputs: 5.0616 split_output_tensors: 2.0334 [2024-12-31 06:51:38,167][00788] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.0803, enqueue_policy_requests: 22.4117, env_step: 190.8220, overhead: 2.7988, complete_rollouts: 1.5903 save_policy_outputs: 5.1037 split_output_tensors: 2.2216 [2024-12-31 06:51:38,169][00788] Loop Runner_EvtLoop terminating... [2024-12-31 06:51:38,171][00788] Runner profile tree view: main_loop: 279.3732 [2024-12-31 06:51:38,172][00788] Collected {0: 5005312}, FPS: 3577.4 [2024-12-31 06:51:38,198][00788] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-12-31 06:51:38,200][00788] Overriding arg 'num_workers' with value 1 passed from command line [2024-12-31 06:51:38,201][00788] Adding new argument 'no_render'=True that is not in the saved config file! [2024-12-31 06:51:38,202][00788] Adding new argument 'save_video'=True that is not in the saved config file! [2024-12-31 06:51:38,203][00788] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-12-31 06:51:38,204][00788] Adding new argument 'video_name'=None that is not in the saved config file! [2024-12-31 06:51:38,205][00788] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-12-31 06:51:38,206][00788] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-12-31 06:51:38,207][00788] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-12-31 06:51:38,208][00788] Adding new argument 'hf_repository'='LunaMeme/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-12-31 06:51:38,209][00788] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-12-31 06:51:38,210][00788] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-12-31 06:51:38,211][00788] Adding new argument 'train_script'=None that is not in the saved config file! [2024-12-31 06:51:38,212][00788] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-12-31 06:51:38,213][00788] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-12-31 06:51:38,259][00788] RunningMeanStd input shape: (3, 72, 128) [2024-12-31 06:51:38,260][00788] RunningMeanStd input shape: (1,) [2024-12-31 06:51:38,273][00788] ConvEncoder: input_channels=3 [2024-12-31 06:51:38,310][00788] Conv encoder output size: 512 [2024-12-31 06:51:38,312][00788] Policy head output size: 512 [2024-12-31 06:51:38,330][00788] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... [2024-12-31 06:51:38,755][00788] Num frames 100... [2024-12-31 06:51:38,873][00788] Num frames 200... [2024-12-31 06:51:38,994][00788] Num frames 300... [2024-12-31 06:51:39,111][00788] Num frames 400... [2024-12-31 06:51:39,262][00788] Avg episode rewards: #0: 7.800, true rewards: #0: 4.800 [2024-12-31 06:51:39,264][00788] Avg episode reward: 7.800, avg true_objective: 4.800 [2024-12-31 06:51:39,290][00788] Num frames 500... [2024-12-31 06:51:39,415][00788] Num frames 600... [2024-12-31 06:51:39,536][00788] Num frames 700... [2024-12-31 06:51:39,657][00788] Num frames 800... [2024-12-31 06:51:39,778][00788] Num frames 900... [2024-12-31 06:51:39,900][00788] Num frames 1000... [2024-12-31 06:51:40,019][00788] Num frames 1100... [2024-12-31 06:51:40,142][00788] Num frames 1200... [2024-12-31 06:51:40,267][00788] Num frames 1300... [2024-12-31 06:51:40,398][00788] Num frames 1400... [2024-12-31 06:51:40,522][00788] Num frames 1500... [2024-12-31 06:51:40,645][00788] Num frames 1600... [2024-12-31 06:51:40,697][00788] Avg episode rewards: #0: 16.500, true rewards: #0: 8.000 [2024-12-31 06:51:40,699][00788] Avg episode reward: 16.500, avg true_objective: 8.000 [2024-12-31 06:51:40,822][00788] Num frames 1700... [2024-12-31 06:51:40,939][00788] Num frames 1800... [2024-12-31 06:51:41,062][00788] Num frames 1900... [2024-12-31 06:51:41,180][00788] Num frames 2000... [2024-12-31 06:51:41,348][00788] Num frames 2100... [2024-12-31 06:51:41,422][00788] Avg episode rewards: #0: 14.360, true rewards: #0: 7.027 [2024-12-31 06:51:41,423][00788] Avg episode reward: 14.360, avg true_objective: 7.027 [2024-12-31 06:51:41,575][00788] Num frames 2200... [2024-12-31 06:51:41,745][00788] Num frames 2300... [2024-12-31 06:51:41,907][00788] Num frames 2400... [2024-12-31 06:51:42,072][00788] Num frames 2500... [2024-12-31 06:51:42,230][00788] Num frames 2600... [2024-12-31 06:51:42,412][00788] Num frames 2700... [2024-12-31 06:51:42,574][00788] Num frames 2800... [2024-12-31 06:51:42,769][00788] Avg episode rewards: #0: 14.705, true rewards: #0: 7.205 [2024-12-31 06:51:42,770][00788] Avg episode reward: 14.705, avg true_objective: 7.205 [2024-12-31 06:51:42,801][00788] Num frames 2900... [2024-12-31 06:51:42,965][00788] Num frames 3000... [2024-12-31 06:51:43,135][00788] Num frames 3100... [2024-12-31 06:51:43,309][00788] Num frames 3200... [2024-12-31 06:51:43,498][00788] Num frames 3300... [2024-12-31 06:51:43,666][00788] Num frames 3400... [2024-12-31 06:51:43,827][00788] Num frames 3500... [2024-12-31 06:51:43,946][00788] Avg episode rewards: #0: 14.308, true rewards: #0: 7.108 [2024-12-31 06:51:43,948][00788] Avg episode reward: 14.308, avg true_objective: 7.108 [2024-12-31 06:51:44,005][00788] Num frames 3600... [2024-12-31 06:51:44,133][00788] Num frames 3700... [2024-12-31 06:51:44,253][00788] Num frames 3800... [2024-12-31 06:51:44,376][00788] Num frames 3900... [2024-12-31 06:51:44,509][00788] Num frames 4000... [2024-12-31 06:51:44,631][00788] Num frames 4100... [2024-12-31 06:51:44,754][00788] Num frames 4200... [2024-12-31 06:51:44,922][00788] Avg episode rewards: #0: 14.488, true rewards: #0: 7.155 [2024-12-31 06:51:44,924][00788] Avg episode reward: 14.488, avg true_objective: 7.155 [2024-12-31 06:51:44,934][00788] Num frames 4300... [2024-12-31 06:51:45,057][00788] Num frames 4400... [2024-12-31 06:51:45,181][00788] Num frames 4500... [2024-12-31 06:51:45,302][00788] Num frames 4600... [2024-12-31 06:51:45,431][00788] Num frames 4700... [2024-12-31 06:51:45,559][00788] Num frames 4800... [2024-12-31 06:51:45,682][00788] Num frames 4900... [2024-12-31 06:51:45,804][00788] Num frames 5000... [2024-12-31 06:51:45,927][00788] Num frames 5100... [2024-12-31 06:51:46,046][00788] Num frames 5200... [2024-12-31 06:51:46,131][00788] Avg episode rewards: #0: 15.602, true rewards: #0: 7.459 [2024-12-31 06:51:46,133][00788] Avg episode reward: 15.602, avg true_objective: 7.459 [2024-12-31 06:51:46,228][00788] Num frames 5300... [2024-12-31 06:51:46,350][00788] Num frames 5400... [2024-12-31 06:51:46,476][00788] Num frames 5500... [2024-12-31 06:51:46,604][00788] Num frames 5600... [2024-12-31 06:51:46,724][00788] Num frames 5700... [2024-12-31 06:51:46,843][00788] Num frames 5800... [2024-12-31 06:51:46,961][00788] Num frames 5900... [2024-12-31 06:51:47,081][00788] Num frames 6000... [2024-12-31 06:51:47,204][00788] Num frames 6100... [2024-12-31 06:51:47,324][00788] Num frames 6200... [2024-12-31 06:51:47,443][00788] Avg episode rewards: #0: 16.806, true rewards: #0: 7.806 [2024-12-31 06:51:47,445][00788] Avg episode reward: 16.806, avg true_objective: 7.806 [2024-12-31 06:51:47,512][00788] Num frames 6300... [2024-12-31 06:51:47,639][00788] Num frames 6400... [2024-12-31 06:51:47,765][00788] Num frames 6500... [2024-12-31 06:51:47,886][00788] Num frames 6600... [2024-12-31 06:51:48,007][00788] Num frames 6700... [2024-12-31 06:51:48,126][00788] Num frames 6800... [2024-12-31 06:51:48,244][00788] Num frames 6900... [2024-12-31 06:51:48,369][00788] Num frames 7000... [2024-12-31 06:51:48,474][00788] Avg episode rewards: #0: 16.818, true rewards: #0: 7.818 [2024-12-31 06:51:48,476][00788] Avg episode reward: 16.818, avg true_objective: 7.818 [2024-12-31 06:51:48,554][00788] Num frames 7100... [2024-12-31 06:51:48,683][00788] Num frames 7200... [2024-12-31 06:51:48,804][00788] Num frames 7300... [2024-12-31 06:51:48,937][00788] Num frames 7400... [2024-12-31 06:51:49,077][00788] Num frames 7500... [2024-12-31 06:51:49,198][00788] Num frames 7600... [2024-12-31 06:51:49,326][00788] Num frames 7700... [2024-12-31 06:51:49,467][00788] Num frames 7800... [2024-12-31 06:51:49,591][00788] Num frames 7900... [2024-12-31 06:51:49,733][00788] Num frames 8000... [2024-12-31 06:51:49,859][00788] Num frames 8100... [2024-12-31 06:51:49,986][00788] Avg episode rewards: #0: 18.160, true rewards: #0: 8.160 [2024-12-31 06:51:49,987][00788] Avg episode reward: 18.160, avg true_objective: 8.160 [2024-12-31 06:52:34,779][00788] Replay video saved to /content/train_dir/default_experiment/replay.mp4!