sf_log.txt

[2023-03-02 18:27:42,253][1037367] Saving configuration to /home/qgallouedec/train_dir/default_experiment/config.json...
[2023-03-02 18:27:42,253][1037367] Rollout worker 0 uses device cpu
[2023-03-02 18:27:42,253][1037367] Rollout worker 1 uses device cpu
[2023-03-02 18:27:42,253][1037367] Rollout worker 2 uses device cpu
[2023-03-02 18:27:42,253][1037367] Rollout worker 3 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 4 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 5 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 6 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 7 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 8 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 9 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 10 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 11 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 12 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 13 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 14 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 15 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 16 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 17 uses device cpu
[2023-03-02 18:27:42,254][1037367] Rollout worker 18 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 19 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 20 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 21 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 22 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 23 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 24 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 25 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 26 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 27 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 28 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 29 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 30 uses device cpu
[2023-03-02 18:27:42,255][1037367] Rollout worker 31 uses device cpu
[2023-03-02 18:27:42,270][1037367] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:27:42,270][1037367] InferenceWorker_p0-w0: min num requests: 10
[2023-03-02 18:27:42,335][1037367] Starting all processes...
[2023-03-02 18:27:42,335][1037367] Starting process learner_proc0
[2023-03-02 18:27:42,385][1037367] Starting all processes...
[2023-03-02 18:27:42,434][1037367] Starting process inference_proc0-0
[2023-03-02 18:27:42,434][1037367] Starting process rollout_proc0
[2023-03-02 18:27:42,434][1037367] Starting process rollout_proc1
[2023-03-02 18:27:42,434][1037367] Starting process rollout_proc2
[2023-03-02 18:27:42,435][1037367] Starting process rollout_proc3
[2023-03-02 18:27:42,435][1037367] Starting process rollout_proc4
[2023-03-02 18:27:42,437][1037367] Starting process rollout_proc5
[2023-03-02 18:27:42,438][1037367] Starting process rollout_proc6
[2023-03-02 18:27:42,438][1037367] Starting process rollout_proc7
[2023-03-02 18:27:42,438][1037367] Starting process rollout_proc8
[2023-03-02 18:27:42,449][1037367] Starting process rollout_proc9
[2023-03-02 18:27:42,451][1037367] Starting process rollout_proc10
[2023-03-02 18:27:42,452][1037367] Starting process rollout_proc11
[2023-03-02 18:27:42,452][1037367] Starting process rollout_proc12
[2023-03-02 18:27:42,452][1037367] Starting process rollout_proc13
[2023-03-02 18:27:42,457][1037367] Starting process rollout_proc14
[2023-03-02 18:27:42,459][1037367] Starting process rollout_proc15
[2023-03-02 18:27:42,460][1037367] Starting process rollout_proc16
[2023-03-02 18:27:42,561][1037367] Starting process rollout_proc31
[2023-03-02 18:27:42,464][1037367] Starting process rollout_proc18
[2023-03-02 18:27:42,470][1037367] Starting process rollout_proc19
[2023-03-02 18:27:42,475][1037367] Starting process rollout_proc20
[2023-03-02 18:27:42,483][1037367] Starting process rollout_proc21
[2023-03-02 18:27:42,486][1037367] Starting process rollout_proc22
[2023-03-02 18:27:42,502][1037367] Starting process rollout_proc24
[2023-03-02 18:27:42,509][1037367] Starting process rollout_proc25
[2023-03-02 18:27:42,496][1037367] Starting process rollout_proc23
[2023-03-02 18:27:42,517][1037367] Starting process rollout_proc26
[2023-03-02 18:27:42,526][1037367] Starting process rollout_proc27
[2023-03-02 18:27:42,544][1037367] Starting process rollout_proc29
[2023-03-02 18:27:42,535][1037367] Starting process rollout_proc28
[2023-03-02 18:27:42,552][1037367] Starting process rollout_proc30
[2023-03-02 18:27:42,463][1037367] Starting process rollout_proc17
[2023-03-02 18:27:44,359][1037628] Worker 3 uses CPU cores [3]
[2023-03-02 18:27:44,419][1037626] Worker 1 uses CPU cores [1]
[2023-03-02 18:27:44,515][1037794] Worker 14 uses CPU cores [14]
[2023-03-02 18:27:44,674][1037630] Worker 5 uses CPU cores [5]
[2023-03-02 18:27:44,678][1037934] Worker 28 uses CPU cores [28]
[2023-03-02 18:27:44,854][1037790] Worker 10 uses CPU cores [10]
[2023-03-02 18:27:44,990][1037830] Worker 21 uses CPU cores [21]
[2023-03-02 18:27:45,050][1037901] Worker 29 uses CPU cores [29]
[2023-03-02 18:27:45,278][1037793] Worker 13 uses CPU cores [13]
[2023-03-02 18:27:45,342][1037798] Worker 18 uses CPU cores [18]
[2023-03-02 18:27:45,506][1037896] Worker 22 uses CPU cores [22]
[2023-03-02 18:27:45,654][1037863] Worker 31 uses CPU cores [31]
[2023-03-02 18:27:45,666][1037795] Worker 15 uses CPU cores [15]
[2023-03-02 18:27:45,822][1037573] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:27:45,822][1037573] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-03-02 18:27:45,833][1037573] Num visible devices: 1
[2023-03-02 18:27:45,867][1037573] WARNING! It is generally recommended to enable Fixed KL loss (https://arxiv.org/pdf/1707.06347.pdf) for continuous action tasks to avoid potential numerical issues. I.e. set --kl_loss_coeff=0.1
[2023-03-02 18:27:45,867][1037573] Starting seed is not provided
[2023-03-02 18:27:45,867][1037573] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:27:45,868][1037573] Initializing actor-critic model on device cuda:0
[2023-03-02 18:27:45,868][1037573] RunningMeanStd input shape: (39,)
[2023-03-02 18:27:45,868][1037573] RunningMeanStd input shape: (1,)
[2023-03-02 18:27:45,870][1037935] Worker 17 uses CPU cores [17]
[2023-03-02 18:27:45,911][1037625] Worker 0 uses CPU cores [0]
[2023-03-02 18:27:45,999][1037573] Created Actor Critic model with architecture:
[2023-03-02 18:27:45,999][1037573] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): MultiInputEncoder(
    (encoders): ModuleDict(
      (obs): MlpEncoder(
        (mlp_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Linear)
          (3): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=8, bias=True)
  )
)
[2023-03-02 18:27:46,049][1037713] Worker 8 uses CPU cores [8]
[2023-03-02 18:27:46,106][1037792] Worker 12 uses CPU cores [12]
[2023-03-02 18:27:46,230][1037797] Worker 19 uses CPU cores [19]
[2023-03-02 18:27:46,467][1037631] Worker 6 uses CPU cores [6]
[2023-03-02 18:27:46,486][1037791] Worker 11 uses CPU cores [11]
[2023-03-02 18:27:46,583][1037899] Worker 26 uses CPU cores [26]
[2023-03-02 18:27:46,851][1037624] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:27:46,851][1037624] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-03-02 18:27:46,875][1037624] Num visible devices: 1
[2023-03-02 18:27:46,965][1037862] Worker 20 uses CPU cores [20]
[2023-03-02 18:27:46,968][1037900] Worker 27 uses CPU cores [27]
[2023-03-02 18:27:47,098][1037796] Worker 16 uses CPU cores [16]
[2023-03-02 18:27:47,176][1037864] Worker 24 uses CPU cores [24]
[2023-03-02 18:27:47,250][1037897] Worker 25 uses CPU cores [25]
[2023-03-02 18:27:47,288][1037898] Worker 23 uses CPU cores [23]
[2023-03-02 18:27:47,467][1037573] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-03-02 18:27:47,467][1037573] No checkpoints found
[2023-03-02 18:27:47,467][1037573] Did not load from checkpoint, starting from scratch!
[2023-03-02 18:27:47,467][1037573] Initialized policy 0 weights for model version 0
[2023-03-02 18:27:47,469][1037573] LearnerWorker_p0 finished initialization!
[2023-03-02 18:27:47,469][1037573] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:27:47,490][1037933] Worker 30 uses CPU cores [30]
[2023-03-02 18:27:47,505][1037694] Worker 9 uses CPU cores [9]
[2023-03-02 18:27:47,526][1037624] RunningMeanStd input shape: (39,)
[2023-03-02 18:27:47,527][1037624] RunningMeanStd input shape: (1,)
[2023-03-02 18:27:47,639][1037627] Worker 2 uses CPU cores [2]
[2023-03-02 18:27:47,693][1037629] Worker 4 uses CPU cores [4]
[2023-03-02 18:27:47,747][1037758] Worker 7 uses CPU cores [7]
[2023-03-02 18:27:48,148][1037367] Inference worker 0-0 is ready!
[2023-03-02 18:27:48,149][1037367] All inference workers are ready! Signal rollout workers to start!
[2023-03-02 18:27:49,284][1037367] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:27:49,695][1037933] EvtLoop [rollout_proc30_evt_loop, process=rollout_proc30] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,697][1037933] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc30_evt_loop
[2023-03-02 18:27:49,748][1037758] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,750][1037758] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc7_evt_loop
[2023-03-02 18:27:49,750][1037934] EvtLoop [rollout_proc28_evt_loop, process=rollout_proc28] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,752][1037934] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc28_evt_loop
[2023-03-02 18:27:49,759][1037901] EvtLoop [rollout_proc29_evt_loop, process=rollout_proc29] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,761][1037901] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc29_evt_loop
[2023-03-02 18:27:49,770][1037625] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,772][1037625] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc0_evt_loop
[2023-03-02 18:27:49,787][1037797] EvtLoop [rollout_proc19_evt_loop, process=rollout_proc19] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,789][1037797] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc19_evt_loop
[2023-03-02 18:27:49,813][1037627] EvtLoop [rollout_proc2_evt_loop, process=rollout_proc2] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,815][1037627] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc2_evt_loop
[2023-03-02 18:27:49,815][1037899] EvtLoop [rollout_proc26_evt_loop, process=rollout_proc26] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,817][1037899] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc26_evt_loop
[2023-03-02 18:27:49,817][1037629] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,817][1037900] EvtLoop [rollout_proc27_evt_loop, process=rollout_proc27] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,817][1037862] EvtLoop [rollout_proc20_evt_loop, process=rollout_proc20] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,818][1037629] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc4_evt_loop
[2023-03-02 18:27:49,818][1037900] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc27_evt_loop
[2023-03-02 18:27:49,818][1037862] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc20_evt_loop
[2023-03-02 18:27:49,840][1037796] EvtLoop [rollout_proc16_evt_loop, process=rollout_proc16] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,841][1037796] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc16_evt_loop
[2023-03-02 18:27:49,843][1037897] EvtLoop [rollout_proc25_evt_loop, process=rollout_proc25] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,845][1037897] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc25_evt_loop
[2023-03-02 18:27:49,845][1037626] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,846][1037626] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc1_evt_loop
[2023-03-02 18:27:49,847][1037935] EvtLoop [rollout_proc17_evt_loop, process=rollout_proc17] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,849][1037935] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc17_evt_loop
[2023-03-02 18:27:49,850][1037863] EvtLoop [rollout_proc31_evt_loop, process=rollout_proc31] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,850][1037795] EvtLoop [rollout_proc15_evt_loop, process=rollout_proc15] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,852][1037863] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc31_evt_loop
[2023-03-02 18:27:49,852][1037795] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc15_evt_loop
[2023-03-02 18:27:49,859][1037798] EvtLoop [rollout_proc18_evt_loop, process=rollout_proc18] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,860][1037798] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc18_evt_loop
[2023-03-02 18:27:49,860][1037791] EvtLoop [rollout_proc11_evt_loop, process=rollout_proc11] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,862][1037791] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc11_evt_loop
[2023-03-02 18:27:49,934][1037631] EvtLoop [rollout_proc6_evt_loop, process=rollout_proc6] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,936][1037631] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc6_evt_loop
[2023-03-02 18:27:49,944][1037630] EvtLoop [rollout_proc5_evt_loop, process=rollout_proc5] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,948][1037630] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc5_evt_loop
[2023-03-02 18:27:49,950][1037694] EvtLoop [rollout_proc9_evt_loop, process=rollout_proc9] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,952][1037694] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc9_evt_loop
[2023-03-02 18:27:49,963][1037830] EvtLoop [rollout_proc21_evt_loop, process=rollout_proc21] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,964][1037830] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc21_evt_loop
[2023-03-02 18:27:49,968][1037864] EvtLoop [rollout_proc24_evt_loop, process=rollout_proc24] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,970][1037864] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc24_evt_loop
[2023-03-02 18:27:49,977][1037713] EvtLoop [rollout_proc8_evt_loop, process=rollout_proc8] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,981][1037713] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc8_evt_loop
[2023-03-02 18:27:49,981][1037896] EvtLoop [rollout_proc22_evt_loop, process=rollout_proc22] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:49,983][1037896] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc22_evt_loop
[2023-03-02 18:27:50,031][1037628] EvtLoop [rollout_proc3_evt_loop, process=rollout_proc3] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,032][1037628] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc3_evt_loop
[2023-03-02 18:27:50,058][1037790] EvtLoop [rollout_proc10_evt_loop, process=rollout_proc10] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,059][1037790] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc10_evt_loop
[2023-03-02 18:27:50,069][1037898] EvtLoop [rollout_proc23_evt_loop, process=rollout_proc23] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,070][1037898] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc23_evt_loop
[2023-03-02 18:27:50,070][1037792] EvtLoop [rollout_proc12_evt_loop, process=rollout_proc12] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,072][1037792] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc12_evt_loop
[2023-03-02 18:27:50,115][1037793] EvtLoop [rollout_proc13_evt_loop, process=rollout_proc13] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,117][1037793] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc13_evt_loop
[2023-03-02 18:27:50,513][1037794] EvtLoop [rollout_proc14_evt_loop, process=rollout_proc14] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=()
Traceback (most recent call last):
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal
    slot_callable(*args)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init
    env_runner.init(self.timing)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init
    self._reset()
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset
    observations, info = e.reset(seed=seed)  # new way of doing seeding since Gym 0.26.0
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 125, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/sample_factory/algo/utils/make_env.py", line 110, in reset
    obs, info = self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/time_limit.py", line 68, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
    return self.env.reset(**kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/qgallouedec/env_sample_factory/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 192, in env_reset_passive_checker
    result = env.reset(**kwargs)
TypeError: reset() got an unexpected keyword argument 'seed'
[2023-03-02 18:27:50,519][1037794] Unhandled exception reset() got an unexpected keyword argument 'seed' in evt loop rollout_proc14_evt_loop
[2023-03-02 18:27:54,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:27:59,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:02,265][1037367] Heartbeat connected on Batcher_0
[2023-03-02 18:28:02,267][1037367] Heartbeat connected on LearnerWorker_p0
[2023-03-02 18:28:02,313][1037367] Heartbeat connected on InferenceWorker_p0-w0
[2023-03-02 18:28:04,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:09,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:14,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:19,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:24,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:29,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:34,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:39,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:44,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:49,284][1037367] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:28:53,935][1037367] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 1037367], exiting...
[2023-03-02 18:28:53,936][1037367] Runner profile tree view:
main_loop: 71.6008
[2023-03-02 18:28:53,936][1037367] Collected {0: 0}, FPS: 0.0
[2023-03-02 18:28:53,936][1037573] Stopping Batcher_0...
[2023-03-02 18:28:53,936][1037573] Loop batcher_evt_loop terminating...
[2023-03-02 18:28:53,937][1037573] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-03-02 18:28:53,972][1037573] Stopping LearnerWorker_p0...
[2023-03-02 18:28:53,972][1037573] Loop learner_proc0_evt_loop terminating...
[2023-03-02 18:28:53,990][1037624] Weights refcount: 2 0
[2023-03-02 18:28:53,991][1037624] Stopping InferenceWorker_p0-w0...
[2023-03-02 18:28:53,991][1037624] Loop inference_proc0-0_evt_loop terminating...
[2023-03-02 18:29:12,579][1041156] Saving configuration to /home/qgallouedec/train_dir/default_experiment/config.json...
[2023-03-02 18:29:12,580][1041156] Rollout worker 0 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 1 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 2 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 3 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 4 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 5 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 6 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 7 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 8 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 9 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 10 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 11 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 12 uses device cpu
[2023-03-02 18:29:12,580][1041156] Rollout worker 13 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 14 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 15 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 16 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 17 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 18 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 19 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 20 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 21 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 22 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 23 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 24 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 25 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 26 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 27 uses device cpu
[2023-03-02 18:29:12,581][1041156] Rollout worker 28 uses device cpu
[2023-03-02 18:29:12,582][1041156] Rollout worker 29 uses device cpu
[2023-03-02 18:29:12,582][1041156] Rollout worker 30 uses device cpu
[2023-03-02 18:29:12,582][1041156] Rollout worker 31 uses device cpu
[2023-03-02 18:29:12,596][1041156] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:29:12,596][1041156] InferenceWorker_p0-w0: min num requests: 10
[2023-03-02 18:29:12,662][1041156] Starting all processes...
[2023-03-02 18:29:12,662][1041156] Starting process learner_proc0
[2023-03-02 18:29:12,712][1041156] Starting all processes...
[2023-03-02 18:29:12,721][1041156] Starting process inference_proc0-0
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc0
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc1
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc2
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc3
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc4
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc5
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc6
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc7
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc8
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc9
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc10
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc11
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc12
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc13
[2023-03-02 18:29:12,722][1041156] Starting process rollout_proc14
[2023-03-02 18:29:12,737][1041156] Starting process rollout_proc16
[2023-03-02 18:29:12,730][1041156] Starting process rollout_proc15
[2023-03-02 18:29:12,744][1041156] Starting process rollout_proc17
[2023-03-02 18:29:12,761][1041156] Starting process rollout_proc18
[2023-03-02 18:29:12,781][1041156] Starting process rollout_proc19
[2023-03-02 18:29:12,784][1041156] Starting process rollout_proc20
[2023-03-02 18:29:12,789][1041156] Starting process rollout_proc21
[2023-03-02 18:29:12,797][1041156] Starting process rollout_proc22
[2023-03-02 18:29:12,803][1041156] Starting process rollout_proc23
[2023-03-02 18:29:12,825][1041156] Starting process rollout_proc24
[2023-03-02 18:29:12,840][1041156] Starting process rollout_proc25
[2023-03-02 18:29:12,851][1041156] Starting process rollout_proc26
[2023-03-02 18:29:12,858][1041156] Starting process rollout_proc27
[2023-03-02 18:29:12,859][1041156] Starting process rollout_proc28
[2023-03-02 18:29:12,860][1041156] Starting process rollout_proc29
[2023-03-02 18:29:12,862][1041156] Starting process rollout_proc30
[2023-03-02 18:29:12,957][1041156] Starting process rollout_proc31
[2023-03-02 18:29:14,702][1041406] Worker 5 uses CPU cores [5]
[2023-03-02 18:29:14,825][1041399] Worker 0 uses CPU cores [0]
[2023-03-02 18:29:14,869][1041633] Worker 18 uses CPU cores [18]
[2023-03-02 18:29:15,038][1041568] Worker 13 uses CPU cores [13]
[2023-03-02 18:29:15,126][1041768] Worker 29 uses CPU cores [29]
[2023-03-02 18:29:15,162][1041634] Worker 19 uses CPU cores [19]
[2023-03-02 18:29:15,386][1041400] Worker 1 uses CPU cores [1]
[2023-03-02 18:29:15,463][1041803] Worker 25 uses CPU cores [25]
[2023-03-02 18:29:15,554][1041403] Worker 6 uses CPU cores [6]
[2023-03-02 18:29:15,612][1041735] Worker 28 uses CPU cores [28]
[2023-03-02 18:29:15,641][1041771] Worker 24 uses CPU cores [24]
[2023-03-02 18:29:15,915][1041699] Worker 22 uses CPU cores [22]
[2023-03-02 18:29:15,972][1041470] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:29:15,972][1041470] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-03-02 18:29:15,982][1041470] Num visible devices: 1
[2023-03-02 18:29:16,066][1041769] Worker 30 uses CPU cores [30]
[2023-03-02 18:29:16,283][1041507] Worker 4 uses CPU cores [4]
[2023-03-02 18:29:16,338][1041601] Worker 15 uses CPU cores [15]
[2023-03-02 18:29:16,476][1041701] Worker 23 uses CPU cores [23]
[2023-03-02 18:29:16,546][1041698] Worker 14 uses CPU cores [14]
[2023-03-02 18:29:16,641][1041402] Worker 3 uses CPU cores [3]
[2023-03-02 18:29:16,735][1041703] Worker 27 uses CPU cores [27]
[2023-03-02 18:29:16,922][1041566] Worker 11 uses CPU cores [11]
[2023-03-02 18:29:17,072][1041348] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:29:17,072][1041348] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-03-02 18:29:17,082][1041348] Num visible devices: 1
[2023-03-02 18:29:17,088][1041404] Worker 8 uses CPU cores [8]
[2023-03-02 18:29:17,111][1041348] WARNING! It is generally recommended to enable Fixed KL loss (https://arxiv.org/pdf/1707.06347.pdf) for continuous action tasks to avoid potential numerical issues. I.e. set --kl_loss_coeff=0.1
[2023-03-02 18:29:17,111][1041348] Starting seed is not provided
[2023-03-02 18:29:17,111][1041348] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:29:17,111][1041348] Initializing actor-critic model on device cuda:0
[2023-03-02 18:29:17,112][1041348] RunningMeanStd input shape: (39,)
[2023-03-02 18:29:17,112][1041348] RunningMeanStd input shape: (1,)
[2023-03-02 18:29:17,206][1041348] Created Actor Critic model with architecture:
[2023-03-02 18:29:17,206][1041348] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): MultiInputEncoder(
    (encoders): ModuleDict(
      (obs): MlpEncoder(
        (mlp_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Linear)
          (3): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=8, bias=True)
  )
)
[2023-03-02 18:29:17,280][1041539] Worker 12 uses CPU cores [12]
[2023-03-02 18:29:17,350][1041405] Worker 7 uses CPU cores [7]
[2023-03-02 18:29:17,527][1041702] Worker 20 uses CPU cores [20]
[2023-03-02 18:29:17,556][1041767] Worker 26 uses CPU cores [26]
[2023-03-02 18:29:17,626][1041600] Worker 16 uses CPU cores [16]
[2023-03-02 18:29:17,738][1041567] Worker 9 uses CPU cores [9]
[2023-03-02 18:29:17,891][1041697] Worker 17 uses CPU cores [17]
[2023-03-02 18:29:18,018][1041401] Worker 2 uses CPU cores [2]
[2023-03-02 18:29:18,126][1041770] Worker 31 uses CPU cores [31]
[2023-03-02 18:29:18,297][1041407] Worker 10 uses CPU cores [10]
[2023-03-02 18:29:18,337][1041700] Worker 21 uses CPU cores [21]
[2023-03-02 18:29:18,445][1041348] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-03-02 18:29:18,446][1041348] Loading state from checkpoint /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-03-02 18:29:18,450][1041348] Loading model from checkpoint
[2023-03-02 18:29:18,451][1041348] Loaded experiment state at self.train_step=0, self.env_steps=0
[2023-03-02 18:29:18,451][1041348] Initialized policy 0 weights for model version 0
[2023-03-02 18:29:18,453][1041348] LearnerWorker_p0 finished initialization!
[2023-03-02 18:29:18,453][1041348] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:29:18,524][1041470] RunningMeanStd input shape: (39,)
[2023-03-02 18:29:18,525][1041470] RunningMeanStd input shape: (1,)
[2023-03-02 18:29:19,153][1041156] Inference worker 0-0 is ready!
[2023-03-02 18:29:19,153][1041156] All inference workers are ready! Signal rollout workers to start!
[2023-03-02 18:29:19,652][1041156] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:29:20,736][1041698] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,754][1041770] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,772][1041701] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,797][1041404] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,802][1041697] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,805][1041699] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,839][1041768] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,856][1041735] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,859][1041702] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,868][1041767] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,876][1041400] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,877][1041771] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,885][1041700] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,886][1041406] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,889][1041566] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,889][1041703] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,892][1041803] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,908][1041399] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,909][1041401] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,911][1041633] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,931][1041634] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,944][1041407] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,947][1041568] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,964][1041402] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,973][1041405] Decorrelating experience for 0 frames...
[2023-03-02 18:29:20,998][1041567] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,001][1041769] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,007][1041507] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,011][1041539] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,106][1041600] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,115][1041403] Decorrelating experience for 0 frames...
[2023-03-02 18:29:21,226][1041601] Decorrelating experience for 0 frames...
[2023-03-02 18:29:22,292][1041770] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,309][1041698] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,349][1041701] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,396][1041768] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,405][1041404] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,405][1041735] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,413][1041697] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,436][1041399] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,448][1041400] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,472][1041634] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,478][1041767] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,496][1041700] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,499][1041803] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,503][1041406] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,517][1041566] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,517][1041407] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,517][1041703] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,543][1041568] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,545][1041567] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,547][1041771] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,551][1041401] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,552][1041633] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,568][1041539] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,574][1041405] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,581][1041699] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,584][1041702] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,584][1041507] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,660][1041769] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,688][1041403] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,717][1041402] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,768][1041600] Decorrelating experience for 32 frames...
[2023-03-02 18:29:22,835][1041601] Decorrelating experience for 32 frames...
[2023-03-02 18:29:23,020][1041348] Signal inference workers to stop experience collection...
[2023-03-02 18:29:23,023][1041470] InferenceWorker_p0-w0: stopping experience collection
[2023-03-02 18:29:23,374][1041348] Signal inference workers to resume experience collection...
[2023-03-02 18:29:23,374][1041470] InferenceWorker_p0-w0: resuming experience collection
[2023-03-02 18:29:24,581][1041470] Updated weights for policy 0, policy_version 10 (0.0212)
[2023-03-02 18:29:24,652][1041156] Fps is (10 sec: 2252.9, 60 sec: 2252.9, 300 sec: 2252.9). Total num frames: 11264. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
[2023-03-02 18:29:25,407][1041470] Updated weights for policy 0, policy_version 20 (0.0007)
[2023-03-02 18:29:26,248][1041470] Updated weights for policy 0, policy_version 30 (0.0007)
[2023-03-02 18:29:27,088][1041470] Updated weights for policy 0, policy_version 40 (0.0007)
[2023-03-02 18:29:27,920][1041470] Updated weights for policy 0, policy_version 50 (0.0007)
[2023-03-02 18:29:28,750][1041470] Updated weights for policy 0, policy_version 60 (0.0007)
[2023-03-02 18:29:29,585][1041470] Updated weights for policy 0, policy_version 70 (0.0006)
[2023-03-02 18:29:29,652][1041156] Fps is (10 sec: 7168.1, 60 sec: 7168.1, 300 sec: 7168.1). Total num frames: 71680. Throughput: 0: 5557.1. Samples: 55570. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:29:29,652][1041156] Avg episode reward: [(0, '6.776')]
[2023-03-02 18:29:30,399][1041470] Updated weights for policy 0, policy_version 80 (0.0006)
[2023-03-02 18:29:31,229][1041470] Updated weights for policy 0, policy_version 90 (0.0006)
[2023-03-02 18:29:32,072][1041470] Updated weights for policy 0, policy_version 100 (0.0006)
[2023-03-02 18:29:32,591][1041156] Heartbeat connected on Batcher_0
[2023-03-02 18:29:32,594][1041156] Heartbeat connected on LearnerWorker_p0
[2023-03-02 18:29:32,599][1041156] Heartbeat connected on RolloutWorker_w0
[2023-03-02 18:29:32,600][1041156] Heartbeat connected on InferenceWorker_p0-w0
[2023-03-02 18:29:32,601][1041156] Heartbeat connected on RolloutWorker_w1
[2023-03-02 18:29:32,604][1041156] Heartbeat connected on RolloutWorker_w2
[2023-03-02 18:29:32,605][1041156] Heartbeat connected on RolloutWorker_w3
[2023-03-02 18:29:32,608][1041156] Heartbeat connected on RolloutWorker_w4
[2023-03-02 18:29:32,611][1041156] Heartbeat connected on RolloutWorker_w6
[2023-03-02 18:29:32,614][1041156] Heartbeat connected on RolloutWorker_w7
[2023-03-02 18:29:32,616][1041156] Heartbeat connected on RolloutWorker_w8
[2023-03-02 18:29:32,619][1041156] Heartbeat connected on RolloutWorker_w10
[2023-03-02 18:29:32,620][1041156] Heartbeat connected on RolloutWorker_w9
[2023-03-02 18:29:32,620][1041156] Heartbeat connected on RolloutWorker_w5
[2023-03-02 18:29:32,622][1041156] Heartbeat connected on RolloutWorker_w11
[2023-03-02 18:29:32,623][1041156] Heartbeat connected on RolloutWorker_w12
[2023-03-02 18:29:32,626][1041156] Heartbeat connected on RolloutWorker_w13
[2023-03-02 18:29:32,628][1041156] Heartbeat connected on RolloutWorker_w14
[2023-03-02 18:29:32,629][1041156] Heartbeat connected on RolloutWorker_w15
[2023-03-02 18:29:32,631][1041156] Heartbeat connected on RolloutWorker_w16
[2023-03-02 18:29:32,634][1041156] Heartbeat connected on RolloutWorker_w17
[2023-03-02 18:29:32,635][1041156] Heartbeat connected on RolloutWorker_w18
[2023-03-02 18:29:32,637][1041156] Heartbeat connected on RolloutWorker_w19
[2023-03-02 18:29:32,639][1041156] Heartbeat connected on RolloutWorker_w20
[2023-03-02 18:29:32,641][1041156] Heartbeat connected on RolloutWorker_w21
[2023-03-02 18:29:32,645][1041156] Heartbeat connected on RolloutWorker_w22
[2023-03-02 18:29:32,645][1041156] Heartbeat connected on RolloutWorker_w23
[2023-03-02 18:29:32,649][1041156] Heartbeat connected on RolloutWorker_w24
[2023-03-02 18:29:32,649][1041156] Heartbeat connected on RolloutWorker_w25
[2023-03-02 18:29:32,651][1041156] Heartbeat connected on RolloutWorker_w26
[2023-03-02 18:29:32,654][1041156] Heartbeat connected on RolloutWorker_w27
[2023-03-02 18:29:32,655][1041156] Heartbeat connected on RolloutWorker_w28
[2023-03-02 18:29:32,658][1041156] Heartbeat connected on RolloutWorker_w29
[2023-03-02 18:29:32,660][1041156] Heartbeat connected on RolloutWorker_w30
[2023-03-02 18:29:32,663][1041156] Heartbeat connected on RolloutWorker_w31
[2023-03-02 18:29:32,889][1041470] Updated weights for policy 0, policy_version 110 (0.0006)
[2023-03-02 18:29:33,723][1041470] Updated weights for policy 0, policy_version 120 (0.0007)
[2023-03-02 18:29:34,567][1041470] Updated weights for policy 0, policy_version 130 (0.0006)
[2023-03-02 18:29:34,652][1041156] Fps is (10 sec: 12185.5, 60 sec: 8874.8, 300 sec: 8874.8). Total num frames: 133120. Throughput: 0: 8620.8. Samples: 129310. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:34,652][1041156] Avg episode reward: [(0, '11.346')]
[2023-03-02 18:29:34,659][1041348] Saving new best policy, reward=11.346!
[2023-03-02 18:29:35,365][1041470] Updated weights for policy 0, policy_version 140 (0.0006)
[2023-03-02 18:29:36,204][1041470] Updated weights for policy 0, policy_version 150 (0.0006)
[2023-03-02 18:29:37,057][1041470] Updated weights for policy 0, policy_version 160 (0.0006)
[2023-03-02 18:29:37,884][1041470] Updated weights for policy 0, policy_version 170 (0.0006)
[2023-03-02 18:29:38,687][1041470] Updated weights for policy 0, policy_version 180 (0.0006)
[2023-03-02 18:29:39,521][1041470] Updated weights for policy 0, policy_version 190 (0.0006)
[2023-03-02 18:29:39,652][1041156] Fps is (10 sec: 12390.4, 60 sec: 9779.3, 300 sec: 9779.3). Total num frames: 195584. Throughput: 0: 8317.0. Samples: 166339. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:39,652][1041156] Avg episode reward: [(0, '16.348')]
[2023-03-02 18:29:39,655][1041348] Saving new best policy, reward=16.348!
[2023-03-02 18:29:40,340][1041470] Updated weights for policy 0, policy_version 200 (0.0006)
[2023-03-02 18:29:41,144][1041470] Updated weights for policy 0, policy_version 210 (0.0007)
[2023-03-02 18:29:41,968][1041470] Updated weights for policy 0, policy_version 220 (0.0006)
[2023-03-02 18:29:42,777][1041470] Updated weights for policy 0, policy_version 230 (0.0006)
[2023-03-02 18:29:43,619][1041470] Updated weights for policy 0, policy_version 240 (0.0008)
[2023-03-02 18:29:44,440][1041470] Updated weights for policy 0, policy_version 250 (0.0007)
[2023-03-02 18:29:44,652][1041156] Fps is (10 sec: 12492.9, 60 sec: 10322.0, 300 sec: 10322.0). Total num frames: 258048. Throughput: 0: 9655.9. Samples: 241394. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:44,652][1041156] Avg episode reward: [(0, '18.740')]
[2023-03-02 18:29:44,652][1041348] Saving new best policy, reward=18.740!
[2023-03-02 18:29:45,246][1041470] Updated weights for policy 0, policy_version 260 (0.0007)
[2023-03-02 18:29:46,072][1041470] Updated weights for policy 0, policy_version 270 (0.0006)
[2023-03-02 18:29:46,915][1041470] Updated weights for policy 0, policy_version 280 (0.0007)
[2023-03-02 18:29:47,729][1041470] Updated weights for policy 0, policy_version 290 (0.0007)
[2023-03-02 18:29:48,542][1041470] Updated weights for policy 0, policy_version 300 (0.0007)
[2023-03-02 18:29:49,366][1041470] Updated weights for policy 0, policy_version 310 (0.0006)
[2023-03-02 18:29:49,652][1041156] Fps is (10 sec: 12492.9, 60 sec: 10683.8, 300 sec: 10683.8). Total num frames: 320512. Throughput: 0: 10536.5. Samples: 316092. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:49,652][1041156] Avg episode reward: [(0, '19.417')]
[2023-03-02 18:29:49,655][1041348] Saving new best policy, reward=19.417!
[2023-03-02 18:29:50,178][1041470] Updated weights for policy 0, policy_version 320 (0.0007)
[2023-03-02 18:29:51,011][1041470] Updated weights for policy 0, policy_version 330 (0.0007)
[2023-03-02 18:29:51,839][1041470] Updated weights for policy 0, policy_version 340 (0.0006)
[2023-03-02 18:29:52,654][1041470] Updated weights for policy 0, policy_version 350 (0.0007)
[2023-03-02 18:29:53,478][1041470] Updated weights for policy 0, policy_version 360 (0.0006)
[2023-03-02 18:29:54,287][1041470] Updated weights for policy 0, policy_version 370 (0.0007)
[2023-03-02 18:29:54,652][1041156] Fps is (10 sec: 12492.7, 60 sec: 10942.2, 300 sec: 10942.2). Total num frames: 382976. Throughput: 0: 10100.3. Samples: 353507. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:54,652][1041156] Avg episode reward: [(0, '18.078')]
[2023-03-02 18:29:55,119][1041470] Updated weights for policy 0, policy_version 380 (0.0007)
[2023-03-02 18:29:55,918][1041470] Updated weights for policy 0, policy_version 390 (0.0006)
[2023-03-02 18:29:56,746][1041470] Updated weights for policy 0, policy_version 400 (0.0006)
[2023-03-02 18:29:57,563][1041470] Updated weights for policy 0, policy_version 410 (0.0006)
[2023-03-02 18:29:58,388][1041470] Updated weights for policy 0, policy_version 420 (0.0007)
[2023-03-02 18:29:59,208][1041470] Updated weights for policy 0, policy_version 430 (0.0007)
[2023-03-02 18:29:59,652][1041156] Fps is (10 sec: 12492.7, 60 sec: 11136.1, 300 sec: 11136.1). Total num frames: 445440. Throughput: 0: 10714.5. Samples: 428578. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:29:59,652][1041156] Avg episode reward: [(0, '22.284')]
[2023-03-02 18:29:59,655][1041348] Saving new best policy, reward=22.284!
[2023-03-02 18:30:00,044][1041470] Updated weights for policy 0, policy_version 440 (0.0007)
[2023-03-02 18:30:00,863][1041470] Updated weights for policy 0, policy_version 450 (0.0006)
[2023-03-02 18:30:01,702][1041470] Updated weights for policy 0, policy_version 460 (0.0006)
[2023-03-02 18:30:02,551][1041470] Updated weights for policy 0, policy_version 470 (0.0007)
[2023-03-02 18:30:03,377][1041470] Updated weights for policy 0, policy_version 480 (0.0006)
[2023-03-02 18:30:04,199][1041470] Updated weights for policy 0, policy_version 490 (0.0006)
[2023-03-02 18:30:04,652][1041156] Fps is (10 sec: 12390.4, 60 sec: 11264.1, 300 sec: 11264.1). Total num frames: 506880. Throughput: 0: 11169.5. Samples: 502623. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:04,652][1041156] Avg episode reward: [(0, '27.631')]
[2023-03-02 18:30:04,652][1041348] Saving new best policy, reward=27.631!
[2023-03-02 18:30:05,041][1041470] Updated weights for policy 0, policy_version 500 (0.0006)
[2023-03-02 18:30:05,889][1041470] Updated weights for policy 0, policy_version 510 (0.0006)
[2023-03-02 18:30:06,704][1041470] Updated weights for policy 0, policy_version 520 (0.0007)
[2023-03-02 18:30:07,530][1041470] Updated weights for policy 0, policy_version 530 (0.0006)
[2023-03-02 18:30:08,365][1041470] Updated weights for policy 0, policy_version 540 (0.0006)
[2023-03-02 18:30:09,185][1041470] Updated weights for policy 0, policy_version 550 (0.0007)
[2023-03-02 18:30:09,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 11366.4, 300 sec: 11366.4). Total num frames: 568320. Throughput: 0: 11991.2. Samples: 539602. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:30:09,652][1041156] Avg episode reward: [(0, '24.110')]
[2023-03-02 18:30:10,027][1041470] Updated weights for policy 0, policy_version 560 (0.0006)
[2023-03-02 18:30:10,846][1041470] Updated weights for policy 0, policy_version 570 (0.0007)
[2023-03-02 18:30:11,651][1041470] Updated weights for policy 0, policy_version 580 (0.0006)
[2023-03-02 18:30:12,473][1041470] Updated weights for policy 0, policy_version 590 (0.0007)
[2023-03-02 18:30:13,310][1041470] Updated weights for policy 0, policy_version 600 (0.0007)
[2023-03-02 18:30:14,132][1041470] Updated weights for policy 0, policy_version 610 (0.0007)
[2023-03-02 18:30:14,651][1041156] Fps is (10 sec: 12390.5, 60 sec: 11468.9, 300 sec: 11468.9). Total num frames: 630784. Throughput: 0: 12406.1. Samples: 613845. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:30:14,652][1041156] Avg episode reward: [(0, '24.942')]
[2023-03-02 18:30:14,952][1041470] Updated weights for policy 0, policy_version 620 (0.0007)
[2023-03-02 18:30:15,790][1041470] Updated weights for policy 0, policy_version 630 (0.0006)
[2023-03-02 18:30:16,623][1041470] Updated weights for policy 0, policy_version 640 (0.0007)
[2023-03-02 18:30:17,458][1041470] Updated weights for policy 0, policy_version 650 (0.0006)
[2023-03-02 18:30:18,309][1041470] Updated weights for policy 0, policy_version 660 (0.0006)
[2023-03-02 18:30:19,150][1041470] Updated weights for policy 0, policy_version 670 (0.0007)
[2023-03-02 18:30:19,652][1041156] Fps is (10 sec: 12390.4, 60 sec: 11537.1, 300 sec: 11537.1). Total num frames: 692224. Throughput: 0: 12405.8. Samples: 687572. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:19,652][1041156] Avg episode reward: [(0, '37.811')]
[2023-03-02 18:30:19,665][1041348] Saving new best policy, reward=37.811!
[2023-03-02 18:30:19,982][1041470] Updated weights for policy 0, policy_version 680 (0.0006)
[2023-03-02 18:30:20,839][1041470] Updated weights for policy 0, policy_version 690 (0.0006)
[2023-03-02 18:30:21,655][1041470] Updated weights for policy 0, policy_version 700 (0.0006)
[2023-03-02 18:30:22,476][1041470] Updated weights for policy 0, policy_version 710 (0.0007)
[2023-03-02 18:30:23,313][1041470] Updated weights for policy 0, policy_version 720 (0.0007)
[2023-03-02 18:30:24,144][1041470] Updated weights for policy 0, policy_version 730 (0.0006)
[2023-03-02 18:30:24,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12373.3, 300 sec: 11594.9). Total num frames: 753664. Throughput: 0: 12399.4. Samples: 724313. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:24,652][1041156] Avg episode reward: [(0, '28.162')]
[2023-03-02 18:30:24,981][1041470] Updated weights for policy 0, policy_version 740 (0.0006)
[2023-03-02 18:30:25,823][1041470] Updated weights for policy 0, policy_version 750 (0.0006)
[2023-03-02 18:30:26,656][1041470] Updated weights for policy 0, policy_version 760 (0.0006)
[2023-03-02 18:30:27,483][1041470] Updated weights for policy 0, policy_version 770 (0.0007)
[2023-03-02 18:30:28,316][1041470] Updated weights for policy 0, policy_version 780 (0.0006)
[2023-03-02 18:30:29,160][1041470] Updated weights for policy 0, policy_version 790 (0.0006)
[2023-03-02 18:30:29,651][1041156] Fps is (10 sec: 12288.1, 60 sec: 12390.4, 300 sec: 11644.4). Total num frames: 815104. Throughput: 0: 12371.6. Samples: 798116. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:29,652][1041156] Avg episode reward: [(0, '23.295')]
[2023-03-02 18:30:29,981][1041470] Updated weights for policy 0, policy_version 800 (0.0007)
[2023-03-02 18:30:30,813][1041470] Updated weights for policy 0, policy_version 810 (0.0007)
[2023-03-02 18:30:31,625][1041470] Updated weights for policy 0, policy_version 820 (0.0006)
[2023-03-02 18:30:32,443][1041470] Updated weights for policy 0, policy_version 830 (0.0007)
[2023-03-02 18:30:33,254][1041470] Updated weights for policy 0, policy_version 840 (0.0006)
[2023-03-02 18:30:34,091][1041470] Updated weights for policy 0, policy_version 850 (0.0007)
[2023-03-02 18:30:34,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12390.4, 300 sec: 11687.3). Total num frames: 876544. Throughput: 0: 12364.9. Samples: 872511. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:34,652][1041156] Avg episode reward: [(0, '30.840')]
[2023-03-02 18:30:34,929][1041470] Updated weights for policy 0, policy_version 860 (0.0007)
[2023-03-02 18:30:35,799][1041470] Updated weights for policy 0, policy_version 870 (0.0007)
[2023-03-02 18:30:36,649][1041470] Updated weights for policy 0, policy_version 880 (0.0007)
[2023-03-02 18:30:37,472][1041470] Updated weights for policy 0, policy_version 890 (0.0007)
[2023-03-02 18:30:38,305][1041470] Updated weights for policy 0, policy_version 900 (0.0007)
[2023-03-02 18:30:39,149][1041470] Updated weights for policy 0, policy_version 910 (0.0006)
[2023-03-02 18:30:39,652][1041156] Fps is (10 sec: 12185.4, 60 sec: 12356.3, 300 sec: 11712.0). Total num frames: 936960. Throughput: 0: 12335.0. Samples: 908583. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:30:39,652][1041156] Avg episode reward: [(0, '30.760')]
[2023-03-02 18:30:40,006][1041470] Updated weights for policy 0, policy_version 920 (0.0006)
[2023-03-02 18:30:40,832][1041470] Updated weights for policy 0, policy_version 930 (0.0007)
[2023-03-02 18:30:41,688][1041470] Updated weights for policy 0, policy_version 940 (0.0007)
[2023-03-02 18:30:42,501][1041470] Updated weights for policy 0, policy_version 950 (0.0006)
[2023-03-02 18:30:43,308][1041470] Updated weights for policy 0, policy_version 960 (0.0006)
[2023-03-02 18:30:44,155][1041470] Updated weights for policy 0, policy_version 970 (0.0006)
[2023-03-02 18:30:44,651][1041156] Fps is (10 sec: 12185.7, 60 sec: 12339.2, 300 sec: 11745.9). Total num frames: 998400. Throughput: 0: 12304.6. Samples: 982283. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:44,652][1041156] Avg episode reward: [(0, '31.192')]
[2023-03-02 18:30:44,989][1041470] Updated weights for policy 0, policy_version 980 (0.0006)
[2023-03-02 18:30:45,832][1041470] Updated weights for policy 0, policy_version 990 (0.0007)
[2023-03-02 18:30:46,653][1041470] Updated weights for policy 0, policy_version 1000 (0.0006)
[2023-03-02 18:30:47,506][1041470] Updated weights for policy 0, policy_version 1010 (0.0006)
[2023-03-02 18:30:48,330][1041470] Updated weights for policy 0, policy_version 1020 (0.0006)
[2023-03-02 18:30:49,144][1041470] Updated weights for policy 0, policy_version 1030 (0.0006)
[2023-03-02 18:30:49,651][1041156] Fps is (10 sec: 12390.5, 60 sec: 12339.2, 300 sec: 11787.4). Total num frames: 1060864. Throughput: 0: 12299.3. Samples: 1056092. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:30:49,652][1041156] Avg episode reward: [(0, '32.916')]
[2023-03-02 18:30:49,982][1041470] Updated weights for policy 0, policy_version 1040 (0.0007)
[2023-03-02 18:30:50,805][1041470] Updated weights for policy 0, policy_version 1050 (0.0007)
[2023-03-02 18:30:51,623][1041470] Updated weights for policy 0, policy_version 1060 (0.0007)
[2023-03-02 18:30:52,416][1041470] Updated weights for policy 0, policy_version 1070 (0.0006)
[2023-03-02 18:30:53,226][1041470] Updated weights for policy 0, policy_version 1080 (0.0007)
[2023-03-02 18:30:54,085][1041470] Updated weights for policy 0, policy_version 1090 (0.0006)
[2023-03-02 18:30:54,652][1041156] Fps is (10 sec: 12492.7, 60 sec: 12339.2, 300 sec: 11824.5). Total num frames: 1123328. Throughput: 0: 12313.9. Samples: 1093727. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:30:54,652][1041156] Avg episode reward: [(0, '26.459')]
[2023-03-02 18:30:54,915][1041470] Updated weights for policy 0, policy_version 1100 (0.0006)
[2023-03-02 18:30:55,754][1041470] Updated weights for policy 0, policy_version 1110 (0.0006)
[2023-03-02 18:30:56,594][1041470] Updated weights for policy 0, policy_version 1120 (0.0006)
[2023-03-02 18:30:57,420][1041470] Updated weights for policy 0, policy_version 1130 (0.0006)
[2023-03-02 18:30:58,248][1041470] Updated weights for policy 0, policy_version 1140 (0.0006)
[2023-03-02 18:30:59,071][1041470] Updated weights for policy 0, policy_version 1150 (0.0006)
[2023-03-02 18:30:59,652][1041156] Fps is (10 sec: 12287.8, 60 sec: 12305.1, 300 sec: 11837.5). Total num frames: 1183744. Throughput: 0: 12308.1. Samples: 1167709. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:30:59,652][1041156] Avg episode reward: [(0, '22.995')]
[2023-03-02 18:30:59,915][1041470] Updated weights for policy 0, policy_version 1160 (0.0007)
[2023-03-02 18:31:00,748][1041470] Updated weights for policy 0, policy_version 1170 (0.0007)
[2023-03-02 18:31:01,582][1041470] Updated weights for policy 0, policy_version 1180 (0.0007)
[2023-03-02 18:31:02,407][1041470] Updated weights for policy 0, policy_version 1190 (0.0006)
[2023-03-02 18:31:03,250][1041470] Updated weights for policy 0, policy_version 1200 (0.0006)
[2023-03-02 18:31:04,068][1041470] Updated weights for policy 0, policy_version 1210 (0.0007)
[2023-03-02 18:31:04,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12322.1, 300 sec: 11868.7). Total num frames: 1246208. Throughput: 0: 12304.7. Samples: 1241284. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:31:04,652][1041156] Avg episode reward: [(0, '33.392')]
[2023-03-02 18:31:04,915][1041470] Updated weights for policy 0, policy_version 1220 (0.0007)
[2023-03-02 18:31:05,723][1041470] Updated weights for policy 0, policy_version 1230 (0.0007)
[2023-03-02 18:31:06,555][1041470] Updated weights for policy 0, policy_version 1240 (0.0007)
[2023-03-02 18:31:07,368][1041470] Updated weights for policy 0, policy_version 1250 (0.0006)
[2023-03-02 18:31:08,187][1041470] Updated weights for policy 0, policy_version 1260 (0.0006)
[2023-03-02 18:31:09,017][1041470] Updated weights for policy 0, policy_version 1270 (0.0007)
[2023-03-02 18:31:09,651][1041156] Fps is (10 sec: 12390.5, 60 sec: 12322.2, 300 sec: 11887.7). Total num frames: 1307648. Throughput: 0: 12311.8. Samples: 1278344. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:31:09,652][1041156] Avg episode reward: [(0, '24.287')]
[2023-03-02 18:31:09,655][1041348] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000001277_1307648.pth...
[2023-03-02 18:31:09,849][1041470] Updated weights for policy 0, policy_version 1280 (0.0006)
[2023-03-02 18:31:10,685][1041470] Updated weights for policy 0, policy_version 1290 (0.0007)
[2023-03-02 18:31:11,545][1041470] Updated weights for policy 0, policy_version 1300 (0.0006)
[2023-03-02 18:31:12,373][1041470] Updated weights for policy 0, policy_version 1310 (0.0007)
[2023-03-02 18:31:13,204][1041470] Updated weights for policy 0, policy_version 1320 (0.0006)
[2023-03-02 18:31:14,045][1041470] Updated weights for policy 0, policy_version 1330 (0.0006)
[2023-03-02 18:31:14,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12305.1, 300 sec: 11905.1). Total num frames: 1369088. Throughput: 0: 12316.8. Samples: 1352372. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:31:14,652][1041156] Avg episode reward: [(0, '24.567')]
[2023-03-02 18:31:14,859][1041470] Updated weights for policy 0, policy_version 1340 (0.0007)
[2023-03-02 18:31:15,707][1041470] Updated weights for policy 0, policy_version 1350 (0.0007)
[2023-03-02 18:31:16,536][1041470] Updated weights for policy 0, policy_version 1360 (0.0006)
[2023-03-02 18:31:17,360][1041470] Updated weights for policy 0, policy_version 1370 (0.0006)
[2023-03-02 18:31:18,188][1041470] Updated weights for policy 0, policy_version 1380 (0.0006)
[2023-03-02 18:31:19,038][1041470] Updated weights for policy 0, policy_version 1390 (0.0006)
[2023-03-02 18:31:19,652][1041156] Fps is (10 sec: 12287.9, 60 sec: 12305.1, 300 sec: 11921.1). Total num frames: 1430528. Throughput: 0: 12305.1. Samples: 1426239. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:31:19,652][1041156] Avg episode reward: [(0, '25.044')]
[2023-03-02 18:31:19,866][1041470] Updated weights for policy 0, policy_version 1400 (0.0007)
[2023-03-02 18:31:20,697][1041470] Updated weights for policy 0, policy_version 1410 (0.0008)
[2023-03-02 18:31:21,521][1041470] Updated weights for policy 0, policy_version 1420 (0.0007)
[2023-03-02 18:31:22,390][1041470] Updated weights for policy 0, policy_version 1430 (0.0005)
[2023-03-02 18:31:23,203][1041470] Updated weights for policy 0, policy_version 1440 (0.0007)
[2023-03-02 18:31:24,057][1041470] Updated weights for policy 0, policy_version 1450 (0.0008)
[2023-03-02 18:31:24,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12305.1, 300 sec: 11935.8). Total num frames: 1491968. Throughput: 0: 12325.3. Samples: 1463221. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:31:24,652][1041156] Avg episode reward: [(0, '21.030')]
[2023-03-02 18:31:24,893][1041470] Updated weights for policy 0, policy_version 1460 (0.0009)
[2023-03-02 18:31:25,683][1041470] Updated weights for policy 0, policy_version 1470 (0.0006)
[2023-03-02 18:31:26,536][1041470] Updated weights for policy 0, policy_version 1480 (0.0007)
[2023-03-02 18:31:27,348][1041470] Updated weights for policy 0, policy_version 1490 (0.0006)
[2023-03-02 18:31:28,161][1041470] Updated weights for policy 0, policy_version 1500 (0.0006)
[2023-03-02 18:31:28,973][1041470] Updated weights for policy 0, policy_version 1510 (0.0007)
[2023-03-02 18:31:29,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12305.0, 300 sec: 11949.3). Total num frames: 1553408. Throughput: 0: 12330.8. Samples: 1537168. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:31:29,652][1041156] Avg episode reward: [(0, '22.561')]
[2023-03-02 18:31:29,837][1041470] Updated weights for policy 0, policy_version 1520 (0.0007)
[2023-03-02 18:31:30,669][1041470] Updated weights for policy 0, policy_version 1530 (0.0006)
[2023-03-02 18:31:31,512][1041470] Updated weights for policy 0, policy_version 1540 (0.0006)
[2023-03-02 18:31:32,334][1041470] Updated weights for policy 0, policy_version 1550 (0.0006)
[2023-03-02 18:31:33,192][1041470] Updated weights for policy 0, policy_version 1560 (0.0006)
[2023-03-02 18:31:34,016][1041470] Updated weights for policy 0, policy_version 1570 (0.0007)
[2023-03-02 18:31:34,652][1041156] Fps is (10 sec: 12288.0, 60 sec: 12305.1, 300 sec: 11961.9). Total num frames: 1614848. Throughput: 0: 12320.2. Samples: 1610504. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
[2023-03-02 18:31:34,652][1041156] Avg episode reward: [(0, '29.190')]
[2023-03-02 18:31:34,880][1041470] Updated weights for policy 0, policy_version 1580 (0.0006)
[2023-03-02 18:31:35,724][1041470] Updated weights for policy 0, policy_version 1590 (0.0006)
[2023-03-02 18:31:36,452][1041156] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 1041156], exiting...
[2023-03-02 18:31:36,452][1041700] Stopping RolloutWorker_w21...
[2023-03-02 18:31:36,452][1041567] Stopping RolloutWorker_w9...
[2023-03-02 18:31:36,452][1041404] Stopping RolloutWorker_w8...
[2023-03-02 18:31:36,452][1041703] Stopping RolloutWorker_w27...
[2023-03-02 18:31:36,452][1041601] Stopping RolloutWorker_w15...
[2023-03-02 18:31:36,453][1041404] Loop rollout_proc8_evt_loop terminating...
[2023-03-02 18:31:36,452][1041702] Stopping RolloutWorker_w20...
[2023-03-02 18:31:36,452][1041568] Stopping RolloutWorker_w13...
[2023-03-02 18:31:36,452][1041401] Stopping RolloutWorker_w2...
[2023-03-02 18:31:36,452][1041402] Stopping RolloutWorker_w3...
[2023-03-02 18:31:36,452][1041405] Stopping RolloutWorker_w7...
[2023-03-02 18:31:36,452][1041700] Loop rollout_proc21_evt_loop terminating...
[2023-03-02 18:31:36,452][1041156] Runner profile tree view:
main_loop: 143.7902
[2023-03-02 18:31:36,453][1041567] Loop rollout_proc9_evt_loop terminating...
[2023-03-02 18:31:36,452][1041803] Stopping RolloutWorker_w25...
[2023-03-02 18:31:36,453][1041703] Loop rollout_proc27_evt_loop terminating...
[2023-03-02 18:31:36,452][1041539] Stopping RolloutWorker_w12...
[2023-03-02 18:31:36,452][1041403] Stopping RolloutWorker_w6...
[2023-03-02 18:31:36,452][1041769] Stopping RolloutWorker_w30...
[2023-03-02 18:31:36,452][1041767] Stopping RolloutWorker_w26...
[2023-03-02 18:31:36,452][1041633] Stopping RolloutWorker_w18...
[2023-03-02 18:31:36,452][1041771] Stopping RolloutWorker_w24...
[2023-03-02 18:31:36,452][1041400] Stopping RolloutWorker_w1...
[2023-03-02 18:31:36,453][1041735] Stopping RolloutWorker_w28...
[2023-03-02 18:31:36,452][1041566] Stopping RolloutWorker_w11...
[2023-03-02 18:31:36,453][1041702] Loop rollout_proc20_evt_loop terminating...
[2023-03-02 18:31:36,453][1041405] Loop rollout_proc7_evt_loop terminating...
[2023-03-02 18:31:36,453][1041601] Loop rollout_proc15_evt_loop terminating...
[2023-03-02 18:31:36,453][1041568] Loop rollout_proc13_evt_loop terminating...
[2023-03-02 18:31:36,452][1041768] Stopping RolloutWorker_w29...
[2023-03-02 18:31:36,452][1041770] Stopping RolloutWorker_w31...
[2023-03-02 18:31:36,453][1041399] Stopping RolloutWorker_w0...
[2023-03-02 18:31:36,453][1041156] Collected {0: 1636352}, FPS: 11380.1
[2023-03-02 18:31:36,452][1041701] Stopping RolloutWorker_w23...
[2023-03-02 18:31:36,453][1041402] Loop rollout_proc3_evt_loop terminating...
[2023-03-02 18:31:36,453][1041401] Loop rollout_proc2_evt_loop terminating...
[2023-03-02 18:31:36,453][1041407] Stopping RolloutWorker_w10...
[2023-03-02 18:31:36,453][1041403] Loop rollout_proc6_evt_loop terminating...
[2023-03-02 18:31:36,453][1041507] Stopping RolloutWorker_w4...
[2023-03-02 18:31:36,453][1041400] Loop rollout_proc1_evt_loop terminating...
[2023-03-02 18:31:36,453][1041735] Loop rollout_proc28_evt_loop terminating...
[2023-03-02 18:31:36,453][1041769] Loop rollout_proc30_evt_loop terminating...
[2023-03-02 18:31:36,453][1041633] Loop rollout_proc18_evt_loop terminating...
[2023-03-02 18:31:36,453][1041699] Stopping RolloutWorker_w22...
[2023-03-02 18:31:36,453][1041539] Loop rollout_proc12_evt_loop terminating...
[2023-03-02 18:31:36,453][1041768] Loop rollout_proc29_evt_loop terminating...
[2023-03-02 18:31:36,453][1041803] Loop rollout_proc25_evt_loop terminating...
[2023-03-02 18:31:36,453][1041399] Loop rollout_proc0_evt_loop terminating...
[2023-03-02 18:31:36,453][1041771] Loop rollout_proc24_evt_loop terminating...
[2023-03-02 18:31:36,453][1041770] Loop rollout_proc31_evt_loop terminating...
[2023-03-02 18:31:36,453][1041566] Loop rollout_proc11_evt_loop terminating...
[2023-03-02 18:31:36,453][1041701] Loop rollout_proc23_evt_loop terminating...
[2023-03-02 18:31:36,453][1041407] Loop rollout_proc10_evt_loop terminating...
[2023-03-02 18:31:36,453][1041507] Loop rollout_proc4_evt_loop terminating...
[2023-03-02 18:31:36,453][1041767] Loop rollout_proc26_evt_loop terminating...
[2023-03-02 18:31:36,453][1041699] Loop rollout_proc22_evt_loop terminating...
[2023-03-02 18:31:36,453][1041634] Stopping RolloutWorker_w19...
[2023-03-02 18:31:36,454][1041634] Loop rollout_proc19_evt_loop terminating...
[2023-03-02 18:31:36,456][1041600] Stopping RolloutWorker_w16...
[2023-03-02 18:31:36,457][1041600] Loop rollout_proc16_evt_loop terminating...
[2023-03-02 18:31:36,457][1041348] Stopping Batcher_0...
[2023-03-02 18:31:36,458][1041348] Loop batcher_evt_loop terminating...
[2023-03-02 18:31:36,459][1041697] Stopping RolloutWorker_w17...
[2023-03-02 18:31:36,460][1041697] Loop rollout_proc17_evt_loop terminating...
[2023-03-02 18:31:36,462][1041698] Stopping RolloutWorker_w14...
[2023-03-02 18:31:36,463][1041698] Loop rollout_proc14_evt_loop terminating...
[2023-03-02 18:31:36,473][1041406] Stopping RolloutWorker_w5...
[2023-03-02 18:31:36,474][1041406] Loop rollout_proc5_evt_loop terminating...
[2023-03-02 18:31:36,483][1041348] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000001599_1637376.pth...
[2023-03-02 18:31:36,516][1041470] Weights refcount: 2 0
[2023-03-02 18:31:36,524][1041470] Stopping InferenceWorker_p0-w0...
[2023-03-02 18:31:36,525][1041470] Loop inference_proc0-0_evt_loop terminating...
[2023-03-02 18:31:36,599][1041348] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth
[2023-03-02 18:31:36,603][1041348] Stopping LearnerWorker_p0...
[2023-03-02 18:31:36,603][1041348] Loop learner_proc0_evt_loop terminating...
[2023-03-02 18:32:22,274][1045180] Saving configuration to /home/qgallouedec/train_dir/default_experiment/config.json...
[2023-03-02 18:32:22,275][1045180] Rollout worker 0 uses device cpu
[2023-03-02 18:32:22,275][1045180] Rollout worker 1 uses device cpu
[2023-03-02 18:32:22,275][1045180] Rollout worker 2 uses device cpu
[2023-03-02 18:32:22,275][1045180] Rollout worker 3 uses device cpu
[2023-03-02 18:32:22,275][1045180] Rollout worker 4 uses device cpu
[2023-03-02 18:32:22,275][1045180] Rollout worker 5 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 6 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 7 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 8 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 9 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 10 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 11 uses device cpu
[2023-03-02 18:32:22,276][1045180] Rollout worker 12 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 13 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 14 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 15 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 16 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 17 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 18 uses device cpu
[2023-03-02 18:32:22,277][1045180] Rollout worker 19 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 20 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 21 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 22 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 23 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 24 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 25 uses device cpu
[2023-03-02 18:32:22,278][1045180] Rollout worker 26 uses device cpu
[2023-03-02 18:32:22,279][1045180] Rollout worker 27 uses device cpu
[2023-03-02 18:32:22,279][1045180] Rollout worker 28 uses device cpu
[2023-03-02 18:32:22,279][1045180] Rollout worker 29 uses device cpu
[2023-03-02 18:32:22,279][1045180] Rollout worker 30 uses device cpu
[2023-03-02 18:32:22,279][1045180] Rollout worker 31 uses device cpu
[2023-03-02 18:32:22,294][1045180] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:32:22,294][1045180] InferenceWorker_p0-w0: min num requests: 10
[2023-03-02 18:32:22,360][1045180] Starting all processes...
[2023-03-02 18:32:22,360][1045180] Starting process learner_proc0
[2023-03-02 18:32:22,410][1045180] Starting all processes...
[2023-03-02 18:32:22,460][1045180] Starting process inference_proc0-0
[2023-03-02 18:32:22,468][1045180] Starting process rollout_proc0
[2023-03-02 18:32:22,468][1045180] Starting process rollout_proc1
[2023-03-02 18:32:22,468][1045180] Starting process rollout_proc2
[2023-03-02 18:32:22,468][1045180] Starting process rollout_proc3
[2023-03-02 18:32:22,469][1045180] Starting process rollout_proc4
[2023-03-02 18:32:22,471][1045180] Starting process rollout_proc5
[2023-03-02 18:32:22,477][1045180] Starting process rollout_proc6
[2023-03-02 18:32:22,477][1045180] Starting process rollout_proc7
[2023-03-02 18:32:22,478][1045180] Starting process rollout_proc8
[2023-03-02 18:32:22,480][1045180] Starting process rollout_proc9
[2023-03-02 18:32:22,480][1045180] Starting process rollout_proc10
[2023-03-02 18:32:22,485][1045180] Starting process rollout_proc11
[2023-03-02 18:32:22,490][1045180] Starting process rollout_proc12
[2023-03-02 18:32:22,490][1045180] Starting process rollout_proc13
[2023-03-02 18:32:22,490][1045180] Starting process rollout_proc14
[2023-03-02 18:32:22,491][1045180] Starting process rollout_proc15
[2023-03-02 18:32:22,491][1045180] Starting process rollout_proc16
[2023-03-02 18:32:22,494][1045180] Starting process rollout_proc17
[2023-03-02 18:32:22,501][1045180] Starting process rollout_proc18
[2023-03-02 18:32:22,502][1045180] Starting process rollout_proc19
[2023-03-02 18:32:22,502][1045180] Starting process rollout_proc20
[2023-03-02 18:32:22,509][1045180] Starting process rollout_proc21
[2023-03-02 18:32:22,526][1045180] Starting process rollout_proc22
[2023-03-02 18:32:22,529][1045180] Starting process rollout_proc23
[2023-03-02 18:32:22,560][1045180] Starting process rollout_proc24
[2023-03-02 18:32:22,565][1045180] Starting process rollout_proc25
[2023-03-02 18:32:22,580][1045180] Starting process rollout_proc26
[2023-03-02 18:32:22,580][1045180] Starting process rollout_proc27
[2023-03-02 18:32:22,612][1045180] Starting process rollout_proc28
[2023-03-02 18:32:22,613][1045180] Starting process rollout_proc29
[2023-03-02 18:32:22,614][1045180] Starting process rollout_proc30
[2023-03-02 18:32:22,619][1045180] Starting process rollout_proc31
[2023-03-02 18:32:24,266][1045448] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:32:24,266][1045448] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-03-02 18:32:24,277][1045448] Num visible devices: 1
[2023-03-02 18:32:24,332][1045448] WARNING! It is generally recommended to enable Fixed KL loss (https://arxiv.org/pdf/1707.06347.pdf) for continuous action tasks to avoid potential numerical issues. I.e. set --kl_loss_coeff=0.1
[2023-03-02 18:32:24,333][1045448] Starting seed is not provided
[2023-03-02 18:32:24,333][1045448] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:32:24,333][1045448] Initializing actor-critic model on device cuda:0
[2023-03-02 18:32:24,333][1045448] RunningMeanStd input shape: (39,)
[2023-03-02 18:32:24,334][1045448] RunningMeanStd input shape: (1,)
[2023-03-02 18:32:24,464][1045448] Created Actor Critic model with architecture:
[2023-03-02 18:32:24,465][1045448] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): MultiInputEncoder(
    (encoders): ModuleDict(
      (obs): MlpEncoder(
        (mlp_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Linear)
          (3): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=8, bias=True)
  )
)
[2023-03-02 18:32:24,527][1045507] Worker 6 uses CPU cores [6]
[2023-03-02 18:32:24,616][1045667] Worker 13 uses CPU cores [13]
[2023-03-02 18:32:24,727][1045501] Worker 1 uses CPU cores [1]
[2023-03-02 18:32:24,727][1045665] Worker 9 uses CPU cores [9]
[2023-03-02 18:32:25,022][1045670] Worker 16 uses CPU cores [16]
[2023-03-02 18:32:25,098][1045500] Worker 0 uses CPU cores [0]
[2023-03-02 18:32:25,108][1045998] Worker 25 uses CPU cores [25]
[2023-03-02 18:32:25,259][1045504] Worker 4 uses CPU cores [4]
[2023-03-02 18:32:25,445][1045666] Worker 8 uses CPU cores [8]
[2023-03-02 18:32:25,530][1045933] Worker 27 uses CPU cores [27]
[2023-03-02 18:32:25,534][1045932] Worker 28 uses CPU cores [28]
[2023-03-02 18:32:25,798][1045671] Worker 18 uses CPU cores [18]
[2023-03-02 18:32:25,871][1045499] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:32:25,871][1045499] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-03-02 18:32:25,882][1045499] Num visible devices: 1
[2023-03-02 18:32:25,953][1045770] Worker 21 uses CPU cores [21]
[2023-03-02 18:32:25,961][1045668] Worker 12 uses CPU cores [12]
[2023-03-02 18:32:26,222][1045664] Worker 11 uses CPU cores [11]
[2023-03-02 18:32:26,223][1045448] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-03-02 18:32:26,223][1045448] Loading state from checkpoint /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000001599_1637376.pth...
[2023-03-02 18:32:26,246][1045448] Loading model from checkpoint
[2023-03-02 18:32:26,263][1045448] Loaded experiment state at self.train_step=1599, self.env_steps=1637376
[2023-03-02 18:32:26,271][1045448] Initialized policy 0 weights for model version 1599
[2023-03-02 18:32:26,286][1045448] LearnerWorker_p0 finished initialization!
[2023-03-02 18:32:26,287][1045448] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-03-02 18:32:26,317][1045834] Worker 22 uses CPU cores [22]
[2023-03-02 18:32:26,368][1045499] RunningMeanStd input shape: (39,)
[2023-03-02 18:32:26,368][1045499] RunningMeanStd input shape: (1,)
[2023-03-02 18:32:26,442][1045706] Worker 19 uses CPU cores [19]
[2023-03-02 18:32:26,542][1045738] Worker 20 uses CPU cores [20]
[2023-03-02 18:32:26,576][1045669] Worker 17 uses CPU cores [17]
[2023-03-02 18:32:26,722][1045705] Worker 15 uses CPU cores [15]
[2023-03-02 18:32:26,790][1045502] Worker 2 uses CPU cores [2]
[2023-03-02 18:32:26,835][1045930] Worker 26 uses CPU cores [26]
[2023-03-02 18:32:27,003][1045578] Worker 7 uses CPU cores [7]
[2023-03-02 18:32:27,034][1045929] Worker 24 uses CPU cores [24]
[2023-03-02 18:32:27,262][1045601] Worker 10 uses CPU cores [10]
[2023-03-02 18:32:27,271][1045503] Worker 3 uses CPU cores [3]
[2023-03-02 18:32:27,275][1045180] Inference worker 0-0 is ready!
[2023-03-02 18:32:27,275][1045180] All inference workers are ready! Signal rollout workers to start!
[2023-03-02 18:32:27,458][1046030] Worker 31 uses CPU cores [31]
[2023-03-02 18:32:27,771][1045965] Worker 29 uses CPU cores [29]
[2023-03-02 18:32:27,885][1045897] Worker 23 uses CPU cores [23]
[2023-03-02 18:32:27,910][1045673] Worker 14 uses CPU cores [14]
[2023-03-02 18:32:28,151][1045505] Worker 5 uses CPU cores [5]
[2023-03-02 18:32:28,219][1045997] Worker 30 uses CPU cores [30]
[2023-03-02 18:32:28,652][1045770] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,673][1045578] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,701][1045667] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,827][1045668] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,857][1045834] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,873][1045671] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,888][1045502] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,890][1045930] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,896][1045998] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,897][1045665] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,904][1045706] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,906][1045933] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,913][1045501] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,913][1045669] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,938][1045507] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,940][1045738] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,959][1045929] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,964][1045705] Decorrelating experience for 0 frames...
[2023-03-02 18:32:28,982][1045670] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,004][1045500] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,007][1045666] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,031][1045932] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,059][1045504] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,186][1046030] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,207][1045503] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,215][1045601] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,313][1045180] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 1637376. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2023-03-02 18:32:29,476][1045965] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,511][1045664] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,675][1045897] Decorrelating experience for 0 frames...
[2023-03-02 18:32:29,822][1045673] Decorrelating experience for 0 frames...
[2023-03-02 18:32:30,126][1045505] Decorrelating experience for 0 frames...
[2023-03-02 18:32:30,214][1045770] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,232][1045578] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,241][1045997] Decorrelating experience for 0 frames...
[2023-03-02 18:32:30,388][1045667] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,404][1045834] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,437][1045930] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,438][1045998] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,460][1045668] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,469][1045671] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,471][1045669] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,471][1045502] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,474][1045706] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,500][1045507] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,537][1045500] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,542][1045501] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,550][1045932] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,575][1045705] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,578][1045933] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,618][1045929] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,645][1045504] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,662][1045666] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,677][1046030] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,681][1045665] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,697][1045503] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,716][1045738] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,727][1045670] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,735][1045601] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,916][1045965] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,938][1045664] Decorrelating experience for 32 frames...
[2023-03-02 18:32:30,944][1045897] Decorrelating experience for 32 frames...
[2023-03-02 18:32:31,067][1045448] Signal inference workers to stop experience collection...
[2023-03-02 18:32:31,071][1045499] InferenceWorker_p0-w0: stopping experience collection
[2023-03-02 18:32:31,250][1045673] Decorrelating experience for 32 frames...
[2023-03-02 18:32:31,274][1045505] Decorrelating experience for 32 frames...
[2023-03-02 18:32:31,344][1045448] Signal inference workers to resume experience collection...
[2023-03-02 18:32:31,345][1045499] InferenceWorker_p0-w0: resuming experience collection
[2023-03-02 18:32:31,510][1045997] Decorrelating experience for 32 frames...
[2023-03-02 18:32:32,593][1045499] Updated weights for policy 0, policy_version 1609 (0.0221)
[2023-03-02 18:32:33,440][1045499] Updated weights for policy 0, policy_version 1619 (0.0007)
[2023-03-02 18:32:34,284][1045499] Updated weights for policy 0, policy_version 1629 (0.0007)
[2023-03-02 18:32:34,313][1045180] Fps is (10 sec: 6144.2, 60 sec: 6144.2, 300 sec: 6144.2). Total num frames: 1668096. Throughput: 0: 3999.1. Samples: 19995. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
[2023-03-02 18:32:34,314][1045180] Avg episode reward: [(0, '28.052')]
[2023-03-02 18:32:35,152][1045499] Updated weights for policy 0, policy_version 1639 (0.0006)
[2023-03-02 18:32:35,961][1045499] Updated weights for policy 0, policy_version 1649 (0.0007)
[2023-03-02 18:32:36,806][1045499] Updated weights for policy 0, policy_version 1659 (0.0007)
[2023-03-02 18:32:37,621][1045499] Updated weights for policy 0, policy_version 1669 (0.0006)
[2023-03-02 18:32:38,466][1045499] Updated weights for policy 0, policy_version 1679 (0.0006)
[2023-03-02 18:32:39,291][1045499] Updated weights for policy 0, policy_version 1689 (0.0007)
[2023-03-02 18:32:39,313][1045180] Fps is (10 sec: 9216.1, 60 sec: 9216.1, 300 sec: 9216.1). Total num frames: 1729536. Throughput: 0: 9334.2. Samples: 93341. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:32:39,314][1045180] Avg episode reward: [(0, '31.189')]
[2023-03-02 18:32:40,122][1045499] Updated weights for policy 0, policy_version 1699 (0.0007)
[2023-03-02 18:32:40,969][1045499] Updated weights for policy 0, policy_version 1709 (0.0007)
[2023-03-02 18:32:41,799][1045499] Updated weights for policy 0, policy_version 1719 (0.0007)
[2023-03-02 18:32:42,289][1045180] Heartbeat connected on Batcher_0
[2023-03-02 18:32:42,291][1045180] Heartbeat connected on LearnerWorker_p0
[2023-03-02 18:32:42,296][1045180] Heartbeat connected on RolloutWorker_w0
[2023-03-02 18:32:42,297][1045180] Heartbeat connected on InferenceWorker_p0-w0
[2023-03-02 18:32:42,298][1045180] Heartbeat connected on RolloutWorker_w1
[2023-03-02 18:32:42,302][1045180] Heartbeat connected on RolloutWorker_w2
[2023-03-02 18:32:42,302][1045180] Heartbeat connected on RolloutWorker_w3
[2023-03-02 18:32:42,307][1045180] Heartbeat connected on RolloutWorker_w4
[2023-03-02 18:32:42,307][1045180] Heartbeat connected on RolloutWorker_w5
[2023-03-02 18:32:42,309][1045180] Heartbeat connected on RolloutWorker_w6
[2023-03-02 18:32:42,311][1045180] Heartbeat connected on RolloutWorker_w7
[2023-03-02 18:32:42,315][1045180] Heartbeat connected on RolloutWorker_w9
[2023-03-02 18:32:42,320][1045180] Heartbeat connected on RolloutWorker_w11
[2023-03-02 18:32:42,321][1045180] Heartbeat connected on RolloutWorker_w12
[2023-03-02 18:32:42,323][1045180] Heartbeat connected on RolloutWorker_w13
[2023-03-02 18:32:42,325][1045180] Heartbeat connected on RolloutWorker_w10
[2023-03-02 18:32:42,326][1045180] Heartbeat connected on RolloutWorker_w14
[2023-03-02 18:32:42,327][1045180] Heartbeat connected on RolloutWorker_w15
[2023-03-02 18:32:42,329][1045180] Heartbeat connected on RolloutWorker_w16
[2023-03-02 18:32:42,330][1045180] Heartbeat connected on RolloutWorker_w8
[2023-03-02 18:32:42,331][1045180] Heartbeat connected on RolloutWorker_w17
[2023-03-02 18:32:42,333][1045180] Heartbeat connected on RolloutWorker_w18
[2023-03-02 18:32:42,336][1045180] Heartbeat connected on RolloutWorker_w19
[2023-03-02 18:32:42,337][1045180] Heartbeat connected on RolloutWorker_w20
[2023-03-02 18:32:42,340][1045180] Heartbeat connected on RolloutWorker_w22
[2023-03-02 18:32:42,342][1045180] Heartbeat connected on RolloutWorker_w21
[2023-03-02 18:32:42,343][1045180] Heartbeat connected on RolloutWorker_w23
[2023-03-02 18:32:42,345][1045180] Heartbeat connected on RolloutWorker_w24
[2023-03-02 18:32:42,346][1045180] Heartbeat connected on RolloutWorker_w25
[2023-03-02 18:32:42,348][1045180] Heartbeat connected on RolloutWorker_w26
[2023-03-02 18:32:42,352][1045180] Heartbeat connected on RolloutWorker_w27
[2023-03-02 18:32:42,353][1045180] Heartbeat connected on RolloutWorker_w28
[2023-03-02 18:32:42,354][1045180] Heartbeat connected on RolloutWorker_w29
[2023-03-02 18:32:42,357][1045180] Heartbeat connected on RolloutWorker_w30
[2023-03-02 18:32:42,358][1045180] Heartbeat connected on RolloutWorker_w31
[2023-03-02 18:32:42,660][1045499] Updated weights for policy 0, policy_version 1729 (0.0006)
[2023-03-02 18:32:43,492][1045499] Updated weights for policy 0, policy_version 1739 (0.0006)
[2023-03-02 18:32:44,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 10240.1, 300 sec: 10240.1). Total num frames: 1790976. Throughput: 0: 8660.7. Samples: 129910. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:32:44,314][1045180] Avg episode reward: [(0, '27.934')]
[2023-03-02 18:32:44,315][1045499] Updated weights for policy 0, policy_version 1749 (0.0007)
[2023-03-02 18:32:45,142][1045499] Updated weights for policy 0, policy_version 1759 (0.0007)
[2023-03-02 18:32:45,969][1045499] Updated weights for policy 0, policy_version 1769 (0.0006)
[2023-03-02 18:32:46,800][1045499] Updated weights for policy 0, policy_version 1779 (0.0007)
[2023-03-02 18:32:47,637][1045499] Updated weights for policy 0, policy_version 1789 (0.0008)
[2023-03-02 18:32:48,459][1045499] Updated weights for policy 0, policy_version 1799 (0.0006)
[2023-03-02 18:32:49,262][1045499] Updated weights for policy 0, policy_version 1809 (0.0007)
[2023-03-02 18:32:49,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 10752.1, 300 sec: 10752.1). Total num frames: 1852416. Throughput: 0: 10205.5. Samples: 204110. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:32:49,314][1045180] Avg episode reward: [(0, '26.200')]
[2023-03-02 18:32:50,097][1045499] Updated weights for policy 0, policy_version 1819 (0.0007)
[2023-03-02 18:32:50,900][1045499] Updated weights for policy 0, policy_version 1829 (0.0006)
[2023-03-02 18:32:51,717][1045499] Updated weights for policy 0, policy_version 1839 (0.0007)
[2023-03-02 18:32:52,571][1045499] Updated weights for policy 0, policy_version 1849 (0.0008)
[2023-03-02 18:32:53,403][1045499] Updated weights for policy 0, policy_version 1859 (0.0006)
[2023-03-02 18:32:54,275][1045499] Updated weights for policy 0, policy_version 1869 (0.0006)
[2023-03-02 18:32:54,313][1045180] Fps is (10 sec: 12287.8, 60 sec: 11059.2, 300 sec: 11059.2). Total num frames: 1913856. Throughput: 0: 11121.8. Samples: 278046. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:32:54,314][1045180] Avg episode reward: [(0, '27.038')]
[2023-03-02 18:32:55,144][1045499] Updated weights for policy 0, policy_version 1879 (0.0007)
[2023-03-02 18:32:55,959][1045499] Updated weights for policy 0, policy_version 1889 (0.0006)
[2023-03-02 18:32:56,821][1045499] Updated weights for policy 0, policy_version 1899 (0.0007)
[2023-03-02 18:32:57,656][1045499] Updated weights for policy 0, policy_version 1909 (0.0007)
[2023-03-02 18:32:58,479][1045499] Updated weights for policy 0, policy_version 1919 (0.0007)
[2023-03-02 18:32:59,311][1045499] Updated weights for policy 0, policy_version 1929 (0.0007)
[2023-03-02 18:32:59,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 11264.0, 300 sec: 11264.0). Total num frames: 1975296. Throughput: 0: 10472.7. Samples: 314181. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:32:59,314][1045180] Avg episode reward: [(0, '30.690')]
[2023-03-02 18:33:00,170][1045499] Updated weights for policy 0, policy_version 1939 (0.0007)
[2023-03-02 18:33:00,999][1045499] Updated weights for policy 0, policy_version 1949 (0.0008)
[2023-03-02 18:33:01,827][1045499] Updated weights for policy 0, policy_version 1959 (0.0007)
[2023-03-02 18:33:02,642][1045499] Updated weights for policy 0, policy_version 1969 (0.0007)
[2023-03-02 18:33:03,507][1045499] Updated weights for policy 0, policy_version 1979 (0.0006)
[2023-03-02 18:33:04,313][1045180] Fps is (10 sec: 12185.7, 60 sec: 11381.0, 300 sec: 11381.0). Total num frames: 2035712. Throughput: 0: 11079.9. Samples: 387797. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:33:04,314][1045180] Avg episode reward: [(0, '41.201')]
[2023-03-02 18:33:04,314][1045448] Saving new best policy, reward=41.201!
[2023-03-02 18:33:04,369][1045499] Updated weights for policy 0, policy_version 1989 (0.0006)
[2023-03-02 18:33:05,165][1045499] Updated weights for policy 0, policy_version 1999 (0.0006)
[2023-03-02 18:33:05,991][1045499] Updated weights for policy 0, policy_version 2009 (0.0007)
[2023-03-02 18:33:06,863][1045499] Updated weights for policy 0, policy_version 2019 (0.0006)
[2023-03-02 18:33:07,660][1045499] Updated weights for policy 0, policy_version 2029 (0.0006)
[2023-03-02 18:33:08,507][1045499] Updated weights for policy 0, policy_version 2039 (0.0006)
[2023-03-02 18:33:09,313][1045180] Fps is (10 sec: 12185.4, 60 sec: 11494.4, 300 sec: 11494.4). Total num frames: 2097152. Throughput: 0: 11538.3. Samples: 461533. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:33:09,314][1045180] Avg episode reward: [(0, '40.773')]
[2023-03-02 18:33:09,322][1045499] Updated weights for policy 0, policy_version 2049 (0.0008)
[2023-03-02 18:33:10,151][1045499] Updated weights for policy 0, policy_version 2059 (0.0007)
[2023-03-02 18:33:10,984][1045499] Updated weights for policy 0, policy_version 2069 (0.0007)
[2023-03-02 18:33:11,809][1045499] Updated weights for policy 0, policy_version 2079 (0.0007)
[2023-03-02 18:33:12,638][1045499] Updated weights for policy 0, policy_version 2089 (0.0007)
[2023-03-02 18:33:13,490][1045499] Updated weights for policy 0, policy_version 2099 (0.0007)
[2023-03-02 18:33:14,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 11582.6, 300 sec: 11582.6). Total num frames: 2158592. Throughput: 0: 11080.8. Samples: 498637. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:33:14,314][1045180] Avg episode reward: [(0, '27.381')]
[2023-03-02 18:33:14,317][1045499] Updated weights for policy 0, policy_version 2109 (0.0006)
[2023-03-02 18:33:15,138][1045499] Updated weights for policy 0, policy_version 2119 (0.0007)
[2023-03-02 18:33:15,965][1045499] Updated weights for policy 0, policy_version 2129 (0.0006)
[2023-03-02 18:33:16,792][1045499] Updated weights for policy 0, policy_version 2139 (0.0007)
[2023-03-02 18:33:17,635][1045499] Updated weights for policy 0, policy_version 2149 (0.0009)
[2023-03-02 18:33:18,464][1045499] Updated weights for policy 0, policy_version 2159 (0.0007)
[2023-03-02 18:33:19,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 11653.1, 300 sec: 11653.1). Total num frames: 2220032. Throughput: 0: 12273.1. Samples: 572287. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:19,314][1045180] Avg episode reward: [(0, '33.109')]
[2023-03-02 18:33:19,327][1045499] Updated weights for policy 0, policy_version 2169 (0.0007)
[2023-03-02 18:33:20,138][1045499] Updated weights for policy 0, policy_version 2179 (0.0006)
[2023-03-02 18:33:20,983][1045499] Updated weights for policy 0, policy_version 2189 (0.0007)
[2023-03-02 18:33:21,812][1045499] Updated weights for policy 0, policy_version 2199 (0.0007)
[2023-03-02 18:33:22,645][1045499] Updated weights for policy 0, policy_version 2209 (0.0006)
[2023-03-02 18:33:23,468][1045499] Updated weights for policy 0, policy_version 2219 (0.0007)
[2023-03-02 18:33:24,308][1045499] Updated weights for policy 0, policy_version 2229 (0.0007)
[2023-03-02 18:33:24,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 11729.5, 300 sec: 11729.5). Total num frames: 2282496. Throughput: 0: 12281.5. Samples: 646009. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:33:24,314][1045180] Avg episode reward: [(0, '34.832')]
[2023-03-02 18:33:25,126][1045499] Updated weights for policy 0, policy_version 2239 (0.0006)
[2023-03-02 18:33:25,922][1045499] Updated weights for policy 0, policy_version 2249 (0.0007)
[2023-03-02 18:33:26,781][1045499] Updated weights for policy 0, policy_version 2259 (0.0006)
[2023-03-02 18:33:27,583][1045499] Updated weights for policy 0, policy_version 2269 (0.0008)
[2023-03-02 18:33:28,417][1045499] Updated weights for policy 0, policy_version 2279 (0.0006)
[2023-03-02 18:33:29,245][1045499] Updated weights for policy 0, policy_version 2289 (0.0007)
[2023-03-02 18:33:29,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 11776.0, 300 sec: 11776.0). Total num frames: 2343936. Throughput: 0: 12302.3. Samples: 683514. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:29,314][1045180] Avg episode reward: [(0, '29.408')]
[2023-03-02 18:33:30,083][1045499] Updated weights for policy 0, policy_version 2299 (0.0006)
[2023-03-02 18:33:30,917][1045499] Updated weights for policy 0, policy_version 2309 (0.0007)
[2023-03-02 18:33:31,777][1045499] Updated weights for policy 0, policy_version 2319 (0.0006)
[2023-03-02 18:33:32,603][1045499] Updated weights for policy 0, policy_version 2329 (0.0006)
[2023-03-02 18:33:33,417][1045499] Updated weights for policy 0, policy_version 2339 (0.0007)
[2023-03-02 18:33:34,264][1045499] Updated weights for policy 0, policy_version 2349 (0.0008)
[2023-03-02 18:33:34,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12288.0, 300 sec: 11815.4). Total num frames: 2405376. Throughput: 0: 12292.6. Samples: 757280. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:34,314][1045180] Avg episode reward: [(0, '30.578')]
[2023-03-02 18:33:35,086][1045499] Updated weights for policy 0, policy_version 2359 (0.0006)
[2023-03-02 18:33:35,899][1045499] Updated weights for policy 0, policy_version 2369 (0.0007)
[2023-03-02 18:33:36,741][1045499] Updated weights for policy 0, policy_version 2379 (0.0007)
[2023-03-02 18:33:37,546][1045499] Updated weights for policy 0, policy_version 2389 (0.0007)
[2023-03-02 18:33:38,362][1045499] Updated weights for policy 0, policy_version 2399 (0.0006)
[2023-03-02 18:33:39,236][1045499] Updated weights for policy 0, policy_version 2409 (0.0007)
[2023-03-02 18:33:39,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12288.0, 300 sec: 11849.2). Total num frames: 2466816. Throughput: 0: 12294.8. Samples: 831309. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:33:39,314][1045180] Avg episode reward: [(0, '34.766')]
[2023-03-02 18:33:40,028][1045499] Updated weights for policy 0, policy_version 2419 (0.0006)
[2023-03-02 18:33:40,840][1045499] Updated weights for policy 0, policy_version 2429 (0.0007)
[2023-03-02 18:33:41,676][1045499] Updated weights for policy 0, policy_version 2439 (0.0007)
[2023-03-02 18:33:42,504][1045499] Updated weights for policy 0, policy_version 2449 (0.0006)
[2023-03-02 18:33:43,318][1045499] Updated weights for policy 0, policy_version 2459 (0.0006)
[2023-03-02 18:33:44,148][1045499] Updated weights for policy 0, policy_version 2469 (0.0006)
[2023-03-02 18:33:44,313][1045180] Fps is (10 sec: 12492.9, 60 sec: 12322.1, 300 sec: 11905.7). Total num frames: 2530304. Throughput: 0: 12329.4. Samples: 869003. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:44,314][1045180] Avg episode reward: [(0, '36.567')]
[2023-03-02 18:33:44,967][1045499] Updated weights for policy 0, policy_version 2479 (0.0007)
[2023-03-02 18:33:45,777][1045499] Updated weights for policy 0, policy_version 2489 (0.0006)
[2023-03-02 18:33:46,618][1045499] Updated weights for policy 0, policy_version 2499 (0.0006)
[2023-03-02 18:33:47,439][1045499] Updated weights for policy 0, policy_version 2509 (0.0007)
[2023-03-02 18:33:48,277][1045499] Updated weights for policy 0, policy_version 2519 (0.0007)
[2023-03-02 18:33:49,092][1045499] Updated weights for policy 0, policy_version 2529 (0.0007)
[2023-03-02 18:33:49,313][1045180] Fps is (10 sec: 12492.9, 60 sec: 12322.1, 300 sec: 11929.6). Total num frames: 2591744. Throughput: 0: 12347.2. Samples: 943419. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:49,314][1045180] Avg episode reward: [(0, '43.631')]
[2023-03-02 18:33:49,317][1045448] Saving new best policy, reward=43.631!
[2023-03-02 18:33:49,950][1045499] Updated weights for policy 0, policy_version 2539 (0.0006)
[2023-03-02 18:33:50,777][1045499] Updated weights for policy 0, policy_version 2549 (0.0006)
[2023-03-02 18:33:51,598][1045499] Updated weights for policy 0, policy_version 2559 (0.0007)
[2023-03-02 18:33:52,422][1045499] Updated weights for policy 0, policy_version 2569 (0.0006)
[2023-03-02 18:33:53,256][1045499] Updated weights for policy 0, policy_version 2579 (0.0007)
[2023-03-02 18:33:54,073][1045499] Updated weights for policy 0, policy_version 2589 (0.0006)
[2023-03-02 18:33:54,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12322.2, 300 sec: 11950.7). Total num frames: 2653184. Throughput: 0: 12355.4. Samples: 1017525. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:33:54,314][1045180] Avg episode reward: [(0, '77.508')]
[2023-03-02 18:33:54,314][1045448] Saving new best policy, reward=77.508!
[2023-03-02 18:33:54,892][1045499] Updated weights for policy 0, policy_version 2599 (0.0007)
[2023-03-02 18:33:55,719][1045499] Updated weights for policy 0, policy_version 2609 (0.0006)
[2023-03-02 18:33:56,543][1045499] Updated weights for policy 0, policy_version 2619 (0.0006)
[2023-03-02 18:33:57,385][1045499] Updated weights for policy 0, policy_version 2629 (0.0007)
[2023-03-02 18:33:58,252][1045499] Updated weights for policy 0, policy_version 2639 (0.0007)
[2023-03-02 18:33:59,071][1045499] Updated weights for policy 0, policy_version 2649 (0.0006)
[2023-03-02 18:33:59,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12339.2, 300 sec: 11980.8). Total num frames: 2715648. Throughput: 0: 12353.1. Samples: 1054528. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
[2023-03-02 18:33:59,314][1045180] Avg episode reward: [(0, '74.012')]
[2023-03-02 18:33:59,877][1045499] Updated weights for policy 0, policy_version 2659 (0.0008)
[2023-03-02 18:34:00,713][1045499] Updated weights for policy 0, policy_version 2669 (0.0007)
[2023-03-02 18:34:01,538][1045499] Updated weights for policy 0, policy_version 2679 (0.0006)
[2023-03-02 18:34:02,368][1045499] Updated weights for policy 0, policy_version 2689 (0.0007)
[2023-03-02 18:34:03,168][1045499] Updated weights for policy 0, policy_version 2699 (0.0006)
[2023-03-02 18:34:03,985][1045499] Updated weights for policy 0, policy_version 2709 (0.0006)
[2023-03-02 18:34:04,313][1045180] Fps is (10 sec: 12492.7, 60 sec: 12373.3, 300 sec: 12007.8). Total num frames: 2778112. Throughput: 0: 12371.6. Samples: 1129007. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:04,314][1045180] Avg episode reward: [(0, '75.130')]
[2023-03-02 18:34:04,834][1045499] Updated weights for policy 0, policy_version 2719 (0.0008)
[2023-03-02 18:34:05,669][1045499] Updated weights for policy 0, policy_version 2729 (0.0007)
[2023-03-02 18:34:06,490][1045499] Updated weights for policy 0, policy_version 2739 (0.0006)
[2023-03-02 18:34:07,330][1045499] Updated weights for policy 0, policy_version 2749 (0.0006)
[2023-03-02 18:34:08,156][1045499] Updated weights for policy 0, policy_version 2759 (0.0006)
[2023-03-02 18:34:09,004][1045499] Updated weights for policy 0, policy_version 2769 (0.0007)
[2023-03-02 18:34:09,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12356.3, 300 sec: 12011.5). Total num frames: 2838528. Throughput: 0: 12374.0. Samples: 1202839. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:09,314][1045180] Avg episode reward: [(0, '79.370')]
[2023-03-02 18:34:09,328][1045448] Saving new best policy, reward=79.370!
[2023-03-02 18:34:09,816][1045499] Updated weights for policy 0, policy_version 2779 (0.0008)
[2023-03-02 18:34:10,675][1045499] Updated weights for policy 0, policy_version 2789 (0.0007)
[2023-03-02 18:34:11,495][1045499] Updated weights for policy 0, policy_version 2799 (0.0006)
[2023-03-02 18:34:12,316][1045499] Updated weights for policy 0, policy_version 2809 (0.0006)
[2023-03-02 18:34:13,151][1045499] Updated weights for policy 0, policy_version 2819 (0.0006)
[2023-03-02 18:34:13,969][1045499] Updated weights for policy 0, policy_version 2829 (0.0007)
[2023-03-02 18:34:14,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12373.4, 300 sec: 12034.5). Total num frames: 2900992. Throughput: 0: 12362.2. Samples: 1239811. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:14,314][1045180] Avg episode reward: [(0, '32.061')]
[2023-03-02 18:34:14,802][1045499] Updated weights for policy 0, policy_version 2839 (0.0006)
[2023-03-02 18:34:15,643][1045499] Updated weights for policy 0, policy_version 2849 (0.0006)
[2023-03-02 18:34:16,488][1045499] Updated weights for policy 0, policy_version 2859 (0.0006)
[2023-03-02 18:34:17,317][1045499] Updated weights for policy 0, policy_version 2869 (0.0006)
[2023-03-02 18:34:18,128][1045499] Updated weights for policy 0, policy_version 2879 (0.0006)
[2023-03-02 18:34:18,947][1045499] Updated weights for policy 0, policy_version 2889 (0.0006)
[2023-03-02 18:34:19,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12046.0). Total num frames: 2962432. Throughput: 0: 12373.8. Samples: 1314102. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:19,314][1045180] Avg episode reward: [(0, '88.601')]
[2023-03-02 18:34:19,331][1045448] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000002894_2963456.pth...
[2023-03-02 18:34:19,362][1045448] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000001277_1307648.pth
[2023-03-02 18:34:19,365][1045448] Saving new best policy, reward=88.601!
[2023-03-02 18:34:19,768][1045499] Updated weights for policy 0, policy_version 2899 (0.0007)
[2023-03-02 18:34:20,599][1045499] Updated weights for policy 0, policy_version 2909 (0.0006)
[2023-03-02 18:34:21,415][1045499] Updated weights for policy 0, policy_version 2919 (0.0008)
[2023-03-02 18:34:22,239][1045499] Updated weights for policy 0, policy_version 2929 (0.0007)
[2023-03-02 18:34:23,070][1045499] Updated weights for policy 0, policy_version 2939 (0.0007)
[2023-03-02 18:34:23,915][1045499] Updated weights for policy 0, policy_version 2949 (0.0007)
[2023-03-02 18:34:24,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12056.5). Total num frames: 3023872. Throughput: 0: 12377.1. Samples: 1388276. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:34:24,314][1045180] Avg episode reward: [(0, '97.890')]
[2023-03-02 18:34:24,322][1045448] Saving new best policy, reward=97.890!
[2023-03-02 18:34:24,749][1045499] Updated weights for policy 0, policy_version 2959 (0.0007)
[2023-03-02 18:34:25,556][1045499] Updated weights for policy 0, policy_version 2969 (0.0007)
[2023-03-02 18:34:26,376][1045499] Updated weights for policy 0, policy_version 2979 (0.0007)
[2023-03-02 18:34:27,231][1045499] Updated weights for policy 0, policy_version 2989 (0.0008)
[2023-03-02 18:34:28,077][1045499] Updated weights for policy 0, policy_version 2999 (0.0006)
[2023-03-02 18:34:28,905][1045499] Updated weights for policy 0, policy_version 3009 (0.0006)
[2023-03-02 18:34:29,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12356.3, 300 sec: 12066.1). Total num frames: 3085312. Throughput: 0: 12361.4. Samples: 1425267. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:29,314][1045180] Avg episode reward: [(0, '100.699')]
[2023-03-02 18:34:29,325][1045448] Saving new best policy, reward=100.699!
[2023-03-02 18:34:29,726][1045499] Updated weights for policy 0, policy_version 3019 (0.0006)
[2023-03-02 18:34:30,577][1045499] Updated weights for policy 0, policy_version 3029 (0.0006)
[2023-03-02 18:34:31,391][1045499] Updated weights for policy 0, policy_version 3039 (0.0006)
[2023-03-02 18:34:32,226][1045499] Updated weights for policy 0, policy_version 3049 (0.0007)
[2023-03-02 18:34:33,077][1045499] Updated weights for policy 0, policy_version 3059 (0.0007)
[2023-03-02 18:34:33,917][1045499] Updated weights for policy 0, policy_version 3069 (0.0008)
[2023-03-02 18:34:34,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12075.0). Total num frames: 3146752. Throughput: 0: 12339.6. Samples: 1498704. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:34:34,314][1045180] Avg episode reward: [(0, '149.621')]
[2023-03-02 18:34:34,314][1045448] Saving new best policy, reward=149.621!
[2023-03-02 18:34:34,733][1045499] Updated weights for policy 0, policy_version 3079 (0.0007)
[2023-03-02 18:34:35,554][1045499] Updated weights for policy 0, policy_version 3089 (0.0007)
[2023-03-02 18:34:36,387][1045499] Updated weights for policy 0, policy_version 3099 (0.0007)
[2023-03-02 18:34:37,192][1045499] Updated weights for policy 0, policy_version 3109 (0.0007)
[2023-03-02 18:34:38,002][1045499] Updated weights for policy 0, policy_version 3119 (0.0007)
[2023-03-02 18:34:38,820][1045499] Updated weights for policy 0, policy_version 3129 (0.0007)
[2023-03-02 18:34:39,313][1045180] Fps is (10 sec: 12390.2, 60 sec: 12373.3, 300 sec: 12091.1). Total num frames: 3209216. Throughput: 0: 12356.9. Samples: 1573586. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:34:39,314][1045180] Avg episode reward: [(0, '172.615')]
[2023-03-02 18:34:39,326][1045448] Saving new best policy, reward=172.615!
[2023-03-02 18:34:39,661][1045499] Updated weights for policy 0, policy_version 3139 (0.0006)
[2023-03-02 18:34:40,492][1045499] Updated weights for policy 0, policy_version 3149 (0.0007)
[2023-03-02 18:34:41,338][1045499] Updated weights for policy 0, policy_version 3159 (0.0007)
[2023-03-02 18:34:42,192][1045499] Updated weights for policy 0, policy_version 3169 (0.0008)
[2023-03-02 18:34:43,043][1045499] Updated weights for policy 0, policy_version 3179 (0.0007)
[2023-03-02 18:34:43,880][1045499] Updated weights for policy 0, policy_version 3189 (0.0007)
[2023-03-02 18:34:44,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12339.2, 300 sec: 12098.4). Total num frames: 3270656. Throughput: 0: 12346.9. Samples: 1610137. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:34:44,314][1045180] Avg episode reward: [(0, '90.885')]
[2023-03-02 18:34:44,732][1045499] Updated weights for policy 0, policy_version 3199 (0.0006)
[2023-03-02 18:34:45,538][1045499] Updated weights for policy 0, policy_version 3209 (0.0007)
[2023-03-02 18:34:46,358][1045499] Updated weights for policy 0, policy_version 3219 (0.0007)
[2023-03-02 18:34:47,186][1045499] Updated weights for policy 0, policy_version 3229 (0.0007)
[2023-03-02 18:34:48,024][1045499] Updated weights for policy 0, policy_version 3239 (0.0007)
[2023-03-02 18:34:48,850][1045499] Updated weights for policy 0, policy_version 3249 (0.0006)
[2023-03-02 18:34:49,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12339.2, 300 sec: 12105.1). Total num frames: 3332096. Throughput: 0: 12327.9. Samples: 1683764. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:34:49,314][1045180] Avg episode reward: [(0, '91.447')]
[2023-03-02 18:34:49,672][1045499] Updated weights for policy 0, policy_version 3259 (0.0007)
[2023-03-02 18:34:50,526][1045499] Updated weights for policy 0, policy_version 3269 (0.0007)
[2023-03-02 18:34:51,351][1045499] Updated weights for policy 0, policy_version 3279 (0.0007)
[2023-03-02 18:34:52,177][1045499] Updated weights for policy 0, policy_version 3289 (0.0008)
[2023-03-02 18:34:52,982][1045499] Updated weights for policy 0, policy_version 3299 (0.0007)
[2023-03-02 18:34:53,826][1045499] Updated weights for policy 0, policy_version 3309 (0.0007)
[2023-03-02 18:34:54,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12339.2, 300 sec: 12111.5). Total num frames: 3393536. Throughput: 0: 12332.9. Samples: 1757821. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:34:54,314][1045180] Avg episode reward: [(0, '58.689')]
[2023-03-02 18:34:54,646][1045499] Updated weights for policy 0, policy_version 3319 (0.0007)
[2023-03-02 18:34:55,469][1045499] Updated weights for policy 0, policy_version 3329 (0.0007)
[2023-03-02 18:34:56,294][1045499] Updated weights for policy 0, policy_version 3339 (0.0007)
[2023-03-02 18:34:57,112][1045499] Updated weights for policy 0, policy_version 3349 (0.0007)
[2023-03-02 18:34:57,991][1045499] Updated weights for policy 0, policy_version 3359 (0.0008)
[2023-03-02 18:34:58,813][1045499] Updated weights for policy 0, policy_version 3369 (0.0006)
[2023-03-02 18:34:59,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12322.1, 300 sec: 12117.3). Total num frames: 3454976. Throughput: 0: 12341.4. Samples: 1795176. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:34:59,314][1045180] Avg episode reward: [(0, '78.526')]
[2023-03-02 18:34:59,641][1045499] Updated weights for policy 0, policy_version 3379 (0.0006)
[2023-03-02 18:35:00,470][1045499] Updated weights for policy 0, policy_version 3389 (0.0006)
[2023-03-02 18:35:01,276][1045499] Updated weights for policy 0, policy_version 3399 (0.0007)
[2023-03-02 18:35:02,108][1045499] Updated weights for policy 0, policy_version 3409 (0.0007)
[2023-03-02 18:35:02,935][1045499] Updated weights for policy 0, policy_version 3419 (0.0009)
[2023-03-02 18:35:03,814][1045499] Updated weights for policy 0, policy_version 3429 (0.0007)
[2023-03-02 18:35:04,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12305.1, 300 sec: 12122.8). Total num frames: 3516416. Throughput: 0: 12331.7. Samples: 1869030. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:04,314][1045180] Avg episode reward: [(0, '37.692')]
[2023-03-02 18:35:04,667][1045499] Updated weights for policy 0, policy_version 3439 (0.0008)
[2023-03-02 18:35:05,503][1045499] Updated weights for policy 0, policy_version 3449 (0.0007)
[2023-03-02 18:35:06,332][1045499] Updated weights for policy 0, policy_version 3459 (0.0007)
[2023-03-02 18:35:07,173][1045499] Updated weights for policy 0, policy_version 3469 (0.0006)
[2023-03-02 18:35:07,988][1045499] Updated weights for policy 0, policy_version 3479 (0.0007)
[2023-03-02 18:35:08,793][1045499] Updated weights for policy 0, policy_version 3489 (0.0007)
[2023-03-02 18:35:09,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12339.2, 300 sec: 12134.4). Total num frames: 3578880. Throughput: 0: 12318.8. Samples: 1942621. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:35:09,314][1045180] Avg episode reward: [(0, '16.756')]
[2023-03-02 18:35:09,617][1045499] Updated weights for policy 0, policy_version 3499 (0.0007)
[2023-03-02 18:35:10,447][1045499] Updated weights for policy 0, policy_version 3509 (0.0007)
[2023-03-02 18:35:11,273][1045499] Updated weights for policy 0, policy_version 3519 (0.0007)
[2023-03-02 18:35:12,092][1045499] Updated weights for policy 0, policy_version 3529 (0.0006)
[2023-03-02 18:35:12,933][1045499] Updated weights for policy 0, policy_version 3539 (0.0007)
[2023-03-02 18:35:13,795][1045499] Updated weights for policy 0, policy_version 3549 (0.0007)
[2023-03-02 18:35:14,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12322.1, 300 sec: 12139.1). Total num frames: 3640320. Throughput: 0: 12320.7. Samples: 1979700. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:14,314][1045180] Avg episode reward: [(0, '15.795')]
[2023-03-02 18:35:14,619][1045499] Updated weights for policy 0, policy_version 3559 (0.0006)
[2023-03-02 18:35:15,460][1045499] Updated weights for policy 0, policy_version 3569 (0.0006)
[2023-03-02 18:35:16,285][1045499] Updated weights for policy 0, policy_version 3579 (0.0007)
[2023-03-02 18:35:17,097][1045499] Updated weights for policy 0, policy_version 3589 (0.0006)
[2023-03-02 18:35:17,956][1045499] Updated weights for policy 0, policy_version 3599 (0.0006)
[2023-03-02 18:35:18,785][1045499] Updated weights for policy 0, policy_version 3609 (0.0007)
[2023-03-02 18:35:19,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12322.1, 300 sec: 12143.4). Total num frames: 3701760. Throughput: 0: 12321.7. Samples: 2053180. Policy #0 lag: (min: 0.0, avg: 1.5, max: 3.0)
[2023-03-02 18:35:19,314][1045180] Avg episode reward: [(0, '24.007')]
[2023-03-02 18:35:19,607][1045499] Updated weights for policy 0, policy_version 3619 (0.0006)
[2023-03-02 18:35:20,448][1045499] Updated weights for policy 0, policy_version 3629 (0.0006)
[2023-03-02 18:35:21,280][1045499] Updated weights for policy 0, policy_version 3639 (0.0008)
[2023-03-02 18:35:22,112][1045499] Updated weights for policy 0, policy_version 3649 (0.0006)
[2023-03-02 18:35:22,956][1045499] Updated weights for policy 0, policy_version 3659 (0.0007)
[2023-03-02 18:35:23,781][1045499] Updated weights for policy 0, policy_version 3669 (0.0007)
[2023-03-02 18:35:24,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12322.1, 300 sec: 12147.6). Total num frames: 3763200. Throughput: 0: 12293.7. Samples: 2126801. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:24,314][1045180] Avg episode reward: [(0, '15.563')]
[2023-03-02 18:35:24,657][1045499] Updated weights for policy 0, policy_version 3679 (0.0007)
[2023-03-02 18:35:25,480][1045499] Updated weights for policy 0, policy_version 3689 (0.0006)
[2023-03-02 18:35:26,302][1045499] Updated weights for policy 0, policy_version 3699 (0.0007)
[2023-03-02 18:35:27,165][1045499] Updated weights for policy 0, policy_version 3709 (0.0006)
[2023-03-02 18:35:27,976][1045499] Updated weights for policy 0, policy_version 3719 (0.0006)
[2023-03-02 18:35:28,839][1045499] Updated weights for policy 0, policy_version 3729 (0.0007)
[2023-03-02 18:35:29,313][1045180] Fps is (10 sec: 12185.7, 60 sec: 12305.1, 300 sec: 12145.8). Total num frames: 3823616. Throughput: 0: 12290.8. Samples: 2163221. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:29,314][1045180] Avg episode reward: [(0, '17.900')]
[2023-03-02 18:35:29,683][1045499] Updated weights for policy 0, policy_version 3739 (0.0007)
[2023-03-02 18:35:30,500][1045499] Updated weights for policy 0, policy_version 3749 (0.0006)
[2023-03-02 18:35:31,301][1045499] Updated weights for policy 0, policy_version 3759 (0.0007)
[2023-03-02 18:35:32,123][1045499] Updated weights for policy 0, policy_version 3769 (0.0007)
[2023-03-02 18:35:32,944][1045499] Updated weights for policy 0, policy_version 3779 (0.0007)
[2023-03-02 18:35:33,776][1045499] Updated weights for policy 0, policy_version 3789 (0.0007)
[2023-03-02 18:35:34,313][1045180] Fps is (10 sec: 12287.8, 60 sec: 12322.1, 300 sec: 12155.2). Total num frames: 3886080. Throughput: 0: 12306.8. Samples: 2237568. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:35:34,314][1045180] Avg episode reward: [(0, '24.087')]
[2023-03-02 18:35:34,595][1045499] Updated weights for policy 0, policy_version 3799 (0.0007)
[2023-03-02 18:35:35,453][1045499] Updated weights for policy 0, policy_version 3809 (0.0006)
[2023-03-02 18:35:36,262][1045499] Updated weights for policy 0, policy_version 3819 (0.0007)
[2023-03-02 18:35:37,081][1045499] Updated weights for policy 0, policy_version 3829 (0.0007)
[2023-03-02 18:35:37,911][1045499] Updated weights for policy 0, policy_version 3839 (0.0007)
[2023-03-02 18:35:38,564][1045448] KL-divergence is very high: 173.6769
[2023-03-02 18:35:38,745][1045499] Updated weights for policy 0, policy_version 3849 (0.0007)
[2023-03-02 18:35:39,314][1045180] Fps is (10 sec: 12492.1, 60 sec: 12322.0, 300 sec: 12164.0). Total num frames: 3948544. Throughput: 0: 12318.6. Samples: 2312164. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:39,314][1045180] Avg episode reward: [(0, '16.004')]
[2023-03-02 18:35:39,545][1045499] Updated weights for policy 0, policy_version 3859 (0.0007)
[2023-03-02 18:35:40,396][1045499] Updated weights for policy 0, policy_version 3869 (0.0006)
[2023-03-02 18:35:41,197][1045499] Updated weights for policy 0, policy_version 3879 (0.0006)
[2023-03-02 18:35:42,048][1045499] Updated weights for policy 0, policy_version 3889 (0.0007)
[2023-03-02 18:35:42,891][1045499] Updated weights for policy 0, policy_version 3899 (0.0007)
[2023-03-02 18:35:43,718][1045499] Updated weights for policy 0, policy_version 3909 (0.0006)
[2023-03-02 18:35:44,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12322.1, 300 sec: 12167.2). Total num frames: 4009984. Throughput: 0: 12306.0. Samples: 2348947. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:44,314][1045180] Avg episode reward: [(0, '25.979')]
[2023-03-02 18:35:44,543][1045499] Updated weights for policy 0, policy_version 3919 (0.0007)
[2023-03-02 18:35:45,376][1045499] Updated weights for policy 0, policy_version 3929 (0.0007)
[2023-03-02 18:35:46,204][1045499] Updated weights for policy 0, policy_version 3939 (0.0006)
[2023-03-02 18:35:47,032][1045499] Updated weights for policy 0, policy_version 3949 (0.0006)
[2023-03-02 18:35:47,866][1045499] Updated weights for policy 0, policy_version 3959 (0.0007)
[2023-03-02 18:35:48,691][1045499] Updated weights for policy 0, policy_version 3969 (0.0007)
[2023-03-02 18:35:49,313][1045180] Fps is (10 sec: 12288.6, 60 sec: 12322.1, 300 sec: 12170.2). Total num frames: 4071424. Throughput: 0: 12310.4. Samples: 2423000. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:49,314][1045180] Avg episode reward: [(0, '31.309')]
[2023-03-02 18:35:49,515][1045499] Updated weights for policy 0, policy_version 3979 (0.0007)
[2023-03-02 18:35:50,353][1045499] Updated weights for policy 0, policy_version 3989 (0.0007)
[2023-03-02 18:35:51,156][1045499] Updated weights for policy 0, policy_version 3999 (0.0007)
[2023-03-02 18:35:51,971][1045499] Updated weights for policy 0, policy_version 4009 (0.0007)
[2023-03-02 18:35:52,797][1045499] Updated weights for policy 0, policy_version 4019 (0.0006)
[2023-03-02 18:35:53,527][1045448] KL-divergence is very high: 147.1295
[2023-03-02 18:35:53,637][1045499] Updated weights for policy 0, policy_version 4029 (0.0007)
[2023-03-02 18:35:54,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12339.2, 300 sec: 12178.1). Total num frames: 4133888. Throughput: 0: 12337.3. Samples: 2497801. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:35:54,314][1045180] Avg episode reward: [(0, '33.488')]
[2023-03-02 18:35:54,450][1045499] Updated weights for policy 0, policy_version 4039 (0.0006)
[2023-03-02 18:35:55,250][1045499] Updated weights for policy 0, policy_version 4049 (0.0007)
[2023-03-02 18:35:56,091][1045499] Updated weights for policy 0, policy_version 4059 (0.0006)
[2023-03-02 18:35:56,910][1045499] Updated weights for policy 0, policy_version 4069 (0.0007)
[2023-03-02 18:35:57,722][1045499] Updated weights for policy 0, policy_version 4079 (0.0007)
[2023-03-02 18:35:58,565][1045499] Updated weights for policy 0, policy_version 4089 (0.0007)
[2023-03-02 18:35:59,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12356.3, 300 sec: 12185.6). Total num frames: 4196352. Throughput: 0: 12346.7. Samples: 2535304. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:35:59,314][1045180] Avg episode reward: [(0, '18.036')]
[2023-03-02 18:35:59,365][1045499] Updated weights for policy 0, policy_version 4099 (0.0007)
[2023-03-02 18:36:00,192][1045499] Updated weights for policy 0, policy_version 4109 (0.0007)
[2023-03-02 18:36:01,055][1045499] Updated weights for policy 0, policy_version 4119 (0.0007)
[2023-03-02 18:36:01,888][1045499] Updated weights for policy 0, policy_version 4129 (0.0008)
[2023-03-02 18:36:02,708][1045499] Updated weights for policy 0, policy_version 4139 (0.0006)
[2023-03-02 18:36:03,538][1045499] Updated weights for policy 0, policy_version 4149 (0.0007)
[2023-03-02 18:36:04,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12356.3, 300 sec: 12188.0). Total num frames: 4257792. Throughput: 0: 12356.3. Samples: 2609214. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:04,314][1045180] Avg episode reward: [(0, '17.536')]
[2023-03-02 18:36:04,352][1045499] Updated weights for policy 0, policy_version 4159 (0.0007)
[2023-03-02 18:36:05,180][1045499] Updated weights for policy 0, policy_version 4169 (0.0006)
[2023-03-02 18:36:06,017][1045499] Updated weights for policy 0, policy_version 4179 (0.0006)
[2023-03-02 18:36:06,842][1045499] Updated weights for policy 0, policy_version 4189 (0.0007)
[2023-03-02 18:36:07,657][1045499] Updated weights for policy 0, policy_version 4199 (0.0007)
[2023-03-02 18:36:08,505][1045499] Updated weights for policy 0, policy_version 4209 (0.0007)
[2023-03-02 18:36:09,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12339.2, 300 sec: 12190.3). Total num frames: 4319232. Throughput: 0: 12367.5. Samples: 2683342. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:09,314][1045180] Avg episode reward: [(0, '12.739')]
[2023-03-02 18:36:09,355][1045499] Updated weights for policy 0, policy_version 4219 (0.0006)
[2023-03-02 18:36:10,169][1045499] Updated weights for policy 0, policy_version 4229 (0.0007)
[2023-03-02 18:36:11,020][1045499] Updated weights for policy 0, policy_version 4239 (0.0006)
[2023-03-02 18:36:11,825][1045499] Updated weights for policy 0, policy_version 4249 (0.0006)
[2023-03-02 18:36:12,648][1045499] Updated weights for policy 0, policy_version 4259 (0.0006)
[2023-03-02 18:36:13,487][1045499] Updated weights for policy 0, policy_version 4269 (0.0007)
[2023-03-02 18:36:14,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12339.2, 300 sec: 12192.4). Total num frames: 4380672. Throughput: 0: 12385.3. Samples: 2720559. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:36:14,314][1045180] Avg episode reward: [(0, '13.930')]
[2023-03-02 18:36:14,316][1045499] Updated weights for policy 0, policy_version 4279 (0.0007)
[2023-03-02 18:36:15,132][1045499] Updated weights for policy 0, policy_version 4289 (0.0007)
[2023-03-02 18:36:15,974][1045499] Updated weights for policy 0, policy_version 4299 (0.0008)
[2023-03-02 18:36:16,799][1045499] Updated weights for policy 0, policy_version 4309 (0.0007)
[2023-03-02 18:36:17,612][1045499] Updated weights for policy 0, policy_version 4319 (0.0007)
[2023-03-02 18:36:18,437][1045499] Updated weights for policy 0, policy_version 4329 (0.0007)
[2023-03-02 18:36:19,272][1045499] Updated weights for policy 0, policy_version 4339 (0.0007)
[2023-03-02 18:36:19,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12356.3, 300 sec: 12199.0). Total num frames: 4443136. Throughput: 0: 12386.8. Samples: 2794972. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:19,314][1045180] Avg episode reward: [(0, '16.766')]
[2023-03-02 18:36:19,324][1045448] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000004340_4444160.pth...
[2023-03-02 18:36:19,356][1045448] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000001599_1637376.pth
[2023-03-02 18:36:20,093][1045499] Updated weights for policy 0, policy_version 4349 (0.0006)
[2023-03-02 18:36:20,916][1045499] Updated weights for policy 0, policy_version 4359 (0.0007)
[2023-03-02 18:36:21,738][1045499] Updated weights for policy 0, policy_version 4369 (0.0007)
[2023-03-02 18:36:22,564][1045499] Updated weights for policy 0, policy_version 4379 (0.0006)
[2023-03-02 18:36:23,389][1045499] Updated weights for policy 0, policy_version 4389 (0.0007)
[2023-03-02 18:36:24,205][1045499] Updated weights for policy 0, policy_version 4399 (0.0006)
[2023-03-02 18:36:24,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12373.3, 300 sec: 12205.2). Total num frames: 4505600. Throughput: 0: 12385.2. Samples: 2869494. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:24,314][1045180] Avg episode reward: [(0, '13.664')]
[2023-03-02 18:36:25,019][1045499] Updated weights for policy 0, policy_version 4409 (0.0006)
[2023-03-02 18:36:25,879][1045499] Updated weights for policy 0, policy_version 4419 (0.0007)
[2023-03-02 18:36:26,699][1045499] Updated weights for policy 0, policy_version 4429 (0.0006)
[2023-03-02 18:36:27,519][1045499] Updated weights for policy 0, policy_version 4439 (0.0006)
[2023-03-02 18:36:28,368][1045499] Updated weights for policy 0, policy_version 4449 (0.0008)
[2023-03-02 18:36:29,208][1045499] Updated weights for policy 0, policy_version 4459 (0.0006)
[2023-03-02 18:36:29,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12390.4, 300 sec: 12206.9). Total num frames: 4567040. Throughput: 0: 12386.7. Samples: 2906347. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:29,314][1045180] Avg episode reward: [(0, '18.686')]
[2023-03-02 18:36:30,048][1045499] Updated weights for policy 0, policy_version 4469 (0.0008)
[2023-03-02 18:36:30,880][1045499] Updated weights for policy 0, policy_version 4479 (0.0006)
[2023-03-02 18:36:31,705][1045499] Updated weights for policy 0, policy_version 4489 (0.0007)
[2023-03-02 18:36:32,541][1045499] Updated weights for policy 0, policy_version 4499 (0.0006)
[2023-03-02 18:36:33,372][1045499] Updated weights for policy 0, policy_version 4509 (0.0006)
[2023-03-02 18:36:34,184][1045499] Updated weights for policy 0, policy_version 4519 (0.0006)
[2023-03-02 18:36:34,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12373.3, 300 sec: 12208.6). Total num frames: 4628480. Throughput: 0: 12381.2. Samples: 2980153. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:34,314][1045180] Avg episode reward: [(0, '15.073')]
[2023-03-02 18:36:35,023][1045499] Updated weights for policy 0, policy_version 4529 (0.0007)
[2023-03-02 18:36:35,833][1045499] Updated weights for policy 0, policy_version 4539 (0.0006)
[2023-03-02 18:36:36,685][1045499] Updated weights for policy 0, policy_version 4549 (0.0007)
[2023-03-02 18:36:37,513][1045499] Updated weights for policy 0, policy_version 4559 (0.0007)
[2023-03-02 18:36:38,352][1045499] Updated weights for policy 0, policy_version 4569 (0.0006)
[2023-03-02 18:36:39,180][1045499] Updated weights for policy 0, policy_version 4579 (0.0007)
[2023-03-02 18:36:39,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12356.4, 300 sec: 12210.2). Total num frames: 4689920. Throughput: 0: 12358.7. Samples: 3053944. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:36:39,314][1045180] Avg episode reward: [(0, '13.465')]
[2023-03-02 18:36:40,002][1045499] Updated weights for policy 0, policy_version 4589 (0.0007)
[2023-03-02 18:36:40,824][1045499] Updated weights for policy 0, policy_version 4599 (0.0006)
[2023-03-02 18:36:41,644][1045499] Updated weights for policy 0, policy_version 4609 (0.0008)
[2023-03-02 18:36:42,453][1045499] Updated weights for policy 0, policy_version 4619 (0.0007)
[2023-03-02 18:36:43,276][1045499] Updated weights for policy 0, policy_version 4629 (0.0007)
[2023-03-02 18:36:44,097][1045499] Updated weights for policy 0, policy_version 4639 (0.0007)
[2023-03-02 18:36:44,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12373.3, 300 sec: 12215.7). Total num frames: 4752384. Throughput: 0: 12361.6. Samples: 3091574. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:44,314][1045180] Avg episode reward: [(0, '9.541')]
[2023-03-02 18:36:44,941][1045499] Updated weights for policy 0, policy_version 4649 (0.0007)
[2023-03-02 18:36:45,799][1045499] Updated weights for policy 0, policy_version 4659 (0.0007)
[2023-03-02 18:36:46,610][1045499] Updated weights for policy 0, policy_version 4669 (0.0006)
[2023-03-02 18:36:47,448][1045499] Updated weights for policy 0, policy_version 4679 (0.0007)
[2023-03-02 18:36:48,264][1045499] Updated weights for policy 0, policy_version 4689 (0.0006)
[2023-03-02 18:36:49,105][1045499] Updated weights for policy 0, policy_version 4699 (0.0007)
[2023-03-02 18:36:49,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12373.3, 300 sec: 12217.1). Total num frames: 4813824. Throughput: 0: 12366.7. Samples: 3165717. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:36:49,314][1045180] Avg episode reward: [(0, '9.880')]
[2023-03-02 18:36:49,938][1045499] Updated weights for policy 0, policy_version 4709 (0.0006)
[2023-03-02 18:36:50,753][1045499] Updated weights for policy 0, policy_version 4719 (0.0007)
[2023-03-02 18:36:51,569][1045499] Updated weights for policy 0, policy_version 4729 (0.0006)
[2023-03-02 18:36:52,406][1045499] Updated weights for policy 0, policy_version 4739 (0.0007)
[2023-03-02 18:36:53,241][1045499] Updated weights for policy 0, policy_version 4749 (0.0007)
[2023-03-02 18:36:54,093][1045499] Updated weights for policy 0, policy_version 4759 (0.0007)
[2023-03-02 18:36:54,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12356.3, 300 sec: 12218.5). Total num frames: 4875264. Throughput: 0: 12359.1. Samples: 3239499. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:36:54,314][1045180] Avg episode reward: [(0, '8.232')]
[2023-03-02 18:36:54,920][1045499] Updated weights for policy 0, policy_version 4769 (0.0007)
[2023-03-02 18:36:55,726][1045499] Updated weights for policy 0, policy_version 4779 (0.0007)
[2023-03-02 18:36:56,553][1045499] Updated weights for policy 0, policy_version 4789 (0.0007)
[2023-03-02 18:36:57,386][1045499] Updated weights for policy 0, policy_version 4799 (0.0007)
[2023-03-02 18:36:58,194][1045499] Updated weights for policy 0, policy_version 4809 (0.0007)
[2023-03-02 18:36:58,988][1045499] Updated weights for policy 0, policy_version 4819 (0.0007)
[2023-03-02 18:36:59,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12356.3, 300 sec: 12223.5). Total num frames: 4937728. Throughput: 0: 12360.8. Samples: 3276795. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:36:59,314][1045180] Avg episode reward: [(0, '9.039')]
[2023-03-02 18:36:59,822][1045499] Updated weights for policy 0, policy_version 4829 (0.0007)
[2023-03-02 18:37:00,647][1045499] Updated weights for policy 0, policy_version 4839 (0.0007)
[2023-03-02 18:37:01,486][1045499] Updated weights for policy 0, policy_version 4849 (0.0007)
[2023-03-02 18:37:02,329][1045499] Updated weights for policy 0, policy_version 4859 (0.0006)
[2023-03-02 18:37:03,150][1045499] Updated weights for policy 0, policy_version 4869 (0.0006)
[2023-03-02 18:37:03,970][1045499] Updated weights for policy 0, policy_version 4879 (0.0007)
[2023-03-02 18:37:04,313][1045180] Fps is (10 sec: 12492.7, 60 sec: 12373.3, 300 sec: 12228.4). Total num frames: 5000192. Throughput: 0: 12370.0. Samples: 3351623. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:37:04,314][1045180] Avg episode reward: [(0, '7.845')]
[2023-03-02 18:37:04,798][1045499] Updated weights for policy 0, policy_version 4889 (0.0008)
[2023-03-02 18:37:05,628][1045499] Updated weights for policy 0, policy_version 4899 (0.0007)
[2023-03-02 18:37:06,449][1045499] Updated weights for policy 0, policy_version 4909 (0.0006)
[2023-03-02 18:37:07,274][1045499] Updated weights for policy 0, policy_version 4919 (0.0006)
[2023-03-02 18:37:08,110][1045499] Updated weights for policy 0, policy_version 4929 (0.0006)
[2023-03-02 18:37:08,928][1045499] Updated weights for policy 0, policy_version 4939 (0.0007)
[2023-03-02 18:37:09,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12229.5). Total num frames: 5061632. Throughput: 0: 12363.4. Samples: 3425849. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:37:09,314][1045180] Avg episode reward: [(0, '7.902')]
[2023-03-02 18:37:09,747][1045499] Updated weights for policy 0, policy_version 4949 (0.0006)
[2023-03-02 18:37:10,555][1045499] Updated weights for policy 0, policy_version 4959 (0.0007)
[2023-03-02 18:37:11,396][1045499] Updated weights for policy 0, policy_version 4969 (0.0007)
[2023-03-02 18:37:12,209][1045499] Updated weights for policy 0, policy_version 4979 (0.0006)
[2023-03-02 18:37:13,047][1045499] Updated weights for policy 0, policy_version 4989 (0.0006)
[2023-03-02 18:37:13,851][1045499] Updated weights for policy 0, policy_version 4999 (0.0007)
[2023-03-02 18:37:14,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12390.4, 300 sec: 12234.1). Total num frames: 5124096. Throughput: 0: 12377.8. Samples: 3463348. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:37:14,314][1045180] Avg episode reward: [(0, '6.443')]
[2023-03-02 18:37:14,657][1045499] Updated weights for policy 0, policy_version 5009 (0.0006)
[2023-03-02 18:37:15,501][1045499] Updated weights for policy 0, policy_version 5019 (0.0007)
[2023-03-02 18:37:16,315][1045499] Updated weights for policy 0, policy_version 5029 (0.0006)
[2023-03-02 18:37:17,133][1045499] Updated weights for policy 0, policy_version 5039 (0.0006)
[2023-03-02 18:37:17,969][1045499] Updated weights for policy 0, policy_version 5049 (0.0006)
[2023-03-02 18:37:18,784][1045499] Updated weights for policy 0, policy_version 5059 (0.0007)
[2023-03-02 18:37:19,313][1045180] Fps is (10 sec: 12492.9, 60 sec: 12390.4, 300 sec: 12238.6). Total num frames: 5186560. Throughput: 0: 12402.5. Samples: 3538266. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:37:19,314][1045180] Avg episode reward: [(0, '6.611')]
[2023-03-02 18:37:19,424][1045448] KL-divergence is very high: 3397.6538
[2023-03-02 18:37:19,499][1045448] KL-divergence is very high: 173.7547
[2023-03-02 18:37:19,606][1045499] Updated weights for policy 0, policy_version 5069 (0.0007)
[2023-03-02 18:37:20,425][1045499] Updated weights for policy 0, policy_version 5079 (0.0007)
[2023-03-02 18:37:21,257][1045499] Updated weights for policy 0, policy_version 5089 (0.0006)
[2023-03-02 18:37:22,065][1045499] Updated weights for policy 0, policy_version 5099 (0.0007)
[2023-03-02 18:37:22,893][1045499] Updated weights for policy 0, policy_version 5109 (0.0006)
[2023-03-02 18:37:23,733][1045499] Updated weights for policy 0, policy_version 5119 (0.0006)
[2023-03-02 18:37:24,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12239.4). Total num frames: 5248000. Throughput: 0: 12406.3. Samples: 3612229. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:37:24,314][1045180] Avg episode reward: [(0, '7.101')]
[2023-03-02 18:37:24,560][1045499] Updated weights for policy 0, policy_version 5129 (0.0007)
[2023-03-02 18:37:25,391][1045499] Updated weights for policy 0, policy_version 5139 (0.0007)
[2023-03-02 18:37:26,239][1045499] Updated weights for policy 0, policy_version 5149 (0.0007)
[2023-03-02 18:37:27,061][1045499] Updated weights for policy 0, policy_version 5159 (0.0007)
[2023-03-02 18:37:27,904][1045499] Updated weights for policy 0, policy_version 5169 (0.0008)
[2023-03-02 18:37:28,715][1045499] Updated weights for policy 0, policy_version 5179 (0.0006)
[2023-03-02 18:37:29,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12390.4, 300 sec: 12347.0). Total num frames: 5310464. Throughput: 0: 12395.8. Samples: 3649383. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:37:29,314][1045180] Avg episode reward: [(0, '6.557')]
[2023-03-02 18:37:29,539][1045499] Updated weights for policy 0, policy_version 5189 (0.0006)
[2023-03-02 18:37:30,391][1045499] Updated weights for policy 0, policy_version 5199 (0.0006)
[2023-03-02 18:37:31,217][1045499] Updated weights for policy 0, policy_version 5209 (0.0007)
[2023-03-02 18:37:32,039][1045499] Updated weights for policy 0, policy_version 5219 (0.0006)
[2023-03-02 18:37:32,870][1045499] Updated weights for policy 0, policy_version 5229 (0.0007)
[2023-03-02 18:37:33,703][1045499] Updated weights for policy 0, policy_version 5239 (0.0007)
[2023-03-02 18:37:34,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12390.4, 300 sec: 12347.0). Total num frames: 5371904. Throughput: 0: 12392.0. Samples: 3723358. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:37:34,314][1045180] Avg episode reward: [(0, '7.085')]
[2023-03-02 18:37:34,508][1045499] Updated weights for policy 0, policy_version 5249 (0.0008)
[2023-03-02 18:37:35,308][1045499] Updated weights for policy 0, policy_version 5259 (0.0006)
[2023-03-02 18:37:36,134][1045499] Updated weights for policy 0, policy_version 5269 (0.0006)
[2023-03-02 18:37:36,952][1045499] Updated weights for policy 0, policy_version 5279 (0.0007)
[2023-03-02 18:37:37,713][1045448] KL-divergence is very high: 644.2097
[2023-03-02 18:37:37,795][1045499] Updated weights for policy 0, policy_version 5289 (0.0007)
[2023-03-02 18:37:38,596][1045499] Updated weights for policy 0, policy_version 5299 (0.0007)
[2023-03-02 18:37:39,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12407.4, 300 sec: 12350.5). Total num frames: 5434368. Throughput: 0: 12417.4. Samples: 3798283. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:37:39,314][1045180] Avg episode reward: [(0, '7.545')]
[2023-03-02 18:37:39,439][1045499] Updated weights for policy 0, policy_version 5309 (0.0008)
[2023-03-02 18:37:40,272][1045499] Updated weights for policy 0, policy_version 5319 (0.0007)
[2023-03-02 18:37:41,120][1045499] Updated weights for policy 0, policy_version 5329 (0.0007)
[2023-03-02 18:37:41,930][1045499] Updated weights for policy 0, policy_version 5339 (0.0007)
[2023-03-02 18:37:42,769][1045499] Updated weights for policy 0, policy_version 5349 (0.0008)
[2023-03-02 18:37:43,606][1045499] Updated weights for policy 0, policy_version 5359 (0.0007)
[2023-03-02 18:37:44,313][1045180] Fps is (10 sec: 12390.6, 60 sec: 12390.4, 300 sec: 12350.5). Total num frames: 5495808. Throughput: 0: 12411.6. Samples: 3835315. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:37:44,314][1045180] Avg episode reward: [(0, '7.400')]
[2023-03-02 18:37:44,431][1045499] Updated weights for policy 0, policy_version 5369 (0.0007)
[2023-03-02 18:37:45,240][1045499] Updated weights for policy 0, policy_version 5379 (0.0006)
[2023-03-02 18:37:46,062][1045499] Updated weights for policy 0, policy_version 5389 (0.0006)
[2023-03-02 18:37:46,888][1045499] Updated weights for policy 0, policy_version 5399 (0.0007)
[2023-03-02 18:37:47,714][1045499] Updated weights for policy 0, policy_version 5409 (0.0006)
[2023-03-02 18:37:48,554][1045499] Updated weights for policy 0, policy_version 5419 (0.0006)
[2023-03-02 18:37:49,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12390.4, 300 sec: 12350.5). Total num frames: 5557248. Throughput: 0: 12401.7. Samples: 3909699. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:37:49,314][1045180] Avg episode reward: [(0, '7.202')]
[2023-03-02 18:37:49,413][1045499] Updated weights for policy 0, policy_version 5429 (0.0006)
[2023-03-02 18:37:50,236][1045499] Updated weights for policy 0, policy_version 5439 (0.0007)
[2023-03-02 18:37:51,066][1045499] Updated weights for policy 0, policy_version 5449 (0.0007)
[2023-03-02 18:37:51,884][1045499] Updated weights for policy 0, policy_version 5459 (0.0006)
[2023-03-02 18:37:52,705][1045499] Updated weights for policy 0, policy_version 5469 (0.0007)
[2023-03-02 18:37:53,561][1045499] Updated weights for policy 0, policy_version 5479 (0.0007)
[2023-03-02 18:37:54,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12407.5, 300 sec: 12354.0). Total num frames: 5619712. Throughput: 0: 12390.5. Samples: 3983418. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:37:54,314][1045180] Avg episode reward: [(0, '7.069')]
[2023-03-02 18:37:54,377][1045499] Updated weights for policy 0, policy_version 5489 (0.0007)
[2023-03-02 18:37:55,200][1045499] Updated weights for policy 0, policy_version 5499 (0.0007)
[2023-03-02 18:37:56,023][1045499] Updated weights for policy 0, policy_version 5509 (0.0007)
[2023-03-02 18:37:56,812][1045499] Updated weights for policy 0, policy_version 5519 (0.0007)
[2023-03-02 18:37:57,639][1045499] Updated weights for policy 0, policy_version 5529 (0.0007)
[2023-03-02 18:37:58,463][1045499] Updated weights for policy 0, policy_version 5539 (0.0006)
[2023-03-02 18:37:59,291][1045499] Updated weights for policy 0, policy_version 5549 (0.0007)
[2023-03-02 18:37:59,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12407.5, 300 sec: 12360.9). Total num frames: 5682176. Throughput: 0: 12396.1. Samples: 4021174. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:37:59,314][1045180] Avg episode reward: [(0, '7.178')]
[2023-03-02 18:38:00,132][1045499] Updated weights for policy 0, policy_version 5559 (0.0007)
[2023-03-02 18:38:00,985][1045499] Updated weights for policy 0, policy_version 5569 (0.0006)
[2023-03-02 18:38:01,825][1045499] Updated weights for policy 0, policy_version 5579 (0.0006)
[2023-03-02 18:38:02,645][1045499] Updated weights for policy 0, policy_version 5589 (0.0007)
[2023-03-02 18:38:03,476][1045499] Updated weights for policy 0, policy_version 5599 (0.0006)
[2023-03-02 18:38:04,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12373.3, 300 sec: 12357.4). Total num frames: 5742592. Throughput: 0: 12368.9. Samples: 4094868. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:04,314][1045180] Avg episode reward: [(0, '6.660')]
[2023-03-02 18:38:04,316][1045499] Updated weights for policy 0, policy_version 5609 (0.0007)
[2023-03-02 18:38:05,125][1045499] Updated weights for policy 0, policy_version 5619 (0.0007)
[2023-03-02 18:38:05,980][1045499] Updated weights for policy 0, policy_version 5629 (0.0009)
[2023-03-02 18:38:06,816][1045499] Updated weights for policy 0, policy_version 5639 (0.0006)
[2023-03-02 18:38:07,629][1045499] Updated weights for policy 0, policy_version 5649 (0.0006)
[2023-03-02 18:38:08,448][1045499] Updated weights for policy 0, policy_version 5659 (0.0007)
[2023-03-02 18:38:09,282][1045499] Updated weights for policy 0, policy_version 5669 (0.0008)
[2023-03-02 18:38:09,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12390.4, 300 sec: 12360.9). Total num frames: 5805056. Throughput: 0: 12370.9. Samples: 4168918. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:38:09,314][1045180] Avg episode reward: [(0, '8.186')]
[2023-03-02 18:38:10,105][1045499] Updated weights for policy 0, policy_version 5679 (0.0007)
[2023-03-02 18:38:10,917][1045499] Updated weights for policy 0, policy_version 5689 (0.0006)
[2023-03-02 18:38:11,736][1045499] Updated weights for policy 0, policy_version 5699 (0.0007)
[2023-03-02 18:38:12,572][1045499] Updated weights for policy 0, policy_version 5709 (0.0006)
[2023-03-02 18:38:13,402][1045499] Updated weights for policy 0, policy_version 5719 (0.0006)
[2023-03-02 18:38:14,240][1045499] Updated weights for policy 0, policy_version 5729 (0.0006)
[2023-03-02 18:38:14,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12373.3, 300 sec: 12360.9). Total num frames: 5866496. Throughput: 0: 12373.6. Samples: 4206195. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:14,314][1045180] Avg episode reward: [(0, '7.735')]
[2023-03-02 18:38:15,074][1045499] Updated weights for policy 0, policy_version 5739 (0.0007)
[2023-03-02 18:38:15,922][1045499] Updated weights for policy 0, policy_version 5749 (0.0006)
[2023-03-02 18:38:16,749][1045499] Updated weights for policy 0, policy_version 5759 (0.0007)
[2023-03-02 18:38:17,571][1045499] Updated weights for policy 0, policy_version 5769 (0.0006)
[2023-03-02 18:38:18,389][1045499] Updated weights for policy 0, policy_version 5779 (0.0006)
[2023-03-02 18:38:19,220][1045499] Updated weights for policy 0, policy_version 5789 (0.0006)
[2023-03-02 18:38:19,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12373.3, 300 sec: 12360.9). Total num frames: 5928960. Throughput: 0: 12373.8. Samples: 4280180. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:38:19,314][1045180] Avg episode reward: [(0, '6.788')]
[2023-03-02 18:38:19,317][1045448] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000005790_5928960.pth...
[2023-03-02 18:38:19,352][1045448] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000002894_2963456.pth
[2023-03-02 18:38:20,052][1045499] Updated weights for policy 0, policy_version 5799 (0.0008)
[2023-03-02 18:38:20,873][1045499] Updated weights for policy 0, policy_version 5809 (0.0007)
[2023-03-02 18:38:21,725][1045499] Updated weights for policy 0, policy_version 5819 (0.0007)
[2023-03-02 18:38:22,556][1045499] Updated weights for policy 0, policy_version 5829 (0.0006)
[2023-03-02 18:38:23,373][1045499] Updated weights for policy 0, policy_version 5839 (0.0006)
[2023-03-02 18:38:24,200][1045499] Updated weights for policy 0, policy_version 5849 (0.0007)
[2023-03-02 18:38:24,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12373.3, 300 sec: 12360.9). Total num frames: 5990400. Throughput: 0: 12359.4. Samples: 4354456. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:38:24,314][1045180] Avg episode reward: [(0, '6.880')]
[2023-03-02 18:38:25,028][1045499] Updated weights for policy 0, policy_version 5859 (0.0006)
[2023-03-02 18:38:25,846][1045499] Updated weights for policy 0, policy_version 5869 (0.0007)
[2023-03-02 18:38:26,669][1045499] Updated weights for policy 0, policy_version 5879 (0.0007)
[2023-03-02 18:38:27,482][1045499] Updated weights for policy 0, policy_version 5889 (0.0006)
[2023-03-02 18:38:28,320][1045499] Updated weights for policy 0, policy_version 5899 (0.0007)
[2023-03-02 18:38:29,157][1045499] Updated weights for policy 0, policy_version 5909 (0.0006)
[2023-03-02 18:38:29,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12373.3, 300 sec: 12364.4). Total num frames: 6052864. Throughput: 0: 12366.8. Samples: 4391822. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:38:29,314][1045180] Avg episode reward: [(0, '6.661')]
[2023-03-02 18:38:29,977][1045499] Updated weights for policy 0, policy_version 5919 (0.0007)
[2023-03-02 18:38:30,782][1045499] Updated weights for policy 0, policy_version 5929 (0.0007)
[2023-03-02 18:38:31,613][1045499] Updated weights for policy 0, policy_version 5939 (0.0007)
[2023-03-02 18:38:32,448][1045499] Updated weights for policy 0, policy_version 5949 (0.0007)
[2023-03-02 18:38:33,271][1045499] Updated weights for policy 0, policy_version 5959 (0.0008)
[2023-03-02 18:38:34,092][1045499] Updated weights for policy 0, policy_version 5969 (0.0007)
[2023-03-02 18:38:34,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12364.4). Total num frames: 6114304. Throughput: 0: 12360.3. Samples: 4465912. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:34,314][1045180] Avg episode reward: [(0, '7.973')]
[2023-03-02 18:38:34,923][1045499] Updated weights for policy 0, policy_version 5979 (0.0007)
[2023-03-02 18:38:35,763][1045499] Updated weights for policy 0, policy_version 5989 (0.0006)
[2023-03-02 18:38:36,624][1045499] Updated weights for policy 0, policy_version 5999 (0.0006)
[2023-03-02 18:38:37,431][1045499] Updated weights for policy 0, policy_version 6009 (0.0006)
[2023-03-02 18:38:38,258][1045499] Updated weights for policy 0, policy_version 6019 (0.0007)
[2023-03-02 18:38:39,111][1045499] Updated weights for policy 0, policy_version 6029 (0.0006)
[2023-03-02 18:38:39,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12357.4). Total num frames: 6175744. Throughput: 0: 12363.4. Samples: 4539770. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:39,314][1045180] Avg episode reward: [(0, '7.047')]
[2023-03-02 18:38:39,958][1045499] Updated weights for policy 0, policy_version 6039 (0.0007)
[2023-03-02 18:38:40,786][1045499] Updated weights for policy 0, policy_version 6049 (0.0007)
[2023-03-02 18:38:41,593][1045499] Updated weights for policy 0, policy_version 6059 (0.0007)
[2023-03-02 18:38:42,421][1045499] Updated weights for policy 0, policy_version 6069 (0.0006)
[2023-03-02 18:38:43,254][1045499] Updated weights for policy 0, policy_version 6079 (0.0007)
[2023-03-02 18:38:44,101][1045499] Updated weights for policy 0, policy_version 6089 (0.0007)
[2023-03-02 18:38:44,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12356.2, 300 sec: 12357.4). Total num frames: 6237184. Throughput: 0: 12348.1. Samples: 4576838. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:38:44,314][1045180] Avg episode reward: [(0, '7.718')]
[2023-03-02 18:38:44,942][1045499] Updated weights for policy 0, policy_version 6099 (0.0007)
[2023-03-02 18:38:45,769][1045499] Updated weights for policy 0, policy_version 6109 (0.0007)
[2023-03-02 18:38:46,595][1045499] Updated weights for policy 0, policy_version 6119 (0.0007)
[2023-03-02 18:38:47,437][1045499] Updated weights for policy 0, policy_version 6129 (0.0007)
[2023-03-02 18:38:48,268][1045499] Updated weights for policy 0, policy_version 6139 (0.0006)
[2023-03-02 18:38:49,066][1045499] Updated weights for policy 0, policy_version 6149 (0.0006)
[2023-03-02 18:38:49,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12357.4). Total num frames: 6298624. Throughput: 0: 12348.9. Samples: 4650568. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:49,314][1045180] Avg episode reward: [(0, '7.382')]
[2023-03-02 18:38:49,922][1045499] Updated weights for policy 0, policy_version 6159 (0.0007)
[2023-03-02 18:38:50,746][1045499] Updated weights for policy 0, policy_version 6169 (0.0006)
[2023-03-02 18:38:51,578][1045499] Updated weights for policy 0, policy_version 6179 (0.0007)
[2023-03-02 18:38:52,424][1045499] Updated weights for policy 0, policy_version 6189 (0.0007)
[2023-03-02 18:38:53,272][1045499] Updated weights for policy 0, policy_version 6199 (0.0008)
[2023-03-02 18:38:54,110][1045499] Updated weights for policy 0, policy_version 6209 (0.0006)
[2023-03-02 18:38:54,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12339.2, 300 sec: 12354.0). Total num frames: 6360064. Throughput: 0: 12337.4. Samples: 4724101. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:38:54,314][1045180] Avg episode reward: [(0, '7.240')]
[2023-03-02 18:38:54,924][1045499] Updated weights for policy 0, policy_version 6219 (0.0006)
[2023-03-02 18:38:55,755][1045499] Updated weights for policy 0, policy_version 6229 (0.0006)
[2023-03-02 18:38:56,587][1045499] Updated weights for policy 0, policy_version 6239 (0.0006)
[2023-03-02 18:38:57,413][1045499] Updated weights for policy 0, policy_version 6249 (0.0007)
[2023-03-02 18:38:58,241][1045499] Updated weights for policy 0, policy_version 6259 (0.0006)
[2023-03-02 18:38:59,044][1045499] Updated weights for policy 0, policy_version 6269 (0.0006)
[2023-03-02 18:38:59,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12339.2, 300 sec: 12354.0). Total num frames: 6422528. Throughput: 0: 12333.6. Samples: 4761209. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:38:59,314][1045180] Avg episode reward: [(0, '7.759')]
[2023-03-02 18:38:59,868][1045499] Updated weights for policy 0, policy_version 6279 (0.0007)
[2023-03-02 18:39:00,674][1045499] Updated weights for policy 0, policy_version 6289 (0.0007)
[2023-03-02 18:39:01,510][1045499] Updated weights for policy 0, policy_version 6299 (0.0007)
[2023-03-02 18:39:02,336][1045499] Updated weights for policy 0, policy_version 6309 (0.0007)
[2023-03-02 18:39:03,172][1045499] Updated weights for policy 0, policy_version 6319 (0.0008)
[2023-03-02 18:39:03,983][1045499] Updated weights for policy 0, policy_version 6329 (0.0007)
[2023-03-02 18:39:04,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12373.4, 300 sec: 12360.9). Total num frames: 6484992. Throughput: 0: 12354.6. Samples: 4836137. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:04,314][1045180] Avg episode reward: [(0, '6.903')]
[2023-03-02 18:39:04,803][1045499] Updated weights for policy 0, policy_version 6339 (0.0006)
[2023-03-02 18:39:05,616][1045499] Updated weights for policy 0, policy_version 6349 (0.0007)
[2023-03-02 18:39:06,437][1045499] Updated weights for policy 0, policy_version 6359 (0.0006)
[2023-03-02 18:39:07,280][1045499] Updated weights for policy 0, policy_version 6369 (0.0007)
[2023-03-02 18:39:08,092][1045499] Updated weights for policy 0, policy_version 6379 (0.0007)
[2023-03-02 18:39:08,918][1045499] Updated weights for policy 0, policy_version 6389 (0.0007)
[2023-03-02 18:39:09,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12356.2, 300 sec: 12357.4). Total num frames: 6546432. Throughput: 0: 12360.0. Samples: 4910657. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:09,314][1045180] Avg episode reward: [(0, '7.851')]
[2023-03-02 18:39:09,729][1045499] Updated weights for policy 0, policy_version 6399 (0.0007)
[2023-03-02 18:39:10,591][1045499] Updated weights for policy 0, policy_version 6409 (0.0007)
[2023-03-02 18:39:11,417][1045499] Updated weights for policy 0, policy_version 6419 (0.0007)
[2023-03-02 18:39:12,256][1045499] Updated weights for policy 0, policy_version 6429 (0.0006)
[2023-03-02 18:39:13,106][1045499] Updated weights for policy 0, policy_version 6439 (0.0007)
[2023-03-02 18:39:13,925][1045499] Updated weights for policy 0, policy_version 6449 (0.0006)
[2023-03-02 18:39:14,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12357.4). Total num frames: 6607872. Throughput: 0: 12350.9. Samples: 4947614. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
[2023-03-02 18:39:14,314][1045180] Avg episode reward: [(0, '7.440')]
[2023-03-02 18:39:14,736][1045499] Updated weights for policy 0, policy_version 6459 (0.0007)
[2023-03-02 18:39:15,563][1045499] Updated weights for policy 0, policy_version 6469 (0.0006)
[2023-03-02 18:39:16,386][1045499] Updated weights for policy 0, policy_version 6479 (0.0006)
[2023-03-02 18:39:17,213][1045499] Updated weights for policy 0, policy_version 6489 (0.0006)
[2023-03-02 18:39:18,031][1045499] Updated weights for policy 0, policy_version 6499 (0.0006)
[2023-03-02 18:39:18,843][1045499] Updated weights for policy 0, policy_version 6509 (0.0006)
[2023-03-02 18:39:19,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12356.3, 300 sec: 12360.9). Total num frames: 6670336. Throughput: 0: 12355.5. Samples: 5021908. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:19,314][1045180] Avg episode reward: [(0, '7.807')]
[2023-03-02 18:39:19,658][1045499] Updated weights for policy 0, policy_version 6519 (0.0007)
[2023-03-02 18:39:20,481][1045499] Updated weights for policy 0, policy_version 6529 (0.0006)
[2023-03-02 18:39:21,332][1045499] Updated weights for policy 0, policy_version 6539 (0.0006)
[2023-03-02 18:39:22,144][1045499] Updated weights for policy 0, policy_version 6549 (0.0007)
[2023-03-02 18:39:22,945][1045499] Updated weights for policy 0, policy_version 6559 (0.0006)
[2023-03-02 18:39:23,783][1045499] Updated weights for policy 0, policy_version 6569 (0.0006)
[2023-03-02 18:39:24,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12373.3, 300 sec: 12364.4). Total num frames: 6732800. Throughput: 0: 12382.7. Samples: 5096993. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
[2023-03-02 18:39:24,314][1045180] Avg episode reward: [(0, '8.402')]
[2023-03-02 18:39:24,581][1045499] Updated weights for policy 0, policy_version 6579 (0.0007)
[2023-03-02 18:39:25,420][1045499] Updated weights for policy 0, policy_version 6589 (0.0007)
[2023-03-02 18:39:26,288][1045499] Updated weights for policy 0, policy_version 6599 (0.0007)
[2023-03-02 18:39:27,091][1045499] Updated weights for policy 0, policy_version 6609 (0.0007)
[2023-03-02 18:39:27,927][1045499] Updated weights for policy 0, policy_version 6619 (0.0007)
[2023-03-02 18:39:28,748][1045499] Updated weights for policy 0, policy_version 6629 (0.0007)
[2023-03-02 18:39:29,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12356.2, 300 sec: 12364.4). Total num frames: 6794240. Throughput: 0: 12375.7. Samples: 5133742. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:39:29,314][1045180] Avg episode reward: [(0, '8.741')]
[2023-03-02 18:39:29,604][1045499] Updated weights for policy 0, policy_version 6639 (0.0006)
[2023-03-02 18:39:30,422][1045499] Updated weights for policy 0, policy_version 6649 (0.0007)
[2023-03-02 18:39:31,236][1045499] Updated weights for policy 0, policy_version 6659 (0.0007)
[2023-03-02 18:39:32,089][1045499] Updated weights for policy 0, policy_version 6669 (0.0007)
[2023-03-02 18:39:32,908][1045499] Updated weights for policy 0, policy_version 6679 (0.0007)
[2023-03-02 18:39:33,712][1045499] Updated weights for policy 0, policy_version 6689 (0.0006)
[2023-03-02 18:39:34,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12356.3, 300 sec: 12360.9). Total num frames: 6855680. Throughput: 0: 12383.7. Samples: 5207834. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:39:34,314][1045180] Avg episode reward: [(0, '8.348')]
[2023-03-02 18:39:34,554][1045499] Updated weights for policy 0, policy_version 6699 (0.0006)
[2023-03-02 18:39:35,354][1045499] Updated weights for policy 0, policy_version 6709 (0.0006)
[2023-03-02 18:39:36,206][1045499] Updated weights for policy 0, policy_version 6719 (0.0007)
[2023-03-02 18:39:37,052][1045499] Updated weights for policy 0, policy_version 6729 (0.0007)
[2023-03-02 18:39:37,883][1045499] Updated weights for policy 0, policy_version 6739 (0.0006)
[2023-03-02 18:39:38,722][1045499] Updated weights for policy 0, policy_version 6749 (0.0007)
[2023-03-02 18:39:39,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12356.3, 300 sec: 12360.9). Total num frames: 6917120. Throughput: 0: 12386.1. Samples: 5281475. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:39,314][1045180] Avg episode reward: [(0, '7.601')]
[2023-03-02 18:39:39,573][1045499] Updated weights for policy 0, policy_version 6759 (0.0006)
[2023-03-02 18:39:40,409][1045499] Updated weights for policy 0, policy_version 6769 (0.0008)
[2023-03-02 18:39:41,243][1045499] Updated weights for policy 0, policy_version 6779 (0.0007)
[2023-03-02 18:39:42,062][1045499] Updated weights for policy 0, policy_version 6789 (0.0007)
[2023-03-02 18:39:42,895][1045499] Updated weights for policy 0, policy_version 6799 (0.0006)
[2023-03-02 18:39:43,724][1045499] Updated weights for policy 0, policy_version 6809 (0.0006)
[2023-03-02 18:39:44,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12373.3, 300 sec: 12364.4). Total num frames: 6979584. Throughput: 0: 12380.0. Samples: 5318310. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:44,314][1045180] Avg episode reward: [(0, '8.140')]
[2023-03-02 18:39:44,572][1045499] Updated weights for policy 0, policy_version 6819 (0.0007)
[2023-03-02 18:39:45,392][1045499] Updated weights for policy 0, policy_version 6829 (0.0006)
[2023-03-02 18:39:46,206][1045499] Updated weights for policy 0, policy_version 6839 (0.0007)
[2023-03-02 18:39:47,064][1045499] Updated weights for policy 0, policy_version 6849 (0.0006)
[2023-03-02 18:39:47,859][1045499] Updated weights for policy 0, policy_version 6859 (0.0006)
[2023-03-02 18:39:48,718][1045499] Updated weights for policy 0, policy_version 6869 (0.0006)
[2023-03-02 18:39:49,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12373.3, 300 sec: 12364.4). Total num frames: 7041024. Throughput: 0: 12366.6. Samples: 5392634. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:39:49,314][1045180] Avg episode reward: [(0, '8.316')]
[2023-03-02 18:39:49,543][1045499] Updated weights for policy 0, policy_version 6879 (0.0006)
[2023-03-02 18:39:50,350][1045499] Updated weights for policy 0, policy_version 6889 (0.0006)
[2023-03-02 18:39:51,182][1045499] Updated weights for policy 0, policy_version 6899 (0.0007)
[2023-03-02 18:39:52,016][1045499] Updated weights for policy 0, policy_version 6909 (0.0008)
[2023-03-02 18:39:52,835][1045499] Updated weights for policy 0, policy_version 6919 (0.0007)
[2023-03-02 18:39:53,647][1045499] Updated weights for policy 0, policy_version 6929 (0.0007)
[2023-03-02 18:39:54,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12390.4, 300 sec: 12367.8). Total num frames: 7103488. Throughput: 0: 12359.8. Samples: 5466845. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:39:54,313][1045180] Avg episode reward: [(0, '8.191')]
[2023-03-02 18:39:54,489][1045499] Updated weights for policy 0, policy_version 6939 (0.0006)
[2023-03-02 18:39:55,310][1045499] Updated weights for policy 0, policy_version 6949 (0.0007)
[2023-03-02 18:39:56,148][1045499] Updated weights for policy 0, policy_version 6959 (0.0007)
[2023-03-02 18:39:57,002][1045499] Updated weights for policy 0, policy_version 6969 (0.0007)
[2023-03-02 18:39:57,819][1045499] Updated weights for policy 0, policy_version 6979 (0.0007)
[2023-03-02 18:39:58,629][1045499] Updated weights for policy 0, policy_version 6989 (0.0007)
[2023-03-02 18:39:59,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12367.8). Total num frames: 7164928. Throughput: 0: 12353.9. Samples: 5503541. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:39:59,314][1045180] Avg episode reward: [(0, '7.433')]
[2023-03-02 18:39:59,444][1045499] Updated weights for policy 0, policy_version 6999 (0.0007)
[2023-03-02 18:40:00,278][1045499] Updated weights for policy 0, policy_version 7009 (0.0007)
[2023-03-02 18:40:01,101][1045499] Updated weights for policy 0, policy_version 7019 (0.0006)
[2023-03-02 18:40:01,945][1045499] Updated weights for policy 0, policy_version 7029 (0.0007)
[2023-03-02 18:40:02,761][1045499] Updated weights for policy 0, policy_version 7039 (0.0006)
[2023-03-02 18:40:03,628][1045499] Updated weights for policy 0, policy_version 7049 (0.0007)
[2023-03-02 18:40:04,313][1045180] Fps is (10 sec: 12287.9, 60 sec: 12356.3, 300 sec: 12364.4). Total num frames: 7226368. Throughput: 0: 12357.6. Samples: 5578000. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:40:04,314][1045180] Avg episode reward: [(0, '19.562')]
[2023-03-02 18:40:04,460][1045499] Updated weights for policy 0, policy_version 7059 (0.0008)
[2023-03-02 18:40:05,296][1045499] Updated weights for policy 0, policy_version 7069 (0.0006)
[2023-03-02 18:40:06,124][1045499] Updated weights for policy 0, policy_version 7079 (0.0007)
[2023-03-02 18:40:06,958][1045499] Updated weights for policy 0, policy_version 7089 (0.0007)
[2023-03-02 18:40:07,821][1045499] Updated weights for policy 0, policy_version 7099 (0.0008)
[2023-03-02 18:40:08,635][1045499] Updated weights for policy 0, policy_version 7109 (0.0007)
[2023-03-02 18:40:09,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12356.3, 300 sec: 12364.4). Total num frames: 7287808. Throughput: 0: 12328.9. Samples: 5651795. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:40:09,314][1045180] Avg episode reward: [(0, '7.684')]
[2023-03-02 18:40:09,438][1045499] Updated weights for policy 0, policy_version 7119 (0.0006)
[2023-03-02 18:40:10,261][1045499] Updated weights for policy 0, policy_version 7129 (0.0006)
[2023-03-02 18:40:11,074][1045499] Updated weights for policy 0, policy_version 7139 (0.0007)
[2023-03-02 18:40:11,893][1045499] Updated weights for policy 0, policy_version 7149 (0.0007)
[2023-03-02 18:40:12,712][1045499] Updated weights for policy 0, policy_version 7159 (0.0007)
[2023-03-02 18:40:13,529][1045499] Updated weights for policy 0, policy_version 7169 (0.0006)
[2023-03-02 18:40:14,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12373.3, 300 sec: 12367.8). Total num frames: 7350272. Throughput: 0: 12348.8. Samples: 5689438. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:40:14,314][1045180] Avg episode reward: [(0, '7.718')]
[2023-03-02 18:40:14,365][1045499] Updated weights for policy 0, policy_version 7179 (0.0006)
[2023-03-02 18:40:15,197][1045499] Updated weights for policy 0, policy_version 7189 (0.0006)
[2023-03-02 18:40:16,018][1045499] Updated weights for policy 0, policy_version 7199 (0.0006)
[2023-03-02 18:40:16,851][1045499] Updated weights for policy 0, policy_version 7209 (0.0006)
[2023-03-02 18:40:17,665][1045499] Updated weights for policy 0, policy_version 7219 (0.0006)
[2023-03-02 18:40:18,520][1045499] Updated weights for policy 0, policy_version 7229 (0.0007)
[2023-03-02 18:40:19,313][1045180] Fps is (10 sec: 12390.5, 60 sec: 12356.3, 300 sec: 12367.8). Total num frames: 7411712. Throughput: 0: 12348.9. Samples: 5763534. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:19,314][1045180] Avg episode reward: [(0, '6.800')]
[2023-03-02 18:40:19,317][1045448] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000007238_7411712.pth...
[2023-03-02 18:40:19,351][1045448] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000004340_4444160.pth
[2023-03-02 18:40:19,376][1045499] Updated weights for policy 0, policy_version 7239 (0.0007)
[2023-03-02 18:40:20,176][1045499] Updated weights for policy 0, policy_version 7249 (0.0007)
[2023-03-02 18:40:21,011][1045499] Updated weights for policy 0, policy_version 7259 (0.0006)
[2023-03-02 18:40:21,824][1045499] Updated weights for policy 0, policy_version 7269 (0.0007)
[2023-03-02 18:40:22,637][1045499] Updated weights for policy 0, policy_version 7279 (0.0006)
[2023-03-02 18:40:23,475][1045499] Updated weights for policy 0, policy_version 7289 (0.0007)
[2023-03-02 18:40:24,280][1045499] Updated weights for policy 0, policy_version 7299 (0.0006)
[2023-03-02 18:40:24,313][1045180] Fps is (10 sec: 12390.3, 60 sec: 12356.3, 300 sec: 12374.8). Total num frames: 7474176. Throughput: 0: 12367.8. Samples: 5838027. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:24,314][1045180] Avg episode reward: [(0, '7.901')]
[2023-03-02 18:40:25,112][1045499] Updated weights for policy 0, policy_version 7309 (0.0007)
[2023-03-02 18:40:25,927][1045499] Updated weights for policy 0, policy_version 7319 (0.0006)
[2023-03-02 18:40:26,750][1045499] Updated weights for policy 0, policy_version 7329 (0.0007)
[2023-03-02 18:40:27,579][1045499] Updated weights for policy 0, policy_version 7339 (0.0007)
[2023-03-02 18:40:28,428][1045499] Updated weights for policy 0, policy_version 7349 (0.0006)
[2023-03-02 18:40:29,234][1045499] Updated weights for policy 0, policy_version 7359 (0.0008)
[2023-03-02 18:40:29,313][1045180] Fps is (10 sec: 12492.8, 60 sec: 12373.3, 300 sec: 12374.8). Total num frames: 7536640. Throughput: 0: 12377.7. Samples: 5875306. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:29,314][1045180] Avg episode reward: [(0, '7.309')]
[2023-03-02 18:40:30,051][1045499] Updated weights for policy 0, policy_version 7369 (0.0006)
[2023-03-02 18:40:30,896][1045499] Updated weights for policy 0, policy_version 7379 (0.0006)
[2023-03-02 18:40:31,749][1045499] Updated weights for policy 0, policy_version 7389 (0.0006)
[2023-03-02 18:40:32,611][1045499] Updated weights for policy 0, policy_version 7399 (0.0007)
[2023-03-02 18:40:33,443][1045499] Updated weights for policy 0, policy_version 7409 (0.0006)
[2023-03-02 18:40:34,278][1045499] Updated weights for policy 0, policy_version 7419 (0.0006)
[2023-03-02 18:40:34,313][1045180] Fps is (10 sec: 12288.1, 60 sec: 12356.3, 300 sec: 12367.9). Total num frames: 7597056. Throughput: 0: 12356.7. Samples: 5948685. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:34,314][1045180] Avg episode reward: [(0, '9.203')]
[2023-03-02 18:40:35,131][1045499] Updated weights for policy 0, policy_version 7429 (0.0007)
[2023-03-02 18:40:35,950][1045499] Updated weights for policy 0, policy_version 7439 (0.0007)
[2023-03-02 18:40:36,799][1045499] Updated weights for policy 0, policy_version 7449 (0.0007)
[2023-03-02 18:40:37,620][1045499] Updated weights for policy 0, policy_version 7459 (0.0007)
[2023-03-02 18:40:38,458][1045499] Updated weights for policy 0, policy_version 7469 (0.0007)
[2023-03-02 18:40:39,280][1045499] Updated weights for policy 0, policy_version 7479 (0.0006)
[2023-03-02 18:40:39,313][1045180] Fps is (10 sec: 12185.5, 60 sec: 12356.3, 300 sec: 12367.8). Total num frames: 7658496. Throughput: 0: 12350.2. Samples: 6022608. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:40:39,314][1045180] Avg episode reward: [(0, '7.892')]
[2023-03-02 18:40:40,126][1045499] Updated weights for policy 0, policy_version 7489 (0.0006)
[2023-03-02 18:40:40,956][1045499] Updated weights for policy 0, policy_version 7499 (0.0007)
[2023-03-02 18:40:41,769][1045499] Updated weights for policy 0, policy_version 7509 (0.0006)
[2023-03-02 18:40:42,603][1045499] Updated weights for policy 0, policy_version 7519 (0.0008)
[2023-03-02 18:40:43,432][1045499] Updated weights for policy 0, policy_version 7529 (0.0007)
[2023-03-02 18:40:44,260][1045499] Updated weights for policy 0, policy_version 7539 (0.0006)
[2023-03-02 18:40:44,313][1045180] Fps is (10 sec: 12288.0, 60 sec: 12339.2, 300 sec: 12367.8). Total num frames: 7719936. Throughput: 0: 12347.9. Samples: 6059196. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:44,314][1045180] Avg episode reward: [(0, '7.893')]
[2023-03-02 18:40:45,078][1045499] Updated weights for policy 0, policy_version 7549 (0.0006)
[2023-03-02 18:40:45,912][1045499] Updated weights for policy 0, policy_version 7559 (0.0005)
[2023-03-02 18:40:46,742][1045499] Updated weights for policy 0, policy_version 7569 (0.0006)
[2023-03-02 18:40:47,555][1045499] Updated weights for policy 0, policy_version 7579 (0.0007)
[2023-03-02 18:40:48,386][1045499] Updated weights for policy 0, policy_version 7589 (0.0007)
[2023-03-02 18:40:49,204][1045499] Updated weights for policy 0, policy_version 7599 (0.0006)
[2023-03-02 18:40:49,313][1045180] Fps is (10 sec: 12390.4, 60 sec: 12356.2, 300 sec: 12367.8). Total num frames: 7782400. Throughput: 0: 12349.7. Samples: 6133737. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
[2023-03-02 18:40:49,314][1045180] Avg episode reward: [(0, '7.195')]
[2023-03-02 18:40:50,046][1045499] Updated weights for policy 0, policy_version 7609 (0.0007)
[2023-03-02 18:40:50,908][1045499] Updated weights for policy 0, policy_version 7619 (0.0006)
[2023-03-02 18:40:51,739][1045499] Updated weights for policy 0, policy_version 7629 (0.0006)
[2023-03-02 18:40:52,549][1045499] Updated weights for policy 0, policy_version 7639 (0.0007)
[2023-03-02 18:40:53,390][1045499] Updated weights for policy 0, policy_version 7649 (0.0006)
[2023-03-02 18:40:54,226][1045499] Updated weights for policy 0, policy_version 7659 (0.0006)
[2023-03-02 18:40:54,313][1045180] Fps is (10 sec: 12287.7, 60 sec: 12322.0, 300 sec: 12360.9). Total num frames: 7842816. Throughput: 0: 12345.7. Samples: 6207353. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:54,314][1045180] Avg episode reward: [(0, '8.553')]
[2023-03-02 18:40:55,052][1045499] Updated weights for policy 0, policy_version 7669 (0.0006)
[2023-03-02 18:40:55,893][1045499] Updated weights for policy 0, policy_version 7679 (0.0007)
[2023-03-02 18:40:56,714][1045499] Updated weights for policy 0, policy_version 7689 (0.0006)
[2023-03-02 18:40:57,575][1045499] Updated weights for policy 0, policy_version 7699 (0.0007)
[2023-03-02 18:40:58,393][1045499] Updated weights for policy 0, policy_version 7709 (0.0006)
[2023-03-02 18:40:59,224][1045499] Updated weights for policy 0, policy_version 7719 (0.0006)
[2023-03-02 18:40:59,313][1045180] Fps is (10 sec: 12185.6, 60 sec: 12322.1, 300 sec: 12360.9). Total num frames: 7904256. Throughput: 0: 12326.2. Samples: 6244117. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
[2023-03-02 18:40:59,314][1045180] Avg episode reward: [(0, '8.064')]
[2023-03-02 18:40:59,383][1045180] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 1045180], exiting...
[2023-03-02 18:40:59,384][1045601] Stopping RolloutWorker_w10...
[2023-03-02 18:40:59,384][1045933] Stopping RolloutWorker_w27...
[2023-03-02 18:40:59,384][1045666] Stopping RolloutWorker_w8...
[2023-03-02 18:40:59,384][1045932] Stopping RolloutWorker_w28...
[2023-03-02 18:40:59,384][1045834] Stopping RolloutWorker_w22...
[2023-03-02 18:40:59,384][1045669] Stopping RolloutWorker_w17...
[2023-03-02 18:40:59,384][1045601] Loop rollout_proc10_evt_loop terminating...
[2023-03-02 18:40:59,384][1045770] Stopping RolloutWorker_w21...
[2023-03-02 18:40:59,384][1045670] Stopping RolloutWorker_w16...
[2023-03-02 18:40:59,384][1045706] Stopping RolloutWorker_w19...
[2023-03-02 18:40:59,384][1045503] Stopping RolloutWorker_w3...
[2023-03-02 18:40:59,384][1045667] Stopping RolloutWorker_w13...
[2023-03-02 18:40:59,384][1045504] Stopping RolloutWorker_w4...
[2023-03-02 18:40:59,384][1045738] Stopping RolloutWorker_w20...
[2023-03-02 18:40:59,385][1045666] Loop rollout_proc8_evt_loop terminating...
[2023-03-02 18:40:59,384][1045930] Stopping RolloutWorker_w26...
[2023-03-02 18:40:59,384][1045665] Stopping RolloutWorker_w9...
[2023-03-02 18:40:59,384][1045929] Stopping RolloutWorker_w24...
[2023-03-02 18:40:59,385][1045834] Loop rollout_proc22_evt_loop terminating...
[2023-03-02 18:40:59,385][1045933] Loop rollout_proc27_evt_loop terminating...
[2023-03-02 18:40:59,384][1045997] Stopping RolloutWorker_w30...
[2023-03-02 18:40:59,384][1045180] Runner profile tree view:
main_loop: 517.0250
[2023-03-02 18:40:59,384][1045578] Stopping RolloutWorker_w7...
[2023-03-02 18:40:59,384][1045705] Stopping RolloutWorker_w15...
[2023-03-02 18:40:59,385][1045770] Loop rollout_proc21_evt_loop terminating...
[2023-03-02 18:40:59,385][1045932] Loop rollout_proc28_evt_loop terminating...
[2023-03-02 18:40:59,385][1045706] Loop rollout_proc19_evt_loop terminating...
[2023-03-02 18:40:59,385][1045503] Loop rollout_proc3_evt_loop terminating...
[2023-03-02 18:40:59,384][1045502] Stopping RolloutWorker_w2...
[2023-03-02 18:40:59,385][1045897] Stopping RolloutWorker_w23...
[2023-03-02 18:40:59,384][1045998] Stopping RolloutWorker_w25...
[2023-03-02 18:40:59,385][1045669] Loop rollout_proc17_evt_loop terminating...
[2023-03-02 18:40:59,385][1045670] Loop rollout_proc16_evt_loop terminating...
[2023-03-02 18:40:59,385][1045667] Loop rollout_proc13_evt_loop terminating...
[2023-03-02 18:40:59,385][1045930] Loop rollout_proc26_evt_loop terminating...
[2023-03-02 18:40:59,385][1045929] Loop rollout_proc24_evt_loop terminating...
[2023-03-02 18:40:59,385][1045997] Loop rollout_proc30_evt_loop terminating...
[2023-03-02 18:40:59,385][1045180] Collected {0: 7905280}, FPS: 12123.0
[2023-03-02 18:40:59,385][1045578] Loop rollout_proc7_evt_loop terminating...
[2023-03-02 18:40:59,385][1045665] Loop rollout_proc9_evt_loop terminating...
[2023-03-02 18:40:59,385][1046030] Stopping RolloutWorker_w31...
[2023-03-02 18:40:59,384][1045671] Stopping RolloutWorker_w18...
[2023-03-02 18:40:59,385][1045897] Loop rollout_proc23_evt_loop terminating...
[2023-03-02 18:40:59,385][1045504] Loop rollout_proc4_evt_loop terminating...
[2023-03-02 18:40:59,385][1045705] Loop rollout_proc15_evt_loop terminating...
[2023-03-02 18:40:59,385][1045501] Stopping RolloutWorker_w1...
[2023-03-02 18:40:59,385][1045738] Loop rollout_proc20_evt_loop terminating...
[2023-03-02 18:40:59,385][1045664] Stopping RolloutWorker_w11...
[2023-03-02 18:40:59,385][1045502] Loop rollout_proc2_evt_loop terminating...
[2023-03-02 18:40:59,385][1045998] Loop rollout_proc25_evt_loop terminating...
[2023-03-02 18:40:59,385][1045501] Loop rollout_proc1_evt_loop terminating...
[2023-03-02 18:40:59,385][1045671] Loop rollout_proc18_evt_loop terminating...
[2023-03-02 18:40:59,385][1045664] Loop rollout_proc11_evt_loop terminating...
[2023-03-02 18:40:59,385][1045668] Stopping RolloutWorker_w12...
[2023-03-02 18:40:59,385][1046030] Loop rollout_proc31_evt_loop terminating...
[2023-03-02 18:40:59,386][1045668] Loop rollout_proc12_evt_loop terminating...
[2023-03-02 18:40:59,386][1045965] Stopping RolloutWorker_w29...
[2023-03-02 18:40:59,389][1045965] Loop rollout_proc29_evt_loop terminating...
[2023-03-02 18:40:59,390][1045505] Stopping RolloutWorker_w5...
[2023-03-02 18:40:59,391][1045505] Loop rollout_proc5_evt_loop terminating...
[2023-03-02 18:40:59,399][1045507] Stopping RolloutWorker_w6...
[2023-03-02 18:40:59,400][1045507] Loop rollout_proc6_evt_loop terminating...
[2023-03-02 18:40:59,400][1045448] Stopping Batcher_0...
[2023-03-02 18:40:59,401][1045448] Loop batcher_evt_loop terminating...
[2023-03-02 18:40:59,401][1045500] Stopping RolloutWorker_w0...
[2023-03-02 18:40:59,402][1045500] Loop rollout_proc0_evt_loop terminating...
[2023-03-02 18:40:59,406][1045673] Stopping RolloutWorker_w14...
[2023-03-02 18:40:59,407][1045673] Loop rollout_proc14_evt_loop terminating...
[2023-03-02 18:40:59,423][1045448] Saving /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000007721_7906304.pth...
[2023-03-02 18:40:59,452][1045499] Weights refcount: 2 0
[2023-03-02 18:40:59,454][1045499] Stopping InferenceWorker_p0-w0...
[2023-03-02 18:40:59,454][1045499] Loop inference_proc0-0_evt_loop terminating...
[2023-03-02 18:40:59,537][1045448] Removing /home/qgallouedec/train_dir/default_experiment/checkpoint_p0/checkpoint_000005790_5928960.pth
[2023-03-02 18:40:59,545][1045448] Stopping LearnerWorker_p0...
[2023-03-02 18:40:59,545][1045448] Loop learner_proc0_evt_loop terminating...