2023-03-18 16:21:39 - r - INFO: - Hyperparameters: 2023-03-18 16:21:39 - r - INFO: - ================================================================================ 2023-03-18 16:21:39 - r - INFO: - Name Value Type 2023-03-18 16:21:39 - r - INFO: - env_name CartPole-v1 2023-03-18 16:21:39 - r - INFO: - new_step_api 1 2023-03-18 16:21:39 - r - INFO: - wrapper None 2023-03-18 16:21:39 - r - INFO: - render 0 2023-03-18 16:21:39 - r - INFO: - algo_name NoisyDQN 2023-03-18 16:21:39 - r - INFO: - mode train 2023-03-18 16:21:39 - r - INFO: - seed 1 2023-03-18 16:21:39 - r - INFO: - device cpu 2023-03-18 16:21:39 - r - INFO: - train_eps 100 2023-03-18 16:21:39 - r - INFO: - test_eps 10 2023-03-18 16:21:39 - r - INFO: - eval_eps 10 2023-03-18 16:21:39 - r - INFO: - eval_per_episode 5 2023-03-18 16:21:39 - r - INFO: - max_steps 200 2023-03-18 16:21:39 - r - INFO: - load_checkpoint 0 2023-03-18 16:21:39 - r - INFO: - load_path Train_CartPole-v1_NoisyDQN 2023-03-18 16:21:39 - r - INFO: - show_fig 0 2023-03-18 16:21:39 - r - INFO: - save_fig 1 2023-03-18 16:21:39 - r - INFO: - epsilon_start 0.95 2023-03-18 16:21:39 - r - INFO: - tau 1.0 2023-03-18 16:21:39 - r - INFO: - epsilon_end 0.01 2023-03-18 16:21:39 - r - INFO: - epsilon_decay 500 2023-03-18 16:21:39 - r - INFO: - hidden_dim 256 2023-03-18 16:21:39 - r - INFO: - gamma 0.95 2023-03-18 16:21:39 - r - INFO: - lr 0.0001 2023-03-18 16:21:39 - r - INFO: - buffer_size 100000 2023-03-18 16:21:39 - r - INFO: - batch_size 64 2023-03-18 16:21:39 - r - INFO: - target_update 4 2023-03-18 16:21:39 - r - INFO: - task_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139 2023-03-18 16:21:39 - r - INFO: - res_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/results 2023-03-18 16:21:39 - r - INFO: - log_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/logs 2023-03-18 16:21:39 - r - INFO: - traj_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/traj 2023-03-18 16:21:39 - r - INFO: - tb_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/tb_logs 2023-03-18 16:21:39 - r - INFO: - ================================================================================ 2023-03-18 16:21:39 - r - INFO: - n_states: 4, n_actions: 2 2023-03-18 16:21:39 - r - INFO: - Start training! 2023-03-18 16:21:39 - r - INFO: - Env: CartPole-v1, Algorithm: NoisyDQN, Device: cpu 2023-03-18 16:21:39 - r - INFO: - Episode: 1/100, Reward: 16.000, Step: 16 2023-03-18 16:21:39 - r - INFO: - Episode: 2/100, Reward: 16.000, Step: 16 2023-03-18 16:21:39 - r - INFO: - Episode: 3/100, Reward: 18.000, Step: 18 2023-03-18 16:21:39 - r - INFO: - Episode: 4/100, Reward: 14.000, Step: 14 2023-03-18 16:21:39 - r - INFO: - Episode: 5/100, Reward: 22.000, Step: 22 2023-03-18 16:21:39 - r - INFO: - Current episode 5 has the best eval reward: 9.300 2023-03-18 16:21:39 - r - INFO: - Episode: 6/100, Reward: 27.000, Step: 27 2023-03-18 16:21:40 - r - INFO: - Episode: 7/100, Reward: 9.000, Step: 9 2023-03-18 16:21:40 - r - INFO: - Episode: 8/100, Reward: 13.000, Step: 13 2023-03-18 16:21:40 - r - INFO: - Episode: 9/100, Reward: 17.000, Step: 17 2023-03-18 16:21:40 - r - INFO: - Episode: 10/100, Reward: 37.000, Step: 37 2023-03-18 16:21:40 - r - INFO: - Current episode 10 has the best eval reward: 9.500 2023-03-18 16:21:40 - r - INFO: - Episode: 11/100, Reward: 15.000, Step: 15 2023-03-18 16:21:40 - r - INFO: - Episode: 12/100, Reward: 22.000, Step: 22 2023-03-18 16:21:40 - r - INFO: - Episode: 13/100, Reward: 9.000, Step: 9 2023-03-18 16:21:40 - r - INFO: - Episode: 14/100, Reward: 14.000, Step: 14 2023-03-18 16:21:40 - r - INFO: - Episode: 15/100, Reward: 12.000, Step: 12 2023-03-18 16:21:40 - r - INFO: - Current episode 15 has the best eval reward: 9.700 2023-03-18 16:21:40 - r - INFO: - Episode: 16/100, Reward: 16.000, Step: 16 2023-03-18 16:21:40 - r - INFO: - Episode: 17/100, Reward: 16.000, Step: 16 2023-03-18 16:21:40 - r - INFO: - Episode: 18/100, Reward: 14.000, Step: 14 2023-03-18 16:21:40 - r - INFO: - Episode: 19/100, Reward: 11.000, Step: 11 2023-03-18 16:21:40 - r - INFO: - Episode: 20/100, Reward: 13.000, Step: 13 2023-03-18 16:21:40 - r - INFO: - Current episode 20 has the best eval reward: 9.700 2023-03-18 16:21:40 - r - INFO: - Episode: 21/100, Reward: 13.000, Step: 13 2023-03-18 16:21:42 - r - INFO: - Episode: 22/100, Reward: 14.000, Step: 14 2023-03-18 16:21:42 - r - INFO: - Episode: 23/100, Reward: 14.000, Step: 14 2023-03-18 16:21:42 - r - INFO: - Episode: 24/100, Reward: 37.000, Step: 37 2023-03-18 16:21:42 - r - INFO: - Episode: 25/100, Reward: 12.000, Step: 12 2023-03-18 16:21:42 - r - INFO: - Episode: 26/100, Reward: 18.000, Step: 18 2023-03-18 16:21:42 - r - INFO: - Episode: 27/100, Reward: 13.000, Step: 13 2023-03-18 16:21:42 - r - INFO: - Episode: 28/100, Reward: 20.000, Step: 20 2023-03-18 16:21:43 - r - INFO: - Episode: 29/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 30/100, Reward: 10.000, Step: 10 2023-03-18 16:21:43 - r - INFO: - Current episode 30 has the best eval reward: 13.700 2023-03-18 16:21:43 - r - INFO: - Episode: 31/100, Reward: 10.000, Step: 10 2023-03-18 16:21:43 - r - INFO: - Episode: 32/100, Reward: 12.000, Step: 12 2023-03-18 16:21:43 - r - INFO: - Episode: 33/100, Reward: 11.000, Step: 11 2023-03-18 16:21:43 - r - INFO: - Episode: 34/100, Reward: 12.000, Step: 12 2023-03-18 16:21:43 - r - INFO: - Episode: 35/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Current episode 35 has the best eval reward: 32.500 2023-03-18 16:21:43 - r - INFO: - Episode: 36/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 37/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 38/100, Reward: 23.000, Step: 23 2023-03-18 16:21:43 - r - INFO: - Episode: 39/100, Reward: 35.000, Step: 35 2023-03-18 16:21:43 - r - INFO: - Episode: 40/100, Reward: 46.000, Step: 46 2023-03-18 16:21:44 - r - INFO: - Episode: 41/100, Reward: 10.000, Step: 10 2023-03-18 16:21:44 - r - INFO: - Episode: 42/100, Reward: 13.000, Step: 13 2023-03-18 16:21:44 - r - INFO: - Episode: 43/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 44/100, Reward: 43.000, Step: 43 2023-03-18 16:21:44 - r - INFO: - Episode: 45/100, Reward: 23.000, Step: 23 2023-03-18 16:21:44 - r - INFO: - Episode: 46/100, Reward: 31.000, Step: 31 2023-03-18 16:21:44 - r - INFO: - Episode: 47/100, Reward: 36.000, Step: 36 2023-03-18 16:21:44 - r - INFO: - Episode: 48/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 49/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 50/100, Reward: 40.000, Step: 40 2023-03-18 16:21:44 - r - INFO: - Current episode 50 has the best eval reward: 36.900 2023-03-18 16:21:45 - r - INFO: - Episode: 51/100, Reward: 47.000, Step: 47 2023-03-18 16:21:45 - r - INFO: - Episode: 52/100, Reward: 60.000, Step: 60 2023-03-18 16:21:45 - r - INFO: - Episode: 53/100, Reward: 104.000, Step: 104 2023-03-18 16:21:45 - r - INFO: - Episode: 54/100, Reward: 70.000, Step: 70 2023-03-18 16:21:45 - r - INFO: - Episode: 55/100, Reward: 65.000, Step: 65 2023-03-18 16:21:46 - r - INFO: - Episode: 56/100, Reward: 96.000, Step: 96 2023-03-18 16:21:46 - r - INFO: - Episode: 57/100, Reward: 34.000, Step: 34 2023-03-18 16:21:46 - r - INFO: - Episode: 58/100, Reward: 30.000, Step: 30 2023-03-18 16:21:46 - r - INFO: - Episode: 59/100, Reward: 63.000, Step: 63 2023-03-18 16:21:46 - r - INFO: - Episode: 60/100, Reward: 32.000, Step: 32 2023-03-18 16:21:46 - r - INFO: - Current episode 60 has the best eval reward: 104.900 2023-03-18 16:21:47 - r - INFO: - Episode: 61/100, Reward: 36.000, Step: 36 2023-03-18 16:21:47 - r - INFO: - Episode: 62/100, Reward: 26.000, Step: 26 2023-03-18 16:21:47 - r - INFO: - Episode: 63/100, Reward: 29.000, Step: 29 2023-03-18 16:21:47 - r - INFO: - Episode: 64/100, Reward: 58.000, Step: 58 2023-03-18 16:21:47 - r - INFO: - Episode: 65/100, Reward: 123.000, Step: 123 2023-03-18 16:21:47 - r - INFO: - Episode: 66/100, Reward: 74.000, Step: 74 2023-03-18 16:21:48 - r - INFO: - Episode: 67/100, Reward: 56.000, Step: 56 2023-03-18 16:21:48 - r - INFO: - Episode: 68/100, Reward: 76.000, Step: 76 2023-03-18 16:21:48 - r - INFO: - Episode: 69/100, Reward: 63.000, Step: 63 2023-03-18 16:21:48 - r - INFO: - Episode: 70/100, Reward: 55.000, Step: 55 2023-03-18 16:21:48 - r - INFO: - Episode: 71/100, Reward: 76.000, Step: 76 2023-03-18 16:21:49 - r - INFO: - Episode: 72/100, Reward: 59.000, Step: 59 2023-03-18 16:21:49 - r - INFO: - Episode: 73/100, Reward: 70.000, Step: 70 2023-03-18 16:21:49 - r - INFO: - Episode: 74/100, Reward: 98.000, Step: 98 2023-03-18 16:21:49 - r - INFO: - Episode: 75/100, Reward: 60.000, Step: 60 2023-03-18 16:21:50 - r - INFO: - Episode: 76/100, Reward: 114.000, Step: 114 2023-03-18 16:21:50 - r - INFO: - Episode: 77/100, Reward: 200.000, Step: 200 2023-03-18 16:21:51 - r - INFO: - Episode: 78/100, Reward: 199.000, Step: 199 2023-03-18 16:21:51 - r - INFO: - Episode: 79/100, Reward: 200.000, Step: 200 2023-03-18 16:21:52 - r - INFO: - Episode: 80/100, Reward: 200.000, Step: 200 2023-03-18 16:21:52 - r - INFO: - Current episode 80 has the best eval reward: 200.000 2023-03-18 16:21:53 - r - INFO: - Episode: 81/100, Reward: 200.000, Step: 200 2023-03-18 16:21:53 - r - INFO: - Episode: 82/100, Reward: 200.000, Step: 200 2023-03-18 16:21:54 - r - INFO: - Episode: 83/100, Reward: 200.000, Step: 200 2023-03-18 16:21:55 - r - INFO: - Episode: 84/100, Reward: 200.000, Step: 200 2023-03-18 16:21:55 - r - INFO: - Episode: 85/100, Reward: 200.000, Step: 200 2023-03-18 16:21:56 - r - INFO: - Current episode 85 has the best eval reward: 200.000 2023-03-18 16:21:56 - r - INFO: - Episode: 86/100, Reward: 200.000, Step: 200 2023-03-18 16:21:57 - r - INFO: - Episode: 87/100, Reward: 200.000, Step: 200 2023-03-18 16:21:57 - r - INFO: - Episode: 88/100, Reward: 200.000, Step: 200 2023-03-18 16:21:58 - r - INFO: - Episode: 89/100, Reward: 200.000, Step: 200 2023-03-18 16:21:58 - r - INFO: - Episode: 90/100, Reward: 200.000, Step: 200 2023-03-18 16:21:59 - r - INFO: - Current episode 90 has the best eval reward: 200.000 2023-03-18 16:21:59 - r - INFO: - Episode: 91/100, Reward: 200.000, Step: 200 2023-03-18 16:22:00 - r - INFO: - Episode: 92/100, Reward: 200.000, Step: 200 2023-03-18 16:22:01 - r - INFO: - Episode: 93/100, Reward: 200.000, Step: 200 2023-03-18 16:22:01 - r - INFO: - Episode: 94/100, Reward: 200.000, Step: 200 2023-03-18 16:22:02 - r - INFO: - Episode: 95/100, Reward: 200.000, Step: 200 2023-03-18 16:22:02 - r - INFO: - Current episode 95 has the best eval reward: 200.000 2023-03-18 16:22:03 - r - INFO: - Episode: 96/100, Reward: 200.000, Step: 200 2023-03-18 16:22:03 - r - INFO: - Episode: 97/100, Reward: 200.000, Step: 200 2023-03-18 16:22:04 - r - INFO: - Episode: 98/100, Reward: 200.000, Step: 200 2023-03-18 16:22:04 - r - INFO: - Episode: 99/100, Reward: 200.000, Step: 200 2023-03-18 16:22:05 - r - INFO: - Episode: 100/100, Reward: 200.000, Step: 200 2023-03-18 16:22:05 - r - INFO: - Current episode 100 has the best eval reward: 200.000 2023-03-18 16:22:05 - r - INFO: - Finish training!