File size: 12,847 Bytes
62e03a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 |
2023-03-18 16:21:39 - r - INFO: - Hyperparameters: 2023-03-18 16:21:39 - r - INFO: - ================================================================================ 2023-03-18 16:21:39 - r - INFO: - Name Value Type 2023-03-18 16:21:39 - r - INFO: - env_name CartPole-v1 <class 'str'> 2023-03-18 16:21:39 - r - INFO: - new_step_api 1 <class 'bool'> 2023-03-18 16:21:39 - r - INFO: - wrapper None <class 'str'> 2023-03-18 16:21:39 - r - INFO: - render 0 <class 'bool'> 2023-03-18 16:21:39 - r - INFO: - algo_name NoisyDQN <class 'str'> 2023-03-18 16:21:39 - r - INFO: - mode train <class 'str'> 2023-03-18 16:21:39 - r - INFO: - seed 1 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - device cpu <class 'str'> 2023-03-18 16:21:39 - r - INFO: - train_eps 100 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - test_eps 10 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - eval_eps 10 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - eval_per_episode 5 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - max_steps 200 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - load_checkpoint 0 <class 'bool'> 2023-03-18 16:21:39 - r - INFO: - load_path Train_CartPole-v1_NoisyDQN <class 'str'> 2023-03-18 16:21:39 - r - INFO: - show_fig 0 <class 'bool'> 2023-03-18 16:21:39 - r - INFO: - save_fig 1 <class 'bool'> 2023-03-18 16:21:39 - r - INFO: - epsilon_start 0.95 <class 'float'> 2023-03-18 16:21:39 - r - INFO: - tau 1.0 <class 'float'> 2023-03-18 16:21:39 - r - INFO: - epsilon_end 0.01 <class 'float'> 2023-03-18 16:21:39 - r - INFO: - epsilon_decay 500 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - hidden_dim 256 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - gamma 0.95 <class 'float'> 2023-03-18 16:21:39 - r - INFO: - lr 0.0001 <class 'float'> 2023-03-18 16:21:39 - r - INFO: - buffer_size 100000 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - batch_size 64 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - target_update 4 <class 'int'> 2023-03-18 16:21:39 - r - INFO: - task_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139 <class 'str'> 2023-03-18 16:21:39 - r - INFO: - res_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/results <class 'str'> 2023-03-18 16:21:39 - r - INFO: - log_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/logs <class 'str'> 2023-03-18 16:21:39 - r - INFO: - traj_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/traj <class 'str'> 2023-03-18 16:21:39 - r - INFO: - tb_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CartPole-v1_NoisyDQN_20230318-162139/tb_logs <class 'str'> 2023-03-18 16:21:39 - r - INFO: - ================================================================================ 2023-03-18 16:21:39 - r - INFO: - n_states: 4, n_actions: 2 2023-03-18 16:21:39 - r - INFO: - Start training! 2023-03-18 16:21:39 - r - INFO: - Env: CartPole-v1, Algorithm: NoisyDQN, Device: cpu 2023-03-18 16:21:39 - r - INFO: - Episode: 1/100, Reward: 16.000, Step: 16 2023-03-18 16:21:39 - r - INFO: - Episode: 2/100, Reward: 16.000, Step: 16 2023-03-18 16:21:39 - r - INFO: - Episode: 3/100, Reward: 18.000, Step: 18 2023-03-18 16:21:39 - r - INFO: - Episode: 4/100, Reward: 14.000, Step: 14 2023-03-18 16:21:39 - r - INFO: - Episode: 5/100, Reward: 22.000, Step: 22 2023-03-18 16:21:39 - r - INFO: - Current episode 5 has the best eval reward: 9.300 2023-03-18 16:21:39 - r - INFO: - Episode: 6/100, Reward: 27.000, Step: 27 2023-03-18 16:21:40 - r - INFO: - Episode: 7/100, Reward: 9.000, Step: 9 2023-03-18 16:21:40 - r - INFO: - Episode: 8/100, Reward: 13.000, Step: 13 2023-03-18 16:21:40 - r - INFO: - Episode: 9/100, Reward: 17.000, Step: 17 2023-03-18 16:21:40 - r - INFO: - Episode: 10/100, Reward: 37.000, Step: 37 2023-03-18 16:21:40 - r - INFO: - Current episode 10 has the best eval reward: 9.500 2023-03-18 16:21:40 - r - INFO: - Episode: 11/100, Reward: 15.000, Step: 15 2023-03-18 16:21:40 - r - INFO: - Episode: 12/100, Reward: 22.000, Step: 22 2023-03-18 16:21:40 - r - INFO: - Episode: 13/100, Reward: 9.000, Step: 9 2023-03-18 16:21:40 - r - INFO: - Episode: 14/100, Reward: 14.000, Step: 14 2023-03-18 16:21:40 - r - INFO: - Episode: 15/100, Reward: 12.000, Step: 12 2023-03-18 16:21:40 - r - INFO: - Current episode 15 has the best eval reward: 9.700 2023-03-18 16:21:40 - r - INFO: - Episode: 16/100, Reward: 16.000, Step: 16 2023-03-18 16:21:40 - r - INFO: - Episode: 17/100, Reward: 16.000, Step: 16 2023-03-18 16:21:40 - r - INFO: - Episode: 18/100, Reward: 14.000, Step: 14 2023-03-18 16:21:40 - r - INFO: - Episode: 19/100, Reward: 11.000, Step: 11 2023-03-18 16:21:40 - r - INFO: - Episode: 20/100, Reward: 13.000, Step: 13 2023-03-18 16:21:40 - r - INFO: - Current episode 20 has the best eval reward: 9.700 2023-03-18 16:21:40 - r - INFO: - Episode: 21/100, Reward: 13.000, Step: 13 2023-03-18 16:21:42 - r - INFO: - Episode: 22/100, Reward: 14.000, Step: 14 2023-03-18 16:21:42 - r - INFO: - Episode: 23/100, Reward: 14.000, Step: 14 2023-03-18 16:21:42 - r - INFO: - Episode: 24/100, Reward: 37.000, Step: 37 2023-03-18 16:21:42 - r - INFO: - Episode: 25/100, Reward: 12.000, Step: 12 2023-03-18 16:21:42 - r - INFO: - Episode: 26/100, Reward: 18.000, Step: 18 2023-03-18 16:21:42 - r - INFO: - Episode: 27/100, Reward: 13.000, Step: 13 2023-03-18 16:21:42 - r - INFO: - Episode: 28/100, Reward: 20.000, Step: 20 2023-03-18 16:21:43 - r - INFO: - Episode: 29/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 30/100, Reward: 10.000, Step: 10 2023-03-18 16:21:43 - r - INFO: - Current episode 30 has the best eval reward: 13.700 2023-03-18 16:21:43 - r - INFO: - Episode: 31/100, Reward: 10.000, Step: 10 2023-03-18 16:21:43 - r - INFO: - Episode: 32/100, Reward: 12.000, Step: 12 2023-03-18 16:21:43 - r - INFO: - Episode: 33/100, Reward: 11.000, Step: 11 2023-03-18 16:21:43 - r - INFO: - Episode: 34/100, Reward: 12.000, Step: 12 2023-03-18 16:21:43 - r - INFO: - Episode: 35/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Current episode 35 has the best eval reward: 32.500 2023-03-18 16:21:43 - r - INFO: - Episode: 36/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 37/100, Reward: 17.000, Step: 17 2023-03-18 16:21:43 - r - INFO: - Episode: 38/100, Reward: 23.000, Step: 23 2023-03-18 16:21:43 - r - INFO: - Episode: 39/100, Reward: 35.000, Step: 35 2023-03-18 16:21:43 - r - INFO: - Episode: 40/100, Reward: 46.000, Step: 46 2023-03-18 16:21:44 - r - INFO: - Episode: 41/100, Reward: 10.000, Step: 10 2023-03-18 16:21:44 - r - INFO: - Episode: 42/100, Reward: 13.000, Step: 13 2023-03-18 16:21:44 - r - INFO: - Episode: 43/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 44/100, Reward: 43.000, Step: 43 2023-03-18 16:21:44 - r - INFO: - Episode: 45/100, Reward: 23.000, Step: 23 2023-03-18 16:21:44 - r - INFO: - Episode: 46/100, Reward: 31.000, Step: 31 2023-03-18 16:21:44 - r - INFO: - Episode: 47/100, Reward: 36.000, Step: 36 2023-03-18 16:21:44 - r - INFO: - Episode: 48/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 49/100, Reward: 27.000, Step: 27 2023-03-18 16:21:44 - r - INFO: - Episode: 50/100, Reward: 40.000, Step: 40 2023-03-18 16:21:44 - r - INFO: - Current episode 50 has the best eval reward: 36.900 2023-03-18 16:21:45 - r - INFO: - Episode: 51/100, Reward: 47.000, Step: 47 2023-03-18 16:21:45 - r - INFO: - Episode: 52/100, Reward: 60.000, Step: 60 2023-03-18 16:21:45 - r - INFO: - Episode: 53/100, Reward: 104.000, Step: 104 2023-03-18 16:21:45 - r - INFO: - Episode: 54/100, Reward: 70.000, Step: 70 2023-03-18 16:21:45 - r - INFO: - Episode: 55/100, Reward: 65.000, Step: 65 2023-03-18 16:21:46 - r - INFO: - Episode: 56/100, Reward: 96.000, Step: 96 2023-03-18 16:21:46 - r - INFO: - Episode: 57/100, Reward: 34.000, Step: 34 2023-03-18 16:21:46 - r - INFO: - Episode: 58/100, Reward: 30.000, Step: 30 2023-03-18 16:21:46 - r - INFO: - Episode: 59/100, Reward: 63.000, Step: 63 2023-03-18 16:21:46 - r - INFO: - Episode: 60/100, Reward: 32.000, Step: 32 2023-03-18 16:21:46 - r - INFO: - Current episode 60 has the best eval reward: 104.900 2023-03-18 16:21:47 - r - INFO: - Episode: 61/100, Reward: 36.000, Step: 36 2023-03-18 16:21:47 - r - INFO: - Episode: 62/100, Reward: 26.000, Step: 26 2023-03-18 16:21:47 - r - INFO: - Episode: 63/100, Reward: 29.000, Step: 29 2023-03-18 16:21:47 - r - INFO: - Episode: 64/100, Reward: 58.000, Step: 58 2023-03-18 16:21:47 - r - INFO: - Episode: 65/100, Reward: 123.000, Step: 123 2023-03-18 16:21:47 - r - INFO: - Episode: 66/100, Reward: 74.000, Step: 74 2023-03-18 16:21:48 - r - INFO: - Episode: 67/100, Reward: 56.000, Step: 56 2023-03-18 16:21:48 - r - INFO: - Episode: 68/100, Reward: 76.000, Step: 76 2023-03-18 16:21:48 - r - INFO: - Episode: 69/100, Reward: 63.000, Step: 63 2023-03-18 16:21:48 - r - INFO: - Episode: 70/100, Reward: 55.000, Step: 55 2023-03-18 16:21:48 - r - INFO: - Episode: 71/100, Reward: 76.000, Step: 76 2023-03-18 16:21:49 - r - INFO: - Episode: 72/100, Reward: 59.000, Step: 59 2023-03-18 16:21:49 - r - INFO: - Episode: 73/100, Reward: 70.000, Step: 70 2023-03-18 16:21:49 - r - INFO: - Episode: 74/100, Reward: 98.000, Step: 98 2023-03-18 16:21:49 - r - INFO: - Episode: 75/100, Reward: 60.000, Step: 60 2023-03-18 16:21:50 - r - INFO: - Episode: 76/100, Reward: 114.000, Step: 114 2023-03-18 16:21:50 - r - INFO: - Episode: 77/100, Reward: 200.000, Step: 200 2023-03-18 16:21:51 - r - INFO: - Episode: 78/100, Reward: 199.000, Step: 199 2023-03-18 16:21:51 - r - INFO: - Episode: 79/100, Reward: 200.000, Step: 200 2023-03-18 16:21:52 - r - INFO: - Episode: 80/100, Reward: 200.000, Step: 200 2023-03-18 16:21:52 - r - INFO: - Current episode 80 has the best eval reward: 200.000 2023-03-18 16:21:53 - r - INFO: - Episode: 81/100, Reward: 200.000, Step: 200 2023-03-18 16:21:53 - r - INFO: - Episode: 82/100, Reward: 200.000, Step: 200 2023-03-18 16:21:54 - r - INFO: - Episode: 83/100, Reward: 200.000, Step: 200 2023-03-18 16:21:55 - r - INFO: - Episode: 84/100, Reward: 200.000, Step: 200 2023-03-18 16:21:55 - r - INFO: - Episode: 85/100, Reward: 200.000, Step: 200 2023-03-18 16:21:56 - r - INFO: - Current episode 85 has the best eval reward: 200.000 2023-03-18 16:21:56 - r - INFO: - Episode: 86/100, Reward: 200.000, Step: 200 2023-03-18 16:21:57 - r - INFO: - Episode: 87/100, Reward: 200.000, Step: 200 2023-03-18 16:21:57 - r - INFO: - Episode: 88/100, Reward: 200.000, Step: 200 2023-03-18 16:21:58 - r - INFO: - Episode: 89/100, Reward: 200.000, Step: 200 2023-03-18 16:21:58 - r - INFO: - Episode: 90/100, Reward: 200.000, Step: 200 2023-03-18 16:21:59 - r - INFO: - Current episode 90 has the best eval reward: 200.000 2023-03-18 16:21:59 - r - INFO: - Episode: 91/100, Reward: 200.000, Step: 200 2023-03-18 16:22:00 - r - INFO: - Episode: 92/100, Reward: 200.000, Step: 200 2023-03-18 16:22:01 - r - INFO: - Episode: 93/100, Reward: 200.000, Step: 200 2023-03-18 16:22:01 - r - INFO: - Episode: 94/100, Reward: 200.000, Step: 200 2023-03-18 16:22:02 - r - INFO: - Episode: 95/100, Reward: 200.000, Step: 200 2023-03-18 16:22:02 - r - INFO: - Current episode 95 has the best eval reward: 200.000 2023-03-18 16:22:03 - r - INFO: - Episode: 96/100, Reward: 200.000, Step: 200 2023-03-18 16:22:03 - r - INFO: - Episode: 97/100, Reward: 200.000, Step: 200 2023-03-18 16:22:04 - r - INFO: - Episode: 98/100, Reward: 200.000, Step: 200 2023-03-18 16:22:04 - r - INFO: - Episode: 99/100, Reward: 200.000, Step: 200 2023-03-18 16:22:05 - r - INFO: - Episode: 100/100, Reward: 200.000, Step: 200 2023-03-18 16:22:05 - r - INFO: - Current episode 100 has the best eval reward: 200.000 2023-03-18 16:22:05 - r - INFO: - Finish training! |