File size: 17,232 Bytes
54d82df |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
2023-05-18 13:39:47 - SimpleLog - INFO: - General Configs:
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
2023-05-18 13:39:47 - SimpleLog - INFO: - env_name gym <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - algo_name NoisyDQN <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - mode train <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - device cpu <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - seed 1 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - max_episode 100 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - max_step 200 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
2023-05-18 13:39:47 - SimpleLog - INFO: - mp_backend ray <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - n_workers 2 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - n_learners 1 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval 1 <class 'bool'>
2023-05-18 13:39:47 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
2023-05-18 13:39:47 - SimpleLog - INFO: - load_path Train_single_CartPole-v1_NoisyDQN_20230518-133737 <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - load_model_step best <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:47 - SimpleLog - INFO: - Algo Configs:
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
2023-05-18 13:39:47 - SimpleLog - INFO: - epsilon_decay 500 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
2023-05-18 13:39:47 - SimpleLog - INFO: - lr 0.0001 <class 'float'>
2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_size 100000 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - batch_size 64 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - target_update 4 <class 'int'>
2023-05-18 13:39:47 - SimpleLog - INFO: - value_layers [{'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}, {'layer_type': 'noisy_linear', 'layer_size': [256], 'activation': 'relu', 'std_init': 0.4}] <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - buffer_type REPLAY_QUE <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:47 - SimpleLog - INFO: - Env Configs:
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:47 - SimpleLog - INFO: - Name Value Type
2023-05-18 13:39:47 - SimpleLog - INFO: - id CartPole-v1 <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - render_mode None <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - wrapper None <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
2023-05-18 13:39:47 - SimpleLog - INFO: - ================================================================================
2023-05-18 13:39:54 - SimpleLog - INFO: - obs_space: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), n_actions: Discrete(2)
2023-05-18 13:39:59 - RayLog - INFO: - Worker 0 finished episode 0 with reward 14.0 in 14 steps
2023-05-18 13:39:59 - RayLog - INFO: - Worker 1 finished episode 0 with reward 22.0 in 22 steps
2023-05-18 13:40:01 - RayLog - INFO: - Worker 1 finished episode 2 with reward 33.0 in 33 steps
2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 2 with reward 38.0 in 38 steps
2023-05-18 13:40:01 - RayLog - INFO: - Worker 0 finished episode 4 with reward 15.0 in 15 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 5 with reward 26.0 in 26 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 3 with reward 53.0 in 53 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 6 with reward 11.0 in 11 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 7 with reward 12.0 in 12 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 8 with reward 14.0 in 14 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 9 with reward 13.0 in 13 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 0 finished episode 10 with reward 12.0 in 12 steps
2023-05-18 13:40:02 - RayLog - INFO: - Worker 1 finished episode 11 with reward 10.0 in 10 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 12 with reward 9.0 in 9 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 13 with reward 12.0 in 12 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 14 with reward 11.0 in 11 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 15 with reward 11.0 in 11 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 16 with reward 9.0 in 9 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 1 finished episode 17 with reward 20.0 in 20 steps
2023-05-18 13:40:03 - RayLog - INFO: - Worker 0 finished episode 18 with reward 20.0 in 20 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 20 with reward 12.0 in 12 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 21 with reward 13.0 in 13 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 19 with reward 31.0 in 31 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 22 with reward 10.0 in 10 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 23 with reward 9.0 in 9 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 24 with reward 12.0 in 12 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 1 finished episode 25 with reward 11.0 in 11 steps
2023-05-18 13:40:04 - RayLog - INFO: - Worker 0 finished episode 26 with reward 12.0 in 12 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 27 with reward 12.0 in 12 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 28 with reward 13.0 in 13 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 29 with reward 13.0 in 13 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 30 with reward 12.0 in 12 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 1 finished episode 31 with reward 14.0 in 14 steps
2023-05-18 13:40:05 - RayLog - INFO: - Worker 0 finished episode 32 with reward 14.0 in 14 steps
2023-05-18 13:40:07 - RayLog - INFO: - learner id: 0, update_step: 500, online_eval_reward: 10.000
2023-05-18 13:40:07 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 10.000, save the best model!
2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 33 with reward 14.0 in 14 steps
2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 34 with reward 11.0 in 11 steps
2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 35 with reward 13.0 in 13 steps
2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 37 with reward 10.0 in 10 steps
2023-05-18 13:40:07 - RayLog - INFO: - Worker 1 finished episode 38 with reward 11.0 in 11 steps
2023-05-18 13:40:07 - RayLog - INFO: - Worker 0 finished episode 36 with reward 29.0 in 29 steps
2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 39 with reward 9.0 in 9 steps
2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 40 with reward 11.0 in 11 steps
2023-05-18 13:40:08 - RayLog - INFO: - Worker 0 finished episode 42 with reward 35.0 in 35 steps
2023-05-18 13:40:08 - RayLog - INFO: - Worker 1 finished episode 41 with reward 49.0 in 49 steps
2023-05-18 13:40:09 - RayLog - INFO: - Worker 0 finished episode 43 with reward 36.0 in 36 steps
2023-05-18 13:40:09 - RayLog - INFO: - Worker 1 finished episode 44 with reward 35.0 in 35 steps
2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 45 with reward 42.0 in 42 steps
2023-05-18 13:40:10 - RayLog - INFO: - Worker 1 finished episode 46 with reward 44.0 in 44 steps
2023-05-18 13:40:10 - RayLog - INFO: - Worker 0 finished episode 47 with reward 36.0 in 36 steps
2023-05-18 13:40:11 - RayLog - INFO: - Worker 1 finished episode 48 with reward 44.0 in 44 steps
2023-05-18 13:40:11 - RayLog - INFO: - Worker 0 finished episode 49 with reward 37.0 in 37 steps
2023-05-18 13:40:12 - RayLog - INFO: - learner id: 0, update_step: 1000, online_eval_reward: 51.000
2023-05-18 13:40:12 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 51.000, save the best model!
2023-05-18 13:40:13 - RayLog - INFO: - Worker 0 finished episode 51 with reward 76.0 in 76 steps
2023-05-18 13:40:13 - RayLog - INFO: - Worker 1 finished episode 50 with reward 109.0 in 109 steps
2023-05-18 13:40:15 - RayLog - INFO: - Worker 0 finished episode 52 with reward 119.0 in 119 steps
2023-05-18 13:40:15 - RayLog - INFO: - Worker 1 finished episode 53 with reward 120.0 in 120 steps
2023-05-18 13:40:16 - RayLog - INFO: - Worker 0 finished episode 54 with reward 57.0 in 57 steps
2023-05-18 13:40:16 - RayLog - INFO: - Worker 1 finished episode 55 with reward 44.0 in 44 steps
2023-05-18 13:40:16 - RayLog - INFO: - learner id: 0, update_step: 1500, online_eval_reward: 53.000
2023-05-18 13:40:16 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 53.000, save the best model!
2023-05-18 13:40:17 - RayLog - INFO: - Worker 1 finished episode 57 with reward 43.0 in 43 steps
2023-05-18 13:40:18 - RayLog - INFO: - Worker 0 finished episode 56 with reward 72.0 in 72 steps
2023-05-18 13:40:18 - RayLog - INFO: - Worker 1 finished episode 58 with reward 35.0 in 35 steps
2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 60 with reward 48.0 in 48 steps
2023-05-18 13:40:19 - RayLog - INFO: - Worker 0 finished episode 59 with reward 64.0 in 64 steps
2023-05-18 13:40:19 - RayLog - INFO: - Worker 1 finished episode 61 with reward 36.0 in 36 steps
2023-05-18 13:40:20 - RayLog - INFO: - Worker 0 finished episode 62 with reward 57.0 in 57 steps
2023-05-18 13:40:20 - RayLog - INFO: - Worker 1 finished episode 63 with reward 42.0 in 42 steps
2023-05-18 13:40:21 - RayLog - INFO: - learner id: 0, update_step: 2000, online_eval_reward: 143.000
2023-05-18 13:40:21 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 143.000, save the best model!
2023-05-18 13:40:23 - RayLog - INFO: - Worker 1 finished episode 65 with reward 100.0 in 100 steps
2023-05-18 13:40:23 - RayLog - INFO: - Worker 0 finished episode 64 with reward 153.0 in 153 steps
2023-05-18 13:40:24 - RayLog - INFO: - Worker 1 finished episode 66 with reward 61.0 in 61 steps
2023-05-18 13:40:25 - RayLog - INFO: - Worker 0 finished episode 67 with reward 111.0 in 111 steps
2023-05-18 13:40:26 - RayLog - INFO: - Worker 1 finished episode 68 with reward 103.0 in 103 steps
2023-05-18 13:40:27 - RayLog - INFO: - learner id: 0, update_step: 2500, online_eval_reward: 85.000
2023-05-18 13:40:28 - RayLog - INFO: - Worker 1 finished episode 70 with reward 127.0 in 127 steps
2023-05-18 13:40:29 - RayLog - INFO: - Worker 0 finished episode 69 with reward 164.0 in 164 steps
2023-05-18 13:40:32 - RayLog - INFO: - learner id: 0, update_step: 3000, online_eval_reward: 74.000
2023-05-18 13:40:32 - RayLog - INFO: - Worker 1 finished episode 71 with reward 200.0 in 200 steps
2023-05-18 13:40:32 - RayLog - INFO: - Worker 0 finished episode 72 with reward 186.0 in 186 steps
2023-05-18 13:40:35 - RayLog - INFO: - Worker 1 finished episode 73 with reward 114.0 in 114 steps
2023-05-18 13:40:35 - RayLog - INFO: - Worker 0 finished episode 74 with reward 109.0 in 109 steps
2023-05-18 13:40:36 - RayLog - INFO: - Worker 1 finished episode 75 with reward 72.0 in 72 steps
2023-05-18 13:40:37 - RayLog - INFO: - learner id: 0, update_step: 3500, online_eval_reward: 83.000
2023-05-18 13:40:37 - RayLog - INFO: - Worker 0 finished episode 76 with reward 96.0 in 96 steps
2023-05-18 13:40:37 - RayLog - INFO: - Worker 1 finished episode 77 with reward 62.0 in 62 steps
2023-05-18 13:40:38 - RayLog - INFO: - Worker 0 finished episode 78 with reward 88.0 in 88 steps
2023-05-18 13:40:39 - RayLog - INFO: - Worker 1 finished episode 79 with reward 70.0 in 70 steps
2023-05-18 13:40:40 - RayLog - INFO: - Worker 1 finished episode 81 with reward 77.0 in 77 steps
2023-05-18 13:40:42 - RayLog - INFO: - learner id: 0, update_step: 4000, online_eval_reward: 123.000
2023-05-18 13:40:42 - RayLog - INFO: - Worker 0 finished episode 80 with reward 181.0 in 181 steps
2023-05-18 13:40:42 - RayLog - INFO: - Worker 1 finished episode 82 with reward 105.0 in 105 steps
2023-05-18 13:40:46 - RayLog - INFO: - Worker 0 finished episode 83 with reward 200.0 in 200 steps
2023-05-18 13:40:47 - RayLog - INFO: - Worker 1 finished episode 84 with reward 200.0 in 200 steps
2023-05-18 13:40:47 - RayLog - INFO: - learner id: 0, update_step: 4500, online_eval_reward: 200.000
2023-05-18 13:40:47 - RayLog - INFO: - learner 0 for current update step obtain a better online_eval_reward: 200.000, save the best model!
2023-05-18 13:40:51 - RayLog - INFO: - Worker 0 finished episode 85 with reward 200.0 in 200 steps
2023-05-18 13:40:51 - RayLog - INFO: - Worker 1 finished episode 86 with reward 200.0 in 200 steps
2023-05-18 13:40:53 - RayLog - INFO: - learner id: 0, update_step: 5000, online_eval_reward: 200.000
2023-05-18 13:40:56 - RayLog - INFO: - Worker 0 finished episode 87 with reward 200.0 in 200 steps
2023-05-18 13:40:56 - RayLog - INFO: - Worker 1 finished episode 88 with reward 200.0 in 200 steps
2023-05-18 13:40:59 - RayLog - INFO: - learner id: 0, update_step: 5500, online_eval_reward: 200.000
2023-05-18 13:41:00 - RayLog - INFO: - Worker 0 finished episode 89 with reward 200.0 in 200 steps
2023-05-18 13:41:01 - RayLog - INFO: - Worker 1 finished episode 90 with reward 200.0 in 200 steps
2023-05-18 13:41:05 - RayLog - INFO: - learner id: 0, update_step: 6000, online_eval_reward: 200.000
2023-05-18 13:41:05 - RayLog - INFO: - Worker 0 finished episode 91 with reward 200.0 in 200 steps
2023-05-18 13:41:05 - RayLog - INFO: - Worker 1 finished episode 92 with reward 200.0 in 200 steps
2023-05-18 13:41:10 - RayLog - INFO: - Worker 0 finished episode 93 with reward 200.0 in 200 steps
2023-05-18 13:41:10 - RayLog - INFO: - Worker 1 finished episode 94 with reward 200.0 in 200 steps
2023-05-18 13:41:11 - RayLog - INFO: - learner id: 0, update_step: 6500, online_eval_reward: 200.000
2023-05-18 13:41:15 - RayLog - INFO: - Worker 0 finished episode 95 with reward 200.0 in 200 steps
2023-05-18 13:41:15 - RayLog - INFO: - Worker 1 finished episode 96 with reward 200.0 in 200 steps
2023-05-18 13:41:17 - RayLog - INFO: - learner id: 0, update_step: 7000, online_eval_reward: 200.000
2023-05-18 13:41:19 - RayLog - INFO: - Worker 0 finished episode 97 with reward 200.0 in 200 steps
2023-05-18 13:41:20 - RayLog - INFO: - Worker 1 finished episode 98 with reward 200.0 in 200 steps
2023-05-18 13:41:23 - RayLog - INFO: - learner id: 0, update_step: 7500, online_eval_reward: 200.000
2023-05-18 13:41:24 - RayLog - INFO: - Worker 0 finished episode 99 with reward 200.0 in 200 steps
2023-05-18 13:41:25 - RayLog - INFO: - Worker 1 finished episode 100 with reward 200.0 in 200 steps
2023-05-18 13:41:28 - SimpleLog - INFO: - Finish training! total time consumed: 100.97s
|