File size: 14,200 Bytes
62e03a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 |
2023-01-14 22:20:12 - r - INFO: - Hyperparameters:
2023-01-14 22:20:12 - r - INFO: - ================================================================================
2023-01-14 22:20:12 - r - INFO: - Name Value Type
2023-01-14 22:20:12 - r - INFO: - env_name CartPole-v1 <class 'str'>
2023-01-14 22:20:12 - r - INFO: - new_step_api 1 <class 'bool'>
2023-01-14 22:20:12 - r - INFO: - wrapper None <class 'str'>
2023-01-14 22:20:12 - r - INFO: - render 0 <class 'bool'>
2023-01-14 22:20:12 - r - INFO: - algo_name RainbowDQN <class 'str'>
2023-01-14 22:20:12 - r - INFO: - mode train <class 'str'>
2023-01-14 22:20:12 - r - INFO: - seed 1 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - device cuda <class 'str'>
2023-01-14 22:20:12 - r - INFO: - train_eps 100 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - test_eps 10 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - eval_eps 10 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - eval_per_episode 5 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - max_steps 200 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - load_checkpoint 0 <class 'bool'>
2023-01-14 22:20:12 - r - INFO: - load_path Train_CartPole-v1_ <class 'str'>
2023-01-14 22:20:12 - r - INFO: - show_fig 0 <class 'bool'>
2023-01-14 22:20:12 - r - INFO: - save_fig 1 <class 'bool'>
2023-01-14 22:20:12 - r - INFO: - Vmin 0.0 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - Vmax 200.0 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - n_atoms 51 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - gamma 0.95 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - tau 1.0 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - hidden_dim 256 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - num_atoms 51 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - support tensor(0.) <class 'str'>
2023-01-14 22:20:12 - r - INFO: - delta_z 4.0 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - n_step 1 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - batch_size 64 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - lr 0.0001 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - target_update 4 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - memory_capacity 10000 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - epsilon_start 0.95 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - epsilon_end 0.01 <class 'float'>
2023-01-14 22:20:12 - r - INFO: - epsilon_decay 500 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - value_layers [{'layer_type': 'linear', 'layer_dim': ['n_states', 256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256, 256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256, 'n_actions'], 'activation': 'none'}] <class 'str'>
2023-01-14 22:20:12 - r - INFO: - buffer_size 100000 <class 'int'>
2023-01-14 22:20:12 - r - INFO: - task_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012 <class 'str'>
2023-01-14 22:20:12 - r - INFO: - model_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/models <class 'str'>
2023-01-14 22:20:12 - r - INFO: - res_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/results <class 'str'>
2023-01-14 22:20:12 - r - INFO: - log_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/logs <class 'str'>
2023-01-14 22:20:12 - r - INFO: - traj_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/traj <class 'str'>
2023-01-14 22:20:12 - r - INFO: - ================================================================================
2023-01-14 22:20:12 - r - INFO: - n_states: 4, n_actions: 2
2023-01-14 22:20:14 - r - INFO: - Start training!
2023-01-14 22:20:14 - r - INFO: - Env: CartPole-v1, Algorithm: RainbowDQN, Device: cuda
2023-01-14 22:20:14 - r - INFO: - Episode: 1/100, Reward: 16.000, Step: 16
2023-01-14 22:20:15 - r - INFO: - Episode: 2/100, Reward: 50.000, Step: 50
2023-01-14 22:20:15 - r - INFO: - Episode: 3/100, Reward: 23.000, Step: 23
2023-01-14 22:20:15 - r - INFO: - Episode: 4/100, Reward: 11.000, Step: 11
2023-01-14 22:20:15 - r - INFO: - Episode: 5/100, Reward: 10.000, Step: 10
2023-01-14 22:20:15 - r - INFO: - Current episode 5 has the best eval reward: 9.200
2023-01-14 22:20:16 - r - INFO: - Episode: 6/100, Reward: 79.000, Step: 79
2023-01-14 22:20:16 - r - INFO: - Episode: 7/100, Reward: 11.000, Step: 11
2023-01-14 22:20:17 - r - INFO: - Episode: 8/100, Reward: 53.000, Step: 53
2023-01-14 22:20:17 - r - INFO: - Episode: 9/100, Reward: 37.000, Step: 37
2023-01-14 22:20:18 - r - INFO: - Episode: 10/100, Reward: 38.000, Step: 38
2023-01-14 22:20:18 - r - INFO: - Current episode 10 has the best eval reward: 46.400
2023-01-14 22:20:19 - r - INFO: - Episode: 11/100, Reward: 51.000, Step: 51
2023-01-14 22:20:19 - r - INFO: - Episode: 12/100, Reward: 42.000, Step: 42
2023-01-14 22:20:20 - r - INFO: - Episode: 13/100, Reward: 43.000, Step: 43
2023-01-14 22:20:20 - r - INFO: - Episode: 14/100, Reward: 34.000, Step: 34
2023-01-14 22:20:20 - r - INFO: - Episode: 15/100, Reward: 21.000, Step: 21
2023-01-14 22:20:21 - r - INFO: - Current episode 15 has the best eval reward: 54.400
2023-01-14 22:20:21 - r - INFO: - Episode: 16/100, Reward: 44.000, Step: 44
2023-01-14 22:20:22 - r - INFO: - Episode: 17/100, Reward: 42.000, Step: 42
2023-01-14 22:20:23 - r - INFO: - Episode: 18/100, Reward: 109.000, Step: 109
2023-01-14 22:20:24 - r - INFO: - Episode: 19/100, Reward: 33.000, Step: 33
2023-01-14 22:20:24 - r - INFO: - Episode: 20/100, Reward: 43.000, Step: 43
2023-01-14 22:20:25 - r - INFO: - Episode: 21/100, Reward: 44.000, Step: 44
2023-01-14 22:20:26 - r - INFO: - Episode: 22/100, Reward: 84.000, Step: 84
2023-01-14 22:20:27 - r - INFO: - Episode: 23/100, Reward: 48.000, Step: 48
2023-01-14 22:20:27 - r - INFO: - Episode: 24/100, Reward: 63.000, Step: 63
2023-01-14 22:20:30 - r - INFO: - Episode: 25/100, Reward: 200.000, Step: 200
2023-01-14 22:20:31 - r - INFO: - Current episode 25 has the best eval reward: 122.800
2023-01-14 22:20:33 - r - INFO: - Episode: 26/100, Reward: 200.000, Step: 200
2023-01-14 22:20:36 - r - INFO: - Episode: 27/100, Reward: 200.000, Step: 200
2023-01-14 22:20:38 - r - INFO: - Episode: 28/100, Reward: 200.000, Step: 200
2023-01-14 22:20:41 - r - INFO: - Episode: 29/100, Reward: 200.000, Step: 200
2023-01-14 22:20:43 - r - INFO: - Episode: 30/100, Reward: 200.000, Step: 200
2023-01-14 22:20:44 - r - INFO: - Current episode 30 has the best eval reward: 200.000
2023-01-14 22:20:47 - r - INFO: - Episode: 31/100, Reward: 200.000, Step: 200
2023-01-14 22:20:49 - r - INFO: - Episode: 32/100, Reward: 200.000, Step: 200
2023-01-14 22:20:52 - r - INFO: - Episode: 33/100, Reward: 200.000, Step: 200
2023-01-14 22:20:54 - r - INFO: - Episode: 34/100, Reward: 200.000, Step: 200
2023-01-14 22:20:57 - r - INFO: - Episode: 35/100, Reward: 200.000, Step: 200
2023-01-14 22:20:58 - r - INFO: - Current episode 35 has the best eval reward: 200.000
2023-01-14 22:21:01 - r - INFO: - Episode: 36/100, Reward: 200.000, Step: 200
2023-01-14 22:21:03 - r - INFO: - Episode: 37/100, Reward: 200.000, Step: 200
2023-01-14 22:21:06 - r - INFO: - Episode: 38/100, Reward: 200.000, Step: 200
2023-01-14 22:21:08 - r - INFO: - Episode: 39/100, Reward: 200.000, Step: 200
2023-01-14 22:21:11 - r - INFO: - Episode: 40/100, Reward: 200.000, Step: 200
2023-01-14 22:21:12 - r - INFO: - Current episode 40 has the best eval reward: 200.000
2023-01-14 22:21:15 - r - INFO: - Episode: 41/100, Reward: 200.000, Step: 200
2023-01-14 22:21:17 - r - INFO: - Episode: 42/100, Reward: 200.000, Step: 200
2023-01-14 22:21:20 - r - INFO: - Episode: 43/100, Reward: 200.000, Step: 200
2023-01-14 22:21:22 - r - INFO: - Episode: 44/100, Reward: 200.000, Step: 200
2023-01-14 22:21:24 - r - INFO: - Episode: 45/100, Reward: 200.000, Step: 200
2023-01-14 22:21:26 - r - INFO: - Current episode 45 has the best eval reward: 200.000
2023-01-14 22:21:28 - r - INFO: - Episode: 46/100, Reward: 200.000, Step: 200
2023-01-14 22:21:31 - r - INFO: - Episode: 47/100, Reward: 200.000, Step: 200
2023-01-14 22:21:33 - r - INFO: - Episode: 48/100, Reward: 200.000, Step: 200
2023-01-14 22:21:36 - r - INFO: - Episode: 49/100, Reward: 200.000, Step: 200
2023-01-14 22:21:38 - r - INFO: - Episode: 50/100, Reward: 200.000, Step: 200
2023-01-14 22:21:40 - r - INFO: - Current episode 50 has the best eval reward: 200.000
2023-01-14 22:21:42 - r - INFO: - Episode: 51/100, Reward: 200.000, Step: 200
2023-01-14 22:21:45 - r - INFO: - Episode: 52/100, Reward: 200.000, Step: 200
2023-01-14 22:21:47 - r - INFO: - Episode: 53/100, Reward: 200.000, Step: 200
2023-01-14 22:21:49 - r - INFO: - Episode: 54/100, Reward: 200.000, Step: 200
2023-01-14 22:21:52 - r - INFO: - Episode: 55/100, Reward: 200.000, Step: 200
2023-01-14 22:21:56 - r - INFO: - Episode: 56/100, Reward: 200.000, Step: 200
2023-01-14 22:21:58 - r - INFO: - Episode: 57/100, Reward: 200.000, Step: 200
2023-01-14 22:22:01 - r - INFO: - Episode: 58/100, Reward: 200.000, Step: 200
2023-01-14 22:22:03 - r - INFO: - Episode: 59/100, Reward: 200.000, Step: 200
2023-01-14 22:22:05 - r - INFO: - Episode: 60/100, Reward: 200.000, Step: 200
2023-01-14 22:22:07 - r - INFO: - Current episode 60 has the best eval reward: 200.000
2023-01-14 22:22:09 - r - INFO: - Episode: 61/100, Reward: 200.000, Step: 200
2023-01-14 22:22:12 - r - INFO: - Episode: 62/100, Reward: 200.000, Step: 200
2023-01-14 22:22:14 - r - INFO: - Episode: 63/100, Reward: 200.000, Step: 200
2023-01-14 22:22:17 - r - INFO: - Episode: 64/100, Reward: 200.000, Step: 200
2023-01-14 22:22:19 - r - INFO: - Episode: 65/100, Reward: 200.000, Step: 200
2023-01-14 22:22:21 - r - INFO: - Current episode 65 has the best eval reward: 200.000
2023-01-14 22:22:24 - r - INFO: - Episode: 66/100, Reward: 200.000, Step: 200
2023-01-14 22:22:26 - r - INFO: - Episode: 67/100, Reward: 200.000, Step: 200
2023-01-14 22:22:29 - r - INFO: - Episode: 68/100, Reward: 200.000, Step: 200
2023-01-14 22:22:31 - r - INFO: - Episode: 69/100, Reward: 200.000, Step: 200
2023-01-14 22:22:34 - r - INFO: - Episode: 70/100, Reward: 200.000, Step: 200
2023-01-14 22:22:38 - r - INFO: - Episode: 71/100, Reward: 200.000, Step: 200
2023-01-14 22:22:40 - r - INFO: - Episode: 72/100, Reward: 200.000, Step: 200
2023-01-14 22:22:42 - r - INFO: - Episode: 73/100, Reward: 174.000, Step: 174
2023-01-14 22:22:45 - r - INFO: - Episode: 74/100, Reward: 200.000, Step: 200
2023-01-14 22:22:47 - r - INFO: - Episode: 75/100, Reward: 200.000, Step: 200
2023-01-14 22:22:51 - r - INFO: - Episode: 76/100, Reward: 200.000, Step: 200
2023-01-14 22:22:53 - r - INFO: - Episode: 77/100, Reward: 200.000, Step: 200
2023-01-14 22:22:56 - r - INFO: - Episode: 78/100, Reward: 200.000, Step: 200
2023-01-14 22:22:58 - r - INFO: - Episode: 79/100, Reward: 200.000, Step: 200
2023-01-14 22:23:01 - r - INFO: - Episode: 80/100, Reward: 200.000, Step: 200
2023-01-14 22:23:04 - r - INFO: - Episode: 81/100, Reward: 200.000, Step: 200
2023-01-14 22:23:07 - r - INFO: - Episode: 82/100, Reward: 200.000, Step: 200
2023-01-14 22:23:10 - r - INFO: - Episode: 83/100, Reward: 200.000, Step: 200
2023-01-14 22:23:12 - r - INFO: - Episode: 84/100, Reward: 200.000, Step: 200
2023-01-14 22:23:15 - r - INFO: - Episode: 85/100, Reward: 193.000, Step: 193
2023-01-14 22:23:16 - r - INFO: - Current episode 85 has the best eval reward: 200.000
2023-01-14 22:23:19 - r - INFO: - Episode: 86/100, Reward: 200.000, Step: 200
2023-01-14 22:23:21 - r - INFO: - Episode: 87/100, Reward: 200.000, Step: 200
2023-01-14 22:23:24 - r - INFO: - Episode: 88/100, Reward: 200.000, Step: 200
2023-01-14 22:23:26 - r - INFO: - Episode: 89/100, Reward: 200.000, Step: 200
2023-01-14 22:23:28 - r - INFO: - Episode: 90/100, Reward: 177.000, Step: 177
2023-01-14 22:23:30 - r - INFO: - Current episode 90 has the best eval reward: 200.000
2023-01-14 22:23:32 - r - INFO: - Episode: 91/100, Reward: 191.000, Step: 191
2023-01-14 22:23:35 - r - INFO: - Episode: 92/100, Reward: 200.000, Step: 200
2023-01-14 22:23:37 - r - INFO: - Episode: 93/100, Reward: 200.000, Step: 200
2023-01-14 22:23:40 - r - INFO: - Episode: 94/100, Reward: 200.000, Step: 200
2023-01-14 22:23:42 - r - INFO: - Episode: 95/100, Reward: 200.000, Step: 200
2023-01-14 22:23:44 - r - INFO: - Current episode 95 has the best eval reward: 200.000
2023-01-14 22:23:46 - r - INFO: - Episode: 96/100, Reward: 200.000, Step: 200
2023-01-14 22:23:49 - r - INFO: - Episode: 97/100, Reward: 200.000, Step: 200
2023-01-14 22:23:51 - r - INFO: - Episode: 98/100, Reward: 200.000, Step: 200
2023-01-14 22:23:54 - r - INFO: - Episode: 99/100, Reward: 200.000, Step: 200
2023-01-14 22:23:56 - r - INFO: - Episode: 100/100, Reward: 200.000, Step: 200
2023-01-14 22:23:58 - r - INFO: - Current episode 100 has the best eval reward: 200.000
2023-01-14 22:23:58 - r - INFO: - Finish training!
|