| 2023-01-14 22:20:12 - r - INFO: - Hyperparameters: | |
| 2023-01-14 22:20:12 - r - INFO: - ================================================================================ | |
| 2023-01-14 22:20:12 - r - INFO: - Name Value Type | |
| 2023-01-14 22:20:12 - r - INFO: - env_name CartPole-v1 <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - new_step_api 1 <class 'bool'> | |
| 2023-01-14 22:20:12 - r - INFO: - wrapper None <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - render 0 <class 'bool'> | |
| 2023-01-14 22:20:12 - r - INFO: - algo_name RainbowDQN <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - mode train <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - seed 1 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - device cuda <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - train_eps 100 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - test_eps 10 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - eval_eps 10 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - eval_per_episode 5 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - max_steps 200 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-01-14 22:20:12 - r - INFO: - load_path Train_CartPole-v1_ <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - show_fig 0 <class 'bool'> | |
| 2023-01-14 22:20:12 - r - INFO: - save_fig 1 <class 'bool'> | |
| 2023-01-14 22:20:12 - r - INFO: - Vmin 0.0 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - Vmax 200.0 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - n_atoms 51 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - gamma 0.95 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - tau 1.0 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - hidden_dim 256 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - num_atoms 51 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - support tensor(0.) <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - delta_z 4.0 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - n_step 1 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - batch_size 64 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - lr 0.0001 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - target_update 4 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - memory_capacity 10000 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - epsilon_start 0.95 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - epsilon_end 0.01 <class 'float'> | |
| 2023-01-14 22:20:12 - r - INFO: - epsilon_decay 500 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - value_layers [{'layer_type': 'linear', 'layer_dim': ['n_states', 256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256, 256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_dim': [256, 'n_actions'], 'activation': 'none'}] <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - buffer_size 100000 <class 'int'> | |
| 2023-01-14 22:20:12 - r - INFO: - task_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012 <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - model_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/models <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - res_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/results <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - log_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/logs <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - traj_dir /home/dingli/rl-tutorials/joyrl/tasks/Train_CartPole-v1_RainbowDQN_20230114-222012/traj <class 'str'> | |
| 2023-01-14 22:20:12 - r - INFO: - ================================================================================ | |
| 2023-01-14 22:20:12 - r - INFO: - n_states: 4, n_actions: 2 | |
| 2023-01-14 22:20:14 - r - INFO: - Start training! | |
| 2023-01-14 22:20:14 - r - INFO: - Env: CartPole-v1, Algorithm: RainbowDQN, Device: cuda | |
| 2023-01-14 22:20:14 - r - INFO: - Episode: 1/100, Reward: 16.000, Step: 16 | |
| 2023-01-14 22:20:15 - r - INFO: - Episode: 2/100, Reward: 50.000, Step: 50 | |
| 2023-01-14 22:20:15 - r - INFO: - Episode: 3/100, Reward: 23.000, Step: 23 | |
| 2023-01-14 22:20:15 - r - INFO: - Episode: 4/100, Reward: 11.000, Step: 11 | |
| 2023-01-14 22:20:15 - r - INFO: - Episode: 5/100, Reward: 10.000, Step: 10 | |
| 2023-01-14 22:20:15 - r - INFO: - Current episode 5 has the best eval reward: 9.200 | |
| 2023-01-14 22:20:16 - r - INFO: - Episode: 6/100, Reward: 79.000, Step: 79 | |
| 2023-01-14 22:20:16 - r - INFO: - Episode: 7/100, Reward: 11.000, Step: 11 | |
| 2023-01-14 22:20:17 - r - INFO: - Episode: 8/100, Reward: 53.000, Step: 53 | |
| 2023-01-14 22:20:17 - r - INFO: - Episode: 9/100, Reward: 37.000, Step: 37 | |
| 2023-01-14 22:20:18 - r - INFO: - Episode: 10/100, Reward: 38.000, Step: 38 | |
| 2023-01-14 22:20:18 - r - INFO: - Current episode 10 has the best eval reward: 46.400 | |
| 2023-01-14 22:20:19 - r - INFO: - Episode: 11/100, Reward: 51.000, Step: 51 | |
| 2023-01-14 22:20:19 - r - INFO: - Episode: 12/100, Reward: 42.000, Step: 42 | |
| 2023-01-14 22:20:20 - r - INFO: - Episode: 13/100, Reward: 43.000, Step: 43 | |
| 2023-01-14 22:20:20 - r - INFO: - Episode: 14/100, Reward: 34.000, Step: 34 | |
| 2023-01-14 22:20:20 - r - INFO: - Episode: 15/100, Reward: 21.000, Step: 21 | |
| 2023-01-14 22:20:21 - r - INFO: - Current episode 15 has the best eval reward: 54.400 | |
| 2023-01-14 22:20:21 - r - INFO: - Episode: 16/100, Reward: 44.000, Step: 44 | |
| 2023-01-14 22:20:22 - r - INFO: - Episode: 17/100, Reward: 42.000, Step: 42 | |
| 2023-01-14 22:20:23 - r - INFO: - Episode: 18/100, Reward: 109.000, Step: 109 | |
| 2023-01-14 22:20:24 - r - INFO: - Episode: 19/100, Reward: 33.000, Step: 33 | |
| 2023-01-14 22:20:24 - r - INFO: - Episode: 20/100, Reward: 43.000, Step: 43 | |
| 2023-01-14 22:20:25 - r - INFO: - Episode: 21/100, Reward: 44.000, Step: 44 | |
| 2023-01-14 22:20:26 - r - INFO: - Episode: 22/100, Reward: 84.000, Step: 84 | |
| 2023-01-14 22:20:27 - r - INFO: - Episode: 23/100, Reward: 48.000, Step: 48 | |
| 2023-01-14 22:20:27 - r - INFO: - Episode: 24/100, Reward: 63.000, Step: 63 | |
| 2023-01-14 22:20:30 - r - INFO: - Episode: 25/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:31 - r - INFO: - Current episode 25 has the best eval reward: 122.800 | |
| 2023-01-14 22:20:33 - r - INFO: - Episode: 26/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:36 - r - INFO: - Episode: 27/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:38 - r - INFO: - Episode: 28/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:41 - r - INFO: - Episode: 29/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:43 - r - INFO: - Episode: 30/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:44 - r - INFO: - Current episode 30 has the best eval reward: 200.000 | |
| 2023-01-14 22:20:47 - r - INFO: - Episode: 31/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:49 - r - INFO: - Episode: 32/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:52 - r - INFO: - Episode: 33/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:54 - r - INFO: - Episode: 34/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:57 - r - INFO: - Episode: 35/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:20:58 - r - INFO: - Current episode 35 has the best eval reward: 200.000 | |
| 2023-01-14 22:21:01 - r - INFO: - Episode: 36/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:03 - r - INFO: - Episode: 37/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:06 - r - INFO: - Episode: 38/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:08 - r - INFO: - Episode: 39/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:11 - r - INFO: - Episode: 40/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:12 - r - INFO: - Current episode 40 has the best eval reward: 200.000 | |
| 2023-01-14 22:21:15 - r - INFO: - Episode: 41/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:17 - r - INFO: - Episode: 42/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:20 - r - INFO: - Episode: 43/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:22 - r - INFO: - Episode: 44/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:24 - r - INFO: - Episode: 45/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:26 - r - INFO: - Current episode 45 has the best eval reward: 200.000 | |
| 2023-01-14 22:21:28 - r - INFO: - Episode: 46/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:31 - r - INFO: - Episode: 47/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:33 - r - INFO: - Episode: 48/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:36 - r - INFO: - Episode: 49/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:38 - r - INFO: - Episode: 50/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:40 - r - INFO: - Current episode 50 has the best eval reward: 200.000 | |
| 2023-01-14 22:21:42 - r - INFO: - Episode: 51/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:45 - r - INFO: - Episode: 52/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:47 - r - INFO: - Episode: 53/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:49 - r - INFO: - Episode: 54/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:52 - r - INFO: - Episode: 55/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:56 - r - INFO: - Episode: 56/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:21:58 - r - INFO: - Episode: 57/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:01 - r - INFO: - Episode: 58/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:03 - r - INFO: - Episode: 59/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:05 - r - INFO: - Episode: 60/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:07 - r - INFO: - Current episode 60 has the best eval reward: 200.000 | |
| 2023-01-14 22:22:09 - r - INFO: - Episode: 61/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:12 - r - INFO: - Episode: 62/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:14 - r - INFO: - Episode: 63/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:17 - r - INFO: - Episode: 64/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:19 - r - INFO: - Episode: 65/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:21 - r - INFO: - Current episode 65 has the best eval reward: 200.000 | |
| 2023-01-14 22:22:24 - r - INFO: - Episode: 66/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:26 - r - INFO: - Episode: 67/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:29 - r - INFO: - Episode: 68/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:31 - r - INFO: - Episode: 69/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:34 - r - INFO: - Episode: 70/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:38 - r - INFO: - Episode: 71/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:40 - r - INFO: - Episode: 72/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:42 - r - INFO: - Episode: 73/100, Reward: 174.000, Step: 174 | |
| 2023-01-14 22:22:45 - r - INFO: - Episode: 74/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:47 - r - INFO: - Episode: 75/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:51 - r - INFO: - Episode: 76/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:53 - r - INFO: - Episode: 77/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:56 - r - INFO: - Episode: 78/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:22:58 - r - INFO: - Episode: 79/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:01 - r - INFO: - Episode: 80/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:04 - r - INFO: - Episode: 81/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:07 - r - INFO: - Episode: 82/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:10 - r - INFO: - Episode: 83/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:12 - r - INFO: - Episode: 84/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:15 - r - INFO: - Episode: 85/100, Reward: 193.000, Step: 193 | |
| 2023-01-14 22:23:16 - r - INFO: - Current episode 85 has the best eval reward: 200.000 | |
| 2023-01-14 22:23:19 - r - INFO: - Episode: 86/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:21 - r - INFO: - Episode: 87/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:24 - r - INFO: - Episode: 88/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:26 - r - INFO: - Episode: 89/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:28 - r - INFO: - Episode: 90/100, Reward: 177.000, Step: 177 | |
| 2023-01-14 22:23:30 - r - INFO: - Current episode 90 has the best eval reward: 200.000 | |
| 2023-01-14 22:23:32 - r - INFO: - Episode: 91/100, Reward: 191.000, Step: 191 | |
| 2023-01-14 22:23:35 - r - INFO: - Episode: 92/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:37 - r - INFO: - Episode: 93/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:40 - r - INFO: - Episode: 94/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:42 - r - INFO: - Episode: 95/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:44 - r - INFO: - Current episode 95 has the best eval reward: 200.000 | |
| 2023-01-14 22:23:46 - r - INFO: - Episode: 96/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:49 - r - INFO: - Episode: 97/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:51 - r - INFO: - Episode: 98/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:54 - r - INFO: - Episode: 99/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:56 - r - INFO: - Episode: 100/100, Reward: 200.000, Step: 200 | |
| 2023-01-14 22:23:58 - r - INFO: - Current episode 100 has the best eval reward: 200.000 | |
| 2023-01-14 22:23:58 - r - INFO: - Finish training! | |