| 2023-04-14 15:02:20 - r - INFO: - Hyperparameters: | |
| 2023-04-14 15:02:20 - r - INFO: - ================================================================================ | |
| 2023-04-14 15:02:20 - r - INFO: - Name Value Type | |
| 2023-04-14 15:02:20 - r - INFO: - env_name Pendulum-v1 <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - new_step_api 1 <class 'bool'> | |
| 2023-04-14 15:02:20 - r - INFO: - wrapper None <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - render 0 <class 'bool'> | |
| 2023-04-14 15:02:20 - r - INFO: - render_mode human <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - algo_name DDPG_HER <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - mode train <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - mp_backend mp <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - seed 0 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - device cpu <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - her_sample_num 4 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - update_every 100 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - train_eps 150 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - test_eps 20 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - eval_eps 10 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - eval_per_episode 5 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - max_steps 200 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - load_checkpoint 0 <class 'bool'> | |
| 2023-04-14 15:02:20 - r - INFO: - load_path tasks <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - show_fig 0 <class 'bool'> | |
| 2023-04-14 15:02:20 - r - INFO: - save_fig 1 <class 'bool'> | |
| 2023-04-14 15:02:20 - r - INFO: - gamma 0.99 <class 'float'> | |
| 2023-04-14 15:02:20 - r - INFO: - critic_lr 0.001 <class 'float'> | |
| 2023-04-14 15:02:20 - r - INFO: - actor_lr 0.0001 <class 'float'> | |
| 2023-04-14 15:02:20 - r - INFO: - buffer_size 8000 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - batch_size 128 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - tau 0.001 <class 'float'> | |
| 2023-04-14 15:02:20 - r - INFO: - critic_hidden_dim 256 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - actor_hidden_dim 256 <class 'int'> | |
| 2023-04-14 15:02:20 - r - INFO: - task_dir D:\MyDoc\joyrl\joyrl-offline/tasks/Train_Pendulum-v1_DDPG_HER_20230414-150220 <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - res_dir D:\MyDoc\joyrl\joyrl-offline/tasks/Train_Pendulum-v1_DDPG_HER_20230414-150220/results <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - log_dir D:\MyDoc\joyrl\joyrl-offline/tasks/Train_Pendulum-v1_DDPG_HER_20230414-150220/logs <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - traj_dir D:\MyDoc\joyrl\joyrl-offline/tasks/Train_Pendulum-v1_DDPG_HER_20230414-150220/traj <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - video_dir D:\MyDoc\joyrl\joyrl-offline/tasks/Train_Pendulum-v1_DDPG_HER_20230414-150220/videos <class 'str'> | |
| 2023-04-14 15:02:20 - r - INFO: - ================================================================================ | |
| 2023-04-14 15:02:20 - r - INFO: - action_bound: 2.0 | |
| 2023-04-14 15:02:20 - r - INFO: - n_states: 3, n_actions: 1 | |
| 2023-04-14 15:02:20 - r - INFO: - Start training! | |
| 2023-04-14 15:02:20 - r - INFO: - Env: Pendulum-v1, Algorithm: DDPG_HER, Device: cpu | |
| 2023-04-14 15:02:21 - r - INFO: - Episode: 1/150, Reward: -1430.631, Step: 200 | |
| 2023-04-14 15:02:22 - r - INFO: - Episode: 2/150, Reward: -1190.537, Step: 200 | |
| 2023-04-14 15:02:24 - r - INFO: - Episode: 3/150, Reward: -1079.347, Step: 200 | |
| 2023-04-14 15:02:26 - r - INFO: - Episode: 4/150, Reward: -1579.293, Step: 200 | |
| 2023-04-14 15:02:28 - r - INFO: - Episode: 5/150, Reward: -1523.855, Step: 200 | |
| 2023-04-14 15:02:29 - r - INFO: - Current episode 5 has the best eval reward: -1410.344 | |
| 2023-04-14 15:02:31 - r - INFO: - Episode: 6/150, Reward: -1764.677, Step: 200 | |
| 2023-04-14 15:02:33 - r - INFO: - Episode: 7/150, Reward: -1739.242, Step: 200 | |
| 2023-04-14 15:02:35 - r - INFO: - Episode: 8/150, Reward: -1730.528, Step: 200 | |
| 2023-04-14 15:02:37 - r - INFO: - Episode: 9/150, Reward: -1780.244, Step: 200 | |
| 2023-04-14 15:02:39 - r - INFO: - Episode: 10/150, Reward: -1712.587, Step: 200 | |
| 2023-04-14 15:02:41 - r - INFO: - Episode: 11/150, Reward: -1750.423, Step: 200 | |
| 2023-04-14 15:02:43 - r - INFO: - Episode: 12/150, Reward: -1589.055, Step: 200 | |
| 2023-04-14 15:02:45 - r - INFO: - Episode: 13/150, Reward: -1601.020, Step: 200 | |
| 2023-04-14 15:02:47 - r - INFO: - Episode: 14/150, Reward: -1538.405, Step: 200 | |
| 2023-04-14 15:02:49 - r - INFO: - Episode: 15/150, Reward: -1437.579, Step: 200 | |
| 2023-04-14 15:02:52 - r - INFO: - Episode: 16/150, Reward: -1479.413, Step: 200 | |
| 2023-04-14 15:02:54 - r - INFO: - Episode: 17/150, Reward: -1260.055, Step: 200 | |
| 2023-04-14 15:02:56 - r - INFO: - Episode: 18/150, Reward: -1442.884, Step: 200 | |
| 2023-04-14 15:02:58 - r - INFO: - Episode: 19/150, Reward: -1143.685, Step: 200 | |
| 2023-04-14 15:03:00 - r - INFO: - Episode: 20/150, Reward: -1305.908, Step: 200 | |
| 2023-04-14 15:03:00 - r - INFO: - Current episode 20 has the best eval reward: -1222.109 | |
| 2023-04-14 15:03:02 - r - INFO: - Episode: 21/150, Reward: -1303.816, Step: 200 | |
| 2023-04-14 15:03:04 - r - INFO: - Episode: 22/150, Reward: -1539.392, Step: 200 | |
| 2023-04-14 15:03:06 - r - INFO: - Episode: 23/150, Reward: -1265.643, Step: 200 | |
| 2023-04-14 15:03:08 - r - INFO: - Episode: 24/150, Reward: -1190.020, Step: 200 | |
| 2023-04-14 15:03:10 - r - INFO: - Episode: 25/150, Reward: -1509.776, Step: 200 | |
| 2023-04-14 15:03:11 - r - INFO: - Current episode 25 has the best eval reward: -1219.529 | |
| 2023-04-14 15:03:13 - r - INFO: - Episode: 26/150, Reward: -1139.533, Step: 200 | |
| 2023-04-14 15:03:15 - r - INFO: - Episode: 27/150, Reward: -1356.016, Step: 200 | |
| 2023-04-14 15:03:17 - r - INFO: - Episode: 28/150, Reward: -1212.430, Step: 200 | |
| 2023-04-14 15:03:19 - r - INFO: - Episode: 29/150, Reward: -1173.039, Step: 200 | |
| 2023-04-14 15:03:21 - r - INFO: - Episode: 30/150, Reward: -1134.589, Step: 200 | |
| 2023-04-14 15:03:22 - r - INFO: - Current episode 30 has the best eval reward: -1103.859 | |
| 2023-04-14 15:03:24 - r - INFO: - Episode: 31/150, Reward: -1539.600, Step: 200 | |
| 2023-04-14 15:03:26 - r - INFO: - Episode: 32/150, Reward: -906.160, Step: 200 | |
| 2023-04-14 15:03:29 - r - INFO: - Episode: 33/150, Reward: -1051.940, Step: 200 | |
| 2023-04-14 15:03:31 - r - INFO: - Episode: 34/150, Reward: -1015.502, Step: 200 | |
| 2023-04-14 15:03:33 - r - INFO: - Episode: 35/150, Reward: -1291.428, Step: 200 | |
| 2023-04-14 15:03:36 - r - INFO: - Episode: 36/150, Reward: -968.884, Step: 200 | |
| 2023-04-14 15:03:38 - r - INFO: - Episode: 37/150, Reward: -1275.947, Step: 200 | |
| 2023-04-14 15:03:41 - r - INFO: - Episode: 38/150, Reward: -942.761, Step: 200 | |
| 2023-04-14 15:03:43 - r - INFO: - Episode: 39/150, Reward: -825.685, Step: 200 | |
| 2023-04-14 15:03:45 - r - INFO: - Episode: 40/150, Reward: -944.424, Step: 200 | |
| 2023-04-14 15:03:46 - r - INFO: - Current episode 40 has the best eval reward: -878.044 | |
| 2023-04-14 15:03:48 - r - INFO: - Episode: 41/150, Reward: -882.493, Step: 200 | |
| 2023-04-14 15:03:50 - r - INFO: - Episode: 42/150, Reward: -927.014, Step: 200 | |
| 2023-04-14 15:03:52 - r - INFO: - Episode: 43/150, Reward: -812.261, Step: 200 | |
| 2023-04-14 15:03:55 - r - INFO: - Episode: 44/150, Reward: -815.314, Step: 200 | |
| 2023-04-14 15:03:57 - r - INFO: - Episode: 45/150, Reward: -661.461, Step: 200 | |
| 2023-04-14 15:03:58 - r - INFO: - Current episode 45 has the best eval reward: -548.563 | |
| 2023-04-14 15:04:00 - r - INFO: - Episode: 46/150, Reward: -845.423, Step: 200 | |
| 2023-04-14 15:04:02 - r - INFO: - Episode: 47/150, Reward: -792.809, Step: 200 | |
| 2023-04-14 15:04:05 - r - INFO: - Episode: 48/150, Reward: -990.300, Step: 200 | |
| 2023-04-14 15:04:07 - r - INFO: - Episode: 49/150, Reward: -383.060, Step: 200 | |
| 2023-04-14 15:04:10 - r - INFO: - Episode: 50/150, Reward: -538.956, Step: 200 | |
| 2023-04-14 15:04:11 - r - INFO: - Current episode 50 has the best eval reward: -383.151 | |
| 2023-04-14 15:04:13 - r - INFO: - Episode: 51/150, Reward: -130.052, Step: 200 | |
| 2023-04-14 15:04:15 - r - INFO: - Episode: 52/150, Reward: -377.034, Step: 200 | |
| 2023-04-14 15:04:17 - r - INFO: - Episode: 53/150, Reward: -132.334, Step: 200 | |
| 2023-04-14 15:04:19 - r - INFO: - Episode: 54/150, Reward: -354.477, Step: 200 | |
| 2023-04-14 15:04:21 - r - INFO: - Episode: 55/150, Reward: -553.161, Step: 200 | |
| 2023-04-14 15:04:22 - r - INFO: - Current episode 55 has the best eval reward: -299.486 | |
| 2023-04-14 15:04:24 - r - INFO: - Episode: 56/150, Reward: -132.422, Step: 200 | |
| 2023-04-14 15:04:26 - r - INFO: - Episode: 57/150, Reward: -386.039, Step: 200 | |
| 2023-04-14 15:04:28 - r - INFO: - Episode: 58/150, Reward: -132.089, Step: 200 | |
| 2023-04-14 15:04:30 - r - INFO: - Episode: 59/150, Reward: -131.928, Step: 200 | |
| 2023-04-14 15:04:32 - r - INFO: - Episode: 60/150, Reward: -402.214, Step: 200 | |
| 2023-04-14 15:04:35 - r - INFO: - Episode: 61/150, Reward: -259.670, Step: 200 | |
| 2023-04-14 15:04:37 - r - INFO: - Episode: 62/150, Reward: -161.742, Step: 200 | |
| 2023-04-14 15:04:39 - r - INFO: - Episode: 63/150, Reward: -136.568, Step: 200 | |
| 2023-04-14 15:04:41 - r - INFO: - Episode: 64/150, Reward: -254.675, Step: 200 | |
| 2023-04-14 15:04:43 - r - INFO: - Episode: 65/150, Reward: -1490.135, Step: 200 | |
| 2023-04-14 15:04:46 - r - INFO: - Episode: 66/150, Reward: -255.318, Step: 200 | |
| 2023-04-14 15:04:47 - r - INFO: - Episode: 67/150, Reward: -1386.437, Step: 200 | |
| 2023-04-14 15:04:50 - r - INFO: - Episode: 68/150, Reward: -358.886, Step: 200 | |
| 2023-04-14 15:04:52 - r - INFO: - Episode: 69/150, Reward: -134.174, Step: 200 | |
| 2023-04-14 15:04:54 - r - INFO: - Episode: 70/150, Reward: -121.258, Step: 200 | |
| 2023-04-14 15:04:54 - r - INFO: - Current episode 70 has the best eval reward: -220.980 | |
| 2023-04-14 15:04:56 - r - INFO: - Episode: 71/150, Reward: -244.039, Step: 200 | |
| 2023-04-14 15:04:58 - r - INFO: - Episode: 72/150, Reward: -2.718, Step: 200 | |
| 2023-04-14 15:05:00 - r - INFO: - Episode: 73/150, Reward: -127.649, Step: 200 | |
| 2023-04-14 15:05:02 - r - INFO: - Episode: 74/150, Reward: -132.686, Step: 200 | |
| 2023-04-14 15:05:04 - r - INFO: - Episode: 75/150, Reward: -235.846, Step: 200 | |
| 2023-04-14 15:05:05 - r - INFO: - Current episode 75 has the best eval reward: -201.119 | |
| 2023-04-14 15:05:07 - r - INFO: - Episode: 76/150, Reward: -396.815, Step: 200 | |
| 2023-04-14 15:05:09 - r - INFO: - Episode: 77/150, Reward: -122.913, Step: 200 | |
| 2023-04-14 15:05:11 - r - INFO: - Episode: 78/150, Reward: -117.540, Step: 200 | |
| 2023-04-14 15:05:13 - r - INFO: - Episode: 79/150, Reward: -127.221, Step: 200 | |
| 2023-04-14 15:05:15 - r - INFO: - Episode: 80/150, Reward: -126.340, Step: 200 | |
| 2023-04-14 15:05:16 - r - INFO: - Current episode 80 has the best eval reward: -125.201 | |
| 2023-04-14 15:05:18 - r - INFO: - Episode: 81/150, Reward: -127.639, Step: 200 | |
| 2023-04-14 15:05:20 - r - INFO: - Episode: 82/150, Reward: -128.077, Step: 200 | |
| 2023-04-14 15:05:22 - r - INFO: - Episode: 83/150, Reward: -116.016, Step: 200 | |
| 2023-04-14 15:05:24 - r - INFO: - Episode: 84/150, Reward: -124.603, Step: 200 | |
| 2023-04-14 15:05:26 - r - INFO: - Episode: 85/150, Reward: -126.350, Step: 200 | |
| 2023-04-14 15:05:29 - r - INFO: - Episode: 86/150, Reward: -117.398, Step: 200 | |
| 2023-04-14 15:05:31 - r - INFO: - Episode: 87/150, Reward: -125.239, Step: 200 | |
| 2023-04-14 15:05:33 - r - INFO: - Episode: 88/150, Reward: -1.460, Step: 200 | |
| 2023-04-14 15:05:35 - r - INFO: - Episode: 89/150, Reward: -367.643, Step: 200 | |
| 2023-04-14 15:05:37 - r - INFO: - Episode: 90/150, Reward: -1.008, Step: 200 | |
| 2023-04-14 15:05:39 - r - INFO: - Episode: 91/150, Reward: -126.855, Step: 200 | |
| 2023-04-14 15:05:41 - r - INFO: - Episode: 92/150, Reward: -374.736, Step: 200 | |
| 2023-04-14 15:05:43 - r - INFO: - Episode: 93/150, Reward: -251.304, Step: 200 | |
| 2023-04-14 15:05:45 - r - INFO: - Episode: 94/150, Reward: -238.669, Step: 200 | |
| 2023-04-14 15:05:47 - r - INFO: - Episode: 95/150, Reward: -119.575, Step: 200 | |
| 2023-04-14 15:05:50 - r - INFO: - Episode: 96/150, Reward: -370.054, Step: 200 | |
| 2023-04-14 15:05:52 - r - INFO: - Episode: 97/150, Reward: -392.496, Step: 200 | |
| 2023-04-14 15:05:54 - r - INFO: - Episode: 98/150, Reward: -240.950, Step: 200 | |
| 2023-04-14 15:05:56 - r - INFO: - Episode: 99/150, Reward: -235.973, Step: 200 | |
| 2023-04-14 15:05:58 - r - INFO: - Episode: 100/150, Reward: -126.822, Step: 200 | |
| 2023-04-14 15:05:58 - r - INFO: - Current episode 100 has the best eval reward: -111.559 | |
| 2023-04-14 15:06:00 - r - INFO: - Episode: 101/150, Reward: -1.238, Step: 200 | |
| 2023-04-14 15:06:02 - r - INFO: - Episode: 102/150, Reward: -129.029, Step: 200 | |
| 2023-04-14 15:06:04 - r - INFO: - Episode: 103/150, Reward: -126.249, Step: 200 | |
| 2023-04-14 15:06:06 - r - INFO: - Episode: 104/150, Reward: -121.460, Step: 200 | |
| 2023-04-14 15:06:08 - r - INFO: - Episode: 105/150, Reward: -275.772, Step: 200 | |
| 2023-04-14 15:06:11 - r - INFO: - Episode: 106/150, Reward: -127.109, Step: 200 | |
| 2023-04-14 15:06:13 - r - INFO: - Episode: 107/150, Reward: -3.014, Step: 200 | |
| 2023-04-14 15:06:15 - r - INFO: - Episode: 108/150, Reward: -0.725, Step: 200 | |
| 2023-04-14 15:06:17 - r - INFO: - Episode: 109/150, Reward: -260.152, Step: 200 | |
| 2023-04-14 15:06:19 - r - INFO: - Episode: 110/150, Reward: -122.403, Step: 200 | |
| 2023-04-14 15:06:21 - r - INFO: - Episode: 111/150, Reward: -1.335, Step: 200 | |
| 2023-04-14 15:06:23 - r - INFO: - Episode: 112/150, Reward: -368.960, Step: 200 | |
| 2023-04-14 15:06:25 - r - INFO: - Episode: 113/150, Reward: -239.405, Step: 200 | |
| 2023-04-14 15:06:27 - r - INFO: - Episode: 114/150, Reward: -241.047, Step: 200 | |
| 2023-04-14 15:06:29 - r - INFO: - Episode: 115/150, Reward: -124.533, Step: 200 | |
| 2023-04-14 15:06:32 - r - INFO: - Episode: 116/150, Reward: -123.599, Step: 200 | |
| 2023-04-14 15:06:34 - r - INFO: - Episode: 117/150, Reward: -125.264, Step: 200 | |
| 2023-04-14 15:06:37 - r - INFO: - Episode: 118/150, Reward: -123.005, Step: 200 | |
| 2023-04-14 15:06:39 - r - INFO: - Episode: 119/150, Reward: -128.860, Step: 200 | |
| 2023-04-14 15:06:41 - r - INFO: - Episode: 120/150, Reward: -126.551, Step: 200 | |
| 2023-04-14 15:06:43 - r - INFO: - Episode: 121/150, Reward: -117.272, Step: 200 | |
| 2023-04-14 15:06:45 - r - INFO: - Episode: 122/150, Reward: -122.532, Step: 200 | |
| 2023-04-14 15:06:47 - r - INFO: - Episode: 123/150, Reward: -333.176, Step: 200 | |
| 2023-04-14 15:06:49 - r - INFO: - Episode: 124/150, Reward: -128.796, Step: 200 | |
| 2023-04-14 15:06:51 - r - INFO: - Episode: 125/150, Reward: -127.390, Step: 200 | |
| 2023-04-14 15:06:54 - r - INFO: - Episode: 126/150, Reward: -124.982, Step: 200 | |
| 2023-04-14 15:06:56 - r - INFO: - Episode: 127/150, Reward: -376.673, Step: 200 | |
| 2023-04-14 15:06:58 - r - INFO: - Episode: 128/150, Reward: -374.774, Step: 200 | |
| 2023-04-14 15:07:00 - r - INFO: - Episode: 129/150, Reward: -236.785, Step: 200 | |
| 2023-04-14 15:07:02 - r - INFO: - Episode: 130/150, Reward: -122.334, Step: 200 | |
| 2023-04-14 15:07:04 - r - INFO: - Episode: 131/150, Reward: -254.140, Step: 200 | |
| 2023-04-14 15:07:06 - r - INFO: - Episode: 132/150, Reward: -247.884, Step: 200 | |
| 2023-04-14 15:07:08 - r - INFO: - Episode: 133/150, Reward: -129.876, Step: 200 | |
| 2023-04-14 15:07:10 - r - INFO: - Episode: 134/150, Reward: -251.932, Step: 200 | |
| 2023-04-14 15:07:12 - r - INFO: - Episode: 135/150, Reward: -124.185, Step: 200 | |
| 2023-04-14 15:07:15 - r - INFO: - Episode: 136/150, Reward: -244.707, Step: 200 | |
| 2023-04-14 15:07:17 - r - INFO: - Episode: 137/150, Reward: -123.827, Step: 200 | |
| 2023-04-14 15:07:19 - r - INFO: - Episode: 138/150, Reward: -242.895, Step: 200 | |
| 2023-04-14 15:07:21 - r - INFO: - Episode: 139/150, Reward: -125.282, Step: 200 | |
| 2023-04-14 15:07:23 - r - INFO: - Episode: 140/150, Reward: -353.050, Step: 200 | |
| 2023-04-14 15:07:26 - r - INFO: - Episode: 141/150, Reward: -1.573, Step: 200 | |
| 2023-04-14 15:07:27 - r - INFO: - Episode: 142/150, Reward: -364.479, Step: 200 | |
| 2023-04-14 15:07:29 - r - INFO: - Episode: 143/150, Reward: -0.872, Step: 200 | |
| 2023-04-14 15:07:31 - r - INFO: - Episode: 144/150, Reward: -120.558, Step: 200 | |
| 2023-04-14 15:07:33 - r - INFO: - Episode: 145/150, Reward: -246.312, Step: 200 | |
| 2023-04-14 15:07:34 - r - INFO: - Current episode 145 has the best eval reward: -99.201 | |
| 2023-04-14 15:07:36 - r - INFO: - Episode: 146/150, Reward: -0.580, Step: 200 | |
| 2023-04-14 15:07:38 - r - INFO: - Episode: 147/150, Reward: -245.957, Step: 200 | |
| 2023-04-14 15:07:40 - r - INFO: - Episode: 148/150, Reward: -120.189, Step: 200 | |
| 2023-04-14 15:07:42 - r - INFO: - Episode: 149/150, Reward: -242.257, Step: 200 | |
| 2023-04-14 15:07:44 - r - INFO: - Episode: 150/150, Reward: -1.290, Step: 200 | |
| 2023-04-14 15:07:45 - r - INFO: - Finish training! | |