2024-07-10 17:35:06 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40005, worker_address='http://10.140.60.25:40005', controller_address='http://10.140.60.209:10075', model_path='share_internvl/InternVL2-26B/', model_name=None, device='cuda', limit_model_concurrency=5, stream_interval=1, load_8bit=False)
2024-07-10 17:35:06 | INFO | model_worker | Loading the model InternVL2-26B on worker 119769 ...
2024-07-10 17:35:06 | WARNING | transformers.tokenization_utils_base | Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-10 17:35:06 | WARNING | transformers.tokenization_utils_base | Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-10 17:35:09 | ERROR | stderr | /mnt/petrelfs/wangweiyun/miniconda3/envs/internvl-apex/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:397: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-10 17:35:09 | ERROR | stderr |   warnings.warn(
2024-07-10 17:35:10 | ERROR | stderr | 
Loading checkpoint shards:   0%|          | 0/11 [00:00<?, ?it/s]
2024-07-10 17:35:21 | ERROR | stderr | 
Loading checkpoint shards:   9%|▉         | 1/11 [00:10<01:47, 10.77s/it]
2024-07-10 17:35:33 | ERROR | stderr | 
Loading checkpoint shards:  18%|█▊        | 2/11 [00:23<01:45, 11.71s/it]
2024-07-10 17:35:42 | ERROR | stderr | 
Loading checkpoint shards:  27%|██▋       | 3/11 [00:32<01:24, 10.55s/it]
2024-07-10 17:35:50 | ERROR | stderr | 
Loading checkpoint shards:  36%|███▋      | 4/11 [00:39<01:05,  9.36s/it]
2024-07-10 17:35:59 | ERROR | stderr | 
Loading checkpoint shards:  45%|████▌     | 5/11 [00:49<00:56,  9.48s/it]
2024-07-10 17:36:09 | ERROR | stderr | 
Loading checkpoint shards:  55%|█████▍    | 6/11 [00:58<00:47,  9.41s/it]
2024-07-10 17:36:18 | ERROR | stderr | 
Loading checkpoint shards:  64%|██████▎   | 7/11 [01:08<00:37,  9.48s/it]
2024-07-10 17:36:25 | ERROR | stderr | 
Loading checkpoint shards:  73%|███████▎  | 8/11 [01:15<00:26,  8.67s/it]
2024-07-10 17:36:32 | ERROR | stderr | 
Loading checkpoint shards:  82%|████████▏ | 9/11 [01:22<00:16,  8.14s/it]
2024-07-10 17:36:39 | ERROR | stderr | 
Loading checkpoint shards:  91%|█████████ | 10/11 [01:29<00:07,  7.80s/it]
2024-07-10 17:36:43 | ERROR | stderr | 
Loading checkpoint shards: 100%|██████████| 11/11 [01:32<00:00,  6.45s/it]
2024-07-10 17:36:43 | ERROR | stderr | 
Loading checkpoint shards: 100%|██████████| 11/11 [01:32<00:00,  8.44s/it]
2024-07-10 17:36:43 | ERROR | stderr | 
2024-07-10 17:37:07 | INFO | model_worker | Register to controller
2024-07-10 17:37:07 | ERROR | stderr | INFO:     Started server process [5180]
2024-07-10 17:37:07 | ERROR | stderr | INFO:     Waiting for application startup.
2024-07-10 17:37:07 | ERROR | stderr | INFO:     Application startup complete.
2024-07-10 17:37:07 | ERROR | stderr | INFO:     Uvicorn running on http://0.0.0.0:40005 (Press CTRL+C to quit)
2024-07-10 17:37:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: None. global_counter: 0
2024-07-10 17:37:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: None. global_counter: 0
2024-07-10 17:37:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: None. global_counter: 0
2024-07-10 17:38:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: None. global_counter: 0
2024-07-10 17:38:15 | INFO | stdout | INFO:     10.140.60.209:59250 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:17 | INFO | stdout | INFO:     10.140.60.209:59320 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: None. global_counter: 0
2024-07-10 17:38:26 | INFO | stdout | INFO:     10.140.60.209:59364 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:29 | INFO | stdout | INFO:     10.140.60.209:59380 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:30 | INFO | stdout | INFO:     10.140.60.209:59394 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:33 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1
2024-07-10 17:38:33 | INFO | stdout | INFO:     10.140.60.209:59400 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-07-10 17:38:33 | INFO | stdout | history: []
2024-07-10 17:38:33 | INFO | stdout | question: Image-1: <image>
2024-07-10 17:38:33 | INFO | stdout | 这是啥
2024-07-10 17:38:33 | INFO | stdout | pil_images: [<PIL.PngImagePlugin.PngImageFile image mode=RGB size=5472x3648 at 0x7FA89C5BC370>]
2024-07-10 17:38:33 | INFO | model_worker | dynamic_image_size: True
2024-07-10 17:38:33 | INFO | model_worker | use_thumbnail: True
2024-07-10 17:38:34 | INFO | model_worker | Resize images to 448x448
2024-07-10 17:38:34 | INFO | model_worker | Split images to torch.Size([7, 3, 448, 448])
2024-07-10 17:38:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1
2024-07-10 17:38:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 1
2024-07-10 17:38:50 | INFO | stdout | INFO:     10.140.60.209:59418 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:38:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 1
2024-07-10 17:38:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 2
2024-07-10 17:38:53 | INFO | stdout | INFO:     10.140.60.209:59424 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-07-10 17:38:53 | INFO | stdout | history: [['Image-1: <image>\n这是啥', '这是一杯饮料，放在一个透明的玻璃杯里，杯子上印有“AURORA GLASS”字样。杯子里有冰块，周围也散落着几块冰块。背景中有一盆绿色的植物，放在一个白色的花盆里。饮料可能是某种碳酸饮料，比如啤酒或汽水，但具体种类无法确定。 ']]
2024-07-10 17:38:53 | INFO | stdout | question: 根据图片写一篇800字的作文
2024-07-10 17:38:53 | INFO | stdout | pil_images: [<PIL.PngImagePlugin.PngImageFile image mode=RGB size=5472x3648 at 0x7FA89C482E00>]
2024-07-10 17:38:53 | INFO | model_worker | dynamic_image_size: True
2024-07-10 17:38:53 | INFO | model_worker | use_thumbnail: True
2024-07-10 17:38:54 | INFO | model_worker | Resize images to 448x448
2024-07-10 17:38:54 | INFO | model_worker | Split images to torch.Size([7, 3, 448, 448])
2024-07-10 17:39:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 2
2024-07-10 17:39:14 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:39:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:39:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:39:42 | INFO | stdout | INFO:     10.140.60.209:59622 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:39:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:40:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:40:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:40:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:40:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:41:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:41:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:41:31 | INFO | stdout | INFO:     10.140.60.209:60036 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:41:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:41:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:42:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:42:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:42:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:42:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:43:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:43:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:43:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:43:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:44:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:44:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:44:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:44:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:45:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:45:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:45:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:45:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:46:07 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:46:22 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:46:37 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:46:52 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:47:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:47:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:47:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:47:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:48:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:48:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:48:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:48:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:49:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:49:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:49:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:49:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:50:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:50:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:50:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:50:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:51:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:51:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:51:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:51:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:52:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:52:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:52:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:52:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:53:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:53:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:53:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:53:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:54:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:54:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:54:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:54:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:55:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:55:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:55:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:55:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:56:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:56:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:56:33 | INFO | stdout | INFO:     10.140.60.209:35500 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:56:36 | INFO | stdout | INFO:     10.140.60.209:35522 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:56:37 | INFO | stdout | INFO:     10.140.60.209:35540 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 17:56:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:56:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:57:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:57:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:57:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:57:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:58:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:58:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:58:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:58:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:59:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:59:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:59:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 17:59:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:00:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:00:15 | INFO | stdout | INFO:     10.140.60.209:36335 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:00:18 | INFO | stdout | INFO:     10.140.60.209:36414 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:00:19 | INFO | stdout | INFO:     10.140.60.209:36432 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:00:23 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:00:38 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:00:53 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:01:08 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:01:24 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:01:30 | INFO | stdout | INFO:     10.140.60.209:36650 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:33 | INFO | stdout | INFO:     10.140.60.209:36664 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:36 | INFO | stdout | INFO:     10.140.60.209:36680 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:38 | INFO | stdout | INFO:     10.140.60.209:36696 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:01:54 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:02:09 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:02:24 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:02:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:02:54 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:03:09 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:03:24 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:03:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:03:54 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:04:09 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:04:24 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:04:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2
2024-07-10 18:04:54 | INFO | model_worker | Send heart beat. Models: ['InternVL2-26B']. Semaphore: Semaphore(value=5, locked=False). global_counter: 2