2024-07-10 18:00:50 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40006, worker_address='http://10.140.60.25:40006', controller_address='http://10.140.60.209:10075', model_path='share_internvl/InternVL2-40B/', model_name=None, device='auto', limit_model_concurrency=5, stream_interval=1, load_8bit=False)
2024-07-10 18:00:50 | INFO | model_worker | Loading the model InternVL2-40B on worker 1c9f40 ...
2024-07-10 18:00:51 | WARNING | transformers.tokenization_utils_base | Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-10 18:00:51 | WARNING | transformers.tokenization_utils_base | Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-10 18:00:54 | ERROR | stderr | /mnt/petrelfs/wangweiyun/miniconda3/envs/internvl/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:397: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-10 18:00:54 | ERROR | stderr |   warnings.warn(
2024-07-10 18:00:56 | ERROR | stderr | 
Loading checkpoint shards:   0%|          | 0/17 [00:00<?, ?it/s]
2024-07-10 18:00:58 | ERROR | stderr | 
Loading checkpoint shards:   6%|▌         | 1/17 [00:02<00:32,  2.00s/it]
2024-07-10 18:01:00 | ERROR | stderr | 
Loading checkpoint shards:  12%|█▏        | 2/17 [00:03<00:29,  1.97s/it]
2024-07-10 18:01:02 | ERROR | stderr | 
Loading checkpoint shards:  18%|█▊        | 3/17 [00:05<00:27,  1.96s/it]
2024-07-10 18:01:04 | ERROR | stderr | 
Loading checkpoint shards:  24%|██▎       | 4/17 [00:07<00:25,  1.98s/it]
2024-07-10 18:01:06 | ERROR | stderr | 
Loading checkpoint shards:  29%|██▉       | 5/17 [00:10<00:24,  2.06s/it]
2024-07-10 18:01:08 | ERROR | stderr | 
Loading checkpoint shards:  35%|███▌      | 6/17 [00:12<00:22,  2.03s/it]
2024-07-10 18:01:10 | ERROR | stderr | 
Loading checkpoint shards:  41%|████      | 7/17 [00:14<00:20,  2.02s/it]
2024-07-10 18:01:12 | ERROR | stderr | 
Loading checkpoint shards:  47%|████▋     | 8/17 [00:16<00:18,  2.05s/it]
2024-07-10 18:01:14 | ERROR | stderr | 
Loading checkpoint shards:  53%|█████▎    | 9/17 [00:18<00:16,  2.03s/it]
2024-07-10 18:01:16 | ERROR | stderr | 
Loading checkpoint shards:  59%|█████▉    | 10/17 [00:20<00:14,  2.02s/it]
2024-07-10 18:01:18 | ERROR | stderr | 
Loading checkpoint shards:  65%|██████▍   | 11/17 [00:22<00:12,  2.07s/it]
2024-07-10 18:01:21 | ERROR | stderr | 
Loading checkpoint shards:  71%|███████   | 12/17 [00:24<00:10,  2.12s/it]
2024-07-10 18:01:23 | ERROR | stderr | 
Loading checkpoint shards:  76%|███████▋  | 13/17 [00:26<00:08,  2.13s/it]
2024-07-10 18:01:25 | ERROR | stderr | 
Loading checkpoint shards:  82%|████████▏ | 14/17 [00:28<00:06,  2.15s/it]
2024-07-10 18:01:27 | ERROR | stderr | 
Loading checkpoint shards:  88%|████████▊ | 15/17 [00:31<00:04,  2.14s/it]
2024-07-10 18:01:29 | ERROR | stderr | 
Loading checkpoint shards:  94%|█████████▍| 16/17 [00:33<00:02,  2.11s/it]
2024-07-10 18:01:30 | ERROR | stderr | 
Loading checkpoint shards: 100%|██████████| 17/17 [00:34<00:00,  1.83s/it]
2024-07-10 18:01:30 | ERROR | stderr | 
Loading checkpoint shards: 100%|██████████| 17/17 [00:34<00:00,  2.02s/it]
2024-07-10 18:01:30 | ERROR | stderr | 
2024-07-10 18:01:31 | INFO | model_worker | Register to controller
2024-07-10 18:01:31 | ERROR | stderr | INFO:     Started server process [62590]
2024-07-10 18:01:31 | ERROR | stderr | INFO:     Waiting for application startup.
2024-07-10 18:01:31 | ERROR | stderr | INFO:     Application startup complete.
2024-07-10 18:01:31 | ERROR | stderr | INFO:     Uvicorn running on http://0.0.0.0:40006 (Press CTRL+C to quit)
2024-07-10 18:01:33 | INFO | stdout | INFO:     10.140.60.209:48796 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:36 | INFO | stdout | INFO:     10.140.60.209:48812 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:38 | INFO | stdout | INFO:     10.140.60.209:48828 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-10 18:01:39 | INFO | model_worker | Send heart beat. Models: ['InternVL2-40B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1
2024-07-10 18:01:39 | INFO | stdout | INFO:     10.140.60.209:48836 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-07-10 18:01:39 | INFO | stdout | history: []
2024-07-10 18:01:39 | INFO | stdout | question: Image-1: <image>
2024-07-10 18:01:39 | INFO | stdout | Describe this image in detail.
2024-07-10 18:01:39 | INFO | stdout | pil_images: [<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x7F01A84C9A80>]
2024-07-10 18:01:39 | INFO | model_worker | dynamic_image_size: True
2024-07-10 18:01:39 | INFO | model_worker | use_thumbnail: True
2024-07-10 18:01:39 | INFO | model_worker | Resize images to 448x448
2024-07-10 18:01:39 | INFO | model_worker | Split images to torch.Size([10, 3, 448, 448])
2024-07-10 18:01:46 | INFO | model_worker | Send heart beat. Models: ['InternVL2-40B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1
2024-07-10 18:02:01 | INFO | model_worker | Send heart beat. Models: ['InternVL2-40B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1
2024-07-10 18:02:16 | INFO | model_worker | Send heart beat. Models: ['InternVL2-40B']. Semaphore: Semaphore(value=4, locked=False). global_counter: 1