chenkq commited on
Commit
b873b63
1 Parent(s): a93e1d3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -88,9 +88,11 @@ with torch.no_grad():
88
  # 4</s>
89
  ```
90
 
91
- 当单卡显存不足时,可以将模型切分到多个小显存GPU
 
92
 
93
- dispatch the model into multiple GPUs with smaller VRAM.
 
94
 
95
  ```python
96
  import torch
 
88
  # 4</s>
89
  ```
90
 
91
+ 当单卡显存不足时,可以将模型切分到多个小显存GPU上。以下是个当你有两张24GB的GPU,16GBCPU内存的例子。
92
+ 你可以将`infer_auto_device_map`的参数改成你的配置。注意这里将GPU显存少写了一点,这是为推理时中间状态预留出一部分显存。
93
 
94
+ dispatch the model into multiple GPUs with smaller VRAM. This is an example for you have two 24GB GPU and 16GB CPU memory.
95
+ you can change the arguments of `infer_auto_device_map` with your own setting.
96
 
97
  ```python
98
  import torch