THUDM/chatglm3-6b-32k · 本地部署 chatglm3 6B， api 官方案列，关闭流式回复就出错

Nov 9, 2023

OS： Ubuntu 22.04.2 LTS
GPU: 3090
描述：按照官方指导部署在本地，
以下为示例代码：
import openai
if name == "main":
openai.api_base = "http://192.168.100.118:8000/v1"
openai.api_key = "none"
for chunk in openai.ChatCompletion.create(
model="chatglm3-6b",
messages=[
{"role": "user", "content": "李焕英"}
],
stream=False
):
if hasattr(chunk.choices[0].delta, "content"):
print(chunk.choices[0].delta.content, end="", flush=True)

如果 stream=True 那回复内容正常。
如果 stream=False
就会出错,信息如下：

Traceback (most recent call last):
File "D:\python提交到qingyingkj\ooo.py", line 12, in
if hasattr(chunk.choices[0].delta, "content"):
AttributeError: 'str' object has no attribute 'choices'

zRzRzRzRzRzRzR

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org Jan 4

更新到最新代码

zRzRzRzRzRzRzR changed discussion status to closed Jan 4