metadata

frameworks:
  - Pytorch
license: other
tasks:
  - image-text-to-text

LLama.cpp克隆与编译

git clone https://github.com/piDack/llama.cpp -b support_glm_edge_model
cmake -B build -DGGML_CUDA=ON #或其他硬件
cmake --build build -- -j

多模态运行

从本repo中下载所需要的gguf，然后通过命令行运行该模型：

llama-llava-cli -m model_path/ggml-model-f16.gguf --mmproj model_path/mmproj-model-f16.gguf --image img_path/image.jpg -p "<|system|>\n system prompt <image><|user|>\n prompt <|assistant|>\n"

下载

#安装ModelScope
pip install modelscope

#SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('ZhipuAI/glm-edge-v-5b-gguf')

Git下载

#Git模型下载
git clone https://www.modelscope.cn/ZhipuAI/glm-edge-v-5b-gguf.git