put "ggml-" infront of the model name

So rename mpt-7b-storywriter.ggmlv3.fp16.bin into ggml-mpt-7b-storywriter.ggmlv3.fp16.bin?

just realized, this is ggmlv3. gpt4all doesnt support ggmlv3, only ggmlv1. you can keep the model if you want ig

Yes it does if you use the ctransformers backend. That's why I linked it, because it supports the one Python binding right now that supports these MPT models.

I've not had a chance to do any kind of tutorial. But it does work in gpt4all-ui + ctransformers

ahh, i see. might give it a go myself

(Not sure if there is anything missing in this or wrong, need someone to confirm this guide)
To set up gpt4all-ui and ctransformers together, you can follow these steps:

• Clone or download the gpt4all-ui repository from GitHub¹. This is a web user interface for interacting with various large language models, such as GPT4All, GPT-J, GPT-Q, and cTransformers. You can use git to clone the repository or download the zip file and unzip it in a folder of your choice. For example:

git clone https://github.com/ParisNeo/gpt4all-ui.git

cd gpt4all-ui

• Create and activate a virtual environment for the project. You can use conda, venv, or any other tool you prefer. For example, using conda:

conda create -n gpt4all python=3.10

conda activate gpt4all

• Install the requirements for the project. You can use pip or conda to install the packages listed in the requirements.txt file. For example, using pip:

pip install -r requirements.txt

• Download and install cTransformers from GitHub. This is a library that provides fast inference for transformer-based language models on CPUs. You can use git to clone the repository or download the zip file and unzip it in a folder of your choice. For example:

git clone https://github.com/ctransformers/ctransformers.git

cd ctransformers

• Build and install cTransformers using cmake and make. You need to have cmake and a C++ compiler installed on your system. For example:

mkdir build

cd build

cmake ..

make -j8

make install

• Download a cTransformers model from the model zoo. You can choose from various models, such as GPT-NeoX-20B, GPT-J-6B, GPT-Q-6B, etc. You need to download the model file (.ggml) and the tokenizer file (.json). For example, to download GPT-J-6B:

wget https://ctransformers.s3.amazonaws.com/gpt-j-6b/gpt-j-6b.ggml

wget https://ctransformers.s3.amazonaws.com/gpt-j-6b/tokenizer.json

• Move the model and tokenizer files to the models folder of the gpt4all-ui project. You need to create a subfolder for each model and place the files there. For example, to move GPT-J-6B:

mkdir ~/gpt4all-ui/models/gpt-j-6b

mv gpt-j-6b.ggml ~/gpt4all-ui/models/gpt-j-6b/

mv tokenizer.json ~/gpt4all-ui/models/gpt-j-6b/

• Update the configuration file of the gpt4all-ui project. You need to edit the configs/default_local.yaml file and add the information about the cTransformers model you downloaded. You need to specify the name, category, language, binding, path, and personality of the model. For example, to add GPT-J-6B:

models:
  - name: "GPT-J 6B"
    category: "General"
    language: "English"
    binding: "ctransformers"
    path: "./models/gpt-j-6b/"
    personality: "default"

You can also change other settings in the configuration file, such as port, database, webui, etc.

• Run the web user interface of the gpt4all-ui project. You can use the webui.bat or webui.sh script depending on your platform. For example:

./webui.sh

This will launch a Flask web application that provides a chat UI for interacting with cTransformers models.

• Enjoy chatting with cTransformers models on your browser. You can access the web UI by going to http://localhost:5000/ (or another port if you changed it in the configuration file). You can select your preferred model and personality from the settings menu and type messages or questions in the message pane. You can also use other features of the web UI, such as rating, copying, editing, removing, searching, exporting, and deleting messages or discussions.

I hope this guide helps you set up gpt4all-ui and ctransformers together. If you have any questions or issues, you can check the README files of each repository or contact their developers on GitHub or Discord.

(1) ParisNeo/Gpt4All-webui: A web user interface for GPT4All - GitHub. https://github.com/ParisNeo/Gpt4All-webui
(2) How to use GPT4ALL — your own local chatbot — for free. https://www.digitaltrends.com/computing/how-to-use-gpt4all/

That's awesome @Lucas3DCG thank you!

And I'm sorry, I completely forgot to link a YT tutorial video made by ParisNeo : https://www.youtube.com/watch?v=ds_U0TDzbzI

I will link both this text tutorial and the YT video in my MPT readmes.

git clone https://github.com/ctransformers/ctransformers.git

should this be marella/ctransformers

cmake ..

should be cmake ../..

import gpt4all

Retrieve model

gptj = gpt4all.GPT4All(model_name="ggml-gpt4all-j-v1.3-groovy.bin")

I'm using this model to deploy over streamlit but facing below error

GGML_ASSERT: C: \Users \circleci.PACKER-64370BA5 \project \gpt4a11-backend\11ama. cpp-230511 \ggmI.c:4411: ctx-›mem buffer !=NULL

@TheBloke kindly help me