|
--- |
|
title: Mistral 7b Pathway 128k 3 |
|
emoji: 🏃 |
|
colorFrom: purple |
|
colorTo: gray |
|
sdk: gradio |
|
sdk_version: 4.4.1 |
|
app_file: app.py |
|
pinned: false |
|
license: apache-2.0 |
|
--- |
|
|
|
# Pseudo-Deterministic Chatbot with Mistral 7B |
|
|
|
## Overview |
|
This repository contains a fine-tuned version of the Mistral 7B model, specifically designed for creating pseudo-deterministic chatbots. The goal of this project is to enhance the predictability and consistency of chatbot responses while maintaining the flexibility and adaptability of the Mistral 7B model. |
|
|
|
## Features |
|
- **Fine-tuned Mistral 7B Model**: Leveraging the power of the Mistral 7B, our model is fine-tuned to offer more deterministic responses, ensuring consistency in conversational contexts. |
|
- **Scalable Hugging Face Endpoint**: We provide a handler script for deploying the chatbot model on a scalable endpoint using Hugging Face's infrastructure. This setup is ideal for handling varying loads with efficient resource management. This can be deployed for public, protected, or private use with ASW privatelink. This handler script can also be used to serve the model on custom hardware. |
|
- **Gradio Interface**: A Gradio demo is included, offering a user-friendly interface to interact with the chatbot. This demo can connect not only to our provided backend but also to any alternative backend setup. |
|
|
|
## Getting Started |
|
|
|
1. **Deploying the Model**: |
|
You can deploy from the model repo ([here](https://huggingface.co/codys12/Mistral-7b-Pathway-128k-3/tree/main)) by clicking "Deploy" in the upper right corner. |
|
or with [Inference Endpoints SDK](https://huggingface.co/docs/inference-endpoints/index) |
|
|
|
3. **Running the Gradio Demo**: |
|
You can deploy directly from hugginface or |
|
*with Python:* |
|
```python |
|
import gradio as gr |
|
|
|
gr.load("models/codys12/Mistral-7b-Pathway-128k-3").launch() |
|
``` |
|
You can embed the space with the URL found in the upper right of the space with "Share" |
|
```javascript |
|
<iframe |
|
src="https://your.hf.space" |
|
frameborder="0" |
|
width="850" |
|
height="450" |
|
></iframe> |
|
``` |
|
|
|
## Usage |
|
- **Model Interaction**: |
|
```python |
|
def generate( |
|
message: str, |
|
chat_history: list[tuple[str, str]],#Conversation history |
|
system_prompt: str = "", |
|
instruction: str = None,#The goal of the current conversation |
|
conclusions: list[tuple[str, str]] = None,#AI classification of conversation ending |
|
#^ Formatted: [["CONCLUSION_KEY","Conclusion criteria"]] |
|
context: list[str] = None,#List of strings to be used as context. Indexes that were used will be returned. |
|
max_new_tokens: int = 1024,#Max new tokens to generate |
|
temperature: float = 0.6,#Temperature hyperparameter |
|
top_p: float = 0.9,#Top-p hyperparameter |
|
top_k: int = 50, #Top-k hyperparameter |
|
repetition_penalty: float = 1.2, #Repitition hyperparameter |
|
end_sequences: list[str] = ["[INST]", "[/INST]", "\n"]#Sequences that break the generation and return |
|
``` |
|
- **Customization**: conversation topics and their possible ansers/paths are stored in topics.json. You can freely change this to fit a desired use case. |
|
|
|
## License |
|
This project is licensed under the Apache License, Version 2.0 - see the `LICENSE` file for details. |
|
|
