Instructions to use Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial")
model = AutoModelForCausalLM.from_pretrained("Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial

SGLang

How to use Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial with Docker Model Runner:
```
docker model run hf.co/Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Name: Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial - Mixture of Experts (MoE)

Description:

This is a cutting-edge Mixture of Experts (MoE) model designed with 24-bit precision, tailored to excel in four key domains: mathematics, coding, storytelling, and general chat. Built with a dynamic mixture of expert layers, this model adapts to different tasks by routing inputs to the most relevant expert network, delivering high-quality outputs efficiently.

Key Features

•	Mathematics Expert: Equipped with specialized mathematical reasoning capabilities, this model is fine-tuned for solving complex mathematical problems, numerical computations, and providing detailed explanations for mathematical concepts.
•	Coding Expert: The model has been trained extensively on various programming languages and software development paradigms. It can help generate, debug, and explain code snippets, offering a comprehensive coding support experience.
•	Storytelling Expert: Designed to assist in creative writing, this expert focuses on generating narratives, constructing dialogues, and offering story-building support for various genres.
•	General Chat Expert: Capable of engaging in everyday conversations, offering accurate and contextually appropriate responses. This expert is versatile and adaptive to different conversational tones, whether it’s casual chit-chat or formal assistance.

Technical Specifications

•	Model Architecture: Mixture of Experts (MoE) with a gating mechanism that routes inputs to the most relevant expert networks.
•	Domains:
•	Mathematics: Advanced reasoning and problem-solving.
•	Coding: Programming support across multiple languages.
•	Storytelling: Creative writing and narrative generation.
•	General Chat: Versatile dialogue handling for various conversational contexts.
•	Training Data: The model was trained on diverse datasets that cover each expert domain, ensuring robustness and versatility.
•	Framework: Developed using [Nom du Framework, par exemple: PyTorch, TensorFlow], optimized for the MoE architecture with gated routing.

Usage

This model can be used for a wide range of applications:

•	Educational Tools: Assisting with mathematical problems, coding exercises, and creative writing tasks.
•	Software Development: Providing coding suggestions, code completion, and debugging support.
•	Creative Writing: Generating stories, dialogues, and narrative content.
•	Conversational Agents: Implementing chatbots with versatile conversational abilities.

Limitations

•	The model may occasionally generate responses that are not entirely contextually appropriate, especially in cases requiring highly specialized domain knowledge.
•	Despite its 24-bit precision, it may not perform well with extremely large datasets or tasks that require higher precision levels.

Downloads last month: 9

Safetensors

Model size

24B params

Tensor type

BF16

Model tree for Marsouuu/MistralBase-4x7B-MoE-ECE-PRYMMAL-Martial

Base model

mistralai/Mistral-7B-v0.3

Finetuned

(384)

this model

Quantizations

2 models