|
--- |
|
license: apache-2.0 |
|
base_model: nidum/Nidum-Llama-3.2-3B-Uncensored |
|
library_name: adapter-transformers |
|
tags: |
|
- chemistry |
|
- biology |
|
- legal |
|
- code |
|
- medical |
|
- finance |
|
- mlx |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
### **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit** |
|
|
|
### **Welcome to Nidum!** |
|
At Nidum, our mission is to bring cutting-edge AI capabilities to everyone with unrestricted access to innovation. With **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit**, you get an optimized, efficient, and versatile AI model for diverse applications. |
|
|
|
--- |
|
|
|
[](https://github.com/NidumAI-Inc) |
|
**Discover Nidum's Open-Source Projects on GitHub**: [https://github.com/NidumAI-Inc](https://github.com/NidumAI-Inc) |
|
|
|
--- |
|
|
|
### **Key Features** |
|
|
|
1. **Efficient and Compact**: Developed in **MLX-8bit format** for improved performance and reduced memory demands. |
|
2. **Wide Applicability**: Suitable for technical problem-solving, educational content, and conversational tasks. |
|
3. **Advanced Context Awareness**: Handles long-context conversations with exceptional coherence. |
|
4. **Streamlined Integration**: Optimized for use with the **mlx-lm library** for effortless development. |
|
5. **Unrestricted Responses**: Offers uncensored answers across all supported domains. |
|
|
|
--- |
|
|
|
### **How to Use** |
|
|
|
To use **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit**, install the **mlx-lm** library and follow these steps: |
|
|
|
#### **Installation** |
|
|
|
```bash |
|
pip install mlx-lm |
|
``` |
|
|
|
#### **Usage** |
|
|
|
```python |
|
from mlx_lm import load, generate |
|
|
|
# Load the model and tokenizer |
|
model, tokenizer = load("nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit") |
|
|
|
# Create a prompt |
|
prompt = "hello" |
|
|
|
# Apply the chat template if available |
|
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None: |
|
messages = [{"role": "user", "content": prompt}] |
|
prompt = tokenizer.apply_chat_template( |
|
messages, tokenize=False, add_generation_prompt=True |
|
) |
|
|
|
# Generate the response |
|
response = generate(model, tokenizer, prompt=prompt, verbose=True) |
|
|
|
# Print the response |
|
print(response) |
|
``` |
|
|
|
--- |
|
|
|
### **About the Model** |
|
|
|
The **nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit** model, converted using **mlx-lm version 0.19.2**, brings: |
|
|
|
- **Memory Efficiency**: Tailored for systems with limited hardware. |
|
- **Performance Optimization**: Matches the capabilities of the original model while delivering faster inference. |
|
- **Plug-and-Play**: Easily integrates with the **mlx-lm** library for deployment ease. |
|
|
|
--- |
|
|
|
### **Use Cases** |
|
|
|
- **Problem Solving in Tech and Science** |
|
- **Educational and Research Assistance** |
|
- **Creative Writing and Brainstorming** |
|
- **Extended Dialogues** |
|
- **Uninhibited Knowledge Exploration** |
|
|
|
--- |
|
|
|
### **Datasets and Fine-Tuning** |
|
|
|
Derived from **Nidum-Llama-3.2-3B-Uncensored**, the MLX-8bit version inherits: |
|
|
|
- **Uncensored Fine-Tuning**: Delivers detailed and open-ended responses. |
|
- **RAG-Based Optimization**: Enhances retrieval-augmented generation for data-driven tasks. |
|
- **Math Reasoning Support**: Precise mathematical computations and explanations. |
|
- **Long-Context Training**: Ensures relevance and coherence in extended conversations. |
|
|
|
--- |
|
|
|
### **Quantized Model Download** |
|
|
|
The **MLX-8bit** format strikes the perfect balance between memory optimization and performance. |
|
|
|
--- |
|
|
|
#### **Benchmark** |
|
|
|
| **Benchmark** | **Metric** | **LLaMA 3B** | **Nidum 3B** | **Observation** | |
|
|-------------------|-----------------------------------|--------------|--------------|-----------------------------------------------------------------------------------------------------| |
|
| **GPQA** | Exact Match (Flexible) | 0.3 | 0.5 | Nidum 3B achieves notable improvement in **generative tasks**. | |
|
| | Accuracy | 0.4 | 0.5 | Demonstrates strong performance, especially in **zero-shot** tasks. | |
|
| **HellaSwag** | Accuracy | 0.3 | 0.4 | Excels in **common-sense reasoning** tasks. | |
|
| | Normalized Accuracy | 0.3 | 0.4 | Strong contextual understanding in sentence completion tasks. | |
|
| | Normalized Accuracy (Stderr) | 0.15275 | 0.1633 | Enhanced consistency in normalized accuracy. | |
|
| | Accuracy (Stderr) | 0.15275 | 0.1633 | Demonstrates robustness in reasoning accuracy compared to LLaMA 3B. | |
|
|
|
--- |
|
|
|
### **Insights** |
|
|
|
1. **High Performance, Low Resource**: The MLX-8bit format is ideal for environments with limited memory and processing power. |
|
2. **Seamless Integration**: Designed for smooth integration into lightweight systems and workflows. |
|
|
|
--- |
|
|
|
### **Contributing** |
|
|
|
Join us in enhancing the **MLX-8bit** model's capabilities. Contact us for collaboration opportunities. |
|
|
|
--- |
|
|
|
### **Contact** |
|
|
|
For questions, support, or feedback, email **info@nidum.ai**. |
|
|
|
--- |
|
|
|
### **Experience the Future** |
|
|
|
Harness the power of **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit** for a perfect blend of performance and efficiency. |
|
|
|
--- |
