File size: 5,652 Bytes
805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 805ec06 38ef389 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 |
---
license: apache-2.0
base_model: nidum/Nidum-Llama-3.2-3B-Uncensored
library_name: adapter-transformers
tags:
- chemistry
- biology
- legal
- code
- medical
- finance
- mlx
pipeline_tag: text-generation
---
### **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit**
### **Welcome to Nidum!**
At Nidum, our mission is to bring cutting-edge AI capabilities to everyone with unrestricted access to innovation. With **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit**, you get an optimized, efficient, and versatile AI model for diverse applications.
---
[](https://github.com/NidumAI-Inc)
**Discover Nidum's Open-Source Projects on GitHub**: [https://github.com/NidumAI-Inc](https://github.com/NidumAI-Inc)
---
### **Key Features**
1. **Efficient and Compact**: Developed in **MLX-8bit format** for improved performance and reduced memory demands.
2. **Wide Applicability**: Suitable for technical problem-solving, educational content, and conversational tasks.
3. **Advanced Context Awareness**: Handles long-context conversations with exceptional coherence.
4. **Streamlined Integration**: Optimized for use with the **mlx-lm library** for effortless development.
5. **Unrestricted Responses**: Offers uncensored answers across all supported domains.
---
### **How to Use**
To use **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit**, install the **mlx-lm** library and follow these steps:
#### **Installation**
```bash
pip install mlx-lm
```
#### **Usage**
```python
from mlx_lm import load, generate
# Load the model and tokenizer
model, tokenizer = load("nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit")
# Create a prompt
prompt = "hello"
# Apply the chat template if available
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
# Generate the response
response = generate(model, tokenizer, prompt=prompt, verbose=True)
# Print the response
print(response)
```
---
### **About the Model**
The **nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit** model, converted using **mlx-lm version 0.19.2**, brings:
- **Memory Efficiency**: Tailored for systems with limited hardware.
- **Performance Optimization**: Matches the capabilities of the original model while delivering faster inference.
- **Plug-and-Play**: Easily integrates with the **mlx-lm** library for deployment ease.
---
### **Use Cases**
- **Problem Solving in Tech and Science**
- **Educational and Research Assistance**
- **Creative Writing and Brainstorming**
- **Extended Dialogues**
- **Uninhibited Knowledge Exploration**
---
### **Datasets and Fine-Tuning**
Derived from **Nidum-Llama-3.2-3B-Uncensored**, the MLX-8bit version inherits:
- **Uncensored Fine-Tuning**: Delivers detailed and open-ended responses.
- **RAG-Based Optimization**: Enhances retrieval-augmented generation for data-driven tasks.
- **Math Reasoning Support**: Precise mathematical computations and explanations.
- **Long-Context Training**: Ensures relevance and coherence in extended conversations.
---
### **Quantized Model Download**
The **MLX-8bit** format strikes the perfect balance between memory optimization and performance.
---
#### **Benchmark**
| **Benchmark** | **Metric** | **LLaMA 3B** | **Nidum 3B** | **Observation** |
|-------------------|-----------------------------------|--------------|--------------|-----------------------------------------------------------------------------------------------------|
| **GPQA** | Exact Match (Flexible) | 0.3 | 0.5 | Nidum 3B achieves notable improvement in **generative tasks**. |
| | Accuracy | 0.4 | 0.5 | Demonstrates strong performance, especially in **zero-shot** tasks. |
| **HellaSwag** | Accuracy | 0.3 | 0.4 | Excels in **common-sense reasoning** tasks. |
| | Normalized Accuracy | 0.3 | 0.4 | Strong contextual understanding in sentence completion tasks. |
| | Normalized Accuracy (Stderr) | 0.15275 | 0.1633 | Enhanced consistency in normalized accuracy. |
| | Accuracy (Stderr) | 0.15275 | 0.1633 | Demonstrates robustness in reasoning accuracy compared to LLaMA 3B. |
---
### **Insights**
1. **High Performance, Low Resource**: The MLX-8bit format is ideal for environments with limited memory and processing power.
2. **Seamless Integration**: Designed for smooth integration into lightweight systems and workflows.
---
### **Contributing**
Join us in enhancing the **MLX-8bit** model's capabilities. Contact us for collaboration opportunities.
---
### **Contact**
For questions, support, or feedback, email **info@nidum.ai**.
---
### **Experience the Future**
Harness the power of **Nidum-Llama-3.2-3B-Uncensored-MLX-8bit** for a perfect blend of performance and efficiency.
--- |