Hugging Face's logo

nidum
/
Nidum-Llama-3.2-3B-Uncensored-MLX-8bit

Text Generation
Adapters
Safetensors
MLX
llama
chemistry
biology
legal
code
medical
finance
conversational
Model card Files Community
1
Nidum-Llama-3.2-3B-Uncensored-MLX-8bit / README.md
rogkesavan's picture
rogkesavan
Update README.md
38ef389 verified
preview code
|
raw
history blame contribute delete
5.65 kB
metadata
license: apache-2.0
base_model: nidum/Nidum-Llama-3.2-3B-Uncensored
library_name: adapter-transformers
tags:
  - chemistry
  - biology
  - legal
  - code
  - medical
  - finance
  - mlx
pipeline_tag: text-generation

Nidum-Llama-3.2-3B-Uncensored-MLX-8bit

Welcome to Nidum!

At Nidum, our mission is to bring cutting-edge AI capabilities to everyone with unrestricted access to innovation. With Nidum-Llama-3.2-3B-Uncensored-MLX-8bit, you get an optimized, efficient, and versatile AI model for diverse applications.

GitHub Icon
Discover Nidum's Open-Source Projects on GitHub: https://github.com/NidumAI-Inc

Key Features

  1. Efficient and Compact: Developed in MLX-8bit format for improved performance and reduced memory demands.
  2. Wide Applicability: Suitable for technical problem-solving, educational content, and conversational tasks.
  3. Advanced Context Awareness: Handles long-context conversations with exceptional coherence.
  4. Streamlined Integration: Optimized for use with the mlx-lm library for effortless development.
  5. Unrestricted Responses: Offers uncensored answers across all supported domains.

How to Use

To use Nidum-Llama-3.2-3B-Uncensored-MLX-8bit, install the mlx-lm library and follow these steps:

Installation 

pip install mlx-lm

Usage 

from mlx_lm import load, generate

# Load the model and tokenizer
model, tokenizer = load("nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit")

# Create a prompt
prompt = "hello"

# Apply the chat template if available
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

# Generate the response
response = generate(model, tokenizer, prompt=prompt, verbose=True)

# Print the response
print(response)

About the Model

The nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-8bit model, converted using mlx-lm version 0.19.2, brings:

  • Memory Efficiency: Tailored for systems with limited hardware.
  • Performance Optimization: Matches the capabilities of the original model while delivering faster inference.
  • Plug-and-Play: Easily integrates with the mlx-lm library for deployment ease.

Use Cases

  • Problem Solving in Tech and Science
  • Educational and Research Assistance
  • Creative Writing and Brainstorming
  • Extended Dialogues
  • Uninhibited Knowledge Exploration

Datasets and Fine-Tuning

Derived from Nidum-Llama-3.2-3B-Uncensored, the MLX-8bit version inherits:

  • Uncensored Fine-Tuning: Delivers detailed and open-ended responses.
  • RAG-Based Optimization: Enhances retrieval-augmented generation for data-driven tasks.
  • Math Reasoning Support: Precise mathematical computations and explanations.
  • Long-Context Training: Ensures relevance and coherence in extended conversations.

Quantized Model Download

The MLX-8bit format strikes the perfect balance between memory optimization and performance.

Benchmark

Benchmark Metric LLaMA 3B Nidum 3B Observation
GPQA Exact Match (Flexible) 0.3 0.5 Nidum 3B achieves notable improvement in generative tasks.
Accuracy 0.4 0.5 Demonstrates strong performance, especially in zero-shot tasks.
HellaSwag Accuracy 0.3 0.4 Excels in common-sense reasoning tasks.
Normalized Accuracy 0.3 0.4 Strong contextual understanding in sentence completion tasks.
Normalized Accuracy (Stderr) 0.15275 0.1633 Enhanced consistency in normalized accuracy.
Accuracy (Stderr) 0.15275 0.1633 Demonstrates robustness in reasoning accuracy compared to LLaMA 3B.

Insights

  1. High Performance, Low Resource: The MLX-8bit format is ideal for environments with limited memory and processing power.
  2. Seamless Integration: Designed for smooth integration into lightweight systems and workflows.

Contributing

Join us in enhancing the MLX-8bit model's capabilities. Contact us for collaboration opportunities.

Contact

For questions, support, or feedback, email info@nidum.ai.

Experience the Future

Harness the power of Nidum-Llama-3.2-3B-Uncensored-MLX-8bit for a perfect blend of performance and efficiency.