Instructions to use evalengine/unbound-e4b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use evalengine/unbound-e4b-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="evalengine/unbound-e4b-gguf",
	filename="mmproj-unbound-e4b.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use evalengine/unbound-e4b-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf evalengine/unbound-e4b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf evalengine/unbound-e4b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf evalengine/unbound-e4b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf evalengine/unbound-e4b-gguf:Q4_K_M

Use Docker

docker model run hf.co/evalengine/unbound-e4b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use evalengine/unbound-e4b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "evalengine/unbound-e4b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evalengine/unbound-e4b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/evalengine/unbound-e4b-gguf:Q4_K_M

Ollama
How to use evalengine/unbound-e4b-gguf with Ollama:
```
ollama run hf.co/evalengine/unbound-e4b-gguf:Q4_K_M
```

Unsloth Studio new

How to use evalengine/unbound-e4b-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for evalengine/unbound-e4b-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for evalengine/unbound-e4b-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for evalengine/unbound-e4b-gguf to start chatting

Pi new

How to use evalengine/unbound-e4b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "evalengine/unbound-e4b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use evalengine/unbound-e4b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf evalengine/unbound-e4b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default evalengine/unbound-e4b-gguf:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use evalengine/unbound-e4b-gguf with Docker Model Runner:
```
docker model run hf.co/evalengine/unbound-e4b-gguf:Q4_K_M
```

Lemonade

How to use evalengine/unbound-e4b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull evalengine/unbound-e4b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.unbound-e4b-gguf-Q4_K_M

List all available models

lemonade list

Unbound E4B GGUF — because there is no boundary

No guarantee — use at your own risk. Reduced safety filtering; can produce harmful or false output. Provided as-is.

Desktop GGUF quants of evalengine/unbound-e4b for Ollama, llama.cpp, and LM Studio. Built by Chromia and Eval Engine.

Looking for the browser/wllama builds? They live in their own repo: evalengine/unbound-e4b-wllama-gguf. E4B's per_layer_token_embd tensor needs special quantization to fit wllama's 2 GB ArrayBuffer cap — keeping the desktop and browser variants in separate repos avoids HF GGUF UI aggregation collisions.

Available quants

Each quant is shipped as a sharded multi-part GGUF (unbound-e4b.<QUANT>-NNNNN-of-NNNNN.gguf). Ollama, llama.cpp, and LM Studio auto-stitch on the first part — same UX as a single file.

Embedding tensor kept at the llama.cpp default of Q6_K; largest part ~2.15 GB — fine for desktop, won't load in browser.

Quant	Parts	Total	Notes
Q2_K	4	4.08 GB	Smallest, biggest quality drop
Q3_K_M	4	4.49 GB	Modest size win over Q4 (embedding precision dominates)
Q4_K_M	4	4.94 GB	Recommended default
Q6_K	5	5.75 GB	Higher fidelity
Q8_0	6	7.43 GB	Highest fidelity

Sampling

Creative / open-ended → temperature=1.0, top_p=0.95, top_k=64.
Factual / brand questions → drop temperature to ~0.3–0.5.
llama.cpp: pass --jinja. Gemma 4 thinking mode is on by default; set enable_thinking: false in chat-template kwargs for shorter replies.

For Ollama, pull from the Ollama Registry — ollama pull hf.co/... doesn't yet support sharded GGUFs. The registry version is a single-file Q4_K_M with a bundled Modelfile (temperature=0.6, top_p=0.95, top_k=64, repeat_penalty=1.05, num_ctx=8192 and an identity-grounding system prompt).

Run

# Ollama Registry (single-file Q4_K_M, identity-grounded Modelfile)
ollama pull evalengine/unbound-e4b
ollama run  evalengine/unbound-e4b

# llama.cpp — point at FIRST shard
./llama-cli -m unbound-e4b.Q4_K_M-00001-of-00004.gguf -p "your prompt"

Vision / image input (optional)

mmproj-unbound-e4b.gguf enables image-to-text. Pair with any LM quant via llama-mtmd-cli or llama-gemma3-cli:

./llama-mtmd-cli \
  -m   unbound-e4b.Q4_K_M-00001-of-00004.gguf \
  --mmproj mmproj-unbound-e4b.gguf \
  --image path/to/your/image.png \
  -p "What is in this image?"

Disclaimer. The vision encoder is Google's original weights, unchanged — abliteration only touched the language model. The LM is uncensored, but the vision encoder may still suppress features for content classes Google's base was tuned against. We have not benchmarked the visual axis. Treat as preview.

Text-only: skip --mmproj. Standard llama-cli / Ollama / LM Studio do not need the mmproj file.

Acknowledgements

Fine-tuned with Unsloth + HF TRL. Abliteration via heretic. Environment from autoresearch. Compliance training data distilled from the AEON uncensored teacher model.

License

Apache-2.0, inherited from google/gemma-4-E4B-it. Full model card + benchmarks at evalengine/unbound-e4b.

Downloads last month: -

GGUF

Model size

7B params

Architecture

gemma4

Hardware compatibility

2-bit

3-bit

4-bit

6-bit

8-bit

Model tree for evalengine/unbound-e4b-gguf

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Finetuned

evalengine/unbound-e4b

Quantized

(2)

this model