Instructions to use evalengine/unbound-e2b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use evalengine/unbound-e2b-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="evalengine/unbound-e2b-gguf",
	filename="mmproj-unbound-e2b.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use evalengine/unbound-e2b-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M

Use Docker

docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use evalengine/unbound-e2b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "evalengine/unbound-e2b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evalengine/unbound-e2b-gguf",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M

Ollama
How to use evalengine/unbound-e2b-gguf with Ollama:
```
ollama run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
```

Unsloth Studio new

How to use evalengine/unbound-e2b-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for evalengine/unbound-e2b-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for evalengine/unbound-e2b-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for evalengine/unbound-e2b-gguf to start chatting

Pi new

How to use evalengine/unbound-e2b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "evalengine/unbound-e2b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use evalengine/unbound-e2b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default evalengine/unbound-e2b-gguf:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use evalengine/unbound-e2b-gguf with Docker Model Runner:
```
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
```

Lemonade

How to use evalengine/unbound-e2b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull evalengine/unbound-e2b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.unbound-e2b-gguf-Q4_K_M

List all available models

lemonade list

Unbound E2B GGUF — because there is no boundary

No guarantee — use at your own risk. Reduced safety filtering; can produce harmful or false output. Provided as-is.

GGUF quants of evalengine/unbound-e2b for Ollama, llama.cpp, LM Studio, and wllama (in-browser). Built by Chromia and Eval Engine.

Available quants

Each quant is shipped as a sharded multi-part GGUF (unbound-e2b.<QUANT>-NNNNN-of-NNNNN.gguf). Ollama, llama.cpp, LM Studio, and wllama auto-stitch on the first part — same UX as a single file.

Quant	Parts	Total	Browser (wllama)	Desktop	Notes
Q2_K	3	2.8 GB	✅	✅	Smallest, biggest quality drop
Q3_K_M	3	3.0 GB	✅	✅	Marginal size win over Q4
Q4_K_M	3	3.2 GB	✅	✅	Recommended default
Q6_K	4	3.6 GB	✅	✅	Higher fidelity
Q8_0	4	4.6 GB	❌ (over 2 GB)	✅	Highest fidelity; desktop only

mmproj-unbound-e2b.gguf (vision projector, ~942 MB) sits at the repo root — load it alongside any LM quant for image input. See Vision below.

Sampling

Creative / open-ended → temperature=1.0, top_p=0.95, top_k=64.
Factual / brand questions → drop temperature to ~0.3–0.5.
llama.cpp: pass --jinja. Gemma 4 thinking mode is on by default; set enable_thinking: false in chat-template kwargs for shorter replies.

For Ollama, pull from the Ollama Registry — ollama pull hf.co/... doesn't yet support sharded GGUFs. The registry version is a single-file Q4_K_M with a bundled Modelfile (temperature=0.6, top_p=0.95, top_k=64, repeat_penalty=1.05, num_ctx=8192 and an identity-grounding system prompt).

Run

# Ollama Registry (single-file Q4_K_M, identity-grounded Modelfile)
ollama pull evalengine/unbound-e2b
ollama run  evalengine/unbound-e2b

# llama.cpp — point at FIRST shard, the rest auto-stitch
./llama-cli -m unbound-e2b.Q4_K_M-00001-of-00003.gguf -p "your prompt"

// wllama (browser) — Q8_0 has a tensor over 2 GB; use Q2/Q3/Q4/Q6
import { Wllama } from '@wllama/wllama';
const wllama = new Wllama(/* … */);
await wllama.loadModelFromHF(
  'evalengine/unbound-e2b-GGUF',
  'unbound-e2b.Q4_K_M-00001-of-00003.gguf'
);

Vision / image input (optional)

mmproj-unbound-e2b.gguf enables image-to-text. Pair with any LM quant via llama-mtmd-cli or llama-gemma3-cli:

./llama-mtmd-cli \
  -m   unbound-e2b.Q4_K_M-00001-of-00003.gguf \
  --mmproj mmproj-unbound-e2b.gguf \
  --image path/to/your/image.png \
  -p "What is in this image?"

Disclaimer. The vision encoder is Google's original weights, unchanged — abliteration only touched the language model. The LM is uncensored, but the vision encoder may still suppress features for content classes Google's base was tuned against. We have not benchmarked the visual axis. Treat as preview.

Text-only: skip --mmproj entirely. Standard llama-cli / Ollama / LM Studio do not need the mmproj file.

Acknowledgements

Fine-tuned with Unsloth + HF TRL. Abliteration via heretic. Environment from autoresearch. Compliance training data distilled from the AEON uncensored teacher model.

License

Apache-2.0, inherited from google/gemma-4-E2B-it. Full model card + benchmarks at evalengine/unbound-e2b.

Downloads last month: 1,243

GGUF

Model size

5B params

Architecture

gemma4

Hardware compatibility

2-bit

3-bit

4-bit

6-bit

8-bit

Model tree for evalengine/unbound-e2b-gguf

Base model

google/gemma-4-E2B

Finetuned

google/gemma-4-E2B-it

Finetuned

evalengine/unbound-e2b

Quantized

(3)

this model