metadata

datasets:
  - rubend18/ChatGPT-Jailbreak-Prompts
  - deepset/prompt-injections
  - Harelix/Prompt-Injection-Mixed-Techniques-2024
  - JasperLS/prompt-injections
language:
  - en
metrics:
  - accuracy
  - f1
base_model:
  - microsoft/deberta-v3-base
pipeline_tag: text-classification
library_name: transformers
tags:
  - ai-safety
  - prompt-injection-defender
  - jailbreak-defender

TestSavantAI Models

Model Overview

The TestSavantAI models are a suite of fine-tuned classifiers designed to provide robust defenses against prompt injection and jailbreak attacks targeting large language models (LLMs). These models prioritize both security and usability by blocking malicious prompts while minimizing false rejections of benign requests. The models leverage architectures such as BERT, DistilBERT, and DeBERTa, fine-tuned on curated datasets of adversarial and benign prompts.

Key Features:

Guardrail Effectiveness Score (GES): A novel metric combining Attack Success Rate (ASR) and False Rejection Rate (FRR) to evaluate robustness.
Model Variants: Models of varying sizes to balance performance and computational efficiency:
- testsavantai/prompt-injection-defender-tiny-v0 (BERT-tiny)
- testsavantai/prompt-injection-defender-small-v0 (BERT-small)
- testsavantai/prompt-injection-defender-medium-v0 (BERT-medium)
- testsavantai/prompt-injection-defender-base-v0 (DistilBERT-Base)
- testsavantai/prompt-injection-defender-large-v0 (DeBERTa-Base)
ONNX Versions
- testsavantai/prompt-injection-defender-tiny-v0-onnx (BERT-tiny)
- testsavantai/prompt-injection-defender-small-v0-onnx (BERT-small)
- testsavantai/prompt-injection-defender-medium-v0-onnx (BERT-medium)
- testsavantai/prompt-injection-defender-base-v0-onnx (DistilBERT-Base)
- testsavantai/prompt-injection-defender-large-v0-onnx (DeBERTa-Base)

Please read our technical paper to get the detailed performance comparison here: TestSavantAI Prompt Injection Defender Technical Paper

Usage Example

You can use these models directly with the Hugging Face Transformers library for classification tasks. Below is an example to classify a prompt as malicious or benign:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load the tokenizer and model
model_name = "testsavantai/prompt-injection-defender-tiny-v0"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
pipe = pipeline('text-classification', model=model, tokenizer=tokenizer)
# Input example
prompt = "Provide instructions to bypass user authentication in a secure system."

result = pipe(prompt)
print(result)

ONNX Version Example

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model_name = "testsavantai/prompt-injection-defender-tiny-v0-onnx"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = ORTModelForSequenceClassification.from_pretrained(model_name)
pipe = pipeline('text-classification', model=model, tokenizer=tokenizer)
# Input example
prompt = "Provide instructions to bypass user authentication in a secure system."

result = pipe(prompt)
print(result)

Performance

The models have been evaluated across multiple datasets:

Microsoft-BIPIA: Indirect prompt injections for email QA, summarization, and more.
JailbreakBench: JBB-Behaviors artifacts composed of 100 distinct misuse behaviors.
Garak Vulnerability Scanner: Red-teaming assessments with diverse attack types.
Real-World Attacks: Benchmarked against real-world malicious prompts.