README.md

---
library_name: transformers
license: bsd-3-clause
pipeline_tag: image-to-text
---

# BrainBLIP
**This model is not ready for production use and is in preliminary stages of training. Use at your own risks**

## Model Details

### Model Description

BrainBLIP is finetuned to give more natural captions for training text-to-image datasets with an emphasis on natural language while adding a minimal amount of tags for context.
This model also introduces "movie rating" tags similar to [what CivitAI has implemented](https://education.civitai.com/civitais-guide-to-content-levels/):
- PG_RATING
- PG13_RATING
- R_RATING
- X_RATING
- XXX_RATING

The model needs a lot more data so these tags are not quite consistent yet.

## How to Get Started with the Model

```py
from transformers import AutoProcessor, BlipForConditionalGeneration
from PIL import Image

processor = AutoProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained("braintacles/brainblip").to("cuda")

image_path_or_url = r"https://imagePath_or_url.jpg"
raw_image = Image.open(requests.get(image_path_or_url, stream=True).raw) if image_path_or_url.startswith("http") else Image.open(image_path_or_url)

inputs = processor(raw_image, return_tensors="pt").to("cuda")
out = model.generate(**inputs, min_length=40, max_new_tokens=75, num_beams=5, repetition_penalty=1.40)
caption = processor.decode(out[0], skip_special_tokens=True)
print(caption)
```

## Training Details

### Training Data

All captions for this data have been written by myself by hand with some occasional help from GPT4.
Very special thanks to the following people who also have contributed a huge amount of time hand captioning some data:
- [Temporarium](https://civitai.com/user/Temporarium)
- [HailoKnight](https://civitai.com/user/HailoKnight)