---
license: cc-by-nc-4.0
---
MiniGPT-4 checkpoint aligned with @panopstor's FF7R dataset (link in the EveryDream discord). Produces captions that are more useful for training SD datasets that MiniGPT4's default output. 

Easiest way to use this is to launch a docker instance for [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui), eg `TheBloke/runpod-pytorch-runclick`, follow the instructions for MiniGPT-4 [here](https://github.com/oobabooga/text-generation-webui/tree/main/extensions/multimodal). For now you'll need to manually edit `minigpt_pipeline.py` ([this line](`https://github.com/Wojtab/minigpt-4-pipeline/blob/16eda85c4bb15e2b1b05b20c55907a8ea2c06764/minigpt4_pipeline.py#L52) to point to [the .pth file in this repo](minigpt4-align-ff7r.pth) instead of the default.

## Dataset

adapted from the @panopstor's FF7R dataset - [zip here](cc_sbu_align_ff7r.zip)

## Sample output:

![](examples/1.png)
![](examples/2.png)
![](examples/3.png)