RS-Painter-Diffusers

RS-Painter-Diffusers

Tackling Few-Shot Segmentation in Remote Sensing via Inpainting Diffusion Model

ICLR Machine Learning for Remote Sensing Workshop, 2025 (Best Paper Award)

• Original Paper: arXiv:2503.03785
• Project Website: https://steveimmanuel.github.io/rs-paint
• Original Repository: https://huggingface.co/SteveImmanuel/RSPaint

Model Description

RS-Painter is an image-conditioned diffusion-based approach for creating diverse sets of novel-class samples for semantic segmentation in few-shot settings in the remote sensing domain. By ensuring semantic consistency using cosine similarity between the generated samples and the conditioning image, and using the Segment Anything Model (SAM) to obtain precise segmentation, this method can train off-the-shelf segmentation models with high-quality synthetic data, significantly improving performance in low-data scenarios.

This model is compatible with the Hugging Face diffusers library and can be used with StableDiffusionInpaintPipeline.

Quick Start

from diffusers import StableDiffusionInpaintPipeline
from PIL import Image

# Load pipeline
pipeline = StableDiffusionInpaintPipeline.from_pretrained(
    "BiliSakura/RS-Painter-Diffusers",
    safety_checker=None,
    requires_safety_checker=False,
)

# Load image and mask
image = Image.open("input_image.png").convert("RGB")
mask = Image.open("mask.png").convert("L")

# Generate
result = pipeline(
    prompt="a beautiful landscape",
    image=image,
    mask_image=mask,
    num_inference_steps=50,
)

# Save result
result.images[0].save("output.png")

Citation

If you use this model in your research, please cite:

@article{2025rspaint,
  title={Tackling Few-Shot Segmentation in Remote Sensing via Inpainting Diffusion Model},
  author={Immanuel, Steve Andreas and Cho, Woojin and Heo, Junhyuk and Kwon, Darongsae},
  journal={arXiv preprint arXiv:2503.03785},
  year={2025}
}