Papers
arxiv:2605.08735

CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models

Published on May 9
· Submitted by
taesiri
on May 12
#3 Paper of the day
Authors:
,

Abstract

A closed-loop framework collaboratively integrates vision-language models with video generation models at step level to improve visual reasoning by enabling real-time failure detection and correction during video generation.

AI-generated summary

Recent "Thinking with Video" approaches use Video Generation Models (VGMs) for visual reasoning by producing temporally coherent Chain-of-Frames as reasoning artifacts. Even strong VGMs, however, exhibit two recurring failure modes on goal-directed tasks: long-horizon drift on multi-step tasks and mid-clip simulation errors that compound. Both stem from the absence of explicit reasoning built upon the VGM's short-horizon visual prior, a role naturally filled by Vision-Language Models (VLMs), but where to place the VLM is non-trivial: upfront plans commit before any frame is generated and post-hoc critiques over whole videos intervene too late. We propose VLM-VGM Collaborative Video Reasoning (CollabVR), a closed-loop framework that couples the VLM with the VGM at step-level granularity: the VLM plans the immediate next action, inspects the clip the VGM generates, and folds the verifier's diagnosis directly into the next action prompt to repair detected failures. On Gen-ViRe and VBVR-Bench, CollabVR improves both open-source and closed-source VGMs over single-inference, Pass@k, and prior test-time scaling baselines at matched compute, with the largest gains on the hardest tasks. It also yields further improvements on top of a reasoning-fine-tuned VGM, indicating that step-level VLM supervision is orthogonal to and stackable with reasoning-oriented fine-tuning. We provide video samples and additional qualitative results at our project page: https://joow0n-kim.github.io/collabvr-project-page.

Community

this collabvr loop feels like a plan-and-revise setup for video reasoning, with a VLM-led planner and a verifier steering the next action at every step. i’d love to see them borrow self-consistency ideas from chain-of-thought work, maybe running multiple verifier diagnoses and using a consensus to stabilize the next prompt rather than trusting a single revision. the arxivlens breakdown helped me parse the per-step prompt evolution (https://arxivlens.com/PaperView/Details/collabvr-collaborative-video-reasoning-with-vision-language-and-video-generation-models-3467-c1a244a4), and i think a lightweight ensemble of verifications could cut drift even more on long-horizon tasks. it would also be cool to connect this with decision-transformer style planning, treating each generated frame as part of a learnable trajectory that the planner can reuse across tasks.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2605.08735 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2605.08735 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2605.08735 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.