Pi

pipwnz

AI & ML interests

None yet

Recent Activity

liked a Space 5 months ago

MeYourHint/MoMask

liked a model 5 months ago

Marqo/marqo-fashionSigLIP

liked a model 5 months ago

patrickjohncyh/fashion-clip

View all activity

Organizations

None yet

liked a Space 5 months ago

105

MoMask

🎭

Generate human motions from text prompts

liked 2 models 5 months ago

Marqo/marqo-fashionSigLIP

Zero-Shot Image Classification • 0.2B • Updated Oct 9, 2024 • 138k • 54

patrickjohncyh/fashion-clip

Zero-Shot Image Classification • 0.2B • Updated Sep 17, 2024 • 2.72M • 239

liked 2 models 6 months ago

deepvk/RuModernBERT-small

Fill-Mask • 0.0B • Updated Jun 9 • 8.96k • • 15

GoidaAlignment/KremlinAI-2

Text Generation • 3B • Updated Nov 19, 2024 • 4 • 8

liked 4 models 8 months ago

DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF

Text Generation • 18B • Updated Jul 28 • 83.8k • 333

replied to singhsidhukuldeep's post 9 months ago

Creates dense visual embeddings for each page while maintaining visual information integrity.

I'm sorry, but where can I read about dense visual embeddings? In this article I found only about the colpali strategy (similar to sparse)

reacted to singhsidhukuldeep's post with 🤗 9 months ago

Post

1321

Exciting breakthrough in Document AI! Researchers from UNC Chapel Hill and Bloomberg have developed M3DocRAG, a revolutionary framework for multi-modal document understanding.

The innovation lies in its ability to handle complex document scenarios that traditional systems struggle with:
- Process 40,000+ pages across 3,000+ documents
- Answer questions requiring information from multiple pages
- Understand visual elements like charts, tables, and figures
- Support both closed-domain (single document) and open-domain (multiple documents) queries

Under the hood, M3DocRAG operates through three sophisticated stages:

>> Document Embedding:
- Converts PDF pages to RGB images
- Uses ColPali to project both text queries and page images into a shared embedding space
- Creates dense visual embeddings for each page while maintaining visual information integrity

>> Page Retrieval:
- Employs MaxSim scoring to compute relevance between queries and pages
- Implements inverted file indexing (IVFFlat) for efficient search
- Reduces retrieval latency from 20s to under 2s when searching 40K+ pages
- Supports approximate nearest neighbor search via Faiss

>> Question Answering:
- Leverages Qwen2-VL 7B as the multi-modal language model
- Processes retrieved pages through a visual encoder
- Generates answers considering both textual and visual context

The results are impressive:
- State-of-the-art performance on MP-DocVQA benchmark
- Superior handling of non-text evidence compared to text-only systems
- Significantly better performance on multi-hop reasoning tasks

This is a game-changer for industries dealing with large document volumes—finance, healthcare, and legal sectors can now process documents more efficiently while preserving crucial visual context.