Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForImageTextToText.from_pretrained("HuggingFaceM4/idefics-80b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

idefics-80b

Commit History

cleanup

02c5697

stas commited on Jul 26, 2023

Update config.json (#4)

92a6c29

stas

ybelkada commited on Jul 26, 2023

Update config.json (#3)

0c44ec5

stas commited on Jul 25, 2023

Update README.md

4a3d381

VictorSanh commited on Jul 25, 2023

preprocessor_config

5215c66

stas commited on Jul 19, 2023

Switch to IDEFICS/ OBELICS

62de87a

Leyo commited on Jul 13, 2023

fix significant digits in table

e23b75c

Leyo commited on Jul 13, 2023

mention the useful tiny model

de16541

stas commited on Jul 12, 2023

rename

45dd546

stas commited on Jul 12, 2023

rename

a3796c6

stas commited on Jul 12, 2023

add missing metadata

8ffb014

stas commited on Jul 12, 2023

winoground fix

6edaaa6

Leyo commited on Jul 12, 2023

IDEFICS

51b1776

Leyo commited on Jul 12, 2023

take off remaining -1

c17962a

Leyo commited on Jul 12, 2023

fix table results rounding

70fea44

Leyo commited on Jul 12, 2023

switch to OE VQA acc

5cc3ed2

Leyo commited on Jul 12, 2023

fix winoground numbers

0b721e1

Leyo commited on Jul 12, 2023

Update result table

052d671

Leyo commited on Jul 12, 2023

Update README.md

4c02d51

HugoLaurencon commited on Jul 12, 2023

add eval tables

7d53dc8

Leyo commited on Jul 12, 2023

new

0940a33

stas commited on Jul 11, 2023

new

6c74def

stas commited on Jul 11, 2023

Update README.md

d3f1708

HugoLaurencon commited on Jul 11, 2023

Update README.md

a93d027

HugoLaurencon commited on Jul 11, 2023

Update README.md

25ff6d2

HugoLaurencon commited on Jul 11, 2023

Update README.md

a8977b9

HugoLaurencon commited on Jul 11, 2023

fix source num tokens

fa62bff

Leyo commited on Jul 11, 2023

add num tokens source

73fda5b

Leyo commited on Jul 11, 2023

fix proportion numbers

436c345

Leyo commited on Jul 11, 2023

Update README.md

2f0f4fc

HugoLaurencon commited on Jul 11, 2023

Update README.md

8683d64

HugoLaurencon commited on Jul 11, 2023

Update README.md

ade3656

HugoLaurencon commited on Jul 11, 2023

vs/path_2 (#2)

de1ddb0

VictorSanh commited on Jul 10, 2023

Fix table layout

08e31d6

Leyo commited on Jul 10, 2023

first pass over the model card (#1)

66a38fb

stas

VictorSanh commited on Jul 7, 2023

metadata

d85c074

stas commited on Jul 6, 2023

metadata

f070274

stas commited on Jul 6, 2023

wip

61f2a7d

stas commited on Jul 6, 2023

start

c7e377b

stas commited on Jul 6, 2023

start

382a0e6

stas commited on Jul 6, 2023

initial commit

19b8bfc

Stas Bekman commited on Jul 5, 2023

Commit History

cleanup 02c5697

Update config.json (#4) 92a6c29

Update config.json (#3) 0c44ec5

Update README.md 4a3d381

preprocessor_config 5215c66

Switch to IDEFICS/ OBELICS 62de87a

fix significant digits in table e23b75c

mention the useful tiny model de16541

rename 45dd546

rename a3796c6

add missing metadata 8ffb014

winoground fix 6edaaa6

IDEFICS 51b1776

take off remaining -1 c17962a

fix table results rounding 70fea44

switch to OE VQA acc 5cc3ed2

fix winoground numbers 0b721e1

Update result table 052d671

Update README.md 4c02d51

add eval tables 7d53dc8

new 0940a33

new 6c74def

Update README.md d3f1708

Update README.md a93d027

Update README.md 25ff6d2

Update README.md a8977b9

fix source num tokens fa62bff

add num tokens source 73fda5b

fix proportion numbers 436c345

Update README.md 2f0f4fc

Update README.md 8683d64

Update README.md ade3656

vs/path_2 (#2) de1ddb0

Fix table layout 08e31d6

first pass over the model card (#1) 66a38fb

metadata d85c074

metadata f070274

wip 61f2a7d

start c7e377b

start 382a0e6

initial commit 19b8bfc

cleanup

02c5697

Update config.json (#4)

92a6c29

Update config.json (#3)

0c44ec5

Update README.md

4a3d381

preprocessor_config

5215c66

Switch to IDEFICS/ OBELICS

62de87a

fix significant digits in table

e23b75c

mention the useful tiny model

de16541

rename

45dd546

rename

a3796c6

add missing metadata

8ffb014

winoground fix

6edaaa6

IDEFICS

51b1776

take off remaining -1

c17962a

fix table results rounding

70fea44

switch to OE VQA acc

5cc3ed2

fix winoground numbers

0b721e1

Update result table

052d671

Update README.md

4c02d51

add eval tables

7d53dc8

new

0940a33

new

6c74def

Update README.md

d3f1708

Update README.md

a93d027

Update README.md

25ff6d2

Update README.md

a8977b9

fix source num tokens

fa62bff

add num tokens source

73fda5b

fix proportion numbers

436c345

Update README.md

2f0f4fc

Update README.md

8683d64

Update README.md

ade3656

vs/path_2 (#2)

de1ddb0

Fix table layout

08e31d6

first pass over the model card (#1)

66a38fb

metadata

d85c074

metadata

f070274

wip

61f2a7d

start

c7e377b

start

382a0e6

initial commit

19b8bfc