Hugging Face's logo

meta-llama
/
Llama-3.1-405B-FP8

Text Generation
Transformers
Safetensors
PyTorch
llama
facebook
meta
llama-3
text-generation-inference
Inference Endpoints
fbgemm_fp8
Model card Files Community
20

Config KV Heads should be 8 now?

#16
by tanmaylaud - opened
Discussion
tanmaylaud

https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-FP8/blob/f6b70b010c2a119cdb3401d158f3d1a347118b1d/config.json#L19

ArthurZ
 Meta Llama org

Yep it should

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment