Tiny models used for testing
Inference Optimization
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Stitched HIGGS Llama3 8B mixed-precision model variants.
-
meta-llama/Llama-3.1-8B-Instruct
Text Generation • 8B • Updated • 6.6M • • 6.08k -
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation • 8B • Updated • 30.9k • 9 -
RedHatAI/Llama-3.1-8B-Instruct-NVFP4
Text Generation • 5B • Updated • 11.9k • 1 -
Qwen/Qwen3-8B
Text Generation • 8B • Updated • 8.57M • • 1.14k
Mixed Precision Models
-
meta-llama/Llama-3.1-8B-Instruct
Text Generation • 8B • Updated • 6.6M • • 6.08k -
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation • 8B • Updated • 30.9k • 9 -
RedHatAI/Llama-3.1-8B-Instruct-NVFP4
Text Generation • 5B • Updated • 11.9k • 1 -
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_hybrid
6B • Updated • 1
Qwen3.6-35B-A3B mixed-precision HIGGS model variants, plus base FP16/FP8/NVFP4 references.
-
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-heuristic
Image-Text-to-Text • 24B • Updated • 99 -
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-hybrid
Image-Text-to-Text • 24B • Updated • 97 -
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-noise
Image-Text-to-Text • 24B • Updated • 62 -
inference-optimization/Qwen3.6-35B-A3B-5.5-bits-mode-heuristic
Image-Text-to-Text • 26B • Updated • 43
-
meta-llama/Llama-3.2-1B-Instruct
Text Generation • 1B • Updated • 5.48M • • 1.48k -
inference-optimization/Llama-3.2-1B-Instruct-FP8-Dynamic
1B • Updated • 1 -
inference-optimization/Llama-3.2-1B-Instruct-NVFP4
0.8B • Updated • 84 -
inference-optimization/Llama-3.2-1B-Instruct-5-bits-mode-heuristic-per-tensor
1B • Updated • 1
-
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch3
2B • Updated -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch1
2B • Updated -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch2
2B • Updated • 1 -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-GSM8K-MTP-finetuned
81B • Updated • 1
Tiny models used for testing
Qwen3.6-35B-A3B mixed-precision HIGGS model variants, plus base FP16/FP8/NVFP4 references.
-
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-heuristic
Image-Text-to-Text • 24B • Updated • 99 -
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-hybrid
Image-Text-to-Text • 24B • Updated • 97 -
inference-optimization/Qwen3.6-35B-A3B-5.0-bits-mode-noise
Image-Text-to-Text • 24B • Updated • 62 -
inference-optimization/Qwen3.6-35B-A3B-5.5-bits-mode-heuristic
Image-Text-to-Text • 26B • Updated • 43
Stitched HIGGS Llama3 8B mixed-precision model variants.
-
meta-llama/Llama-3.1-8B-Instruct
Text Generation • 8B • Updated • 6.6M • • 6.08k -
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation • 8B • Updated • 30.9k • 9 -
RedHatAI/Llama-3.1-8B-Instruct-NVFP4
Text Generation • 5B • Updated • 11.9k • 1 -
Qwen/Qwen3-8B
Text Generation • 8B • Updated • 8.57M • • 1.14k
-
meta-llama/Llama-3.2-1B-Instruct
Text Generation • 1B • Updated • 5.48M • • 1.48k -
inference-optimization/Llama-3.2-1B-Instruct-FP8-Dynamic
1B • Updated • 1 -
inference-optimization/Llama-3.2-1B-Instruct-NVFP4
0.8B • Updated • 84 -
inference-optimization/Llama-3.2-1B-Instruct-5-bits-mode-heuristic-per-tensor
1B • Updated • 1
Mixed Precision Models
-
meta-llama/Llama-3.1-8B-Instruct
Text Generation • 8B • Updated • 6.6M • • 6.08k -
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation • 8B • Updated • 30.9k • 9 -
RedHatAI/Llama-3.1-8B-Instruct-NVFP4
Text Generation • 5B • Updated • 11.9k • 1 -
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_hybrid
6B • Updated • 1
-
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch3
2B • Updated -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch1
2B • Updated -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch2
2B • Updated • 1 -
inference-optimization/Qwen3-Next-80B-A3B-Instruct-GSM8K-MTP-finetuned
81B • Updated • 1