Data-free quantization w/ Q-Palette
Collection
30 items
•
Updated
Model Card
- Base model:
meta-llama/Llama-3.1-8B - Quantization method: Latency constrained MSQ with Q-Palette
- Backend kernel: Q-Palette kernel
- Calibration data: N/A (data-free)
- See Figure 1 (c) of our paper.
How to run
- Follow the instruction in https://github.com/snu-mllab/Q-Palette.
References
- Downloads last month
- 18
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for bdbj/Llama-3.1-8b-qpal-df-msq-fig1c
Base model
meta-llama/Llama-3.1-8B