Exl2
Collection
Models that we have quantized to exllamav2's quantization format
•
22 items
•
Updated
•
2
Aetheria-L2-70B-exl2
Exllama v2 quant of royallab/Aetheria-L2-70B
Branches:
- main: measurement.json calculated at 2048 token calibration rows on PIPPA
- 5.0bpw-h6: 5 decoder bits per weight, 6 head bits
- ideal for 2x 24gb GPUs at 8192 context, or 1x 48gb GPU at 8192 context with CFG cache
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model authors have turned it off explicitly.