For inference. CPU is enough for both quantization and inference.
ONEKQ AI
company
AI & ML interests
Benchmark, Code Generation, LLM
Organization Card
Edit this README.md
markdown file to author your organization card.
models
5
onekq-ai/starcoder2-3b-instruct-v0.1
Text Generation
•
Updated
•
14
onekq-ai/DeepSeek-Coder-V2-Lite-Base-bnb-4bit
Text Generation
•
Updated
•
110
onekq-ai/starcoder2-3b-bnb-4bit
Text Generation
•
Updated
•
48
onekq-ai/starcoder2-7b-bnb-4bit
Text Generation
•
Updated
•
8
onekq-ai/starcoder2-15b-bnb-4bit
Text Generation
•
Updated
•
30
•
1