ksukrit/qwen2-vl-2b-4bit

Qwen2-VL-2B-Instruct 4-bit Quantized

This is a 4-bit quantized version of the Qwen2-VL-2B-Instruct model.

Model Description

• Original Model: Qwen/Qwen2-VL-2B-Instruct
• Quantization: 4-bit quantization using bitsandbytes
• Usage: This model is optimized for memory efficiency while maintaining performance
• License: Same as original model

Usage

from transformers import Qwen2VLModel, AutoTokenizer
import torch

model = Qwen2VLModel.from_pretrained("ksukrit/qwen2-vl-2b-4bit", trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("ksukrit/qwen2-vl-2b-4bit", trust_remote_code=True)

Quantization Details

• Quantization Method: bitsandbytes 4-bit quantization
• Compute dtype: float16
• Uses double quantization: True
• Quantization type: nf4