arbib/darija-qwen2.5-multigpu-lora_v3

Darija Qwen2.5-1.5B — LoRA (LLaMA-Factory + Unsloth + 2x T4)

Fine-tune sur Qwen/Qwen2.5-1.5B-Instruct pour le dialecte marocain (Darija).

Stack

• LLaMA-Factory (orchestration multi-GPU)
• Unsloth (backend rapide)
• 2x GPU T4 — fp16 pur (pas de quantization)

Dataset

• MBZUAI-Paris/Darija-SFT-Mixture — 5 000 samples (direction=None)

Config LoRA

• lora_rank: 16 | lora_alpha: 32 | target: all

Utilisation

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct", torch_dtype=torch.float16, device_map="auto"
)
model     = PeftModel.from_pretrained(base, "REPO_ID")
tokenizer = AutoTokenizer.from_pretrained("REPO_ID")