Vishva007/Qwen2.5-3B-Instruct-RBI-QA

Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A

A specialized model for answering questions about Reserve Bank of India (RBI) regulations and banking policies.

Performance: 57.6% accuracy (8.2x improvement over base model's 7%)

Quick Facts

• 🎯 Accuracy: 57.6% on 1000-sample evaluation set
• 📚 Coverage: 100+ regulation areas (Basel III, FEMA, AML, PSL, etc.)
• 🚀 Training: 47K QA pairs with rephrased variants
• ⚡ Efficient: 3B parameters, optimized for deployment

Performance Highlights

Category	Base Model	Fine-tuned	Improvement
Overall	7.0%	57.6%	+50.6%
Fact-based	6.8%	57.6%	+50.8%
Reasoning	37.5%	62.5%	+25.0%

Top Categories (70%+ accuracy): Anti-Money Laundering, Digital Payments, Government Banking, MSME Finance, Currency Management

Training Details

• Method: LoRA fine-tuning (r=16, alpha=32)
• Dataset: RBI-Circular-QA-Dataset (47K samples)
• Training: 1 epoch, 2 hours on NVIDIA L40S
• Loss: 0.79 → 0.57 (train), 0.58 (eval)

Code & Resources

• 💻 Training Code: GitHub Repository
• 📊 Dataset: RBI-Circular-QA-Dataset
• 🤗 Base Model: Qwen2.5-3B-Instruct

📝 Technical Deep Dive

Want to understand the theory and methodology behind this model?

Read the full article: Fine-tuning Qwen 2.5 3B for RBI Regulations: Achieving 8x Performance with Smart Data Augmentation

The article covers:

• 🧠 LoRA Theory: Why and how Low-Rank Adaptation works
• ⚡ Unsloth Deep Dive: Technical advantages and performance optimizations
• 📊 Data Augmentation Strategy: The rephrasing technique that delivered 40% of improvement
• 🎓 Hyperparameter Analysis: Detailed explanation of every training choice
• 📈 Evaluation Methodology: Stratified sampling and LLM-as-judge approach
• 🔬 Ablation Studies: What really mattered for the 8x improvement

Perfect for ML engineers who want to replicate or adapt this approach for their own domain-specific fine-tuning projects.

Use Cases

✅ Banking compliance chatbots
✅ Regulatory Q&A systems
✅ Training tools for banking professionals
✅ RBI circular analysis

⚠️ Not for: Legal compliance decisions (requires expert review), real-time updates

Limitations

• Knowledge cutoff: 2024 regulations
• 57.6% accuracy means ~42% of complex queries need verification
• Optimized for English only

Citation

  @misc{
    qwen25-3b-rbi-qa,
    author = {Vishva007},
    title = {Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A},
    year = {2025},
    publisher = {HuggingFace},
    howpublished = {\url{https://huggingface.co/Vishva007/Qwen2.5-3B-Instruct-RBI-QA}},
  }

License

Apache 2.0 (inherits from base model)

---

Author: @Vishva007 | Updated: November 2025