README.md

---
license: apache-2.0
datasets:
- harishnair04/mtsamples
language:
- en
base_model:
- google/gemma-2-2b
tags:
- trl
- sft
- quantization
- 4bit
- LoRA
library_name: transformers
---


# Model Card for Medical Transcription Model (Gemma-MedTr)

This model is a fine-tuned variant of `Gemma-2-2b`, optimized for medical transcription tasks with efficient 4-bit quantization and Low-Rank Adaptation (LoRA). It handles transcription processing, keyword extraction, and medical specialty classification.

## Model Details

- **Developed by:** Harish Nair
- **Organization:** University of Ottawa
- **License:** Apache 2.0
- **Fine-tuned from:** [Gemma-2-2b](https://huggingface.co/google/gemma-2-2b)
- **Model type:** Transformer-based language model for medical transcription processing
- **Language(s):** English

### Training Details

- **Training Loss:** Final training loss at step 10: 1.4791
- **Training Configuration:** 
  - LoRA with `r=8`, targeting specific transformer modules for adaptation.
  - 4-bit quantization using `nf4` quantization type and `bfloat16` compute precision.
- **Training Runtime:** 20.85 seconds, with approximately 1.92 samples processed per second.

## How to Use

To load and use this model, initialize it with the following configuration:
```python
import pandas as pd
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, PeftModel

model_id = "harishnair04/Gemma-medtr-2b-sft"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(model_id, token=access_token_read)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map='auto', token=access_token_read)