gpt-oss-20b-finetune-mini

This is a fine-tuned version of openai/gpt-oss-20b using LoRA (Low-Rank Adaptation) on healthcare discharge data.

Model Details

• Base Model: openai/gpt-oss-20b
• Fine-tuning Method: LoRA (rank=32, alpha=64)
• Training Data: Healthcare discharge records (50,000 samples)
• Training Loss: 0.653

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("openai/gpt-oss-20b")
tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")

# Load fine-tuned adapter
model = PeftModel.from_pretrained(base_model, "nessa272/gpt-oss-20b-finetune-mini")

# Use for inference
inputs = tokenizer("Your medical query here", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Training Configuration

• Learning Rate: 1e-5
• Batch Size: 4
• Epochs: 3
• Max Length: 128
• Gradient Accumulation Steps: 4
• GPUs: 4x H100