SeifElden2342532/Code-Optimizer

Code Optimization Fine-tuned Qwen2.5-Coder-7B-Instruct (LoRA Adapter)

This repository contains a fine-tuned LoRA adapter for the Qwen/Qwen2.5-Coder-7B-Instruct model, specialized for Python code optimization. The model was fine-tuned using QLoRA on the SeifElden2342532/Code-Optimization dataset.

Model Description

This model is a LoRA adapter of the Qwen2.5-Coder-7B-Instruct base model. It has been fine-tuned to act as an expert Python code optimizer, capable of taking user-provided Python code and optimizing it for performance, readability, or conciseness, along with providing explanations and complexity comparisons.

Training Details

• Base Model: Qwen/Qwen2.5-Coder-7B-Instruct
• Fine-tuning Framework: QLoRA (Parameter-Efficient Fine-Tuning)
• Dataset: SeifElden2342532/Code-Optimization
• Training Parameters:
  • num_train_epochs: 3
  • lora_r: 64
  • lora_alpha: 128
  • max_seq_length: 2048
  • per_device_train_batch_size: 4
  • gradient_accumulation_steps: 4
  • learning_rate: 2e-4
  • bf16: True
  • gradient_checkpointing: True

Github Repo

https://github.com/SeifEldenOsama/Code-Optimizer/tree/master

How to Use

To use this LoRA adapter, you need to load the base model and then apply the adapter. Here's a complete example to test the model:

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# 1. Configuration
base_model_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
adapter_repo_id = "SeifElden2342532/Code-Optimizer"

# 2. Load Tokenizer and Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id, 
    torch_dtype=torch.bfloat16, 
    device_map="auto"
)

# 3. Load and Merge the LoRA Adapter
model = PeftModel.from_pretrained(model, adapter_repo_id)
model = model.merge_and_unload() # Merging for faster inference

# 4. Prepare the Input
messages = [
    {
        "role": "system", 
        "content": "You are an expert Python code optimizer. Your goal is to take user-provided Python code and optimize it for performance, readability, or conciseness, based on the user's specified category. Provide the optimized code, a brief explanation of the changes, and a complexity comparison table (e.g., time and space complexity before and after optimization)."
    },
    {
        "role": "user", 
        "content": "Original Code:\n```python\ndef factorial(n):\n    if n == 0:\n        return 1\n    else:\n        return n * factorial(n-1)\n```\nCategory: Performance"
    }
]

# 5. Generate Optimization
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(**model_inputs, max_new_tokens=1024)
output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

# 6. Print Result
print(output)