Hugging Face's logo

steve-sli
/
0721_201833-google-gemma-2b

PEFT
Safetensors
trl
reward-trainer
Generated from Trainer
Model card Files Community
0721_201833-google-gemma-2b / README.md
steve-sli's picture
steve-sli
End of training
030ac67 verified
preview code
|
raw
history blame contribute delete
2.39 kB
metadata
base_model: google/gemma-2b
library_name: peft
license: gemma
metrics:
  - accuracy
tags:
  - trl
  - reward-trainer
  - generated_from_trainer
model-index:
  - name: 0721_201833-google-gemma-2b
    results: []

Visualize in Weights & Biases

0721_201833-google-gemma-2b

This model is a fine-tuned version of google/gemma-2b on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7621
  • Accuracy: 0.515

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-08
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10.0

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.734 0.7692 5 0.7622 0.515
0.8399 1.5385 10 0.7622 0.515
0.7834 2.3077 15 0.7622 0.515
0.7012 3.0769 20 0.7622 0.515
0.7259 3.8462 25 0.7621 0.515
0.8204 4.6154 30 0.7621 0.515
0.7768 5.3846 35 0.7621 0.515
0.8371 6.1538 40 0.7621 0.515
0.7098 6.9231 45 0.7621 0.515
0.8716 7.6923 50 0.7621 0.515
0.6456 8.4615 55 0.7621 0.515
0.7848 9.2308 60 0.7621 0.515

Framework versions

  • PEFT 0.11.1
  • Transformers 4.42.4
  • Pytorch 2.3.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1