Edit model card

finetune_colpali_v1_2-german_ver2-4bit

This model is a fine-tuned version of vidore/colpaligemma-3b-pt-448-base on the German_docx dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0559
  • Model Preparation Time: 0.0099

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Model Preparation Time
No log 0.0816 1 0.3744 0.0099
1.6073 0.8163 10 0.3027 0.0099
1.2318 1.6327 20 0.2157 0.0099
0.6498 2.4490 30 0.1428 0.0099
0.5073 3.2653 40 0.1181 0.0099
0.5106 4.0816 50 0.1069 0.0099
0.2965 4.8980 60 0.0969 0.0099
0.3175 5.7143 70 0.0922 0.0099
0.1775 6.5306 80 0.1089 0.0099
0.1966 7.3469 90 0.0649 0.0099
0.1317 8.1633 100 0.0477 0.0099
0.1287 8.9796 110 0.0503 0.0099
0.2576 9.7959 120 0.0559 0.0099

Framework versions

  • Transformers 4.46.1
  • Pytorch 2.3.1
  • Datasets 3.1.0
  • Tokenizers 0.20.1
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for svenbl80/finetune_colpali_v1_2-german_ver2-4bit

Finetuned
(18)
this model