Edit model card

llama3.2-1B-Function-calling

⚠️ Important: This model is still under development and has not been fully fine-tuned. It is not yet suitable for use in production and should be treated as a work-in-progress. The results and performance metrics shared here are preliminary and subject to change.

Model description

This model was trained from scratch on an unknown dataset and is intended for function-calling tasks. As it is still in early stages, further development is required to optimize its performance.

Intended uses & limitations

Currently, this model is not fully trained or optimized for any specific task. It is intended to handle function-calling tasks but should not be used in production until more comprehensive fine-tuning and evaluation are completed.

Training and evaluation data

More information is needed regarding the dataset used for training. The model has not yet been fully evaluated, and additional testing is required to confirm its capabilities.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 32
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.3083 0.9997 1687 0.3622
0.202 2.0 3375 0.2844
0.1655 2.9997 5061 0.1491

These results are preliminary, and further training will be necessary to refine the model's performance.

Framework versions

  • Transformers 4.45.2
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.0
Downloads last month
61
Safetensors
Model size
1.24B params
Tensor type
F32
·
Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for Kanonenbombe/llama3.2-1B-Function-calling

Finetuned
(90)
this model

Dataset used to train Kanonenbombe/llama3.2-1B-Function-calling