---
base_model: unsloth/llama-3-8b
library_name: peft
license: llama3
tags:
- unsloth
- generated_from_trainer
model-index:
- name: Meta-Llama-3-8B_pct_default
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Meta-Llama-3-8B_pct_default

This model is a fine-tuned version of [unsloth/llama-3-8b](https://huggingface.co/unsloth/llama-3-8b) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.1916

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.02
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 2.2535        | 0.0206 | 8    | 2.2735          |
| 2.2839        | 0.0412 | 16   | 2.2722          |
| 2.2318        | 0.0618 | 24   | 2.2795          |
| 2.3022        | 0.0824 | 32   | 2.2721          |
| 2.2843        | 0.1030 | 40   | 2.2608          |
| 2.2433        | 0.1236 | 48   | 2.2513          |
| 2.2617        | 0.1442 | 56   | 2.2632          |
| 2.2962        | 0.1648 | 64   | 2.2767          |
| 2.2573        | 0.1854 | 72   | 2.2800          |
| 2.2856        | 0.2060 | 80   | 2.2750          |
| 2.3158        | 0.2266 | 88   | 2.2799          |
| 2.3622        | 0.2472 | 96   | 2.2860          |
| 2.357         | 0.2678 | 104  | 2.2901          |
| 2.3124        | 0.2884 | 112  | 2.2985          |
| 2.3646        | 0.3090 | 120  | 2.2943          |
| 2.3591        | 0.3296 | 128  | 2.2891          |
| 2.3085        | 0.3502 | 136  | 2.2923          |
| 2.3054        | 0.3708 | 144  | 2.2878          |
| 2.3203        | 0.3914 | 152  | 2.2829          |
| 2.2995        | 0.4120 | 160  | 2.2783          |
| 2.356         | 0.4326 | 168  | 2.2759          |
| 2.2942        | 0.4532 | 176  | 2.2720          |
| 2.2987        | 0.4738 | 184  | 2.2650          |
| 2.3025        | 0.4944 | 192  | 2.2645          |
| 2.294         | 0.5150 | 200  | 2.2624          |
| 2.2959        | 0.5356 | 208  | 2.2678          |
| 2.3074        | 0.5562 | 216  | 2.2525          |
| 2.2862        | 0.5768 | 224  | 2.2530          |
| 2.2745        | 0.5974 | 232  | 2.2494          |
| 2.2422        | 0.6180 | 240  | 2.2398          |
| 2.275         | 0.6386 | 248  | 2.2399          |
| 2.2632        | 0.6592 | 256  | 2.2398          |
| 2.2198        | 0.6798 | 264  | 2.2288          |
| 2.2732        | 0.7004 | 272  | 2.2233          |
| 2.2576        | 0.7210 | 280  | 2.2178          |
| 2.2606        | 0.7416 | 288  | 2.2098          |
| 2.2559        | 0.7621 | 296  | 2.2151          |
| 2.2852        | 0.7827 | 304  | 2.2048          |
| 2.2252        | 0.8033 | 312  | 2.2026          |
| 2.2024        | 0.8239 | 320  | 2.2029          |
| 2.2339        | 0.8445 | 328  | 2.1969          |
| 2.2468        | 0.8651 | 336  | 2.1979          |
| 2.2582        | 0.8857 | 344  | 2.1932          |
| 2.223         | 0.9063 | 352  | 2.1925          |
| 2.1887        | 0.9269 | 360  | 2.1937          |
| 2.218         | 0.9475 | 368  | 2.1924          |
| 2.258         | 0.9681 | 376  | 2.1917          |
| 2.2479        | 0.9887 | 384  | 2.1916          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.44.0
- Pytorch 2.4.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1