model_output
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.6558
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_steps: 100
- num_epochs: 3
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
0.8594 | 0.11 | 20 | 0.8079 |
0.7572 | 0.22 | 40 | 0.7334 |
0.7202 | 0.33 | 60 | 0.7059 |
0.6849 | 0.44 | 80 | 0.6899 |
0.7245 | 0.55 | 100 | 0.6865 |
0.6738 | 0.66 | 120 | 0.6752 |
0.703 | 0.77 | 140 | 0.6705 |
0.6725 | 0.87 | 160 | 0.6702 |
0.6976 | 0.98 | 180 | 0.6617 |
0.6018 | 1.09 | 200 | 0.6663 |
0.6093 | 1.2 | 220 | 0.6593 |
0.6167 | 1.31 | 240 | 0.6611 |
0.6128 | 1.42 | 260 | 0.6611 |
0.6098 | 1.53 | 280 | 0.6590 |
0.6463 | 1.64 | 300 | 0.6568 |
0.6174 | 1.75 | 320 | 0.6534 |
0.5943 | 1.86 | 340 | 0.6519 |
0.602 | 1.97 | 360 | 0.6516 |
0.5815 | 2.08 | 380 | 0.6540 |
0.5496 | 2.19 | 400 | 0.6576 |
0.5597 | 2.3 | 420 | 0.6609 |
0.5433 | 2.4 | 440 | 0.6691 |
0.5498 | 2.51 | 460 | 0.6742 |
0.5418 | 2.62 | 480 | 0.6684 |
0.5337 | 2.73 | 500 | 0.6672 |
0.5045 | 2.84 | 520 | 0.6553 |
0.5659 | 2.95 | 540 | 0.6558 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.0.1+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
- Downloads last month
- 6
Inference API (serverless) is not available, repository is disabled.
Model tree for chrisgru/llama-7b-v1
Base model
meta-llama/Llama-2-7b-hf
Quantized
this model