---
library_name: transformers
license: apache-2.0
base_model: albert/albert-base-v2
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: albert-base-v2-grammar-ner
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# albert-base-v2-grammar-ner

This model is a fine-tuned version of [albert/albert-base-v2](https://huggingface.co/albert/albert-base-v2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1134
- Accuracy: 0.9870
- F1 Macro: 0.7941
- F1 Micro: 0.9008
- Precision Macro: 0.8789
- Precision Micro: 0.9569
- Recall Macro: 0.7518
- Recall Micro: 0.8510

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 24
- eval_batch_size: 24
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 18

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Micro | Precision Macro | Precision Micro | Recall Macro | Recall Micro |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:--------:|:---------------:|:---------------:|:------------:|:------------:|
| 0.4297        | 1.0   | 93   | 0.2896          | 0.9313   | 0.1318   | 0.4462   | 0.1897          | 0.5163          | 0.1281       | 0.3928       |
| 0.2521        | 2.0   | 186  | 0.2192          | 0.9452   | 0.2315   | 0.5160   | 0.3282          | 0.6752          | 0.1962       | 0.4176       |
| 0.167         | 3.0   | 279  | 0.1630          | 0.9662   | 0.3546   | 0.7198   | 0.4142          | 0.8358          | 0.3295       | 0.6321       |
| 0.1026        | 4.0   | 372  | 0.1343          | 0.9733   | 0.4185   | 0.7769   | 0.5241          | 0.8732          | 0.3797       | 0.6998       |
| 0.0718        | 5.0   | 465  | 0.1231          | 0.9738   | 0.4644   | 0.7794   | 0.5584          | 0.8525          | 0.4382       | 0.7178       |
| 0.0483        | 6.0   | 558  | 0.1269          | 0.9778   | 0.4778   | 0.8204   | 0.6262          | 0.9415          | 0.4164       | 0.7269       |
| 0.0335        | 7.0   | 651  | 0.1162          | 0.9804   | 0.6028   | 0.8416   | 0.6985          | 0.8834          | 0.5846       | 0.8036       |
| 0.0233        | 8.0   | 744  | 0.1203          | 0.9813   | 0.5736   | 0.8475   | 0.7429          | 0.9496          | 0.4988       | 0.7652       |
| 0.0171        | 9.0   | 837  | 0.1052          | 0.9836   | 0.6502   | 0.8671   | 0.7023          | 0.8964          | 0.6490       | 0.8397       |
| 0.01          | 10.0  | 930  | 0.1125          | 0.9805   | 0.6681   | 0.8477   | 0.6854          | 0.8535          | 0.6875       | 0.8420       |
| 0.0084        | 11.0  | 1023 | 0.1058          | 0.9862   | 0.7195   | 0.8894   | 0.8004          | 0.9287          | 0.6870       | 0.8533       |
| 0.0051        | 12.0  | 1116 | 0.1092          | 0.9870   | 0.8015   | 0.9015   | 0.8810          | 0.95            | 0.7612       | 0.8578       |
| 0.0031        | 13.0  | 1209 | 0.1131          | 0.9865   | 0.8006   | 0.8983   | 0.8827          | 0.9429          | 0.7592       | 0.8578       |
| 0.0017        | 14.0  | 1302 | 0.1106          | 0.9873   | 0.8058   | 0.9039   | 0.8748          | 0.9525          | 0.7749       | 0.8600       |
| 0.0012        | 15.0  | 1395 | 0.1111          | 0.9875   | 0.7985   | 0.9058   | 0.8818          | 0.9596          | 0.7576       | 0.8578       |
| 0.0009        | 16.0  | 1488 | 0.1128          | 0.9870   | 0.7941   | 0.9008   | 0.8789          | 0.9569          | 0.7518       | 0.8510       |
| 0.0008        | 17.0  | 1581 | 0.1133          | 0.9870   | 0.7941   | 0.9008   | 0.8789          | 0.9569          | 0.7518       | 0.8510       |
| 0.0008        | 18.0  | 1674 | 0.1134          | 0.9870   | 0.7941   | 0.9008   | 0.8789          | 0.9569          | 0.7518       | 0.8510       |


### Framework versions

- Transformers 4.46.3
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.20.3