End of training

README.md

@@ -0,0 +1,86 @@
+---
+library_name: transformers
+base_model: KennethEnevoldsen/dfm-sentence-encoder-large-exp2-no-lang-align
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+model-index:
+- name: dfm1
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# dfm1
+
+This model is a fine-tuned version of [KennethEnevoldsen/dfm-sentence-encoder-large-exp2-no-lang-align](https://huggingface.co/KennethEnevoldsen/dfm-sentence-encoder-large-exp2-no-lang-align) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Accuracy: 0.8868
+- Precision: 0.8861
+- Recall: 0.8868
+- F1: 0.8855
+- Loss: 0.5432
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 8
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 20
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch   | Step | Accuracy | Precision | Recall | F1     | Validation Loss |
+|:-------------:|:-------:|:----:|:--------:|:---------:|:------:|:------:|:---------------:|
+| No log        | 0.9412  | 8    | 0.7844   | 0.7464    | 0.7844 | 0.7612 | 0.7436          |
+| No log        | 2.0     | 17   | 0.8999   | 0.8922    | 0.8999 | 0.8914 | 0.3252          |
+| No log        | 2.9412  | 25   | 0.9214   | 0.9226    | 0.9214 | 0.9121 | 0.3213          |
+| No log        | 4.0     | 34   | 0.9164   | 0.9235    | 0.9164 | 0.9176 | 0.3572          |
+| No log        | 4.9412  | 42   | 0.8880   | 0.8875    | 0.8880 | 0.8857 | 0.3576          |
+| No log        | 6.0     | 51   | 0.8907   | 0.8894    | 0.8907 | 0.8898 | 0.3993          |
+| No log        | 6.9412  | 59   | 0.8822   | 0.8822    | 0.8822 | 0.8806 | 0.4444          |
+| No log        | 8.0     | 68   | 0.8876   | 0.8867    | 0.8876 | 0.8865 | 0.4480          |
+| No log        | 8.9412  | 76   | 0.8987   | 0.8978    | 0.8987 | 0.8979 | 0.4688          |
+| No log        | 10.0    | 85   | 0.8984   | 0.8972    | 0.8984 | 0.8975 | 0.4845          |
+| No log        | 10.9412 | 93   | 0.8895   | 0.8887    | 0.8895 | 0.8884 | 0.5172          |
+| No log        | 12.0    | 102  | 0.8891   | 0.8882    | 0.8891 | 0.8881 | 0.5349          |
+| No log        | 12.9412 | 110  | 0.8907   | 0.8897    | 0.8907 | 0.8896 | 0.5343          |
+| No log        | 14.0    | 119  | 0.8895   | 0.8886    | 0.8895 | 0.8884 | 0.5374          |
+| No log        | 14.9412 | 127  | 0.8868   | 0.8861    | 0.8868 | 0.8855 | 0.5317          |
+| No log        | 16.0    | 136  | 0.8853   | 0.8847    | 0.8853 | 0.8839 | 0.5383          |
+| No log        | 16.9412 | 144  | 0.8853   | 0.8847    | 0.8853 | 0.8839 | 0.5402          |
+| No log        | 18.0    | 153  | 0.8865   | 0.8858    | 0.8865 | 0.8851 | 0.5429          |
+| No log        | 18.8235 | 160  | 0.8868   | 0.8861    | 0.8868 | 0.8855 | 0.5432          |
+
+
+### Framework versions
+
+- Transformers 4.44.2
+- Pytorch 2.5.0+cu121
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "KennethEnevoldsen/dfm-sentence-encoder-large-exp2-no-lang-align",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2",
+    "3": "LABEL_3",
+    "4": "LABEL_4"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2,
+    "LABEL_3": 3,
+    "LABEL_4": 4
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 50000
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04e4b69004af1123efca6cde5fd5665d1ecb4afc9d92ea7a15603cf6c13599a9
+size 1416218404

runs/Oct24_09-52-01_55f39f7a0968/events.out.tfevents.1729763524.55f39f7a0968.571.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945238118f5fa887f06d8c14f2570bd9ab1891af0c690ae1de19ebe33ad3584d
+size 14515

runs/Oct24_09-52-01_55f39f7a0968/events.out.tfevents.1729763719.55f39f7a0968.571.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aea21dc5e7145b22456359d3e0944a91ff3d1f9fe2bbead45f7aa3c7d91785b4
+size 560

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20e37548f1bed2c12a9f4882eb0a54e885f75ec4ad6d459fbfa3a5b81bb71371
+size 5304