metadata

base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
library_name: setfit
metrics:
  - accuracy
pipeline_tag: text-classification
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      Bir öğrenci her gün 5 sayfa kitap okuyor. 5 gün süresince kaç sayfa kitap
      okur?
  - text: >-
      Bir inşaat firmasında 74 işçi vardı, Firma 29 işçi daha işe aldı. Firmada
      şimdi kaç işçi var?
  - text: >-
      If you have 4 coins that total 35 cents, and 3 of them are dimes, what is
      the other coin?
  - text: Bir üçgenin tabanı 12 cm, yüksekliği 8 cm'dir. Üçgenin alanı kaç cm²'dir?
  - text: >-
      Bir üçgenin kenar uzunlukları 8 cm, 10 cm ve 12 cm'dir. Üçgenin çevresi
      kaç cm'dir?
inference: true
model-index:
  - name: SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 1
            name: Accuracy

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
non_math	'Güneş sisteminde kaç gezegen vardır?' 'What is the largest ocean on Earth?' 'What is the intake of energy and nutrients by living organisms called?'
math	"Bir düzensiz çokgenin kenar uzunlukları 5 cm, 8 cm, 7 cm ve 6 cm'dir. Çokgenin çevresi kaç cm'dir?" '27 ÷ 3 = ?' 'Bir altıgenin bir iç açısının ölçüsü 120° ise, tüm iç açıların toplamı kaç derecedir?'

Evaluation

Metrics

Label	Accuracy
all	1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("serdarcaglar/primary-school-math-question-multi-lang")
# Run inference
preds = model("Bir üçgenin tabanı 12 cm, yüksekliği 8 cm'dir. Üçgenin alanı kaç cm²'dir?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	10.3435	33

Label	Training Sample Count
math	459
non_math	129

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0007	1	0.1982	-
0.0340	50	0.1009	-
0.0680	100	0.0054	-
0.1020	150	0.002	-
0.1361	200	0.001	-
0.1701	250	0.0023	-
0.2041	300	0.0002	-
0.2381	350	0.0013	-
0.2721	400	0.0004	-
0.3061	450	0.0004	-
0.3401	500	0.0002	-
0.3741	550	0.0001	-
0.4082	600	0.0001	-
0.4422	650	0.0002	-
0.4762	700	0.0001	-
0.5102	750	0.0001	-
0.5442	800	0.0002	-
0.5782	850	0.0002	-
0.6122	900	0.0001	-
0.6463	950	0.0005	-
0.6803	1000	0.0001	-
0.7143	1050	0.0001	-
0.7483	1100	0.0001	-
0.7823	1150	0.0001	-
0.8163	1200	0.0001	-
0.8503	1250	0.0001	-
0.8844	1300	0.0	-
0.9184	1350	0.0002	-
0.9524	1400	0.0001	-
0.9864	1450	0.0001	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 3.0.1
Transformers: 4.39.0
PyTorch: 2.4.0+cu121
Datasets: 2.21.0
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}