metadata
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
base_model: sentence-transformers/paraphrase-MiniLM-L6-v2
metrics:
- accuracy
widget:
- text: >-
What fabric has a comfortable feel and is suitable for people with
sensitive skin?
- text: >-
What is the most recommended fabric for making outerwear that requires a
blend of comfort and resilience?
- text: >-
What fabric has a fluid drape and is ideal for creating lightweight summer
dresses?
- text: >-
Which fabric is best for creating versatile clothing items like casual
shirts, blouses, and dresses in a periwinkle blue hue?
- text: >-
What kind of fabric is suitable for making form-fitting activewear like
yoga pants and t-shirts?
pipeline_tag: text-classification
inference: true
model-index:
- name: SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.3836898395721925
name: Accuracy
SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-MiniLM-L6-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/paraphrase-MiniLM-L6-v2
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 128 tokens
- Number of Classes: 75 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
Fabric ID 0462 |
|
Fabric ID 0719_1 |
|
Fabric ID 0862 |
|
Fabric ID 0573_1 |
|
Fabric ID 0455 |
|
Fabric ID 0735 |
|
Fabric ID 0863 |
|
Fabric ID 0600 |
|
Fabric ID 0736 |
|
Fabric ID 0527_1 |
|
Fabric ID 0453 |
|
Fabric ID 0859 |
|
Fabric ID 0745 |
|
Fabric ID 0513 |
|
Fabric ID 0873 |
|
Fabric ID 0576_1 |
|
Fabric ID 0456 |
|
Fabric ID 0571 |
|
Fabric ID 0462_1 |
|
Fabric ID 0447 |
|
Fabric ID 0645 |
|
Fabric ID 0756 |
|
Fabric ID 0612 |
|
Fabric ID 0613 |
|
Fabric ID 0768 |
|
Fabric ID 0748 |
|
Fabric ID 0528_1 |
|
Fabric ID 0874 |
|
Fabric ID 0742 |
|
Fabric ID 0769 |
|
Fabric ID 0770 |
|
Fabric ID 0448 |
|
Fabric ID 0725 |
|
Fabric ID 0579 |
|
Fabric ID 0522 |
|
Fabric ID 0578 |
|
Fabric ID 0526_1 |
|
Fabric ID 0733 |
|
Fabric ID 0575_1 |
|
Fabric ID 0579_1 |
|
Fabric ID 0722 |
|
Fabric ID 0614 |
|
Fabric ID 0575 |
|
Fabric ID 0723 |
|
Fabric ID 0598 |
|
Fabric ID 0565 |
|
Fabric ID 0512 |
|
Fabric ID 0876 |
|
Fabric ID 0856 |
|
Fabric ID 0608 |
|
Fabric ID 0573 |
|
Fabric ID 0880 |
|
Fabric ID 0450 |
|
Fabric ID 0459 |
|
Fabric ID 0564 |
|
Fabric ID 0731 |
|
Fabric ID 0578_1 |
|
Fabric ID 0855 |
|
Fabric ID 0772 |
|
Fabric ID 0606 |
|
Fabric ID 0596 |
|
Fabric ID 0458 |
|
Fabric ID 0523_1 |
|
Fabric ID 0730 |
|
Fabric ID 0449 |
|
Fabric ID 0724 |
|
Fabric ID 0734 |
|
Fabric ID 0615 |
|
Fabric ID 0869 |
|
Fabric ID 0864 |
|
Fabric ID 0616 |
|
Fabric ID 0866 |
|
Fabric ID 0601 |
|
Fabric ID 0618 |
|
Fabric ID 0773 |
|
Evaluation
Metrics
Label | Accuracy |
---|---|
all | 0.3837 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Jazielinho/fabric_model_1")
# Run inference
preds = model("What fabric has a comfortable feel and is suitable for people with sensitive skin?")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 7 | 15.4858 | 30 |
Label | Training Sample Count |
---|---|
Fabric ID 0447 | 39 |
Fabric ID 0448 | 40 |
Fabric ID 0449 | 41 |
Fabric ID 0450 | 32 |
Fabric ID 0453 | 37 |
Fabric ID 0455 | 33 |
Fabric ID 0456 | 36 |
Fabric ID 0458 | 40 |
Fabric ID 0459 | 30 |
Fabric ID 0462 | 36 |
Fabric ID 0462_1 | 42 |
Fabric ID 0512 | 38 |
Fabric ID 0513 | 39 |
Fabric ID 0522 | 43 |
Fabric ID 0523_1 | 41 |
Fabric ID 0526_1 | 41 |
Fabric ID 0527_1 | 35 |
Fabric ID 0528_1 | 42 |
Fabric ID 0564 | 40 |
Fabric ID 0565 | 43 |
Fabric ID 0571 | 44 |
Fabric ID 0573 | 36 |
Fabric ID 0573_1 | 37 |
Fabric ID 0575 | 40 |
Fabric ID 0575_1 | 44 |
Fabric ID 0576_1 | 42 |
Fabric ID 0578 | 41 |
Fabric ID 0578_1 | 38 |
Fabric ID 0579 | 41 |
Fabric ID 0579_1 | 46 |
Fabric ID 0596 | 41 |
Fabric ID 0598 | 38 |
Fabric ID 0600 | 40 |
Fabric ID 0601 | 39 |
Fabric ID 0606 | 41 |
Fabric ID 0608 | 44 |
Fabric ID 0612 | 45 |
Fabric ID 0613 | 40 |
Fabric ID 0614 | 37 |
Fabric ID 0615 | 44 |
Fabric ID 0616 | 39 |
Fabric ID 0618 | 42 |
Fabric ID 0645 | 36 |
Fabric ID 0719_1 | 43 |
Fabric ID 0722 | 42 |
Fabric ID 0723 | 37 |
Fabric ID 0724 | 41 |
Fabric ID 0725 | 44 |
Fabric ID 0730 | 36 |
Fabric ID 0731 | 40 |
Fabric ID 0733 | 43 |
Fabric ID 0734 | 44 |
Fabric ID 0735 | 39 |
Fabric ID 0736 | 38 |
Fabric ID 0742 | 38 |
Fabric ID 0745 | 43 |
Fabric ID 0748 | 41 |
Fabric ID 0756 | 44 |
Fabric ID 0768 | 40 |
Fabric ID 0769 | 41 |
Fabric ID 0770 | 35 |
Fabric ID 0772 | 43 |
Fabric ID 0773 | 41 |
Fabric ID 0855 | 43 |
Fabric ID 0856 | 37 |
Fabric ID 0859 | 41 |
Fabric ID 0862 | 36 |
Fabric ID 0863 | 38 |
Fabric ID 0864 | 42 |
Fabric ID 0866 | 41 |
Fabric ID 0869 | 39 |
Fabric ID 0873 | 43 |
Fabric ID 0874 | 34 |
Fabric ID 0876 | 40 |
Fabric ID 0880 | 41 |
Training Hyperparameters
- batch_size: (256, 256)
- num_epochs: (20, 20)
- max_steps: -1
- sampling_strategy: undersampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0021 | 1 | 0.2732 | - |
0.1040 | 50 | 0.2348 | - |
0.2079 | 100 | 0.2277 | - |
0.3119 | 150 | 0.2419 | - |
0.4158 | 200 | 0.2401 | - |
0.5198 | 250 | 0.2367 | - |
0.6237 | 300 | 0.237 | - |
0.7277 | 350 | 0.2372 | - |
0.8316 | 400 | 0.2283 | - |
0.9356 | 450 | 0.223 | - |
1.0 | 481 | - | 0.207 |
1.0395 | 500 | 0.2075 | - |
1.1435 | 550 | 0.2162 | - |
1.2474 | 600 | 0.1984 | - |
1.3514 | 650 | 0.2173 | - |
1.4553 | 700 | 0.2154 | - |
1.5593 | 750 | 0.1912 | - |
1.6632 | 800 | 0.2014 | - |
1.7672 | 850 | 0.1866 | - |
1.8711 | 900 | 0.1933 | - |
1.9751 | 950 | 0.1821 | - |
2.0 | 962 | - | 0.1863 |
2.0790 | 1000 | 0.1607 | - |
2.1830 | 1050 | 0.1544 | - |
2.2869 | 1100 | 0.1624 | - |
2.3909 | 1150 | 0.1586 | - |
2.4948 | 1200 | 0.1445 | - |
2.5988 | 1250 | 0.1662 | - |
2.7027 | 1300 | 0.1515 | - |
2.8067 | 1350 | 0.158 | - |
2.9106 | 1400 | 0.1316 | - |
3.0 | 1443 | - | 0.1824 |
3.0146 | 1450 | 0.138 | - |
3.1185 | 1500 | 0.1414 | - |
3.2225 | 1550 | 0.1249 | - |
3.3264 | 1600 | 0.1336 | - |
3.4304 | 1650 | 0.1249 | - |
3.5343 | 1700 | 0.1308 | - |
3.6383 | 1750 | 0.1088 | - |
3.7422 | 1800 | 0.122 | - |
3.8462 | 1850 | 0.1029 | - |
3.9501 | 1900 | 0.1065 | - |
4.0 | 1924 | - | 0.1836 |
4.0541 | 1950 | 0.1133 | - |
4.1580 | 2000 | 0.1102 | - |
4.2620 | 2050 | 0.1209 | - |
4.3659 | 2100 | 0.1054 | - |
4.4699 | 2150 | 0.0874 | - |
4.5738 | 2200 | 0.0896 | - |
4.6778 | 2250 | 0.1104 | - |
4.7817 | 2300 | 0.0912 | - |
4.8857 | 2350 | 0.0766 | - |
4.9896 | 2400 | 0.0778 | - |
5.0 | 2405 | - | 0.1952 |
5.0936 | 2450 | 0.114 | - |
5.1975 | 2500 | 0.0869 | - |
5.3015 | 2550 | 0.0912 | - |
5.4054 | 2600 | 0.103 | - |
5.5094 | 2650 | 0.0748 | - |
5.6133 | 2700 | 0.0911 | - |
5.7173 | 2750 | 0.0721 | - |
5.8212 | 2800 | 0.0964 | - |
5.9252 | 2850 | 0.0712 | - |
6.0 | 2886 | - | 0.1938 |
6.0291 | 2900 | 0.0831 | - |
6.1331 | 2950 | 0.0924 | - |
6.2370 | 3000 | 0.0862 | - |
6.3410 | 3050 | 0.0725 | - |
6.4449 | 3100 | 0.0828 | - |
6.5489 | 3150 | 0.0645 | - |
6.6528 | 3200 | 0.0741 | - |
6.7568 | 3250 | 0.0589 | - |
6.8607 | 3300 | 0.075 | - |
6.9647 | 3350 | 0.075 | - |
7.0 | 3367 | - | 0.2016 |
7.0686 | 3400 | 0.0893 | - |
7.1726 | 3450 | 0.0727 | - |
7.2765 | 3500 | 0.0669 | - |
7.3805 | 3550 | 0.0702 | - |
7.4844 | 3600 | 0.0636 | - |
7.5884 | 3650 | 0.0605 | - |
7.6923 | 3700 | 0.0707 | - |
7.7963 | 3750 | 0.0597 | - |
7.9002 | 3800 | 0.0577 | - |
8.0 | 3848 | - | 0.2067 |
8.0042 | 3850 | 0.0515 | - |
8.1081 | 3900 | 0.0686 | - |
8.2121 | 3950 | 0.0587 | - |
8.3160 | 4000 | 0.057 | - |
8.4200 | 4050 | 0.0693 | - |
8.5239 | 4100 | 0.0812 | - |
8.6279 | 4150 | 0.0592 | - |
8.7318 | 4200 | 0.07 | - |
8.8358 | 4250 | 0.064 | - |
8.9397 | 4300 | 0.0503 | - |
9.0 | 4329 | - | 0.2122 |
9.0437 | 4350 | 0.0489 | - |
9.1476 | 4400 | 0.0602 | - |
9.2516 | 4450 | 0.0673 | - |
9.3555 | 4500 | 0.0665 | - |
9.4595 | 4550 | 0.0672 | - |
9.5634 | 4600 | 0.07 | - |
9.6674 | 4650 | 0.042 | - |
9.7713 | 4700 | 0.0656 | - |
9.8753 | 4750 | 0.0557 | - |
9.9792 | 4800 | 0.0648 | - |
10.0 | 4810 | - | 0.215 |
10.0832 | 4850 | 0.0455 | - |
10.1871 | 4900 | 0.0668 | - |
10.2911 | 4950 | 0.0453 | - |
10.3950 | 5000 | 0.0555 | - |
10.4990 | 5050 | 0.0679 | - |
10.6029 | 5100 | 0.0516 | - |
10.7069 | 5150 | 0.0448 | - |
10.8108 | 5200 | 0.0458 | - |
10.9148 | 5250 | 0.0544 | - |
11.0 | 5291 | - | 0.2172 |
11.0187 | 5300 | 0.0453 | - |
11.1227 | 5350 | 0.0442 | - |
11.2266 | 5400 | 0.0396 | - |
11.3306 | 5450 | 0.0507 | - |
11.4345 | 5500 | 0.0515 | - |
11.5385 | 5550 | 0.0503 | - |
11.6424 | 5600 | 0.0521 | - |
11.7464 | 5650 | 0.0551 | - |
11.8503 | 5700 | 0.0572 | - |
11.9543 | 5750 | 0.0604 | - |
12.0 | 5772 | - | 0.2245 |
12.0582 | 5800 | 0.0445 | - |
12.1622 | 5850 | 0.0564 | - |
12.2661 | 5900 | 0.0449 | - |
12.3701 | 5950 | 0.0502 | - |
12.4740 | 6000 | 0.0517 | - |
12.5780 | 6050 | 0.0426 | - |
12.6819 | 6100 | 0.0386 | - |
12.7859 | 6150 | 0.0446 | - |
12.8898 | 6200 | 0.0574 | - |
12.9938 | 6250 | 0.0546 | - |
13.0 | 6253 | - | 0.223 |
13.0977 | 6300 | 0.0381 | - |
13.2017 | 6350 | 0.047 | - |
13.3056 | 6400 | 0.0425 | - |
13.4096 | 6450 | 0.0445 | - |
13.5135 | 6500 | 0.056 | - |
13.6175 | 6550 | 0.0533 | - |
13.7214 | 6600 | 0.0466 | - |
13.8254 | 6650 | 0.0506 | - |
13.9293 | 6700 | 0.0402 | - |
14.0 | 6734 | - | 0.2238 |
14.0333 | 6750 | 0.0375 | - |
14.1372 | 6800 | 0.0447 | - |
14.2412 | 6850 | 0.0584 | - |
14.3451 | 6900 | 0.0348 | - |
14.4491 | 6950 | 0.0459 | - |
14.5530 | 7000 | 0.0465 | - |
14.6570 | 7050 | 0.0421 | - |
14.7609 | 7100 | 0.0537 | - |
14.8649 | 7150 | 0.041 | - |
14.9688 | 7200 | 0.0281 | - |
15.0 | 7215 | - | 0.2247 |
15.0728 | 7250 | 0.0431 | - |
15.1767 | 7300 | 0.039 | - |
15.2807 | 7350 | 0.0408 | - |
15.3846 | 7400 | 0.048 | - |
15.4886 | 7450 | 0.0354 | - |
15.5925 | 7500 | 0.0626 | - |
15.6965 | 7550 | 0.0396 | - |
15.8004 | 7600 | 0.045 | - |
15.9044 | 7650 | 0.0432 | - |
16.0 | 7696 | - | 0.2246 |
16.0083 | 7700 | 0.0385 | - |
16.1123 | 7750 | 0.0368 | - |
16.2162 | 7800 | 0.0628 | - |
16.3202 | 7850 | 0.035 | - |
16.4241 | 7900 | 0.0264 | - |
16.5281 | 7950 | 0.0275 | - |
16.6320 | 8000 | 0.0383 | - |
16.7360 | 8050 | 0.0469 | - |
16.8399 | 8100 | 0.0445 | - |
16.9439 | 8150 | 0.0357 | - |
17.0 | 8177 | - | 0.2268 |
17.0478 | 8200 | 0.0456 | - |
17.1518 | 8250 | 0.053 | - |
17.2557 | 8300 | 0.0498 | - |
17.3597 | 8350 | 0.0368 | - |
17.4636 | 8400 | 0.0473 | - |
17.5676 | 8450 | 0.0422 | - |
17.6715 | 8500 | 0.0362 | - |
17.7755 | 8550 | 0.0292 | - |
17.8794 | 8600 | 0.0431 | - |
17.9834 | 8650 | 0.0412 | - |
18.0 | 8658 | - | 0.2276 |
18.0873 | 8700 | 0.0655 | - |
18.1913 | 8750 | 0.0405 | - |
18.2952 | 8800 | 0.0455 | - |
18.3992 | 8850 | 0.0324 | - |
18.5031 | 8900 | 0.038 | - |
18.6071 | 8950 | 0.0315 | - |
18.7110 | 9000 | 0.0468 | - |
18.8150 | 9050 | 0.0451 | - |
18.9189 | 9100 | 0.032 | - |
19.0 | 9139 | - | 0.2268 |
19.0229 | 9150 | 0.0371 | - |
19.1268 | 9200 | 0.0439 | - |
19.2308 | 9250 | 0.0472 | - |
19.3347 | 9300 | 0.0362 | - |
19.4387 | 9350 | 0.0341 | - |
19.5426 | 9400 | 0.036 | - |
19.6466 | 9450 | 0.0382 | - |
19.7505 | 9500 | 0.0288 | - |
19.8545 | 9550 | 0.04 | - |
19.9584 | 9600 | 0.0277 | - |
20.0 | 9620 | - | 0.2277 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.1
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}