Hugging Face's logo

firqaaa
/
setfit-indo-absa-restaurant-aspect

Text Classification
setfit
Safetensors
sentence-transformers
bert
absa
generated_from_setfit_trainer
Eval Results
Model card Files Community
setfit-indo-absa-restaurant-aspect / README.md
firqaaa's picture
firqaaa
Update README.md
6d0c783 verified
preview code
|
raw
history blame contribute delete
20.6 kB
metadata
library_name: setfit
tags:
  - setfit
  - absa
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      timur:unggul di atas tetangga di jalan 6 timur, taj mahal juga sangat
      sebanding, dalam kualitas makanan, dengan baluchi yang terlalu dipuji (dan
      kurang layak).
  - text: >-
      makanan:saya sangat merekomendasikan cafe st bart's untuk makanan mereka,
      suasana dan layanan yang luar biasa melayani
  - text: >-
      terong parmesan:parmesan terung juga enak, dan teman saya yang besar di
      manhattan metakan bahwa tidak ada orang yang pantas mendapatkan ziti
      panggang yang lebih enak dengan saus daging terong parmesan
  - text: >-
      tuna lelehan:kami memesan tuna lelehan - itu datang dengan keluar keju
      yang ha membuat sandwich tuna daging tuna
  - text: >-
      manhattan metakan:parmesan terung juga enak, dan teman saya yang besar di
      manhattan metakan bahwa tidak ada orang yang pantas mendapatkan ziti
      panggang yang lebih enak dengan saus daging ziti panggang dengan saus
      daging
pipeline_tag: text-classification
inference: false
base_model: firqaaa/indo-sentence-bert-base
model-index:
  - name: SetFit Aspect Model with firqaaa/indo-sentence-bert-base
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.9087072065030483
            name: Accuracy

SetFit Aspect Model with firqaaa/indo-sentence-bert-base

This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses firqaaa/indo-sentence-bert-base as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification. In particular, this model is in charge of filtering aspect span candidates.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

This model was trained within the context of a larger system for ABSA, which looks like so:

  1. Use a spaCy model to select possible aspect span candidates.
  2. Use this SetFit model to filter these possible aspect span candidates.
  3. Use a SetFit model to classify the filtered aspect span candidates.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
aspect
  • 'reservasi:restoran ini sangat kecil sehingga reservasi adalah suatu keharusan.'
  • 'nyonya rumah:di sebelah kanan saya, nyo rumah berdiri di dekat seorang busboy dan mendesiskan rapido, rapido ketika dia mencoba membersihkan dan mengatur ulang meja untuk enam orang nyonya rumah'
  • 'busboy:di sebelah kanan saya, nyo rumah berdiri di dekat seorang busboy dan mendesiskan rapido, rapido ketika dia mencoba membersihkan dan mengatur ulang meja untuk enam orang.'
no aspect
  • 'restoran:restoran ini sangat kecil sehingga reservasi adalah suatu keharusan.'
  • 'keharusan:restoran ini sangat kecil sehingga reservasi adalah suatu keharusan.'
  • 'sebelah kanan:di sebelah kanan saya, nyo rumah berdiri di dekat seorang busboy dan mendesiskan rapido, rapido ketika dia mencoba membersihkan dan mengatur ulang meja untuk enam orang nyonya rumah'

Evaluation

Metrics

Label Accuracy
all 0.9087

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import AbsaModel

# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
    "firqaaa/setfit-indo-absa-restaurants-aspect",
    "firqaaa/setfit-indo-absa-restaurants-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 2 19.7819 59
Label Training Sample Count
no aspect 2939
aspect 1468

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: True
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0000 1 0.3135 -
0.0001 50 0.3401 -
0.0001 100 0.3212 -
0.0002 150 0.3641 -
0.0003 200 0.3317 -
0.0004 250 0.2809 -
0.0004 300 0.2446 -
0.0005 350 0.284 -
0.0006 400 0.3257 -
0.0007 450 0.2996 -
0.0007 500 0.209 0.295
0.0008 550 0.2121 -
0.0009 600 0.2204 -
0.0010 650 0.3023 -
0.0010 700 0.3253 -
0.0011 750 0.233 -
0.0012 800 0.3131 -
0.0013 850 0.2873 -
0.0013 900 0.2028 -
0.0014 950 0.2608 -
0.0015 1000 0.2842 0.2696
0.0016 1050 0.2297 -
0.0016 1100 0.266 -
0.0017 1150 0.2771 -
0.0018 1200 0.2347 -
0.0019 1250 0.2539 -
0.0019 1300 0.3409 -
0.0020 1350 0.2925 -
0.0021 1400 0.2608 -
0.0021 1450 0.2792 -
0.0022 1500 0.261 0.2636
0.0023 1550 0.2596 -
0.0024 1600 0.2563 -
0.0024 1650 0.2329 -
0.0025 1700 0.2954 -
0.0026 1750 0.3329 -
0.0027 1800 0.2138 -
0.0027 1850 0.2591 -
0.0028 1900 0.268 -
0.0029 1950 0.2144 -
0.0030 2000 0.2361 0.2586
0.0030 2050 0.2322 -
0.0031 2100 0.2646 -
0.0032 2150 0.2018 -
0.0033 2200 0.2579 -
0.0033 2250 0.2501 -
0.0034 2300 0.2657 -
0.0035 2350 0.2272 -
0.0036 2400 0.2383 -
0.0036 2450 0.2615 -
0.0037 2500 0.2818 0.2554
0.0038 2550 0.2616 -
0.0039 2600 0.2225 -
0.0039 2650 0.2749 -
0.0040 2700 0.2572 -
0.0041 2750 0.2729 -
0.0041 2800 0.2559 -
0.0042 2850 0.2363 -
0.0043 2900 0.2518 -
0.0044 2950 0.1948 -
0.0044 3000 0.2842 0.2538
0.0045 3050 0.2243 -
0.0046 3100 0.2186 -
0.0047 3150 0.2829 -
0.0047 3200 0.2101 -
0.0048 3250 0.2156 -
0.0049 3300 0.2539 -
0.0050 3350 0.3005 -
0.0050 3400 0.2699 -
0.0051 3450 0.2431 -
0.0052 3500 0.2931 0.2515
0.0053 3550 0.2032 -
0.0053 3600 0.2451 -
0.0054 3650 0.2419 -
0.0055 3700 0.2267 -
0.0056 3750 0.2945 -
0.0056 3800 0.2689 -
0.0057 3850 0.2596 -
0.0058 3900 0.2978 -
0.0059 3950 0.2876 -
0.0059 4000 0.2484 0.2482
0.0060 4050 0.2698 -
0.0061 4100 0.2155 -
0.0061 4150 0.2474 -
0.0062 4200 0.2683 -
0.0063 4250 0.2979 -
0.0064 4300 0.2866 -
0.0064 4350 0.2604 -
0.0065 4400 0.1989 -
0.0066 4450 0.2708 -
0.0067 4500 0.2705 0.2407
0.0067 4550 0.2144 -
0.0068 4600 0.2503 -
0.0069 4650 0.2193 -
0.0070 4700 0.1796 -
0.0070 4750 0.2384 -
0.0071 4800 0.1933 -
0.0072 4850 0.2248 -
0.0073 4900 0.22 -
0.0073 4950 0.2052 -
0.0074 5000 0.2314 0.224
0.0075 5050 0.2279 -
0.0076 5100 0.2198 -
0.0076 5150 0.2332 -
0.0077 5200 0.1666 -
0.0078 5250 0.1949 -
0.0079 5300 0.1802 -
0.0079 5350 0.2496 -
0.0080 5400 0.2399 -
0.0081 5450 0.2042 -
0.0082 5500 0.1859 0.2077
0.0082 5550 0.2216 -
0.0083 5600 0.1227 -
0.0084 5650 0.2351 -
0.0084 5700 0.2735 -
0.0085 5750 0.1008 -
0.0086 5800 0.1568 -
0.0087 5850 0.1211 -
0.0087 5900 0.0903 -
0.0088 5950 0.1473 -
0.0089 6000 0.1167 0.1877
0.0090 6050 0.206 -
0.0090 6100 0.2392 -
0.0091 6150 0.116 -
0.0092 6200 0.1493 -
0.0093 6250 0.1373 -
0.0093 6300 0.1163 -
0.0094 6350 0.0669 -
0.0095 6400 0.0756 -
0.0096 6450 0.0788 -
0.0096 6500 0.1816 0.1838
0.0097 6550 0.1288 -
0.0098 6600 0.0946 -
0.0099 6650 0.1374 -
0.0099 6700 0.2167 -
0.0100 6750 0.0759 -
0.0101 6800 0.1543 -
0.0102 6850 0.0573 -
0.0102 6900 0.1169 -
0.0103 6950 0.0294 -
0.0104 7000 0.1241 0.1769
0.0104 7050 0.0803 -
0.0105 7100 0.0139 -
0.0106 7150 0.01 -
0.0107 7200 0.0502 -
0.0107 7250 0.0647 -
0.0108 7300 0.0117 -
0.0109 7350 0.0894 -
0.0110 7400 0.0101 -
0.0110 7450 0.0066 -
0.0111 7500 0.0347 0.1899
0.0112 7550 0.0893 -
0.0113 7600 0.0127 -
0.0113 7650 0.1285 -
0.0114 7700 0.0049 -
0.0115 7750 0.0571 -
0.0116 7800 0.0068 -
0.0116 7850 0.0586 -
0.0117 7900 0.0788 -
0.0118 7950 0.0655 -
0.0119 8000 0.0052 0.1807
0.0119 8050 0.0849 -
0.0120 8100 0.0133 -
0.0121 8150 0.0445 -
0.0122 8200 0.0118 -
0.0122 8250 0.0118 -
0.0123 8300 0.063 -
0.0124 8350 0.0751 -
0.0124 8400 0.058 -
0.0125 8450 0.002 -
0.0126 8500 0.0058 0.1804
0.0127 8550 0.0675 -
0.0127 8600 0.0067 -
0.0128 8650 0.0087 -
0.0129 8700 0.0028 -
0.0130 8750 0.0626 -
0.0130 8800 0.0563 -
0.0131 8850 0.0012 -
0.0132 8900 0.0067 -
0.0133 8950 0.0011 -
0.0133 9000 0.0105 0.189
0.0134 9050 0.101 -
0.0135 9100 0.1162 -
0.0136 9150 0.0593 -
0.0136 9200 0.0004 -
0.0137 9250 0.0012 -
0.0138 9300 0.0022 -
0.0139 9350 0.0033 -
0.0139 9400 0.0025 -
0.0140 9450 0.0578 -
0.0141 9500 0.0012 0.1967
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.13
  • SetFit: 1.0.3
  • Sentence Transformers: 2.2.2
  • spaCy: 3.7.4
  • Transformers: 4.36.2
  • PyTorch: 2.1.2+cu121
  • Datasets: 2.16.1
  • Tokenizers: 0.15.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}