SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 3 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
1	'I was spending too much money on beer and it wasn’t helping my life in any capacity, so I cut it out. I have enough other expensive hobbies I don’t need liver damage to be one of them.' "And I forgot the worst: eating out is expensive, and beer is crazy expensive. That's really annoying." 'Young me also didn’t realize a few ballpark beers could have you reevaluating your monthly budget'
2	'Mental health problems and obesity often go hand in hand. In particular depression can be countered through endorphines released through simple Workouts including (!) normal paced walking outside. I do factor these things in. But if you eat unhealthy, only sit at home in the shadow, smoke tobacco or even worse weed with the occasional beer, you do not give yourself a fighting chance. There are exceptions. Yes.' 'It's also essential for vitamin d Alcohol isn't essential and has no positive health outcomes. Even when you consider "getting together with the boys" as a positive mental health aspect, it's negated by all the other effects. I still have a few beer a week, but I'm aware of its consequences' "I drink on SSRI but I know two things. If I drink a lot the other day my anxiety is hell and I have to double the dose of my anti anxiety meds so I do it only if I don't have to do anything important the other day and veeeery occasionally. If I occasionally drink one or two beers yes it hits me more hard, I used to be that kind of person who needed a lot of alcohol to feel the high and now with one glass of wine I feel it, but I don't have any problem the other day. Be careful because the first time I found out the first thing I hated myself, I had to sleep all day to get through the hangxiety"
0	"I'm not sure if that's actually true (maybe I'm wrong) cause with the exception of the occasional craft brew, I always found the alcohol level to be the same on both sides of the border. Budweiser down there and Molson up here are both 5%." '?? angolbryggeri - Hazy Crazy\n\n✴️ IPA\n\n?? Sweden ????\n\n??Abv 6.5%\n\n⭐️ 3.60 / 5.0 ~ avg 3.67\n\n?? systembolaget\n\n#beer #bier #birra #öl #cerveza #øl #craftbeer #ipa #dipa #tipa #sour #gose #berlinerweisse #paleale #pilsner #lager #stout #beeroftheday #beerphotografy #hantverksöl #untappd #beergeek #beerlover #ilovebeer #cheers #beerstagram #instabeer #beerporn #ängöl #sweden' 'Lately some popular breweries around me have catered to lighter beers away from mostly pales les. Hefeweizens, Pilsner’s, blondes, and it’s been really nice. My local had 3 awesome pilsners that taste straight out of Europe.'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("bhaskars113/beer-budget-health-model")
# Run inference
preds = model("By my house pizza is pretty inexpensive. I might be able to get two cheap beers too!")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	12	50.7391	177

Label	Training Sample Count
0	16
1	15
2	15

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0087	1	0.203	-
0.4348	50	0.003	-
0.8696	100	0.0007	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 2.3.1
Transformers: 4.35.2
PyTorch: 2.1.0+cu121
Datasets: 2.17.1
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

bhaskars113
/

beer-budget-health-model