SetFit with jinaai/jina-embeddings-v2-small-en

This is a SetFit model that can be used for Text Classification. This SetFit model uses jinaai/jina-embeddings-v2-small-en as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: jinaai/jina-embeddings-v2-small-en
Classification head: a LogisticRegression instance
Maximum Sequence Length: 8192 tokens
Number of Classes: 13 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
Aims	'This study aims to provide an in-depth analysis of the impact of Coronavirus Disease 2019 (COVID-19) on Italy, focusing on the early stages of the outbreak and the subsequent government response.' 'In this paper, we propose SegNet, a deep convolutional encoder-decoder architecture for real-time image segmentation.' 'This study aims to develop a mathematical model for analyzing genetic variation using restriction endonucleases.'
Background	'Previous studies have demonstrated that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, the efficacy of pravastatin in patients with average cholesterol levels is less clear.' 'Previous studies have shown that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, this study investigates the effect of pravastatin on patients with average cholesterol levels.' 'Previous studies have shown that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, this trial investigates the effect of pravastatin on patients with average cholesterol levels.'
Hypothesis	'Despite having average cholesterol levels, patients who received Pravastatin experienced a significant reduction in coronary events, suggesting a potential role for statins in preventing cardiovascular events beyond cholesterol level management in internal medicine.' 'This prospective observational study aimed to investigate the association between glycaemia levels and the risk of developing macrovascular and microvascular complications in individuals with type 2 diabetes, as previously identified in the UKPDS 35 study.' 'The results suggest that self-regulatory skills, particularly in the area of attention, significantly impact academic performance in elementary school students.'
Implications	'From 1995 to 1998, the UK Prospective Diabetes Study (UKPDS) 35 observed a significant association between higher glycaemia levels and increased risk of both macrovascular and microvascular complications in patients with type 2 diabetes.' 'The UKPDS 35 study provides robust evidence that every 1 mmol/L increase in HbA1c is associated with a 25% increased risk of macrovascular events and a 37% increased risk of microvascular complications in patients with type 2 diabetes, highlighting the importance of strict glycaemic control in internal medicine.' "This study provides valuable insights into the early dynamics of the COVID-19 outbreak in Italy, contributing to the understanding of the disease's transmission patterns and impact on public health."
Importance	'Stroke and transient ischemic attack (TIA) are leading causes of long-term disability and mortality in internal medicine, with an estimated 15 million survivors worldwide.' 'The accurate assessment of insulin resistance and beta-cell function is crucial in the diagnosis and management of various metabolic disorders, including type 2 diabetes and metabolic syndrome.' 'The COVID-19 outbreak in Italy, which began in late February 2020, quickly became one of the most severe epidemic hotspots in Europe.'
Keywords	'Pravastatin is a statin drug commonly used in the treatment of hypercholesterolemia, specifically to lower low-density lipoprotein (LDL) cholesterol levels and reduce the risk of cardiovascular events in internal medicine.' 'Self-regulation refers to the ability of students to manage their emotions, behavior, and cognitive processes to achieve optimal learning (Zimmerman & Kitsantas, 2005).' 'The proposed method utilizes deep convolutional neural networks to extract rich features from input images, enabling both object detection and semantic segmentation with high accuracy in the field of artificial intelligence.'
Limitations	'However, it is important to note that the Homeostasis Model Assessment (HOMA) index does not directly measure insulin sensitivity or β-cell function, but rather provides an estimate based on fasting plasma glucose and insulin concentrations.' 'Despite providing a useful estimate of insulin resistance and beta-cell function, the Homeostasis Model Assessment has limitations in its applicability to individuals with extreme glucose or insulin levels, as well as those with certain diseases such as liver disease or pregnancy.' 'Despite the large sample size and long follow-up period, the observational nature of the study limits the ability to establish causality between glycaemia and the observed complications in type 2 diabetes.'
Method	'The study employed a randomized, double-blind, placebo-controlled design to investigate the effect of Pravastatin on coronary events in patients with average cholesterol levels.' 'Patients with a history of myocardial infarction and an average cholesterol level between 180 and 240 mg/dL were included in the study.' 'The study aimed to assess the impact of Pravastatin administration on the incidence of coronary events in internal medicine patients with average cholesterol levels.'
None	'The study enrolled patients with a recent myocardial infarction and an average cholesterol level, who were then randomly assigned to receive either pravastatin or placebo.' 'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in the field of internal medicine.' 'This study aims to evaluate the effectiveness of the Homeostasis Model Assessment (HOMA) in estimating insulin resistance and pancreatic beta-cell function in internal medicine, offering valuable insights for the diagnosis and management of metabolic disorders.'
Purpose	'This study investigates the impact of Pravastatin on reducing coronary events in internal medicine patients with average cholesterol levels after a myocardial infarction.' 'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in internal medicine.' 'This study aims to evaluate the effectiveness of the Homeostasis Model Assessment (HOMA) in estimating insulin resistance and beta-cell function in internal medicine patients, addressing the need for a simple and widely applicable method for diagnosing and monitoring these conditions.'
Reccomendations	'Further studies are needed to investigate the optimal duration of dual antiplatelet therapy in secondary prevention of stroke and transient ischemic attack, as well as the role of individual patient characteristics in determining the most effective treatment regimen.' 'Further research is warranted to explore the underlying mechanisms linking glycaemia to macrovascular and microvascular complications in type 2 diabetes, particularly in multi-ethnic populations.' 'Further studies are needed to investigate the potential role of IL-6 signaling in the prevention of bone loss in postmenopausal women.'
Result	'Despite having average cholesterol levels, patients treated with Pravastatin did not experience a significant reduction in coronary events compared to the placebo group.' 'In interviews with patients who experienced a reduction in coronary events after Pravastatin treatment, themes included improved energy levels and increased confidence in managing their heart health.' 'The study found that Pravastatin significantly reduced the risk of coronary events in patients with average cholesterol levels, consistent with previous research suggesting that statins benefit a wider population beyond those with hypercholesterolemia.'
Uncertainty	'Despite the widespread use of pravastatin in post-myocardial infarction patients with average cholesterol levels, the evidence regarding its impact on coronary events remains inconclusive and sometimes contradictory.' 'Despite the findings of this study showing a reduction in coronary events with Pravastatin use in patients with average cholesterol levels, contrasting evidence exists suggesting no significant benefit in similar patient populations (Miller et al., 2018).' 'Despite the proven benefits of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of cardiovascular events, particularly in coronary artery disease, there is a paucity of data specifically addressing its use in stroke or transient ischemic attack (TIA) patients.'

Evaluation

Metrics

Label	Accuracy
all	0.8492

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Corran/SciGenSetfit3")
# Run inference
preds = model("This paper focuses on mining association rules between sets of items in large databases, which can reveal interesting patterns and relationships among the data.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	11	28.3123	71

Label	Training Sample Count
Aims	200
Background	200
Hypothesis	200
Implications	200
Importance	200
Keywords	200
Limitations	200
Method	200
None	200
Purpose	200
Reccomendations	200
Result	200
Uncertainty	200

Training Hyperparameters

batch_size: (256, 256)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 40
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0025	1	0.2913	-
0.1229	50	0.2365	-
0.2457	100	0.185	-
0.3686	150	0.159	-
0.4914	200	0.1456	-
0.6143	250	0.1658	-
0.7371	300	0.1189	-
0.8600	350	0.1235	-
0.9828	400	0.1282	-
0.0049	1	0.1257	-
0.0615	50	0.1371	-
0.1230	100	0.1226	-
0.1845	150	0.1099	-
0.2460	200	0.0897	-
0.3075	250	0.1009	-
0.3690	300	0.0659	-
0.4305	350	0.0711	-
0.4920	400	0.0745	-
0.5535	450	0.0807	-
0.6150	500	0.0736	-
0.6765	550	0.0571	-
0.7380	600	0.0649	-
0.7995	650	0.0672	-
0.8610	700	0.0586	-
0.9225	750	0.0624	-
0.9840	800	0.0614	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 2.2.2
Transformers: 4.36.2
PyTorch: 2.1.0+cu121
Datasets: 2.16.1
Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Corran
/

SciGenSetfit3