Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,271 @@
+---
+base_model: sentence-transformers/all-mpnet-base-v2
+library_name: setfit
+metrics:
+- accuracy
+pipeline_tag: text-classification
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget:
+- text: 'I have alcoholism, drug abouse and suicide all over my family as far back
+    as three generations. After seeing several friend in college (class of ''76, U
+    of Arkansas, Go Hogs!) get blind drunk and  raped at frat parties, I decided I
+    could live without it. And I have -- even through five years active duty in the
+    army. I cook with wine and my husband likes a daily beer in the summer. I haven''t
+    missed a thing, I''m height-weight proportionate and probably a few pesos richer
+    for not having squandered money on booze. I live outside the US and I''ve seen
+    dozens of women battered beyond recognition by drunk husbands, children neglected
+    by their parents almost to a point of starvation, and families ruptured and ruined
+    by alcohol. It ain''t worth it.
+
+    '
+- text: 'The War Between the Catholic Cardinals Two essays make plain the different
+    views often obscured by careful political maneuvering within the church. The death
+    of the pope emeritus, Benedict XVI, was succeeded by a small literary outpouring,
+    a rush of publications that were interpreted as salvos in the Catholic Church’s
+    civil war. The list includes a memoir by Benedict’s longtime secretary that mentioned
+    the former pontiff’s disappointment at his successor’s restriction of the Latin
+    Mass, a posthumous essay collection by Benedict himself that’s being mined for
+    controversial quotes, and an Associated Press interview with Pope Francis that
+    made news for its call to decriminalize homosexuality around the world. Two essays
+    make plain the different views often obscured by careful political maneuvering
+    within the church.
+
+    '
+- text: '"As one of the 100,000 or so Catholics in this country who attend the old
+    Mass each week, I will always be grateful to him for allowing for its widespread
+    celebration despite the promulgation of a new, vernacular liturgy."This really
+    says it all. "Soren Kierkegaard?"? Really? Mr. Walther may be the editor of "The
+    Lamp," but his lamp sheds no light on Ratzinger or the fundamental evils of the
+    continuous and painfully slow downward spiral that has been the trajectory of
+    the papacy and the Roman Catholic Church for a very long time.Vatican II opened
+    up a great hope. Ratzinger and his followers saw in it only a threat to the cult
+    of secrecy, both of the sacraments, and the sins. They have done much to unravel
+    the all of the inherent good of Vatican II -- which actually made Catholicism
+    interesting and meaningful to youths at a time of great cynicism in the world.
+    Walther and his 100,000 should form their own 4th century Catholic schism, "despite
+    the promulgation of a new, vernacular liturgy," and leave what''s left of the
+    Catholic church alone to re-build.
+
+    '
+- text: 'Benedict, the reluctant popeThe former Cardinal Ratzinger had never wanted
+    to be pope, planning at age 78 to spend his final years writing in the “peace
+    and quiet” of his native Bavaria.Instead, he was forced to follow the footsteps
+    of the beloved St. John Paul II and run the church through the fallout of the
+    clerical sex abuse scandal Being elected pope, he once said, felt like a “guillotine”
+    had come down on him. Nevertheless, he set about the job with a single-minded
+    vision to rekindle the faith in a world that, he frequently lamented, seemed to
+    think it could do without God.“In vast areas of the world today, there is a strange
+    forgetfulness of God,” he told one million young people gathered on a vast field
+    for his first foreign trip as pope, to World Youth Day in Cologne, Germany, in
+    2005. “It seems as if everything would be just the same even without him.”With
+    some decisive, .. he tried to remind Europe of its Christian heritage. And he
+    set the Catholic Church on a conservative, tradition-minded path that often alienated
+    progressives. He relaxed the restrictions on celebrating the old Latin Mass. It
+    was a path that in many ways was reversed by his successor, Francis, whose mercy-over-morals
+    priorities alienated the traditionalists Benedict’s style couldn’t have been more
+    different from that of Francis. No globe-trotting media darling or populist, Benedict
+    was a teacher, theologian and academic to the core: quiet and pensive with a fierce
+    mind. El Pais Dec
+
+    '
+- text: 'Willy Stone  Wouldn''t it be a pity if all ancient art could only be seen
+    in the location where it was made?
+
+    '
+inference: true
+model-index:
+- name: SetFit with sentence-transformers/all-mpnet-base-v2
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 1.0
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/all-mpnet-base-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 384 tokens
+- **Number of Classes:** 2 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+|:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| yes   | <ul><li>'The First Afterlife of Pope Benedict XVI Pope Benedict’s legacy will be felt across decades or even centuries.  The first pope to resign was Celestine V, born Pietro Da Morrone, who was living the life of a pious hermit when he was elevated to the papacy in 1294, in his 80s, to break a two-year deadlock in the College of Cardinals. Feeling overmastered by the job, he soon resigned in the expectation that he could return to his monastic existence. Instead, he was imprisoned by his successor, Boniface VIII, who feared that some rival faction might make Celestine an antipope. Pope Benedict’s legacy will be felt across decades or even centuries.\n'</li><li>'Here is the statement on Ratzinger\'s death from SNAP, an organization representing victims of abuse from the Catholic Church: "In our view, the death of Pope Benedict XVI is a reminder that, much like John Paul II, Benedict was more concerned about the church’s deteriorating image and financial flow to the hierarchy versus grasping the concept of true apologies followed by true amends to victims of abuse. The rot of clergy sexual abuse of children and adults, even their own professed religious, runs throughout the Catholic church, to every country, and we now have incontrovertible evidence, all the way to the top.Any celebration that marks the life of abuse enablers like Benedict must end. It is past time for the Vatican to refocus on change: tell the truth about known abusive clergy, protect children and adults, and allow justice to those who have been hurt. Honoring Pope Benedict XVI now is not only wrong. It is shameful.It is almost a year after a report into decades of abuse allegations by a law firm in Germany has shown that Pope Benedict XVI did not take action against abusive priests in four child abuse cases while he was Archbishop (Josef Ratzinger). In our view, Pope Benedict XVI is taking decades of the church’s darkest secrets to his grave with him..."<a href="https://www.snapnetwork.org/snap_reacts_to_the_death_of_pope_benedict_xvi" target="_blank">https://www.snapnetwork.org/snap_reacts_to_the_death_of_pope_benedict_xvi</a>\n'</li><li>'I found the statement "While Benedict felt that celebrations of the new Mass were frequently unedifying and even banal..." to be flawed when compared to the post-Vatican II church in the USA.  Where was  Benedict when Catholic churches in the US held "folk masses" using guitars and keyboard instruments instead of organs?  My confirmation ceremony in 1970 in NJ was held that way and one I remember clearly to this very day.If Benedict was really looking for the "cosmic" dimension of the liturgy, maybe he should have attended Maronite Catholic Masses on a regular basis.  At the very least, he would have observed the Maronites\' liturgical rituals which date back into the First Century A.D., not to mention the use of Aramaic, the language of Christ, during the consecration of the Eucharist.\n'</li></ul> |
+| no    | <ul><li>'I’m not a Rogan fan, but this is just a few adventurers adventuring. As long as they are experienced divers with the proper equipment and aware of the dangers, who knows what they might find at the bottom of the East River? Inquiring minds want to know.\n'</li><li>'Mike DiNovi One thing we can agree on is that, depending on our individual backgrounds, there are for each of us, a whole treasure trove of "missing" words. Often but not always, these words may be found in the crosswords, particularly older crosswords. But to ask for all of them to be included here would be asking to change the whole gestalt, as well as immodest. But I think longtime players of the Bee still see some value and delight in posting missing words. It may not be included in the official list but the Hive gives them currency. I personally enjoy all the "missing" word posts, however redundant they often are. I find in them some commonality and I have learned also probably several dozen new words - unofficial and official - including many chemistry words. I was a lousy chemistry student but I absolutely love the vocabulary of it.\n'</li><li>'If "work on what comes next" and "innovation" means expanding the definition of "life" to mean all stages of life from womb to tomb (e.g., paternal leave, pre-school, school choice, basic universal healthcare, baby bonds, etc.) that would be a positive and hopefully inclusive step forward that might bridge the awful divide we see across the country and even in the comments of this article.\n'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 1.0      |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("davidadamczyk/setfit-model-10")
+# Run inference
+preds = model("Willy Stone  Wouldn't it be a pity if all ancient art could only be seen in the location where it was made?
+")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 15  | 123.625 | 286 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| no    | 18                    |
+| yes   | 22                    |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 120
+- body_learning_rate: (2e-05, 2e-05)
+- head_learning_rate: 2e-05
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- l2_weight: 0.01
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0017 | 1    | 0.365         | -               |
+| 0.0833 | 50   | 0.1213        | -               |
+| 0.1667 | 100  | 0.0018        | -               |
+| 0.25   | 150  | 0.0004        | -               |
+| 0.3333 | 200  | 0.0002        | -               |
+| 0.4167 | 250  | 0.0002        | -               |
+| 0.5    | 300  | 0.0001        | -               |
+| 0.5833 | 350  | 0.0001        | -               |
+| 0.6667 | 400  | 0.0001        | -               |
+| 0.75   | 450  | 0.0001        | -               |
+| 0.8333 | 500  | 0.0001        | -               |
+| 0.9167 | 550  | 0.0001        | -               |
+| 1.0    | 600  | 0.0001        | -               |
+
+### Framework Versions
+- Python: 3.10.13
+- SetFit: 1.1.0
+- Sentence Transformers: 3.0.1
+- Transformers: 4.45.2
+- PyTorch: 2.4.0+cu124
+- Datasets: 2.21.0
+- Tokenizers: 0.20.0
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "sentence-transformers/all-mpnet-base-v2",
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.45.2",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.45.2",
+    "pytorch": "2.4.0+cu124"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

config_setfit.json

@@ -0,0 +1,7 @@
+{
+  "normalize_embeddings": false,
+  "labels": [
+    "no",
+    "yes"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acf0e4c3ff98b99796ff63f901b966b7f2c1bab776f52290682de1b0ed24d6b3
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f138402e3ef1afe4f71f1b5c77a69a3a7d93f5972cb66a77dfb0f38ac5e5925
+size 7023

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 384,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,72 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "max_length": 128,
+  "model_max_length": 384,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}