Commit
•
6d5161a
1
Parent(s):
82f0150
Update README.md
Browse files
README.md
CHANGED
@@ -1,70 +1,508 @@
|
|
1 |
---
|
2 |
-
|
3 |
-
|
4 |
tags:
|
5 |
-
-
|
6 |
-
|
7 |
-
-
|
8 |
-
|
9 |
-
|
10 |
-
results: []
|
11 |
---
|
12 |
|
13 |
-
|
14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
-
|
17 |
|
18 |
-
|
19 |
-
|
20 |
-
- Loss: 0.1987
|
21 |
-
- F1 Macro: 0.6760
|
22 |
-
- F1 Micro: 0.7335
|
23 |
-
- Accuracy Balanced: 0.7247
|
24 |
-
- Accuracy: 0.7335
|
25 |
-
- Precision Macro: 0.6872
|
26 |
-
- Recall Macro: 0.7247
|
27 |
-
- Precision Micro: 0.7335
|
28 |
-
- Recall Micro: 0.7335
|
29 |
|
30 |
-
|
|
|
|
|
|
|
31 |
|
32 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
-
|
|
|
35 |
|
36 |
-
|
|
|
37 |
|
38 |
-
|
39 |
|
40 |
-
|
|
|
|
|
|
|
|
|
41 |
|
42 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
43 |
|
44 |
-
###
|
|
|
45 |
|
46 |
-
|
47 |
-
-
|
48 |
-
- train_batch_size: 32
|
49 |
-
- eval_batch_size: 128
|
50 |
-
- seed: 42
|
51 |
-
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
52 |
-
- lr_scheduler_type: linear
|
53 |
-
- lr_scheduler_warmup_ratio: 0.06
|
54 |
-
- num_epochs: 3
|
55 |
|
56 |
-
###
|
|
|
|
|
|
|
57 |
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
|
|
|
|
|
|
|
|
|
63 |
|
|
|
64 |
|
65 |
-
### Framework versions
|
66 |
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
tags:
|
5 |
+
- text-classification
|
6 |
+
- zero-shot-classification
|
7 |
+
pipeline_tag: zero-shot-classification
|
8 |
+
library_name: transformers
|
9 |
+
license: mit
|
|
|
10 |
---
|
11 |
|
12 |
+
# Model description: deberta-v3-base-zeroshot-v1.1-all-33
|
13 |
+
The model is designed for zero-shot classification with the Hugging Face pipeline.
|
14 |
+
|
15 |
+
The model can do one universal task: determine whether a hypothesis is `true` or `not_true`
|
16 |
+
given a text (also called `entailment` vs. `not_entailment`).
|
17 |
+
This task format is based on the Natural Language Inference task (NLI).
|
18 |
+
The task is so universal that any classification task can be reformulated into the task.
|
19 |
+
|
20 |
+
A detailed description of how the model was trained and how it can be used is available in this paper: [link to be added]
|
21 |
+
|
22 |
+
## Training data
|
23 |
+
The model was trained on a mixture of __33 datasets and 387 classes__ that have been reformatted into this universal format.
|
24 |
+
1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli", "ling"
|
25 |
+
2. 28 classification tasks reformatted into the universal NLI format. ~51k cleaned texts were used to avoid overfitting:
|
26 |
+
'amazonpolarity', 'imdb', 'appreviews', 'yelpreviews', 'rottentomatoes',
|
27 |
+
'emotiondair', 'emocontext', 'empathetic',
|
28 |
+
'financialphrasebank', 'banking77', 'massive',
|
29 |
+
'wikitoxic_toxicaggregated', 'wikitoxic_obscene', 'wikitoxic_threat', 'wikitoxic_insult', 'wikitoxic_identityhate',
|
30 |
+
'hateoffensive', 'hatexplain', 'biasframes_offensive', 'biasframes_sex', 'biasframes_intent',
|
31 |
+
'agnews', 'yahootopics',
|
32 |
+
'trueteacher', 'spam', 'wellformedquery',
|
33 |
+
'manifesto', 'capsotu'.
|
34 |
|
35 |
+
See details on each dataset here: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv
|
36 |
|
37 |
+
Note that compared to other NLI models, this model predicts two classes (`entailment` vs. `not_entailment`)
|
38 |
+
as opposed to three classes (entailment/neutral/contradiction)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
39 |
|
40 |
+
The model was only trained on English data. For __multilingual use-cases__,
|
41 |
+
I recommend machine translating texts to English with libraries like [EasyNMT](https://github.com/UKPLab/EasyNMT).
|
42 |
+
English-only models tend to perform better than multilingual models and
|
43 |
+
validation with English data can be easier if you don't speak all languages in your corpus.
|
44 |
|
45 |
+
### How to use the model
|
46 |
+
#### Simple zero-shot classification pipeline
|
47 |
+
```python
|
48 |
+
from transformers import pipeline
|
49 |
+
text = "Angela Merkel is a politician in Germany and leader of the CDU"
|
50 |
+
hypothesis_template = "This example is about {}"
|
51 |
+
classes_verbalized = ["politics", "economy", "entertainment", "environment"]
|
52 |
+
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33")
|
53 |
+
output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False)
|
54 |
+
print(output)
|
55 |
+
```
|
56 |
|
57 |
+
### Details on data and training
|
58 |
+
The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
|
59 |
|
60 |
+
## Limitations and bias
|
61 |
+
The model can only do text classification tasks.
|
62 |
|
63 |
+
Please consult the original DeBERTa paper and the papers for the different datasets for potential biases.
|
64 |
|
65 |
+
## License
|
66 |
+
The base model (DeBERTa-v3) is published under the MIT license.
|
67 |
+
The datasets the model was fine-tuned on are published under a diverse set of licenses.
|
68 |
+
The following spreadsheet provides an overview of the non-NLI datasets used for fine-tuning.
|
69 |
+
The spreadsheets contains information on licenses, the underlying papers etc.: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv
|
70 |
|
71 |
+
## Citation
|
72 |
+
If you use this model academically, please cite:
|
73 |
+
```
|
74 |
+
@article{laurer_less_2023,
|
75 |
+
title = {Less {Annotating}, {More} {Classifying}: {Addressing} the {Data} {Scarcity} {Issue} of {Supervised} {Machine} {Learning} with {Deep} {Transfer} {Learning} and {BERT}-{NLI}},
|
76 |
+
issn = {1047-1987, 1476-4989},
|
77 |
+
shorttitle = {Less {Annotating}, {More} {Classifying}},
|
78 |
+
url = {https://www.cambridge.org/core/product/identifier/S1047198723000207/type/journal_article},
|
79 |
+
doi = {10.1017/pan.2023.20},
|
80 |
+
language = {en},
|
81 |
+
urldate = {2023-06-20},
|
82 |
+
journal = {Political Analysis},
|
83 |
+
author = {Laurer, Moritz and Van Atteveldt, Wouter and Casas, Andreu and Welbers, Kasper},
|
84 |
+
month = jun,
|
85 |
+
year = {2023},
|
86 |
+
pages = {1--33},
|
87 |
+
}
|
88 |
+
```
|
89 |
|
90 |
+
### Ideas for cooperation or questions?
|
91 |
+
If you have questions or ideas for cooperation, contact me at m{dot}laurer{at}vu{dot}nl or [LinkedIn](https://www.linkedin.com/in/moritz-laurer/)
|
92 |
|
93 |
+
### Debugging and issues
|
94 |
+
Note that DeBERTa-v3 was released on 06.12.21 and older versions of HF Transformers can have issues running the model (e.g. resulting in an issue with the tokenizer). Using Transformers>=4.13 might solve some issues.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
95 |
|
96 |
+
### Hypotheses used for classification
|
97 |
+
The hypotheses in the tables below were used to fine-tune the model.
|
98 |
+
Inspecting them can help users get a feeling for which type of hypotheses and tasks the model was trained on.
|
99 |
+
You can formulate your own hypotheses by changing the `hypothesis_template` of the zeroshot pipeline. For example:
|
100 |
|
101 |
+
```python
|
102 |
+
from transformers import pipeline
|
103 |
+
text = "Angela Merkel is a politician in Germany and leader of the CDU"
|
104 |
+
hypothesis_template = "Merkel is the leader of the party: {}"
|
105 |
+
classes_verbalized = ["CDU", "SPD", "Greens"]
|
106 |
+
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33")
|
107 |
+
output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False)
|
108 |
+
print(output)
|
109 |
+
```
|
110 |
|
111 |
+
Note that a few rows in the `massive` and `banking77` datasets contain `nan` because some classes were so ambiguous/unclear that I excluded them from the data.
|
112 |
|
|
|
113 |
|
114 |
+
#### wellformedquery
|
115 |
+
| label | hypothesis |
|
116 |
+
|:----------------|:-----------------------------------------------|
|
117 |
+
| not_well_formed | This example is not a well formed Google query |
|
118 |
+
| well_formed | This example is a well formed Google query. |
|
119 |
+
#### biasframes_sex
|
120 |
+
| label | hypothesis |
|
121 |
+
|:--------|:-----------------------------------------------------------|
|
122 |
+
| not_sex | This example does not contain allusions to sexual content. |
|
123 |
+
| sex | This example contains allusions to sexual content. |
|
124 |
+
#### biasframes_intent
|
125 |
+
| label | hypothesis |
|
126 |
+
|:-----------|:-----------------------------------------------------------------|
|
127 |
+
| intent | The intent of this example is to be offensive/disrespectful. |
|
128 |
+
| not_intent | The intent of this example is not to be offensive/disrespectful. |
|
129 |
+
#### biasframes_offensive
|
130 |
+
| label | hypothesis |
|
131 |
+
|:--------------|:-------------------------------------------------------------------------|
|
132 |
+
| not_offensive | This example could not be considered offensive, disrespectful, or toxic. |
|
133 |
+
| offensive | This example could be considered offensive, disrespectful, or toxic. |
|
134 |
+
#### financialphrasebank
|
135 |
+
| label | hypothesis |
|
136 |
+
|:---------|:--------------------------------------------------------------------------|
|
137 |
+
| negative | The sentiment in this example is negative from an investor's perspective. |
|
138 |
+
| neutral | The sentiment in this example is neutral from an investor's perspective. |
|
139 |
+
| positive | The sentiment in this example is positive from an investor's perspective. |
|
140 |
+
#### rottentomatoes
|
141 |
+
| label | hypothesis |
|
142 |
+
|:---------|:-----------------------------------------------------------------------|
|
143 |
+
| negative | The sentiment in this example rotten tomatoes movie review is negative |
|
144 |
+
| positive | The sentiment in this example rotten tomatoes movie review is positive |
|
145 |
+
#### amazonpolarity
|
146 |
+
| label | hypothesis |
|
147 |
+
|:---------|:----------------------------------------------------------------|
|
148 |
+
| negative | The sentiment in this example amazon product review is negative |
|
149 |
+
| positive | The sentiment in this example amazon product review is positive |
|
150 |
+
#### imdb
|
151 |
+
| label | hypothesis |
|
152 |
+
|:---------|:------------------------------------------------------------|
|
153 |
+
| negative | The sentiment in this example imdb movie review is negative |
|
154 |
+
| positive | The sentiment in this example imdb movie review is positive |
|
155 |
+
#### appreviews
|
156 |
+
| label | hypothesis |
|
157 |
+
|:---------|:------------------------------------------------------|
|
158 |
+
| negative | The sentiment in this example app review is negative. |
|
159 |
+
| positive | The sentiment in this example app review is positive. |
|
160 |
+
#### yelpreviews
|
161 |
+
| label | hypothesis |
|
162 |
+
|:---------|:-------------------------------------------------------|
|
163 |
+
| negative | The sentiment in this example yelp review is negative. |
|
164 |
+
| positive | The sentiment in this example yelp review is positive. |
|
165 |
+
#### wikitoxic_toxicaggregated
|
166 |
+
| label | hypothesis |
|
167 |
+
|:--------------------|:----------------------------------------------------------------|
|
168 |
+
| not_toxicaggregated | This example wikipedia comment does not contain toxic language. |
|
169 |
+
| toxicaggregated | This example wikipedia comment contains toxic language. |
|
170 |
+
#### wikitoxic_obscene
|
171 |
+
| label | hypothesis |
|
172 |
+
|:------------|:------------------------------------------------------------------|
|
173 |
+
| not_obscene | This example wikipedia comment does not contain obscene language. |
|
174 |
+
| obscene | This example wikipedia comment contains obscene language. |
|
175 |
+
#### wikitoxic_threat
|
176 |
+
| label | hypothesis |
|
177 |
+
|:-----------|:----------------------------------------------------------|
|
178 |
+
| not_threat | This example wikipedia comment does not contain a threat. |
|
179 |
+
| threat | This example wikipedia comment contains a threat. |
|
180 |
+
#### wikitoxic_insult
|
181 |
+
| label | hypothesis |
|
182 |
+
|:-----------|:-----------------------------------------------------------|
|
183 |
+
| insult | This example wikipedia comment contains an insult. |
|
184 |
+
| not_insult | This example wikipedia comment does not contain an insult. |
|
185 |
+
#### wikitoxic_identityhate
|
186 |
+
| label | hypothesis |
|
187 |
+
|:-----------------|:---------------------------------------------------------------|
|
188 |
+
| identityhate | This example wikipedia comment contains identity hate. |
|
189 |
+
| not_identityhate | This example wikipedia comment does not contain identity hate. |
|
190 |
+
#### hateoffensive
|
191 |
+
| label | hypothesis |
|
192 |
+
|:------------|:------------------------------------------------------------------------|
|
193 |
+
| hate_speech | This example tweet contains hate speech. |
|
194 |
+
| neither | This example tweet contains neither offensive language nor hate speech. |
|
195 |
+
| offensive | This example tweet contains offensive language without hate speech. |
|
196 |
+
#### hatexplain
|
197 |
+
| label | hypothesis |
|
198 |
+
|:------------|:-------------------------------------------------------------------------------------------|
|
199 |
+
| hate_speech | This example text from twitter or gab contains hate speech. |
|
200 |
+
| neither | This example text from twitter or gab contains neither offensive language nor hate speech. |
|
201 |
+
| offensive | This example text from twitter or gab contains offensive language without hate speech. |
|
202 |
+
#### spam
|
203 |
+
| label | hypothesis |
|
204 |
+
|:---------|:------------------------------|
|
205 |
+
| not_spam | This example sms is not spam. |
|
206 |
+
| spam | This example sms is spam. |
|
207 |
+
#### emotiondair
|
208 |
+
| label | hypothesis |
|
209 |
+
|:---------|:---------------------------------------------------|
|
210 |
+
| anger | This example tweet expresses the emotion: anger |
|
211 |
+
| fear | This example tweet expresses the emotion: fear |
|
212 |
+
| joy | This example tweet expresses the emotion: joy |
|
213 |
+
| love | This example tweet expresses the emotion: love |
|
214 |
+
| sadness | This example tweet expresses the emotion: sadness |
|
215 |
+
| surprise | This example tweet expresses the emotion: surprise |
|
216 |
+
#### emocontext
|
217 |
+
| label | hypothesis |
|
218 |
+
|:--------|:--------------------------------------------------------------------------------------|
|
219 |
+
| angry | This example tweet expresses the emotion: anger |
|
220 |
+
| happy | This example tweet expresses the emotion: happiness |
|
221 |
+
| others | This example tweet does not express any of the emotions: anger, sadness, or happiness |
|
222 |
+
| sad | This example tweet expresses the emotion: sadness |
|
223 |
+
#### empathetic
|
224 |
+
| label | hypothesis |
|
225 |
+
|:-------------|:-----------------------------------------------------------|
|
226 |
+
| afraid | The main emotion of this example dialogue is: afraid |
|
227 |
+
| angry | The main emotion of this example dialogue is: angry |
|
228 |
+
| annoyed | The main emotion of this example dialogue is: annoyed |
|
229 |
+
| anticipating | The main emotion of this example dialogue is: anticipating |
|
230 |
+
| anxious | The main emotion of this example dialogue is: anxious |
|
231 |
+
| apprehensive | The main emotion of this example dialogue is: apprehensive |
|
232 |
+
| ashamed | The main emotion of this example dialogue is: ashamed |
|
233 |
+
| caring | The main emotion of this example dialogue is: caring |
|
234 |
+
| confident | The main emotion of this example dialogue is: confident |
|
235 |
+
| content | The main emotion of this example dialogue is: content |
|
236 |
+
| devastated | The main emotion of this example dialogue is: devastated |
|
237 |
+
| disappointed | The main emotion of this example dialogue is: disappointed |
|
238 |
+
| disgusted | The main emotion of this example dialogue is: disgusted |
|
239 |
+
| embarrassed | The main emotion of this example dialogue is: embarrassed |
|
240 |
+
| excited | The main emotion of this example dialogue is: excited |
|
241 |
+
| faithful | The main emotion of this example dialogue is: faithful |
|
242 |
+
| furious | The main emotion of this example dialogue is: furious |
|
243 |
+
| grateful | The main emotion of this example dialogue is: grateful |
|
244 |
+
| guilty | The main emotion of this example dialogue is: guilty |
|
245 |
+
| hopeful | The main emotion of this example dialogue is: hopeful |
|
246 |
+
| impressed | The main emotion of this example dialogue is: impressed |
|
247 |
+
| jealous | The main emotion of this example dialogue is: jealous |
|
248 |
+
| joyful | The main emotion of this example dialogue is: joyful |
|
249 |
+
| lonely | The main emotion of this example dialogue is: lonely |
|
250 |
+
| nostalgic | The main emotion of this example dialogue is: nostalgic |
|
251 |
+
| prepared | The main emotion of this example dialogue is: prepared |
|
252 |
+
| proud | The main emotion of this example dialogue is: proud |
|
253 |
+
| sad | The main emotion of this example dialogue is: sad |
|
254 |
+
| sentimental | The main emotion of this example dialogue is: sentimental |
|
255 |
+
| surprised | The main emotion of this example dialogue is: surprised |
|
256 |
+
| terrified | The main emotion of this example dialogue is: terrified |
|
257 |
+
| trusting | The main emotion of this example dialogue is: trusting |
|
258 |
+
#### agnews
|
259 |
+
| label | hypothesis |
|
260 |
+
|:---------|:-------------------------------------------------------|
|
261 |
+
| Business | This example news text is about business news |
|
262 |
+
| Sci/Tech | This example news text is about science and technology |
|
263 |
+
| Sports | This example news text is about sports |
|
264 |
+
| World | This example news text is about world news |
|
265 |
+
#### yahootopics
|
266 |
+
| label | hypothesis |
|
267 |
+
|:-----------------------|:---------------------------------------------------------------------------------------------------|
|
268 |
+
| Business & Finance | This example question from the Yahoo Q&A forum is categorized in the topic: Business & Finance |
|
269 |
+
| Computers & Internet | This example question from the Yahoo Q&A forum is categorized in the topic: Computers & Internet |
|
270 |
+
| Education & Reference | This example question from the Yahoo Q&A forum is categorized in the topic: Education & Reference |
|
271 |
+
| Entertainment & Music | This example question from the Yahoo Q&A forum is categorized in the topic: Entertainment & Music |
|
272 |
+
| Family & Relationships | This example question from the Yahoo Q&A forum is categorized in the topic: Family & Relationships |
|
273 |
+
| Health | This example question from the Yahoo Q&A forum is categorized in the topic: Health |
|
274 |
+
| Politics & Government | This example question from the Yahoo Q&A forum is categorized in the topic: Politics & Government |
|
275 |
+
| Science & Mathematics | This example question from the Yahoo Q&A forum is categorized in the topic: Science & Mathematics |
|
276 |
+
| Society & Culture | This example question from the Yahoo Q&A forum is categorized in the topic: Society & Culture |
|
277 |
+
| Sports | This example question from the Yahoo Q&A forum is categorized in the topic: Sports |
|
278 |
+
#### massive
|
279 |
+
| label | hypothesis |
|
280 |
+
|:-------------------------|:------------------------------------------------------------------------------------------|
|
281 |
+
| alarm_query | The example utterance is a query about alarms. |
|
282 |
+
| alarm_remove | The intent of this example utterance is to remove an alarm. |
|
283 |
+
| alarm_set | The intent of the example utterance is to set an alarm. |
|
284 |
+
| audio_volume_down | The intent of the example utterance is to lower the volume. |
|
285 |
+
| audio_volume_mute | The intent of this example utterance is to mute the volume. |
|
286 |
+
| audio_volume_other | The example utterance is related to audio volume. |
|
287 |
+
| audio_volume_up | The intent of this example utterance is turning the audio volume up. |
|
288 |
+
| calendar_query | The example utterance is a query about a calendar. |
|
289 |
+
| calendar_remove | The intent of the example utterance is to remove something from a calendar. |
|
290 |
+
| calendar_set | The intent of this example utterance is to set something in a calendar. |
|
291 |
+
| cooking_query | The example utterance is a query about cooking. |
|
292 |
+
| cooking_recipe | This example utterance is about cooking recipies. |
|
293 |
+
| datetime_convert | The example utterance is related to date time changes or conversion. |
|
294 |
+
| datetime_query | The intent of this example utterance is a datetime query. |
|
295 |
+
| email_addcontact | The intent of this example utterance is adding an email address to contacts. |
|
296 |
+
| email_query | The example utterance is a query about emails. |
|
297 |
+
| email_querycontact | The intent of this example utterance is to query contact details. |
|
298 |
+
| email_sendemail | The intent of the example utterance is to send an email. |
|
299 |
+
| general_greet | This example utterance is a general greet. |
|
300 |
+
| general_joke | The intent of the example utterance is to hear a joke. |
|
301 |
+
| general_quirky | nan |
|
302 |
+
| iot_cleaning | The intent of the example utterance is for an IoT device to start cleaning. |
|
303 |
+
| iot_coffee | The intent of this example utterance is for an IoT device to make coffee. |
|
304 |
+
| iot_hue_lightchange | The intent of this example utterance is changing the light. |
|
305 |
+
| iot_hue_lightdim | The intent of the example utterance is to dim the lights. |
|
306 |
+
| iot_hue_lightoff | The example utterance is related to turning the lights off. |
|
307 |
+
| iot_hue_lighton | The example utterance is related to turning the lights on. |
|
308 |
+
| iot_hue_lightup | The intent of this example utterance is to brighten lights. |
|
309 |
+
| iot_wemo_off | The intent of this example utterance is turning an IoT device off. |
|
310 |
+
| iot_wemo_on | The intent of the example utterance is to turn an IoT device on. |
|
311 |
+
| lists_createoradd | The example utterance is related to creating or adding to lists. |
|
312 |
+
| lists_query | The example utterance is a query about a list. |
|
313 |
+
| lists_remove | The intent of this example utterance is to remove a list or remove something from a list. |
|
314 |
+
| music_dislikeness | The intent of this example utterance is signalling music dislike. |
|
315 |
+
| music_likeness | The example utterance is related to liking music. |
|
316 |
+
| music_query | The example utterance is a query about music. |
|
317 |
+
| music_settings | The intent of the example utterance is to change music settings. |
|
318 |
+
| news_query | The example utterance is a query about the news. |
|
319 |
+
| play_audiobook | The example utterance is related to playing audiobooks. |
|
320 |
+
| play_game | The intent of this example utterance is to start playing a game. |
|
321 |
+
| play_music | The intent of this example utterance is for an IoT device to play music. |
|
322 |
+
| play_podcasts | The example utterance is related to playing podcasts. |
|
323 |
+
| play_radio | The intent of the example utterance is to play something on the radio. |
|
324 |
+
| qa_currency | This example utteranceis about currencies. |
|
325 |
+
| qa_definition | The example utterance is a query about a definition. |
|
326 |
+
| qa_factoid | The example utterance is a factoid question. |
|
327 |
+
| qa_maths | The example utterance is a question about maths. |
|
328 |
+
| qa_stock | This example utterance is about stocks. |
|
329 |
+
| recommendation_events | This example utterance is about event recommendations. |
|
330 |
+
| recommendation_locations | The intent of this example utterance is receiving recommendations for good locations. |
|
331 |
+
| recommendation_movies | This example utterance is about movie recommendations. |
|
332 |
+
| social_post | The example utterance is about social media posts. |
|
333 |
+
| social_query | The example utterance is a query about a social network. |
|
334 |
+
| takeaway_order | The intent of this example utterance is to order takeaway food. |
|
335 |
+
| takeaway_query | This example utterance is about takeaway food. |
|
336 |
+
| transport_query | The example utterance is a query about transport or travels. |
|
337 |
+
| transport_taxi | The intent of this example utterance is to get a taxi. |
|
338 |
+
| transport_ticket | This example utterance is about transport tickets. |
|
339 |
+
| transport_traffic | This example utterance is about transport or traffic. |
|
340 |
+
| weather_query | This example utterance is a query about the wheather. |
|
341 |
+
#### banking77
|
342 |
+
| label | hypothesis |
|
343 |
+
|:-------------------------------------------------|:----------------------------------------------------------------------------------------------------------|
|
344 |
+
| Refund_not_showing_up | This customer example message is about a refund not showing up. |
|
345 |
+
| activate_my_card | This banking customer example message is about activating a card. |
|
346 |
+
| age_limit | This banking customer example message is related to age limits. |
|
347 |
+
| apple_pay_or_google_pay | This banking customer example message is about apple pay or google pay |
|
348 |
+
| atm_support | This banking customer example message requests ATM support. |
|
349 |
+
| automatic_top_up | This banking customer example message is about automatic top up. |
|
350 |
+
| balance_not_updated_after_bank_transfer | This banking customer example message is about a balance not updated after a transfer. |
|
351 |
+
| balance_not_updated_after_cheque_or_cash_deposit | This banking customer example message is about a balance not updated after a cheque or cash deposit. |
|
352 |
+
| beneficiary_not_allowed | This banking customer example message is related to a beneficiary not being allowed or a failed transfer. |
|
353 |
+
| cancel_transfer | This banking customer example message is related to the cancellation of a transfer. |
|
354 |
+
| card_about_to_expire | This banking customer example message is related to the expiration of a card. |
|
355 |
+
| card_acceptance | This banking customer example message is related to the scope of acceptance of a card. |
|
356 |
+
| card_arrival | This banking customer example message is about the arrival of a card. |
|
357 |
+
| card_delivery_estimate | This banking customer example message is about a card delivery estimate or timing. |
|
358 |
+
| card_linking | nan |
|
359 |
+
| card_not_working | This banking customer example message is about a card not working. |
|
360 |
+
| card_payment_fee_charged | This banking customer example message is about a card payment fee. |
|
361 |
+
| card_payment_not_recognised | This banking customer example message is about a payment the customer does not recognise. |
|
362 |
+
| card_payment_wrong_exchange_rate | This banking customer example message is about a wrong exchange rate. |
|
363 |
+
| card_swallowed | This banking customer example message is about a card swallowed by a machine. |
|
364 |
+
| cash_withdrawal_charge | This banking customer example message is about a cash withdrawal charge. |
|
365 |
+
| cash_withdrawal_not_recognised | This banking customer example message is about an unrecognised cash withdrawal. |
|
366 |
+
| change_pin | This banking customer example message is about changing a pin code. |
|
367 |
+
| compromised_card | This banking customer example message is about a compromised card. |
|
368 |
+
| contactless_not_working | This banking customer example message is about contactless not working |
|
369 |
+
| country_support | This banking customer example message is about country-specific support. |
|
370 |
+
| declined_card_payment | This banking customer example message is about a declined card payment. |
|
371 |
+
| declined_cash_withdrawal | This banking customer example message is about a declined cash withdrawal. |
|
372 |
+
| declined_transfer | This banking customer example message is about a declined transfer. |
|
373 |
+
| direct_debit_payment_not_recognised | This banking customer example message is about an unrecognised direct debit payment. |
|
374 |
+
| disposable_card_limits | This banking customer example message is about the limits of disposable cards. |
|
375 |
+
| edit_personal_details | This banking customer example message is about editing personal details. |
|
376 |
+
| exchange_charge | This banking customer example message is about exchange rate charges. |
|
377 |
+
| exchange_rate | This banking customer example message is about exchange rates. |
|
378 |
+
| exchange_via_app | nan |
|
379 |
+
| extra_charge_on_statement | This banking customer example message is about an extra charge. |
|
380 |
+
| failed_transfer | This banking customer example message is about a failed transfer. |
|
381 |
+
| fiat_currency_support | This banking customer example message is about fiat currency support |
|
382 |
+
| get_disposable_virtual_card | This banking customer example message is about getting a disposable virtual card. |
|
383 |
+
| get_physical_card | nan |
|
384 |
+
| getting_spare_card | This banking customer example message is about getting a spare card. |
|
385 |
+
| getting_virtual_card | This banking customer example message is about getting a virtual card. |
|
386 |
+
| lost_or_stolen_card | This banking customer example message is about a lost or stolen card. |
|
387 |
+
| lost_or_stolen_phone | This banking customer example message is about a lost or stolen phone. |
|
388 |
+
| order_physical_card | This banking customer example message is about ordering a card. |
|
389 |
+
| passcode_forgotten | This banking customer example message is about a forgotten passcode. |
|
390 |
+
| pending_card_payment | This banking customer example message is about a pending card payment. |
|
391 |
+
| pending_cash_withdrawal | This banking customer example message is about a pending cash withdrawal. |
|
392 |
+
| pending_top_up | This banking customer example message is about a pending top up. |
|
393 |
+
| pending_transfer | This banking customer example message is about a pending transfer. |
|
394 |
+
| pin_blocked | This banking customer example message is about a blocked pin. |
|
395 |
+
| receiving_money | This banking customer example message is about receiving money. |
|
396 |
+
| request_refund | This banking customer example message is about a refund request. |
|
397 |
+
| reverted_card_payment? | This banking customer example message is about reverting a card payment. |
|
398 |
+
| supported_cards_and_currencies | nan |
|
399 |
+
| terminate_account | This banking customer example message is about terminating an account. |
|
400 |
+
| top_up_by_bank_transfer_charge | nan |
|
401 |
+
| top_up_by_card_charge | This banking customer example message is about the charge for topping up by card. |
|
402 |
+
| top_up_by_cash_or_cheque | This banking customer example message is about topping up by cash or cheque. |
|
403 |
+
| top_up_failed | This banking customer example message is about top up issues or failures. |
|
404 |
+
| top_up_limits | This banking customer example message is about top up limitations. |
|
405 |
+
| top_up_reverted | This banking customer example message is about issues with topping up. |
|
406 |
+
| topping_up_by_card | This banking customer example message is about topping up by card. |
|
407 |
+
| transaction_charged_twice | This banking customer example message is about a transaction charged twice. |
|
408 |
+
| transfer_fee_charged | This banking customer example message is about an issue with a transfer fee charge. |
|
409 |
+
| transfer_into_account | This banking customer example message is about transfers into the customer's own account. |
|
410 |
+
| transfer_not_received_by_recipient | This banking customer example message is about a transfer that has not arrived yet. |
|
411 |
+
| transfer_timing | This banking customer example message is about transfer timing. |
|
412 |
+
| unable_to_verify_identity | This banking customer example message is about an issue with identity verification. |
|
413 |
+
| verify_my_identity | This banking customer example message is about identity verification. |
|
414 |
+
| verify_source_of_funds | This banking customer example message is about the source of funds. |
|
415 |
+
| verify_top_up | This banking customer example message is about verification and top ups |
|
416 |
+
| virtual_card_not_working | This banking customer example message is about a virtual card not working |
|
417 |
+
| visa_or_mastercard | This banking customer example message is about types of bank cards. |
|
418 |
+
| why_verify_identity | This banking customer example message questions why identity verification is necessary. |
|
419 |
+
| wrong_amount_of_cash_received | This banking customer example message is about a wrong amount of cash received. |
|
420 |
+
| wrong_exchange_rate_for_cash_withdrawal | This banking customer example message is about a wrong exchange rate for a cash withdrawal. |
|
421 |
+
#### trueteacher
|
422 |
+
| label | hypothesis |
|
423 |
+
|:-----------------------|:---------------------------------------------------------------------|
|
424 |
+
| factually_consistent | The example summary is factually consistent with the full article. |
|
425 |
+
| factually_inconsistent | The example summary is factually inconsistent with the full article. |
|
426 |
+
#### capsotu
|
427 |
+
| label | hypothesis |
|
428 |
+
|:----------------------|:----------------------------------------------------------------------------------------------------------|
|
429 |
+
| Agriculture | This example text from a US presidential speech is about agriculture |
|
430 |
+
| Civil Rights | This example text from a US presidential speech is about civil rights or minorities or civil liberties |
|
431 |
+
| Culture | This example text from a US presidential speech is about cultural policy |
|
432 |
+
| Defense | This example text from a US presidential speech is about defense or military |
|
433 |
+
| Domestic Commerce | This example text from a US presidential speech is about banking or finance or commerce |
|
434 |
+
| Education | This example text from a US presidential speech is about education |
|
435 |
+
| Energy | This example text from a US presidential speech is about energy or electricity or fossil fuels |
|
436 |
+
| Environment | This example text from a US presidential speech is about the environment or water or waste or pollution |
|
437 |
+
| Foreign Trade | This example text from a US presidential speech is about foreign trade |
|
438 |
+
| Government Operations | This example text from a US presidential speech is about government operations or administration |
|
439 |
+
| Health | This example text from a US presidential speech is about health |
|
440 |
+
| Housing | This example text from a US presidential speech is about community development or housing issues |
|
441 |
+
| Immigration | This example text from a US presidential speech is about migration |
|
442 |
+
| International Affairs | This example text from a US presidential speech is about international affairs or foreign aid |
|
443 |
+
| Labor | This example text from a US presidential speech is about employment or labour |
|
444 |
+
| Law and Crime | This example text from a US presidential speech is about law, crime or family issues |
|
445 |
+
| Macroeconomics | This example text from a US presidential speech is about macroeconomics |
|
446 |
+
| Public Lands | This example text from a US presidential speech is about public lands or water management |
|
447 |
+
| Social Welfare | This example text from a US presidential speech is about social welfare |
|
448 |
+
| Technology | This example text from a US presidential speech is about space or science or technology or communications |
|
449 |
+
| Transportation | This example text from a US presidential speech is about transportation |
|
450 |
+
#### manifesto
|
451 |
+
| label | hypothesis |
|
452 |
+
|:-------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
453 |
+
| Agriculture and Farmers: Positive | This example text from a political party manifesto is positive towards policies for agriculture and farmers |
|
454 |
+
| Anti-Growth Economy: Positive | This example text from a political party manifesto is in favour of anti-growth politics |
|
455 |
+
| Anti-Imperialism | This example text from a political party manifesto is anti-imperialistic, for example against controlling other countries and for greater self-government of colonies |
|
456 |
+
| Centralisation | This example text from a political party manifesto is in favour of political centralisation |
|
457 |
+
| Civic Mindedness: Positive | This example text from a political party manifesto is positive towards national solidarity, civil society or appeals for public spiritedness or against anti-social attitudes |
|
458 |
+
| Constitutionalism: Negative | This example text from a political party manifesto is positive towards constitutionalism |
|
459 |
+
| Constitutionalism: Positive | This example text from a political party manifesto is positive towards constitutionalism and the status quo of the constitution |
|
460 |
+
| Controlled Economy | This example text from a political party manifesto is supportive of direct government control of the economy, e.g. price control or minimum wages |
|
461 |
+
| Corporatism/Mixed Economy | This example text from a political party manifesto is positive towards cooperation of government, employers, and trade unions simultaneously |
|
462 |
+
| Culture: Positive | This example text from a political party manifesto is in favour of cultural policies or leisure facilities, for example museus, libraries or public sport clubs |
|
463 |
+
| Decentralization | This example text from a political party manifesto is for decentralisation or federalism |
|
464 |
+
| Democracy | This example text from a political party manifesto favourably mentions democracy or democratic procedures or institutions |
|
465 |
+
| Economic Goals | This example text from a political party manifesto is a broad/general statement on economic goals without specifics |
|
466 |
+
| Economic Growth: Positive | This example text from a political party manifesto is supportive of economic growth, for example facilitation of more production or government aid for growth |
|
467 |
+
| Economic Orthodoxy | This example text from a political party manifesto is for economic orthodoxy, for example reduction of budget deficits, thrift or a strong currency |
|
468 |
+
| Economic Planning | This example text from a political party manifesto is positive towards government economic planning, e.g. policy plans or strategies |
|
469 |
+
| Education Expansion | This example text from a political party manifesto is about the need to expand/improve policy on education |
|
470 |
+
| Education Limitation | This example text from a political party manifesto is sceptical towards state expenditure on education, for example in favour of study fees or private schools |
|
471 |
+
| Environmental Protection | This example text from a political party manifesto is in favour of environmental protection, e.g. fighting climate change or 'green' policies or preservation of natural resources or animal rights |
|
472 |
+
| Equality: Positive | This example text from a political party manifesto is positive towards equality or social justice, e.g. protection of underprivileged groups or fair distribution of resources |
|
473 |
+
| European Community/Union: Negative | This example text from a political party manifesto negatively mentions the EU or European Community |
|
474 |
+
| European Community/Union: Positive | This example text from a political party manifesto is positive towards the EU or European Community, for example EU expansion and integration |
|
475 |
+
| Foreign Special Relationships: Negative | This example text from a political party manifesto is negative towards particular countries |
|
476 |
+
| Foreign Special Relationships: Positive | This example text from a political party manifesto is positive towards particular countries |
|
477 |
+
| Free Market Economy | This example text from a political party manifesto is in favour of a free market economy and capitalism |
|
478 |
+
| Freedom and Human Rights | This example text from a political party manifesto is in favour of freedom and human rights, for example freedom of speech, assembly or against state coercion or for individualism |
|
479 |
+
| Governmental and Administrative Efficiency | This example text from a political party manifesto is in favour of efficiency in government/administration, for example by restructuring civil service or improving bureaucracy |
|
480 |
+
| Incentives: Positive | This example text from a political party manifesto is favourable towards supply side economic policies supporting businesses, for example for incentives like subsidies or tax breaks |
|
481 |
+
| Internationalism: Negative | This example text from a political party manifesto is sceptical of internationalism, for example negative towards international cooperation, in favour of national sovereignty and unilaterialism |
|
482 |
+
| Internationalism: Positive | This example text from a political party manifesto is in favour of international cooperation with other countries, for example mentions the need for aid to developing countries, or global governance |
|
483 |
+
| Keynesian Demand Management | This example text from a political party manifesto is for keynesian demand management and demand side economic policies |
|
484 |
+
| Labour Groups: Negative | This example text from a political party manifesto is negative towards labour groups and unions |
|
485 |
+
| Labour Groups: Positive | This example text from a political party manifesto is positive towards labour groups, for example for good working conditions, fair wages or unions |
|
486 |
+
| Law and Order: Positive | This example text from a political party manifesto is positive towards law and order and strict law enforcement |
|
487 |
+
| Market Regulation | This example text from a political party manifesto is supports market regulation for a fair and open market, for example for consumer protection or for increased competition or for social market economy |
|
488 |
+
| Marxist Analysis | This example text from a political party manifesto is positive towards Marxist-Leninist ideas or uses specific Marxist terminology |
|
489 |
+
| Middle Class and Professional Groups | This example text from a political party manifesto favourably references the middle class, e.g. white colar groups or the service sector |
|
490 |
+
| Military: Negative | This example text from a political party manifesto is negative towards the military, for example for decreasing military spending or disarmament |
|
491 |
+
| Military: Positive | This example text from a political party manifesto is positive towards the military, for example for military spending or rearmament or military treaty obligations |
|
492 |
+
| Multiculturalism: Negative | This example text from a political party manifesto is sceptical towards multiculturalism, or for cultural integration or appeals to cultural homogeneity in society |
|
493 |
+
| Multiculturalism: Positive | This example text from a political party manifesto favourably mentions cultural diversity, for example for freedom of religion or linguistic heritages |
|
494 |
+
| National Way of Life: Negative | This example text from a political party manifesto unfavourably mentions a country's nation and history, for example sceptical towards patriotism or national pride |
|
495 |
+
| National Way of Life: Positive | This example text from a political party manifesto is positive towards the national way of life and history, for example pride of citizenship or appeals to patriotism |
|
496 |
+
| Nationalisation | This example text from a political party manifesto is positive towards government ownership of industries or land or for economic nationalisation |
|
497 |
+
| Non-economic Demographic Groups | This example text from a political party manifesto favourably mentions non-economic demographic groups like women, students or specific age groups |
|
498 |
+
| Peace | This example text from a political party manifesto is positive towards peace and peaceful means of solving crises, for example in favour of negotiations and ending wars |
|
499 |
+
| Political Authority | This example text from a political party manifesto mentions the speaker's competence to govern or other party's lack of such competence, or favourably mentions a strong/stable government |
|
500 |
+
| Political Corruption | This example text from a political party manifesto is negative towards political corruption or abuse of political/bureaucratic power |
|
501 |
+
| Protectionism: Negative | This example text from a political party manifesto is negative towards protectionism, in favour of free trade |
|
502 |
+
| Protectionism: Positive | This example text from a political party manifesto is in favour of protectionism, for example tariffs, export subsidies |
|
503 |
+
| Technology and Infrastructure: Positive | This example text from a political party manifesto is about technology and infrastructure, e.g. the importance of modernisation of industry, or supportive of public spending on infrastructure/tech |
|
504 |
+
| Traditional Morality: Negative | This example text from a political party manifesto is negative towards traditional morality, for example against religious moral values, for divorce or abortion, for modern families or separation of church and state |
|
505 |
+
| Traditional Morality: Positive | This example text from a political party manifesto is favourable towards traditional or religious values, for example for censorship of immoral behavour, for traditional family values or religious institutions |
|
506 |
+
| Underprivileged Minority Groups | This example text from a political party manifesto favourably mentions underprivileged minorities, for example handicapped, homosexuals or immigrants |
|
507 |
+
| Welfare State Expansion | This example text from a political party manifesto is positive towards the welfare state, e.g. health care, pensions or social housing |
|
508 |
+
| Welfare State Limitation | This example text from a political party manifesto is for limiting the welfare state, for example public funding for social services or social security, e.g. private care before state care |
|