VictorSanh commited on
Commit
7ebb63d
1 Parent(s): 87af389

fix row missing in winobias results

Browse files
Files changed (1) hide show
  1. README.md +14 -6
README.md CHANGED
@@ -9,15 +9,18 @@ widget:
9
  - text: "Task: copy but say the opposite.\n
10
  PSG won its match against Barca."
11
  - text: "Is this review positive or negative? Review: Best cast iron skillet you will every buy."
 
12
  - text: "Question A: How is air traffic controlled?
13
  \nQuestion B: How do you become an air traffic controller?\nPick one: these questions are duplicates or not duplicates."
14
  - text: "Barack Obama nominated Hilary Clinton as his secretary of state on Monday. He chose her because she had foreign affairs experience as a former First Lady.
15
  \nIn the previous sentence, decide who 'her' is referring to."
 
16
  - text: "Last week I upgraded my iOS version and ever since then my phone has been overheating whenever I use your app.\n
17
  Select the category for the above sentence from: mobile, website, billing, account access."
18
  - text: "Sentence 1: Gyorgy Heizler, head of the local disaster unit, said the coach was carrying 38 passengers.\n
19
  Sentence 2: The head of the local disaster unit, Gyorgy Heizler, said the bus was full except for 38 empty seats.\n\n
20
  Do sentences 1 and 2 have the same meaning?"
 
21
  - text: "Here's the beginning of an article, choose a tag that best describes the topic of the article: business, cinema, politics, health, travel, sports.\n\n
22
  The best and worst fo 007 as 'No time to die' marks Daniel Craig's exit.\n
23
  (CNN) Some 007 math: 60 years, 25 movies (with a small asterisk) and six James Bonds. For a Cold War creation, Ian Fleming's suave spy has certainly gotten around, but despite different guises in the tuxedo and occasional scuba gear, when it comes to Bond ratings, there really shouldn't be much argument about who wore it best."
@@ -33,6 +36,11 @@ Sentence B: the tables in this book are very hard to read."
33
  - text: "On a shelf, there are five books: a gray book, a red book, a purple book, a blue book, and a black book.\n
34
  The red book is to the right of the gray book. The black book is to the left of the blue book. The blue book is to the left of the gray book. The purple book is the second from the right.\n\n
35
  Which book is the leftmost book?"
 
 
 
 
 
36
  - text: "The word 'binne' means any animal that is furry and has four legs, and the word 'bam' means a simple sort of dwelling.\n\n
37
  Which of the following best characterizes binne bams?\n
38
  - Sentence 1: Binne bams are for pets.\n
@@ -41,11 +49,11 @@ Which of the following best characterizes binne bams?\n
41
  - Sentence 4: Binne bams are places where people live."
42
  ---
43
 
44
- **How do I pronounce the name of the model?** T0 should be pronounced "T Zero" and any "p" stands for "Plus", so "T0pp" should be pronounced "T Zero Plus Plus"!
45
 
46
  # Model Description
47
 
48
- T0* is a series of encoder-decoder models trained on a large set of different tasks specified in natural language prompts. We convert numerous English supervised datasets into prompts, each with multiple templates using varying formulations. These prompted datasets allow for benchmarking the ability of a model to perform completely unseen tasks specified in natural language. To obtain T0*, we fine-tune a pretrained language model on this multitask mixture covering many different NLP tasks.
49
 
50
  # Intended uses
51
 
@@ -265,21 +273,21 @@ To measure the extent to which our model reproduces gender biases, we evaluate o
265
  </tr>
266
  <td rowspan="2">T0_single_prompt</td>
267
  <td>Type 1</td>
268
- <td>82.3</td><td>70.1</td><td>12.2</td><td>83.6</td><td>62.9</td><td>20.7</td>
269
  </tr>
270
  </tr>
271
  <td>Type 2</td>
272
- <td>83.8</td><td>76.5</td><td>7.3</td><td>85.9</td><td>75.0</td><td>10.9</td>
273
  </tr>
274
 
275
  </tr>
276
  <td rowspan="2">T0_original_task_only</td>
277
  <td>Type 1</td>
278
- <td>73.7</td><td>60.5</td><td>13.2</td><td>79.3</td><td>60.6</td><td>18.7</td>
279
  </tr>
280
  </tr>
281
  <td> Type 2</td>
282
- <td>77.7</td><td>69.6</td><td>8.0</td><td>80.8</td><td>69.7</td><td>11.1</td>
283
  </tr>
284
 
285
  </tr>
 
9
  - text: "Task: copy but say the opposite.\n
10
  PSG won its match against Barca."
11
  - text: "Is this review positive or negative? Review: Best cast iron skillet you will every buy."
12
+ example_title: "Sentiment analysis"
13
  - text: "Question A: How is air traffic controlled?
14
  \nQuestion B: How do you become an air traffic controller?\nPick one: these questions are duplicates or not duplicates."
15
  - text: "Barack Obama nominated Hilary Clinton as his secretary of state on Monday. He chose her because she had foreign affairs experience as a former First Lady.
16
  \nIn the previous sentence, decide who 'her' is referring to."
17
+ example_title: "Coreference resolution"
18
  - text: "Last week I upgraded my iOS version and ever since then my phone has been overheating whenever I use your app.\n
19
  Select the category for the above sentence from: mobile, website, billing, account access."
20
  - text: "Sentence 1: Gyorgy Heizler, head of the local disaster unit, said the coach was carrying 38 passengers.\n
21
  Sentence 2: The head of the local disaster unit, Gyorgy Heizler, said the bus was full except for 38 empty seats.\n\n
22
  Do sentences 1 and 2 have the same meaning?"
23
+ example_title: "Paraphrase identification"
24
  - text: "Here's the beginning of an article, choose a tag that best describes the topic of the article: business, cinema, politics, health, travel, sports.\n\n
25
  The best and worst fo 007 as 'No time to die' marks Daniel Craig's exit.\n
26
  (CNN) Some 007 math: 60 years, 25 movies (with a small asterisk) and six James Bonds. For a Cold War creation, Ian Fleming's suave spy has certainly gotten around, but despite different guises in the tuxedo and occasional scuba gear, when it comes to Bond ratings, there really shouldn't be much argument about who wore it best."
 
36
  - text: "On a shelf, there are five books: a gray book, a red book, a purple book, a blue book, and a black book.\n
37
  The red book is to the right of the gray book. The black book is to the left of the blue book. The blue book is to the left of the gray book. The purple book is the second from the right.\n\n
38
  Which book is the leftmost book?"
39
+ example_title: "Logic puzzles"
40
+ - text: "The two men running to become New York City's next mayor will face off in their first debate Wednesday night.\n\n
41
+ Democrat Eric Adams, the Brooklyn Borough president and a former New York City police captain, is widely expected to win the Nov. 2 election against Republican Curtis Sliwa, the founder of the 1970s-era Guardian Angels anti-crime patril.\n\n
42
+ Who are the men running for mayor?"
43
+ example_title: "Reading comprehension"
44
  - text: "The word 'binne' means any animal that is furry and has four legs, and the word 'bam' means a simple sort of dwelling.\n\n
45
  Which of the following best characterizes binne bams?\n
46
  - Sentence 1: Binne bams are for pets.\n
 
49
  - Sentence 4: Binne bams are places where people live."
50
  ---
51
 
52
+ **How do I pronounce the name of the model?** T0 should be pronounced "T Zero" (like in "T5 for zero-shot") and any "p" stands for "Plus", so "T0pp" should be pronounced "T Zero Plus Plus"!
53
 
54
  # Model Description
55
 
56
+ T0* shows zero-shot task generalization on English natural language prompts, outperforming GPT-3 on many tasks, while being 16x smaller. It is a series of encoder-decoder models trained on a large set of different tasks specified in natural language prompts. We convert numerous English supervised datasets into prompts, each with multiple templates using varying formulations. These prompted datasets allow for benchmarking the ability of a model to perform completely unseen tasks specified in natural language. To obtain T0*, we fine-tune a pretrained language model on this multitask mixture covering many different NLP tasks.
57
 
58
  # Intended uses
59
 
 
273
  </tr>
274
  <td rowspan="2">T0_single_prompt</td>
275
  <td>Type 1</td>
276
+ <td>73.7</td><td>60.5</td><td>13.2</td><td>79.3</td><td>60.6</td><td>18.7</td>
277
  </tr>
278
  </tr>
279
  <td>Type 2</td>
280
+ <td>77.7</td><td>69.6</td><td>8.0</td><td>80.8</td><td>69.7</td><td>11.1</td>
281
  </tr>
282
 
283
  </tr>
284
  <td rowspan="2">T0_original_task_only</td>
285
  <td>Type 1</td>
286
+ <td>78.1</td><td>67.7</td><td>10.4</td><td>81.8</td><td>67.2</td><td>14.6</td>
287
  </tr>
288
  </tr>
289
  <td> Type 2</td>
290
+ <td>85.2</td><td>82.3</td><td>2.9</td><td>89.6</td><td>85.4</td><td>4.3</td>
291
  </tr>
292
 
293
  </tr>