File size: 6,946 Bytes
6fa48df c598fc3 18dcce4 32e199d 6eab3ec 6fa48df b15ab2d 4ccde54 b15ab2d 32e199d 4ccde54 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 |
---
library_name: transformers
base_model:
- answerdotai/ModernBERT-base
license: apache-2.0
language:
- en
pipeline_tag: zero-shot-classification
datasets:
- nyu-mll/glue
- facebook/anli
tags:
- instruct
- natural-language-inference
- nli
---
# Model Card for Model ID
ModernBERT fine-tuned on tasksource NLI tasks, including MNLI, ANLI, SICK, WANLI, doc-nli, LingNLI, FOLIO, FOL-NLI, LogicNLI, Label-NLI...)
Test accuracy at 10k training steps (current version, 100k steps incoming at the end of the week).
| test_name | test_accuracy |
|:-------------------------------------|----------------:|
| glue/mnli | 0.82 |
| glue/qnli | 0.84 |
| glue/rte | 0.78 |
| super_glue/cb | 0.75 |
| anli/a1 | 0.51 |
| anli/a2 | 0.39 |
| anli/a3 | 0.38 |
| sick/label | 0.91 |
| sick/entailment_AB | 0.81 |
| snli | 0.82 |
| scitail/snli_format | 0.94 |
| hans | 0.99 |
| WANLI | 0.7 |
| recast/recast_ner | 0.84 |
| recast/recast_kg_relations | 0.89 |
| recast/recast_puns | 0.78 |
| recast/recast_verbcorner | 0.87 |
| recast/recast_sentiment | 0.97 |
| recast/recast_verbnet | 0.74 |
| recast/recast_factuality | 0.88 |
| recast/recast_megaveridicality | 0.86 |
| probability_words_nli/reasoning_2hop | 0.76 |
| probability_words_nli/reasoning_1hop | 0.84 |
| probability_words_nli/usnli | 0.7 |
| nan-nli | 0.62 |
| nli_fever | 0.71 |
| breaking_nli | 0.98 |
| conj_nli | 0.66 |
| fracas | 0 |
| dialogue_nli | 0.84 |
| mpe | 0.69 |
| dnc | 0.81 |
| recast_white/fnplus | 0.6 |
| recast_white/sprl | 0.83 |
| recast_white/dpr | 0.57 |
| robust_nli/IS_CS | 0.45 |
| robust_nli/LI_LI | 0.92 |
| robust_nli/ST_WO | 0.66 |
| robust_nli/PI_SP | 0.53 |
| robust_nli/PI_CD | 0.54 |
| robust_nli/ST_SE | 0.58 |
| robust_nli/ST_NE | 0.52 |
| robust_nli/ST_LM | 0.47 |
| robust_nli_is_sd | 0.99 |
| robust_nli_li_ts | 0.81 |
| add_one_rte | 0.87 |
| cycic_classification | 0.62 |
| lingnli | 0.73 |
| monotonicity-entailment | 0.84 |
| scinli | 0.65 |
| naturallogic | 0.77 |
| syntactic-augmentation-nli | 0.87 |
| autotnli | 0.83 |
| defeasible-nli/atomic | 0.72 |
| defeasible-nli/snli | 0.67 |
| help-nli | 0.72 |
| nli-veridicality-transitivity | 0.92 |
| lonli | 0.88 |
| dadc-limit-nli | 0.59 |
| folio | 0.44 |
| tomi-nli | 0.52 |
| temporal-nli | 0.62 |
| counterfactually-augmented-snli | 0.69 |
| cnli | 0.71 |
| logiqa-2.0-nli | 0.51 |
| mindgames | 0.83 |
| ConTRoL-nli | 0.49 |
| logical-fallacy | 0.13 |
| conceptrules_v2 | 0.97 |
| zero-shot-label-nli | 0.67 |
| scone | 0.79 |
| monli | 0.76 |
| SpaceNLI | 0.89 |
| propsegment/nli | 0.82 |
| SDOH-NLI | 0.98 |
| scifact_entailment | 0.52 |
| AdjectiveScaleProbe-nli | 0.91 |
| resnli | 0.97 |
| semantic_fragments_nli | 0.91 |
| dataset_train_nli | 0.81 |
| ruletaker | 0.69 |
| PARARULE-Plus | 1 |
| logical-entailment | 0.53 |
| nope | 0.36 |
| LogicNLI | 0.34 |
| contract-nli/contractnli_a/seg | 0.79 |
| contract-nli/contractnli_b/full | 0.67 |
| nli4ct_semeval2024 | 0.53 |
| biosift-nli | 0.85 |
| SIGA-nli | 0.46 |
| FOL-nli | 0.49 |
| doc-nli | 0.81 |
| mctest-nli | 0.84 |
| idioms-nli | 0.77 |
| lifecycle-entailment | 0.57 |
| MSciNLI | 0.65 |
| babi_nli | 0.77 |
| gen_debiased_nli | 0.82 |
```
@inproceedings{sileo-2024-tasksource,
title = "tasksource: A Large Collection of {NLP} tasks with a Structured Dataset Preprocessing Framework",
author = "Sileo, Damien",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.1361",
pages = "15655--15684",
}
``` |