AntoineBlanot
commited on
Commit
•
11589e4
1
Parent(s):
47ceb17
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
datasets:
|
3 |
+
- multi_nli
|
4 |
+
- snli
|
5 |
+
- scitail
|
6 |
+
language:
|
7 |
+
- en
|
8 |
+
metrics:
|
9 |
+
- accuracy
|
10 |
+
- f1
|
11 |
+
pipeline_tag: zero-shot-classification
|
12 |
+
---
|
13 |
+
# RoBERTa NLI (Natural Language Inference)
|
14 |
+
This model is a fine-tuned model of [roberta-large](https://huggingface.co/roberta-large) after being trained on a **mixture of NLI datasets**.
|
15 |
+
|
16 |
+
This model can classify a pair of sentence (a <u>premise</u> and a <u>claim</u>) into 3 classes:
|
17 |
+
- 'entailment': the claim can logically be inferred from the premise
|
18 |
+
- 'contradiction': the claim contradicts the premise
|
19 |
+
- 'neutral': the premise is unrelated or do not provide sufficient information to validate the claim
|
20 |
+
|
21 |
+
This model can also be used for **zero-shot classification tasks** !
|
22 |
+
Please take a look at this [repo](https://github.com/AntoineBlanot/zero-nlp) for more information on zero-shot classification tasks.
|
23 |
+
|
24 |
+
# Usage
|
25 |
+
This model has been trained in an efficient way and thus cannot be load directly from HuggingFace's hub. To use that model, please follow instructions on this [repo](https://github.com/AntoineBlanot/efficient-llm).
|
26 |
+
|
27 |
+
For **zero-shot classification** tasks, please take a look at this [repo](https://github.com/AntoineBlanot/zero-nlp).
|
28 |
+
|
29 |
+
# Data used for training
|
30 |
+
- multi_nli
|
31 |
+
- snli
|
32 |
+
- scitail
|
33 |
+
|
34 |
+
# Evaluation results
|
35 |
+
|
36 |
+
| Data | Accuracy |
|
37 |
+
|:---:|:---------:|
|
38 |
+
| MNLI (val. m) | 0.894 |
|
39 |
+
| MNLI (val. mm) | 0.895 |
|
40 |
+
| SNLI (val.) | 0.920 |
|
41 |
+
| SciTail (val.) | 0.934 |
|