Update README.md
Browse files
README.md
CHANGED
@@ -36,14 +36,34 @@ For this purpose, we translated the sentence pairs in these dataset to German.
|
|
36 |
### Model Details
|
37 |
|
38 |
| | Description or Link |
|
39 |
-
|
40 |
|---|---|
|
|
|
|
|
|
|
41 |
|
42 |
-
|
43 |
|
44 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
|
46 |
-
|
47 |
|
48 |
|
49 |
|
|
|
36 |
### Model Details
|
37 |
|
38 |
| | Description or Link |
|
|
|
39 |
|---|---|
|
40 |
+
|**Base model** | [```gbert-large```](https://huggingface.co/deepset/gbert-large) |
|
41 |
+
|**Finetuning task**| Text Pair Classification / Natural Language Inference |
|
42 |
+
|**Source dataset**| [```mnli```](https://huggingface.co/datasets/multi_nli) ; [```anli```](https://huggingface.co/datasets/anli) ; [```snli```](https://huggingface.co/datasets/snli) |
|
43 |
|
44 |
+
### Performance
|
45 |
|
46 |
+
We evaluated our model for the nli task using the TEST set of the German part of the [xnli](https://huggingface.co/datasets/xnli dataset).
|
47 |
+
|
48 |
+
TEST-Set Accuracy: 86%
|
49 |
+
|
50 |
+
|
51 |
+
## Zeroshot Text Classification Task Benchmark
|
52 |
+
|
53 |
+
We further tested our model for a zeroshot text classification task using a part of the [10kGNAD Dataset](https://tblock.github.io/10kGNAD/).
|
54 |
+
Specifically, we used all articles that were labeled "Kultur", "Sport", "Web", "Wirtschaft" und "Wissenschaft".
|
55 |
+
|
56 |
+
The next table shows the results as well as a comparison with other German language zeroshot options performing the same task:
|
57 |
+
|
58 |
+
| Model | NDCG@1 | NDCG@5 | NDCG@10 | Recall@1 | Recall@5 | Recall@10 |
|
59 |
+
|
60 |
+
|:-------------------:|:------:|:------:|:-------:|:--------:|:--------:|:---------:|
|
61 |
+
|
62 |
+
| BM25 | 0.1463 | 0.3451 | 0.4097 | 0.1463 | 0.5424 | 0.7415 |
|
63 |
+
|
64 |
+
| BM25(Top 100) +Ours | 0.6410 | 0.7885 | 0.7943 | 0.6410 | 0.8576 | 0.9024 |
|
65 |
|
66 |
+
## Other Applications
|
67 |
|
68 |
|
69 |
|