Dehnes commited on
Commit
7079f3b
1 Parent(s): d161770

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -4
README.md CHANGED
@@ -36,14 +36,34 @@ For this purpose, we translated the sentence pairs in these dataset to German.
36
  ### Model Details
37
 
38
  | | Description or Link |
39
-
40
  |---|---|
 
 
 
41
 
42
- |**Base model** | [```gbert-large```](https://huggingface.co/deepset/gbert-large) |
43
 
44
- |**Finetuning task**| Text Pair Classification / Natural Language Inference |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
- |**Source dataset**| [```mnli```](https://huggingface.co/datasets/multi_nli) ; [```anli```](https://huggingface.co/datasets/anli) ; [```snli```](https://huggingface.co/datasets/snli) |
47
 
48
 
49
 
 
36
  ### Model Details
37
 
38
  | | Description or Link |
 
39
  |---|---|
40
+ |**Base model** | [```gbert-large```](https://huggingface.co/deepset/gbert-large) |
41
+ |**Finetuning task**| Text Pair Classification / Natural Language Inference |
42
+ |**Source dataset**| [```mnli```](https://huggingface.co/datasets/multi_nli) ; [```anli```](https://huggingface.co/datasets/anli) ; [```snli```](https://huggingface.co/datasets/snli) |
43
 
44
+ ### Performance
45
 
46
+ We evaluated our model for the nli task using the TEST set of the German part of the [xnli](https://huggingface.co/datasets/xnli dataset).
47
+
48
+ TEST-Set Accuracy: 86%
49
+
50
+
51
+ ## Zeroshot Text Classification Task Benchmark
52
+
53
+ We further tested our model for a zeroshot text classification task using a part of the [10kGNAD Dataset](https://tblock.github.io/10kGNAD/).
54
+ Specifically, we used all articles that were labeled "Kultur", "Sport", "Web", "Wirtschaft" und "Wissenschaft".
55
+
56
+ The next table shows the results as well as a comparison with other German language zeroshot options performing the same task:
57
+
58
+ | Model | NDCG@1 | NDCG@5 | NDCG@10 | Recall@1 | Recall@5 | Recall@10 |
59
+
60
+ |:-------------------:|:------:|:------:|:-------:|:--------:|:--------:|:---------:|
61
+
62
+ | BM25 | 0.1463 | 0.3451 | 0.4097 | 0.1463 | 0.5424 | 0.7415 |
63
+
64
+ | BM25(Top 100) +Ours | 0.6410 | 0.7885 | 0.7943 | 0.6410 | 0.8576 | 0.9024 |
65
 
66
+ ## Other Applications
67
 
68
 
69