Commit
•
bfa57dd
1
Parent(s):
b0238e5
Update README.md
Browse files
README.md
CHANGED
|
@@ -157,6 +157,21 @@ Glue test results:
|
|
| 157 |
| | 72/73 | 84 | 80 | 95 | 69 | 79 | 76 | 63| 76.7 |
|
| 158 |
|
| 159 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 160 |
### BibTeX entry and citation info
|
| 161 |
|
| 162 |
```bibtex
|
|
|
|
| 157 |
| | 72/73 | 84 | 80 | 95 | 69 | 79 | 76 | 63| 76.7 |
|
| 158 |
|
| 159 |
|
| 160 |
+
The following table contains test results on the HuggingFace model in comparison with [bert-base-cased](https://hf.co/models/bert-base-cased). The training was done on a single 16GB NVIDIA Tesla V100 GPU. For MRPC/WNLI, the models were trained for 5 epochs, while for other tasks, the models were trained for 3 epochs. Please refer to the checkpoints linked with the scores. The sequence length used for 512 with batch size 16 and learning rate 2e-5.
|
| 161 |
+
|
| 162 |
+
| Task | Metric | Result | | Training time | |
|
| 163 |
+
| ----- | ---------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------- | -------- |
|
| 164 |
+
| | | Bert | FNet | Bert | FNet |
|
| 165 |
+
| CoLA | Matthews corr | [59.57](https://huggingface.co/gchhablani/bert-base-cased-finetuned-cola) | [35.94](https://huggingface.co/gchhablani/fnet-base-finetuned-cola) | 14:20 | 09:47 |
|
| 166 |
+
| SST-2 | Accuracy | [92.32](https://huggingface.co/gchhablani/bert-base-cased-finetuned-sst2) | [89.45](https://huggingface.co/gchhablani/fnet-base-finetuned-sst2) | 01:42:17 | 01:09:27 |
|
| 167 |
+
| MRPC | F1/Accuracy | [90.26/86.03](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mrpc) | [85.02/77.21](https://huggingface.co/gchhablani/fnet-base-finetuned-mrpc) | 11:12 | 07:48 |
|
| 168 |
+
| STS-B | Pearson/Spearman corr. | [89.26/88.98](https://huggingface.co/gchhablani/bert-base-cased-finetuned-stsb) | [82.56/82.19](https://huggingface.co/gchhablani/fnet-base-finetuned-stsb) | 10:24 | 07:09 |
|
| 169 |
+
| QQP | Accuracy/F1 | [90.84/87.68](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp) | [88.47/84.66](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp) | 09:25:01 | 06:21:16 |
|
| 170 |
+
| MNLI | Accuracy | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli) | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli) | 09:52:33 | 06:40:55 |
|
| 171 |
+
| QNLI | Accuracy | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 02:40:22 | 01:48:22 |
|
| 172 |
+
| RTE | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 04:51 | 03:24 |
|
| 173 |
+
| WNLI | Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | 03:23 | 02:37 |
|
| 174 |
+
|
| 175 |
### BibTeX entry and citation info
|
| 176 |
|
| 177 |
```bibtex
|