This is a reproduction of SumCSE. I don't know the author of the original paper.
5401.reimple-robert-large
epoch = 3.0
eval_CR = 92.18
eval_MPQA = 90.08
eval_MR = 87.44
eval_MRPC = 77.01
eval_SST2 = 92.43
eval_SUBJ = 93.46
eval_TREC = 81.49
eval_avg_sts = 0.8624794115602034
eval_avg_transfer = 87.72714285714285
eval_sickr_spearman = 0.8479586922026686
eval_stsb_spearman = 0.8770001309177381
------ test ------
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness | Avg. |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| 79.55 | 87.14 | 81.30 | 87.50 | 85.13 | 85.21 | 82.10 | 83.99 |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
+------+------+------+------+------+------+------+------+
| MR | CR | SUBJ | MPQA | SST2 | TREC | MRPC | Avg. |
+------+------+------+------+------+------+------+------+
| 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
+------+------+------+------+------+------+------+------+
- Downloads last month
- 3