TomatenMarc
commited on
Commit
•
87a8dbd
1
Parent(s):
2476254
Upload 10 files
Browse files
README.md
CHANGED
@@ -141,14 +141,13 @@ Parameters of the fit()-Method:
|
|
141 |
"lr": 4e-05
|
142 |
},
|
143 |
"scheduler": "WarmupLinear",
|
144 |
-
"warmup_steps": 66
|
145 |
-
"weight_decay": 0.06
|
146 |
}
|
147 |
```
|
148 |
|
149 |
## Evaluation
|
150 |
|
151 |
-
We applied a 6-fold (
|
152 |
described in the *Training* section, where we trained on k-1 splits and made predictions using the kth split.
|
153 |
|
154 |
Additionally, we assessed its ability to generalize across the 6 topics (Cross-Topic) of TACO. Each of the k topics was utilized for testing, while
|
@@ -156,19 +155,19 @@ the remaining k-1 topics were used for training purposes.
|
|
156 |
|
157 |
In total, the WRAP classifier performs as follows:
|
158 |
|
159 |
-
###
|
160 |
|
161 |
-
| Macro-F1
|
162 |
-
|
163 |
-
|
|
164 |
-
| Cross-Topic
|
165 |
|
166 |
-
### Classification
|
167 |
|
168 |
-
| Micro-F1
|
169 |
-
|
170 |
-
|
|
171 |
-
| Cross-Topic
|
172 |
|
173 |
# Environmental Impact
|
174 |
|
|
|
141 |
"lr": 4e-05
|
142 |
},
|
143 |
"scheduler": "WarmupLinear",
|
144 |
+
"warmup_steps": 66
|
|
|
145 |
}
|
146 |
```
|
147 |
|
148 |
## Evaluation
|
149 |
|
150 |
+
We applied a 6-fold (Closed-Topic) cross-validation method to demonstrate WRAP's optimal performance. This involved using the same dataset and parameters
|
151 |
described in the *Training* section, where we trained on k-1 splits and made predictions using the kth split.
|
152 |
|
153 |
Additionally, we assessed its ability to generalize across the 6 topics (Cross-Topic) of TACO. Each of the k topics was utilized for testing, while
|
|
|
155 |
|
156 |
In total, the WRAP classifier performs as follows:
|
157 |
|
158 |
+
### Binary Classification Tasks
|
159 |
|
160 |
+
| Macro-F1 | Inference | Information | Multi-Class |
|
161 |
+
|--------------|-----------|-------------|-------------|
|
162 |
+
| Closed-Topic | 86.62% | 86.30% | 75.29% |
|
163 |
+
| Cross-Topic | 86.27% | 84.90% | 73.54% |
|
164 |
|
165 |
+
### Multi-Class Classification Task
|
166 |
|
167 |
+
| Micro-F1 | Reason | Statement | Notification | None |
|
168 |
+
|--------------|--------|-----------|--------------|--------|
|
169 |
+
| Closed-Topic | 78.14% | 60.96% | 79.36% | 82.72% |
|
170 |
+
| Cross-Topic | 77.05% | 58.33% | 78.45% | 80.33% |
|
171 |
|
172 |
# Environmental Impact
|
173 |
|