Commit
•
db7f095
1
Parent(s):
7054526
Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (#11)
Browse files- Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (797b6e8ba53c5f6efa593e246e0d5dd56005c17c)
Co-authored-by: Evaluation Bot <autoevaluator@users.noreply.huggingface.co>
README.md
CHANGED
@@ -1,6 +1,9 @@
|
|
1 |
---
|
2 |
language:
|
3 |
- en
|
|
|
|
|
|
|
4 |
tags:
|
5 |
- summarization
|
6 |
- led
|
@@ -9,9 +12,6 @@ tags:
|
|
9 |
- booksum
|
10 |
- long-document
|
11 |
- long-form
|
12 |
-
license:
|
13 |
-
- apache-2.0
|
14 |
-
- bsd-3-clause
|
15 |
datasets:
|
16 |
- kmfoda/booksum
|
17 |
metrics:
|
@@ -30,39 +30,38 @@ widget:
|
|
30 |
deviation of the average recurrence interval, the more specific could be the long
|
31 |
term prediction of a future mainshock.
|
32 |
example_title: earthquakes
|
33 |
-
- text:
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
-
\ this function space (Section 5)."
|
66 |
example_title: scientific paper
|
67 |
- text: ' the big variety of data coming from diverse sources is one of the key properties
|
68 |
of the big data phenomenon. It is, therefore, beneficial to understand how data
|
@@ -107,50 +106,62 @@ widget:
|
|
107 |
in their business An important area of data analytics on the edge of corporate
|
108 |
IT and the Internet is Web Analytics.'
|
109 |
example_title: data science textbook
|
110 |
-
- text:
|
111 |
-
|
112 |
-
|
113 |
-
|
114 |
-
|
115 |
-
|
116 |
-
|
117 |
-
|
118 |
-
|
119 |
-
|
120 |
-
|
121 |
-
|
122 |
-
|
123 |
-
|
124 |
-
|
125 |
-
|
126 |
-
|
127 |
-
|
128 |
-
|
129 |
-
|
130 |
-
|
131 |
-
|
132 |
-
|
133 |
-
|
134 |
-
|
135 |
-
|
136 |
-
|
137 |
-
|
138 |
-
|
139 |
-
|
140 |
-
|
141 |
-
|
142 |
-
|
143 |
-
|
144 |
-
|
145 |
-
|
146 |
-
|
147 |
-
|
148 |
-
|
149 |
-
|
150 |
-
|
151 |
-
|
152 |
-
|
153 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
154 |
example_title: bigbird blog intro
|
155 |
- text: 'The majority of available text summarization datasets include short-form
|
156 |
source documents that lack long-range causal and temporal dependencies, and often
|
@@ -188,30 +199,36 @@ model-index:
|
|
188 |
config: kmfoda--booksum
|
189 |
split: test
|
190 |
metrics:
|
191 |
-
-
|
192 |
-
type: rouge
|
193 |
value: 31.7308
|
|
|
194 |
verified: true
|
195 |
-
|
196 |
-
|
197 |
value: 5.3311
|
|
|
198 |
verified: true
|
199 |
-
|
200 |
-
|
201 |
value: 16.1465
|
|
|
202 |
verified: true
|
203 |
-
|
204 |
-
|
205 |
value: 29.0883
|
|
|
206 |
verified: true
|
207 |
-
|
208 |
-
|
209 |
value: 4.815707206726074
|
|
|
210 |
verified: true
|
211 |
-
|
212 |
-
|
213 |
value: 154.9036
|
|
|
214 |
verified: true
|
|
|
215 |
- task:
|
216 |
type: summarization
|
217 |
name: Summarization
|
@@ -221,30 +238,36 @@ model-index:
|
|
221 |
config: samsum
|
222 |
split: test
|
223 |
metrics:
|
224 |
-
-
|
225 |
-
type: rouge
|
226 |
value: 33.4484
|
|
|
227 |
verified: true
|
228 |
-
|
229 |
-
|
230 |
value: 10.4249
|
|
|
231 |
verified: true
|
232 |
-
|
233 |
-
|
234 |
value: 24.5802
|
|
|
235 |
verified: true
|
236 |
-
|
237 |
-
|
238 |
value: 29.8226
|
|
|
239 |
verified: true
|
240 |
-
|
241 |
-
|
242 |
value: 4.176078796386719
|
|
|
243 |
verified: true
|
244 |
-
|
245 |
-
|
246 |
value: 65.4005
|
|
|
247 |
verified: true
|
|
|
248 |
- task:
|
249 |
type: summarization
|
250 |
name: Summarization
|
@@ -254,30 +277,36 @@ model-index:
|
|
254 |
config: default
|
255 |
split: test
|
256 |
metrics:
|
257 |
-
-
|
258 |
-
type: rouge
|
259 |
value: 40.5843
|
|
|
260 |
verified: true
|
261 |
-
|
262 |
-
|
263 |
value: 17.3401
|
|
|
264 |
verified: true
|
265 |
-
|
266 |
-
|
267 |
value: 25.1256
|
|
|
268 |
verified: true
|
269 |
-
|
270 |
-
|
271 |
value: 34.6619
|
|
|
272 |
verified: true
|
273 |
-
|
274 |
-
|
275 |
value: 4.792657375335693
|
|
|
276 |
verified: true
|
277 |
-
|
278 |
-
|
279 |
value: 163.9394
|
|
|
280 |
verified: true
|
|
|
281 |
- task:
|
282 |
type: summarization
|
283 |
name: Summarization
|
@@ -287,30 +316,36 @@ model-index:
|
|
287 |
config: default
|
288 |
split: test
|
289 |
metrics:
|
290 |
-
-
|
291 |
-
type: rouge
|
292 |
value: 39.0834
|
|
|
293 |
verified: true
|
294 |
-
|
295 |
-
|
296 |
value: 11.4043
|
|
|
297 |
verified: true
|
298 |
-
|
299 |
-
|
300 |
value: 19.1813
|
|
|
301 |
verified: true
|
302 |
-
|
303 |
-
|
304 |
value: 35.1581
|
|
|
305 |
verified: true
|
306 |
-
|
307 |
-
|
308 |
value: 4.654905319213867
|
|
|
309 |
verified: true
|
310 |
-
|
311 |
-
|
312 |
value: 186.2494
|
|
|
313 |
verified: true
|
|
|
314 |
---
|
315 |
|
316 |
# Longformer Encoder-Decoder (LED) for Narrative-Esque Long Text Summarization
|
|
|
1 |
---
|
2 |
language:
|
3 |
- en
|
4 |
+
license:
|
5 |
+
- apache-2.0
|
6 |
+
- bsd-3-clause
|
7 |
tags:
|
8 |
- summarization
|
9 |
- led
|
|
|
12 |
- booksum
|
13 |
- long-document
|
14 |
- long-form
|
|
|
|
|
|
|
15 |
datasets:
|
16 |
- kmfoda/booksum
|
17 |
metrics:
|
|
|
30 |
deviation of the average recurrence interval, the more specific could be the long
|
31 |
term prediction of a future mainshock.
|
32 |
example_title: earthquakes
|
33 |
+
- text: ' A typical feed-forward neural field algorithm. Spatiotemporal coordinates
|
34 |
+
are fed into a neural network that predicts values in the reconstructed domain.
|
35 |
+
Then, this domain is mapped to the sensor domain where sensor measurements are
|
36 |
+
available as supervision. Class and Section Problems Addressed Generalization
|
37 |
+
(Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid
|
38 |
+
Representations (Section 3) Computation & memory efficiency, representation capacity,
|
39 |
+
editability: Forward Maps (Section 4) Inverse problems Network Architecture (Section
|
40 |
+
5) Spectral bias, integration & derivatives. Manipulating Neural Fields (Section
|
41 |
+
6) Edit ability, constraints, regularization. Table 2: The five classes of techniques
|
42 |
+
in the neural field toolbox each addresses problems that arise in learning, inference,
|
43 |
+
and control. (Section 3). We can supervise reconstruction via differentiable forward
|
44 |
+
maps that transform Or project our domain (e.g, 3D reconstruction via 2D images;
|
45 |
+
Section 4) With appropriate network architecture choices, we can overcome neural
|
46 |
+
network spectral biases (blurriness) and efficiently compute derivatives and integrals
|
47 |
+
(Section 5). Finally, we can manipulate neural fields to add constraints and regularizations,
|
48 |
+
and to achieve editable representations (Section 6). Collectively, these classes
|
49 |
+
constitute a ''toolbox'' of techniques to help solve problems with neural fields
|
50 |
+
There are three components in a conditional neural field: (1) An encoder or inference
|
51 |
+
function € that outputs the conditioning latent variable 2 given an observation
|
52 |
+
0 E(0) =2. 2 is typically a low-dimensional vector, and is often referred to aS
|
53 |
+
a latent code Or feature code_ (2) A mapping function 4 between Z and neural field
|
54 |
+
parameters O: Y(z) = O; (3) The neural field itself $. The encoder € finds the
|
55 |
+
most probable z given the observations O: argmaxz P(2/0). The decoder maximizes
|
56 |
+
the inverse conditional probability to find the most probable 0 given Z: arg-
|
57 |
+
max P(Olz). We discuss different encoding schemes with different optimality guarantees
|
58 |
+
(Section 2.1.1), both global and local conditioning (Section 2.1.2), and different
|
59 |
+
mapping functions Y (Section 2.1.3) 2. Generalization Suppose we wish to estimate
|
60 |
+
a plausible 3D surface shape given a partial or noisy point cloud. We need a suitable
|
61 |
+
prior over the sur- face in its reconstruction domain to generalize to the partial
|
62 |
+
observations. A neural network expresses a prior via the function space of its
|
63 |
+
architecture and parameters 0, and generalization is influenced by the inductive
|
64 |
+
bias of this function space (Section 5).'
|
|
|
65 |
example_title: scientific paper
|
66 |
- text: ' the big variety of data coming from diverse sources is one of the key properties
|
67 |
of the big data phenomenon. It is, therefore, beneficial to understand how data
|
|
|
106 |
in their business An important area of data analytics on the edge of corporate
|
107 |
IT and the Internet is Web Analytics.'
|
108 |
example_title: data science textbook
|
109 |
+
- text: 'Transformer-based models have shown to be very useful for many NLP tasks.
|
110 |
+
However, a major limitation of transformers-based models is its O(n^2)O(n 2) time
|
111 |
+
& memory complexity (where nn is sequence length). Hence, it''s computationally
|
112 |
+
very expensive to apply transformer-based models on long sequences n > 512n>512.
|
113 |
+
Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention
|
114 |
+
try to remedy this problem by approximating the full attention matrix. You can
|
115 |
+
checkout 🤗''s recent blog post in case you are unfamiliar with these models.
|
116 |
+
|
117 |
+
BigBird (introduced in paper) is one of such recent models to address this issue.
|
118 |
+
BigBird relies on block sparse attention instead of normal attention (i.e. BERT''s
|
119 |
+
attention) and can handle sequences up to a length of 4096 at a much lower computational
|
120 |
+
cost compared to BERT. It has achieved SOTA on various tasks involving very long
|
121 |
+
sequences such as long documents summarization, question-answering with long contexts.
|
122 |
+
|
123 |
+
BigBird RoBERTa-like model is now available in 🤗Transformers. The goal of this
|
124 |
+
post is to give the reader an in-depth understanding of big bird implementation
|
125 |
+
& ease one''s life in using BigBird with 🤗Transformers. But, before going into
|
126 |
+
more depth, it is important to remember that the BigBird''s attention is an approximation
|
127 |
+
of BERT''s full attention and therefore does not strive to be better than BERT''s
|
128 |
+
full attention, but rather to be more efficient. It simply allows to apply transformer-based
|
129 |
+
models to much longer sequences since BERT''s quadratic memory requirement quickly
|
130 |
+
becomes unbearable. Simply put, if we would have ∞ compute & ∞ time, BERT''s attention
|
131 |
+
would be preferred over block sparse attention (which we are going to discuss
|
132 |
+
in this post).
|
133 |
+
|
134 |
+
If you wonder why we need more compute when working with longer sequences, this
|
135 |
+
blog post is just right for you!
|
136 |
+
|
137 |
+
Some of the main questions one might have when working with standard BERT-like
|
138 |
+
attention include:
|
139 |
+
|
140 |
+
Do all tokens really have to attend to all other tokens? Why not compute attention
|
141 |
+
only over important tokens? How to decide what tokens are important? How to attend
|
142 |
+
to just a few tokens in a very efficient way? In this blog post, we will try to
|
143 |
+
answer those questions.
|
144 |
+
|
145 |
+
What tokens should be attended to? We will give a practical example of how attention
|
146 |
+
works by considering the sentence ''BigBird is now available in HuggingFace for
|
147 |
+
extractive question answering''. In BERT-like attention, every word would simply
|
148 |
+
attend to all other tokens.
|
149 |
+
|
150 |
+
Let''s think about a sensible choice of key tokens that a queried token actually
|
151 |
+
only should attend to by writing some pseudo-code. Will will assume that the token
|
152 |
+
available is queried and build a sensible list of key tokens to attend to.
|
153 |
+
|
154 |
+
>>> # let''s consider following sentence as an example >>> example = [''BigBird'',
|
155 |
+
''is'', ''now'', ''available'', ''in'', ''HuggingFace'', ''for'', ''extractive'',
|
156 |
+
''question'', ''answering'']
|
157 |
+
|
158 |
+
>>> # further let''s assume, we''re trying to understand the representation of
|
159 |
+
''available'' i.e. >>> query_token = ''available'' >>> # We will initialize an
|
160 |
+
empty `set` and fill up the tokens of our interest as we proceed in this section.
|
161 |
+
>>> key_tokens = [] # => currently ''available'' token doesn''t have anything
|
162 |
+
to attend Nearby tokens should be important because, in a sentence (sequence of
|
163 |
+
words), the current word is highly dependent on neighboring past & future tokens.
|
164 |
+
This intuition is the idea behind the concept of sliding attention.'
|
165 |
example_title: bigbird blog intro
|
166 |
- text: 'The majority of available text summarization datasets include short-form
|
167 |
source documents that lack long-range causal and temporal dependencies, and often
|
|
|
199 |
config: kmfoda--booksum
|
200 |
split: test
|
201 |
metrics:
|
202 |
+
- type: rouge
|
|
|
203 |
value: 31.7308
|
204 |
+
name: ROUGE-1
|
205 |
verified: true
|
206 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjJmZjMxYTY0OGU3MzNjNmIzNmYyODNlNDg2ZGRhZDAzNTMwMDM5YWMxODc1OTc1ZWE3MzM2OTg1ODFhZDBkNCIsInZlcnNpb24iOjF9.B8BCKgySYVZW910_1zP0LfCpQYJbAe6loyWut76JlgZb2kV1_x9ybqtNESX0ka-lNqhYyXUNDpuS-7pTmsJVDg
|
207 |
+
- type: rouge
|
208 |
value: 5.3311
|
209 |
+
name: ROUGE-2
|
210 |
verified: true
|
211 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzViMmY4ODFjYTc5ODk5MmRhMDQ3ZDRiYWQwMDg0OTk3ZTA4NDAxYTNiNDgyMmI4NDA3ZDMwYWViOTBkODBjNyIsInZlcnNpb24iOjF9.MOhJLDcgvv93mVFL1igIgIiTAH3b2Xa4gmBObq7RF44Mmu8Kxtd1KP7rOlDVFOrtrsooGPGsyE1GMCQ2kqeMDg
|
212 |
+
- type: rouge
|
213 |
value: 16.1465
|
214 |
+
name: ROUGE-L
|
215 |
verified: true
|
216 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzNjMzEwMTliZGE3ZmQ4M2UxMDAyMTY3YzJjZmMyMDYyN2YyNDM0N2VhNzI1MDc1YTg4MTRjMmEzNjVkNTk1NCIsInZlcnNpb24iOjF9.XLJ-DVKiYLlbw5E5rWADKbzUzf5fNHhlTCWPCC5dU4NI9Yeh76aR7TPt36ZzLDwTBknnR8KHqlaF8F8YAvBUAg
|
217 |
+
- type: rouge
|
218 |
value: 29.0883
|
219 |
+
name: ROUGE-LSUM
|
220 |
verified: true
|
221 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTcwNzEwMmE5NjQxZTkzYmQyZDZmNzllYzYyNGI5OTMyNWMwNjdiM2I2YmM5YjdmY2E5OWQ3OTk3ZDA1MTc3YyIsInZlcnNpb24iOjF9.d6rFxjCB6RJNI_pn2DNNSjuZe4rdvj0RatkaTJRp5lP0F_AFfU5Zn9zRWzZJV7V-xMauIc4UhfdoLp9r_-CABA
|
222 |
+
- type: loss
|
223 |
value: 4.815707206726074
|
224 |
+
name: loss
|
225 |
verified: true
|
226 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTMwMTgxMmJkODY3MjkzOWJhMzJhOTIxMWVkODhjZmM0MWUzMWQ1N2JkZjRhOTQxNmU1YWVjYzQ0MDNlZWI3OSIsInZlcnNpb24iOjF9.mkBQHYhYFfDV6F4klXGJ1dSsF-pbCs-6F9zcw6IYznwmXUjtk7m5J4Zt4JAju5LKz4YizvEcUCl_L0WddnfvDA
|
227 |
+
- type: gen_len
|
228 |
value: 154.9036
|
229 |
+
name: gen_len
|
230 |
verified: true
|
231 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTc0ZmM1ZDM4MDE0MzY3MDM3OWJhNDkzZjJkZDdkMjU5M2JmMDJjYTIxODA1OTllNmY5ZWQzZDlmNWFiYzk4NiIsInZlcnNpb24iOjF9.VQ_O_xSTz870tnM08PJXQOwg9OsNNwI_HVX4S7AuW57_FzGGyRaWSuGE5SWzRS4Tur9YP0QxV4VV0Yoaoi3IAA
|
232 |
- task:
|
233 |
type: summarization
|
234 |
name: Summarization
|
|
|
238 |
config: samsum
|
239 |
split: test
|
240 |
metrics:
|
241 |
+
- type: rouge
|
|
|
242 |
value: 33.4484
|
243 |
+
name: ROUGE-1
|
244 |
verified: true
|
245 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTk4Yjg1YTc4YmY0MzBiZDU4ZjFhNzI4MjZkMWU1MzBlOWNlMjQ5ODMzY2YzYzRhYjJkMGUzNmI3ZjdkMzIzZSIsInZlcnNpb24iOjF9.AqS8A1OUiM0IZFBEGirv5F3Novk8lSUYSfPc3bYWLA6t-W7wgup3qA207eGbE5j9CkDWZ7QrSG1U6Z9A0sOqAA
|
246 |
+
- type: rouge
|
247 |
value: 10.4249
|
248 |
+
name: ROUGE-2
|
249 |
verified: true
|
250 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2U4NjUyNTFmOGM5OTlhZDMyMTlmM2E4OWI2NGFiMDAyMGJjMzRjNWNlMGEyYWFmNTE5ZWMxM2I0ZGZmNWNmOCIsInZlcnNpb24iOjF9.SgJcHJ4qoRWXFvFiwv1PUutWktvsxQNynVPEv-GtBgxd6WI7o561ONyco5U-5tcyE_1SbSCJzz-L-R-q3cvoDA
|
251 |
+
- type: rouge
|
252 |
value: 24.5802
|
253 |
+
name: ROUGE-L
|
254 |
verified: true
|
255 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmQ5MDI5MzdiNGE5NDM0MmU5OThmZTBkNjkxMzg5N2IxNGVlODdhZTZhNjg3NzFjYWEyMzA3MTQxNjMyMjRkOCIsInZlcnNpb24iOjF9.Bg5dHqCcJjmxa-xGWNR5lD9g3quX7lKkH0pjiTd2xE5WiPoLLN2c0mYa2GovdW7__WnYwhhHC7es03jmvyZbCw
|
256 |
+
- type: rouge
|
257 |
value: 29.8226
|
258 |
+
name: ROUGE-LSUM
|
259 |
verified: true
|
260 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGFhOTEwNGM1MmZkNDk2ZjQ1Y2MyNjM3MGI5MGY3MWVkM2I0MjU2NWFiYmEwMjE4MTJlZWIwOGQ2MjQ3YjgzYSIsInZlcnNpb24iOjF9.W_aQKs10oXQdKEczJBGM3iiwJgb-VaXTpyA3sGof5WbhHf9vITAQA-xvynh5LgKtXQ1zjx737hnHgjEsu_Y0Cw
|
261 |
+
- type: loss
|
262 |
value: 4.176078796386719
|
263 |
+
name: loss
|
264 |
verified: true
|
265 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2JhODQ5YTZkNDZkZGYyNGU2MzkxMWU5MTEwMGM2YmVjZTA5YzI5NTMxMDNhYjhlOTAxMzFiMDYwYmM0MjEzZCIsInZlcnNpb24iOjF9.OvZrPBOR5jhkoTGBgsInkH7j3_xpacXHDoT7UIXEnyXzadfBO-O-K6fjalLNZw8wSkbjHIFcL_6S_qTTxPsNAQ
|
266 |
+
- type: gen_len
|
267 |
value: 65.4005
|
268 |
+
name: gen_len
|
269 |
verified: true
|
270 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2NhYjc3ZjQzNDEwYmMzOTM0ODkyZTJhZWNhNzZhYmEyZTYxMzA2YTYzMWFjOTA5ZjlhYWMzODg3NzY1ZTUwYSIsInZlcnNpb24iOjF9.vk9bgmtQFeRwdY3VXjtrJr_5wUCIeoAkI3kO0cHxhxmJo6RvUnyXiut72FuB-mlLZvqgiNkaZ-u_bh0Z3DjuCw
|
271 |
- task:
|
272 |
type: summarization
|
273 |
name: Summarization
|
|
|
277 |
config: default
|
278 |
split: test
|
279 |
metrics:
|
280 |
+
- type: rouge
|
|
|
281 |
value: 40.5843
|
282 |
+
name: ROUGE-1
|
283 |
verified: true
|
284 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTVjMDkyMWZjYTQ0NzgzNGUxZjNiMTg3NjU1MWJlNTQ2MWQ1NjE1MDk1OTU4ZjJiNGQ5ODg3Y2VlMWUyMzllNyIsInZlcnNpb24iOjF9.OhqBcVIuHk7fzmdrsWMvUe1bLeVMZVstZUoZpP7C1vR-3aIDl7r6eBmPrt5w-KcNq5p4teNPBsq7oKzbd5ZgDQ
|
285 |
+
- type: rouge
|
286 |
value: 17.3401
|
287 |
+
name: ROUGE-2
|
288 |
verified: true
|
289 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGQxYmQzMmE0OTcyNTM5NmMwNjIxNzYxZDcwMDFkYzJkOWY4YWY3NTdhZGRhZDdlMDAxNzcwODQ5OGM3Mzc1MCIsInZlcnNpb24iOjF9.Pksn25EEqvmx757N7Swrd4yXc_xU7-AMN9yNe8lrbBa-l1LoI_2PUASvnjML4f705cfuyMAfb0FkFp5WfER2AA
|
290 |
+
- type: rouge
|
291 |
value: 25.1256
|
292 |
+
name: ROUGE-L
|
293 |
verified: true
|
294 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjhjYzI5MDBiMjk2NTY3MDNmZTdiOGYwMTRlYjIwZjAwMjdlNTAyYzdhYTJlODQ4MjYzYmQ3MjRlYTA2YzhhZSIsInZlcnNpb24iOjF9.1jPepsweS2bzIqDverQzzhmhFGch7gpoEGFGqQ8zW7K10aUKWFX8lt-uZAmTa1Z5ZhzyXGBzc3dReFPhWRRJBg
|
295 |
+
- type: rouge
|
296 |
value: 34.6619
|
297 |
+
name: ROUGE-LSUM
|
298 |
verified: true
|
299 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2VkZDIxNWJjOTA0NzFjOTIwOTdjYjc1M2EyNDVjZjY2ZjY3MjIxNDk3YTc5YWExNzAwN2FhOTc1NjVhYjBkYiIsInZlcnNpb24iOjF9.8opqHSUckPohoSF9jfPTpXDz2AtDwvdMqOdIXx2kE1tkOcbLPbOBfcc8RhRR98y8S26yC6EYFhFnf03CV2ejAQ
|
300 |
+
- type: loss
|
301 |
value: 4.792657375335693
|
302 |
+
name: loss
|
303 |
verified: true
|
304 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTY5ZTRkMGU3OGVkODMzMDU5OWE1NTM5YjA4NDliZDlmNzc2NzZjNjFmNTA3M2EwY2NmN2E0MWJmZjQ5ZDliMiIsInZlcnNpb24iOjF9.KCKdk8xt2NWcMmYKV3-9eVEsFm9MqGllSMu9QCFJFIQlnyNXllHKdBLouoaGQz8IRYXvZKH8_TLDPIQx-31jAg
|
305 |
+
- type: gen_len
|
306 |
value: 163.9394
|
307 |
+
name: gen_len
|
308 |
verified: true
|
309 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzdkZDYyZGUzYmFkZmI2NjUwYmQ0MzZjMmIyZjI1YTFiMzM4OThiZjBiMzljOTVkZTgwMjA0NTE5OGM2YmFjMiIsInZlcnNpb24iOjF9.XyMZLUdkUIF32KTJMuv_bJswQCx_Tfg4Fx823cURUixSeoIKps8_a634AreZ3Z8kb7bfE_sFGh3rM9KWsMxlDw
|
310 |
- task:
|
311 |
type: summarization
|
312 |
name: Summarization
|
|
|
316 |
config: default
|
317 |
split: test
|
318 |
metrics:
|
319 |
+
- type: rouge
|
|
|
320 |
value: 39.0834
|
321 |
+
name: ROUGE-1
|
322 |
verified: true
|
323 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjYzMmVlMDM4MTNkMTI4MjAyMTU2YTg1ZWQwNTI1MmJlNGUwZmE1NTRmYTljZTQwY2RlMjcxOTgyZGMyYTc0ZiIsInZlcnNpb24iOjF9.6yuSr7UmsFatwqQ-mEO4gmsEtWI05kGB5Ib2pnl05H1OiPT2uUwmqdUytUw8KTx9u1jv9q0cTF1cL-n2kPEJAA
|
324 |
+
- type: rouge
|
325 |
value: 11.4043
|
326 |
+
name: ROUGE-2
|
327 |
verified: true
|
328 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWI5N2U2ZWI1ODM2MWUwOTIzYTAzNmRhNDA2OWEzZWRjMGEzMjBmY2EwN2YyYzU1NWE0YjIyZDE3MWE0MmMxZCIsInZlcnNpb24iOjF9.wonuxbBl25TzEaHUH_E816nHJ1OSXKfkaq7eJzbLpsfeGwcDklxUSxZxRO7VBiBMaY3Qttf9ywmEIPp40HnpBA
|
329 |
+
- type: rouge
|
330 |
value: 19.1813
|
331 |
+
name: ROUGE-L
|
332 |
verified: true
|
333 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjU1NDZhN2NkMzZiZGJkODE4NDZiYjViOTZkNGMyNDlkNjBlZmFjYzU1N2IzMjFjYjY1MDU1Zjk2MzA0M2U4NyIsInZlcnNpb24iOjF9.bTCRzv3J9NiCh4aV23tAWGTvrdQCv_RS40zGwC4AJXtGS40cY7tJHYwBf9U9_rCetDBxqfjJpdaUbCAOglxLAA
|
334 |
+
- type: rouge
|
335 |
value: 35.1581
|
336 |
+
name: ROUGE-LSUM
|
337 |
verified: true
|
338 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDNhNTUyZjE4NjYxYjIzYThmMDM2YWNhM2QwYzY1ODI2ZTE3NmNjMmVhOTAzZjZlOWQwYzc1NzU2NDNjNzIxMyIsInZlcnNpb24iOjF9.cWlSbEBgrMN5D-fV_yL9geNMyMkIItcVO3wehNJPzFi3E0v1-4q8pnX-UgjLzto8X7JLi6as2V_HtZE4-C-CDw
|
339 |
+
- type: loss
|
340 |
value: 4.654905319213867
|
341 |
+
name: loss
|
342 |
verified: true
|
343 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTc5Nzk0ODhiNWUzNTAxNzk2YzZmMjU2NDliY2UzOTYyYTdmZGEyYjI5NDNhOTE0MGUxOTgxMGVjMmNhM2UyMSIsInZlcnNpb24iOjF9.eBBAebcl3AwkrjR6a8BvoSjDfpw8LWTRFjyIFHVzspvoOKVfnO8_NB_UeR_K127OwXyoZ70Z7X_aKJOe-2kTDA
|
344 |
+
- type: gen_len
|
345 |
value: 186.2494
|
346 |
+
name: gen_len
|
347 |
verified: true
|
348 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWI2NjVlYjgwYWJiMjcyMDUzMzEwNDNjZTMxMDM0MjAzMzk1ZmIwY2Q1ZDQ2Y2M5NDBlMDEzYzFkNWEyNzJmNiIsInZlcnNpb24iOjF9.iZ1Iy7FuWL4GH7LS5EylVj5eZRC3L2ZsbYQapAkMNzR_VXPoMGvoM69Hp-kU7gW55tmz2V4Qxhvoz9cM8fciBA
|
349 |
---
|
350 |
|
351 |
# Longformer Encoder-Decoder (LED) for Narrative-Esque Long Text Summarization
|