|
--- |
|
language: |
|
- en |
|
datasets: |
|
- squad |
|
model: |
|
- facebook/data2vec-text-base |
|
--- |
|
<h1>data2vec squad</h1> |
|
This is a testing fine tuned data2vec model in the squad dataset, any improvements and suggestions are welcome! |
|
|
|
<h3>Intended use</h3> |
|
Question Answering |
|
|
|
<h3>Training results</h3> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>Epoch</th> |
|
<th>Training Loss</th> |
|
<th>Validation Loss</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>1</td> |
|
<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">1.015800</span><br></td> |
|
<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.997690</span><br></td> |
|
</tr> |
|
<tr> |
|
<td>2</td> |
|
<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.804400</span></td> |
|
<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.950322</span><br></td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
<h3>Hyperparameters</h3> |
|
<ul> |
|
<li>evaluation_strategy="epoch"</li> |
|
<li>learning_rate=2e-5</li> |
|
<li>per_device_train_batch_size=15</li> |
|
<li>per_device_eval_batch_size=15</li> |
|
<li>num_train_epochs=2</li> |
|
<li>weight_decay=0.01</li> |
|
</ul> |
|
|
|
<h3>Frameworks and libraries used:</h3> |
|
<ul> |
|
<li>transformers</li> |
|
<li>datasets</li> |
|
<li>evaluate</li> |
|
</ul> |
