|
--- |
|
license: mit |
|
datasets: |
|
- squad_v2 |
|
language: |
|
- en |
|
tags: |
|
- Bert |
|
- SQuAD2.0 |
|
- SQuAD |
|
pipeline_tag: question-answering |
|
--- |
|
# Extract QA Model (SQuAD2.0) |
|
|
|
## Model Information |
|
|
|
Pretrained model: google/bert_uncased_L-12_H-768_A-12 |
|
|
|
## Training Hyperparameters |
|
|
|
```Python |
|
epochs = 2 |
|
batch_size = 24 |
|
learning_rate = 3e-5 |
|
|
|
max_seq_length = 384 |
|
doc_stride = 128 |
|
max_query_length = 256 |
|
``` |
|
|
|
## Latest Result |
|
```json |
|
// 11 Apr 2023, 11:08 (Staging Epoch 1, total epoch 4, Correct Tokenizer) |
|
{ |
|
"exact": 74.43780005053483, |
|
"f1": 77.49749091378419, |
|
"total": 11873, |
|
"HasAns_exact": 73.73481781376518, |
|
"HasAns_f1": 79.86297395738177, |
|
"HasAns_total": 5928, |
|
"NoAns_exact": 75.13877207737595, |
|
"NoAns_f1": 75.13877207737595, |
|
"NoAns_total": 5945, |
|
"best_exact": 74.43780005053483, |
|
"best_exact_thresh": 0.0, |
|
"best_f1": 77.49749091378418, |
|
"best_f1_thresh": 0.0 |
|
} |
|
``` |
|
|
|
```json |
|
// 11 Apr 2023, 09:38 (Staging Epoch 3, total epoch 4, Correct Tokenizer) |
|
{ |
|
"exact": 73.99983155057694, |
|
"f1": 77.23749498407376, |
|
"total": 11873, |
|
"HasAns_exact": 72.85762483130904, |
|
"HasAns_f1": 79.3422364955984, |
|
"HasAns_total": 5928, |
|
"NoAns_exact": 75.13877207737595, |
|
"NoAns_f1": 75.13877207737595, |
|
"NoAns_total": 5945, |
|
"best_exact": 73.99983155057694, |
|
"best_exact_thresh": 0.0, |
|
"best_f1": 77.23749498407373, |
|
"best_f1_thresh": 0.0 |
|
} |
|
``` |
|
|
|
|
|
```json |
|
// 24 Mar 2023, 19:57 (Invalid Tokenizer) |
|
{ |
|
"exact": 74.12616861787248, |
|
"f1": 77.34212395572948, |
|
"total": 11873, |
|
"HasAns_exact": 72.72267206477733, |
|
"HasAns_f1": 79.16380528447645, |
|
"HasAns_total": 5928, |
|
"NoAns_exact": 75.52565180824222, |
|
"NoAns_f1": 75.52565180824222, |
|
"NoAns_total": 5945, |
|
"best_exact": 74.12616861787248, |
|
"best_exact_thresh": 0.0, |
|
"best_f1": 77.34212395572948, |
|
"best_f1_thresh": 0.0 |
|
} |
|
``` |
|
|
|
```json |
|
// 23 Mar 2023, (Invalid Tokenizer) |
|
{ |
|
"exact": 73.77242482944496, |
|
"f1": 76.83752039897598, |
|
"total": 11873, |
|
"HasAns_exact": 71.72739541160594, |
|
"HasAns_f1": 77.86637646711235, |
|
"HasAns_total": 5928, |
|
"NoAns_exact": 75.81160639192599, |
|
"NoAns_f1": 75.81160639192599, |
|
"NoAns_total": 5945, |
|
"best_exact": 73.77242482944496, |
|
"best_exact_thresh": 0.0, |
|
"best_f1": 76.83752039897604, |
|
"best_f1_thresh": 0.0 |
|
} |
|
``` |
