README.md

metadata

language: en
datasets:
  - squad
metrics:
  - squad
license: apache-2.0

DistilBERT base cased distilled SQuAD

Note: This model is a clone of distilbert-base-cased-distilled-squad for internal testing.

This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7).