Upload 13 files

README.md

@@ -0,0 +1,105 @@
+---
+language: en
+thumbnail:
+tags:
+- QA
+
+---
+
+# BERT-Tiny fine-tuned on SQuAD v2
+
+[BERT-Tiny](https://github.com/google-research/bert/) created by [Google Research](https://github.com/google-research) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task.
+
+**Mode size** (after training): **16.74 MB**
+
+## Details of BERT-Tiny and its 'family' (from their documentation)
+
+Released on March 11th, 2020
+
+This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962).
+
+The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher.
+
+## Details of the downstream task (Q&A) - Dataset
+
+[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering.
+
+| Dataset  | Split | # samples |
+| -------- | ----- | --------- |
+| SQuAD2.0 | train | 130k      |
+| SQuAD2.0 | eval  | 12.3k     |
+
+## Model training
+
+The model was trained on a Tesla P100 GPU and 25GB of RAM.
+The script for fine tuning can be found [here](https://github.com/huggingface/transformers/tree/main/examples/legacy/question-answering)
+
+## Results:
+
+| Metric | # Value   |
+| ------ | --------- |
+| **EM** | **48.60** |
+| **F1** | **49.73** |
+
+
+| Model                                                                                     | EM        | F1 score  | SIZE (MB) |
+| ----------------------------------------------------------------------------------------- | --------- | --------- | --------- |
+| [bert-tiny-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-finetuned-squadv2) | 48.60     | 49.73     | **16.74** |
+| [bert-tiny-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-5-finetuned-squadv2) | **57.12** | **60.86** | 24.34  
+
+## Model in action
+
+Fast usage with **pipelines**:
+
+```python
+from transformers import pipeline
+
+qa_pipeline = pipeline(
+    "question-answering",
+    model="mrm8488/bert-tiny-finetuned-squadv2",
+    tokenizer="mrm8488/bert-tiny-finetuned-squadv2"
+)
+
+qa_pipeline({
+    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
+    'question': "Who has been working hard for hugginface/transformers lately?"
+
+})
+
+# Output:
+```
+
+```json
+{
+  "answer": "Manuel Romero",
+  "end": 13,
+  "score": 0.05684709993458714,
+  "start": 0
+}
+```
+
+### Yes! That was easy 🎉 Let's try with another example
+
+```python
+qa_pipeline({
+    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
+    'question': "For which company has worked Manuel Romero?"
+})
+
+# Output:
+```
+
+```json
+{
+  "answer": "hugginface/transformers",
+  "end": 79,
+  "score": 0.11613431826808274,
+  "start": 56
+}
+```
+
+### It works!! 🎉 🎉 🎉
+
+> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
+
+> Made with <span style="color: #e25555;">&hearts;</span> in Spain

config.json

@@ -0,0 +1,20 @@
+{
+  "architectures": [
+    "BertForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "output_past": true,
+  "pad_token_id": 0,
+  "type_vocab_size": 2,
+  "vocab_size": 30522
+}

flax_model.msgpack

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ed0e94d84dbc40d763d5fff5b11ce849426755c312978e4431586b6ecd5dde7
+size 17480080

gitattributes.txt

@@ -0,0 +1,10 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85de102568316fe22c76f686893a1cca0df6fee00f432fd0f5600b649bce8649
+size 17549310

model.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73a283022b2553b466662c883cc362832862f69c5be8bee62ae9c629ea66c6eb
+size 16258966

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b0e28d9e21beb1df101fd56b6290f4fe7ad70a5daf101251d594efe3bd80752
+size 17553971

pytorch_model.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcc403a1c3fd5a8e2093dff046f6f35ba7a454ce5bf1b8e1ad8d0f697e0c6bd8
+size 16260877

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": false}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c26310a645b2de935c72b13d46fc84f6ad0ac3dff8f756e1202d86a0b4857e2
+size 1491

training_args.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10bd9a1d54114ef0ffed0f4fce87e26e2893d94deb467851675ad19ea211ac7f
+size 1073