Update README.md
Browse files
README.md
CHANGED
@@ -63,15 +63,8 @@ scheduler.remove_pruning()
|
|
63 |
```
|
64 |
For more code examples, refer to the [GitHub Repo](https://github.com/IntelLabs/Model-Compression-Research-Package/tree/main/research/prune-once-for-all).
|
65 |
|
66 |
-
| Factors | Description |
|
67 |
-
| ----------- | ----------- |
|
68 |
-
| Groups | Many Wikipedia articles with question and answer labels are contained in the training data |
|
69 |
-
| Instrumentation | - |
|
70 |
-
| Environment | - |
|
71 |
-
| Card Prompts | - |
|
72 |
-
|
73 |
|
74 |
-
|
75 |
| Model | Model Size | SQuADv1.1 (EM/F1) | MNLI-m (Acc) | MNLI-mm (Acc) | QQP (Acc/F1) | QNLI (Acc) | SST-2 (Acc) |
|
76 |
|-------------------------------|:----------:|:-----------------:|:------------:|:-------------:|:------------:|:----------:|:-----------:|
|
77 |
| [80% Sparse BERT-Base uncased](https://huggingface.co/Intel/bert-base-uncased-squadv1.1-sparse-80-1x4-block-pruneofa) | - | 81.29/88.47 | - | - | - | - | - |
|
@@ -87,7 +80,7 @@ All the results are the mean of two seperate experiments with the same hyper-par
|
|
87 |
| ----------- | ----------- |
|
88 |
| Datasets | SQuAD1.1: "Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable." (https://huggingface.co/datasets/squad)|
|
89 |
| Motivation | To build an efficient and accurate model for the question answering task. |
|
90 |
-
| Preprocessing | "We use the English Wikipedia dataset (2500M words) for training the models on the pre-training task. We split the data into train (95%) and validation (5%) sets. Both sets are preprocessed as described in the models’ original papers ([Devlin et al., 2019](https://arxiv.org/abs/1810.04805), [Sanh et al., 2019](https://arxiv.org/abs/1910.01108)). We process the data to use the maximum sequence length allowed by the models, however, we allow shorter sequences at a probability of 0:1." |
|
91 |
|
92 |
| Ethical Considerations | Description |
|
93 |
| ----------- | ----------- |
|
|
|
63 |
```
|
64 |
For more code examples, refer to the [GitHub Repo](https://github.com/IntelLabs/Model-Compression-Research-Package/tree/main/research/prune-once-for-all).
|
65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
|
67 |
+
### Metrics (Model Performance):
|
68 |
| Model | Model Size | SQuADv1.1 (EM/F1) | MNLI-m (Acc) | MNLI-mm (Acc) | QQP (Acc/F1) | QNLI (Acc) | SST-2 (Acc) |
|
69 |
|-------------------------------|:----------:|:-----------------:|:------------:|:-------------:|:------------:|:----------:|:-----------:|
|
70 |
| [80% Sparse BERT-Base uncased](https://huggingface.co/Intel/bert-base-uncased-squadv1.1-sparse-80-1x4-block-pruneofa) | - | 81.29/88.47 | - | - | - | - | - |
|
|
|
80 |
| ----------- | ----------- |
|
81 |
| Datasets | SQuAD1.1: "Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable." (https://huggingface.co/datasets/squad)|
|
82 |
| Motivation | To build an efficient and accurate model for the question answering task. |
|
83 |
+
| Preprocessing | "We use the English Wikipedia dataset (2500M words) for training the models on the pre-training task. We split the data into train (95%) and validation (5%) sets. Both sets are preprocessed as described in the models’ original papers ([Devlin et al., 2019](https://arxiv.org/abs/1810.04805), [Sanh et al., 2019](https://arxiv.org/abs/1910.01108)). We process the data to use the maximum sequence length allowed by the models, however, we allow shorter sequences at a probability of 0:1." Following the pre-training on Wikipedia, fine-tuning is completed on the SQuAD1.1 dataset. |
|
84 |
|
85 |
| Ethical Considerations | Description |
|
86 |
| ----------- | ----------- |
|