julien-c HF staff commited on
Commit
be24ece
1 Parent(s): 479be83

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/csarron/bert-base-uncased-squad-v1/README.md

Files changed (1) hide show
  1. README.md +113 -0
README.md ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ thumbnail:
4
+ license: mit
5
+ tags:
6
+ - question-answering
7
+ - bert
8
+ - bert-base
9
+ datasets:
10
+ - squad
11
+ metrics:
12
+ - squad
13
+ widget:
14
+ - text: "Which name is also used to describe the Amazon rainforest in English?"
15
+ context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
16
+ - text: "How many square kilometers of rainforest is covered in the basin?"
17
+ context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
18
+ ---
19
+
20
+ ## BERT-base uncased model fine-tuned on SQuAD v1
21
+
22
+ This model was fine-tuned from the HuggingFace [BERT](https://www.aclweb.org/anthology/N19-1423/) base uncased checkpoint on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer).
23
+ This model is case-insensitive: it does not make a difference between english and English.
24
+
25
+ ## Details
26
+
27
+ | Dataset | Split | # samples |
28
+ | -------- | ----- | --------- |
29
+ | SQuAD1.1 | train | 90.6K |
30
+ | SQuAD1.1 | eval | 11.1k |
31
+
32
+
33
+ ### Fine-tuning
34
+ - Python: `3.7.5`
35
+
36
+ - Machine specs:
37
+
38
+ `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz`
39
+
40
+ `Memory: 32 GiB`
41
+
42
+ `GPUs: 2 GeForce GTX 1070, each with 8GiB memory`
43
+
44
+ `GPU driver: 418.87.01, CUDA: 10.1`
45
+
46
+ - script:
47
+
48
+ ```shell
49
+ # after install https://github.com/huggingface/transformers
50
+
51
+ cd examples/question-answering
52
+ mkdir -p data
53
+
54
+ wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json
55
+
56
+ wget -O data/dev-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json
57
+
58
+ python run_squad.py \
59
+ --model_type bert \
60
+ --model_name_or_path bert-base-uncased \
61
+ --do_train \
62
+ --do_eval \
63
+ --do_lower_case \
64
+ --train_file train-v1.1.json \
65
+ --predict_file dev-v1.1.json \
66
+ --per_gpu_train_batch_size 12 \
67
+ --per_gpu_eval_batch_size=16 \
68
+ --learning_rate 3e-5 \
69
+ --num_train_epochs 2.0 \
70
+ --max_seq_length 320 \
71
+ --doc_stride 128 \
72
+ --data_dir data \
73
+ --output_dir data/bert-base-uncased-squad-v1 2>&1 | tee train-energy-bert-base-squad-v1.log
74
+ ```
75
+
76
+ It took about 2 hours to finish.
77
+
78
+ ### Results
79
+
80
+ **Model size**: `418M`
81
+
82
+ | Metric | # Value | # Original ([Table 2](https://www.aclweb.org/anthology/N19-1423.pdf))|
83
+ | ------ | --------- | --------- |
84
+ | **EM** | **80.9** | **80.8** |
85
+ | **F1** | **88.2** | **88.5** |
86
+
87
+ Note that the above results didn't involve any hyperparameter search.
88
+
89
+ ## Example Usage
90
+
91
+
92
+ ```python
93
+ from transformers import pipeline
94
+
95
+ qa_pipeline = pipeline(
96
+ "question-answering",
97
+ model="csarron/bert-base-uncased-squad-v1",
98
+ tokenizer="csarron/bert-base-uncased-squad-v1"
99
+ )
100
+
101
+ predictions = qa_pipeline({
102
+ 'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
103
+ 'question': "What day was the game played on?"
104
+ })
105
+
106
+ print(predictions)
107
+ # output:
108
+ # {'score': 0.8730505704879761, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}
109
+ ```
110
+
111
+ > Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp)
112
+
113
+ > Made with ❤️ in New York.