model documentation

README.md

@@ -1,12 +1,114 @@
 ---
+license: apache-2.0
 language: 
 - zh
-license: "apache-2.0"
 ---
 
-## Chinese MRC roberta_wwm_ext_large
 
-* 使用大量中文MRC数据训练的roberta_wwm_ext_large模型，详情可查看：https://github.com/basketballandlearn/MRC_Competition_Dureader
+
+# Model Card for Chinese MRC roberta_wwm_ext_large
+ 
+# Model Details
+ 
+## Model Description
+ 
+使用大量中文MRC数据训练的roberta_wwm_ext_large模型，[详情可查看](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+ 
+- **Developed by:** luhua-rain
+- **Shared by [Optional]:**  luhua-rain
+- **Model type:** Question Answering 
+- **Language(s) (NLP):** Chinese
+- **License:** Apache 2.0 
+- **Parent Model:** BERT
+- **Resources for more information:**
+  - [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+ 	
+
+
+# Uses
+ 
+
+## Direct Use
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+> 此mrc模型可直接用于open domain，点击体验
+ 
+## Downstream Use [Optional]
+ 
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+> 将此模型放到下游 MRC/分类 任务微调可比直接使用预训练语言模型提高2个点/1个点以上
+ 
+## Out-of-Scope Use
+ 
+The model should not be used to intentionally create hostile or alienating environments for people. 
+ 
+# Bias, Risks, and Limitations
+ 
+ 
+Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
+
+
+
+## Recommendations
+ 
+ 
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+
+# Training Details
+ 
+## Training Data
+ 
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+> 网上收集的大量中文MRC数据 （其中包括公开的MRC数据集以及自己爬取的网页数据等， 囊括了医疗、教育、娱乐、百科、军事、法律、等领域。）
+ 
+## Training Procedure
+
+ 
+### Preprocessing
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader): 
+>**清洗**
+舍弃：context>1024的舍弃、question>64的舍弃、网页标签占比超过30%的舍弃。
+重新标注：若answer>64且不完全出现在文档中，则采用模糊匹配: 计算所有片段与answer的相似度(F1值)，取相似度最高的且高于阈值（0.8）
+**数据标注**
+收集的数据有一部分是不包含的位置标签的，仅仅是(问题-文章-答案)的三元组形式。 所以，对于只有答案而没有位置标签的数据通过正则匹配进行位置标注：
+ 若答案片段多次出现在文章中，选择上下文与问题最相似的答案片段作为标准答案（使用F1值计算相似度，答案片段的上文48和下文48个字符作为上下文）；
+ 若答案片段只出现一次，则默认该答案为标准答案。
+采用滑动窗口将长文档切分为多个重叠的子文档，故一个文档可能会生成多个有答案的子文档。
+**无答案数据构造**
+     在跨领域数据上训练可以增加数据的领域多样性，进而提高模型的泛化能力，而负样本的引入恰好能使得模型编码尽可能多的数据，加强模型对难样本的识别能力：
+1.)  对于每一个问题，随机从数据中捞取context，并保留对应的title作为负样本;（50%）
+2.)  对于每一个问题，将其正样本中答案出现的句子删除，以此作为负样本；（20%）
+3.)  对于每一个问题，使用BM25算法召回得分最高的前十个文档，然后根据得分采样出一个context作为负样本， 对于非实体类答案，剔除得分最高的context（30%）
+ 
+ 
+ 
+
+
+ 
+### Speeds, Sizes, Times
+More information needed 
+
+ 
+# Evaluation
+ 
+ 
+## Testing Data, Factors & Metrics
+ 
+### Testing Data
+ 
+More information needed 
+ 
+ 
+### Factors
+More information needed
+ 
+### Metrics
+ 
+More information needed
+ 
+ 
+## Results 
+
 * 此库发布的再训练模型，在 阅读理解/分类 等任务上均有大幅提高<br/>
 （已有多位小伙伴在Dureader-2021等多个比赛中取得**top5**的成绩😁）
 
@@ -19,4 +121,97 @@ license: "apache-2.0"
 | macbert-large (ours)                      | 70.45 / **68.13**|   **83.4**    |
 | roberta-wwm-ext-large (ours)              | 68.91 / 66.91   |    83.1        |
 
+       | 68.91 / 66.91   |    83.1        |
+
+
+
+
+ 
+# Model Examination
+ 
+More information needed
+ 
+# Environmental Impact
+ 
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+ 
+- **Hardware Type:** More information needed
+- **Hours used:** More information needed
+- **Cloud Provider:** More information needed
+- **Compute Region:** More information needed
+- **Carbon Emitted:** More information needed
+ 
+# Technical Specifications [optional]
+ 
+## Model Architecture and Objective
+
+More information needed 
+ 
+## Compute Infrastructure
+ 
+More information needed 
+ 
+### Hardware
+ 
+ 
+More information needed
+ 
+### Software
+ 
+More information needed.
+ 
+# Citation
+
+ 
+**BibTeX:**
+ 
+ 
+More information needed 
+ 
+ 
+ 
+ 
+# Glossary [optional]
+More information needed 
+ 
+# More Information [optional]
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+> 代码上传前已经跑通。文件不多，所以如果碰到报错之类的信息，可能是代码路径不对、缺少安装包等问题，一步步解决，可以提issue
+环境
+ 
+
+ 
+# Model Card Authors [optional]
+ 
+Luhua-rain in collaboration with Ezi Ozoani and the Hugging Face team
+
+
+# Model Card Contact
+ 
+The model authors also note in the [GitHub Repo](https://github.com/basketballandlearn/MRC_Competition_Dureader)
+> 合作
+相关训练数据以及使用更多数据训练的模型/一起打比赛 可邮箱联系(luhua98@foxmail.com)~
+
+ 
+# How to Get Started with the Model
+ 
+Use the code below to get started with the model.
+ 
+<details>
+<summary> Click to expand </summary>
+
+```python
+ ----- 使用方法 -----
+from transformers import AutoTokenizer, AutoModelForQuestionAnswering
+
+model_name = "chinese_pretrain_mrc_roberta_wwm_ext_large" # "chinese_pretrain_mrc_macbert_large"
+
+# Use in Transformers
+tokenizer = AutoTokenizer.from_pretrained(f"luhua/{model_name}")
+model = AutoModelForQuestionAnswering.from_pretrained(f"luhua/{model_name}")
 
+# Use locally（通过 https://huggingface.co/luhua 下载模型及配置文件）
+tokenizer = BertTokenizer.from_pretrained(f'./{model_name}')
+model = AutoModelForQuestionAnswering.from_pretrained(f'./{model_name}')
+ ```
+</details>