|
--- |
|
language: zh |
|
--- |
|
|
|
# ernie-health-zh |
|
|
|
## Introduction |
|
ERNIE-health is a Chinese biomedical language model pre-trained from in-domain text of de-identified online doctor-patient dialogues, electronic medical records, and textbooks. |
|
|
|
More detail: |
|
https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/ernie-health/ |
|
https://arxiv.org/pdf/2110.07244.pdf |
|
|
|
## Released Model Info |
|
|
|
|Model Name|Language|Model Structure| |
|
|:---:|:---:|:---:| |
|
|ernie-health-zh| Chinese |Layer:12, Hidden:768, Heads:12| |
|
|
|
This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and |
|
a series of experiments have been conducted to check the accuracy of the conversion. |
|
|
|
- Official PaddlePaddle ERNIE repo:https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/ernie-health/ |
|
- Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch |
|
|
|
## How to use |
|
```Python |
|
from transformers import AutoTokenizer, AutoModel |
|
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-health-zh") |
|
model = AutoModel.from_pretrained("nghuyong/ernie-health-zh") |
|
``` |
|
|
|
## Citation |
|
|
|
```bibtex |
|
@article{wang2021building, |
|
title={Building Chinese Biomedical Language Models via Multi-Level Text Discrimination}, |
|
author={Wang, Quan and Dai, Songtai and Xu, Benfeng and Lyu, Yajuan and Zhu, Yong and Wu, Hua and Wang, Haifeng}, |
|
journal={arXiv preprint arXiv:2110.07244}, |
|
year={2021} |
|
} |
|
``` |
|
|
