README.md

---
language:
- ko
- en
tags:
- bert
- korean-english
- clinical nlp
- pharmacovigilance
- adverse events
license: mit
base_model:
- skt/kobert-base-v1
---

# KAERS-BERT

## Model Description

KAERS-BERT is a domain-specific Korean BERT model specialized for clinical text analysis, particularly for processing adverse drug event (ADE) narratives. It was developed by pretraining KoBERT (developed by SK Telecom) using 1.2 million ADE narratives reported through the Korea Adverse Event Reporting System (KAERS) between January 2015 and December 2019.

The model is specifically designed to handle clinical texts where code-switching between Korean and English is frequent, making it particularly effective for processing medical terms and abbreviations in a bilingual context.

## Usage
You can load the model from HuggingFace Hub while using the local tokenizer:

```python
from transformers import BertForPreTraining
from tokenization_kobert import KoBERTTokenizer
```

# Load model from HuggingFace 
```python
model = BertForPreTraining.from_pretrained("kimsiun/kaers-bert-241101")
```

# Load tokenizer from local file
```python
tokenizer = KoBERTTokenizer.from_pretrained('skt/kobert-base-v1')
```

## Key Features

- Specialized in clinical and pharmaceutical domain text
- Handles Korean-English code-switching common in medical texts
- Optimized for processing adverse drug event narratives
- Built upon KoBERT architecture with domain-specific pretraining

## Training Data

The model was pretrained on:
- 1.2 million ADE narratives from KAERS
- Training data specifically focused on 'disease history in detail' and 'adverse event in detail' sections
- Masked language modeling with 15% token masking rate
- Maximum sequence length of 200
- Learning rate: 5×10^-5

## Performance

The model demonstrated strong performance in various NLP tasks related to drug safety information extraction:
- Named Entity Recognition (NER): 83.81% F1-score
- Sentence Extraction: 76.62% F1-score
- Relation Extraction: 64.37% F1-score (weighted)
- Label Classification:
  - 'Occurred' Label: 81.33% F1-score
  - 'Concerned' Label: 77.62% F1-score

When applied to the KAERS database, the model achieved an average increase of 3.24% in data completeness for structured data fields.

## Intended Use

This model is designed for:
- Extracting drug safety information from clinical narratives
- Processing Korean medical texts with English medical terminology
- Supporting pharmacovigilance activities
- Improving data quality in adverse event reporting systems

## Limitations

- The model is specifically trained on adverse event narratives and may not generalize well to other clinical domains
- Performance may vary for texts significantly different from KAERS narratives
- The model works best with Korean clinical texts containing English medical terminology

## Citation

```bibtex
@article{kim2023automatic,
  title={Automatic Extraction of Comprehensive Drug Safety Information from Adverse Drug Event Narratives in the Korea Adverse Event Reporting System Using Natural Language Processing Techniques},
  author={Kim, Siun and Kang, Taegwan and Chung, Tae Kyu and Choi, Yoona and Hong, YeSol and Jung, Kyomin and Lee, Howard},
  journal={Drug Safety},
  volume={46},
  pages={781--795},
  year={2023}