Upload folder using huggingface_hub

README.md

@@ -1,3 +1,73 @@
+
 ---
+language: ko
+tags:
+- bert
+- korean-english
+- clinical nlp
+- pharmacovigilance
+- adverse events
 license: mit
 ---
+
+# KAERS-BERT
+
+## Model Description
+
+KAERS-BERT is a domain-specific Korean BERT model specialized for clinical text analysis, particularly for processing adverse drug event (ADE) narratives. It was developed by pretraining KoBERT (developed by SK Telecom) using 1.2 million ADE narratives reported through the Korea Adverse Event Reporting System (KAERS) between January 2015 and December 2019.
+
+The model is specifically designed to handle clinical texts where code-switching between Korean and English is frequent, making it particularly effective for processing medical terms and abbreviations in a bilingual context.
+
+## Key Features
+
+- Specialized in clinical and pharmaceutical domain text
+- Handles Korean-English code-switching common in medical texts
+- Optimized for processing adverse drug event narratives
+- Built upon KoBERT architecture with domain-specific pretraining
+
+## Training Data
+
+The model was pretrained on:
+- 1.2 million ADE narratives from KAERS
+- Training data specifically focused on 'disease history in detail' and 'adverse event in detail' sections
+- Masked language modeling with 15% token masking rate
+- Maximum sequence length of 200
+- Learning rate: 5×10^-5
+
+## Performance
+
+The model demonstrated strong performance in various NLP tasks related to drug safety information extraction:
+- Named Entity Recognition (NER): 83.81% F1-score
+- Sentence Extraction: 76.62% F1-score
+- Relation Extraction: 64.37% F1-score (weighted)
+- Label Classification:
+  - 'Occurred' Label: 81.33% F1-score
+  - 'Concerned' Label: 77.62% F1-score
+
+When applied to the KAERS database, the model achieved an average increase of 3.24% in data completeness for structured data fields.
+
+## Intended Use
+
+This model is designed for:
+- Extracting drug safety information from clinical narratives
+- Processing Korean medical texts with English medical terminology
+- Supporting pharmacovigilance activities
+- Improving data quality in adverse event reporting systems
+
+## Limitations
+
+- The model is specifically trained on adverse event narratives and may not generalize well to other clinical domains
+- Performance may vary for texts significantly different from KAERS narratives
+- The model works best with Korean clinical texts containing English medical terminology
+
+## Citation
+
+```bibtex
+@article{kim2023automatic,
+  title={Automatic Extraction of Comprehensive Drug Safety Information from Adverse Drug Event Narratives in the Korea Adverse Event Reporting System Using Natural Language Processing Techniques},
+  author={Kim, Siun and Kang, Taegwan and Chung, Tae Kyu and Choi, Yoona and Hong, YeSol and Jung, Kyomin and Lee, Howard},
+  journal={Drug Safety},
+  volume={46},
+  pages={781--795},
+  year={2023}
+}

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "monologg/kobert",
+  "architectures": [
+    "BertForPreTraining"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.31.0.dev0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 8002
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b00afd760a17c718872dffa81038a8f708672e4e132de711bf7b27d41e43cbea
+size 371225993

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17dc471055592d3cc9e0a5831e769246a8a001a4d27551c9ed79668173c7b407
+size 371427

tokenizer_config.json

@@ -0,0 +1,24 @@
+{
+  "additional_special_tokens": null,
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "keep_accents": false,
+  "mask_token": {
+    "__type": "AddedToken",
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "remove_space": true,
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "KoBERTTokenizer",
+  "unk_token": "[UNK]"
+}