---
library_name: transformers
tags:
- BC5CDR-disease
- NER
license: apache-2.0
language:
- en
metrics:
- seqeval
base_model:
- distilbert/distilbert-base-uncased
---

# Model Card for Model ID

Fine-tuned distilbert model. Trained on train set of BC5CDR-disease dataset taken from [BLURB](https://microsoft.github.io/BLURB/tasks.html). 


## Model Details


### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/kbulutozler/medical-llm-benchmark



## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Train set of BC5CDR-disease dataset.

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Classical fine-tuning. 

#### Training Hyperparameters

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

learning_rate=5e-5
per_device_train_batch_size=16
per_device_eval_batch_size=16
num_train_epochs=3
weight_decay=0.01


## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

Test set of BC5CDR-disease dataset.

### Results

Precision: 0.76
Recall: 0.81
Micro-F1: 0.79


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

- **Hardware Type:** 1xRTX A4000
- **Hours used:** 00:07:00