---
license: other
base_model: microsoft/phi-2
tags:
- generated_from_trainer
- medical
- peft
- 'lora '
model-index:
- name: Thealth-phi-2-tunned-9_medalpaca_medical_meadow
  results: []
datasets:
- medalpaca/medical_meadow_mediqa
- medalpaca/medical_meadow_mmmlu
- medalpaca/medical_meadow_medical_flashcards
- medalpaca/medical_meadow_wikidoc_patient_information
- medalpaca/medical_meadow_wikidoc
- medalpaca/medical_meadow_pubmed_causal
- medalpaca/medical_meadow_medqa
- medalpaca/medical_meadow_health_advice
- medalpaca/medical_meadow_cord19
pipeline_tag: conversational
---
![Medical Phi Symbol Cartoon](https://github.com/mostafadentist/huggingface/blob/main/medical_phi_%CF%95_cartoon.png?raw=true)
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Thealth-phi-2-tunned-9_medalpaca_medical_meadow

This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 6.6588

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

Training is done one 9 medalpaca/medical_meadow datasets combined and splited to 90% train and 10% Evaluation 

| Dataset                                               |
|:-----------------------------------------------------:|
| medalpaca/medical_meadow_mediqa                       |
| medalpaca/medical_meadow_mmmlu                        | 
| medalpaca/medical_meadow_medical_flashcards           | 
| medalpaca/medical_meadow_wikidoc_patient_information  | 
| medalpaca/medical_meadow_wikidoc                      | 
| medalpaca/medical_meadow_pubmed_causal                | 
| medalpaca/medical_meadow_medqa                        | 
| medalpaca/medical_meadow_health_advice                |
| medalpaca/medical_meadow_cord19                       |

## Training procedure
Used different tokenizer [stanford-crfm/BioMedLM](https://huggingface.co/stanford-crfm/BioMedLM)

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 1000

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 6.8245        | 0.0   | 500  | 6.7654          |
| 6.7944        | 0.0   | 1000 | 6.6588          |


### Usage
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("stanford-crfm/BioMedLM", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("TachyHealthResearch/Thealth-phi-2-tunned-9_medalpaca_medical_meadow", trust_remote_code=True, torch_dtype=torch.float32)
```

```python
inputs = tokenizer(
    """
    question: ****** ? answer:
    """,
    return_tensors="pt",
    return_attention_mask=False)
```

```python
outputs = model.generate(**inputs, max_length=512)
text = tokenizer.batch_decode(outputs)[0]
print(text)
```

### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0