Model Card abhilashkrish/nursing-pharmacology:
- base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
- library_name: peft
- license: mit
- datasets:
- timzhou99/nursing-pharmacology
- language:
- en
- description:
- Healthcare Inference Engine
Model Summary
abhilashkrish/nursing-pharmacology is a domain-specific large language model fine-tuned to provide responses for healthcare training and related applications. Designed to assist healthcare professionals, this model excels at generating precise explanations and adhering to controlled-substance protocols. With over 41 million parameters, it leverages state-of-the-art fine-tuning techniques to optimize for domain specificity and accuracy.
Model Details
Model Description
- Developed by: Abhilash Krishnan
- Model type: Fine-tuned version of
meta-llama-3.1-8b-bnb-4bit
- Language(s): English
- License: MIT
- Finetuned from model:
unsloth/meta-llama-3.1-8b-bnb-4bit
Uses
Direct Use
This model can be directly used for:
- Assisting healthcare professionals with training tasks.
- Providing detailed explanations for controlled-substance protocols.
- Serving as an inference engine in EdTech platforms for medical training.
Downstream Use
With further fine-tuning, this model can be adapted for:
- Broader healthcare applications like patient assistance tools.
- Domain-specific professional training across industries.
Out-of-Scope Use
- General-purpose conversation or text generation beyond the healthcare domain.
- Use cases requiring multilingual or culturally specific nuances outside of English.
Bias, Risks, and Limitations
Bias
- While the model is fine-tuned on healthcare-specific data, it may not account for regional differences in medical practices or laws.
Risks
- Incorrect or outdated training data may lead to inaccuracies in responses, particularly in critical medical contexts.
Limitations
- The model is trained exclusively on English-language datasets and is not designed for multilingual support or non-healthcare applications.
How to Get Started with the Model
Here’s a simple example to load and use the model:
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("abhilashkrish/nursing-pharmacology")
model = AutoModelForCausalLM.from_pretrained("abhilashkrish/nursing-pharmacology")
# Generate a response
input_text = "What are the protocols for handling controlled substances?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Model Training Details
- training_details:
- training_data:
- dataset: timzhou99/nursing-pharmacology
- description:
- The dataset includes text focused on nursing pharmacology and healthcare-specific training materials.
- training_procedure:
- preprocessing:
- Tokenized using Hugging Face's AutoTokenizer with healthcare domain-specific vocabulary.
- training_regime:
- Mixed precision with bfloat16 to optimize GPU memory usage and accelerate fine-tuning.
- hyperparameters:
- learning_rate: 4e-5
- batch_size: 8
- gradient_accumulation_steps: 4
- epochs: 5
- preprocessing:
- compute_infrastructure:
- hardware: NVIDIA A40 GPU (44 GB VRAM)
- software:
- PEFT_library: v0.13.2
- PyTorch: 2.5.1+cu12
- CUDA: 12.4
- training_data:
- evaluation:
- testing_data:
- dataset: timzhou99/nursing-pharmacology
- description:
- The model was evaluated on a subset of the timzhou99/nursing-pharmacology dataset using specific healthcare-related tasks.
- metrics:
- perplexity:
- Evaluated as a measure of model fluency.
- domain_specific_accuracy:
- Assessed based on its ability to generate accurate healthcare protocols.
- perplexity:
- testing_data:
- model_card_contact:
- contact: Abhilash Krishnan
- email: abhilashk.cse@gmail.com
- framework_versions:
- PEFT: 0.13.2
- Transformers: 4.46.3
- Downloads last month
- 23