Hugging Face's logo

BrunoGR
/
JUST_HEAR_ME-PEFT_Adapter

Text Generation
PEFT
Safetensors
Spanish
Emotional
Assistant
Emotional Support
chatbot
conversational
Model card Files Community
JUST_HEAR_ME-PEFT_Adapter / README.md
BrunoGR's picture
BrunoGR
Update README.md
dacf556 verified
preview code
|
raw
history blame contribute delete
4.1 kB
metadata
base_model: meta-llama/Llama-2-7b-chat-hf
tags:
  - Emotional
  - Assistant
  - Emotional Support
  - chatbot
model-index:
  - name: checkpoint-41_9k-lr5em5-bs15n5_r64LA128dt01_2811
    results: []
license: mit
language:
  - es
datasets:
  - BrunoGR/HEAR-Hispanic_Emotional_Accompaniment_Responses
  - >-
    BrunoGR/HRECPW-Hispanic_Responses_for_Emotional_Classification_based_on_Plutchik_Wheel
library_name: peft
pipeline_tag: text-generation

Sólo Escúchame: PEFT Adapter

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on BrunoGR/HEAR-Hispanic_Emotional_Accompaniment_Responses. It achieves the following results on the evaluation set:

  • Loss: 0.8265

Model Description

Sólo Escúchame is an emotional accompaniment chatbot designed to provide psychological support in Spanish. This model, built upon the LLaMA-2-7b-Chat architecture, has been fine-tuned to deliver empathetic responses tailored to users' emotional states.

Intended Uses & Limitations

This model is intended for use in providing emotional support and companionship. It is not a substitute for professional mental health services. Users should be aware of its limitations in handling severe psychological conditions and emergencies.

Training and Evaluation Data

The model was trained using the "BrunoGR/HEAR-Hispanic_Emotional_Accompaniment_Responses" dataset, which includes diverse emotional responses to simulate a wide range of psychological support scenarios.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 5
  • eval_batch_size: 8
  • seed: 1
  • gradient_accumulation_steps: 3
  • total_train_batch_size: 15
  • optimizer: Adam with betas=(0.9,0.95) and epsilon=2e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 300
  • num_epochs: 3

LoRA Parameters:

  • r: 64
  • Alpha: 128
  • LoRA_DROPOUT: 0.1
  • bias: "none"
  • task_type: "CAUSAL_LM"
  • LoRA_TARGET_MODULES = ["q_proj", "v_proj"]

Training Results

Training Loss Epoch Step Validation Loss
0.7717 0.16 450 0.9065
0.7716 0.32 900 0.8863
0.7856 0.48 1350 0.8679
0.7463 0.64 1800 0.8602
0.7298 0.81 2250 0.8531
0.7426 0.97 2700 0.8415
0.6991 1.13 3150 0.8425
0.6923 1.29 3600 0.8382
0.6917 1.45 4050 0.8361
0.6681 1.61 4500 0.8328
0.6764 1.77 4950 0.8297
0.6684 1.93 5400 0.8275
0.6395 2.09 5850 0.8268
0.6735 2.25 6300 0.8272
0.6416 2.42 6750 0.8275
0.651 2.58 7200 0.8265
0.6514 2.74 7650 0.8265
0.6535 2.9 8100 0.8265

Framework Versions

  • Transformers 4.35.2
  • Pytorch 2.1.1+cu121
  • Datasets 2.16.0
  • Tokenizers 0.15.0
  • torch: 2.1.1
  • trl: 0.7.5
  • tokenizers: 0.15.0
  • peft: 0.7.0
  • evaluate: 0.4.1
  • accelerate: 0.25.0
  • seaborn: 0.13.0

License

Sólo Escúchame is released under the MIT License.

Citation

If you use this Adapter in your research, please cite the following paper:

@misc{ramírez2024soloescuchamespanishemotional,
      title={S\'olo Esc\'uchame: Spanish Emotional Accompaniment Chatbot}, 
      author={Bruno Gil Ramírez and Jessica López Espejel and María del Carmen Santiago Díaz and Gustavo Trinidad Rubín Linares},
      year={2024},
      eprint={2408.01852},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2408.01852}, 
}

Contact

For any questions or inquiries, please contact: