Update README.md

e1d5826 verified 7 months ago

8.35 kB

	---
	library_name: transformers
	tags:
	- SFT
	- PEFT
	- Mental Health
	- Psychotherapy
	- Fine-tuning
	- Text Generation
	- Chatbo
	license: mit
	datasets:
	- mpingale/mental-health-chat-dataset
	- Amod/mental_health_counseling_conversations
	- heliosbrahma/mental_health_chatbot_dataset
	---

	# Model Card for Model ID

	<!-- Provide a quick summary of what the model is/does. -->
	A fine-tuned Llama-3-8B-instruct trained to provide initial support and guidance in mental health by leveraging the power of large language models (LLMs).
	It's a fine-tuned version of the LLaMA 3 model specifically trained on mental health counseling conversations.

	<img src="https://cdn-uploads.huggingface.co/production/uploads/65f2cc4457fbd3c9fb65b34c/ujViqdS9zgW-IrEFbAKLw.jpeg" width="480" style="width:400px;">



	## Model Details

	Llama-3-8B-chat-psychotherapist

	Provides active listening: The model is trained to follow conversation flows and respond attentively to user expressions.
	Offers empathetic support: The model's responses are designed to be supportive and understanding, fostering a safe space for users to share their concerns.
	Guides users towards self-reflection: The model can ask open-ended questions and prompt users to explore their thoughts and feelings.
	Offers resources and information: Depending on the conversation, the model can provide users with relevant mental health resources and information.
	Important to Note:

	**This model is intended for initial support and guidance, not a replacement for professional mental health care.
	It's crucial to emphasize that users should seek professional help if needed.
	The model is still under development, and its responses may require human oversight for accuracy and safety.**

	Further Considerations:
	You can specify the type of mental health conversations the model is trained on (e.g., anxiety, depression, general well-being).
	Briefly mention the dataset used for fine-tuning to showcase the model's training grounds.
	If applicable, mention any limitations of the model, such as inability to diagnose or provide specific treatment plans.



	## How to Get Started with the Model
	Use the code below to get started with the model.

	```

	from transformers import AutoTokenizer, AutoModel

	tokenizer = AutoTokenizer.from_pretrained("zementalist/llama-3-8B-chat-psychotherapist")
	model = AutoModel.from_pretrained("zementalist/llama-3-8B-chat-psychotherapist")

	dialogue = "I feel like I don't exist and my body is not my own, I'm sombody else observing me, what causes these ideas?"

	messages = [
	{"role": "system", "content": "Reply to the following inquiry:"},
	{"role": "user", "content": dialogue}
	]

	input_ids = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	return_tensors="pt"
	).to(model.device)


	terminators = [
	tokenizer.eos_token_id,
	tokenizer.convert_tokens_to_ids("<\|eot_id\|>")
	]

	outputs = model.generate(
	input_ids,
	max_new_tokens=256,
	eos_token_id=terminators,
	do_sample=True,
	temperature=0.01
	)
	response = outputs[0][input_ids.shape[-1]:]
	output = tokenizer.decode(response, skip_special_tokens=True)


	print(output)

	```



	## Training Details

	### Training Data

	1. [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations)
	2. [mpingale/mental-health-chat-dataset](https://huggingface.co/datasets/mpingale/mental-health-chat-dataset)
	3. [heliosbrahma/mental_health_chatbot_dataset](https://huggingface.co/datasets/heliosbrahma/mental_health_chatbot_dataset)


	#### Training Hyperparameters

	- Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

	* epochs: 2
	* learning rate: 6e-5
	* learning rate scheduler type: constant
	* Warmup ratio: 0.03
	* optimizer: paged_adamw_32bit
	* weight_decay: 0.001
	* fp16: True

	## Evaluation

	<!-- This section describes the evaluation protocols and provides the results. -->



	\| Step \| Training Loss \|
	\|---------\|-------------------\|
	\| 100 \| 2.637700 \|
	\| 200 \| 2.350900 \|
	\| 300 \| 2.234500 \|
	\| 400 \| 2.230800 \|
	\| 500 \| 2.194900 \|
	\| 600 \| 2.041100 \|
	\| 700 \| 2.033800 \|
	\| 800 \| 1.988600 \|
	\| 900 \| 1.898900 \|
	\| 1000 \| 1.823600 \|
	\| 1100 \| 1.943700 \|
	\| 1200 \| 1.789900 \|
	\| 1300 \| 1.840700 \|
	\| 1400 \| 1.760900 \|
	\| 1500 \| 1.718400 \|
	\| 1600 \| 1.725600 \|
	\| 1700 \| 1.717600 \|
	\| 1800 \| 1.628200 \|
	\| 1900 \| 1.693300 \|
	\| 2000 \| 1.535600 \|
	\| 2100 \| 1.630100 \|
	\| 2200 \| 1.592300 \|
	\| 2300 \| 1.628400 \|
	\| 2400 \| 1.462500 \|
	\| 2500 \| 1.504500 \|
	\| 2600 \| 1.517500 \|
	\| 2700 \| 1.375100 \|
	\| 2800 \| 1.452800 \|
	\| 2900 \| 1.527200 \|
	\| 3000 \| 1.418600 \|
	\| 3100 \| 1.437700 \|
	\| 3200 \| 1.247900 \|
	\| 3300 \| 1.134900 \|
	\| 3400 \| 1.283800 \|
	\| 3500 \| 1.378600 \|
	\| 3600 \| 1.114600 \|
	\| 3700 \| 1.300900 \|
	\| 3800 \| 1.225700 \|
	\| 3900 \| 1.190000 \|
	\| 4000 \| 1.260200 \|
	\| 4100 \| 1.053300 \|
	\| 4200 \| 1.173000 \|
	\| 4300 \| 1.275900 \|
	\| 4400 \| 1.079800 \|
	\| 4500 \| 1.121100 \|
	\| 4600 \| 1.043100 \|
	\| 4700 \| 1.121000 \|
	\| 4800 \| 1.087200 \|
	\| 4900 \| 1.102400 \|
	\| 5000 \| 1.126100 \|
	\| 5100 \| 1.182200 \|
	\| 5200 \| 1.156500 \|
	\| 5300 \| 0.979700 \|
	\| 5400 \| 1.145300 \|
	\| 5500 \| 1.018100 \|
	\| 5600 \| 1.070500 \|
	\| 5700 \| 0.954900 \|
	\| 5800 \| 1.074200 \|
	\| 5900 \| 1.022800 \|
	\| 6000 \| 1.037700 \|
	\| 6100 \| 0.913400 \|
	\| 6200 \| 0.929500 \|



	### Results

	1. Question: I feel like I don't exist and my body is not my own,
	like if I'm sombody else observin me, what could be this disorder?
	* * Answer: You're describing a phenomenon called depersonalization. It's a common symptom of anxiety disorders, post-traumatic stress disorder (PTSD), and other mental health conditions.

	Depersonalization can make you feel like you're observing yourself from the outside, like you're watching a movie or reading a book about yourself. You might also feel like you're not in control of your body or that you're just a puppet.

	This feeling can be very distressing and can make you wonder if you're even alive. It's a very disorienting and unsettling experience.

	If you're experiencing depersonalization, it's important to seek help from a mental health professional. They can help you identify the underlying causes of your feelings and develop coping strategies to manage them.


	2. Question: I'm always confused when making decisions, I cannot choose one option,
	I must have only one option so I can make a decision, what are the causes of being
	indecisive?
	* * Answer: Indecisiveness is a common phenomenon that many people experience. It's not a mental health condition, but it can be frustrating and time-consuming. Here are some possible causes of indecisiveness:

	1. Fear of making the wrong decision: Sometimes, people are afraid of making the wrong choice, which can lead to indecisiveness. This fear can be due to various factors such as fear of failure, fear of rejection, or fear of the unknown.
	2. Lack of clear goals: When people don't have clear goals, they can be indecisive. This is because they don't know what they want, which can lead to confusion and indecisiveness.
	3. Overthinking: Overthinking can be a major cause of indecisiveness. When people overthink, they can get caught up in their thoughts and be unable to make a decision.
	4. Lack of self-confidence: People who lack self-confidence can be indecisive. This is because they don't believe in themselves and are afraid of making the wrong decision.
	5. Avoidance: Avoiding a decision can be a cause of indecisiveness. When people avoid making a decision, they can be indecisive because they don't know



	- Developed by: [Zeyad Khalid]