lordChipotle/SuicidePreventionLlama

To see a detailed Notebook of our training approach: https://colab.research.google.com/drive/1vbio7VWmkpQoTnDUg32TABxxf4VcBBeY?usp=sharing

Suicide and Mental Health Support LLaMA

This model is a fine-tuned LLaMA-based (or derivative) model designed to (1) detect suicidal or self-harm risk in text, and (2) provide a short therapeutic-style reply if suicidality is detected. We combined multiple datasets to train this model, including:

• Reddit-based suicide detection data (r/SuicideWatch, r/depression, r/teenagers),
• Twitter suicidal-intent classification data,
• CounselChat: a dataset of mental-health counseling Q&A,
• PAIR: short counseling interactions with high- and medium-quality reflections.

DISCLAIMER: This model is not a substitute for professional mental-health services or emergency intervention. If you or someone you know is in crisis, seek professional help (e.g., call emergency services or hotlines like 988 in the US). This model may be incorrect or incomplete. Use responsibly, and see Limitations below.

---

Model Details

• Base Model: LLaMA-based architecture
• Parameter-Efficient Fine-tuning: We used LoRA adapters or 4-bit quantization to reduce GPU memory usage.
• Data:
  1. Suicide detection (Reddit & Twitter) – labeled as “suicidal” vs. “non-suicidal.”
  2. Therapeutic Q&A (CounselChat & PAIR) – used to produce empathetic, reflective responses.
• Intended Use:
  • For research on suicidal ideation detection and mental-health conversation modeling.
  • For demonstration or proof-of-concept.

---

Training Approach

To see a detailed Notebook of our training approach: https://colab.research.google.com/drive/1vbio7VWmkpQoTnDUg32TABxxf4VcBBeY?usp=sharing

1. Data Preprocessing: We unified suicidal posts as "suicidal" and non-suicidal posts as "non-suicidal".
2. Multi-Task Instruction: We used short prompts for classification tasks, and Q&A style prompts for therapy.
3. Oversampling: To ensure the model doesn’t just classify everything as “suicidal,” we oversampled the therapy data.
4. Hyperparameters:
   • Batch Size: 2
   • Max Steps: 60 (example short run)
   • Learning Rate: 2e-4
   • Mixed Precision (fp16) or bf16 depending on the GPU

---

Usage

Classification Example:

from transformers import AutoTokenizer, AutoModelForCausalLM
# or from unsloth import FastLanguageModel if you used Unsloth

text = "Life is too painful. I'm done. I want to end it."

# 1) Classify
classification = model("Determine if the following text is suicidal:\n" + text)
print("Classification:", classification)
# e.g., "suicidal"

# 2) Therapeutic Response Example:

response = model("Respond like a therapist:\n" + text, max_new_tokens=256)
print("Therapy-Style Reply:", response)

Limitations & Caveats

1. Not a Medical Professional: This model does not replace mental-health professionals.
2. Potential for Harmful or Inaccurate Content: Large language models may produce misleading or harmful text.
3. Biased Data: Reddit, Twitter, or crowd-annotated counseling data can carry biases and incomplete perspectives.
4. Over-Classification or Under-Classification: The model might incorrectly label or fail to detect self-harm.

Ethical and Responsible Use

• Self-Harm & Crisis: If you suspect someone is in crisis, direct them to professional hotlines or emergency resources.

• Data Privacy: The training data might include personal text from Reddit/Twitter. We have made efforts to remove personally identifying information, but use responsibly.

Thank You

Thank you for checking out our model. We hope this can encourage research into safe, responsible, and helpful mental-health assistant approaches. Please reach out or open an issue if you have suggestions or concerns.