
# Thesa

Thesa is an experimental project of a therapy chatbot trained on mental health data and fine-tuned with the Zephyr GPTQ model that uses quantization to decrease high computatinal and storage costs.

## Model description

- Fine-tuned from [TheBloke/zephyr-7B-alpha-GPTQ](https://huggingface.co/TheBloke/zephyr-7B-alpha-GPTQ)

## Intended uses & limitations

The intended use is experimental.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 250
- mixed_precision_training: Native AMP


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1

## More info
More info at https://github.com/johnhandleyd/thesa
{"language": ["en"], "license": "mit", "tags": ["trl", "sft", "generated_from_trainer", "peft", "gptq"], "datasets": ["loaiabdalslam/counselchat"], "base_model": "TheBloke/zephyr-7B-alpha-GPTQ", "pipeline_tag": "text-generation", "widget": [{"text": "<|system|>You are a therapist helping patients.<|user|>I'm fighting with my boyfriend and he's not talking to me. I don't know what to do<|assistant|>", "example_title": "Example 1"}], "model-index": [{"name": "thesa", "results": []}]}
peft tensorboard safetensors trl sft generated_from_trainer gptq text-generation en dataset:loaiabdalslam/counselchat base_model:TheBloke/zephyr-7B-alpha-GPTQ license:mit region:us