README.md

---
base_model: unsloth/mistral-7b-instruct-v0.1-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- mistral
- trl
license: apache-2.0
language:
- en
datasets:
- Laurent1/MedQuad-MedicalQnADataset_128tokens_max
---

# Model Card for Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA

<img src="https://files.oaiusercontent.com/file-SRkkqbc6KKUWGAfvWfrZpA?se=2025-01-11T20%3A14%3A07Z&sp=r&sv=2024-08-04&sr=b&rscc=max-age%3D604800%2C%20immutable%2C%20private&rscd=attachment%3B%20filename%3D9f951e1f-ad60-431b-b016-e4d79f30a3ab.webp&sig=PwbELJUHXlMlgk3T4MoDPH7nVYfPEXN0ypjadk1DuEc%3D" alt="drawing" width="400"/>

<font color="FF0000" size="5"><b>
This is a medical question-answering model fine-tuned for healthcare domain</b></font>

<br><b>Foundation Model: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1<br/>
Dataset: https://huggingface.co/datasets/Laurent1/MedQuad-MedicalQnADataset_128tokens_max<br/></b>

The model has been fine-tuned using CUDA-enabled GPU hardware with optimized training through [Unsloth](https://github.com/unslothai/unsloth).

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="100"/>](https://github.com/unslothai/unsloth)


## Model Details

The model is based upon the foundation model: Mistral-7B-Instruct-v0.1.<br/>
It has been tuned with Supervised Fine-tuning Trainer using the Unsloth optimization framework for faster and more efficient training.

### Libraries
- unsloth
- transformers
- torch
- trl
- peft
- einops
- bitsandbytes
- datasets

## Training Configuration

### Model Parameters
- max_sequence_length = 2048
- load_in_4bit = True
- LoRA rank (r) = 32
- lora_alpha = 16
- lora_dropout = 0

### Target Modules for LoRA
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj

### Training Hyperparameters
- per_device_train_batch_size = 2
- gradient_accumulation_steps = 16
- warmup_steps = 5
- warmup_ratio = 0.03
- max_steps = 1600
- learning_rate = 1e-4
- weight_decay = 0.01
- lr_scheduler_type = "linear"
- optimizer = "paged_adamw_32bit"

## Training Statistics

### Hardware Utilization
- Training duration: 10,561.28 seconds (approximately 176.02 minutes)
- Peak reserved memory: 5.416 GB
- Peak reserved memory for training: 0.748 GB
- Peak reserved memory % of max memory: 13.689%
- Peak reserved memory for training % of max memory: 1.891%

### Dataset
The model was trained on the MedQuad dataset, which contains medical questions and answers. The training data was processed using a chat template format for instruction-tuning.

## Bias, Risks, and Limitations

<font color="FF0000">
Users (both direct and downstream) should be aware of the following:

1. This model is intended for medical question-answering but should not be used as a substitute for professional medical advice.
2. The model's responses should be verified by healthcare professionals before making any medical decisions.
3. Generation of plausible yet incorrect medical information remains a possibility.
4. The model's knowledge is limited to its training data and may not cover all medical conditions or recent medical developments.
</font>

## Usage

The model can be loaded and used with the Unsloth library:

```python
from unsloth import FastLanguageModel
max_seq_length = 2048  # Choose any! We auto support RoPE Scaling internally!
dtype = (
    None  # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
)
model, tokenizer = FastLanguageModel.from_pretrained(
    "bouthros/Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA",
    max_seq_length=2048,
    load_in_4bit=True,
)
```

Example usage:
```python
messages = [
    {"from": "human", "value": "What are the types of liver cancer?"},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")
```

## Model Access

The model is available on Hugging Face Hub at: bouthros/Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA

## Citation

If you use this model, please cite the original Mistral-7B-Instruct-v0.1 model and the MedQuad dataset.