
# TinyLlama 1.1B Medical 🤏🦙

### Model Description

A smaller version of https://huggingface.co/therealcyberlord/llama2-qlora-finetuned-medical, which used Llama 2 7B. 

Finetuned on <|user|> <|assistant|> instructions


## How to Get Started with the Model

```
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM

config = PeftConfig.from_pretrained("therealcyberlord/TinyLlama-1.1B-Medical")
model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model = PeftModel.from_pretrained(model, "therealcyberlord/TinyLlama-1.1B-Medical")
```

## Training Details

### Training Data

Used two data sources:

**BI55/MedText**: https://huggingface.co/datasets/BI55/MedText

**MedQuad-MedicalQnADataset**: https://huggingface.co/datasets/keivalya/MedQuad-MedicalQnADataset


### Training Procedure 

Trained on 1000 steps on a shuffled **combined** dataset


### Framework versions

- PEFT 0.7.2.dev0
{"library_name": "peft", "datasets": ["BI55/MedText", "keivalya/MedQuad-MedicalQnADataset"], "base_model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0", "pipeline_tag": "text-generation"}
peft safetensors text-generation dataset:BI55/MedText dataset:keivalya/MedQuad-MedicalQnADataset base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0 region:us