
# Meditron-Guanaco-7B-Chat
This is Guanaco family chat model finetuned from base [`epfl-llm/meditron-7b`](https://huggingface.co/epfl-llm/meditron-7b) with [open assist dataset](https://huggingface.co/datasets/mlabonne/guanaco-llama2) using SFT [QLora](https://arxiv.org/abs/2305.14314) .<br>
All the linear parameters were made trainable with a rank of 64.<br>

# Prompt template: Llama

```
'<s> [INST] <<SYS>>
You are a helpful, respectful and medical honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
<</SYS>> {question} [/INST] {Model answer } </s>'
```

# Usage:
```python
model_name='jiviadmin/meditron-7b-guanaco-chat'

# Load the model
base_model = AutoModelForCausalLM.from_pretrained(
model_name,
low_cpu_mem_usage=True,
return_dict=True,
torch_dtype=torch.float16,
device_map={"": 0},
)
# Load tokenizer to save it
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True,add_eos_token=True)
tokenizer.add_special_tokens({'pad_token': '[PAD]'})
tokenizer.pad_token_id = 18610
tokenizer.padding_side = "right"

default_system_prompt="You are a helpful, respectful and honest medical assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.Please consider the context below if applicable:
Context:NA"

#Initialize the hugging face pipeline
def format_prompt(question):
  return f'''<s> [INST] <<SYS>> {default_system_prompt} <</SYS>> [INST] {question} [/INST]'''

question=' My father has a big white colour patch inside of his right cheek. please suggest a reason.'

pipe = pipeline(task="text-generation", model=base_model, tokenizer=tokenizer, max_length=512,repetition_penalty=1.1,return_full_text=False)
result = pipe(format_prompt(question))
answer=result[0]['generated_text']
print(answer)
```
{"datasets": ["timdettmers/openassistant-guanaco", "mlabonne/guanaco-llama2"], "pipeline_tag": "text-generation"}
transformers pytorch llama text-generation dataset:timdettmers/openassistant-guanaco dataset:mlabonne/guanaco-llama2 arxiv:2305.14314 autotrain_compatible endpoints_compatible text-generation-inference region:us