peft safetensors arxiv:1910.09700 base_model:NousResearch/Llama-2-7b-chat-hf region:us
---
library_name: peft
base_model: NousResearch/Llama-2-7b-chat-hf
---

# Model Card for Model ID

The model is a variant of Llama-2 finetuned on Intel Cloud with a dataset of 120k pubmed articles to best summarize each article. 


## Model Details

### Model Description


- **Developed by:** Infogenesis
- **Funded by [optional]:** Hacklytics 2024, Intel
- **Model type:** Large Language Model
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Finetuned from model [optional]:** Llama-2-7b-chat-hf

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/ShivanshSharma04/HacklyticsProject

## Uses

This model is fine-tuned on the summarization and analysis of schoarly medical articles, and as such, it should primarily be used for the summarization and analysis of said models.

### Direct Use


This model is intended solely for the summarization and analysis of schoarly medical articles. 

Here is the code to run the model directly:
```
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

peft_model_id = "AndyYu25/hacklytics24-medsummarizer"
model_id = "NousResearch/Llama-2-7b-chat-hf"

model = AutoModelForCausalLM.from_pretrained(model_id)
model.load_adapter(peft_model_id)

pipe = pipeline(task="text-generation", model=model)


result = pipe(f"<s>[INST] PROMPT [/INST]")


print(result)
```

### Downstream Use [optional]

Fine-tuned variations of this model can probably be extrapolated to parse non-medical scholarly articles and perform a broader set of tasks that merely summarization.

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

The model, regardless of finetuning, should not be used to generate medical advice without Prediction Guard API, as it may hallucinate false medical information, something can be detrimental to the health of anyone that makes medical decisions based on the model. 

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The model has not been fine-tuned outside the scope of medical journals published in the United States, so it may have an American-centric slant when interpreting or summarizing articles.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

The model can be implemented the same way as Llama-2

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The AI was trained on 120,000 open-access PubMed articles provided by the National Institute of Health. Information about accessing the API for PubMed can be found here: https://www.ncbi.nlm.nih.gov/books/NBK25499/

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

Each paper in the dataset was split into the main body and the abstract, where the abstract would be a hypothetical summary that the model was asked to create. 

#### Training Hyperparameters

- **Training regime:** fp16

Learning rate: 2e-4
Batch size: 4

## Evaluation

Evaluation was based on a 80-10-10 train-test-validate split.

### Testing Data, Factors & Metrics

#### Testing Data

The testing data was a randomly selected sample of ~12,000 articles from the general dataset.


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** Intel Max Series GPU (4 GPUs with 4th Gen CPU, 2 sockets, 256 GB memory, 2 TB disk with OneAPI base kit and tooling)
- **Hours used:** 1 hour
- **Cloud Provider:** Intel
- **Compute Region:** us-region-1
- **Carbon Emitted:** <0.1 kg CO2


## Model Card Contact

ayu303@gatech.edu

### Framework versions

- PEFT 0.8.2