peft pytorch safetensors mistral trl sft generated_from_trainer base_model:mistralai/Mistral-7B-Instruct-v0.2 license:apache-2.0 4-bit bitsandbytes region:us
---
license: apache-2.0
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: mistralai/Mistral-7B-Instruct-v0.2
model-index:
- name: Mistral-7b-Javelin-med
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7b-Javelin-med

This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7900

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 30

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.0565        | 0.03  | 1    | 1.9073          |
| 1.9762        | 0.06  | 2    | 1.8026          |
| 1.7179        | 0.1   | 3    | 1.7420          |
| 1.7005        | 0.13  | 4    | 1.6867          |
| 1.5088        | 0.16  | 5    | 1.6289          |
| 1.5717        | 0.19  | 6    | 1.5767          |
| 1.6219        | 0.23  | 7    | 1.5234          |
| 1.552         | 0.26  | 8    | 1.4657          |
| 1.3237        | 0.29  | 9    | 1.4097          |
| 1.2997        | 0.32  | 10   | 1.3543          |
| 1.1756        | 0.35  | 11   | 1.2965          |
| 1.2622        | 0.39  | 12   | 1.2412          |
| 1.148         | 0.42  | 13   | 1.1946          |
| 1.2081        | 0.45  | 14   | 1.1595          |
| 1.0854        | 0.48  | 15   | 1.1244          |
| 1.2107        | 0.52  | 16   | 1.0912          |
| 1.0053        | 0.55  | 17   | 1.0595          |
| 1.025         | 0.58  | 18   | 1.0267          |
| 1.1565        | 0.61  | 19   | 0.9961          |
| 0.6961        | 0.65  | 20   | 0.9649          |
| 0.9958        | 0.68  | 21   | 0.9365          |
| 0.7097        | 0.71  | 22   | 0.9085          |
| 0.8728        | 0.74  | 23   | 0.8838          |
| 0.4737        | 0.77  | 24   | 0.8610          |
| 0.5576        | 0.81  | 25   | 0.8418          |
| 0.9218        | 0.84  | 26   | 0.8261          |
| 0.8374        | 0.87  | 27   | 0.8123          |
| 0.5547        | 0.9   | 28   | 0.8016          |
| 0.4174        | 0.94  | 29   | 0.7938          |
| 0.8493        | 0.97  | 30   | 0.7900          |


### Framework versions

- PEFT 0.7.1
- Transformers 4.37.0
- Pytorch 2.0.0
- Datasets 2.16.1
- Tokenizers 0.15.1