safetensors generated_from_trainer base_model:mistralai/Mistral-7B-v0.1 license:apache-2.0 region:us
---
license: apache-2.0
base_model: mistralai/Mistral-7B-v0.1
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: Mistral-7B-v0.1_cola_rsb_relu
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-v0.1_cola_rsb_relu

This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.3524
- Accuracy: 0.6577

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 2
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 2
- total_train_batch_size: 256
- total_eval_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 750

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 2.5204        | 0.33  | 10   | 2.7397          | 0.5503   |
| 2.415         | 0.66  | 20   | 2.5289          | 0.5475   |
| 2.208         | 0.98  | 30   | 2.2658          | 0.5417   |
| 1.4886        | 1.31  | 40   | 1.9140          | 0.5580   |
| 1.0463        | 1.64  | 50   | 1.5899          | 0.5436   |
| 0.9207        | 1.97  | 60   | 1.1666          | 0.5916   |
| 0.6876        | 2.3   | 70   | 0.9105          | 0.6290   |
| 0.7447        | 2.62  | 80   | 0.7802          | 0.6385   |
| 0.664         | 2.95  | 90   | 0.7189          | 0.6270   |
| 0.6109        | 3.28  | 100  | 0.6891          | 0.6539   |
| 0.5817        | 3.61  | 110  | 0.6786          | 0.6625   |
| 0.6261        | 3.93  | 120  | 0.6674          | 0.6280   |
| 0.5419        | 4.26  | 130  | 0.6621          | 0.6577   |
| 0.5489        | 4.59  | 140  | 0.6757          | 0.6625   |
| 0.4903        | 4.92  | 150  | 0.6603          | 0.6596   |
| 0.495         | 5.25  | 160  | 0.6607          | 0.6520   |
| 0.5505        | 5.57  | 170  | 0.6742          | 0.6309   |
| 0.5342        | 5.9   | 180  | 0.6733          | 0.6558   |
| 0.4534        | 6.23  | 190  | 0.6735          | 0.6539   |
| 0.4728        | 6.56  | 200  | 0.6843          | 0.6577   |
| 0.4911        | 6.89  | 210  | 0.7004          | 0.6510   |
| 0.399         | 7.21  | 220  | 0.7090          | 0.6644   |
| 0.4304        | 7.54  | 230  | 0.7368          | 0.6750   |
| 0.4325        | 7.87  | 240  | 0.7183          | 0.6616   |
| 0.3899        | 8.2   | 250  | 0.7379          | 0.6568   |
| 0.3659        | 8.52  | 260  | 0.7466          | 0.6644   |
| 0.4117        | 8.85  | 270  | 0.7640          | 0.6529   |
| 0.3277        | 9.18  | 280  | 0.7780          | 0.6242   |
| 0.3311        | 9.51  | 290  | 0.7982          | 0.6817   |
| 0.337         | 9.84  | 300  | 0.8088          | 0.6625   |
| 0.297         | 10.16 | 310  | 0.8086          | 0.6500   |
| 0.2856        | 10.49 | 320  | 0.8621          | 0.6644   |
| 0.2939        | 10.82 | 330  | 0.8666          | 0.6357   |
| 0.2517        | 11.15 | 340  | 0.9056          | 0.6769   |
| 0.2726        | 11.48 | 350  | 0.8940          | 0.6318   |
| 0.2985        | 11.8  | 360  | 0.9314          | 0.6529   |
| 0.1915        | 12.13 | 370  | 0.9358          | 0.6731   |
| 0.2081        | 12.46 | 380  | 0.9824          | 0.6405   |
| 0.2147        | 12.79 | 390  | 1.0395          | 0.6577   |
| 0.1644        | 13.11 | 400  | 1.0678          | 0.6606   |
| 0.154         | 13.44 | 410  | 1.1975          | 0.6462   |
| 0.1734        | 13.77 | 420  | 1.2171          | 0.6510   |
| 0.1008        | 14.1  | 430  | 1.2565          | 0.6596   |
| 0.1104        | 14.43 | 440  | 1.3389          | 0.6318   |
| 0.1125        | 14.75 | 450  | 1.3177          | 0.6500   |
| 0.067         | 15.08 | 460  | 1.4552          | 0.6520   |
| 0.0897        | 15.41 | 470  | 1.4729          | 0.6548   |
| 0.0945        | 15.74 | 480  | 1.4543          | 0.6500   |
| 0.0602        | 16.07 | 490  | 1.6014          | 0.6491   |
| 0.0525        | 16.39 | 500  | 1.6417          | 0.6337   |
| 0.0701        | 16.72 | 510  | 1.5756          | 0.6414   |
| 0.0413        | 17.05 | 520  | 1.6458          | 0.6491   |
| 0.0885        | 17.38 | 530  | 1.6182          | 0.6443   |
| 0.0318        | 17.7  | 540  | 1.6987          | 0.6405   |
| 0.0498        | 18.03 | 550  | 1.6695          | 0.6596   |
| 0.0531        | 18.36 | 560  | 1.7738          | 0.6347   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.1+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0
