safetensors generated_from_trainer base_model:mistralai/Mistral-7B-v0.1 license:apache-2.0 region:us
---
license: apache-2.0
base_model: mistralai/Mistral-7B-v0.1
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: Mistral-7B-v0.1_cola_silu
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-v0.1_cola_silu

This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3753
- Accuracy: 0.8575

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 2
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 2
- total_train_batch_size: 256
- total_eval_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 750

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.8023        | 0.33  | 10   | 1.7689          | 0.5465   |
| 1.3283        | 0.66  | 20   | 1.4353          | 0.5954   |
| 1.1049        | 0.98  | 30   | 1.1974          | 0.6328   |
| 0.8542        | 1.31  | 40   | 0.9991          | 0.6337   |
| 0.7607        | 1.64  | 50   | 0.7994          | 0.6711   |
| 0.661         | 1.97  | 60   | 0.6935          | 0.7354   |
| 0.5306        | 2.3   | 70   | 0.5793          | 0.7661   |
| 0.6456        | 2.62  | 80   | 0.5240          | 0.7900   |
| 0.5685        | 2.95  | 90   | 0.4889          | 0.7996   |
| 0.4556        | 3.28  | 100  | 0.4674          | 0.7967   |
| 0.6054        | 3.61  | 110  | 0.4582          | 0.8102   |
| 0.3784        | 3.93  | 120  | 0.4379          | 0.8121   |
| 0.3644        | 4.26  | 130  | 0.4198          | 0.8130   |
| 0.4346        | 4.59  | 140  | 0.4197          | 0.8178   |
| 0.4284        | 4.92  | 150  | 0.4075          | 0.8255   |
| 0.3597        | 5.25  | 160  | 0.3816          | 0.8380   |
| 0.3898        | 5.57  | 170  | 0.3905          | 0.8293   |
| 0.3815        | 5.9   | 180  | 0.3902          | 0.8351   |
| 0.307         | 6.23  | 190  | 0.3756          | 0.8418   |
| 0.3967        | 6.56  | 200  | 0.3706          | 0.8418   |
| 0.3673        | 6.89  | 210  | 0.3767          | 0.8428   |
| 0.3036        | 7.21  | 220  | 0.4043          | 0.8236   |
| 0.2788        | 7.54  | 230  | 0.3686          | 0.8447   |
| 0.3103        | 7.87  | 240  | 0.3909          | 0.8245   |
| 0.3109        | 8.2   | 250  | 0.3682          | 0.8408   |
| 0.3152        | 8.52  | 260  | 0.3627          | 0.8408   |
| 0.3514        | 8.85  | 270  | 0.3599          | 0.8456   |
| 0.2984        | 9.18  | 280  | 0.4051          | 0.8332   |
| 0.3046        | 9.51  | 290  | 0.3775          | 0.8370   |
| 0.2388        | 9.84  | 300  | 0.3745          | 0.8360   |
| 0.2641        | 10.16 | 310  | 0.3593          | 0.8408   |
| 0.2567        | 10.49 | 320  | 0.3537          | 0.8485   |
| 0.2414        | 10.82 | 330  | 0.3655          | 0.8466   |
| 0.2385        | 11.15 | 340  | 0.3656          | 0.8476   |
| 0.3334        | 11.48 | 350  | 0.3617          | 0.8504   |
| 0.2424        | 11.8  | 360  | 0.3520          | 0.8485   |
| 0.3105        | 12.13 | 370  | 0.3608          | 0.8447   |
| 0.2301        | 12.46 | 380  | 0.3510          | 0.8552   |
| 0.2869        | 12.79 | 390  | 0.3541          | 0.8447   |
| 0.2197        | 13.11 | 400  | 0.3518          | 0.8514   |
| 0.2533        | 13.44 | 410  | 0.3588          | 0.8466   |
| 0.2426        | 13.77 | 420  | 0.3545          | 0.8581   |
| 0.2097        | 14.1  | 430  | 0.3649          | 0.8456   |
| 0.1854        | 14.43 | 440  | 0.3858          | 0.8504   |
| 0.1908        | 14.75 | 450  | 0.3636          | 0.8466   |
| 0.1651        | 15.08 | 460  | 0.3692          | 0.8504   |
| 0.1954        | 15.41 | 470  | 0.3866          | 0.8514   |
| 0.1904        | 15.74 | 480  | 0.3763          | 0.8514   |
| 0.1437        | 16.07 | 490  | 0.3838          | 0.8514   |
| 0.1535        | 16.39 | 500  | 0.3889          | 0.8485   |
| 0.1086        | 16.72 | 510  | 0.3960          | 0.8514   |
| 0.1327        | 17.05 | 520  | 0.4210          | 0.8447   |
| 0.1104        | 17.38 | 530  | 0.4173          | 0.8408   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.1+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0
