peft safetensors generated_from_trainer base_model:AIDC-ai-business/Marcoroni-7B-v3 license:apache-2.0 region:us
---
license: apache-2.0
library_name: peft
tags:
- generated_from_trainer
base_model: AIDC-ai-business/Marcoroni-7B-v3
model-index:
- name: 20231216-045106
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 20231216-045106

This model is a fine-tuned version of [AIDC-ai-business/Marcoroni-7B-v3](https://huggingface.co/AIDC-ai-business/Marcoroni-7B-v3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2843

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2.5e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- training_steps: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.2541        | 0.02  | 1    | 2.1106          |
| 1.5517        | 0.03  | 2    | 2.1106          |
| 2.347         | 0.05  | 3    | 2.0350          |
| 2.006         | 0.06  | 4    | 1.9297          |
| 1.8398        | 0.08  | 5    | 1.8406          |
| 1.8506        | 0.09  | 6    | 1.8406          |
| 1.8804        | 0.11  | 7    | 1.7621          |
| 1.9034        | 0.12  | 8    | 1.6988          |
| 0.9835        | 0.14  | 9    | 1.6494          |
| 1.4725        | 0.15  | 10   | 1.5915          |
| 1.1824        | 0.17  | 11   | 1.5557          |
| 1.3356        | 0.18  | 12   | 1.5557          |
| 1.1077        | 0.2   | 13   | 1.5242          |
| 1.4959        | 0.22  | 14   | 1.4908          |
| 1.4881        | 0.23  | 15   | 1.4550          |
| 1.2674        | 0.25  | 16   | 1.4201          |
| 1.2133        | 0.26  | 17   | 1.3872          |
| 1.5423        | 0.28  | 18   | 1.3521          |
| 0.9551        | 0.29  | 19   | 1.3212          |
| 1.1868        | 0.31  | 20   | 1.2881          |
| 1.4799        | 0.32  | 21   | 1.2526          |
| 1.2903        | 0.34  | 22   | 1.2154          |
| 1.3357        | 0.35  | 23   | 1.1770          |
| 0.868         | 0.37  | 24   | 1.1383          |
| 1.1978        | 0.38  | 25   | 1.0982          |
| 1.2102        | 0.4   | 26   | 1.0575          |
| 0.8946        | 0.42  | 27   | 1.0575          |
| 1.0671        | 0.43  | 28   | 1.0151          |
| 1.0761        | 0.45  | 29   | 0.9715          |
| 0.9568        | 0.46  | 30   | 0.9263          |
| 0.8203        | 0.48  | 31   | 0.8848          |
| 0.8601        | 0.49  | 32   | 0.8408          |
| 0.8253        | 0.51  | 33   | 0.7945          |
| 0.8098        | 0.52  | 34   | 0.7504          |
| 0.6567        | 0.54  | 35   | 0.7110          |
| 0.6545        | 0.55  | 36   | 0.6695          |
| 0.6036        | 0.57  | 37   | 0.6338          |
| 0.6746        | 0.58  | 38   | 0.6018          |
| 0.5376        | 0.6   | 39   | 0.5731          |
| 0.5674        | 0.62  | 40   | 0.5471          |
| 0.4762        | 0.63  | 41   | 0.5218          |
| 0.4755        | 0.65  | 42   | 0.4965          |
| 0.3974        | 0.66  | 43   | 0.4756          |
| 0.8437        | 0.68  | 44   | 0.4607          |
| 0.4766        | 0.69  | 45   | 0.4485          |
| 0.5403        | 0.71  | 46   | 0.4369          |
| 0.4201        | 0.72  | 47   | 0.4244          |
| 0.4326        | 0.74  | 48   | 0.4115          |
| 0.4669        | 0.75  | 49   | 0.3978          |
| 0.5259        | 0.77  | 50   | 0.3867          |
| 0.3906        | 0.78  | 51   | 0.3782          |
| 0.369         | 0.8   | 52   | 0.3724          |
| 0.3314        | 0.82  | 53   | 0.3658          |
| 0.4164        | 0.83  | 54   | 0.3585          |
| 0.3339        | 0.85  | 55   | 0.3499          |
| 0.315         | 0.86  | 56   | 0.3405          |
| 0.5404        | 0.88  | 57   | 0.3340          |
| 0.3136        | 0.89  | 58   | 0.3302          |
| 0.3574        | 0.91  | 59   | 0.3275          |
| 0.3466        | 0.92  | 60   | 0.3249          |
| 0.3097        | 0.94  | 61   | 0.3226          |
| 0.4212        | 0.95  | 62   | 0.3208          |
| 0.3183        | 0.97  | 63   | 0.3184          |
| 0.2762        | 0.98  | 64   | 0.3162          |
| 0.3936        | 1.0   | 65   | 0.3143          |
| 0.303         | 1.02  | 66   | 0.3128          |
| 0.3813        | 1.03  | 67   | 0.3108          |
| 0.3065        | 1.05  | 68   | 0.3089          |
| 0.2457        | 1.06  | 69   | 0.3069          |
| 0.4008        | 1.08  | 70   | 0.3053          |
| 0.3131        | 1.09  | 71   | 0.3040          |
| 0.2483        | 1.11  | 72   | 0.3027          |
| 0.2445        | 1.12  | 73   | 0.3016          |
| 0.2717        | 1.14  | 74   | 0.3003          |
| 0.6308        | 1.15  | 75   | 0.2991          |
| 0.3271        | 1.17  | 76   | 0.2980          |
| 0.3013        | 1.18  | 77   | 0.2971          |
| 0.2951        | 1.2   | 78   | 0.2960          |
| 0.2514        | 1.22  | 79   | 0.2950          |
| 0.2836        | 1.23  | 80   | 0.2940          |
| 0.2538        | 1.25  | 81   | 0.2929          |
| 0.312         | 1.26  | 82   | 0.2921          |
| 0.2773        | 1.28  | 83   | 0.2912          |
| 0.2957        | 1.29  | 84   | 0.2904          |
| 0.3997        | 1.31  | 85   | 0.2896          |
| 0.3084        | 1.32  | 86   | 0.2889          |
| 0.2521        | 1.34  | 87   | 0.2882          |
| 0.2873        | 1.35  | 88   | 0.2877          |
| 0.2419        | 1.37  | 89   | 0.2872          |
| 0.2691        | 1.38  | 90   | 0.2867          |
| 0.6368        | 1.4   | 91   | 0.2864          |
| 0.2299        | 1.42  | 92   | 0.2860          |
| 0.4967        | 1.43  | 93   | 0.2857          |
| 0.2827        | 1.45  | 94   | 0.2855          |
| 0.2501        | 1.46  | 95   | 0.2853          |
| 0.2285        | 1.48  | 96   | 0.2851          |
| 0.269         | 1.49  | 97   | 0.2849          |
| 0.2539        | 1.51  | 98   | 0.2847          |
| 0.2798        | 1.52  | 99   | 0.2845          |
| 0.2695        | 1.54  | 100  | 0.2843          |


### Framework versions

- PEFT 0.7.2.dev0
- Transformers 4.37.0.dev0
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0