transformers safetensors llama text-generation trl sft generated_from_trainer base_model:deepseek-ai/deepseek-math-7b-base license:other autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: other
base_model: deepseek-ai/deepseek-math-7b-base
tags:
- trl
- sft
- generated_from_trainer
model-index:
- name: output-itemHalfTax-wd0.1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# output-itemHalfTax-wd0.1

This model is a fine-tuned version of [deepseek-ai/deepseek-math-7b-base](https://huggingface.co/deepseek-ai/deepseek-math-7b-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0475

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant_with_warmup
- lr_scheduler_warmup_steps: 1
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.8425        | 0.01  | 1    | 0.4687          |
| 0.8496        | 0.01  | 2    | 0.4640          |
| 0.819         | 0.02  | 3    | 0.4579          |
| 0.8173        | 0.03  | 4    | 0.4532          |
| 0.8297        | 0.03  | 5    | 0.4495          |
| 0.8014        | 0.04  | 6    | 0.4447          |
| 0.7981        | 0.04  | 7    | 0.4399          |
| 0.7716        | 0.05  | 8    | 0.4375          |
| 0.6996        | 0.06  | 9    | 0.4332          |
| 0.6631        | 0.06  | 10   | 0.4302          |
| 0.6596        | 0.07  | 11   | 0.4247          |
| 0.6887        | 0.08  | 12   | 0.4221          |
| 0.6373        | 0.08  | 13   | 0.4180          |
| 0.6536        | 0.09  | 14   | 0.4161          |
| 0.6112        | 0.1   | 15   | 0.4102          |
| 0.6113        | 0.1   | 16   | 0.4072          |
| 0.5427        | 0.11  | 17   | 0.4041          |
| 0.5434        | 0.12  | 18   | 0.4005          |
| 0.5098        | 0.12  | 19   | 0.3955          |
| 0.5567        | 0.13  | 20   | 0.3914          |
| 0.4984        | 0.13  | 21   | 0.3889          |
| 0.5414        | 0.14  | 22   | 0.3852          |
| 0.4851        | 0.15  | 23   | 0.3818          |
| 0.4599        | 0.15  | 24   | 0.3769          |
| 0.4423        | 0.16  | 25   | 0.3736          |
| 0.4562        | 0.17  | 26   | 0.3711          |
| 0.4359        | 0.17  | 27   | 0.3677          |
| 0.4182        | 0.18  | 28   | 0.3631          |
| 0.4302        | 0.19  | 29   | 0.3612          |
| 0.4294        | 0.19  | 30   | 0.3556          |
| 0.4169        | 0.2   | 31   | 0.3527          |
| 0.3764        | 0.2   | 32   | 0.3483          |
| 0.3773        | 0.21  | 33   | 0.3451          |
| 0.3611        | 0.22  | 34   | 0.3423          |
| 0.3524        | 0.22  | 35   | 0.3380          |
| 0.3528        | 0.23  | 36   | 0.3337          |
| 0.3666        | 0.24  | 37   | 0.3306          |
| 0.3536        | 0.24  | 38   | 0.3266          |
| 0.3812        | 0.25  | 39   | 0.3223          |
| 0.2976        | 0.26  | 40   | 0.3191          |
| 0.2805        | 0.26  | 41   | 0.3157          |
| 0.2739        | 0.27  | 42   | 0.3115          |
| 0.2994        | 0.28  | 43   | 0.3089          |
| 0.2864        | 0.28  | 44   | 0.3031          |
| 0.2745        | 0.29  | 45   | 0.3003          |
| 0.2666        | 0.29  | 46   | 0.2968          |
| 0.3109        | 0.3   | 47   | 0.2914          |
| 0.2603        | 0.31  | 48   | 0.2873          |
| 0.2761        | 0.31  | 49   | 0.2821          |
| 0.2566        | 0.32  | 50   | 0.2792          |
| 0.2637        | 0.33  | 51   | 0.2758          |
| 0.2388        | 0.33  | 52   | 0.2706          |
| 0.2474        | 0.34  | 53   | 0.2651          |
| 0.2462        | 0.35  | 54   | 0.2607          |
| 0.2617        | 0.35  | 55   | 0.2567          |
| 0.2387        | 0.36  | 56   | 0.2513          |
| 0.2171        | 0.36  | 57   | 0.2482          |
| 0.2324        | 0.37  | 58   | 0.2428          |
| 0.227         | 0.38  | 59   | 0.2384          |
| 0.2366        | 0.38  | 60   | 0.2332          |
| 0.2174        | 0.39  | 61   | 0.2288          |
| 0.2083        | 0.4   | 62   | 0.2237          |
| 0.1935        | 0.4   | 63   | 0.2198          |
| 0.2141        | 0.41  | 64   | 0.2152          |
| 0.2044        | 0.42  | 65   | 0.2111          |
| 0.2061        | 0.42  | 66   | 0.2064          |
| 0.2005        | 0.43  | 67   | 0.2011          |
| 0.1895        | 0.44  | 68   | 0.1944          |
| 0.1823        | 0.44  | 69   | 0.1902          |
| 0.1763        | 0.45  | 70   | 0.1874          |
| 0.1897        | 0.45  | 71   | 0.1814          |
| 0.1853        | 0.46  | 72   | 0.1764          |
| 0.1674        | 0.47  | 73   | 0.1706          |
| 0.1642        | 0.47  | 74   | 0.1648          |
| 0.1677        | 0.48  | 75   | 0.1604          |
| 0.1551        | 0.49  | 76   | 0.1544          |
| 0.1457        | 0.49  | 77   | 0.1486          |
| 0.1497        | 0.5   | 78   | 0.1434          |
| 0.1389        | 0.51  | 79   | 0.1374          |
| 0.1396        | 0.51  | 80   | 0.1325          |
| 0.1297        | 0.52  | 81   | 0.1266          |
| 0.1298        | 0.52  | 82   | 0.1211          |
| 0.1162        | 0.53  | 83   | 0.1164          |
| 0.13          | 0.54  | 84   | 0.1127          |
| 0.1207        | 0.54  | 85   | 0.1085          |
| 0.1165        | 0.55  | 86   | 0.1039          |
| 0.1088        | 0.56  | 87   | 0.1012          |
| 0.1082        | 0.56  | 88   | 0.0969          |
| 0.109         | 0.57  | 89   | 0.0952          |
| 0.1102        | 0.58  | 90   | 0.0924          |
| 0.1026        | 0.58  | 91   | 0.0891          |
| 0.1058        | 0.59  | 92   | 0.0875          |
| 0.0973        | 0.6   | 93   | 0.0861          |
| 0.0963        | 0.6   | 94   | 0.0859          |
| 0.0941        | 0.61  | 95   | 0.0838          |
| 0.0983        | 0.61  | 96   | 0.0824          |
| 0.0884        | 0.62  | 97   | 0.0812          |
| 0.0855        | 0.63  | 98   | 0.0798          |
| 0.0875        | 0.63  | 99   | 0.0781          |
| 0.0833        | 0.64  | 100  | 0.0766          |
| 0.0836        | 0.65  | 101  | 0.0757          |
| 0.0809        | 0.65  | 102  | 0.0745          |
| 0.0839        | 0.66  | 103  | 0.0731          |
| 0.0748        | 0.67  | 104  | 0.0717          |
| 0.0779        | 0.67  | 105  | 0.0710          |
| 0.0768        | 0.68  | 106  | 0.0705          |
| 0.0794        | 0.68  | 107  | 0.0693          |
| 0.079         | 0.69  | 108  | 0.0679          |
| 0.0808        | 0.7   | 109  | 0.0667          |
| 0.0785        | 0.7   | 110  | 0.0658          |
| 0.0669        | 0.71  | 111  | 0.0649          |
| 0.0715        | 0.72  | 112  | 0.0640          |
| 0.0751        | 0.72  | 113  | 0.0632          |
| 0.0727        | 0.73  | 114  | 0.0626          |
| 0.0725        | 0.74  | 115  | 0.0625          |
| 0.0665        | 0.74  | 116  | 0.0614          |
| 0.0627        | 0.75  | 117  | 0.0605          |
| 0.0681        | 0.76  | 118  | 0.0597          |
| 0.0673        | 0.76  | 119  | 0.0593          |
| 0.0741        | 0.77  | 120  | 0.0592          |
| 0.0686        | 0.77  | 121  | 0.0584          |
| 0.0618        | 0.78  | 122  | 0.0584          |
| 0.065         | 0.79  | 123  | 0.0574          |
| 0.061         | 0.79  | 124  | 0.0572          |
| 0.0685        | 0.8   | 125  | 0.0571          |
| 0.0621        | 0.81  | 126  | 0.0560          |
| 0.0636        | 0.81  | 127  | 0.0558          |
| 0.0581        | 0.82  | 128  | 0.0550          |
| 0.0576        | 0.83  | 129  | 0.0547          |
| 0.0628        | 0.83  | 130  | 0.0544          |
| 0.056         | 0.84  | 131  | 0.0542          |
| 0.0572        | 0.84  | 132  | 0.0536          |
| 0.0605        | 0.85  | 133  | 0.0529          |
| 0.0626        | 0.86  | 134  | 0.0520          |
| 0.0566        | 0.86  | 135  | 0.0517          |
| 0.0575        | 0.87  | 136  | 0.0519          |
| 0.0571        | 0.88  | 137  | 0.0514          |
| 0.0594        | 0.88  | 138  | 0.0510          |
| 0.0528        | 0.89  | 139  | 0.0513          |
| 0.0507        | 0.9   | 140  | 0.0508          |
| 0.0587        | 0.9   | 141  | 0.0503          |
| 0.0558        | 0.91  | 142  | 0.0504          |
| 0.0538        | 0.92  | 143  | 0.0500          |
| 0.0509        | 0.92  | 144  | 0.0509          |
| 0.0538        | 0.93  | 145  | 0.0504          |
| 0.0524        | 0.93  | 146  | 0.0498          |
| 0.059         | 0.94  | 147  | 0.0496          |
| 0.0508        | 0.95  | 148  | 0.0494          |
| 0.0563        | 0.95  | 149  | 0.0491          |
| 0.0472        | 0.96  | 150  | 0.0484          |
| 0.0526        | 0.97  | 151  | 0.0482          |
| 0.0525        | 0.97  | 152  | 0.0482          |
| 0.0483        | 0.98  | 153  | 0.0478          |
| 0.0541        | 0.99  | 154  | 0.0483          |
| 0.0521        | 0.99  | 155  | 0.0474          |
| 0.0556        | 1.0   | 156  | 0.0475          |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.1.0a0+32f93b1
- Datasets 2.17.1
- Tokenizers 0.15.2
