
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
# GMarcO-ft

This model was finetuned from [Mediform/GMarcO](https://huggingface.co/Mediform/GMarcO) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0123

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 50
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 1.1974        | 0.05  | 4    | 1.1860          |
| 1.1084        | 0.09  | 8    | 0.8903          |
| 0.7682        | 0.14  | 12   | 0.5366          |
| 0.3806        | 0.19  | 16   | 0.3049          |
| 0.2491        | 0.24  | 20   | 0.1645          |
| 0.0959        | 0.28  | 24   | 0.1158          |
| 0.0709        | 0.33  | 28   | 0.0862          |
| 0.1118        | 0.38  | 32   | 0.0696          |
| 0.0494        | 0.42  | 36   | 0.0574          |
| 0.0843        | 0.47  | 40   | 0.0482          |
| 0.0661        | 0.52  | 44   | 0.0410          |
| 0.1191        | 0.56  | 48   | 0.0391          |
| 0.0312        | 0.61  | 52   | 0.0371          |
| 0.0422        | 0.66  | 56   | 0.0331          |
| 0.0365        | 0.71  | 60   | 0.0340          |
| 0.0403        | 0.75  | 64   | 0.0300          |
| 0.0303        | 0.8   | 68   | 0.0288          |
| 0.0313        | 0.85  | 72   | 0.0280          |
| 0.0592        | 0.89  | 76   | 0.0277          |
| 0.0218        | 0.94  | 80   | 0.0259          |
| 0.0267        | 0.99  | 84   | 0.0253          |
| 0.0395        | 1.04  | 88   | 0.0242          |
| 0.021         | 1.08  | 92   | 0.0247          |
| 0.0173        | 1.13  | 96   | 0.0245          |
| 0.0204        | 1.18  | 100  | 0.0236          |
| 0.0332        | 1.22  | 104  | 0.0230          |
| 0.0184        | 1.27  | 108  | 0.0221          |
| 0.014         | 1.32  | 112  | 0.0218          |
| 0.0259        | 1.36  | 116  | 0.0212          |
| 0.0107        | 1.41  | 120  | 0.0212          |
| 0.0212        | 1.46  | 124  | 0.0220          |
| 0.0087        | 1.51  | 128  | 0.0211          |
| 0.0118        | 1.55  | 132  | 0.0205          |
| 0.0175        | 1.6   | 136  | 0.0209          |
| 0.0246        | 1.65  | 140  | 0.0211          |
| 0.016         | 1.69  | 144  | 0.0218          |
| 0.0145        | 1.74  | 148  | 0.0187          |
| 0.022         | 1.79  | 152  | 0.0183          |
| 0.0143        | 1.84  | 156  | 0.0173          |
| 0.0062        | 1.88  | 160  | 0.0162          |
| 0.0258        | 1.93  | 164  | 0.0155          |
| 0.0234        | 1.98  | 168  | 0.0153          |
| 0.0084        | 2.02  | 172  | 0.0155          |
| 0.0084        | 2.07  | 176  | 0.0161          |
| 0.0163        | 2.12  | 180  | 0.0155          |
| 0.0078        | 2.16  | 184  | 0.0152          |
| 0.0042        | 2.21  | 188  | 0.0153          |
| 0.0067        | 2.26  | 192  | 0.0157          |
| 0.014         | 2.31  | 196  | 0.0168          |
| 0.0045        | 2.35  | 200  | 0.0153          |
| 0.0221        | 2.4   | 204  | 0.0150          |
| 0.0172        | 2.45  | 208  | 0.0147          |
| 0.0147        | 2.49  | 212  | 0.0147          |
| 0.0175        | 2.54  | 216  | 0.0149          |
| 0.0165        | 2.59  | 220  | 0.0152          |
| 0.0079        | 2.64  | 224  | 0.0156          |
| 0.0127        | 2.68  | 228  | 0.0157          |
| 0.003         | 2.73  | 232  | 0.0154          |
| 0.0085        | 2.78  | 236  | 0.0154          |
| 0.0129        | 2.82  | 240  | 0.0156          |
| 0.0096        | 2.87  | 244  | 0.0155          |
| 0.0164        | 2.92  | 248  | 0.0149          |
| 0.0168        | 2.96  | 252  | 0.0142          |
| 0.0107        | 3.01  | 256  | 0.0139          |
| 0.0058        | 3.06  | 260  | 0.0138          |
| 0.0124        | 3.11  | 264  | 0.0134          |
| 0.0059        | 3.15  | 268  | 0.0131          |
| 0.0079        | 3.2   | 272  | 0.0129          |
| 0.0048        | 3.25  | 276  | 0.0127          |
| 0.0056        | 3.29  | 280  | 0.0128          |
| 0.0045        | 3.34  | 284  | 0.0130          |
| 0.0159        | 3.39  | 288  | 0.0133          |
| 0.005         | 3.44  | 292  | 0.0133          |
| 0.0043        | 3.48  | 296  | 0.0132          |
| 0.0054        | 3.53  | 300  | 0.0134          |
| 0.0028        | 3.58  | 304  | 0.0133          |
| 0.0051        | 3.62  | 308  | 0.0131          |
| 0.0048        | 3.67  | 312  | 0.0129          |
| 0.0034        | 3.72  | 316  | 0.0128          |
| 0.0265        | 3.76  | 320  | 0.0126          |
| 0.0056        | 3.81  | 324  | 0.0124          |
| 0.0052        | 3.86  | 328  | 0.0122          |
| 0.0027        | 3.91  | 332  | 0.0122          |
| 0.002         | 3.95  | 336  | 0.0123          |
| 0.0064        | 4.0   | 340  | 0.0124          |
| 0.0071        | 4.05  | 344  | 0.0124          |
| 0.004         | 4.09  | 348  | 0.0123          |
| 0.0054        | 4.14  | 352  | 0.0121          |
| 0.0044        | 4.19  | 356  | 0.0121          |
| 0.0019        | 4.24  | 360  | 0.0121          |
| 0.0012        | 4.28  | 364  | 0.0123          |
| 0.0051        | 4.33  | 368  | 0.0122          |
| 0.0045        | 4.38  | 372  | 0.0123          |
| 0.0081        | 4.42  | 376  | 0.0123          |
| 0.0057        | 4.47  | 380  | 0.0123          |
| 0.0026        | 4.52  | 384  | 0.0122          |
| 0.0076        | 4.56  | 388  | 0.0122          |
| 0.0042        | 4.61  | 392  | 0.0122          |
| 0.0056        | 4.66  | 396  | 0.0122          |
| 0.0038        | 4.71  | 400  | 0.0122          |
| 0.0101        | 4.75  | 404  | 0.0122          |
| 0.0052        | 4.8   | 408  | 0.0122          |
| 0.0063        | 4.85  | 412  | 0.0122          |
| 0.0065        | 4.89  | 416  | 0.0122          |
| 0.0094        | 4.94  | 420  | 0.0122          |
| 0.0053        | 4.99  | 424  | 0.0123          |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.0.1+cu117
- Datasets 2.14.6
- Tokenizers 0.14.1

{"tags": ["generated_from_trainer"], "model-index": [{"name": "GMarcO-ft", "results": []}]}
gguf generated_from_trainer region:us