
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# repro_jeopardy

This model is a fine-tuned version of [Qwen/Qwen-14B](https://huggingface.co/Qwen/Qwen-14B) on the jeopardy dataset.
It achieves the following results on the evaluation set:
- Loss: 2.3458

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.01
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.7911        | 0.02  | 250   | 2.5658          |
| 2.529         | 0.04  | 500   | 2.5132          |
| 2.4848        | 0.06  | 750   | 2.4896          |
| 2.4718        | 0.08  | 1000  | 2.4665          |
| 2.4424        | 0.1   | 1250  | 2.4525          |
| 2.4381        | 0.12  | 1500  | 2.4451          |
| 2.4259        | 0.14  | 1750  | 2.4348          |
| 2.4164        | 0.16  | 2000  | 2.4301          |
| 2.4233        | 0.18  | 2250  | 2.4248          |
| 2.4008        | 0.2   | 2500  | 2.4196          |
| 2.4012        | 0.22  | 2750  | 2.4147          |
| 2.394         | 0.24  | 3000  | 2.4071          |
| 2.3751        | 0.26  | 3250  | 2.4086          |
| 2.3871        | 0.28  | 3500  | 2.3987          |
| 2.3804        | 0.3   | 3750  | 2.3945          |
| 2.4           | 0.32  | 4000  | 2.3899          |
| 2.3849        | 0.34  | 4250  | 2.3882          |
| 2.3721        | 0.36  | 4500  | 2.3856          |
| 2.363         | 0.38  | 4750  | 2.3812          |
| 2.3586        | 0.4   | 5000  | 2.3767          |
| 2.3665        | 0.42  | 5250  | 2.3734          |
| 2.3557        | 0.44  | 5500  | 2.3730          |
| 2.3483        | 0.46  | 5750  | 2.3683          |
| 2.3656        | 0.48  | 6000  | 2.3670          |
| 2.3574        | 0.5   | 6250  | 2.3668          |
| 2.352         | 0.52  | 6500  | 2.3647          |
| 2.3384        | 0.54  | 6750  | 2.3630          |
| 2.3437        | 0.56  | 7000  | 2.3599          |
| 2.3571        | 0.58  | 7250  | 2.3588          |
| 2.3377        | 0.6   | 7500  | 2.3575          |
| 2.3534        | 0.62  | 7750  | 2.3560          |
| 2.3461        | 0.64  | 8000  | 2.3556          |
| 2.3382        | 0.66  | 8250  | 2.3535          |
| 2.3288        | 0.68  | 8500  | 2.3521          |
| 2.3196        | 0.7   | 8750  | 2.3522          |
| 2.347         | 0.72  | 9000  | 2.3511          |
| 2.3167        | 0.74  | 9250  | 2.3506          |
| 2.3405        | 0.76  | 9500  | 2.3492          |
| 2.3087        | 0.78  | 9750  | 2.3488          |
| 2.3212        | 0.8   | 10000 | 2.3486          |
| 2.3352        | 0.82  | 10250 | 2.3471          |
| 2.3194        | 0.84  | 10500 | 2.3468          |
| 2.3417        | 0.86  | 10750 | 2.3463          |
| 2.3465        | 0.88  | 11000 | 2.3462          |
| 2.3412        | 0.9   | 11250 | 2.3456          |
| 2.3209        | 0.92  | 11500 | 2.3458          |
| 2.3273        | 0.94  | 11750 | 2.3459          |
| 2.3254        | 0.96  | 12000 | 2.3458          |
| 2.3282        | 0.98  | 12250 | 2.3458          |
| 2.3342        | 1.0   | 12500 | 2.3458          |


### Framework versions

- Transformers 4.32.0
- Pytorch 2.1.0
- Datasets 2.15.0
- Tokenizers 0.13.3

{"tags": ["generated_from_trainer"], "datasets": ["jeopardy"], "base_model": "Qwen/Qwen-14B", "model-index": [{"name": "repro_jeopardy", "results": []}]}
generated_from_trainer dataset:jeopardy base_model:Qwen/Qwen-14B region:us