
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# synpre_extract_q10_a5_1M_q_middle_t5-small

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the tyzhu/synpre_extract_q10_a5_1M_q_middle dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0303
- Bleu: 99.0663
- Gen Len: 6.2413

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant_with_warmup
- lr_scheduler_warmup_steps: 10000
- training_steps: 200000

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|
| 7.3474        | 0.66  | 5000   | 7.1194          | 1.8181  | 8.2388  |
| 6.7358        | 1.31  | 10000  | 6.6147          | 2.5865  | 6.9488  |
| 6.4733        | 1.97  | 15000  | 6.2818          | 2.6374  | 6.9203  |
| 6.2082        | 2.62  | 20000  | 6.1461          | 2.6078  | 7.076   |
| 6.0355        | 3.28  | 25000  | 6.0499          | 2.7372  | 6.8757  |
| 6.0461        | 3.93  | 30000  | 6.0051          | 2.7729  | 6.8961  |
| 5.8582        | 4.59  | 35000  | 5.9518          | 2.9219  | 6.8637  |
| 5.7091        | 5.24  | 40000  | 5.9180          | 2.9882  | 6.8949  |
| 5.7644        | 5.9   | 45000  | 5.8884          | 3.0533  | 6.9172  |
| 5.5539        | 6.55  | 50000  | 5.8648          | 3.1978  | 6.8483  |
| 5.4095        | 7.21  | 55000  | 5.8422          | 3.2665  | 6.8366  |
| 5.4713        | 7.87  | 60000  | 5.7989          | 3.4296  | 6.8062  |
| 5.3088        | 8.52  | 65000  | 5.7341          | 3.2485  | 6.9185  |
| 5.1461        | 9.18  | 70000  | 5.6393          | 3.4836  | 6.8063  |
| 5.1468        | 9.83  | 75000  | 5.4932          | 3.4589  | 6.8234  |
| 4.756         | 10.49 | 80000  | 5.0665          | 3.4641  | 6.9106  |
| 4.0087        | 11.14 | 85000  | 4.1256          | 4.0029  | 6.7468  |
| 3.7821        | 11.8  | 90000  | 3.7954          | 4.6263  | 6.6409  |
| 3.4823        | 12.45 | 95000  | 3.5778          | 4.9723  | 6.6464  |
| 0.8927        | 13.11 | 100000 | 0.7053          | 72.0211 | 6.1621  |
| 0.3604        | 13.77 | 105000 | 0.2818          | 94.0056 | 6.1649  |
| 0.2185        | 14.42 | 110000 | 0.1744          | 96.4397 | 6.2273  |
| 0.1241        | 15.08 | 115000 | 0.0979          | 98.1597 | 6.2426  |
| 0.0789        | 15.73 | 120000 | 0.0436          | 98.8179 | 6.2435  |
| 0.0465        | 16.39 | 125000 | 0.0303          | 99.0663 | 6.2413  |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/synpre_extract_q10_a5_1M_q_middle"], "metrics": ["bleu"], "base_model": "t5-small", "model-index": [{"name": "synpre_extract_q10_a5_1M_q_middle_t5-small", "results": [{"task": {"type": "translation", "name": "Translation"}, "dataset": {"name": "tyzhu/synpre_extract_q10_a5_1M_q_middle", "type": "tyzhu/synpre_extract_q10_a5_1M_q_middle"}, "metrics": [{"type": "bleu", "value": 99.0663, "name": "Bleu"}]}]}]}
transformers pytorch t5 text2text-generation generated_from_trainer dataset:tyzhu/synpre_extract_q10_a5_1M_q_middle base_model:t5-small license:apache-2.0 model-index autotrain_compatible endpoints_compatible text-generation-inference region:us