transformers safetensors t5 text2text-generation generated_from_trainer base_model:google/flan-t5-base license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: google/flan-t5-base
tags:
- generated_from_trainer
model-index:
- name: flan-t5-base-sheldon-chat-v2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan-t5-base-sheldon-chat-v2

This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2012
- Rouge 1: 12.02
- Rouge 2: 2.8059
- Rouge L: 11.2005
- Avg Len: 14.229
- Bertscore Prec: 0.8673
- Bertscore Rec: 0.8568
- Bertscore F1: 0.8616

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge 1 | Rouge 2 | Rouge L | Avg Len | Bertscore Prec | Bertscore Rec | Bertscore F1 |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:-------:|:--------------:|:-------------:|:------------:|
| 3.5731        | 0.07  | 200   | 3.2556          | 9.2886  | 1.7826  | 8.8558  | 10.8814 | 0.8661         | 0.8494        | 0.8573       |
| 3.421         | 0.13  | 400   | 3.1758          | 9.2559  | 1.9064  | 8.8243  | 10.7076 | 0.8714         | 0.8538        | 0.8622       |
| 3.3457        | 0.2   | 600   | 3.1197          | 9.0991  | 2.1854  | 8.7076  | 10.1063 | 0.8741         | 0.8546        | 0.8639       |
| 3.3013        | 0.26  | 800   | 3.0755          | 8.5523  | 1.8896  | 8.2224  | 12.3681 | 0.8706         | 0.8537        | 0.8617       |
| 3.2509        | 0.33  | 1000  | 3.0383          | 9.2396  | 2.0409  | 8.8314  | 12.0511 | 0.8729         | 0.8552        | 0.8636       |
| 3.2376        | 0.4   | 1200  | 3.0005          | 8.7356  | 1.8181  | 8.3039  | 12.9796 | 0.8702         | 0.8541        | 0.8617       |
| 3.2075        | 0.46  | 1400  | 2.9602          | 9.4303  | 2.0003  | 8.7571  | 16.0736 | 0.8633         | 0.8548        | 0.8587       |
| 3.1342        | 0.53  | 1600  | 2.9263          | 10.8148 | 2.2698  | 9.8735  | 15.4826 | 0.866          | 0.8571        | 0.8611       |
| 3.1376        | 0.6   | 1800  | 2.8915          | 9.9381  | 1.8569  | 9.1291  | 15.1718 | 0.8635         | 0.8553        | 0.859        |
| 3.0862        | 0.66  | 2000  | 2.8648          | 9.6534  | 1.7925  | 8.9539  | 14.9939 | 0.8603         | 0.8538        | 0.8567       |
| 3.0633        | 0.73  | 2200  | 2.8372          | 7.4573  | 1.6883  | 7.2688  | 15.411  | 0.8597         | 0.8512        | 0.855        |
| 3.0286        | 0.79  | 2400  | 2.8118          | 9.964   | 2.1269  | 9.281   | 13.5706 | 0.8708         | 0.8556        | 0.8628       |
| 3.0248        | 0.86  | 2600  | 2.7839          | 8.9688  | 1.901   | 8.5439  | 14.5665 | 0.8649         | 0.8527        | 0.8583       |
| 2.9777        | 0.93  | 2800  | 2.7637          | 10.2338 | 2.0737  | 9.67    | 13.3415 | 0.8692         | 0.8555        | 0.8619       |
| 2.9803        | 0.99  | 3000  | 2.7438          | 8.8705  | 1.8115  | 8.3491  | 15.7914 | 0.8644         | 0.8537        | 0.8586       |
| 2.9313        | 1.06  | 3200  | 2.7196          | 8.5001  | 1.8396  | 8.1555  | 14.4008 | 0.8684         | 0.854         | 0.8607       |
| 2.9094        | 1.12  | 3400  | 2.7013          | 9.9201  | 1.8958  | 9.1863  | 16.0777 | 0.8587         | 0.8542        | 0.856        |
| 2.8625        | 1.19  | 3600  | 2.6804          | 9.6675  | 2.0412  | 9.1107  | 13.5235 | 0.8691         | 0.8546        | 0.8614       |
| 2.8871        | 1.26  | 3800  | 2.6595          | 9.9853  | 2.0365  | 9.3471  | 14.9836 | 0.8655         | 0.8546        | 0.8596       |
| 2.8607        | 1.32  | 4000  | 2.6436          | 9.5726  | 1.625   | 8.9096  | 14.4151 | 0.8635         | 0.8541        | 0.8584       |
| 2.8028        | 1.39  | 4200  | 2.6298          | 9.6501  | 1.8484  | 9.1131  | 13.4765 | 0.8636         | 0.8544        | 0.8586       |
| 2.8357        | 1.46  | 4400  | 2.6080          | 8.961   | 1.9923  | 8.5776  | 13.6115 | 0.864          | 0.853         | 0.8581       |
| 2.8085        | 1.52  | 4600  | 2.5954          | 9.2942  | 1.8072  | 8.7018  | 15.9427 | 0.8611         | 0.8529        | 0.8566       |
| 2.7853        | 1.59  | 4800  | 2.5773          | 9.6865  | 2.001   | 9.1058  | 14.6626 | 0.8658         | 0.8548        | 0.8599       |
| 2.7934        | 1.65  | 5000  | 2.5613          | 9.3635  | 2.0014  | 8.8713  | 13.6401 | 0.8699         | 0.8533        | 0.8611       |
| 2.7697        | 1.72  | 5200  | 2.5508          | 8.6314  | 1.7812  | 8.2446  | 13.4417 | 0.8662         | 0.8523        | 0.8588       |
| 2.7451        | 1.79  | 5400  | 2.5361          | 9.7799  | 2.1524  | 9.2527  | 12.3926 | 0.8711         | 0.854         | 0.8621       |
| 2.7692        | 1.85  | 5600  | 2.5220          | 9.7707  | 2.258   | 9.2042  | 14.0102 | 0.8665         | 0.8539        | 0.8598       |
| 2.7435        | 1.92  | 5800  | 2.5110          | 9.6982  | 2.1742  | 9.2157  | 14.773  | 0.8621         | 0.8536        | 0.8574       |
| 2.7033        | 1.98  | 6000  | 2.4948          | 9.7766  | 2.0843  | 9.141   | 13.3926 | 0.8677         | 0.8537        | 0.8602       |
| 2.6664        | 2.05  | 6200  | 2.4792          | 10.4019 | 2.2247  | 9.8155  | 13.5174 | 0.8684         | 0.8547        | 0.861        |
| 2.6542        | 2.12  | 6400  | 2.4708          | 10.2339 | 2.2096  | 9.5914  | 14.7669 | 0.8668         | 0.8548        | 0.8603       |
| 2.6467        | 2.18  | 6600  | 2.4582          | 10.3021 | 2.3528  | 9.7578  | 12.9448 | 0.8723         | 0.8547        | 0.863        |
| 2.6376        | 2.25  | 6800  | 2.4477          | 10.3056 | 2.1854  | 9.8499  | 12.9489 | 0.8712         | 0.855         | 0.8626       |
| 2.6267        | 2.31  | 7000  | 2.4364          | 10.6358 | 2.2434  | 10.002  | 13.4254 | 0.8691         | 0.8559        | 0.8621       |
| 2.6489        | 2.38  | 7200  | 2.4246          | 10.5459 | 2.1935  | 9.8531  | 13.4949 | 0.8682         | 0.8549        | 0.861        |
| 2.6174        | 2.45  | 7400  | 2.4160          | 9.8943  | 2.2126  | 9.2111  | 15.0164 | 0.8649         | 0.854         | 0.859        |
| 2.6094        | 2.51  | 7600  | 2.4052          | 10.8941 | 2.3246  | 10.1182 | 14.5337 | 0.8676         | 0.8561        | 0.8614       |
| 2.5877        | 2.58  | 7800  | 2.3966          | 10.5092 | 2.2398  | 9.7792  | 14.454  | 0.8675         | 0.8553        | 0.861        |
| 2.5525        | 2.65  | 8000  | 2.3845          | 10.0878 | 2.1701  | 9.4132  | 13.9059 | 0.8678         | 0.8547        | 0.8608       |
| 2.5912        | 2.71  | 8200  | 2.3709          | 10.504  | 2.3081  | 9.8336  | 13.7076 | 0.8686         | 0.8558        | 0.8618       |
| 2.5748        | 2.78  | 8400  | 2.3625          | 10.6169 | 2.4216  | 10.0053 | 13.9202 | 0.8671         | 0.8554        | 0.8608       |
| 2.5921        | 2.84  | 8600  | 2.3550          | 10.4233 | 2.3283  | 9.9305  | 14.1963 | 0.8642         | 0.8547        | 0.859        |
| 2.5617        | 2.91  | 8800  | 2.3414          | 10.5877 | 2.3495  | 9.9807  | 13.7894 | 0.8669         | 0.8555        | 0.8608       |
| 2.5632        | 2.98  | 9000  | 2.3338          | 10.8744 | 2.5212  | 10.2281 | 14.1022 | 0.8665         | 0.8554        | 0.8605       |
| 2.5257        | 3.04  | 9200  | 2.3217          | 11.3215 | 2.4995  | 10.589  | 14.9264 | 0.8647         | 0.8562        | 0.8601       |
| 2.4889        | 3.11  | 9400  | 2.3154          | 11.2608 | 2.4103  | 10.5319 | 14.1043 | 0.8668         | 0.8554        | 0.8606       |
| 2.5173        | 3.17  | 9600  | 2.3077          | 10.9017 | 2.3331  | 10.3352 | 14.4315 | 0.8643         | 0.8553        | 0.8593       |
| 2.5151        | 3.24  | 9800  | 2.3006          | 10.7286 | 2.2598  | 10.2093 | 13.728  | 0.8666         | 0.8557        | 0.8607       |
| 2.4883        | 3.31  | 10000 | 2.2952          | 10.858  | 2.3735  | 10.3329 | 14.1227 | 0.8655         | 0.8555        | 0.8601       |
| 2.5006        | 3.37  | 10200 | 2.2861          | 10.7892 | 2.2598  | 10.1382 | 13.4254 | 0.8688         | 0.8563        | 0.8621       |
| 2.4661        | 3.44  | 10400 | 2.2827          | 10.917  | 2.3892  | 10.1933 | 13.9693 | 0.8685         | 0.8556        | 0.8616       |
| 2.4945        | 3.51  | 10600 | 2.2767          | 11.0385 | 2.551   | 10.3031 | 14.2004 | 0.867          | 0.8558        | 0.8609       |
| 2.4588        | 3.57  | 10800 | 2.2692          | 10.9102 | 2.3942  | 10.169  | 14.2331 | 0.8671         | 0.8562        | 0.8612       |
| 2.4649        | 3.64  | 11000 | 2.2632          | 10.9139 | 2.3421  | 10.2154 | 14.3374 | 0.8654         | 0.8556        | 0.86         |
| 2.4763        | 3.7   | 11200 | 2.2556          | 11.7755 | 2.6463  | 11.0983 | 14.2065 | 0.8684         | 0.8563        | 0.8619       |
| 2.4497        | 3.77  | 11400 | 2.2493          | 11.1075 | 2.372   | 10.4027 | 14.0777 | 0.8662         | 0.8556        | 0.8605       |
| 2.4567        | 3.84  | 11600 | 2.2467          | 11.1779 | 2.2802  | 10.4564 | 14.6646 | 0.8666         | 0.8565        | 0.8611       |
| 2.4421        | 3.9   | 11800 | 2.2395          | 11.0069 | 2.3448  | 10.3473 | 14.3681 | 0.8662         | 0.856         | 0.8607       |
| 2.4269        | 3.97  | 12000 | 2.2360          | 11.0892 | 2.3662  | 10.4705 | 14.9325 | 0.8651         | 0.8562        | 0.8602       |
| 2.4406        | 4.03  | 12200 | 2.2318          | 11.4857 | 2.4196  | 10.7859 | 14.3783 | 0.8669         | 0.8565        | 0.8613       |
| 2.4121        | 4.1   | 12400 | 2.2283          | 11.3331 | 2.3757  | 10.6601 | 14.3661 | 0.867          | 0.8561        | 0.8611       |
| 2.4076        | 4.17  | 12600 | 2.2246          | 11.7127 | 2.437   | 10.98   | 14.5583 | 0.8664         | 0.8559        | 0.8607       |
| 2.4133        | 4.23  | 12800 | 2.2199          | 11.3607 | 2.467   | 10.65   | 13.9877 | 0.8667         | 0.8557        | 0.8607       |
| 2.4241        | 4.3   | 13000 | 2.2158          | 11.4633 | 2.5719  | 10.7229 | 14.0757 | 0.8664         | 0.8562        | 0.8608       |
| 2.4116        | 4.37  | 13200 | 2.2137          | 11.6146 | 2.4491  | 10.8822 | 14.0777 | 0.8662         | 0.8559        | 0.8606       |
| 2.4188        | 4.43  | 13400 | 2.2130          | 11.2618 | 2.3805  | 10.5944 | 14.2883 | 0.866          | 0.8557        | 0.8604       |
| 2.4074        | 4.5   | 13600 | 2.2105          | 11.3653 | 2.4117  | 10.6008 | 14.1472 | 0.8668         | 0.8559        | 0.8609       |
| 2.4063        | 4.56  | 13800 | 2.2092          | 11.6648 | 2.7294  | 10.9428 | 14.3395 | 0.8665         | 0.8566        | 0.8611       |
| 2.3941        | 4.63  | 14000 | 2.2058          | 11.7881 | 2.7584  | 11.0606 | 14.2086 | 0.8661         | 0.8562        | 0.8607       |
| 2.3943        | 4.7   | 14200 | 2.2051          | 11.6491 | 2.7424  | 10.8839 | 14.2597 | 0.8665         | 0.8564        | 0.861        |
| 2.3948        | 4.76  | 14400 | 2.2038          | 11.6683 | 2.7736  | 10.8596 | 14.4315 | 0.8662         | 0.8565        | 0.8609       |
| 2.3716        | 4.83  | 14600 | 2.2024          | 11.8931 | 2.7546  | 11.0867 | 14.3088 | 0.8666         | 0.8565        | 0.8611       |
| 2.4017        | 4.89  | 14800 | 2.2017          | 11.973  | 2.7664  | 11.1738 | 14.2495 | 0.8671         | 0.8567        | 0.8615       |
| 2.4215        | 4.96  | 15000 | 2.2012          | 12.02   | 2.8059  | 11.2005 | 14.229  | 0.8673         | 0.8568        | 0.8616       |


### Framework versions

- Transformers 4.38.1
- Pytorch 2.0.1+cu118
- Datasets 2.18.0
- Tokenizers 0.15.2
