generated_from_trainer dataset:mbpp base_model:Salesforce/codet5p-770m-py license:bsd-3-clause region:us
---
license: bsd-3-clause
base_model: Salesforce/codet5p-770m-py
tags:
- generated_from_trainer
datasets:
- mbpp
model-index:
- name: codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_0_9_21_23
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_0_9_21_23

This model is a fine-tuned version of [Salesforce/codet5p-770m-py](https://huggingface.co/Salesforce/codet5p-770m-py) on the mbpp dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6910
- Codebleu: 0.1101
- Ngram Match Score: 0.0179
- Weighted Ngram Match Score: 0.0340
- Syntax Match Score: 0.1177
- Dataflow Match Score: 0.1446

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 64

### Training results

| Training Loss | Epoch | Step | Validation Loss | Codebleu | Ngram Match Score | Weighted Ngram Match Score | Syntax Match Score | Dataflow Match Score |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-----------------:|:--------------------------:|:------------------:|:--------------------:|
| 0.9819        | 1.0   | 15   | 0.9236          | 0.0072   | 0.0000            | 0.0000                     | 0.0079             | 0.0100               |
| 0.9665        | 2.0   | 30   | 0.9191          | 0.0072   | 0.0000            | 0.0000                     | 0.0079             | 0.0100               |
| 0.9678        | 3.0   | 45   | 0.9089          | 0.0080   | 0.0000            | 0.0001                     | 0.0079             | 0.0120               |
| 0.9211        | 4.0   | 60   | 0.8882          | 0.0332   | 0.0001            | 0.0151                     | 0.0331             | 0.0462               |
| 0.919         | 5.0   | 75   | 0.8541          | 0.0834   | 0.0115            | 0.0349                     | 0.1005             | 0.0964               |
| 0.8921        | 6.0   | 90   | 0.8245          | 0.0982   | 0.0213            | 0.0512                     | 0.1270             | 0.1004               |
| 0.8575        | 7.0   | 105  | 0.7992          | 0.0952   | 0.0189            | 0.0470                     | 0.1190             | 0.1024               |
| 0.8053        | 8.0   | 120  | 0.7769          | 0.0968   | 0.0176            | 0.0458                     | 0.1217             | 0.1044               |
| 0.7762        | 9.0   | 135  | 0.7629          | 0.0989   | 0.0153            | 0.0426                     | 0.1204             | 0.1124               |
| 0.7671        | 10.0  | 150  | 0.7534          | 0.0992   | 0.0091            | 0.0277                     | 0.1204             | 0.1185               |
| 0.7504        | 11.0  | 165  | 0.7458          | 0.1129   | 0.0143            | 0.0442                     | 0.1310             | 0.1365               |
| 0.7502        | 12.0  | 180  | 0.7393          | 0.1187   | 0.0225            | 0.0576                     | 0.1402             | 0.1365               |
| 0.7452        | 13.0  | 195  | 0.7351          | 0.1172   | 0.0230            | 0.0576                     | 0.1362             | 0.1365               |
| 0.7128        | 14.0  | 210  | 0.7298          | 0.1170   | 0.0222            | 0.0563                     | 0.1323             | 0.1406               |
| 0.7337        | 15.0  | 225  | 0.7247          | 0.1210   | 0.0245            | 0.0599                     | 0.1389             | 0.1426               |
| 0.7374        | 16.0  | 240  | 0.7200          | 0.1216   | 0.0251            | 0.0595                     | 0.1442             | 0.1386               |
| 0.722         | 17.0  | 255  | 0.7158          | 0.1140   | 0.0218            | 0.0513                     | 0.1362             | 0.1305               |
| 0.7002        | 18.0  | 270  | 0.7133          | 0.1156   | 0.0227            | 0.0530                     | 0.1415             | 0.1285               |
| 0.6985        | 19.0  | 285  | 0.7116          | 0.1198   | 0.0279            | 0.0582                     | 0.1455             | 0.1325               |
| 0.6981        | 20.0  | 300  | 0.7091          | 0.1204   | 0.0279            | 0.0582                     | 0.1468             | 0.1325               |
| 0.6788        | 21.0  | 315  | 0.7059          | 0.1165   | 0.0263            | 0.0562                     | 0.1402             | 0.1305               |
| 0.6747        | 22.0  | 330  | 0.7020          | 0.1162   | 0.0275            | 0.0570                     | 0.1389             | 0.1305               |
| 0.6621        | 23.0  | 345  | 0.7013          | 0.1178   | 0.0227            | 0.0481                     | 0.1362             | 0.1406               |
| 0.6599        | 24.0  | 360  | 0.7005          | 0.1110   | 0.0228            | 0.0471                     | 0.1336             | 0.1265               |
| 0.6726        | 25.0  | 375  | 0.6983          | 0.1158   | 0.0220            | 0.0470                     | 0.1296             | 0.1426               |
| 0.6463        | 26.0  | 390  | 0.6932          | 0.1179   | 0.0237            | 0.0481                     | 0.1362             | 0.1406               |
| 0.6269        | 27.0  | 405  | 0.6925          | 0.1150   | 0.0225            | 0.0467                     | 0.1257             | 0.1446               |
| 0.6454        | 28.0  | 420  | 0.6932          | 0.1094   | 0.0225            | 0.0465                     | 0.1177             | 0.1386               |
| 0.6241        | 29.0  | 435  | 0.6922          | 0.1111   | 0.0233            | 0.0469                     | 0.1217             | 0.1386               |
| 0.6253        | 30.0  | 450  | 0.6928          | 0.1198   | 0.0229            | 0.0464                     | 0.1296             | 0.1526               |
| 0.6226        | 31.0  | 465  | 0.6920          | 0.1120   | 0.0241            | 0.0467                     | 0.1257             | 0.1365               |
| 0.6156        | 32.0  | 480  | 0.6925          | 0.1153   | 0.0245            | 0.0476                     | 0.1257             | 0.1446               |
| 0.613         | 33.0  | 495  | 0.6916          | 0.1140   | 0.0251            | 0.0473                     | 0.1243             | 0.1426               |
| 0.612         | 34.0  | 510  | 0.6892          | 0.1112   | 0.0215            | 0.0415                     | 0.1257             | 0.1365               |
| 0.5982        | 35.0  | 525  | 0.6894          | 0.1115   | 0.0203            | 0.0399                     | 0.1230             | 0.1406               |
| 0.6148        | 36.0  | 540  | 0.6897          | 0.1091   | 0.0208            | 0.0397                     | 0.1190             | 0.1386               |
| 0.5948        | 37.0  | 555  | 0.6899          | 0.1068   | 0.0172            | 0.0333                     | 0.1177             | 0.1365               |
| 0.5946        | 38.0  | 570  | 0.6900          | 0.1076   | 0.0182            | 0.0353                     | 0.1190             | 0.1365               |
| 0.5992        | 39.0  | 585  | 0.6907          | 0.1125   | 0.0176            | 0.0343                     | 0.1177             | 0.1506               |
| 0.5966        | 40.0  | 600  | 0.6912          | 0.1116   | 0.0183            | 0.0354                     | 0.1230             | 0.1426               |
| 0.586         | 41.0  | 615  | 0.6918          | 0.1149   | 0.0181            | 0.0339                     | 0.1257             | 0.1486               |
| 0.5936        | 42.0  | 630  | 0.6906          | 0.1116   | 0.0179            | 0.0333                     | 0.1177             | 0.1486               |
| 0.5739        | 43.0  | 645  | 0.6911          | 0.1117   | 0.0177            | 0.0340                     | 0.1217             | 0.1446               |
| 0.5809        | 44.0  | 660  | 0.6911          | 0.1119   | 0.0176            | 0.0335                     | 0.1164             | 0.1506               |
| 0.5752        | 45.0  | 675  | 0.6917          | 0.1083   | 0.0169            | 0.0324                     | 0.1098             | 0.1486               |
| 0.5794        | 46.0  | 690  | 0.6918          | 0.1216   | 0.0215            | 0.0415                     | 0.1257             | 0.1627               |
| 0.5816        | 47.0  | 705  | 0.6917          | 0.1200   | 0.0217            | 0.0415                     | 0.1257             | 0.1586               |
| 0.5738        | 48.0  | 720  | 0.6919          | 0.1160   | 0.0187            | 0.0358                     | 0.1257             | 0.1506               |
| 0.5819        | 49.0  | 735  | 0.6911          | 0.1123   | 0.0180            | 0.0340                     | 0.1190             | 0.1486               |
| 0.5672        | 50.0  | 750  | 0.6918          | 0.1110   | 0.0184            | 0.0345                     | 0.1217             | 0.1426               |
| 0.5852        | 51.0  | 765  | 0.6915          | 0.1098   | 0.0179            | 0.0341                     | 0.1190             | 0.1426               |
| 0.5721        | 52.0  | 780  | 0.6912          | 0.1144   | 0.0177            | 0.0342                     | 0.1243             | 0.1486               |
| 0.5823        | 53.0  | 795  | 0.6906          | 0.1077   | 0.0183            | 0.0338                     | 0.1138             | 0.1426               |
| 0.5503        | 54.0  | 810  | 0.6904          | 0.1112   | 0.0190            | 0.0355                     | 0.1217             | 0.1426               |
| 0.5701        | 55.0  | 825  | 0.6905          | 0.1098   | 0.0179            | 0.0341                     | 0.1190             | 0.1426               |
| 0.577         | 56.0  | 840  | 0.6908          | 0.1101   | 0.0179            | 0.0340                     | 0.1177             | 0.1446               |
| 0.5759        | 57.0  | 855  | 0.6912          | 0.1099   | 0.0180            | 0.0343                     | 0.1190             | 0.1426               |
| 0.5615        | 58.0  | 870  | 0.6911          | 0.1099   | 0.0180            | 0.0343                     | 0.1190             | 0.1426               |
| 0.576         | 59.0  | 885  | 0.6910          | 0.1098   | 0.0179            | 0.0341                     | 0.1190             | 0.1426               |
| 0.5577        | 60.0  | 900  | 0.6909          | 0.1099   | 0.0180            | 0.0343                     | 0.1190             | 0.1426               |
| 0.5743        | 61.0  | 915  | 0.6910          | 0.1111   | 0.0188            | 0.0352                     | 0.1217             | 0.1426               |
| 0.5576        | 62.0  | 930  | 0.6911          | 0.1098   | 0.0179            | 0.0341                     | 0.1190             | 0.1426               |
| 0.5676        | 63.0  | 945  | 0.6910          | 0.1101   | 0.0179            | 0.0340                     | 0.1177             | 0.1446               |
| 0.5567        | 64.0  | 960  | 0.6910          | 0.1101   | 0.0179            | 0.0340                     | 0.1177             | 0.1446               |


### Framework versions

- Transformers 4.31.0
- Pytorch 2.0.1
- Datasets 2.14.4
- Tokenizers 0.13.3
