generated_from_trainer dataset:mbpp base_model:Salesforce/codet5p-770m-py license:bsd-3-clause region:us
---
license: bsd-3-clause
base_model: Salesforce/codet5p-770m-py
tags:
- generated_from_trainer
datasets:
- mbpp
model-index:
- name: codet5p-770m-py-sanitized-codebleu-1-True-0.0001-0.1-lora-layer_20_21_22_23
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# codet5p-770m-py-sanitized-codebleu-1-True-0.0001-0.1-lora-layer_20_21_22_23

This model is a fine-tuned version of [Salesforce/codet5p-770m-py](https://huggingface.co/Salesforce/codet5p-770m-py) on the mbpp dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7246
- Codebleu: 0.1153
- Ngram Match Score: 0.0337
- Weighted Ngram Match Score: 0.0601
- Syntax Match Score: 0.1402
- Dataflow Match Score: 0.1245

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 64

### Training results

| Training Loss | Epoch | Step | Validation Loss | Codebleu | Ngram Match Score | Weighted Ngram Match Score | Syntax Match Score | Dataflow Match Score |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-----------------:|:--------------------------:|:------------------:|:--------------------:|
| 0.9741        | 1.0   | 15   | 0.9220          | 0.0072   | 0.0000            | 0.0000                     | 0.0079             | 0.0100               |
| 0.9661        | 2.0   | 30   | 0.9098          | 0.0080   | 0.0000            | 0.0000                     | 0.0079             | 0.0120               |
| 0.9382        | 3.0   | 45   | 0.8789          | 0.0526   | 0.0016            | 0.0263                     | 0.0582             | 0.0663               |
| 0.8856        | 4.0   | 60   | 0.8316          | 0.0981   | 0.0221            | 0.0518                     | 0.1283             | 0.0984               |
| 0.8406        | 5.0   | 75   | 0.7946          | 0.1019   | 0.0205            | 0.0492                     | 0.1310             | 0.1064               |
| 0.79          | 6.0   | 90   | 0.7619          | 0.1083   | 0.0208            | 0.0535                     | 0.1296             | 0.1225               |
| 0.7691        | 7.0   | 105  | 0.7426          | 0.1043   | 0.0169            | 0.0417                     | 0.1217             | 0.1245               |
| 0.7374        | 8.0   | 120  | 0.7276          | 0.1011   | 0.0196            | 0.0469                     | 0.1217             | 0.1145               |
| 0.7237        | 9.0   | 135  | 0.7185          | 0.1141   | 0.0273            | 0.0596                     | 0.1310             | 0.1325               |
| 0.7011        | 10.0  | 150  | 0.7136          | 0.1110   | 0.0276            | 0.0603                     | 0.1270             | 0.1285               |
| 0.6903        | 11.0  | 165  | 0.7049          | 0.1001   | 0.0227            | 0.0498                     | 0.1177             | 0.1145               |
| 0.6824        | 12.0  | 180  | 0.7003          | 0.1094   | 0.0281            | 0.0600                     | 0.1230             | 0.1285               |
| 0.6642        | 13.0  | 195  | 0.7011          | 0.0987   | 0.0245            | 0.0521                     | 0.1151             | 0.1124               |
| 0.6377        | 14.0  | 210  | 0.6989          | 0.0960   | 0.0235            | 0.0473                     | 0.1098             | 0.1124               |
| 0.6622        | 15.0  | 225  | 0.7008          | 0.1000   | 0.0265            | 0.0498                     | 0.1204             | 0.1104               |
| 0.6408        | 16.0  | 240  | 0.6974          | 0.0930   | 0.0221            | 0.0457                     | 0.1111             | 0.1044               |
| 0.6238        | 17.0  | 255  | 0.6993          | 0.0972   | 0.0180            | 0.0386                     | 0.1243             | 0.1044               |
| 0.602         | 18.0  | 270  | 0.6975          | 0.1023   | 0.0238            | 0.0467                     | 0.1296             | 0.1084               |
| 0.5989        | 19.0  | 285  | 0.7000          | 0.0994   | 0.0258            | 0.0481                     | 0.1257             | 0.1044               |
| 0.5867        | 20.0  | 300  | 0.6999          | 0.0957   | 0.0203            | 0.0407                     | 0.1257             | 0.0984               |
| 0.5761        | 21.0  | 315  | 0.6938          | 0.1047   | 0.0210            | 0.0421                     | 0.1376             | 0.1084               |
| 0.5667        | 22.0  | 330  | 0.6957          | 0.1081   | 0.0243            | 0.0436                     | 0.1349             | 0.1185               |
| 0.5593        | 23.0  | 345  | 0.6944          | 0.0996   | 0.0257            | 0.0499                     | 0.1257             | 0.1044               |
| 0.5535        | 24.0  | 360  | 0.6986          | 0.1045   | 0.0219            | 0.0421                     | 0.1389             | 0.1064               |
| 0.5537        | 25.0  | 375  | 0.6990          | 0.1072   | 0.0242            | 0.0426                     | 0.1389             | 0.1124               |
| 0.539         | 26.0  | 390  | 0.6991          | 0.1035   | 0.0270            | 0.0504                     | 0.1349             | 0.1044               |
| 0.5201        | 27.0  | 405  | 0.6999          | 0.1016   | 0.0225            | 0.0439                     | 0.1310             | 0.1064               |
| 0.5245        | 28.0  | 420  | 0.7014          | 0.1041   | 0.0210            | 0.0410                     | 0.1323             | 0.1124               |
| 0.5112        | 29.0  | 435  | 0.7031          | 0.1095   | 0.0266            | 0.0522                     | 0.1415             | 0.1124               |
| 0.4964        | 30.0  | 450  | 0.7013          | 0.1037   | 0.0261            | 0.0505                     | 0.1336             | 0.1064               |
| 0.501         | 31.0  | 465  | 0.6992          | 0.1016   | 0.0253            | 0.0491                     | 0.1230             | 0.1124               |
| 0.489         | 32.0  | 480  | 0.7072          | 0.0954   | 0.0336            | 0.0583                     | 0.1190             | 0.0964               |
| 0.5003        | 33.0  | 495  | 0.7060          | 0.0960   | 0.0282            | 0.0520                     | 0.1217             | 0.0984               |
| 0.4845        | 34.0  | 510  | 0.7069          | 0.1108   | 0.0357            | 0.0670                     | 0.1310             | 0.1205               |
| 0.4733        | 35.0  | 525  | 0.7048          | 0.1030   | 0.0347            | 0.0609                     | 0.1190             | 0.1145               |
| 0.4832        | 36.0  | 540  | 0.7059          | 0.1095   | 0.0373            | 0.0625                     | 0.1243             | 0.1245               |
| 0.4722        | 37.0  | 555  | 0.7111          | 0.1062   | 0.0397            | 0.0670                     | 0.1243             | 0.1145               |
| 0.4679        | 38.0  | 570  | 0.7122          | 0.0951   | 0.0340            | 0.0602                     | 0.1138             | 0.1004               |
| 0.4721        | 39.0  | 585  | 0.7113          | 0.1007   | 0.0325            | 0.0589                     | 0.1204             | 0.1084               |
| 0.4706        | 40.0  | 600  | 0.7135          | 0.1075   | 0.0368            | 0.0610                     | 0.1217             | 0.1225               |
| 0.4694        | 41.0  | 615  | 0.7147          | 0.1053   | 0.0325            | 0.0576                     | 0.1243             | 0.1165               |
| 0.4571        | 42.0  | 630  | 0.7146          | 0.1052   | 0.0369            | 0.0649                     | 0.1230             | 0.1145               |
| 0.4584        | 43.0  | 645  | 0.7170          | 0.1089   | 0.0359            | 0.0639                     | 0.1310             | 0.1165               |
| 0.4592        | 44.0  | 660  | 0.7173          | 0.1043   | 0.0342            | 0.0616                     | 0.1243             | 0.1124               |
| 0.4474        | 45.0  | 675  | 0.7154          | 0.1092   | 0.0350            | 0.0620                     | 0.1283             | 0.1205               |
| 0.4424        | 46.0  | 690  | 0.7215          | 0.1080   | 0.0364            | 0.0641                     | 0.1283             | 0.1165               |
| 0.4467        | 47.0  | 705  | 0.7204          | 0.1117   | 0.0339            | 0.0581                     | 0.1296             | 0.1265               |
| 0.4476        | 48.0  | 720  | 0.7215          | 0.1136   | 0.0357            | 0.0624                     | 0.1310             | 0.1285               |
| 0.4488        | 49.0  | 735  | 0.7205          | 0.1092   | 0.0335            | 0.0602                     | 0.1270             | 0.1225               |
| 0.4413        | 50.0  | 750  | 0.7249          | 0.1036   | 0.0337            | 0.0605                     | 0.1230             | 0.1124               |
| 0.4432        | 51.0  | 765  | 0.7231          | 0.1068   | 0.0327            | 0.0555                     | 0.1204             | 0.1245               |
| 0.446         | 52.0  | 780  | 0.7197          | 0.1085   | 0.0321            | 0.0545                     | 0.1230             | 0.1265               |
| 0.4386        | 53.0  | 795  | 0.7235          | 0.1175   | 0.0374            | 0.0630                     | 0.1362             | 0.1325               |
| 0.4276        | 54.0  | 810  | 0.7242          | 0.1177   | 0.0348            | 0.0592                     | 0.1402             | 0.1305               |
| 0.4388        | 55.0  | 825  | 0.7244          | 0.1108   | 0.0325            | 0.0598                     | 0.1376             | 0.1165               |
| 0.4341        | 56.0  | 840  | 0.7256          | 0.1141   | 0.0346            | 0.0638                     | 0.1402             | 0.1205               |
| 0.4334        | 57.0  | 855  | 0.7280          | 0.1127   | 0.0347            | 0.0629                     | 0.1389             | 0.1185               |
| 0.425         | 58.0  | 870  | 0.7280          | 0.1123   | 0.0350            | 0.0640                     | 0.1376             | 0.1185               |
| 0.4376        | 59.0  | 885  | 0.7248          | 0.1054   | 0.0322            | 0.0592                     | 0.1283             | 0.1124               |
| 0.4199        | 60.0  | 900  | 0.7236          | 0.1153   | 0.0337            | 0.0601                     | 0.1402             | 0.1245               |
| 0.4349        | 61.0  | 915  | 0.7241          | 0.1153   | 0.0337            | 0.0601                     | 0.1402             | 0.1245               |
| 0.4302        | 62.0  | 930  | 0.7241          | 0.1153   | 0.0337            | 0.0601                     | 0.1402             | 0.1245               |
| 0.4333        | 63.0  | 945  | 0.7244          | 0.1153   | 0.0337            | 0.0601                     | 0.1402             | 0.1245               |
| 0.4212        | 64.0  | 960  | 0.7246          | 0.1153   | 0.0337            | 0.0601                     | 0.1402             | 0.1245               |


### Framework versions

- Transformers 4.31.0
- Pytorch 2.0.1
- Datasets 2.14.4
- Tokenizers 0.13.3
