
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# BERTModified-finetuned-wikitext-full

This model was trained from scratch on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 13.7327
- Precision: 0.1676
- Recall: 0.1676
- F1: 0.1676
- Accuracy: 0.1676

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 120

### Training results

| Training Loss | Epoch | Step   | Accuracy | F1     | Validation Loss | Precision | Recall |
|:-------------:|:-----:|:------:|:--------:|:------:|:---------------:|:---------:|:------:|
| 22.0255       | 1.0   | 4253   | 0.0916   | 0.0916 | 20.2496         | 0.0916    | 0.0916 |
| 19.842        | 2.0   | 8506   | 0.1143   | 0.1143 | 19.0691         | 0.1143    | 0.1143 |
| 18.775        | 3.0   | 12759  | 0.1206   | 0.1206 | 18.5569         | 0.1206    | 0.1206 |
| 18.1046       | 4.0   | 17012  | 0.1251   | 0.1251 | 17.9528         | 0.1251    | 0.1251 |
| 17.683        | 5.0   | 21265  | 0.1251   | 0.1251 | 17.5635         | 0.1251    | 0.1251 |
| 17.5511       | 6.0   | 25518  | 0.1259   | 0.1259 | 16.8760         | 0.1259    | 0.1259 |
| 16.6864       | 7.0   | 29771  | 0.1335   | 0.1335 | 16.4200         | 0.1335    | 0.1335 |
| 16.0819       | 8.0   | 34024  | 0.1314   | 0.1314 | 16.0207         | 0.1314    | 0.1314 |
| 15.7008       | 9.0   | 38277  | 0.1378   | 0.1378 | 15.7501         | 0.1378    | 0.1378 |
| 15.4118       | 10.0  | 42530  | 0.1388   | 0.1388 | 15.5762         | 0.1388    | 0.1388 |
| 15.1596       | 11.0  | 46783  | 0.1449   | 0.1449 | 15.3382         | 0.1449    | 0.1449 |
| 14.9354       | 12.0  | 51036  | 0.1452   | 0.1452 | 15.2556         | 0.1452    | 0.1452 |
| 14.7266       | 13.0  | 55289  | 0.1464   | 0.1464 | 15.0213         | 0.1464    | 0.1464 |
| 14.5535       | 14.0  | 59542  | 0.1442   | 0.1442 | 14.9560         | 0.1442    | 0.1442 |
| 14.3903       | 15.0  | 63795  | 0.1538   | 0.1538 | 14.8365         | 0.1538    | 0.1538 |
| 14.2298       | 16.0  | 68048  | 0.1523   | 0.1523 | 14.8610         | 0.1523    | 0.1523 |
| 14.0871       | 17.0  | 72301  | 0.1536   | 0.1536 | 14.6533         | 0.1536    | 0.1536 |
| 13.9559       | 18.0  | 76554  | 0.1613   | 0.1613 | 14.6200         | 0.1613    | 0.1613 |
| 13.8601       | 19.0  | 80807  | 0.1624   | 0.1624 | 14.5995         | 0.1624    | 0.1624 |
| 13.7507       | 20.0  | 85060  | 0.1539   | 0.1539 | 14.4666         | 0.1539    | 0.1539 |
| 13.6544       | 21.0  | 89313  | 0.1575   | 0.1575 | 14.5517         | 0.1575    | 0.1575 |
| 13.5541       | 22.0  | 93566  | 0.1590   | 0.1590 | 14.5004         | 0.1590    | 0.1590 |
| 13.4733       | 23.0  | 97819  | 0.1648   | 0.1648 | 14.4233         | 0.1648    | 0.1648 |
| 13.3851       | 24.0  | 102072 | 0.1561   | 0.1561 | 14.3381         | 0.1561    | 0.1561 |
| 13.3094       | 25.0  | 106325 | 0.1630   | 0.1630 | 14.3428         | 0.1630    | 0.1630 |
| 13.2396       | 26.0  | 110578 | 0.1602   | 0.1602 | 14.3448         | 0.1602    | 0.1602 |
| 13.1548       | 27.0  | 114831 | 0.1603   | 0.1603 | 14.4059         | 0.1603    | 0.1603 |
| 13.0713       | 28.0  | 119084 | 0.1647   | 0.1647 | 14.2224         | 0.1647    | 0.1647 |
| 13.0317       | 29.0  | 123337 | 0.1619   | 0.1619 | 14.1962         | 0.1619    | 0.1619 |
| 12.9748       | 30.0  | 127590 | 0.1633   | 0.1633 | 14.2379         | 0.1633    | 0.1633 |
| 12.9308       | 31.0  | 131843 | 0.1648   | 0.1648 | 14.3591         | 0.1648    | 0.1648 |
| 12.8504       | 32.0  | 136096 | 0.1612   | 0.1612 | 14.1515         | 0.1612    | 0.1612 |
| 12.8249       | 33.0  | 140349 | 0.1625   | 0.1625 | 14.2247         | 0.1625    | 0.1625 |
| 12.76         | 34.0  | 144602 | 0.1588   | 0.1588 | 14.3861         | 0.1588    | 0.1588 |
| 12.7458       | 35.0  | 148855 | 0.1677   | 0.1677 | 14.1783         | 0.1677    | 0.1677 |
| 12.6804       | 36.0  | 153108 | 0.1639   | 0.1639 | 14.0968         | 0.1639    | 0.1639 |
| 12.6653       | 37.0  | 157361 | 0.1604   | 0.1604 | 13.9660         | 0.1604    | 0.1604 |
| 12.6191       | 38.0  | 161614 | 0.1680   | 0.1680 | 14.1075         | 0.1680    | 0.1680 |
| 12.5721       | 39.0  | 165867 | 0.1664   | 0.1664 | 13.9293         | 0.1664    | 0.1664 |
| 12.5647       | 40.0  | 170120 | 0.1693   | 0.1693 | 14.0956         | 0.1693    | 0.1693 |
| 12.5205       | 41.0  | 174373 | 0.1680   | 0.1680 | 14.1204         | 0.1680    | 0.1680 |
| 12.4846       | 42.0  | 178626 | 0.1655   | 0.1655 | 14.0288         | 0.1655    | 0.1655 |
| 12.4415       | 43.0  | 182879 | 0.1659   | 0.1659 | 14.1088         | 0.1659    | 0.1659 |
| 12.4219       | 44.0  | 187132 | 0.1661   | 0.1661 | 14.1751         | 0.1661    | 0.1661 |
| 12.4161       | 45.0  | 191385 | 0.1691   | 0.1691 | 14.0942         | 0.1691    | 0.1691 |
| 12.3982       | 46.0  | 195638 | 0.1607   | 0.1607 | 14.1141         | 0.1607    | 0.1607 |
| 12.3699       | 47.0  | 199891 | 0.1660   | 0.1660 | 14.0529         | 0.1660    | 0.1660 |
| 12.3591       | 48.0  | 204144 | 0.1693   | 0.1693 | 13.8278         | 0.1693    | 0.1693 |
| 12.3295       | 49.0  | 208397 | 0.1662   | 0.1662 | 14.2327         | 0.1662    | 0.1662 |
| 12.3284       | 50.0  | 212650 | 0.1685   | 0.1685 | 14.1071         | 0.1685    | 0.1685 |
| 12.5499       | 51.0  | 216903 | 0.1703   | 0.1703 | 14.0200         | 0.1703    | 0.1703 |
| 12.5456       | 52.0  | 221156 | 0.1679   | 0.1679 | 14.1215         | 0.1679    | 0.1679 |
| 12.5275       | 53.0  | 225409 | 0.1645   | 0.1645 | 14.0103         | 0.1645    | 0.1645 |
| 12.4778       | 54.0  | 229662 | 0.1669   | 0.1669 | 14.0708         | 0.1669    | 0.1669 |
| 12.4607       | 55.0  | 233915 | 0.1681   | 0.1681 | 13.9272         | 0.1681    | 0.1681 |
| 12.3859       | 56.0  | 238168 | 0.1653   | 0.1653 | 14.0765         | 0.1653    | 0.1653 |
| 12.3655       | 57.0  | 242421 | 0.1667   | 0.1667 | 13.9649         | 0.1667    | 0.1667 |
| 12.3407       | 58.0  | 246674 | 0.1671   | 0.1671 | 13.9990         | 0.1671    | 0.1671 |
| 12.3088       | 59.0  | 250927 | 0.1656   | 0.1656 | 14.1460         | 0.1656    | 0.1656 |
| 12.2647       | 60.0  | 255180 | 0.1634   | 0.1634 | 14.0701         | 0.1634    | 0.1634 |
| 12.2248       | 61.0  | 259433 | 0.1647   | 0.1647 | 14.1750         | 0.1647    | 0.1647 |
| 12.1574       | 62.0  | 263686 | 0.1664   | 0.1664 | 14.0358         | 0.1664    | 0.1664 |
| 12.1868       | 63.0  | 267939 | 0.1655   | 0.1655 | 14.1684         | 0.1655    | 0.1655 |
| 12.1522       | 64.0  | 272192 | 0.1629   | 0.1629 | 13.8736         | 0.1629    | 0.1629 |
| 12.1013       | 65.0  | 276445 | 0.1679   | 0.1679 | 14.0409         | 0.1679    | 0.1679 |
| 12.0726       | 66.0  | 280698 | 0.1664   | 0.1664 | 14.1370         | 0.1664    | 0.1664 |
| 12.0498       | 67.0  | 284951 | 0.1636   | 0.1636 | 14.3254         | 0.1636    | 0.1636 |
| 12.016        | 68.0  | 289204 | 0.1656   | 0.1656 | 13.9675         | 0.1656    | 0.1656 |
| 12.0105       | 69.0  | 293457 | 0.1683   | 0.1683 | 13.9938         | 0.1683    | 0.1683 |
| 11.9617       | 70.0  | 297710 | 0.1691   | 0.1691 | 13.8582         | 0.1691    | 0.1691 |
| 11.9515       | 71.0  | 301963 | 0.1633   | 0.1633 | 14.1426         | 0.1633    | 0.1633 |
| 11.9394       | 72.0  | 306216 | 0.1666   | 0.1666 | 14.1693         | 0.1666    | 0.1666 |
| 11.8926       | 73.0  | 310469 | 0.1686   | 0.1686 | 13.8987         | 0.1686    | 0.1686 |
| 11.8816       | 74.0  | 314722 | 0.1703   | 0.1703 | 13.8562         | 0.1703    | 0.1703 |
| 11.8537       | 75.0  | 318975 | 0.1664   | 0.1664 | 13.8956         | 0.1664    | 0.1664 |
| 11.8447       | 76.0  | 323228 | 0.1672   | 0.1672 | 13.9239         | 0.1672    | 0.1672 |
| 11.8166       | 77.0  | 327481 | 0.1712   | 0.1712 | 13.9084         | 0.1712    | 0.1712 |
| 11.7861       | 78.0  | 331734 | 0.1721   | 0.1721 | 13.8685         | 0.1721    | 0.1721 |
| 11.7681       | 79.0  | 335987 | 0.1679   | 0.1679 | 13.8594         | 0.1679    | 0.1679 |
| 11.7506       | 80.0  | 340240 | 0.1655   | 0.1655 | 13.8944         | 0.1655    | 0.1655 |
| 11.7237       | 81.0  | 344493 | 0.1678   | 0.1678 | 13.8826         | 0.1678    | 0.1678 |
| 11.7115       | 82.0  | 348746 | 0.1650   | 0.1650 | 13.7157         | 0.1650    | 0.1650 |
| 11.6859       | 83.0  | 352999 | 0.1713   | 0.1713 | 14.0802         | 0.1713    | 0.1713 |
| 11.6991       | 84.0  | 357252 | 0.1722   | 0.1722 | 13.8076         | 0.1722    | 0.1722 |
| 11.6674       | 85.0  | 361505 | 0.1712   | 0.1712 | 13.9484         | 0.1712    | 0.1712 |
| 11.6401       | 86.0  | 365758 | 0.1718   | 0.1718 | 13.8485         | 0.1718    | 0.1718 |
| 11.6477       | 87.0  | 370011 | 0.1701   | 0.1701 | 13.7326         | 0.1701    | 0.1701 |
| 11.6111       | 88.0  | 374264 | 0.1695   | 0.1695 | 14.0255         | 0.1695    | 0.1695 |
| 11.6031       | 89.0  | 378517 | 0.1704   | 0.1704 | 13.7595         | 0.1704    | 0.1704 |
| 11.5898       | 90.0  | 382770 | 0.1677   | 0.1677 | 14.0630         | 0.1677    | 0.1677 |
| 11.5808       | 91.0  | 387023 | 0.1743   | 0.1743 | 13.9293         | 0.1743    | 0.1743 |
| 11.5664       | 92.0  | 391276 | 0.1712   | 0.1712 | 13.9230         | 0.1712    | 0.1712 |
| 11.5692       | 93.0  | 395529 | 0.1694   | 0.1694 | 13.9230         | 0.1694    | 0.1694 |
| 11.5578       | 94.0  | 399782 | 0.1713   | 0.1713 | 13.9748         | 0.1713    | 0.1713 |
| 11.5067       | 95.0  | 404035 | 0.1770   | 0.1770 | 13.8655         | 0.1770    | 0.1770 |
| 11.5176       | 96.0  | 408288 | 0.1740   | 0.1740 | 14.0284         | 0.1740    | 0.1740 |
| 11.5408       | 97.0  | 412541 | 0.1718   | 0.1718 | 13.8855         | 0.1718    | 0.1718 |
| 11.4876       | 98.0  | 416794 | 0.1746   | 0.1746 | 13.6721         | 0.1746    | 0.1746 |
| 11.5056       | 99.0  | 421047 | 0.1734   | 0.1734 | 13.8167         | 0.1734    | 0.1734 |
| 11.5185       | 100.0 | 425300 | 0.1716   | 0.1716 | 13.8581         | 0.1716    | 0.1716 |
| 11.5619       | 101.0 | 429553 | 13.9219  | 0.1687 | 0.1687          | 0.1687    | 0.1687 |
| 11.5759       | 102.0 | 433806 | 14.1460  | 0.1663 | 0.1663          | 0.1663    | 0.1663 |
| 11.5661       | 103.0 | 438059 | 13.8979  | 0.1724 | 0.1724          | 0.1724    | 0.1724 |
| 11.5127       | 104.0 | 442312 | 13.7580  | 0.1727 | 0.1727          | 0.1727    | 0.1727 |
| 11.5476       | 105.0 | 446565 | 14.1029  | 0.1675 | 0.1675          | 0.1675    | 0.1675 |
| 11.5309       | 106.0 | 450818 | 14.0572  | 0.1704 | 0.1704          | 0.1704    | 0.1704 |
| 11.507        | 107.0 | 455071 | 13.8523  | 0.1734 | 0.1734          | 0.1734    | 0.1734 |
| 11.5251       | 108.0 | 459324 | 13.9018  | 0.1701 | 0.1701          | 0.1701    | 0.1701 |
| 11.4931       | 109.0 | 463577 | 13.7898  | 0.1786 | 0.1786          | 0.1786    | 0.1786 |
| 11.5107       | 110.0 | 467830 | 13.8426  | 0.1764 | 0.1764          | 0.1764    | 0.1764 |
| 11.4665       | 111.0 | 472083 | 13.9605  | 0.1705 | 0.1705          | 0.1705    | 0.1705 |
| 11.4901       | 112.0 | 476336 | 13.8094  | 0.1743 | 0.1743          | 0.1743    | 0.1743 |
| 11.4702       | 113.0 | 480589 | 13.9445  | 0.1664 | 0.1664          | 0.1664    | 0.1664 |
| 11.4617       | 114.0 | 484842 | 13.8887  | 0.1707 | 0.1707          | 0.1707    | 0.1707 |
| 11.4458       | 115.0 | 489095 | 13.8951  | 0.1701 | 0.1701          | 0.1701    | 0.1701 |
| 11.4361       | 116.0 | 493348 | 13.7645  | 0.1705 | 0.1705          | 0.1705    | 0.1705 |
| 11.4176       | 117.0 | 497601 | 13.7752  | 0.1734 | 0.1734          | 0.1734    | 0.1734 |
| 11.4359       | 118.0 | 501854 | 13.9083  | 0.1714 | 0.1714          | 0.1714    | 0.1714 |
| 11.409        | 119.0 | 506107 | 13.9772  | 0.1668 | 0.1668          | 0.1668    | 0.1668 |
| 11.4238       | 120.0 | 510360 | 13.7327  | 0.1676 | 0.1676          | 0.1676    | 0.1676 |


### Framework versions

- Transformers 4.24.0
- Pytorch 2.0.0
- Datasets 2.6.1
- Tokenizers 0.13.1

{"tags": ["generated_from_trainer"], "metrics": ["precision", "recall", "f1", "accuracy"], "model-index": [{"name": "BERTModified-finetuned-wikitext-full", "results": []}]}
pytorch generated_from_trainer region:us
TAGS
#pytorch #generated_from_trainer #region-us 

BERTModified-finetuned-wikitext-full
====================================


This model was trained from scratch on the None dataset.
It achieves the following results on the evaluation set:


* Loss: 13.7327
* Precision: 0.1676
* Recall: 0.1676
* F1: 0.1676
* Accuracy: 0.1676


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.0001
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 120


### Training results



### Framework versions


* Transformers 4.24.0
* Pytorch 2.0.0
* Datasets 2.6.1
* Tokenizers 0.13.1



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.0001
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 120 ### Training results ### Framework versions


* Transformers 4.24.0
* Pytorch 2.0.0
* Datasets 2.6.1
* Tokenizers 0.13.1
TAGS
#pytorch #generated_from_trainer #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.0001
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 120 ### Training results ### Framework versions


* Transformers 4.24.0
* Pytorch 2.0.0
* Datasets 2.6.1
* Tokenizers 0.13.1
TAGS
#pytorch #generated_from_trainer #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.0001
* train\_batch\_size: 4
* eval\_batch\_size: 4
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 120### Training results### Framework versions


* Transformers 4.24.0
* Pytorch 2.0.0
* Datasets 2.6.1
* Tokenizers 0.13.1