
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# pretrained-m-bert-300

This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 5.8273
- Validation Loss: 15.6623
- Epoch: 299

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 10.2479    | 10.9372         | 0     |
| 7.7731     | 10.9191         | 1     |
| 6.8702     | 11.5201         | 2     |
| 6.4849     | 11.6086         | 3     |
| 6.3725     | 11.5271         | 4     |
| 6.3243     | 12.1350         | 5     |
| 6.4515     | 11.7665         | 6     |
| 6.0675     | 12.1761         | 7     |
| 5.9322     | 12.1155         | 8     |
| 6.0672     | 12.0390         | 9     |
| 5.9976     | 12.5114         | 10    |
| 5.9208     | 12.7953         | 11    |
| 5.9503     | 12.4924         | 12    |
| 5.9696     | 12.7799         | 13    |
| 6.0537     | 12.3489         | 14    |
| 5.8556     | 12.5165         | 15    |
| 5.8976     | 12.8338         | 16    |
| 5.9458     | 13.0800         | 17    |
| 5.8258     | 12.9819         | 18    |
| 5.8284     | 13.0523         | 19    |
| 5.8739     | 13.0829         | 20    |
| 5.7537     | 13.1990         | 21    |
| 5.8624     | 13.2222         | 22    |
| 5.8871     | 13.1393         | 23    |
| 5.7382     | 13.0271         | 24    |
| 5.6791     | 13.3209         | 25    |
| 5.8651     | 13.5971         | 26    |
| 5.7795     | 14.0682         | 27    |
| 5.7961     | 13.5632         | 28    |
| 5.9525     | 13.0326         | 29    |
| 5.8251     | 13.0935         | 30    |
| 5.7616     | 13.5397         | 31    |
| 5.9793     | 13.4677         | 32    |
| 5.6852     | 13.6610         | 33    |
| 5.7826     | 13.6501         | 34    |
| 5.7675     | 13.3981         | 35    |
| 5.7075     | 13.6568         | 36    |
| 5.8363     | 13.5032         | 37    |
| 5.8045     | 13.6162         | 38    |
| 5.8582     | 13.5919         | 39    |
| 5.6427     | 13.8740         | 40    |
| 5.7807     | 13.7311         | 41    |
| 5.7421     | 14.1702         | 42    |
| 5.7074     | 13.8185         | 43    |
| 5.7145     | 14.0385         | 44    |
| 5.6605     | 14.0947         | 45    |
| 5.6647     | 13.9634         | 46    |
| 5.6628     | 14.1416         | 47    |
| 5.6652     | 13.9625         | 48    |
| 5.8173     | 14.0109         | 49    |
| 5.8535     | 14.0783         | 50    |
| 5.6777     | 14.4908         | 51    |
| 5.7189     | 14.2846         | 52    |
| 5.7306     | 13.9430         | 53    |
| 5.9265     | 14.2692         | 54    |
| 5.6752     | 13.7434         | 55    |
| 5.8745     | 14.2234         | 56    |
| 5.7229     | 14.4659         | 57    |
| 5.7215     | 14.0766         | 58    |
| 5.7540     | 14.3406         | 59    |
| 5.7831     | 13.9421         | 60    |
| 5.6559     | 14.0940         | 61    |
| 5.6964     | 14.4394         | 62    |
| 5.6707     | 14.4002         | 63    |
| 5.7088     | 14.3143         | 64    |
| 5.7738     | 14.3808         | 65    |
| 5.7194     | 14.6182         | 66    |
| 5.7911     | 14.2589         | 67    |
| 5.9282     | 14.3536         | 68    |
| 5.8769     | 14.5976         | 69    |
| 5.7150     | 14.3358         | 70    |
| 5.6573     | 14.2675         | 71    |
| 5.8684     | 14.2212         | 72    |
| 5.6871     | 14.0757         | 73    |
| 5.7349     | 14.9877         | 74    |
| 5.8587     | 14.1604         | 75    |
| 5.8195     | 14.4759         | 76    |
| 5.7681     | 14.4587         | 77    |
| 5.7803     | 14.4228         | 78    |
| 5.6986     | 14.1285         | 79    |
| 5.7369     | 14.5417         | 80    |
| 5.7565     | 14.2100         | 81    |
| 5.7648     | 14.4228         | 82    |
| 5.6307     | 15.0572         | 83    |
| 5.8166     | 14.6594         | 84    |
| 5.7945     | 14.9603         | 85    |
| 5.8273     | 14.6196         | 86    |
| 5.6483     | 15.2973         | 87    |
| 5.7982     | 14.9318         | 88    |
| 5.7286     | 14.4151         | 89    |
| 5.7488     | 14.2480         | 90    |
| 5.7564     | 15.2868         | 91    |
| 5.7200     | 14.9984         | 92    |
| 5.6758     | 14.8934         | 93    |
| 5.8600     | 14.6392         | 94    |
| 5.6302     | 14.9115         | 95    |
| 5.7530     | 14.8292         | 96    |
| 5.6311     | 14.9683         | 97    |
| 5.6845     | 14.8707         | 98    |
| 5.7639     | 15.2866         | 99    |
| 5.7692     | 15.1005         | 100   |
| 5.7279     | 15.5260         | 101   |
| 5.8349     | 14.8966         | 102   |
| 5.7720     | 14.2529         | 103   |
| 5.6082     | 15.5972         | 104   |
| 5.7725     | 15.1931         | 105   |
| 5.8239     | 15.1119         | 106   |
| 5.7973     | 14.8203         | 107   |
| 5.7439     | 15.2762         | 108   |
| 5.7344     | 15.2897         | 109   |
| 5.8002     | 14.8071         | 110   |
| 5.7978     | 15.3206         | 111   |
| 5.8302     | 15.1250         | 112   |
| 5.6829     | 15.3822         | 113   |
| 5.8658     | 14.7853         | 114   |
| 5.7236     | 15.1413         | 115   |
| 5.8151     | 14.9191         | 116   |
| 5.6697     | 15.2308         | 117   |
| 5.8450     | 15.2055         | 118   |
| 5.6843     | 15.3117         | 119   |
| 5.7215     | 15.1254         | 120   |
| 5.8230     | 15.1992         | 121   |
| 5.7106     | 15.2795         | 122   |
| 5.7720     | 15.6248         | 123   |
| 5.7214     | 15.0411         | 124   |
| 5.6302     | 15.2897         | 125   |
| 5.7151     | 15.7383         | 126   |
| 5.7107     | 15.5989         | 127   |
| 5.6569     | 15.2202         | 128   |
| 5.9129     | 15.1588         | 129   |
| 5.5289     | 15.4879         | 130   |
| 5.7570     | 15.5103         | 131   |
| 5.8748     | 15.3842         | 132   |
| 5.7679     | 15.6996         | 133   |
| 5.6655     | 15.2690         | 134   |
| 5.7573     | 15.2401         | 135   |
| 5.7238     | 15.5996         | 136   |
| 5.7273     | 15.3198         | 137   |
| 5.7344     | 15.3389         | 138   |
| 5.8311     | 14.8744         | 139   |
| 5.6549     | 15.6956         | 140   |
| 5.6496     | 15.2694         | 141   |
| 5.7590     | 15.0076         | 142   |
| 5.7703     | 15.3850         | 143   |
| 5.7206     | 15.4296         | 144   |
| 5.8623     | 14.8546         | 145   |
| 5.7601     | 15.4164         | 146   |
| 5.7175     | 15.8795         | 147   |
| 5.6459     | 15.8282         | 148   |
| 5.8591     | 15.3127         | 149   |
| 5.7940     | 16.0000         | 150   |
| 5.8439     | 15.5051         | 151   |
| 5.7669     | 15.9199         | 152   |
| 5.6481     | 15.2306         | 153   |
| 5.7793     | 15.4377         | 154   |
| 5.8167     | 15.7849         | 155   |
| 5.7556     | 15.2991         | 156   |
| 5.7905     | 15.5514         | 157   |
| 5.5980     | 15.6595         | 158   |
| 5.7624     | 15.7794         | 159   |
| 5.7073     | 15.7131         | 160   |
| 5.7823     | 15.6013         | 161   |
| 5.6993     | 15.3206         | 162   |
| 5.8054     | 15.1585         | 163   |
| 5.7734     | 15.3361         | 164   |
| 5.6832     | 16.0706         | 165   |
| 5.6192     | 15.7624         | 166   |
| 5.8735     | 15.9157         | 167   |
| 5.7212     | 15.5399         | 168   |
| 5.7479     | 15.7155         | 169   |
| 5.6542     | 16.2107         | 170   |
| 5.7076     | 15.7150         | 171   |
| 5.7149     | 15.8730         | 172   |
| 5.8877     | 15.2373         | 173   |
| 5.6803     | 16.1623         | 174   |
| 5.7420     | 15.9171         | 175   |
| 5.6912     | 15.5799         | 176   |
| 5.7350     | 16.0120         | 177   |
| 5.6631     | 15.9157         | 178   |
| 5.7305     | 16.1250         | 179   |
| 5.7077     | 15.8018         | 180   |
| 5.6688     | 16.1011         | 181   |
| 5.7675     | 15.6628         | 182   |
| 5.6747     | 15.6886         | 183   |
| 5.7921     | 15.6053         | 184   |
| 5.6793     | 15.5329         | 185   |
| 5.6993     | 15.4673         | 186   |
| 5.8451     | 15.6634         | 187   |
| 5.7389     | 15.9733         | 188   |
| 5.7486     | 15.8548         | 189   |
| 5.7089     | 16.1267         | 190   |
| 5.8106     | 15.4471         | 191   |
| 5.7402     | 15.8568         | 192   |
| 5.6393     | 15.9586         | 193   |
| 5.7403     | 15.2678         | 194   |
| 5.7854     | 15.5638         | 195   |
| 5.5414     | 16.1871         | 196   |
| 5.7082     | 15.9706         | 197   |
| 5.6636     | 16.2550         | 198   |
| 5.6875     | 15.9385         | 199   |
| 5.7139     | 15.6730         | 200   |
| 5.6601     | 15.4174         | 201   |
| 5.6422     | 16.1655         | 202   |
| 5.7642     | 16.3103         | 203   |
| 5.7039     | 16.4020         | 204   |
| 5.7237     | 15.8775         | 205   |
| 5.7529     | 15.7237         | 206   |
| 5.6827     | 16.1514         | 207   |
| 5.7591     | 16.0905         | 208   |
| 5.7899     | 15.6417         | 209   |
| 5.7775     | 16.3878         | 210   |
| 5.6634     | 15.9944         | 211   |
| 5.5958     | 16.1042         | 212   |
| 5.8629     | 16.6206         | 213   |
| 5.7548     | 16.3826         | 214   |
| 5.7512     | 16.2234         | 215   |
| 5.6905     | 16.5029         | 216   |
| 5.6434     | 16.8345         | 217   |
| 5.6728     | 15.8749         | 218   |
| 5.7253     | 16.1679         | 219   |
| 5.6529     | 15.9138         | 220   |
| 5.6542     | 16.4299         | 221   |
| 5.6646     | 15.9442         | 222   |
| 5.7054     | 16.3624         | 223   |
| 5.7083     | 16.1256         | 224   |
| 5.8134     | 15.8207         | 225   |
| 5.7805     | 16.2750         | 226   |
| 5.7037     | 15.9758         | 227   |
| 5.7653     | 16.2336         | 228   |
| 5.7890     | 16.4635         | 229   |
| 5.7060     | 16.2425         | 230   |
| 5.7508     | 16.2569         | 231   |
| 5.6349     | 16.4228         | 232   |
| 5.7062     | 16.5237         | 233   |
| 5.7277     | 16.4191         | 234   |
| 5.7827     | 16.0735         | 235   |
| 5.7090     | 16.3830         | 236   |
| 5.6960     | 16.3506         | 237   |
| 5.7367     | 15.9862         | 238   |
| 5.7863     | 16.2742         | 239   |
| 5.5916     | 16.3640         | 240   |
| 5.6753     | 16.7890         | 241   |
| 5.6915     | 16.5041         | 242   |
| 5.7292     | 16.4998         | 243   |
| 5.7814     | 16.1040         | 244   |
| 5.6399     | 16.4167         | 245   |
| 5.6281     | 16.1772         | 246   |
| 5.7067     | 16.5245         | 247   |
| 5.7268     | 16.3465         | 248   |
| 5.7664     | 16.5136         | 249   |
| 5.7020     | 16.1559         | 250   |
| 5.6693     | 16.8744         | 251   |
| 5.6625     | 15.9549         | 252   |
| 5.6282     | 16.4120         | 253   |
| 5.6190     | 15.9476         | 254   |
| 5.6562     | 16.2114         | 255   |
| 5.6690     | 16.2859         | 256   |
| 5.7533     | 16.3209         | 257   |
| 5.7191     | 16.3224         | 258   |
| 5.8181     | 16.1149         | 259   |
| 5.6598     | 16.2559         | 260   |
| 5.6762     | 16.5949         | 261   |
| 5.6452     | 16.2653         | 262   |
| 5.6691     | 16.2993         | 263   |
| 5.7951     | 16.0316         | 264   |
| 5.8137     | 16.3896         | 265   |
| 5.7124     | 16.3996         | 266   |
| 5.7853     | 16.6237         | 267   |
| 5.7931     | 15.6052         | 268   |
| 5.7788     | 16.5983         | 269   |
| 5.7472     | 16.0878         | 270   |
| 5.6607     | 16.6207         | 271   |
| 5.8085     | 16.5659         | 272   |
| 5.7699     | 16.1165         | 273   |
| 5.6865     | 16.3090         | 274   |
| 5.7237     | 16.1727         | 275   |
| 5.8241     | 16.1545         | 276   |
| 5.6519     | 16.5434         | 277   |
| 5.6718     | 16.4884         | 278   |
| 5.6988     | 16.4953         | 279   |
| 5.7020     | 16.8616         | 280   |
| 5.7338     | 16.3847         | 281   |
| 5.6695     | 16.4040         | 282   |
| 5.6916     | 16.3199         | 283   |
| 5.7519     | 15.6585         | 284   |
| 5.7317     | 16.4947         | 285   |
| 5.8143     | 15.9633         | 286   |
| 5.6979     | 16.5859         | 287   |
| 5.7405     | 16.5161         | 288   |
| 5.7338     | 16.4144         | 289   |
| 5.5844     | 16.5315         | 290   |
| 5.6871     | 16.4282         | 291   |
| 5.8713     | 15.5593         | 292   |
| 5.6710     | 15.8436         | 293   |
| 5.7074     | 16.4072         | 294   |
| 5.6212     | 16.4969         | 295   |
| 5.7022     | 16.3911         | 296   |
| 5.6552     | 16.8670         | 297   |
| 5.7888     | 16.2774         | 298   |
| 5.8273     | 15.6623         | 299   |


### Framework versions

- Transformers 4.27.0.dev0
- TensorFlow 2.9.2
- Datasets 2.9.0
- Tokenizers 0.13.2

{"tags": ["generated_from_keras_callback"], "model-index": [{"name": "pretrained-m-bert-300", "results": []}]}
transformers tf bert pretraining generated_from_keras_callback endpoints_compatible region:us
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 

pretrained-m-bert-300
=====================


This model is a fine-tuned version of [](URL on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 5.8273
* Validation Loss: 15.6623
* Epoch: 299


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 1e-04, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.27.0.dev0
* TensorFlow 2.9.2
* Datasets 2.9.0
* Tokenizers 0.13.2



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 1e-04, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.27.0.dev0
* TensorFlow 2.9.2
* Datasets 2.9.0
* Tokenizers 0.13.2
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 1e-04, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.27.0.dev0
* TensorFlow 2.9.2
* Datasets 2.9.0
* Tokenizers 0.13.2
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 1e-04, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32### Training results### Framework versions


* Transformers 4.27.0.dev0
* TensorFlow 2.9.2
* Datasets 2.9.0
* Tokenizers 0.13.2