
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# skillsBERT_v3

This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.0728
- Validation Loss: 6.7366
- Epoch: 49

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamW', 'weight_decay': 0.004, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': False, 'is_legacy_optimizer': False, 'learning_rate': 5e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 6.8303     | 6.7512          | 0     |
| 6.2128     | 6.1686          | 1     |
| 5.3526     | 5.2872          | 2     |
| 4.5883     | 4.9001          | 3     |
| 4.0815     | 4.7446          | 4     |
| 3.6906     | 4.6986          | 5     |
| 3.3715     | 4.8142          | 6     |
| 3.0918     | 4.5540          | 7     |
| 2.8173     | 4.6815          | 8     |
| 2.5603     | 4.6359          | 9     |
| 2.3075     | 4.8909          | 10    |
| 2.0662     | 4.7510          | 11    |
| 1.8325     | 4.8515          | 12    |
| 1.6146     | 4.7591          | 13    |
| 1.4106     | 4.8230          | 14    |
| 1.2215     | 5.0016          | 15    |
| 1.0493     | 4.9657          | 16    |
| 0.8918     | 5.1592          | 17    |
| 0.7619     | 5.3179          | 18    |
| 0.6336     | 5.2910          | 19    |
| 0.5381     | 5.6262          | 20    |
| 0.4488     | 5.5304          | 21    |
| 0.3816     | 5.4275          | 22    |
| 0.3209     | 5.8982          | 23    |
| 0.2814     | 5.7482          | 24    |
| 0.2441     | 5.7706          | 25    |
| 0.2153     | 6.0133          | 26    |
| 0.1918     | 6.1146          | 27    |
| 0.1782     | 6.4018          | 28    |
| 0.1598     | 6.0096          | 29    |
| 0.1482     | 6.2542          | 30    |
| 0.1408     | 6.3236          | 31    |
| 0.1293     | 6.4049          | 32    |
| 0.1243     | 6.2547          | 33    |
| 0.1204     | 6.3997          | 34    |
| 0.1111     | 6.5252          | 35    |
| 0.1049     | 6.5424          | 36    |
| 0.1021     | 6.6899          | 37    |
| 0.1029     | 6.8212          | 38    |
| 0.0949     | 6.7073          | 39    |
| 0.0900     | 6.8162          | 40    |
| 0.0904     | 6.7465          | 41    |
| 0.0894     | 6.9141          | 42    |
| 0.0860     | 6.8801          | 43    |
| 0.0819     | 6.7101          | 44    |
| 0.0779     | 7.0270          | 45    |
| 0.0785     | 6.6004          | 46    |
| 0.0756     | 6.6370          | 47    |
| 0.0735     | 6.8723          | 48    |
| 0.0728     | 6.7366          | 49    |


### Framework versions

- Transformers 4.28.0.dev0
- TensorFlow 2.11.0
- Datasets 2.10.1
- Tokenizers 0.13.2

{"tags": ["generated_from_keras_callback"], "model-index": [{"name": "skillsBERT_v3", "results": []}]}
transformers tf bert pretraining generated_from_keras_callback endpoints_compatible region:us
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 

skillsBERT\_v3
==============


This model is a fine-tuned version of [](URL on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.0728
* Validation Loss: 6.7366
* Epoch: 49


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamW', 'weight\_decay': 0.004, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': False, 'is\_legacy\_optimizer': False, 'learning\_rate': 5e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.28.0.dev0
* TensorFlow 2.11.0
* Datasets 2.10.1
* Tokenizers 0.13.2



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamW', 'weight\_decay': 0.004, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': False, 'is\_legacy\_optimizer': False, 'learning\_rate': 5e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.0.dev0
* TensorFlow 2.11.0
* Datasets 2.10.1
* Tokenizers 0.13.2
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamW', 'weight\_decay': 0.004, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': False, 'is\_legacy\_optimizer': False, 'learning\_rate': 5e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.0.dev0
* TensorFlow 2.11.0
* Datasets 2.10.1
* Tokenizers 0.13.2
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamW', 'weight\_decay': 0.004, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': False, 'is\_legacy\_optimizer': False, 'learning\_rate': 5e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32### Training results### Framework versions


* Transformers 4.28.0.dev0
* TensorFlow 2.11.0
* Datasets 2.10.1
* Tokenizers 0.13.2