
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# TGPT-2-345M

This model is a fine-tuned version of [](https://huggingface.co/) on an Thai Wikipedia dataset.
It achieves the following results on the evaluation set:
- Train Loss: 2.6217
- Train Accuracy: 0.5400
- Epoch: 3

## Model description

-

## Intended uses & limitations

-

## Training and evaluation data

-

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': 1.0, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 3e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train Accuracy | Epoch |
|:----------:|:--------------:|:-----:|
| 3.5955     | 0.4289         | 0     |
| 2.8630     | 0.5107         | 1     |
| 2.6217     | 0.5400         | 2     |


### Framework versions

- Transformers 4.26.0
- TensorFlow 2.11.0
- Datasets 2.10.0
- Tokenizers 0.13.2

{"language": ["th"], "license": "apache-2.0", "library_name": "transformers", "tags": ["TGPT"], "datasets": ["th_wikipedia"], "pipeline_tag": "text-generation", "widget": [{"text": "\u0e1b\u0e23\u0e30\u0e40\u0e17\u0e28\u0e44\u0e17\u0e22\u0e40\u0e1b\u0e47\u0e19\u0e1b\u0e23\u0e30\u0e40\u0e17\u0e28\u0e17\u0e35\u0e48"}, {"text": "\u0e01\u0e23\u0e38\u0e07\u0e40\u0e17\u0e1e\u0e40\u0e04\u0e22\u0e40\u0e01\u0e34\u0e14"}, {"text": "\u0e23\u0e30\u0e1a\u0e1a\u0e2a\u0e38\u0e23\u0e34\u0e22\u0e30\u0e08\u0e31\u0e01\u0e23\u0e27\u0e32\u0e25\u0e40\u0e1b\u0e47\u0e19"}], "model-index": [{"name": "TGPT-2-345M", "results": []}]}
transformers tf gpt2 text-generation TGPT th dataset:th_wikipedia license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #tf #gpt2 #text-generation #TGPT #th #dataset-th_wikipedia #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

TGPT-2-345M
===========


This model is a fine-tuned version of [](URL on an Thai Wikipedia dataset.
It achieves the following results on the evaluation set:


* Train Loss: 2.6217
* Train Accuracy: 0.5400
* Epoch: 3


Model description
-----------------


* 


Intended uses & limitations
---------------------------


* 


Training and evaluation data
----------------------------


* 


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': 1.0, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 3e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.26.0
* TensorFlow 2.11.0
* Datasets 2.10.0
* Tokenizers 0.13.2



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': 1.0, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 3e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.26.0
* TensorFlow 2.11.0
* Datasets 2.10.0
* Tokenizers 0.13.2
TAGS
#transformers #tf #gpt2 #text-generation #TGPT #th #dataset-th_wikipedia #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': 1.0, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 3e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.26.0
* TensorFlow 2.11.0
* Datasets 2.10.0
* Tokenizers 0.13.2
TAGS
#transformers #tf #gpt2 #text-generation #TGPT #th #dataset-th_wikipedia #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': 1.0, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 3e-05, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32### Training results### Framework versions


* Transformers 4.26.0
* TensorFlow 2.11.0
* Datasets 2.10.0
* Tokenizers 0.13.2