Pretrained toy model. Made with Andrej Karpathy's NanoGPT, ~2023. Trained on part of Tagalog portion of Belebele.

Parameters:
* batch_size = 64
* block_size = 256
* n_layer = 8
* n_head = 8
* n_embd = 768

Everything else is left as is.
{"license": "cc-by-sa-4.0", "datasets": ["facebook/belebele"]}
dataset:facebook/belebele license:cc-by-sa-4.0 region:us