This repository is created as part of the Flax/Jax community week by Huggingface. The aim of this project is to pre-train a language model using GPT-2 specifically for Tamil language.