
# rugpt3small\_based\_on\_gpt2 safetensors variant
Model was trained with sequence length 1024 using transformers by [SberDevices](https://sberdevices.ru/) team on 80B tokens around 3 epoch. After that model was finetuned on 2048 context.

Total training time took around one week on 32 GPUs.

# Authors
+ NLP core team RnD [Telegram channel](https://t.me/nlpcoreteam):
  + Dmitry Zmitrovich
+ Safetensors variant by [Sashkanik13](https://huggingface.co/Sashkanik13)

{"language": ["ru"], "tags": ["PyTorch", "Transformers"], "thumbnail": "https://github.com/sberbank-ai/ru-gpts"}
transformers pytorch jax safetensors gpt2 text-generation PyTorch Transformers ru autotrain_compatible endpoints_compatible text-generation-inference region:us