rugpt3small_based_on_gpt2 safetensors variant
Model was trained with sequence length 1024 using transformers by SberDevices team on 80B tokens around 3 epoch. After that model was finetuned on 2048 context.
Total training time took around one week on 32 GPUs.
Authors
- NLP core team RnD Telegram channel:
- Dmitry Zmitrovich
- Safetensors variant by Sashkanik13
- Downloads last month
- 5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.