GPT2 Ukrainian
A generative language model for the Ukrainian language follows the GPT-2 architecture (124M parameters).
- hidden size: 768
- number of heads: 12
- number of layers: 12
- seq length: 1024
- tokens: 11238113280 (3 epochs)
- steps: 57167
Training data
- OSCAR
- Wikimedia dumps
License
MIT