t5-sl-large

t5-sl-large model is a Slovene T5 model. It has 24 encoder and 24 decoder layers, in total about 750 million parameters. It was trained for 3 epochs on the following corpora:

Corpora

The following corpora were used for training the model:

Evaluation

The model is described in detail and evaluated in our paper "Sequence to sequence pretraining for a less-resourced Slovenian language"