Norwegian Electra

Trained on Oscar + wikipedia + opensubtitles + some other data I had with the awesome power of TPUs(V3-8)

Use with caution. I have no downstream tasks in Norwegian to test on so I have no idea of its performance yet.

Model

Electra: Pre-training Text Encoders as Discriminators Rather Than Generators

Kevin Clark and Minh-Thang Luong and Quoc V. Le and Christopher D. Manning

• https://openreview.net/pdf?id=r1xMH1BtvB
• https://github.com/google-research/electra

Acknowledgments

TensorFlow Research Cloud

Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). Thanks for providing access to the TFRC ❤️

• https://www.tensorflow.org/tfrc

OSCAR corpus

• https://oscar-corpus.com/

OPUS

• http://opus.nlpl.eu/
• http://www.opensubtitles.org/