Model description
We pretrained a RoBERTa-based Japanese masked language model on paper abstracts from the academic database CiNii Articles.
A Japanese Masked Language Model for Academic Domain
Vocabulary
The vocabulary consists of 32000 tokens including subwords induced by the unigram language model of sentencepiece.
