tgf-sp-unigram-tokenizer-ptbr

作者 thegoodfellas · 下载 0 · 点赞 0 · 更新 2023-11-01

This tokenizer was created from the BrWac dataset (full version). It is been used by models originated from Flan-T5.