This KenLM model is trained on https://huggingface.co/datasets/indonesian-nlp/id_newspapers_2018 dataset.
This model is 4-gram and it was pruned.
Used command:
../kenlm/build/bin/lmplz -T tmp -o 4 --prune 0 1 1 < "texts.txt" > "4gram.arpa"
This KenLM model is trained on https://huggingface.co/datasets/indonesian-nlp/id_newspapers_2018 dataset.
This model is 4-gram and it was pruned.
Used command:
../kenlm/build/bin/lmplz -T tmp -o 4 --prune 0 1 1 < "texts.txt" > "4gram.arpa"