• Continue pre-training RoBERTa-base using discharge summaries from MIMIC-III datasets.

• Details can be found in the following paper

Xiang Dai and Ilias Chalkidis and Sune Darkner and Desmond Elliott. 2022. Revisiting Transformer-based Models for Long Document Classification. (https://arxiv.org/abs/2204.06683)

• Important hyper-parameters