Model from ACL 2023 paper Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization.

Our Socratic model continue-pretrained over 30M instances from the Book3 corpus.