30B (act-order true-sequential groupsize)
wikitext2 4.100694179534912 (stock 16bit)
wikitext2 4.179347991943359 (32g)
wikitext2 4.222894191741943 (128g)
ptb-new 8.13940715789795 (stock 16bit)
ptb-new 8.201859474182129 (32g)
ptb-new 8.227158546447754 (128g)
c4-new 6.129664421081543 (stock 16bit)
c4-new 6.190909385681152 (32g)
c4-new 6.230474948883057 (128g)