As a proof of concept, trying to improve accuracy went from 2 epochs to 100 epochs on the 10k Covid19 questions in the CDC corpus I collected.
Details and code available here. https://github.com/jasondeden/capstone
Model was having issues loading, but those appear to have been back-end and are now fixed, or else I was doing something wrong that I somehow missed.