Small-Bench NLP is a benchmark for small efficient neural language models trained on a single GPU.