microsoft/xtremedistil-l6-h256-uncased fined-tuned on SQuAD (https://huggingface.co/datasets/squad)

Hyperparameters:

• epochs: 1
• lr: 1e-5
• train batch sie: 16
• optimizer: adamW
• lr_scheduler: linear
• num warming steps: 0
• max_length: 512

Results on the dev set:

• 'exact_match': 62.66792809839168
• 'f1': 74.99490608582015