xtremedistil-l6-h384-go-emotion

作者 bergum · 下载 8902 · 点赞 6 · 更新 2023-10-31

xtremedistil-l6-h384-go-emotion

This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased on the go_emotions dataset.

See notebook for how the model was trained and converted to ONNX format Training Notebook

This model is deployed to aiserv.cloud for live demo of the model.

See https://github.com/jobergum/browser-ml-inference for how to reproduce.

Training hyperparameters

  Num examples = 211225
  Num Epochs = 4
  Instantaneous batch size per device = 128
  Total train batch size (w. parallel, distributed & accumulation) = 128
  Gradient Accumulation steps = 1
  Total optimization steps = 6604
 [6604/6604 53:23, Epoch 4/4]
Step	Training Loss
500	0.263200
1000	0.156900
1500	0.152500
2000	0.145400
2500	0.140500
3000	0.135900
3500	0.132800
4000	0.129400
4500	0.127200
5000	0.125700
5500	0.124400
6000	0.124100
6500	0.123400