xtremedistil-l6-h384-go-emotion
This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased on the go_emotions dataset.
See notebook for how the model was trained and converted to ONNX format 
This model is deployed to aiserv.cloud for live demo of the model.
See https://github.com/jobergum/browser-ml-inference for how to reproduce.
Training hyperparameters
- batch size 128
- learning_rate=3e-05
- epocs 4
Num examples = 211225 Num Epochs = 4 Instantaneous batch size per device = 128 Total train batch size (w. parallel, distributed & accumulation) = 128 Gradient Accumulation steps = 1 Total optimization steps = 6604 [6604/6604 53:23, Epoch 4/4] Step Training Loss 500 0.263200 1000 0.156900 1500 0.152500 2000 0.145400 2500 0.140500 3000 0.135900 3500 0.132800 4000 0.129400 4500 0.127200 5000 0.125700 5500 0.124400 6000 0.124100 6500 0.123400
- Downloads last month
- 609
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.