

TITLE
Support on Mixture of expert models

COMMENTS
8

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Hi, I find that there are emerging works in the field of NLP on **Mixture of experts** based models, such as Switch Transformers from Google. However, I do not find such mixture of expert models in huggingface transformers. Do you have the plan to support such models? Thanks !