

TITLE
Converting fairseq NMT to transformers misses model weight

COMMENTS
8

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Hi there, question about fairseq NMT model ([FSMT](https://huggingface.co/transformers/model_doc/fsmt.html)) conversion.

I tried to convert my own fairseq-nmt model ([`transformer_wmt_en_de`](https://github.com/pytorch/fairseq/blob/master/fairseq/models/transformer.py#L1046)) based on [this conversion script](https://github.com/huggingface/transformers/blob/master/src/transformers/models/fsmt/convert_fsmt_original_pytorch_checkpoint_to_pytorch.py). 
However, `decoder.embed_out` weight is missing after converting fairseq model to transformers FSMT model. This parameter exists when not specifing  `--share-all-embeddings` or `--share-decoder-input-output-embed`, while official fairseq wmt models do not have `decoder.embed_out` weight because specifying `--share-all-embedding`.
https://github.com/pytorch/fairseq/issues/2537

Are there any solution or tips to converting own fairseq model?