text: 
  
  TITLE
  Advice needed: Adding more FSMT models
  
  COMMENTS
  1
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🌟 New model addition
  
  ## Model description
  
  I am planning to contribute a series of FSMT models to the model hub. The models have been trained for a paper that is currently under review.
  
  Before working on a PR I wanted to ask for some advice:
  
  ### normalize_before
  The new models have been trained with Fairseq's option `normalize_before=True`, while the existing FSMT implementation uses `normalize_before=False`. I understand that copy-pasting model code is preferred to extending the configuration. This would mean that a near-duplicate module `fsmt_prenorm` needs to be created. Is this correct?
  
  ### Adequate base branch
  The FSMT module is currently being refactored (https://github.com/huggingface/transformers/pull/11218). Do you recommend that I start from the master branch or from the PR's feature branch, which is nearly completed?