text: 
  
  TITLE
  [feature request] a tool to clone existing models to make new models with small changes
  
  COMMENTS
  10
  
  REACTIONS
  +1: 1
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🚀 Feature request
  
  So we have great templates for creating a new model.
  
  ### Can you think of a way to create full clones of existing models? 
  
  Practically for BigScience needs we will have to create something like GPTMeg which is 99.9% identical to GPT2 with 2-3 tiny changes. And then we will need another GPT2 variant that replaces Positional Embeddings with ALiBi. And there will be more variants.
  
  Using templates would be quite expensive, when always everything is really identical.
  
  So ideally a user will do:
  ```
  transformers-clone-model GPT2 GPTMeg
  ```
  and voila it'd replicate model's files, tests and docs.
  
  If all source files could be easily identified this perhaps could be done in a few perl one liners. Here is a very rough outline:
  
  1. find the pertinent source files grep -Irl GPT2 .
  2. rename files/dirs while copying s/gpt2/gpt_meg/
  3. rename internals to s/GPT2/GPTMeg/g
  
  The hard to automate part is the index files as they is only one of each
  
  I think I can work it out, but I'm afraid that the end result would be a set of Perl one-liners only Stas will know what to do with. So perhaps long term this is not a good solution.
  
  Here is the Issue where we need to implement this: https://github.com/bigscience-workshop/Megatron-DeepSpeed/issues/138
  and 2 more will be coming soon.
  
  @LysandreJik, @patrickvonplaten, @sgugger 