text: 
  
  TITLE
  Pretrain BERT with user defined vocabulary
  
  COMMENTS
  3
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🚀 Feature request
  
  I'm wondering if there is a way to pretrain BERT with user-defined vocabulary, number of layers/heads, and use a WordLevel Tokenizer instead of a WordPiece Tokenizer current BERT uses. Thank you!
  
  ## Motivation
  
  This could help the BERT mode to adapt to different tasks.
  
  