

TITLE
Converting pretrained tf2 bert model to pytorch model for using FillMaskPipeline

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Sincerely greeting to every NLP researchers and HuggingFace members, 
I am currently working on masked word prediction in a sentence like what FillMaskPipeline did. 
(i.e. This is a  [MASK] test.  Where the result of [MASK] might gives 
{'sequence': '[CLS]This is a [MASK] test.[SEP]', 'score': floating point, 'token': integer, 'token_str': string } 
as one of the candidate output) 

In my past few works I downloaded bert-base-chinese as my base model, which can be found in [original bert](https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip), then keep running the pretraining steps on my custom dataset using run_pretraining [script](https://github.com/google-research/bert/blob/master/run_pretraining.py), finally using [transformers-cli](https://huggingface.co/transformers/converting_tensorflow_models.html) to convert my bert_model.ckpt to pytorch_model.bin and apply FillMaskPipeline to do masked word prediction.
Everything work fine when I use the script under tensorflow-gpu==1.13.1 and CUDA 10.0 with GTX 1080 Ti. However, in current task I want to switch from 1080ti to RTX3090 to have larger batch size but in [Access to checkpoint](https://github.com/tensorflow/models/tree/master/official/nlp/bert#access-to-pretrained-checkpoints) the bert-base-chinese model haven't release yet, so I first convert the tf1 bert-base-chinese model to tf2 by using this [script](https://github.com/tensorflow/models/blob/master/official/nlp/bert/tf2_encoder_checkpoint_converter.py) with following args,

!python tf2_encoder_checkpoint_converter.py --checkpoint_to_convert=$BASE_DIR/bert_model.ckpt --converted_checkpoint_path=tmp/ --bert_config_file=$BASE_DIR/bert_config.json

after model being converted I use tf2 [create_pretraining_data.py](https://github.com/tensorflow/models/blob/master/official/nlp/data/create_pretraining_data.py) on my dataset and run [run_pretraining.py](https://github.com/tensorflow/models/blob/master/official/nlp/bert/run_pretraining.py). The problem I encounter now is after finished training I have no idea how to do masked prediction task on my data now.
I tried [converting ckpt script](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_original_tf2_checkpoint_to_pytorch.py) but notice the BertSelfAttention layer can not be convert due to MLM heads, also I didn't found any related script doing similar work as FillMaskPipeline in tensorflow, my questions are,

- Is it possible to convert pretrained tf2 model to pytorch without any addition modification on convert_bert_original_tf2_checkpoint_to_pytorch.py (I have no idea how to move forward and I also try to ignore the layer that can not convert but it raise the error as following png)
![image](https://user-images.githubusercontent.com/56808566/106563015-206cc180-6566-11eb-9356-000d9281df64.png)

- Is there any related script in tensorflow 1 and tensorflow 2 I can refer to, I spent some time finding related posts but with no luck. I merely understand some tensorflow so if anyone can give me direction on doing masked prediction would be much more appreciated.

### System Info
My environments are: (using pipreqs)
Python version: 3.7.5
CUDA used to build PyTorch: cuda_11.0_bu.TC445_37.28845127_0
OS: Ubuntu 18.04.5 LTS
GCC version: 7.5.0

### Versions of relevant libraries: (using pipreqs)
numpy==1.19.5
transformers==4.2.2
six==1.15.0
torch==1.7.1+cu110
tensorflow_gpu==2.2.0
gin_config==0.1.1
absl_py==0.11.0
tensorflow_hub==0.11.0
sentencepiece==0.1.94
absl==0.0
torchvision==0.8.2+cu110
bert4keras==0.9.9
gin==0.1.006
tensorflow==2.4.1

Any suggestions, helps, and relative post that I can dig into would be appreciated. Thanks in advance!