

TITLE
Is there a `DataCollator` cat mask n-gram words for LM?

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 📚 Migration

## Information

I want a mask n-gram words when pre-train bert model? but I can't find  a DataCollator from the lib
https://github.com/huggingface/transformers/blob/master/src/transformers/data/data_collator.py

I want to build it by myself, but I don't know how to build my own DataCollator, who can give me a demo?

## Checklist

- [ ] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
  [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [ ] I checked if a related official extension example runs on my machine.
