text: 
  
  TITLE
  Improve PyTorch examples for FP16
  
  COMMENTS
  1
  
  REACTIONS
  +1: 4
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  To get the full speed-up of FP16 training, every tensor passed through the model should have all its dimensions be a multiple of 8. In the new PyTorch examples, when using dynamic padding, the tensors are padded to the length of the biggest sentence of the batch, but that number is not necessarily a multiple of 8.
  
  The examples should be improved to pass along the option `pad_to_multiple_of=8` when `fp16` is True, if using a data collator that applies padding (or replace the `None` passed along to `Trainer` for `data_collator` by a `DataCollatorWithPadding(tokenizer, pad_to_multiple_of=8)`).