

TITLE
[performance doc] add tables with DDP/DP/no-DP benchmarks

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Users ask of DDP vs. DP vs. single GPU vs w/ w/o Nvlink and this summary is insufficient 
https://huggingface.co/transformers/performance.html#nvlink
luckily we have great numbers posted - so should link to those from the guide:

1. table 1 where DDP wins by miles over any other setup: https://github.com/huggingface/transformers/issues/9371#issuecomment-768656711
2. table 2 where DDP is terrible https://github.com/huggingface/transformers/issues/9371#issuecomment-771302472

