

TITLE
[deepspeed doc] install issues + 1-gpu deployment

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
This PR extends the DeepSpeed/FairScale integration documentation to:

* add extensive general troubleshooting for CUDA-extensions (applies to fairscale, deepspeed, apex or any other python pytorch extension with CUDA C++ code) - these are very likely to be encountered by our users - all notes are based on my first hand encounters with these issues - 2 of which I run into yesterday while trying to build fairscale and deepspeed on Sylvain's hardware which he let me use to run the recent benchmarks. so I figured others are likely to have similar issues and neither fairscale nor deepspeed have these documented anywhere.
* adds deployment for 1 gpu DeepSpeed notes 
* reformats sub-headers so that it's easier to link to specific sections

@sgugger 