

TITLE
[DeepSpeed] restore memory for evaluation

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
I spent some time trying to see if we could gain from DeepSpeed during inference - and while in the future there will be goodies to make it useful at the moment we don't need it, so let's make DeepSpeed cleanly contained to `train` for now.
 
This PR has a few small tweaks:
- frees up all the memory used by DeepSpeed at the end of training
- makes a clean way of not switching `model.to()` - only for when `--do_train` is used with deepspeed (so this is the case where you @sgugger were concerned about eval before train - no problem now)
- adds a warning if a user tries to use `--deepspeed` without `--do_train`
- re-works the test suite
- applies consistent json config formatting

@sgugger, @LysandreJik 