

TITLE
[Deepspeed] ZeRO-Infinity integration plus config revamp

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
This PR:
- [x] integrates ZeRO-Infinity 
- [x] revamps the configuration process, instead of the confusing to users sometimes-we-override-values, sometimes-we-don't - all values are now explicit unless they are set to `auto`, then and only then the Trainer will set them to the correct or recommended values.
- [x] massively revamps the way the configuration is done. now splitting the config parsing into 2 phases - one happening at the very end of `TrainingArguments` and then a weak ref global module var is created which can then be queried by various `transformers` components w/o needing to change any APIs. The global object cleanly goes away when `TrainingArguments` goes out of scope. Users no longer need to make any special calls - just need to ensure the  `TrainingArguments` object is created before `model.from_pretrained()` is called (like we do in all examples). Phase 2 happens during `train` where we get a few variables that weren't there during `TrainingArguments`, so the config gets completed here.
- [x] ds_config is now passed to `zero.Init` in `from_pretrained` under ZeRO-3 since it now needs several configuration values - this is in preparation for fp32 and other important features.
- [x] adds new tests for ZeRO-Inf and configuration.
- [x] adds a minor fix in  `get_regression_trainer`

If you're testing this PR please make sure you install deepspeed master branch:
```
git clone https://github.com/microsoft/DeepSpeed
cd DeepSpeed
pip install -e .
```

## Important changes

Please note a major change is that now only params that are set to `auto` will get automatically overriden/set to the correct/recommended values, everything else is left as is. This is to avoid the previously confusing behavior of never being quite sure what gets overridden and what not despite the logger telling what it did override. The new behavior is completely unambiguous.

See: examples
- [zero2](https://github.com/huggingface/transformers/blob/0f221d2cce751182c455295ef2c03a2c1bd3d66b/tests/deepspeed/ds_config_zero2.json)
- [zero3](https://github.com/huggingface/transformers/blob/0f221d2cce751182c455295ef2c03a2c1bd3d66b/tests/deepspeed/ds_config_zero3.json)


It's ready to release now. 0.3.15 just has a debug print that is loud, fixed in their master.

<!--
TODO:

The following is probably best saved for the next PR as it'd probably require waiting for deepspeed==0.3.16

- [ ] may be revamp the resume to avoid first loading the model weights. can do it in another PR.

PRs waiting to be integrated before this PR can be merged:
- [ ] zero.init() ds_config arg - not yet created
- [ ] new release is needed 0.3.16
-->

@sgugger 