

TITLE
[wip] [deepspeed] AdamW is now supported by default

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
This PR syncs with changes in DeepSpeed since `deepspeed==0.3.10` and can only be merged when `deepspeed==0.3.11` or higher is released. So it may sit here for a while aggregating adjustments

* [x] AdamW is now supported by default so we can remove the now redundant config options and comments https://github.com/microsoft/DeepSpeed/pull/670
