

TITLE
ReduceLROnPlateau-like functionality?

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

**Description:** Dynamic learning rate reduction upon metric saturation, as in `torch.optim.lr_scheduler.ReduceLROnPlateau`, integrated into the `Trainer` API.

Alternately, if there's any way (if hacky) to get dynamic learning rate reduction using the `Trainer` API as it is, that would be extremely helpful as well.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

LR schedules are a commonly used trick for ML optimization, and the `transformers` library already provides a significant number of baseline schedules (i.e. linear, cosine schedulers, warmup/no-warmup, restarts). However, these schedules are all static: updates to them occur at fixed steps in the optimization -- one can always tell what the learning rate at, say, step 1000 will be given these fixed schedules.

Reducing learning rate dynamically is also a common practical technique, usually applied when loss saturates (fails to improve after N iterations).

The difficulty is that dynamic learning rate reduction follows a non-fixed update schedule, meaning that working within the `LambdaLR` framework used by the other scheduler is less straightforward.

## Your contribution

I don't have a working implementation yet. At a high level, I tried to implement this myself as a `TrainerCallback` modeled on both the `EarlyStoppingCallback` in the `transformers` library as well as the `ReduceLROnPlateau` implementation in PyTorch. I was able to modify the optimizer object; however, learning rate updates to the optimizer would get overwritten by the scheduler. In any case, I also don't know if it's good style/even possible to modify the optimizer and scheduler in this way using a Callback -- seems like the `control` object is the only thing that changes within a `TrainerCallback`.

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->
