

TITLE
Update no_trainer scripts with new Accelerate functionalities

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# Update the `no_trainer` scripts to keep aligned with Accelerate capabilities

## What does this add?

Updates all `no_trainer` scripts to use the latest capabilities.

## Why is it needed?

Accelerate had a number of new capabilities added, including better saving/loading, experiment tracking, and support for LR Schedulers. As a result, much of the current scripts can either be simplified from their hard-coded behaviors, or have these features added

Modified scripts with potential major changes:
- `language-modeling`
- `multiple-choice`
- `question-answering`
- `summarization`
- `text-classification`
- `token-classification`
- `translation`

The speech fine-tuning will be updated in a later PR

## Basic usage examples:

* Saving checkpoints each epoch or number of steps:
```bash
accelerate launch language-modeling/run_clm_no_trainer --checkpointing_steps "epoch"
```
```bash
accelerate launch language-modeling/run_clm_no_trainer --checkpointing_steps 100
```
* Resuming training from a saved checkpoint: 
```bash
accelerate launch language-modeling/run_clm_no_trainer --resume_from_checkpoint "epoch_1"
```
* Use any available trackers that Accelerate can automatically pick up including Weights and Biases, TensorBoard, and CometML
```bash
accelerate launch language_modeling/run_clm_no_trainer --with_tracking
```
## Anticipated maintence burden? (What will happen in say, 3 months if something changes)

As it gets more widly used, these scripts might need small updates if we find the end-users prefer a different experience when it comes to logging, or other small bugfixes we find as time goes on.