

TITLE
Fix no_trainer examples to properly calculate the number of samples

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# Fix number of samples for `no_trainer` scripts

## What does this add?

This PR fixes all of the no_trainer scripts to properly use the right number of training steps after the length of the dataloader was changed with `accelerator.prepare`

## Why is it needed?

Currently in a multi-process setup, the progress bar still shows the old number of samples. As a result the old number of steps before breaking is set at the original amount, even though the length of the dataloaders changed. The progress bar reflects this too.

Simplified example:

If the dataloader starts with 128 batches, if 2 GPUs are used then each dataloader has 64 batches. As a result the progress bar should use `64`, and the break condition needs to also know there is only 64. Both currently use 128 still

## What parts of the API does this impact?

### User-facing:

All scripts have a recalculation of the max_train_steps after `accelerate.prepare`

## Basic Usage Example(s):

```python
    # Prepare everything with our `accelerator`.
    model, optimizer, train_dataloader, eval_dataloader, lr_scheduler = accelerator.prepare(
        model, optimizer, train_dataloader, eval_dataloader, lr_scheduler
    )

    # We need to recalculate our total training steps
    num_update_steps_per_epoch = math.ceil(len(train_dataloader) / args.gradient_accumulation_steps)
    args.max_train_steps = args.num_train_epochs * num_update_steps_per_epoch
```

## When would I use it, and when wouldn't I?

While this is always used, technically it is only needed when the number of nodes > 1. 