README.md

metadata

title: Accelerate Examples
emoji: 🌖
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 3.14.0
app_file: src/app.py
pinned: false

Accelerate Integration Examples

This is an interactive utility to show users how to incorporate parts of 🤗 Accelerate into their code. To use it select a feature to add and a github-like diff will be rendered showing what code to remove and add based on the initial template code.

These are more simplified versions of examples that exist in the accelerate and transformers repositories.

How each example is made

In the code_examples folder are basic text-like files which contain a much simplified version of some integration. For example:

<pre>
from accelerate import Accelerator
accelerator = Accelerator()
train_dataloader, model, optimizer scheduler = accelerator.prepare(
        dataloader, model, optimizer, scheduler
)

model.train()
for batch in train_dataloader:
&nbsp;&nbsp;&nbsp;&nbsp;optimizer.zero_grad()
    inputs, targets = batch
    outputs = model(inputs)
    loss = loss_function(outputs, targets)
    accelerator.backward(loss)
    optimizer.step()
    scheduler.step()
</pre>

These are done in an HTML-like syntax, with pre being used instead of three back-ticks for formatting purposes.

Creating a diff

To create a diff, a similar pre tag should be wrapped around the code, and a single + or - (showing an addition or subtraction) should be added to the code with no extra spacing or formatting. The tool will automatically know how to render these properly.

For example:

<pre>
+from accelerate import Accelerator
+accelerator = Accelerator()
+dataloader, model, optimizer scheduler = accelerator.prepare(
+        dataloader, model, optimizer, scheduler
+)

for batch in dataloader:
    optimizer.zero_grad()
    inputs, targets = batch
-    inputs = inputs.to(device)
-    targets = targets.to(device)
    outputs = model(inputs)
    loss = loss_function(outputs, targets)
-    loss.backward()
+    accelerator.backward(loss)
    optimizer.step()
    scheduler.step()</pre>

Also note that the initial pre is on a newline, and the latter </pre> is not.

Rendering the diff

After a diff and starter (if needed) has been made, if a new template was created add it to TEMPLATES in src/template.py. Otherwise in app.py modify the change function to properly point to the new integration example to show on a particular selection:

def change(inp):
    if inp == "Basic":
        return (templates["initial"], highlight(inp), "## Accelerate Code (Base Integration)")
    elif inp == "Calculating Metrics":
        return (templates["initial_with_metrics"], highlight(inp), f"## Accelerate Code ({inp})")
    else:
        return (templates["accelerate"], highlight(inp), f"## Accelerate Code ({inp})")