Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
File size: 1,504 Bytes
4bd209d 7576dfc 1cbe8ed |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
##
<pre>
from accelerate import Accelerator
-accelerator = Accelerator()
+accelerator = Accelerator(log_with="wandb")
train_dataloader, model, optimizer scheduler = accelerator.prepare(
dataloader, model, optimizer, scheduler
)
+accelerator.init_trackers()
model.train()
for batch in train_dataloader:
optimizer.zero_grad()
inputs, targets = batch
outputs = model(inputs)
loss = loss_function(outputs, targets)
+ accelerator.log({"loss":loss})
accelerator.backward(loss)
optimizer.step()
scheduler.step()
+accelerator.end_training()
</pre>
##
To use experiment trackers with `accelerate`, simply pass the desired tracker to the `log_with` parameter
when building the `Accelerator` object. Then initialize the tracker(s) by running `Accelerator.init_trackers()`
passing in any configurations they may need. Afterwards call `Accelerator.log` to log a particular value to your tracker.
At the end of training call `accelerator.end_training()` to call any finalization functions a tracking library
may need automatically.
##
To learn more checkout the related documentation:
- <a href="https://huggingface.co/docs/accelerate/usage_guides/tracking" target="_blank">Using experiment trackers</a>
- <a href="https://huggingface.co/docs/accelerate/package_reference/accelerator#accelerate.Accelerator.log" target="_blank">Accelerator API Reference</a>
- <a href="https://huggingface.co/docs/accelerate/package_reference/tracking" target="_blank">Tracking API Reference</a>
|