Document optional steps for converting Fabric code (#19486)

This commit is contained in:
awaelchli 2024-02-18 00:37:35 +01:00 committed by GitHub
parent bb35e8e0d3
commit c5ab34876b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 15 additions and 0 deletions

View File

@ -90,6 +90,21 @@ Check out our before-and-after example for `image classification <https://github
----
****************
Optional changes
****************
Here are a few optional upgrades you can make to your code, if applicable:
- Replace ``torch.save()`` and ``torch.load()`` with Fabric's :doc:`save and load methods <../guide/checkpoint/checkpoint>`.
- Replace collective operations from ``torch.distributed`` (barrier, broadcast, etc.) with Fabric's :doc:`collective methods <../advanced/distributed_communication>`.
- Use Fabric's :doc:`no_backward_sync() context manager <../advanced/gradient_accumulation>` if you implemented gradient accumulation.
- Initialize your model under the :doc:`init_module() <../advanced/model_init>` context manager.
----
**********
Next steps
**********