Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Document `ddp_find_unused_parameters_true` in Fabric (#19564)

This commit is contained in:
Leng Yue 2024-03-04 03:10:38 -08:00 committed by GitHub
parent 48c39ce24f
commit 34f036917d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/source-fabric/api/fabric_args.rst
View File

@ -36,13 +36,16 @@ See also: :doc:`../fundamentals/accelerators`
strategy
========
Choose a training strategy: ``"dp"``, ``"ddp"``, ``"ddp_spawn"``, ``"xla"``, ``"deepspeed"``, ``"fsdp"````.
Choose a training strategy: ``"dp"``, ``"ddp"``, ``"ddp_spawn"``, ``"ddp_find_unused_parameters_true"``, ``"xla"``, ``"deepspeed"``, ``"fsdp"``.
.. code-block:: python
# Running with the DistributedDataParallel strategy on 4 GPUs
fabric = Fabric(strategy="ddp", accelerator="gpu", devices=4)
# Running with the DDP strategy with find unused parameters enabled on 4 GPUs
fabric = Fabric(strategy="ddp_find_unused_parameters_true", accelerator="gpu", devices=4)
# Running with the DDP Spawn strategy using 4 CPU processes
fabric = Fabric(strategy="ddp_spawn", accelerator="cpu", devices=4)