From 34f036917d83a0805c46c32cf4035a5dffbd5c76 Mon Sep 17 00:00:00 2001 From: Leng Yue Date: Mon, 4 Mar 2024 03:10:38 -0800 Subject: [PATCH] Document `ddp_find_unused_parameters_true` in Fabric (#19564) --- docs/source-fabric/api/fabric_args.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source-fabric/api/fabric_args.rst b/docs/source-fabric/api/fabric_args.rst index a126667259..4396aa26e9 100644 --- a/docs/source-fabric/api/fabric_args.rst +++ b/docs/source-fabric/api/fabric_args.rst @@ -36,13 +36,16 @@ See also: :doc:`../fundamentals/accelerators` strategy ======== -Choose a training strategy: ``"dp"``, ``"ddp"``, ``"ddp_spawn"``, ``"xla"``, ``"deepspeed"``, ``"fsdp"````. +Choose a training strategy: ``"dp"``, ``"ddp"``, ``"ddp_spawn"``, ``"ddp_find_unused_parameters_true"``, ``"xla"``, ``"deepspeed"``, ``"fsdp"``. .. code-block:: python # Running with the DistributedDataParallel strategy on 4 GPUs fabric = Fabric(strategy="ddp", accelerator="gpu", devices=4) + # Running with the DDP strategy with find unused parameters enabled on 4 GPUs + fabric = Fabric(strategy="ddp_find_unused_parameters_true", accelerator="gpu", devices=4) + # Running with the DDP Spawn strategy using 4 CPU processes fabric = Fabric(strategy="ddp_spawn", accelerator="cpu", devices=4)