Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[bugfix] Group defaults to WORLD if None (#5125) 

* [bugfix] Group defaults to WORLD if None

* fix no_grad

* Update pytorch_lightning/utilities/distributed.py

* Update pytorch_lightning/utilities/distributed.py

Co-authored-by: Gregor Koporec <gregork@unicorn.gorenje.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
(cherry picked from commit 176735097a)
This commit is contained in:
Gregor 2020-12-23 19:16:45 +01:00 committed by Jirka Borovec
parent 4349de8316
commit 0858beaf6b
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
pytorch_lightning/utilities/distributed.py
View File

@ -202,6 +202,7 @@ def all_gather_ddp_if_available(
Return:
A tensor of shape (world_size, batch, ...)
"""
group = group if group is not None else torch.distributed.group.WORLD
if torch.distributed.is_available() and torch.distributed.is_initialized():
if sync_grads:
return AllGatherGrad.apply(tensor, group)