Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
lightning/pl_examples/multi_node_examples
History
Jirka Borovec f18aee30a5 Minor imports cleaning (#402) 
* code cleaning

* drop unused imports

* optimize imports
 		2019-10-22 11:32:40 +03:00
..
README.md changes examples to pl_examples for name connflict 2019-10-19 00:41:17 +02:00
__init__.py changes examples to pl_examples for name connflict 2019-10-19 00:41:17 +02:00
ddp2_job_submit.sh changes examples to pl_examples for name connflict 2019-10-19 00:41:17 +02:00
ddp_job_submit.sh changes examples to pl_examples for name connflict 2019-10-19 00:41:17 +02:00
multi_node_ddp2_demo.py Minor imports cleaning (#402) 2019-10-22 11:32:40 +03:00
multi_node_ddp_demo.py Minor imports cleaning (#402) 2019-10-22 11:32:40 +03:00

README.md

Multi-node example

This demo launches a job using 2 GPUs on 2 different nodes (4 GPUs total). To run this demo do the following:

  1. Log into the jumphost node of your SLURM-managed cluster.
  2. Create a conda environment with Lightning and a GPU PyTorch version.
  3. Choose a script to submit

DDP

Submit this job to run with distributedDataParallel (2 nodes, 2 gpus each)

sbatch ddp_job_submit.sh YourEnv

DDP2

Submit this job to run with a different implementation of distributedDataParallel. In this version, each node acts like DataParallel but syncs across nodes like DDP.

sbatch ddp2_job_submit.sh YourEnv