lightning/tests
Danielle Pintz 6329be60be
Replace PostLocalSGDOptimizer with a dedicated model averaging component (#12378)
2022-03-24 17:33:19 -07:00
..
accelerators Enable tpu device ids test (#12428) 2022-03-25 09:19:08 +09:00
benchmarks add parity test for sync batchnorm (#12021) 2022-02-26 03:51:57 +00:00
callbacks [3/3] Update lightning callbacks to `Stateful`, deprecations for old `on_save/load_checkpoint` signatures (#11887) 2022-03-25 00:06:10 +00:00
checkpointing [3/3] Update lightning callbacks to `Stateful`, deprecations for old `on_save/load_checkpoint` signatures (#11887) 2022-03-25 00:06:10 +00:00
core Do not mark LightningModule methods as abstract (#12381) 2022-03-23 08:55:12 +00:00
deprecated_api [3/3] Update lightning callbacks to `Stateful`, deprecations for old `on_save/load_checkpoint` signatures (#11887) 2022-03-25 00:06:10 +00:00
helpers unify logger testing (#9081) 2022-03-11 14:24:30 +00:00
lite Return the output of the optimizer step (#11711) 2022-02-09 09:37:13 +00:00
loggers `ModelCheckpoint`'s `save_last` now ignores `every_n_epochs` (#12418) 2022-03-24 20:06:52 +01:00
loops Have the outputs match the loops format (#12182) 2022-03-08 18:10:18 +00:00
models [3/3] Update lightning callbacks to `Stateful`, deprecations for old `on_save/load_checkpoint` signatures (#11887) 2022-03-25 00:06:10 +00:00
overrides Fix retrieval of batch indices when dataloader num_workers > 0 (#10870) 2021-12-02 10:36:10 +00:00
plugins Refactor `TorchElasticEnvironment.detect` to use `torch.distributed.is_torchelastic_launched` (#12376) 2022-03-21 16:51:24 +01:00
profiler Deprecate `LoggerCollection` in favor of `trainer.loggers` (#12147) 2022-03-04 23:01:43 +00:00
strategies Replace PostLocalSGDOptimizer with a dedicated model averaging component (#12378) 2022-03-24 17:33:19 -07:00
trainer [3/3] Update lightning callbacks to `Stateful`, deprecations for old `on_save/load_checkpoint` signatures (#11887) 2022-03-25 00:06:10 +00:00
tuner Disable tuner with distributed strategies (#12179) 2022-03-07 08:45:07 +00:00
utilities Remove Accelerator.parallel_device_ids and deprecate Trainer.data_parallel_device_ids (#12072) 2022-03-23 22:18:30 +00:00
README.md CI: add mdformat (#8673) 2021-08-03 18:19:09 +00:00
__init__.py Replace `yapf` with `black` (#7783) 2021-07-26 13:37:35 +02:00
conftest.py unify logger testing (#9081) 2022-03-11 14:24:30 +00:00
standalone_tests.sh Fix selection of standalone tests (#10857) 2021-12-01 09:48:37 +01:00

README.md

PyTorch-Lightning Tests

Most PL tests train a full MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). This provides testing for most combinations of important settings. The tests expect the model to perform to a reasonable degree of testing accuracy to pass.

Running tests

git clone https://github.com/PyTorchLightning/pytorch-lightning
cd pytorch-lightning

# install dev deps
pip install -r requirements/devel.txt

# run tests
py.test -v

To test models that require GPU make sure to run the above command on a GPU machine. The GPU machine must have at least 2 GPUs to run distributed tests.

Note that this setup will not run tests that require specific packages installed such as Horovod, FairScale, NVIDIA/apex, NVIDIA/DALI, etc. You can rely on our CI to make sure all these tests pass.

Running Coverage

Make sure to run coverage on a GPU machine with at least 2 GPUs and NVIDIA apex installed.

cd pytorch-lightning

# generate coverage (coverage is also installed as part of dev dependencies under requirements/devel.txt)
coverage run --source pytorch_lightning -m py.test pytorch_lightning tests examples -v

# print coverage stats
coverage report -m

# exporting results
coverage xml

Building test image

You can build it on your own, note it takes lots of time, be prepared.

git clone <git-repository>
docker image build -t pytorch_lightning:devel-torch1.9 -f dockers/cuda-extras/Dockerfile --build-arg TORCH_VERSION=1.9 .

To build other versions, select different Dockerfile.

docker image list
docker run --rm -it pytorch_lightning:devel-torch1.9 bash
docker image rm pytorch_lightning:devel-torch1.9