Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
lightning/tests
History
Jirka Borovec 53b0cb910b update isort config (#5335) 
* update isort config

* apply

(cherry picked from commit 724f1051f0)
 		2021-01-06 12:49:23 +01:00
..
backends Refactor: clean trainer device & distrib setters (#5297) 2021-01-04 17:10:13 +00:00
base add memory parity for PL vs Vanilla (#5170) 2021-01-06 11:40:01 +01:00
callbacks Disable pl optimizer temporarily to fix AMP issues (#5163) 2021-01-05 09:58:37 +01:00
checkpointing Fix saved filename in ModelCheckpoint if it already exists (#4861) 2021-01-05 09:57:37 +01:00
core drop deprecated TrainResult (#5323) 2021-01-04 09:54:21 +08:00
deprecated_api Refactor: clean trainer device & distrib setters (#5297) 2021-01-04 17:10:13 +00:00
loggers flake8 ++ 2021-01-05 09:58:37 +01:00
metrics Fix metric state reset (#5273) 2021-01-06 12:48:01 +01:00
models flake8 ++ 2021-01-05 09:58:37 +01:00
plugins Refactor: clean trainer device & distrib setters (#5297) 2021-01-04 17:10:13 +00:00
trainer Trainer.test should return only test metrics (#5214) 2021-01-06 11:55:21 +01:00
tuner refactor - check F841 (#5202) 2020-12-21 11:10:55 +05:30
utilities Unify names in Utils (#5199) 2020-12-22 00:23:33 +01:00
README.md Horovod: fixed early stopping and added metrics aggregation (#3775) 2020-11-05 12:52:02 -05:00
__init__.py flake8 ++ 2021-01-05 09:58:37 +01:00
collect_env_details.py add copyright to tests (#5143) 2021-01-05 09:57:37 +01:00
conftest.py update isort config (#5335) 2021-01-06 12:49:23 +01:00
special_tests.sh support number for logging with sync_dist=True (#5080) 2021-01-05 09:58:37 +01:00
test_profiler.py update isort config (#5335) 2021-01-06 12:49:23 +01:00

README.md

PyTorch-Lightning Tests

Most PL tests train a full MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). This provides testing for most combinations of important settings. The tests expect the model to perform to a reasonable degree of testing accuracy to pass.

Running tests

The automatic travis tests ONLY run CPU-based tests. Although these cover most of the use cases, run on a 2-GPU machine to validate the full test-suite.

To run all tests do the following:

Install Open MPI or another MPI implementation. Learn how to install Open MPI on this page.

git clone https://github.com/PyTorchLightning/pytorch-lightning
cd pytorch-lightning

# install AMP support
bash requirements/install_AMP.sh

# install dev deps
pip install -r requirements/devel.txt

# run tests
py.test -v

To test models that require GPU make sure to run the above command on a GPU machine. The GPU machine must have:

  1. At least 2 GPUs.
  2. NVIDIA-apex installed.
  3. Horovod with NCCL support: HOROVOD_GPU_OPERATIONS=NCCL pip install horovod

Running Coverage

Make sure to run coverage on a GPU machine with at least 2 GPUs and NVIDIA apex installed.

cd pytorch-lightning

# generate coverage (coverage is also installed as part of dev dependencies under requirements/devel.txt)
coverage run --source pytorch_lightning -m py.test pytorch_lightning tests examples -v

# print coverage stats
coverage report -m

# exporting results
coverage xml

Building test image

You can build it on your own, note it takes lots of time, be prepared.

git clone <git-repository>
docker image build -t pytorch_lightning:devel-torch1.4 -f dockers/cuda-extras/Dockerfile --build-arg TORCH_VERSION=1.4 .

To build other versions, select different Dockerfile.

docker image list
docker run --rm -it pytorch_lightning:devel-torch1.4 bash
docker image rm pytorch_lightning:devel-torch1.4