diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 794fb36d45..dc9323c351 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -130,108 +130,11 @@ In case you adding new dependencies, make sure that they are compatible with the ### Documentation -We are using Sphinx with Napoleon extension. -Moreover, we set Google style to follow with type convention. - -- [Napoleon formatting with Google style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) -- [ReStructured Text (reST)](https://docs.pylonsproject.org/projects/docs-style-guide/) -- [Paragraph-level markup](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#paragraphs) - -See following short example of a sample function taking one position string and optional - -```python -from typing import Optional - - -def my_func(param_a: int, param_b: Optional[float] = None) -> str: - """Sample function. - - Args: - param_a: first parameter - param_b: second parameter - - Return: - sum of both numbers - - Example:: - - Sample doctest example... - >>> my_func(1, 2) - 3 - - Note: - If you want to add something. - """ - p = param_b if param_b else 0 - return str(param_a + p) -``` - -When updating the docs make sure to build them first locally and visually inspect the html files (in the browser) for -formatting errors. In certain cases, a missing blank line or a wrong indent can lead to a broken layout. -Run these commands - -```bash -git submodule update --init --recursive -pip install -r requirements/docs.txt -make clean -cd docs -make html -``` - -and open `docs/build/html/index.html` in your browser. - -Notes: - -- You need to have LaTeX installed for rendering math equations. You can for example install TeXLive by doing one of the following: - - on Ubuntu (Linux) run `apt-get install texlive` or otherwise follow the instructions on the TeXLive website - - use the [RTD docker image](https://hub.docker.com/r/readthedocs/build) -- with PL used class meta you need to use python 3.7 or higher - -When you send a PR the continuous integration will run tests and build the docs. You can access a preview of the html pages in the -_Artifacts_ tab in CircleCI when you click on the task named _ci/circleci: Build-Docs_ at the bottom of the PR page. +To learn about development of docs, check out the docs [README.md](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/README.md). ### Testing -**Local:** Testing your work locally will help you speed up the process since it allows you to focus on particular (failing) test-cases. -To setup a local development environment, install both local and test dependencies: - -```bash -python -m pip install ".[dev, examples]" -python -m pip install pre-commit -pre-commit install -``` - -Additionally, for testing backward compatibility with older versions of PyTorch Lightning, you also need to download all saved version-checkpoints from the public AWS storage. Run the following script to get all saved version-checkpoints: - -```bash -wget https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip -P legacy/ -unzip -o legacy/checkpoints.zip -d legacy/ -``` - -Note: These checkpoints are generated to set baselines for maintaining backward compatibility with legacy versions of PyTorch Lightning. Details of checkpoints for back-compatibility can be found [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/legacy/README.md). - -You can run the full test-case in your terminal via this make script: - -```bash -make test -``` - -Note: if your computer does not have multi-GPU nor TPU these tests are skipped. - -**GitHub Actions:** For convenience, you can also use your own GHActions building which will be triggered with each commit. -This is useful if you do not test against all required dependency versions. - -**Docker:** Another option is to utilize the [pytorch lightning cuda base docker image](https://hub.docker.com/repository/docker/pytorchlightning/pytorch_lightning/tags?page=1&name=cuda). You can then run: - -```bash -python -m pytest pytorch_lightning tests pl_examples -v -``` - -You can also run a single test as follows: - -```bash -python -m pytest -v tests/trainer/test_trainer_cli.py::test_default_args -``` +To learn about tests, check out the tests [README.md](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/tests/README.md). ### Pull Request diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000000..171d7b63ae --- /dev/null +++ b/docs/README.md @@ -0,0 +1,61 @@ +# PyTorch-Lightning Docs + +We are using Sphinx with Napoleon extension. +Moreover, we set Google style to follow with type convention. + +- [Napoleon formatting with Google style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) +- [ReStructured Text (reST)](https://docs.pylonsproject.org/projects/docs-style-guide/) +- [Paragraph-level markup](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#paragraphs) + +See following short example of a sample function taking one position string and optional + +```python +from typing import Optional + + +def my_func(param_a: int, param_b: Optional[float] = None) -> str: + """Sample function. + + Args: + param_a: first parameter + param_b: second parameter + + Return: + sum of both numbers + + Example:: + + >>> my_func(1, 2) + 3 + + Note: + If you want to add something. + """ + p = param_b if param_b else 0 + return str(param_a + p) +``` + +## Building Docs + +When updating the docs, make sure to build them first locally and visually inspect the html files in your browser for +formatting errors. In certain cases, a missing blank line or a wrong indent can lead to a broken layout. +Run these commands + +```bash +git submodule update --init --recursive +pip install -r requirements/docs.txt +make clean +cd docs +make html +``` + +and open `docs/build/html/index.html` in your browser. + +When you send a PR the continuous integration will run tests and build the docs. You can access a preview of the html pages in the +_Artifacts_ tab in CircleCI when you click on the task named _build-Docs_ of _ci-tests_ at the bottom of the PR page. + +Notes (Optional): + +- You need to have LaTeX installed for rendering math equations. You can for example install TeXLive by doing one of the following: + - on Ubuntu (Linux) run `apt-get install texlive` or otherwise follow the instructions on the TeXLive website + - use the [RTD docker image](https://hub.docker.com/r/readthedocs/build) diff --git a/tests/README.md b/tests/README.md index fc63d353eb..a6908b415b 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,22 +1,58 @@ # PyTorch-Lightning Tests -Most PL tests train a full MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). -This provides testing for most combinations of important settings. -The tests expect the model to perform to a reasonable degree of testing accuracy to pass. +Most of the tests in PyTorch Lightning train a [BoringModel](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/tests/helpers/boring_model.py) under various trainer conditions (ddp, ddp2+amp, etc...). Want to add a new test case and not sure how? [Talk to us!](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ) ## Running tests +**Local:** Testing your work locally will help you speed up the process since it allows you to focus on particular (failing) test-cases. +To setup a local development environment, install both local and test dependencies: + ```bash +# clone the repo git clone https://github.com/PyTorchLightning/pytorch-lightning cd pytorch-lightning -# install dev deps -pip install -r requirements/devel.txt - -# run tests -py.test -v +# install required depedencies +python -m pip install ".[dev, examples]" +# install pre-commit (optional) +python -m pip install pre-commit +pre-commit install ``` +Additionally, for testing backward compatibility with older versions of PyTorch Lightning, you also need to download all saved version-checkpoints from the public AWS storage. Run the following script to get all saved version-checkpoints: + +```bash +wget https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip -P legacy/ +unzip -o legacy/checkpoints.zip -d legacy/ +``` + +Note: These checkpoints are generated to set baselines for maintaining backward compatibility with legacy versions of PyTorch Lightning. Details of checkpoints for back-compatibility can be found [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/legacy/README.md). + +You can run the full test suite in your terminal via this make script: + +```bash +make test +``` + +Note: if your computer does not have multi-GPU or TPU these tests are skipped. + +**GitHub Actions:** For convenience, you can also use your own GHActions building which will be triggered with each commit. +This is useful if you do not test against all required dependency versions. + +**Docker:** Another option is to utilize the [pytorch lightning cuda base docker image](https://hub.docker.com/repository/docker/pytorchlightning/pytorch_lightning/tags?page=1&name=cuda). You can then run: + +```bash +python -m pytest pytorch_lightning tests pl_examples -v +``` + +You can also run a single test as follows: + +```bash +python -m pytest -v tests/trainer/test_trainer_cli.py::test_default_args +``` + +### Conditional Tests + To test models that require GPU make sure to run the above command on a GPU machine. The GPU machine must have at least 2 GPUs to run distributed tests. @@ -24,6 +60,16 @@ Note that this setup will not run tests that require specific packages installed such as Horovod, FairScale, NVIDIA/apex, NVIDIA/DALI, etc. You can rely on our CI to make sure all these tests pass. +### Standalone Tests + +There are certain standalone tests, which you can run using: + +```bash +PL_RUN_STANDALONE_TESTS=1 python -m pytest -v tests/trainer/ +# or +./tests/standalone_tests.sh tests/trainer +``` + ## Running Coverage Make sure to run coverage on a GPU machine with at least 2 GPUs and NVIDIA apex installed. @@ -32,7 +78,7 @@ Make sure to run coverage on a GPU machine with at least 2 GPUs and NVIDIA apex cd pytorch-lightning # generate coverage (coverage is also installed as part of dev dependencies under requirements/devel.txt) -coverage run --source pytorch_lightning -m py.test pytorch_lightning tests examples -v +coverage run --source pytorch_lightning -m pytest pytorch_lightning tests pl_examples -v # print coverage stats coverage report -m