Commit Graph

70 Commits

Author SHA1 Message Date
Jirka Borovec 8cd75a4dd5
fix comparing versions (#6434)
* fix comparing versions

* chlog

* .

* ...

* datasets
2021-03-23 07:51:45 +00:00
Jirka Borovec 1fae10a2dc
refactoring setup (#6590)
* refactoring setup

* .

* docs

* flake8
2021-03-22 08:39:19 -04:00
Carlos Mocholí efd272a3ca
Pass {fit,validate,test,predict} to setup() and teardown() (#6386) 2021-03-08 15:27:07 +01:00
Rohit Gupta bcc0004955
Add before_batch_transfer and after_batch_transfer hooks (#3671)
* add hooks

* comment

* docs

* add tests

* make it private

* fix tests

* docs

* chlog

* testcode

* codefactor

* fix doctest

* fix doctest

* suggestions

* is always overriden

* pep and BoringModel

* BoringModel

* docs

* docs

* docs

* fix

* rebase

* rebase

* suggestions

* docs

* suggestions

* try fix docs

* docs

* update name

* yapf

* docs

* rebase

* yapf
2021-02-18 06:58:12 -05:00
tchaton 77be6f6e24 resolve conflits
resolve doc

boring commit

docs

torchvision

tpu

Update dockers/tpu-tests/tpu_test_cases.jsonnet

Update dockers/tpu-tests/tpu_test_cases.jsonnet
2021-02-05 21:43:10 +01:00
Alexander Reshytko 6d7c01b1b4 [docs] Add docs for non-SLURM cluster setup (#5754)
* Add docs for non-slurm cluster setup

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update docs/source/cluster.rst

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update docs/source/cluster.rst

Co-authored-by: Alexander <alexander@reshytko.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-02-05 21:43:10 +01:00
siahuat0727 b2b27e4018 Fix docs typo (#4930)
* Fix docs

* typo

* import

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-05 21:40:40 +01:00
Jirka Borovec d5c107fa96
Docs: move images (#5756)
* move images

* logo
2021-02-03 15:08:19 +00:00
Jirka Borovec 99ea2a3b35
define Yapf config (#5591)
* define YAPF

* add check

* add check

* add temp ignore

* apply yapf

* ex
2021-01-27 21:58:33 -05:00
Jirka Borovec dee5553b2b
move to Pages dir (#4869)
* folders

* common / advanced / extensions

* paths

* flake8

* isort

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-26 15:07:07 -05:00
Jirka Borovec f782230412
docs cleaning - testcode (#5595)
* testcode - python

* revert

* simple

* testcode @rst

* pl

* fix

* pip

* update

* conf

* conf

* nn.

* typo
2021-01-26 04:44:54 -05:00
Jirka Borovec 0c9960bfbb
Docs: upgrade packages (#5600)
* upgrade docs packages

* cmd

* -cmd
2021-01-24 11:15:52 -05:00
tarepan bb366232e7 Add non-existing resume_from_checkpoint acceptance for auto-resubmit (#4402)
* Add empty resume_from_checkpoint acceptance #4366

* Fix general error catch with focused file check

* Add fsspec HTTP extras

Add fsspec's HTTPFileSystem  support through http extras.
pl has supported remote http file (e.g. #2925),
so this commit do not add new functionality.

* Fix potential too much logging in DDP

* Add PR changelog

* Add well-written argument explanation

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Fix DDP-compatible restore logging

Notify from where the states are restored.
This feature temporally deleted as a result of PR review.
With succeeding review, added with DDP compatibility.

* Fix utility import pathes

* Refactor load step commentaries

* Refactor hpc ckpt suffix acquisition

* Refactor restore/hpc_load match

* Refactor hpc load trial

* Refactor checkpoint dir check

* Refactor unneeded function nest

* Refactor nested If

* Refactor duplicated cache clear

* Refactor attempt flow with if/elif

* Fix pip8

* Refactor hook commentary

Co-authored-by: chaton <thomas@grid.ai>

* Fix pep8

* Refactor hpc load checkpoint path acquisition

* Fix pip8

* Fix typo

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Fix typo

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Fix doc

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Refactor None Union type with Optional

* Fix build-doc CI failure debuged in #5329

* Fix fsspec import during build-doc #5329

* Fix test epoch

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Fix test with latest test models

* .

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

(cherry picked from commit b0051e8c03)
2021-01-06 12:55:38 +01:00
Jirka Borovec 0f36525e8f
fix/enable - check F401 (#5201)
* refactor - check F401

* missed

* fix
2020-12-21 10:15:04 +01:00
Jirka Borovec 059eaecbb4
set xxx_AVAILABLE as protected (#5082)
* sett xxx_AVAILABLE as protected

* docs
2020-12-14 20:19:05 +05:30
Lezwon Castelino 12cb9942a1
Tpu save (#4309)
* convert xla tensor to cpu before save

* move_to_cpu

* updated CHANGELOG.md

* added on_save to accelerators

* if accelerator is not None

* refactors

* change filename to run test

* run test_tpu_backend

* added xla_device_utils to tests

* added xla_device_utils to test

* removed tests

* Revert "added xla_device_utils to test"

This reverts commit 0c9316bb

* fixed pep

* increase timeout and print traceback

* lazy check tpu exists

* increased timeout
removed barrier for tpu during test
reduced epochs

* fixed torch_xla imports

* fix tests

* define xla utils

* fix test

* aval

* chlog

* docs

* aval

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-02 13:05:11 +00:00
Jirka Borovec 217650320e
simplify imports Omegaconf (#4873)
* hydra

* omegaconf
2020-11-27 01:00:56 +01:00
Jirka Borovec 442d57f1e9
simplify imports xla / TPU (#4872)
* xla

* tpu

* fix

* fix

* flake8
2020-11-27 00:37:48 +01:00
Jirka Borovec 11e73ceaa6
fix import and typo in AMP (#4871)
* fix import and typo

* docs

* apex

* fix

* typo
2020-11-26 23:45:52 +01:00
Jirka Borovec 9a5d40aff4
test PL examples (#4551)
* test PL examples

* minor formatting

* skip failing

* skip failing

* args

* mnist datamodule

* refactor tests

* refactor tests

* skip

* skip

* drop DM

* drop DM

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-11-17 19:35:17 +01:00
Nathan Painchaud 2d78d9b84a
CI: Added isort import check for the code on pull-request (#4242)
* added isort CI job and updated isort config

* changed CI check output from files to full diff

* added isort pre-commit hook

* Added missing first party and restricted files affected by isort

* Applied isort to root-level, docs and benchmarks

* Apply suggestions from code review

Co-authored-by: Nathan Painchaud <nathanpainchaud@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-13 22:57:46 +01:00
Jirka Borovec abf1d4b992
fix mock pkgs in docs (#4591)
* fix mock pkgs in docs

* sphinx

* CI

Co-authored-by: chaton <thomas@grid.ai>
2020-11-10 14:57:21 +01:00
Akihiro Nitta 207ff728c9
Add favicon path (#4331) 2020-10-24 16:32:06 +05:30
Jirka Borovec d3567c33a6
move base req. to root (#4219)
* move base req. to root

* check-manifest

* check-manifest

* manifest

* req
2020-10-18 20:40:18 +02:00
Jeff Yang 9be26d0c1b
Use `viewcode` extension instead of `linkcode` (#3503)
* use viewcode instead of linkcode

* use viewcode instead of linkcode

* Apply suggestions from code review

Co-authored-by: William Falcon <waf2107@columbia.edu>

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-10-07 23:55:24 +02:00
Jirka Borovec fef9b7777f
missing logs [to be merged before release][skip ci] (#3131)
* missing logs

* chlog

* add chlo to docs

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* chlog

* docs

* log

* metrics

* logs

* log

* format

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-10-07 14:51:35 -04:00
Jeff Yang fe5b943965
Callback docs with autosummary (#3908)
* callback docs with autosummary

* do not show private methods

* callback base docstring
2020-10-06 17:28:45 -04:00
Adrian Wälchli 3ff5327e83
Mocking loggers (part 1, wandb) (#3596)
* mocking for wandb

* remove wandb import in amp test

* mock loggers in sphinx

* check tests

* Update extra.txt

* setup

* dev

* min

* revert

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-09-25 16:00:02 +02:00
Nathan Raw 6685b49db3
add lightning colab tutorial notebooks (#3591)
*  add lightning colab tutorial notebooks

* Update notebooks/01-mnist-hello-world.ipynb

* Update notebooks/03-basic-gan.ipynb

* Update notebooks/04-transformers-text-classification.ipynb

* Update notebooks/02-datamodules.ipynb

* add jupytext

* 🐛 comment ipynb related things in conf.py

* 🔥 remove .py files

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-09-21 19:31:59 -06:00
Jirka Borovec a81ca4d1c1
drop nb sphinx (#3452) 2020-09-10 19:51:09 -04:00
edenlightning ee4eae884b
[docs] Add copy and toggle buttons to sphinx (#3054)
* Support sphinx toggle and copy buttons

* add buttons to conf
2020-08-19 18:59:45 -04:00
Jirka Borovec f8c058215f
simplify tests & cleaning (#2588)
* simplify

* tmpdir

* revert

* clean

* accel

* types

* test

* edit test acc

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update test acc

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-08-07 23:22:05 +02:00
Jirka Borovec 949734489a
remove deprecated in v0.9 (#2760)
* remove deprecated in v0.9

* data_loader

* import

* hook

* args
2020-07-30 23:19:28 +02:00
William Falcon 4dbd761a1c
refactor 3/n (#2709)
* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator
2020-07-25 20:56:50 -04:00
William Falcon 071e09fe38
refactor 1/n for v1.0.0 (#2704)
* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator
2020-07-25 14:38:51 -04:00
William Falcon 325852c6df
enabled no returns from eval (#2446)
* enabled no returns from eval

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs

* fixed docs
2020-07-01 07:38:00 -04:00
Adrian Wälchli d910cc5200
docs: dont mock imports when running sphinx doctest (#2396)
* skip if no amp

* dont mock when doctesting

* install extra
2020-06-27 23:31:06 -04:00
Jirka Borovec a5cc4e8a2a
hide doctest decoration (#2194)
* hide doctest decoration

* Update docs/source/conf.py

* Update docs/source/_static/copybutton.js

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-06-15 17:02:21 -04:00
Jirka Borovec bfaabd7b7f
clean requirements (#2128)
* clean requirements

* missing

* missing

* req

* min

* default >> base

* base.txt
2020-06-13 10:15:22 -04:00
Jirka Borovec 2674976f2c
remove deprecated API for v0.8 (#2073)
* remove deprecated API

* chlog

* times

* missed

* formatting check

* missing

* missing

* miss

* fix docs build error

* fix pep whitespace error

* docs

* wip

* amp_level

* amp_level

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-06-12 14:37:52 -04:00
Justus Schock bd49b07fbb
Rework of Sklearn Metrics (#1327)
* Create utils.py

* Create __init__.py

* redo sklearn metrics

* add some more metrics

* add sklearn metrics

* Create __init__.py

* redo sklearn metrics

* New metric classes (#1326)

* Create metrics package

* Create metric.py

* Create utils.py

* Create __init__.py

* add tests for metric utils

* add docstrings for metrics utils

* add function to recursively apply other function to collection

* add tests for this function

* update test

* Update pytorch_lightning/metrics/metric.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* update metric name

* remove example docs

* fix tests

* add metric tests

* fix to tensor conversion

* fix apply to collection

* Update CHANGELOG.md

* Update pytorch_lightning/metrics/metric.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* remove tests from init

* add missing type annotations

* rename utils to convertors

* Create metrics.rst

* Update index.rst

* Update index.rst

* Update pytorch_lightning/metrics/convertors.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/metrics/convertors.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* add doctest example

* rename file and fix imports

* added parametrized test

* replace lambda with inlined function

* rename apply_to_collection to apply_func

* Separated class description from init args

* Apply suggestions from code review

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* adjust random values

* suppress output when seeding

* remove gpu from doctest

* Add requested changes and add ellipsis for doctest

* forgot to push these files...

* add explicit check for dtype to convert to

* fix ddp tests

* remove explicit ddp destruction

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* add sklearn metrics

* start adding sklearn tests

* fix typo

* return x and y only for curves

* fix typo

* add missing tests for sklearn funcs

* imports

* __all__

* imports

* fix sklearn arguments

* fix imports

* update requirements

* Update CHANGELOG.md

* Update test_sklearn_metrics.py

* formatting

* formatting

* format

* fix all warnings and formatting problems

* Update environment.yml

* Update requirements-extra.txt

* Update environment.yml

* Update requirements-extra.txt

* fix all warnings and formatting problems

* Update CHANGELOG.md

* docs

* inherit

* docs inherit.

* docs

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* docs

* req

* min

* Apply suggestions from code review

Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
2020-06-10 15:43:12 +02:00
Jirka Borovec 8ee6d91d0e
code guideline (#1949)
* code rule

* Apply suggestions from code review

Co-authored-by: Jeremy Jordan <13970565+jeremyjordan@users.noreply.github.com>

* chlog

Co-authored-by: Jeremy Jordan <13970565+jeremyjordan@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-05-28 14:40:49 +00:00
Jirka Borovec 1c10560531
Fix failing docs (#1821)
* missing pkg

* update CI

* strict RTD

* strict RTD

* make

* missing

* ignore

* ignore

* mock

* typo
2020-05-14 08:25:06 -04:00
Adrian Wälchli 0cb58fbb4c
Mock packages for RTD docs build (follow up to doctests) (#1739)
* mock all packages on RTD

* update
2020-05-05 16:48:45 -04:00
Adrian Wälchli a6de1b8d75
doctest for .rst files (#1511)
* add doctest to circleci

* Revert "add doctest to circleci"

This reverts commit c45b34ea911a81f87989f6c3a832b1e8d8c471c6.

* Revert "Revert "add doctest to circleci""

This reverts commit 41fca97fdcfe1cf4f6bdb3bbba75d25fa3b11f70.

* doctest docs rst files

* Revert "doctest docs rst files"

This reverts commit b4a2e83e3da5ed1909de500ec14b6b614527c07f.

* doctest only rst

* doctest debugging.rst

* doctest apex

* doctest callbacks

* doctest early stopping

* doctest for child modules

* doctest experiment reporting

* indentation

* doctest fast training

* doctest for hyperparams

* doctests for lr_finder

* doctests multi-gpu

* more doctest

* make doctest drone

* fix label build error

* update fast training

* update invalid imports

* fix problem with int device count

* rebase stuff

* wip

* wip

* wip

* intro guide

* add missing code block

* circleci

* logger import for doctest

* test if doctest runs on drone

* fix mnist download

* also run install deps for building docs

* install cmake

* try sudo

* hide output

* try pip stuff

* try to mock horovod

* Tranfer -> Transfer

* add torchvision to extras

* revert pip stuff

* mlflow file location

* do not mock torch

* torchvision

* drone extra req.

* try higher sphinx version

* Revert "try higher sphinx version"

This reverts commit 490ac28e46d6fd52352640dfdf0d765befa56988.

* try coverage command

* try coverage command

* try undoc flag

* newline

* undo drone

* report coverage

* review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* remove torchvision from extras

* skip tests only if torchvision not available

* fix testoutput torchvision

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-05-04 22:16:54 -04:00
Adrian Wälchli 48e808c20e
Move generated RST files to subfolder (#1555)
* move generated files to subfolder

* remove if exists

* reformat argv

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* update rebase

* rebase yml

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-05-04 16:53:06 -04:00
Justus Schock 29c7d2f195
Revert namespace package search to normal package search (#1545)
* Revert this

* typos

* version++

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-21 08:26:47 -04:00
Jirka Borovec bd168819f2
fix changelog (#1452)
* fix changelog

* formatting

* add ddp_cpu

* docs

* add another
2020-04-20 17:36:26 -04:00
Adrian Wälchli 6e1d72d98a
Improved docs for Loggers (#1484)
* improve __init__

* improve logger base

* improve comet logger docs

* improved docs for mlflow

* improved nepune logger docs

* fix matplotlib import issue

* improve tensorboard docs

* improve docs for test tube

* improved trains logger docs

* improve wandb logger docs

* improved docs in experiment_logging.rst

* added MLflow to the list of loggers

* fix too long lines

* fix trains doctest

* fix neptune doctest

* fix mlflow doctest

* Apply suggestions from code review

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Apply suggestions from code review

* fix whitespace

* try bypass mode for neptune (fix doctest api key error)

* try "test" as api key

* Revert "try "test" as api key"

This reverts commit fd77db26d551f08b4b4a12bb93cbd8f7a0814f29.

* try test as api key

* update neptune docs

* bump neptune minimal version

* revert unnecessary bypass code

* test if CI runs doctests in .rst files

* Revert "test if CI runs doctests in .rst files"

This reverts commit a45aeb460a8c4b7445a35dd7b49265f48d11c485.

* add doctest directive

* neptune demo links

* added tutorial link for W&B

* fix line too long

* fix merge error

* fix merge error

* add instructions how to install loggers

* add instructions how to install the loggers

* hide _abc_impl property from docs

* review Borda, 4 spaces

* indentation in example sections

* blank

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-16 12:04:12 -04:00
Adrian Wälchli 5f6be4dd53
Pin sphinx version (fix for current docs build errors) (#1382)
* Update requirements.txt

* Update docs/requirements.txt

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* freeze sphinx version

* Update docs/source/conf.py

Co-Authored-By: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-06 06:42:33 +00:00