Commit Graph

55 Commits

Author SHA1 Message Date
Carlos Mocholí ad87d2cad0
Future 5/n: Move requirements (#13306)
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2022-06-21 17:11:33 +02:00
Jirka Borovec 759e89df21
Future 1/n: package in src/ folder (#13293)
* move: pytorch_lightning >> src/

* update setup & install

* update CI

* ci

* update CI for examples

* Self review

* mypy

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* ci

* make

* docs

* typo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ci: gpu

* .

* hpu

* typing

* docs

* tpu

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-14 20:54:55 -04:00
Aki Nitta e48f136088
Split jobs into two workflows (#12449) 2022-04-04 09:08:14 -04:00
Kaushik B bd035af78a
Fix TPU CI (#12419) 2022-03-23 11:35:38 +05:30
Carlos Mocholí 3692eba807
Drop Python 3.6 support (#11117) 2021-12-21 17:06:15 +00:00
Adrian Wälchli 3d59a2faff
triger ci only with pull request (#10932) 2021-12-05 06:55:47 +00:00
thomas chaton 8d0df6fad2
[Feat] Improve TPU CI (#6078)
* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* i

* update

* update ci

* i

* i

* i

* i
2021-07-19 19:43:21 +05:30
Kaushik B 2f3c65e57b
XLA Profiler integration (#8014) 2021-06-29 00:58:05 +05:30
Jirka Borovec 0fda862274
Refactor notebooks (#7752)
* drop notebooks

* add submodule

* copy notebooks

* docs include ipynb

* fix headers

* CI

* readthedocs

* manifest

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* req

* workdir

* pandoc

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pandoc

* manifest

* Apply suggestions from code review

* fix versions

* checkout

* `git submodule update --init --recursive --remote`

* notebooks @docs

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-06-08 16:30:13 +00:00
Jirka Borovec 851fd7fae7
Merge pull request #6885 from PyTorchLightning/v1.3.0rc
prepare v1.3.0rc
2021-04-08 14:01:26 -04:00
Jirka Borovec fcfa7fabbf
move TPU cleaning to GH actions (#5991)
* move TPU cleaning to GH actions

* test

* .
2021-02-16 18:01:22 +00:00
Jirka Borovec 27ab76923a
try random TPU config (#5992)
* try random TPU config

* random

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-16 17:19:41 +01:00
Adrian Wälchli 4bdf2fe55f
remove executable bit on source files (#5929)
* 644
2021-02-12 00:06:40 +01:00
Jirka Borovec f782230412
docs cleaning - testcode (#5595)
* testcode - python

* revert

* simple

* testcode @rst

* pl

* fix

* pip

* update

* conf

* conf

* nn.

* typo
2021-01-26 04:44:54 -05:00
Jirka Borovec 405645daf8
formatting (#4898)
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-11-30 00:57:28 -05:00
Jirka Borovec 597dfa174c
build dockers XLA 1.7 (#4891)
* build XLA 1.7

* night XLA 1.7

* rename

* use 1.7

* tpu ver
2020-11-29 15:14:19 -04:00
Jirka Borovec ab508dae0c
run TPU tests with multiple versions (#3024)
* rename

* multi build

* multi build

* copy

* copy

* copy

* copy

* copy

* copy

* clean

* note

* docker

* formatting

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-09-30 08:36:02 -04:00
zcain117 a783e75dc3
Fix tpu cleanup (#3056)
* Only try to delete jobs if there are any to delete.

* Reorder jobs.

* Remove cleanup from the jobs that run on every commit.
2020-08-24 11:04:20 +02:00
zcain117 580a5bd1df
Use kubectl to get logs from TPU CI instead of gcloud logging. (#2918)
* Use kubectl to get logs from TPU CI instead of gcloud logging.

* Update Github Action to read logs from kubectl rather than gcloud logging.
2020-08-11 19:30:56 -04:00
zcain117 35a3fd2f97
Add missing arg to docker build. (#2905) 2020-08-10 18:37:36 +00:00
Jirka Borovec 2242af11b6
another try to filter master from CircleCI jobs (#2734)
* circleci config

* Apply suggestions from code review

* miss
2020-08-05 21:43:33 +02:00
zcain117 eca7d0a6d3
Check CI_PULL_REQUEST and set GITHUB_REF accordingly. (#2741) 2020-07-29 18:35:32 -04:00
Jirka Borovec 40337cce58
freeze PT 1.5 for Horovod issue (#2744)
* freeze pt 1.5

* torchtext

* Apply suggestions from code review

Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>

* timeout

Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
2020-07-28 15:52:23 -04:00
Jirka Borovec a3aebc1350
skip CircleCI config on master (#2732)
* circleci config

* circleci config

* circleci config

* circleci config
2020-07-28 06:34:01 -04:00
Jirka Borovec 0fe933e23d
fixing TPU tests (#2632)
* init

* rename

* tpu_core_idx

* idx 8

* idxs

* @pl_multi_process_test

* assert

* assert

* deamon

* no close

* imort

* msg

* use_single_gpu

* dataset

* idx

* fix idx

* dataset

* format

* add pickable

* typo

* apex

* typo

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* docs

* typo

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* docs

* docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* docs

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-07-27 19:07:09 -04:00
zcain117 3f2c1022ab
Add a GKE cleanup workflow to run once per hour. (#2682)
* Add a GKE cleanup workflow to run once per hour.

* Add fixes. Temp use workflow as triggered by commit so we can see that command works.

* Add back in schedule.
2020-07-26 09:44:57 -04:00
zcain117 d0b8e850a4
integrate with CircleCI (#2486)
* add circleCI

* wip

* CircleCI setup that worked on my private repo. Use a working pytorch-lightning commit

* Fix the orb imports

* Update circleci header comment

* Try to pull the GITHUB_REF from the CI_PULL_REQUEST

* Use null instead of space for 'sed'

* Add TODO for codecov

* Remove echo of GKE_CLUSTER since it will be redacted by CircleCI.

* Try running codecov upload.

* Try using codecov orb

* Use pip install codecov

* Use codecov orb again since it should be approved

* dockers/tpu-tests/Dockerfile

* action

* suggestions

* drop suggestion

* suggestion

Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-23 12:13:10 -04:00
Jirka Borovec 548dbd171b
add simple CircleCI (#2609)
* add simple CircleCI

* ignore some

* note
2020-07-14 14:04:04 -04:00
Jirka Borovec d4a02e3bd8
tests: drop CircleCI (#2412)
* drop CircleCI

* add PT testing

* fix

* cpu

* conda

* conda

* req

* base

* conda

* conda

* conda

* conda

* conda

* conda

* conda

* name

* req

* info

* tests

* pt 1.6

* drop 1.6

* info
2020-06-30 10:56:05 -04:00
Jirka Borovec 73a78a13c7
CI: partial move from CircleCI (#2378)
* move from CircleCI

* req

* tex

* tex

* sudo

* extra

* recom

* pic

* dvipng
2020-06-27 16:25:33 -04:00
Jirka Borovec 0be78d13aa
native amp (#2373)
* native amp

* typo

* imports

* apex
2020-06-26 21:45:13 -04:00
Jirka Borovec bfaabd7b7f
clean requirements (#2128)
* clean requirements

* missing

* missing

* req

* min

* default >> base

* base.txt
2020-06-13 10:15:22 -04:00
Justus Schock bd49b07fbb
Rework of Sklearn Metrics (#1327)
* Create utils.py

* Create __init__.py

* redo sklearn metrics

* add some more metrics

* add sklearn metrics

* Create __init__.py

* redo sklearn metrics

* New metric classes (#1326)

* Create metrics package

* Create metric.py

* Create utils.py

* Create __init__.py

* add tests for metric utils

* add docstrings for metrics utils

* add function to recursively apply other function to collection

* add tests for this function

* update test

* Update pytorch_lightning/metrics/metric.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* update metric name

* remove example docs

* fix tests

* add metric tests

* fix to tensor conversion

* fix apply to collection

* Update CHANGELOG.md

* Update pytorch_lightning/metrics/metric.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* remove tests from init

* add missing type annotations

* rename utils to convertors

* Create metrics.rst

* Update index.rst

* Update index.rst

* Update pytorch_lightning/metrics/convertors.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/metrics/convertors.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* add doctest example

* rename file and fix imports

* added parametrized test

* replace lambda with inlined function

* rename apply_to_collection to apply_func

* Separated class description from init args

* Apply suggestions from code review

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* adjust random values

* suppress output when seeding

* remove gpu from doctest

* Add requested changes and add ellipsis for doctest

* forgot to push these files...

* add explicit check for dtype to convert to

* fix ddp tests

* remove explicit ddp destruction

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* add sklearn metrics

* start adding sklearn tests

* fix typo

* return x and y only for curves

* fix typo

* add missing tests for sklearn funcs

* imports

* __all__

* imports

* fix sklearn arguments

* fix imports

* update requirements

* Update CHANGELOG.md

* Update test_sklearn_metrics.py

* formatting

* formatting

* format

* fix all warnings and formatting problems

* Update environment.yml

* Update requirements-extra.txt

* Update environment.yml

* Update requirements-extra.txt

* fix all warnings and formatting problems

* Update CHANGELOG.md

* docs

* inherit

* docs inherit.

* docs

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* docs

* req

* min

* Apply suggestions from code review

Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
2020-06-10 15:43:12 +02:00
Jirka Borovec c438d0dd90
increase acc (#2039)
* increase acc

* try 0.45

* @pytest

* @pytest

* try .50

* duration

* pytest
2020-06-03 08:28:19 -04:00
Justus Schock ceecf1cea9
Graceful shutdown on python interpreter exit (#1631)
* Fraceful shutdown on python interpreter exit

* Update CHANGELOG.md

* Update training_loop.py

* Update training_loop.py

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* pep8, move to constant

* Update training_loop.py

* Update training_loop.py

* Update training_loop.py

* pep8, move to constant

* pep8

* timeout

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2020-05-29 16:20:04 +02:00
Jirka Borovec 033ddc0c29
update min req (#1934) 2020-05-25 07:43:17 -04:00
Jirka Borovec d610f3bb53
set min PT 1.3 (#1917)
* set min PT 1.3

* circleCI

* mergify

* min

* chlog

* skip
2020-05-22 07:14:08 -04:00
Jirka Borovec 1c10560531
Fix failing docs (#1821)
* missing pkg

* update CI

* strict RTD

* strict RTD

* make

* missing

* ignore

* ignore

* mock

* typo
2020-05-14 08:25:06 -04:00
Adrian Wälchli a6de1b8d75
doctest for .rst files (#1511)
* add doctest to circleci

* Revert "add doctest to circleci"

This reverts commit c45b34ea911a81f87989f6c3a832b1e8d8c471c6.

* Revert "Revert "add doctest to circleci""

This reverts commit 41fca97fdcfe1cf4f6bdb3bbba75d25fa3b11f70.

* doctest docs rst files

* Revert "doctest docs rst files"

This reverts commit b4a2e83e3da5ed1909de500ec14b6b614527c07f.

* doctest only rst

* doctest debugging.rst

* doctest apex

* doctest callbacks

* doctest early stopping

* doctest for child modules

* doctest experiment reporting

* indentation

* doctest fast training

* doctest for hyperparams

* doctests for lr_finder

* doctests multi-gpu

* more doctest

* make doctest drone

* fix label build error

* update fast training

* update invalid imports

* fix problem with int device count

* rebase stuff

* wip

* wip

* wip

* intro guide

* add missing code block

* circleci

* logger import for doctest

* test if doctest runs on drone

* fix mnist download

* also run install deps for building docs

* install cmake

* try sudo

* hide output

* try pip stuff

* try to mock horovod

* Tranfer -> Transfer

* add torchvision to extras

* revert pip stuff

* mlflow file location

* do not mock torch

* torchvision

* drone extra req.

* try higher sphinx version

* Revert "try higher sphinx version"

This reverts commit 490ac28e46d6fd52352640dfdf0d765befa56988.

* try coverage command

* try coverage command

* try undoc flag

* newline

* undo drone

* report coverage

* review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* remove torchvision from extras

* skip tests only if torchvision not available

* fix testoutput torchvision

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-05-04 22:16:54 -04:00
Jirka Borovec 0b22b64a10
Tests/docker (#1573)
* devel image

* try parallel

* new image
2020-04-23 12:52:59 -04:00
Travis Addair 7024177f7d
Added Horovod distributed backend (#1529)
* Initial commit of Horovod distributed backend implementation

* Update distrib_data_parallel.py

* Update distrib_data_parallel.py

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Fixed tests

* Added six

* tests

* Install tox for GitHub CI

* Retry tests

* Catch all exceptions

* Skip cache

* Remove tox

* Restore pip cache

* Remove the cache

* Restore pip cache

* Remove AMP

Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-22 17:39:08 -04:00
Jirka Borovec 4d24032ea5
tests for pytorch 1.5 (#1552)
* tests for pytorch 1.5

* up Win

* win

* win

* win

* win

* win

* win
2020-04-22 10:10:23 -04:00
Adrian Wälchli 7131685ae3
CI: Docs build preview in each PR (#1494)
* store html artifacts in circle ci

* add note to contributing.md
2020-04-19 17:07:16 -04:00
Jirka Borovec 724b787cd1
faster CI testing (#1323)
* MNIST digits

* increase test acc

* smaller parity

* drone builds

* increase GH action timeout

* drone format

* fix paths

* drone cache

* circle cache

* fix test

* lower nb epochs

* circleCI

* user orb

* fix test

* fix test

* circle cache

* circle cache

* circle cache

* comment caches

* benchmark batch size

* cache dataset

* smaller dataset

* smaller dataset

* fix nb samples

* batch size

* fix test
2020-04-02 12:28:44 -04:00
Jirka Borovec 45d671a4a8
CI: split tests-examples (#990)
* CI: split tests-examples

* tests without template

* comment depends

* CircleCI typo

* add doctest

* update test req.

* CI tests

* setup macOS

* longer train

* lover pred acc

* fix model

* rename default model

* lower tests acc

* typo

* imports

* fix test optimizer

* update calls

* fix Win

* lower Drone image

* fix call

* pytorch image

* fix test

* add dev image

* add dev image

* update image

* drone volume

* lint

* update test notes

* rename tests/models >> tests/base

* group models

* conftest

* optim imports

* typos

* fix import

* fix tests

* install AMP

* tests

* fix import
2020-03-25 07:46:27 -04:00
Adrian Wälchli 792962ecc9
CI: Force docs warnings to be raised as errors (+ fix all) (#1191)
* add argument to force warn

* fix automodule error

* fix permalink error

* fix indentation warning

* fix warning

* fix import warnings

* fix duplicate label warning

* fix bullet point indentation warning

* fix duplicate label warning

* fix "import not top level" warning

* line too long

* fix indentation

* fix bullet points indentation warning

* fix hooks warnings

* fix reference problem with excluded test_tube

* fix indentation in print

* change imports for trains logger

* remove pandas type annotation

* Update pytorch_lightning/core/lightning.py

* include bullet points inside note

* remove old quick start guide (unused)

* fix unused warning

* fix formatting

* fix duplicate label issue

* fix duplicate label warning (replaced by class ref)

* fix tick

* fix indentation warnings

* docstring ticks

* remove obsolete docstring typing

* Revert "remove old quick start guide (unused)"

This reverts commit d51bb40695.

* added old quick start guide to navigation

* remove unused  tutorials file

* ignore some modules that got deprecated and are not used anymore

* fix duplicate label warning

* move examples doc and exclude pl_examples from autodoc

* fix formatting for configure_optimizer

* fix no blank line warnings

* fix "see also" labels and add paramref extension

* fix more reference problems

* fix multi-gpu reference

* fix weird warning

* fix indentation and unrecognized characters in code block

* fix warning "... not included in toctree"

* fix PIL import error

* fix duplicate target "here" warning

* fix broken link

* revert accidentally moved pl_examples

* changelog

* stdout

* note some things to know

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-20 20:49:01 +01:00
Jirka Borovec 479a35d94e
fix docs (#982) 2020-02-28 18:48:07 -05:00
Jirka Borovec 6029fad989
fix test pkg create (#873) 2020-02-16 21:08:20 -05:00
Jirka Borovec c48541dedc
fix missing PyPI images & CI badges (#853) 2020-02-15 20:30:00 -05:00
Jirka Borovec 21d0f32047
Add GitHub Actions build (#823)
* init GH building

* try run

* circleci: install

* add cache

* add artifacts

* fix cache

* update
2020-02-12 11:28:47 -05:00