Commit Graph

106 Commits

Author SHA1 Message Date
Justus Schock 5d239ccd70
Base classes for accelerator refactoring (#5715)
* add basic accelerator class.
Co-Authored with @awaelchi

* Add base plugin class.
Co-authored with @awaelchi

* add basic trainign type plugin.
Co-Authored with @awaelchi

* add basic precision plugin.
Co-Authored with @awaelchi

* Add missing inits.
Co-authored with @awaelchi

* pep8

Co-authored-by: @awaelchi

* ignore  flake8

* coverage omit

* imports in init

* lost

* imports

* flake8

* .

* .

* chlog

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-30 14:55:28 -05:00
Jirka Borovec 99ea2a3b35
define Yapf config (#5591)
* define YAPF

* add check

* add check

* add temp ignore

* apply yapf

* ex
2021-01-27 21:58:33 -05:00
Jirka Borovec 7e2e874d95
Refactor: legacy accelerators and plugins (#5645)
* tests: legacy

* legacy: accel

* legacy: plug

* fix imports

* mypy

* flake8
2021-01-26 20:04:36 -05:00
Jirka Borovec cb58fdeb3d
fix: freeze mypy (#5634)
* update mypy for tests

* freeze
2021-01-24 20:09:08 -05:00
Alan Du 1c8ad3a94b Tighten up mypy config (#5237) 2021-01-05 09:58:37 +01:00
Jirka Borovec 0f36525e8f
fix/enable - check F401 (#5201)
* refactor - check F401

* missed

* fix
2020-12-21 10:15:04 +01:00
Jirka Borovec 35fd6e93c7
refactor - check E501 (#5200) 2020-12-21 14:23:09 +05:30
Jirka Borovec 6d2c564bc6
refactor - check F841 (#5202) 2020-12-21 11:10:55 +05:30
Jirka Borovec 2c11d96012
replace pyright by mypy (#5021)
* drop pyright & add mypy

* detail

* name

* fix

* flake8

* ver

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-09 10:57:11 +08:00
chaton c2e6e68c7e
optimizer clean up (#4658)
* add LightningOptimizer

* typo

* add mock closure

* typo

* remove logic in optimizer_step

* update

* update

* update

* desactivate LightningOptimizer for hovorod

* resolve flake

* typo

* check optimizer name

* change name

* added backward to LightningOptimizer

* remove use_lightning_optimizer

* move update

* simplify init

* resolve comments

* resolve bug

* update

* update

* resolve bugs

* resolve flake8

* set state

* work manual_optimizer_step

* add doc

* add enable_pl_optimizer

* make optimizer_step

* add make_optimizer_step

* add examples

* resolve test

* add test_optimizer_return_options_enable_pl_optimizer

* add enable_pl_optimizer=True

* update

* update tests

* resolve bugs

* update

* set Trainer to False

* update

* resolve bugs

* update

* remove from doc

* resolve bug

* typo

* update

* set to True

* simplification

* typo

* resolve horovod

* unwrap horovod

* remove Optimizer

* resolve horovod

* move logic to amp_backend

* doesn't seem to be pickable

* update

* add again

* resolve some bugs

* cleanup

* resolve bug with AMP

* change __repr__

* round at -12

* udpate

* update

* update

* remove from horovod

* typo

* add convert_to_lightning_optimizers in each accelerators

* typo

* forgot

* forgot a convert_to_lightning_optimizers

* update

* update

* update

* increase coverage

* update

* resolve flake8

* update

* remove useless code

* resolve comments + add support for LightningOptimizer base class

* resolve flake

* check optimizer get wrapped back

* resolve DDPSharded

* reduce code

* lightningoptimizer

* Update pytorch_lightning/core/optimizer.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update pytorch_lightning/core/lightning.py

* remove reference to step function

* Apply suggestions from code review

* update on comments

* resolve

* Update CHANGELOG.md

* add back training_step in apex and native_amp

* rename optimizer_step

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-01 00:09:46 +00:00
Jirka Borovec bddc6cd77a
pytest default color (#4703)
* pytest default color

* time

Co-authored-by: chaton <thomas@grid.ai>
2020-11-18 10:53:44 +00:00
Nathan Painchaud 2d78d9b84a
CI: Added isort import check for the code on pull-request (#4242)
* added isort CI job and updated isort config

* changed CI check output from files to full diff

* added isort pre-commit hook

* Added missing first party and restricted files affected by isort

* Applied isort to root-level, docs and benchmarks

* Apply suggestions from code review

Co-authored-by: Nathan Painchaud <nathanpainchaud@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-13 22:57:46 +01:00
Nicki Skafte edcb6e49b9
Speedup of metric tests (#4122)
* speedup

* something working

* update the rest

* more desc

* recurse tests/metrics again

* pep8

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-14 13:51:58 -04:00
Jirka Borovec 2c3b512c50
reverted "temporary drop metrics tests while speeding them up" and SKIP (#4115)
* Revert "temporary drop metrics tests while speeding them up (#4071)"

This reverts commit 86c70622fb.

* skip metrics tests

* skipping
2020-10-14 19:01:43 +02:00
William Falcon 5b645d713e
Covv1 (#4072)
* temporary drop metrics tests while speeding them up

* cov

* cov

* docs
2020-10-11 10:21:53 -04:00
William Falcon 2b255a3df4
ref: enable custom clusters (1/n) (#4048)
* enable cluster plugins

* enable cluster plugins + test backend choices

* enable cluster plugins + test backend choices

* enable cluster plugins + test backend choices

* enable cluster plugins + test backend choices

* enable cluster plugins + test backend choices

* enable cluster plugins + test backend choices
2020-10-10 08:09:29 -04:00
Jirka Borovec c77073f040
skip files in coverage (#3944) 2020-10-07 12:37:01 -04:00
William Falcon f43028f3ae
added copyright notices (#3062) 2020-08-19 22:03:22 -04:00
Simon-Martin Schröder a5736d244f
Configure isort (#2136)
* Configure isort

* Fix whitespace

* Line length, make THIRDPARTY the default
2020-06-16 14:36:19 -04:00
Jirka Borovec 2674976f2c
remove deprecated API for v0.8 (#2073)
* remove deprecated API

* chlog

* times

* missed

* formatting check

* missing

* missing

* miss

* fix docs build error

* fix pep whitespace error

* docs

* wip

* amp_level

* amp_level

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-06-12 14:37:52 -04:00
kumuji fd7814d287
Added black formater for the code with code-checker on pull (#1610)
* black

Added throught black.toml other options are hard so far

No caching for black github action

Moved from black.toml to pyproject.toml

Exclude not only yml but also yaml

Update pyproject.toml

Co-authored-by: Thomas Johansen <thomasjo@gmail.com>

Update .github/workflows/code-formatting-check.yml

mergify

Remove formating check

E231 error ignoring because of black formating

Updated CONTRIBUTING to the master

* Update .github/workflows/code-formatting-check.yml

* Bump black to 19.10b0 version

* resolved incorrect merge of CONTRIBUTING,

Black skipping string normalization

* Minor fixes in CONTRIBUTING, two typos

* Update setup.cfg

* chlog

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-06-03 18:23:14 +02:00
Jirka Borovec c438d0dd90
increase acc (#2039)
* increase acc

* try 0.45

* @pytest

* @pytest

* try .50

* duration

* pytest
2020-06-03 08:28:19 -04:00
William Falcon d40425d257
added warning to crash (#1625)
* added warning to crash

* formatting

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-30 08:04:18 -04:00
Jirka Borovec bd168819f2
fix changelog (#1452)
* fix changelog

* formatting

* add ddp_cpu

* docs

* add another
2020-04-20 17:36:26 -04:00
Jirka Borovec 61177cd1c8
system info (#1234)
* system info

* update big info

* test script

* update config

* rename script

* import path
2020-03-27 08:45:52 -04:00
Jirka Borovec 45d671a4a8
CI: split tests-examples (#990)
* CI: split tests-examples

* tests without template

* comment depends

* CircleCI typo

* add doctest

* update test req.

* CI tests

* setup macOS

* longer train

* lover pred acc

* fix model

* rename default model

* lower tests acc

* typo

* imports

* fix test optimizer

* update calls

* fix Win

* lower Drone image

* fix call

* pytorch image

* fix test

* add dev image

* add dev image

* update image

* drone volume

* lint

* update test notes

* rename tests/models >> tests/base

* group models

* conftest

* optim imports

* typos

* fix import

* fix tests

* install AMP

* tests

* fix import
2020-03-25 07:46:27 -04:00
Jirka Borovec 22a7264e9a
improve partial Codecov (#1172)
* ignore in setup

* show report

* abs imports

* abstract pass

* cover loggers

* doctest trains

* locals

* pass

* revert tensorboard

* use tensorboardX

* revert tensorboardX

* fix trains

* Add TrainsLogger.set_credentials (#1179)

* Add TrainsLogger.set_credentials to control trains server configuration and authentication from code. Sync trains package version.
Fix CI Trains tests

* Add global TrainsLogger set_bypass_mode (#1187)

* Add global TrainsLogger set_bypass_mode skips all external communication

Co-authored-by: bmartinn <>

* rm some no-cov

Co-authored-by: Martin.B <51887611+bmartinn@users.noreply.github.com>
2020-03-19 09:14:29 -04:00
Jirka Borovec c89e482f85
prune codecov exceptions (#1107) 2020-03-16 14:40:34 -04:00
Jirka Borovec 462788738b CI buils with minimal and latest requirements (#500)
* install nim req.

* update requirements

* drop Cython
2019-11-25 06:39:19 -05:00
Jirka Borovec 1fd1e42aa6 Fix setup-doc for pypi (#472)
* add Twine to CI

* freeze Twine

* freeze Twine

* minor refactoring

* try another

* fix req.

* update README

* fix __doc__

* fix multiple req. test-tube
2019-11-09 00:59:14 -05:00
Jirka Borovec 37647d835a add package info (#395)
* add package info #358

* Update __init__.py

* wrap package info

* update CI

* fix package info

* fix for #388

* prune duplicated configs

* fix install

* use req from file

* move info to sep. module
drop comments from req

* add setup req.

* add setup req.

* update get info

* refactor init

* update pip

* fix failing on buildins

* fix failing open

* fix test imports

* fix tests

* fix pep8
2019-10-28 18:41:13 -04:00
William Falcon 0d5da5f29b
added gan template (#115)
* added gan template

* ommit templates folder
2019-08-14 08:38:49 -04:00
William Falcon 5785e5965d last config try 2019-08-07 12:51:52 -04:00
William Falcon 0985a1875c last config try 2019-08-07 12:42:36 -04:00
William Falcon 8391b744c0 pt dpp some ignores 2019-07-24 19:36:35 -04:00
William Falcon e3463c8fe3 pt dpp some ignores 2019-07-24 19:35:31 -04:00
William Falcon 6fb27c4526 pt dpp some ignores 2019-07-24 19:29:51 -04:00
William Falcon 70a2e66ae9 running ddp tests 2019-07-24 18:30:47 -04:00
William Falcon abbbcac9fa running ddp tests 2019-07-24 18:30:35 -04:00
William Falcon 9d588f337f running ddp tests 2019-07-24 18:30:08 -04:00
William Falcon 5ebe494212 test memory printing 2019-07-24 17:34:08 -04:00
William Falcon 1d28b468bd remove exception line 2019-07-24 17:30:16 -04:00
William Falcon d372b21b5e ignore test module model 2019-07-24 17:28:23 -04:00
William Falcon c277ab1036 ignore tests file 2019-07-24 17:27:33 -04:00
William Falcon 2f4bd676e8 added coverage file 2019-07-24 16:09:24 -04:00
William Falcon ad3d00bcae added coverage file 2019-07-24 16:08:35 -04:00
William Falcon cfd2792d76 added coverage file 2019-07-24 16:04:36 -04:00
William Falcon dfccc03da8 added coverage file 2019-07-24 16:04:18 -04:00
William Falcon 843675e9a1 added coverage file 2019-07-24 16:00:48 -04:00
William Falcon eae4fa0495 added coverage file 2019-07-24 15:57:18 -04:00
William Falcon cc875cd603 added coverage file 2019-07-24 15:56:33 -04:00
William Falcon 23e0986141 removed coverage file 2019-07-24 15:54:14 -04:00
William Falcon ea7be12bb1 added coverage file 2019-07-24 15:52:59 -04:00
William Falcon b836e6f321 removed dead code in model save 2019-07-24 15:43:10 -04:00
William Falcon 08c76c47bd removed opt check 2019-07-24 15:38:15 -04:00
Shreyas Bapat 4809de8765 Fix pip install too 2019-04-03 22:47:55 +05:30