b8619a695f
new LightningModule hook "configure_callbacks" ( #5621 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-12 19:27:44 -05:00
da6dbc8d1d
PoC: Accelerator refactor ( #5743 )
...
* restoring the result from subprocess
* fix queue.get() order for results
* add missing "block_backward_sync" context manager
* add missing "block_backward_sync" context manager
* fix sync_batchnorm
* fix supported gpu-ids for tuple
* fix clip gradients and inf recursion
* accelerator selection: added cluster_environment plugin
* fix torchelastic test
* fix reduce early stopping decision for DDP
* fix tests: callbacks, conversion to lightning optimizer
* fix lightning optimizer does not pickle
* fix setting benchmark and deterministic option
* fix slurm amp test
* fix prepare_data test and determine node_rank
* fix retrieving last path when testing
* remove obsolete plugin argument
* fix test: test_trainer_config
* fix torchscript tests
* fix trainer.model access
* move properties
* fix test_transfer_batch_hook
* fix auto_select_gpus
* fix omegaconf test
* fix test that needs to simulate slurm ddp
* add horovod plugin
* fix test with named arguments
* clean up whitespace
* fix datamodules test
* remove old accelerators
* fix naming
* move old plugins
* move to plugins
* create precision subpackage
* create training_type subpackage
* fix all new import errors
* fix wrong arguments order passed to test
* fix LR finder
* Added sharded training type and amp plugin
* Move clip grad to precision plugin
* Added sharded spawn, select accelerators based on distributed_backend + enable custom fp16 plugin automatically
* Fix import issue, attempting to fix tests
* Fix initial test
* Reflect hook logic from master, should wrap model after move to device
* Optional state consolidation, since master has optimizers not wrapped
* change attribute for instance test
* reset optimizers
optimizers are not used in main process, so state would be wrong.
* legacy
* imports in accel
* legacy2
* trainer imports
* fix import errors after rebase
* move hook to new setup location
* provide unwrapping logic
* fix trainer callback system
* added ddp2 implementation
* fix imports .legacy
* move plugins
* restore legacy
* drop test.py from root
* add tpu accelerator and plugins
* fixes
* fix lightning optimizer merge
* reset bugreportmodel
* unwrapping
* step routing forward
* model access
* unwrap
* opt
* integrate distrib_type
* sync changes
* sync
* fixes
* add forgotten generators
* add missing logic
* update
* import
* missed imports
* import fixes
* isort
* mv f
* changelog
* format
* move helper to parallel plugin
* d
* add world size
* clean up
* duplicate
* activate ddp_sharded and tpu
* set nvidia flags
* remove unused colab var
* use_tpu <-> on_tpu attrs
* make some ddp_cpu and clusterplugin tests pass
* Ref/accelerator connector (#5742 )
* final cleanup
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* connector cleanup
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* trainer cleanup
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* accelerator cleanup + missing logic in accelerator connector
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add missing changes to callbacks
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* reflect accelerator changes to lightning module
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* clean cluster envs
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* cleanup plugins
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add broadcasting
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* yapf
* remove plugin connector
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* plugins
* manual optimization
* update optimizer routing
* add rank to torchelastic
* fix memory mixed precision
* setstate on trainer for pickling in ddp spawn
* add predict method
* add back commented accelerator code
* adapt test for sync_batch_norm to new plugin
* fix deprecated tests
* fix ddp cpu choice when no num_processes are given
* yapf format
* skip a memory test that cannot pass anymore
* fix pickle error in spawn plugin
* x
* avoid
* x
* fix cyclic import in docs build
* add support for sharded
* update typing
* add sharded and sharded_spawn to distributed types
* make unwrap model default
* refactor LightningShardedDataParallel similar to LightningDistributedDataParallel
* update sharded spawn to reflect changes
* update sharded to reflect changes
* Merge 1.1.5 changes
* fix merge
* fix merge
* yapf isort
* fix merge
* yapf isort
* fix indentation in test
* copy over reinit scheduler implementation from dev1.2
* fix apex tracking calls with dev_debugger
* reduce diff to dev1.2, clean up
* fix trainer config test when gpus>0 and num_processes >0 and ddp_cpu
* sort plugin tests legacy/new
* fix error handling for amp on cpu
* fix merge
fix merge
fix merge
* [Feat] Resolve manual_backward (#5837 )
* resolve manual_backward
* resolve flake8
* update
* resolve for ddp_spawn
* resolve flake8
* resolve flake8
* resolve flake8
Co-authored-by: Ubuntu <ubuntu@ip-172-31-88-60.ec2.internal>
* fix tests/accelerator tests on cpu
* [BugFix] Resolve manual optimization (#5852 )
* resolve manual_optimization
* update
* update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-88-60.ec2.internal>
* Remove copy trainer parameters to happen earlier within the loop and add safe guard to get ref model (#5856 )
* resovle a bug
* Accelerator refactor sharded rpc (#5854 )
* rpc branch
* merge
* update handling of rpc
* make devices etc. Optional in RPC
* set devices etc. later if necessary
* remove devices from sequential
* make devices optional in rpc
* fix import
* uncomment everything
* fix cluster selection
Co-authored-by: Ubuntu <ubuntu@ip-172-31-88-60.ec2.internal>
* resolve bug
* fix assert in rpc test
* resolve a test
* fix docs compilation
* accelerator refactor - fix for sharded parity test (#5866 )
* fix memory issue with ddp_spawn
* x
x
x
x
x
x
x
x
x
* x
* Remove DDP2 as this does not apply
* Add missing pre optimizer hook to ensure lambda closure is called
* fix apex docstring
* [accelerator][BugFix] Resolve some test for 1 gpu (#5863 )
* update
* revert init
* resolve a bug
* update
* resolve flake8
* update
* update
* update
* revert init
* resolve a bug
* update
* resolve flake8
* update
* update
* update
* update
* update
* revert init
* resolve a bug
* update
* resolve flake8
* update
* update
* update
* revert init
* update
* resolve flake8
* update
* update
* update
* update
* update
* all_gather
* update
* make plugins work, add misconfig for RPC
* update
* update
* remove breaking test
* resolve some tests
* resolve flake8
* revert to ddp_spawn
Co-authored-by: root <root@ip-172-31-88-60.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-88-60.ec2.internal>
Co-authored-by: Justus Schock <justus.schock@rwth-aachen.de>
* yapf isort
* resolve flake8
* fix apex doctests
* fix apex doctests 2
* resolve docs
* update drone
* clean env
* update
* update
* update
* update
* merge
* Fix RPC related tests, clean out old API, update for new accelerator API [skip ci] (#5881 )
* Fix RPC related tests, clean out old API, update for new accelerator API
* Move tests out of legacy folder, update paths and names
* Update test_remove_1-4.py
* Expose properties for tpu cores/gpus/num_gpus
* Add root GPU property
* Move properties to properties.py
* move tests that were previously in drone
* Fix root GPU property (#5908 )
* Move root GPU to property, remove horovod set as this is handled in horovod plugin, ensure we mock correctly to set GPU accelerator
* Add missing tests back
* fix best model path transfer when no checkpoint callback available
* Fix setup hook order [wip] (#5858 )
* Call trainer setup hook before accelerator setup
* Add test case
* add new test
* typo
* fix callback order in test
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* rename ddp sequential -> rpc sequential for special test
* revert
* fix stupid merge problem
* Use property in connector for sampler (#5913 )
* merge the import conflicts
* fix spawning of processes in slurm
* [wip] Fix some bugs for TPU [skip ci] (#5878 )
* fixed for single tpu
* fixed spawn
* fixed spawn
* update
* update
* wip
* resolve bugs
* resolve bug
* update on comment
* removed decorator
* resolve comments
* set to 4
* update
* update
* need cleaning
* update
* update
* update
* resolve flake8
* resolve bugs
* exclude broadcast
* resolve bugs
* change test
* update
* update
* skip if meet fails
* properly raise trace
* update
* add catch
* wrap test
* resolve typo
* update
* typo
Co-authored-by: Lezwon Castelino <lezwon@gmail.com>
Co-authored-by: Your Name <you@example.com>
* resolve some tests
* update
* fix imports
* update
* resolve flake8
* update azure pipeline
* skip a sharded test on cpu that requires a gpu
* resolve tpus
* resolve bug
* resolve flake8
* update
* updat utils
* revert permission change on files
* suggestions from carlos
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* remove unrelated formatting changes
* remove incomplete comment
* Update pytorch_lightning/accelerators/__init__.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* remove unrelated formatting change
* add types
* warn 1.7 ddp manual backward only if ddp kwarg unset
* yapf + isort
* pep8 unused imports
* fix cyclic import in docs
* Apply suggestions from code review
* typer in accelerator.py
* typo
* Apply suggestions from code review
* formatting
* update on comments
* update typo
* Update pytorch_lightning/trainer/properties.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* update
* suggestion from code review
* suggestion from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: SeanNaren <sean@grid.ai>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-88-60.ec2.internal>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: root <root@ip-172-31-88-60.ec2.internal>
Co-authored-by: Lezwon Castelino <lezwon@gmail.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-12 15:48:56 -05:00
4bdf2fe55f
remove executable bit on source files ( #5929 )
...
* 644
2021-02-12 00:06:40 +01:00
31da16344c
add docs ( #5902 )
2021-02-11 14:32:32 +00:00
9475c845cb
Docs/fixes ( #5914 )
...
* wip
* ..
* ...
* Apply suggestions from code review
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-02-11 10:22:07 +00:00
7b00894130
[feat] Add StochasticWeightAveragingCallback ( #5640 )
...
* add swa callback
* switch back to 1.6.0
* remove optimizer_step
* move super
* update
* forgot update_parameters
* update on comments
* works for ddp
* resolve flake8
* remove set_model
* resolve flake8
* resolve cpu
* resolve flake8
* resolve flake8
* update
* update on comments
2021-02-11 00:05:59 +00:00
2008d77f28
Fix broken code in docs ( #5859 )
...
Co-authored-by: Roman Tezikov <roman.tezikov@lamoda.ru>
2021-02-08 11:52:12 -05:00
74fb87552a
Update README .gif ( #5777 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit e8c1755129
2021-02-05 21:44:01 +01:00
77be6f6e24
resolve conflits
...
resolve doc
boring commit
docs
torchvision
tpu
Update dockers/tpu-tests/tpu_test_cases.jsonnet
Update dockers/tpu-tests/tpu_test_cases.jsonnet
2021-02-05 21:43:10 +01:00
6d7c01b1b4
[docs] Add docs for non-SLURM cluster setup ( #5754 )
...
* Add docs for non-slurm cluster setup
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update docs/source/cluster.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/cluster.rst
Co-authored-by: Alexander <alexander@reshytko.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-02-05 21:43:10 +01:00
a3161267d9
link blog ( #5740 )
2021-02-05 21:40:40 +01:00
ad9b188b78
Fix typo in LightningOptimizer ( #5736 )
2021-02-05 21:40:40 +01:00
b2b27e4018
Fix docs typo ( #4930 )
...
* Fix docs
* typo
* import
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-05 21:40:40 +01:00
78918a9e8e
Rename opt_idx to optimizer_idx in docs for complex training loops ( #5712 )
2021-02-05 21:40:40 +01:00
2c9f606af9
Fix tensor printing in `trainer.test()` ( #5138 )
...
* Fix showing test results for tensors
* Fix docs
* Update pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Fix lint issues
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-02-04 20:55:41 +01:00
e425bf3ba9
[BugOnFeat] Resolve bug with Finetuning ( #5744 )
...
* resolve bug + add doc
* Update pytorch_lightning/callbacks/finetuning.py
* resolve bug
* start adding more test
* add more tests for finetuning callback functions
* rename to flatten_modules
* resolve doc
* Update pytorch_lightning/callbacks/finetuning.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* resolve comments
* remove update on BoringModel
* update on comments
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-02-04 18:36:54 +00:00
d05cdf83f1
Merge remote-tracking branch 'carmocca/sync-1.1.5' into release/1.2-dev
2021-02-04 09:42:59 -05:00
9555043a29
Force ModelCheckpoint callback to run last ( #5731 )
2021-02-03 16:40:57 -05:00
630a88addb
drop deprecated docs ( #5768 )
...
* drop deprecated docs
* notes
* .
2021-02-03 20:21:19 +00:00
0ff4c56d9b
fix typo in multi-gpu docs ( #5402 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-02-03 19:39:28 +01:00
d5c107fa96
Docs: move images ( #5756 )
...
* move images
* logo
2021-02-03 15:08:19 +00:00
b3ebc18bcb
Hardware specific parts of Accelerator Refactoring ( #5719 )
...
* add basic accelerator class.
Co-Authored with @awaelchi
* pep8
Co-authored-by: @awaelchi
* add cpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add gpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add tpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add accelerator connector
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add single device training
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add single tpu
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add tpu spawn
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* make on_colab_kaggle utility func
* add basic accelerator class.
Co-Authored with @awaelchi
* pep8
Co-authored-by: @awaelchi
* add cpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add gpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add tpu accelerator
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add accelerator connector
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add single device training
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add single tpu
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add tpu spawn
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* make on_colab_kaggle utility func
* fixes
* move
* yapf
* .
* .
* .
* flake8
* sync accelerator connector changes from dev1.2
* changelog
* fix tpu handling
* tpu
* aval
* yapf
* Update pytorch_lightning/plugins/training_type/tpu_spawn.py
Co-authored-by: chaton <thomas@grid.ai>
* Update pytorch_lightning/accelerators/accelerator_connector.py
Co-authored-by: chaton <thomas@grid.ai>
* Update pytorch_lightning/plugins/training_type/tpu_spawn.py
Co-authored-by: chaton <thomas@grid.ai>
* Update tpu_spawn.py
* Update pytorch_lightning/accelerators/accelerator_connector.py
Co-authored-by: chaton <thomas@grid.ai>
* indentation
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: chaton <thomas@grid.ai>
2021-02-01 08:34:59 -05:00
a17c941cc4
LR scheduler docs update ( #5678 )
...
* doc updates
* typo suggestions by rohit
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* update based on suggestions
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-01-29 10:08:29 +01:00
99ea2a3b35
define Yapf config ( #5591 )
...
* define YAPF
* add check
* add check
* add temp ignore
* apply yapf
* ex
2021-01-27 21:58:33 -05:00
3da28fd634
[feat] 1/2 Add trainer.predict ( #5579 )
...
* start adding predict
* add predict
* resolve test
* add predict
* remove limit_predict
* update
* add test for predict
* typo
* update on comments
* remove predict_step
* update ddp_shareded
* check ddp_sharded
* resolve on comments
* resolve isort
* update dp
* add test dp 1 gpu
* made default forward
* resolve path
* resolve bug
* update on comments
* resolve doc
* resolve bug
* update
* resolve bug
* update on comments
* resolve pep8
* update test doc
* update on comments
* solve special tests
* resolve bug
* resolve flake8
* Update pytorch_lightning/callbacks/progress.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* add predict to LightningModule
* missing predict
* typo
* rename is_prediction to _predicting
* add
* update
* update
* update doc
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-27 11:38:14 -05:00
221c4a0ba2
[Metrics] AUC/AUROC class interface ( #5479 )
...
* base files
* auc done
* init files
* auc class interface
* fixing auc
* more fixes
* working auroc
* update auc
* add docs
* remove leftovers from merge
* suggestions
* fix f-string
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add deprecated tests
* make logic clearer
* Update pytorch_lightning/metrics/classification/auroc.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* fix
* fix
* fix docs
* fix isort
* fix deprecated test
* fix tests
* fix tests
* fix isort
* Apply suggestions from code review
* add enum
* deprecate old impl
* update from suggestions
* chlog
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
2021-01-27 08:16:54 -05:00
d0aaf983b9
[Feat] Adding PruningCallback ( #5618 )
...
* wip
* add pruning callback
* add condition for duplicated weights
* update on comments
* update on comments
* update on comments
* add more tests
* resolve flake8
* resolve on comments
* update changelog
* update on comments
* update on comments
* change order
* remove ddp_spawn skip
* update
* typo
* Update pytorch_lightning/callbacks/pruning.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/callbacks/pruning.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update on comments
* forgot platform
* update on comments
* remove @rank_zero_only
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-27 01:00:42 -05:00
7e2e874d95
Refactor: legacy accelerators and plugins ( #5645 )
...
* tests: legacy
* legacy: accel
* legacy: plug
* fix imports
* mypy
* flake8
2021-01-26 20:04:36 -05:00
dee5553b2b
move to Pages dir ( #4869 )
...
* folders
* common / advanced / extensions
* paths
* flake8
* isort
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-26 15:07:07 -05:00
8c55a08e85
Compositional metrics ( #5464 )
...
* implement compositional metrics
* implement composition functions for metrics
* test compositions
* docs
* pytest
* pep8
* fix argument resolution
* return all kwargs if filtering not possible
* fix typo
* implement hashing
* Update pytorch_lightning/metrics/compositional.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add representation
* Apply suggestions from code review
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* chlog
* flake8
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
2021-01-26 11:56:12 -05:00
0435e23a64
deprecate enable_pl_optimizer as it is not restored properly ( #5244 )
...
* update
* clean test
* still in progress
* udpdate test
* update
* update
* resolve flake
* add test for zero_grad
* update
* works without accumulated_grad
* update
* update
* resolve amp
* revert back to True
* update
* clean tests
* cleaned out
* typo
* update test
* git repare bug
* remove print
* udpate
* Fix formatting/optimizer imports
* Refactor the test for cleanliness
* Add vanilla model to the test, better var names
* Fixed var names, let's clean up these mock tests
* repare test
* update test
* resolve flake8
* add manual_optimization
* update tests
* resolve flake8
* add random accumulate_grad_batches
* improve test
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update
* clean tests
* correct bug
* Apply suggestions from code review
* format
* adress comments
* update on comments
* wip
* typo
* depreceate enable_pl_optimizer
* resolve latest bugs
* update
* resolve merge
* add comment
* Update pytorch_lightning/core/lightning.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/deprecated_api/test_remove_1-3.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/connectors/optimizer_connector.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update on comments
* update restore
* add a property
* remove setstate as not needed anymore
* update test
* provide optimizer to on_before_zero_grad
* update on comments
* update on comments
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* mofidy import
* update changelog
* resolve flake8
* update
* update
* clean doc
Co-authored-by: SeanNaren <sean@grid.ai>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
(cherry picked from commit f2e99d617f
2021-01-26 14:29:46 +01:00
cccbcc5420
[docs] Add ananthsub to core ( #5476 )
...
* Update test_manual_optimization.py
* Update governance.rst
* Update test_manual_optimization.py
* Update test_manual_optimization.py
(cherry picked from commit d30e316a35
2021-01-26 14:28:47 +01:00
fb6c33a52d
Fix pre-commit trailing-whitespace and end-of-file-fixer hooks. ( #5387 )
...
(cherry picked from commit 4c6f36e6e1
2021-01-26 14:27:56 +01:00
edfd6b2c93
Update sharded install to latest fairscale release, add reasoning why fork required for sequential parallelism ( #5380 )
...
(cherry picked from commit ee8373110a
2021-01-26 14:27:56 +01:00
f782230412
docs cleaning - testcode ( #5595 )
...
* testcode - python
* revert
* simple
* testcode @rst
* pl
* fix
* pip
* update
* conf
* conf
* nn.
* typo
2021-01-26 04:44:54 -05:00
30f31d32c8
docs: Add BackboneLambdaFinetuningCallback ( #5553 )
...
* Add and fix the docs of BackboneLambdaFinetuningCallback
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-25 12:07:02 +00:00
0c9960bfbb
Docs: upgrade packages ( #5600 )
...
* upgrade docs packages
* cmd
* -cmd
2021-01-24 11:15:52 -05:00
24462dc5fd
Set progressbar refresh rate in Google Colab ( #5516 )
...
* refresh
* add tests
* docs
* chlog
* chlog
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* update docstring
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2021-01-19 12:47:14 -05:00
c8f605e47d
Classification metrics overhaul: precision & recall (4/n) ( #4842 )
...
* Add stuff
* Change metrics documentation layout
* Add stuff
* Add stat scores
* Change testing utils
* Replace len(*.shape) with *.ndim
* More descriptive error message for input formatting
* Replace movedim with permute
* PEP 8 compliance
* WIP
* Add reduce_scores function
* Temporarily add back legacy class_reduce
* Division with float
* PEP 8 compliance
* Remove precision recall
* Replace movedim with permute
* Add back tests
* Add empty newlines
* Add precision recall back
* Add empty line
* Fix permute
* Fix some issues with old versions of PyTorch
* Style changes in error messages
* More error message style improvements
* Fix typo in docs
* Add more descriptive variable names in utils
* Change internal var names
* Revert unwanted changes
* Revert unwanted changes pt 2
* Update metrics interface
* Add top_k parameter
* Add back reduce function
* Add stuff
* PEP3
* Add depreciation
* PEP8
* Deprecate param
* PEP8
* Fix and simplify testing for older PT versions
* Update Changelog
* Remove redundant import
* Add tests to increase coverage
* Remove zero_division
* fix zero_division
* Add zero_div + edge case tests
* Reorder cls metric args
* Add back quotes for is_multiclass
* Add precision_recall and tests
* PEP8
* Fix docs
* Fix docs
* Update
* Change precision_recall output
* PEP8/isort
* Add method _get_final_stats
* Fix depr test
* Add comment to deprecation tests
* isort
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Add typing to test
* Add matc str to pytest.raises
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-18 03:24:13 -05:00
61f415f2ac
Add LambdaCallback ( #5347 )
...
* Add LambdaCallback
* docs
* add pr link
# Conflicts:
# CHANGELOG.md
* convention
* Fix Callback Typo
* Update pytorch_lightning/callbacks/lambda_cb.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update pytorch_lightning/callbacks/lambda_cb.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update pytorch_lightning/callbacks/lambda_cb.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* use Misconfigureation
* update docs
* sort export
* use inspect
* string fill
* use fast dev run
* isort
* remove unused import
* sort
* hilightning
* highlighting
* highlighting
* remove debug log
* eq
* res
* results
* add misconfig exception test
* use pytest raises
* fix
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/callbacks/lambda_cb.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* hc
* rm pt
* fix
* try fix
* whitespace
* new hook
* add raise
* fix
* remove unused
* rename
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
2021-01-13 04:42:49 -05:00
02acb21e55
[Metrics] Disable default reset after compute ( #5409 )
...
* reset
* self._cache -> cache (make cache local variable so it is not overwritten)
* pep8
* fix metric result integration
* rm print statements
* better comment
* changelog
* Update docs/source/metrics.rst
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-01-13 04:01:08 -05:00
0192f0ce40
Add a performance section to TPU docs to address FAQ ( #5445 )
...
* header
* update docs
* punctuation
* adding another note
* some more notes
* Update docs/source/tpu.rst
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* punctuation
Co-authored-by: Lezwon Castelino <lezwon@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
2021-01-11 08:12:38 -05:00
3ee1b55bf1
Reformat iou [func] and add IoU class ( #4704 )
...
* added Iou
* Create iou.py
* Update iou.py
* Update iou.py
* Update CHANGELOG.md
* Update metrics.rst
* Update iou.py
* Update iou.py
* Update __init__.py
* Update iou.py
* Update iou.py
* Update classification.py
* Update classification.py
* Update classification.py
* Update __init__.py
* Update __init__.py
* Update iou.py
* Update classification.py
* Update metrics.rst
* Update CHANGELOG.md
* Update CHANGELOG.md
* add iou
* add test
* add test
* removed iou
* add iou
* add iou test
* add float
* reformat test_iou
* removed test_iou
* updated format
* updated format
* Update CHANGELOG.md
* updated format
* Update metrics.rst
* Apply suggestions from code review
merge suggestions
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* added equations
* reformat init
* change format
* change format
* deprecate iou and test for this
* fix changelog
* delete iou test in test_classification
* format change
* format change
* format
* format
* format
* delete white space
* delete white space
* fix tests
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* better deprecation
* fix docs
* Apply suggestions from code review
* fix todo
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-08 13:36:08 +00:00
06668c0ddf
[Metrics] MetricCollection ( #4318 )
...
* docs + precision + recall + f_beta + refactor
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* rebase
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* fixes
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* added missing file
* docs
* docs
* extra import
* add metric collection
* add docs + integration with log_dict
* add test
* update
* update
* more test
* more test
* pep8
* fix doctest
* pep8
* add clone method
* add clone method
* merge-2
* changelog
* kwargs filtering and tests
* pep8
* fix test
* update docs
* Update docs/source/metrics.rst
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
* fix docs
* fix tests
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* fix docs
* fix doctest
* fix doctest
* fix doctest
* fix doctest
Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-08 11:09:07 +01:00
06f36092a4
added image-gradients ( #4763 ) [1/2] ( #5056 )
...
* added iamge-gradients (#4763 )
* fixed tests code format
* made recommended fixes
* removed explicit device flags
* tried to fix doctest failure
* pep8 and doctest fixes
* added to docs/metrics
* updated CHANGELOG
* added the noqa flag
* added suggested modification to changelog
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* recommended update to docstring
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* removed device from docstring
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* made recommended fixes
* Update CHANGELOG.md
* Apply suggestions from code review
* added 1-line docstrings
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-01-07 20:34:38 +01:00
a46cbdbd76
Change the classifier input from 2048 to 1000. ( #5232 )
...
* Change the classifier input from 2048 to 1000.
* Update docs for Imagenet example
Thanks @rohitgr7
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
(cherry picked from commit a40e3a325e
2021-01-06 15:14:16 +01:00
9cfbf8d609
Disable checkpointing, earlystopping and logging with fast_dev_run ( #5277 )
...
* Disable checkpointing, earlystopping and logger with fast_dev_run
* docs
* chlog
* disable callbacks and enable DummyLogger
* add log
* use dummy logger method
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit f740245521
2021-01-06 12:57:24 +01:00
bb366232e7
Add non-existing resume_from_checkpoint acceptance for auto-resubmit ( #4402 )
...
* Add empty resume_from_checkpoint acceptance #4366
* Fix general error catch with focused file check
* Add fsspec HTTP extras
Add fsspec's HTTPFileSystem support through http extras.
pl has supported remote http file (e.g. #2925 ),
so this commit do not add new functionality.
* Fix potential too much logging in DDP
* Add PR changelog
* Add well-written argument explanation
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix DDP-compatible restore logging
Notify from where the states are restored.
This feature temporally deleted as a result of PR review.
With succeeding review, added with DDP compatibility.
* Fix utility import pathes
* Refactor load step commentaries
* Refactor hpc ckpt suffix acquisition
* Refactor restore/hpc_load match
* Refactor hpc load trial
* Refactor checkpoint dir check
* Refactor unneeded function nest
* Refactor nested If
* Refactor duplicated cache clear
* Refactor attempt flow with if/elif
* Fix pip8
* Refactor hook commentary
Co-authored-by: chaton <thomas@grid.ai>
* Fix pep8
* Refactor hpc load checkpoint path acquisition
* Fix pip8
* Fix typo
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix typo
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix doc
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Refactor None Union type with Optional
* Fix build-doc CI failure debuged in #5329
* Fix fsspec import during build-doc #5329
* Fix test epoch
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix test with latest test models
* .
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
(cherry picked from commit b0051e8c03
2021-01-06 12:55:38 +01:00
cc607d5937
[Docs] update docs for resume_from_checkpoint ( #5164 )
...
* update docs and add pathlib support
* fix
(cherry picked from commit dd442b6d33
2021-01-06 12:53:23 +01:00
2d70a80148
docs: logits -> probs in Accuracy metric documentation ( #5340 )
...
* fix: logits -> probs in accuracy metrics documentation
* Update metrics.rst
* Update metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit 15a400b95f
2021-01-06 12:49:37 +01:00
Adrian Wälchli