Commit Graph

178 Commits

Author SHA1 Message Date
Rohit Gupta 6609b2e46f
enable `on_load_checkpoint` for `datamodule` for all `trainer_fn` (#10238) 2021-11-01 14:20:46 +00:00
Carlos Mocholí 2b24be2e45
Simplify `LightningOptimizer` (#10224) 2021-10-30 15:56:15 +00:00
Carlos Mocholí 81d15c5986
Implement double optimizer closure for hook structure consistency (#10167) 2021-10-29 13:03:04 +00:00
Ning dbfadedfe7
Revert "Add support for `len(datamodule)` (#9895)" (#10072)
This reverts commit 6429de8944.
2021-10-29 13:33:51 +02:00
Carlos Mocholí dbe1662dc3
Replace `_TORCH_GREATER_EQUAL_DEV_1_10` with `_TORCH_GREATER_EQUAL_1_10` (#10157) 2021-10-27 13:38:39 +01:00
jjenniferdai 6d79184ec5
Unify checkpoint load paths [redo #9693] (#10061) 2021-10-25 19:05:31 +00:00
Carlos Mocholí b376799430
Minor fixes related to clipping (#10130)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-10-25 16:40:22 +00:00
Adrian Wälchli 7eb2edf421
rename set_random_master_port (#10104)
Co-authored-by: tchaton <thomas@grid.ai>
2021-10-25 12:09:05 +00:00
Danielle Pintz e94dcf6936
Mark `trainer.data_connector` as protected (#10031)
Co-authored-by: tchaton <thomas@grid.ai>
2021-10-25 12:29:09 +01:00
Kaushik B c3614f1c07
Fix: skip importing DistributedOptimizer for Windows (#10071) 2021-10-21 21:01:56 +00:00
Carlos Mocholí 6aeebf1bd3
Remove unnecessary dependency available checks (#10050) 2021-10-20 16:21:37 +00:00
thomas chaton 1759403c8d
Add check for callable with datamodule len (#10003)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-10-19 14:51:08 +00:00
Carlos Mocholí 3f355d0eb7
Remove manual tracking of optimizer steps (#9957) 2021-10-18 12:43:06 +00:00
Kaushik B 5e8829b97d
(1/n) tests: Use strategy flag instead of accelerator for training strategies (#9931)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-10-16 20:40:25 +05:30
kingyiusuen 6429de8944
Add support for `len(datamodule)` (#9895)
Co-authored-by: tchaton <thomas@grid.ai>
2021-10-15 14:19:50 +02:00
Rohit Gupta 23e8b59ae7
Add `configure_gradient_clipping` hook in `LightningModule` (#9584)
* init hook

* docs

* dep train args

* update tests

* doc

* doc

* .gitignore

* not dep

* add trainer args

* add & update tests

* fix tests

* pre-commit

* docs

* add docs

* add exception

* code review

* deepspeed

* update tests

* not

* try fix

* Apply suggestions from code review

* update deepspeed

* disable some tests

* disable some tests

* enable all tests
2021-10-13 20:15:13 +05:30
ananthsub 28fc8d2016
Add `enable_model_summary` flag and deprecate `weights_summary` (#9699)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
2021-10-13 17:20:54 +05:30
Rohit Gupta 4decbc0d95
Deprecate `dataloader_idx` from `on_train_batch_start/end` (#9816)
* deprecate hooks

* dep todo

* explicit

* Apply suggestions from code review

* Apply suggestions from code review

* code review

* base
2021-10-07 10:18:11 +00:00
Carlos Mocholí 8f1c855d97
Fix `ResultCollection._get_cache` with multielement tensors (#9582) 2021-09-22 14:03:20 +01:00
Carlos Mocholí 48d3a10c9b
Add `OutputResult` [1/2] (#9437)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-09-14 15:48:27 +02:00
Carlos Mocholí 9eccb3148e
Loop and test restructuring (#9383) 2021-09-10 13:18:24 +00:00
Carlos Mocholí e0f2e041b9
Share the training step output data via `ClosureResult` (#9349) 2021-09-10 11:40:20 +00:00
Carlos Mocholí 15d943089d
Enforce that the optimizer closure is executed when `optimizer_step` is overridden (#9360) 2021-09-08 12:24:57 +02:00
Jirka Borovec 6e124e7207
CI: precommit - docformatter (#8584)
* CI: precommit - docformatter
* fix deprecated

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-09-06 12:49:09 +00:00
Roger Shieh 904dde7573
Fix inspection of unspecified args for container hparams (#9125)
* Update parsing.py

* add todo (for single arg)

* unblock non container single arg

* init test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update CHANGELOG.md

* pep8 line length

* Update pytorch_lightning/utilities/parsing.py

* remove dict namespace conversion

* add omegaconf support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add dict test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add omegaconf test

* Update CHANGELOG.md

* Update pytorch_lightning/utilities/parsing.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/utilities/parsing.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-09-06 09:48:11 +00:00
Carlos Mocholí 73fca23bed
Add typing for `ResultCollection` [3/3] (#9271) 2021-09-06 09:34:40 +00:00
B. Kerim Tshimanga f0788b3bbc
scheduled removal of auto_move_data decorator (#9231)
* scheduled removal of auto_move_data decorator

* update CHANGELOG.md

* remove unused import

* remove test_decorators.py

* fix missed merge conflict

Co-authored-by: thomas chaton <thomas@grid.ai>
2021-09-03 00:54:36 +02:00
Himanshu Dutta 5fbf04a145
DataModule compatiblity with Python dataclass (#9039)
* added support and checks required for use of datamodule as python dataclass
* made changes required for dataclass support for LightningDataModule and required tests
* made the code compliant with future releases
* edited tests - removed training call. left dataclass decorator to defaults.
* added tests to check for multilevel inheritence and make sure init isn't called on the parent of defined class
* modified __new__ to ensure calling of init on LightningDataModule impliciltly
* added relevant tests for multilevel inheritence cases
* removed default values from tests

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-09-03 00:43:38 +02:00
B. Kerim Tshimanga c993d0ce33
Make unimplemented dataloader hooks raise `NotImplementedError` (#9161) 2021-08-28 16:07:47 +00:00
thomas chaton 045c879e08
Fix `self.log(sync_dist=True, reduce_fx={mean,max})` (#9142)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-08-27 15:40:51 +00:00
Adrian Wälchli 6592d0e454
generalize closure api in Lightning (#8642)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-08-26 08:36:21 +00:00
Yifu Wang 3d5e71a767
Add `ShardedTensor` support in `LightningModule` (#8944)
* Add `ShardedTensor` support in `LightningModule`

Co-authored-by: Yifu Wang <yifuwang@2012@gmail.com>
Co-authored-by: Sean Naren <sean@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
2021-08-23 19:59:38 +00:00
Ning 2481816490
Deprecate `prepare_data_per_node` flag on Trainer and set it as a property for DataHooks (#8958)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-08-23 12:43:45 +00:00
Carlos Mocholí e1442d247e
Always use `trainer.call_hook` (#8498) 2021-08-20 18:22:03 +02:00
Adrian Wälchli 522df2b89b
3/n integrate new LightningDataFetcher into loop (#8953)
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2021-08-17 21:42:22 +00:00
Carlos Mocholí d0efb55b0f
Delete `TrainingEpochLoop._dataloader_idx` which always equals 0 (#8911) 2021-08-16 13:34:42 +02:00
ananthsub 037a86c873
Remove write_predictions from LightningModule (#8850)
* Remove write_predictions from LightningModule
2021-08-14 02:00:23 +00:00
Carlos Mocholí ed13040729
Connect the model to the training type plugin at the start of run (#8536) 2021-08-04 17:43:34 +02:00
samlurye f90849cc95
Deprecate LightningModule.summarize() in favor of pl.utilities.model_summary.summarize() (#8513)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-08-03 22:08:51 +00:00
thomas chaton dd8216a6b8
Save the `ResultCollection` in the loops state dict (#8641)
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-08-02 20:52:24 +00:00
thomas chaton 15fb32037d
Test `metric_attribute` for different children module structures (#8675)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-08-02 20:51:15 +01:00
Carlos Mocholí 9720e264f5
Fix references for `ResultCollection.extra` and improve `str` and `repr` (#8622) 2021-07-30 12:47:34 +02:00
Carlos Mocholí e63968ab88
Add `pyupgrade` to `pre-commit` (#8557)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-26 14:38:12 +02:00
Carlos Mocholí a64cc37394
Replace `yapf` with `black` (#7783)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-26 13:37:35 +02:00
Carlos Mocholí 321689f52e
Add `ModelCheckpoint(save_on_train_epoch_end)` (#8389)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-07-13 14:47:59 +00:00
Tilman Krokotsch 09ff295177
Hyperparameters for datamodule (#3792)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Tilman Krokotsch <tilman.krokotsch@iav.de>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
2021-07-09 15:10:00 +00:00
Sidhant Sundrani 20df24d2a2
Enables reload of dataloaders on every n epochs from every epoch (#5043)
* edit arg to reload_dataloaders_every_n_epoch

* init reload_dataloaders_every_n_epoch

* edit logic to reload dl

* update arg to test datamodule

* update arg test dataloader

* edit reload dl logic in eval loop

* fix var name in reset_train_val_dataloaders

* fix error, use current_epoch attribute

* edit every_n_epoch to every_n_epochs

* edit every_n_epoch to every_n_epochs

* edit every_n_epoch to every_n_epochs

* edit every_n_epoch to every_n_epochs

* edit every_n_epoch to every_n_epochs

* edit every_n_epoch to every_n_epochs

* assert reload_dataloaders_every_n_epochs positive

* assert reload_dataloaders_every_n_epochs positive

* add trainer property should reload dl

* update should reload dl in train loop

* condition on should reload dl in eval loop

* pep8

* fix update should reload dl in train loop

* add test case

* replace assertion with misconfig exception

* remove unused variable

* remove unnecessary checks

* replace to BoringModel

* remove unrequired comment

* deprecate _every_epoch

* add deprecated argument to trainer

* test case for deprecated arg

* remove unrequired assertion in train loop

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* modify misconfig exception for int

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* conv bool to int of depreciated _every_epoch

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* update description of deprecated param

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* update deprecation warning

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* modify argument to int only

* fix deprecated test function name

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* merge tests for reload dls

* add propery should reload dl

* removed and added to trainer property

* use property in train loop

* remove deprecated test

* add deprecated test to new file

* test case for exception

* update test datamodule every_n_epochs

* update trainer docs

* update hooks with every_n_epochs

* edit format if statement

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update CHANGELOG.md

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* typo in exception

* pytest check only misconfig exception

* remove unnecessary code in test

* remove unnecessary code in deprec test

* added match in test

* typo in comment

* revert to prev, keep only req in context manager

* Apply suggestions from code review

* docs

* rebase

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix import: model_helpers instead of model_utils

* fix, add reload_dataloaders_every_n_epochs argument to data connector

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add required imports

* move deprecated log

* add missing import rank_zero_warn

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update varname in should_reload_dl_epoch

suggestion from code review

* Fix CHANGELOG. Update deprecation versions

* Minor change

* change property name, mark protected

* update property name

* update property name

* Remove deprecated *_loop.py files

* Rename test func

* Update CHANGELOG.md

* use rank_zero_deprecation

* update deprecation message in trainer api docs

* test deprecation with real arg name in message

* fix typo in trainer docs

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-07-07 13:10:08 +02:00
Adrian Wälchli ced2c94a3e
fix missing call to untoggle_optimizer when accumulating gradients (#8284)
* add fix

* toggle test

* re-structure

* update changelog

* update comment

* remove debugging assertion
2021-07-05 11:59:04 +00:00
Palermo 36b893c43e
Add `ModelSummary.max_depth` (#8062)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-07-01 12:08:16 +02:00
thomas chaton 24db914093
Support state restoration of logged results 2/2(#7966)
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-06-25 19:16:11 +00:00