Commit Graph

86 Commits

Author SHA1 Message Date
Danielle Pintz 160e7e1289
Deprecate LightningModule.get_progress_bar_dict (#8985)
* Move get_progress_bar_dict from lightning module to progress bar callback
2021-09-09 20:53:47 +00:00
Jirka Borovec 6e124e7207
CI: precommit - docformatter (#8584)
* CI: precommit - docformatter
* fix deprecated

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-09-06 12:49:09 +00:00
Carlos Mocholí 73fca23bed
Add typing for `ResultCollection` [3/3] (#9271) 2021-09-06 09:34:40 +00:00
thomas chaton f6d40871bd
Prevent loss to be moved to the cpu before backward call. (#9308) 2021-09-03 16:26:26 +00:00
Carlos Mocholí 171d242a89
Add typing for `_FxValidator` [1/3] (#9269) 2021-09-03 13:41:05 +00:00
Danielle Pintz b046bd0670
Add on_exception callback hook (#9183) 2021-09-01 10:49:00 +02:00
thomas chaton 045c879e08
Fix `self.log(sync_dist=True, reduce_fx={mean,max})` (#9142)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-08-27 15:40:51 +00:00
Nicki Skafte 81145ca990
Fig logging with log_gpu_memory='min_max' (#9013) 2021-08-24 15:00:59 +02:00
Carlos Mocholí e1442d247e
Always use `trainer.call_hook` (#8498) 2021-08-20 18:22:03 +02:00
thomas chaton f0a105bf3e
[bugfix] Resolve lost reference to meta object in `ResultMetricCollection` (#8932) 2021-08-16 19:21:03 +02:00
ananthsub c4a1c8ba20
Fix truncated backprop through time when set on LightningModule and not Trainer (#8804)
* Fix truncated backprop through time set on LightningModule and not Trainer
2021-08-09 21:23:05 -07:00
edward-io 8473cf44ec
Remove rank 0 restrictions from logger (#8608) 2021-08-06 04:13:56 +00:00
Binh Tang efec3d461c
Move logger and profiler finalization to trainer's teardown (#8685)
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-08-05 10:09:43 +02:00
Carlos Mocholí c2199fbbee
Fix `trainer.fit_loop.split_idx` reference (#8601)
* Fix split idx reference

* Update CHANGELOG

* Add comment
2021-07-29 08:00:04 +00:00
Adrian Wälchli 8c27fa71fa
[1 / 3] improvements to saving and loading callback state (#6886)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-29 00:12:32 +02:00
Jirka Borovec 0c0b24c031
Prune deprecated metrics (#8586)
* drop metrics

* drop tests

* fix imports

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-28 16:57:31 +00:00
Carlos Mocholí 47c47faeae
Remove `outputs` in `on_train_epoch_end` hooks (#8587) 2021-07-28 18:27:54 +02:00
Sean Naren aadd2a9d9c
Load ckpt path when model provided in validate/test/predict (#8352)
* Change trainer loading behaviour for validate/test/predict

* Fix

* Fix/add tests

* remove

* Cleanups

* Space

* cleanups

* Add CHANGELOG.md

* Move after setup

* Cleanups on logic

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remve

* fix test

* feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update pytorch_lightning/trainer/properties.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Feedback

* Same fix

* Same fix

* Add test for behaviour, modify based on feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Wording

* Apply suggestions from code review

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Cleanup docs

* Update pytorch_lightning/trainer/trainer.py

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* feedback

* Fixes to test API

* Add carlos description

* Move logic further

* Move checkpoint connector logic

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2021-07-28 10:12:46 +00:00
Carlos Mocholí e63968ab88
Add `pyupgrade` to `pre-commit` (#8557)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-26 14:38:12 +02:00
Carlos Mocholí a64cc37394
Replace `yapf` with `black` (#7783)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-26 13:37:35 +02:00
Dusan Drevicky 1b06edf2f2
Add the `on_before_optimizer_step` hook (#8048)
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-07-09 13:30:52 +02:00
thomas chaton 1c825a2a9c
Add the `on_before_backward` hook (#7865)
* Add callback to hook tests and add predict test

* Fix lambda callback test

* Simplify lambda call test

* Use LambdaCallback

* Dynamically append to called for the model

* Remove print

* Consistency

* Consistency

* Prepare args/kwargs testing

* yapf doesn't like dict literals

* Add arguments for fit no val test

* Add arguments for fit no val test

* add before_backward_hook

* add test

* resolve flake8

* resolve tests

* update changelog

* add on_before_backward to LightningModule

* update on comments

* Test arguments

* Datamodule refactor

* Fix eval test

* remove extra file

* resolve bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* move to hooks

* update

* resolve flake8

* update on comments

* Update full fit + val test

* Update test

* Remove FIXME

* Remove FIXME

* Undo change

* Fix

* Parametrize fit hook test

* Comment

* Parametrize fit hook test with different precision plugins

* Fix tests

* Parametrize fit hook test with manual optimization

* Unnecessary parenthesis

* WIP

* Comments

* Fix message

* Test CI error

* Revert "Test CI error"

This reverts commit 39c4a85a83.

* Add ddp training type teardown

* Update CHANGELOG

* Adrian's fix

* Use destructor

* Update CHANGELOG.md

* RPC destructor

* Update pytorch_lightning/plugins/training_type/ddp.py

* Why do you not work :(

* Missing condition

* Fix deepspeed test

* GC collect in conftest

* Do not show warnings for special tests

* Needs to run on 1.8

To avoid: "RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/ProcessGroupNCCL.cpp:32, unhandled cuda error, NCCL version 2.4.8"

* Run torch 1.8

* Skip test due to 'Python bus error'

* Debug NCCL

* shm size

* Disable warnings for special tests

* Remove NCCL_DEBUG statement

* Try smaller shm size

* Revert "Skip test due to 'Python bus error'"

This reverts commit e0a3e8785d.

* README and adjust versions

* Avoid self.on_gpu call

* empty cache cleanup

* More garbage collection

* Unroll parametrizations

* Do not reuse mock

* Undo changes

* Undo notebooks modification

* resolve test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* delete file

* Undo

* Fix test

* Revert "WIP"

This reverts commit f5828a8c42.

* Rename

* Remove optimizers

* Fix bug with LightningOptimizer

* Add optimizers

* update

* update

* Update CHANGELOG

* On after backward refactor

* Do not call super

* Fixes

* Remove should_accumulate

* pre/post backward refactor

* Call the LM backward hook

* Update tests

* Remove dev debug patch

* Fix test

* Remove optimizer arguments and typing

* Docs fixes

* Fix comment

* Undo changes

* Split manual and auto

* Undo change

* Deepsource

* Remove optimizers

* Undo changes

* Call the hook

* Docs

* Docs

Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-07-09 06:15:57 +00:00
Carlos Mocholí 9877265887
Simplify logger connector access (#8318) 2021-07-07 14:13:30 +02:00
Carlos Mocholí 8a7f504b6f
Detach hiddens and add test (#8249) 2021-07-02 14:03:12 +02:00
Carlos Mocholí 87b1b86e2f
Add missing logging tests (#8195) 2021-06-29 22:52:50 +00:00
thomas chaton 1f025789fc
[bugfix] Clean Validation Sanity Checking metrics (#8171)
* resolve logging issue

* update changelog

* remove breakpoint

* resolve bugs

* remove pass
2021-06-28 13:49:56 -04:00
thomas chaton c4492ad6aa
Merge pull request #8174 from PyTorchLightning/bugfix/8159_log_gpu_memory_on_step
[bugfix] Resolve memory not logged when missing metrics
2021-06-28 09:39:17 -04:00
deepsource-autofix[bot] 03154eb30a
Refactor unnecessary `else` / `elif` when `if` block has a `return` statement (#8156)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2021-06-28 15:27:41 +05:30
thomas chaton 24db914093
Support state restoration of logged results 2/2(#7966)
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-06-25 19:16:11 +00:00
Carlos Mocholí a23a69965e
Deprecate returning extras with grads (#7994)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-06-18 13:05:37 +01:00
Carlos Mocholí bc2c2db2bf
Do not override the logged epoch in `logged_metrics` (#7982) 2021-06-16 13:36:58 +00:00
Carlos Mocholí b45a89a256
Clean-up after logger connector redesign 2/2 (#7631)
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-06-10 12:09:01 +00:00
Carlos Mocholí ec4f8856af
Enable logger connector re-design (#7891)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-06-09 14:24:45 +00:00
Carlos Mocholí c310ce661e
Logger connector re-design `_Metadata.reduce_fx` fixes. (#7890) 2021-06-09 01:21:01 -07:00
Carlos Mocholí b214442e74
New logger connector code (#7882)
* New logger connector code

* Update CHANGELOG

* Update requirements

* Fix import path

* Add new suffix

* Tests

* Minor changes

* Rename and reorder

* Fix import

* Formatting

* Fix with seed_everything?

* Fix test?

* Fix test?

* Minor change

* Minor changes

* Minor changes

* Force float

* Fix minimal bug

* Fix minimal bug

* Update with latest changes

* Fix import

* bad merge

* update typing

Co-authored-by: tchaton <thomas@grid.ai>
2021-06-08 20:20:17 +00:00
Carlos Mocholí b74f8ac149
Use `apply_to_collection` in `metrics_to_scalars` (#7888)
* Use `apply_to_collection` in `metrics_to_scalars`

* Typing

* Update CHANGELOG

* Update pytorch_lightning/utilities/metrics.py

* Whitespace
2021-06-08 12:54:32 -04:00
Carlos Mocholí 5593b6f772
Merge pull request #7872 from PyTorchLightning/refactor/logger-poc-changes
Random fixes for logger connector PoC
2021-06-08 09:04:16 -04:00
Carlos Mocholí 8cc55ebdb0
Add `log_grad_norm` hook to `LightningModule` (#7873) 2021-06-08 12:09:06 +01:00
Carlos Mocholí 3427cb728d
Stricter `FxValidator` and add hooks (#7874)
* Stricter FxValidator and add hooks

* Update CHANGELOG
2021-06-08 08:26:05 +01:00
Carlos Mocholí 1dd61e4e35
Extend support for logging a collection (#7771) 2021-06-01 12:51:50 +01:00
Carlos Mocholí a69beab499
Clean existing logging tests (#7760)
* Remove dev debugger metric tracking

* Fix tests

* Fix test

* Import

* Clean logging tests

* flake8

* Docstring
2021-05-30 16:36:52 +02:00
Carlos Mocholí fa8f0363ee
Some test updates (#7761)
* Some test updates

* flake8
2021-05-30 13:15:25 +02:00
Carlos Mocholí bc3238be8c
Remove metric tracking from dev debugger (#7759)
* Remove dev debugger metric tracking

* Fix tests

* Fix test

* Import

* Fix tests

* Fix test

* flake8

* Fix tests
2021-05-30 12:03:42 +02:00
Carlos Mocholí 9304c0df8f
Rename and move Result (#7736)
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2021-05-27 12:27:52 +00:00
Carlos Mocholí 3d4dd28bec
Replace `CallbackHookNameValidator` with `FxValidator` [3/n] (#7627)
* Refactor FxValidator

* Fix tests

* Fix tests

* Class attribute

* Fix tests

* Better error message

* Fix tests

* Update pytorch_lightning/trainer/connectors/logger_connector/fx_validator.py
2021-05-21 11:54:16 +01:00
Carlos Mocholí 901b2bac98
Unify `current_fx_name` and `current_hook_fx_name` [2/n] (#7594)
* Minor loggger connector cleanup [1/n]

* Missing line

* Address comments

* Rely on validator

* Unify `current_fx_name` and `current_hook_fx_name`

* Fix test
2021-05-19 20:31:06 +00:00
Ethan Harris 45143fd825
Improve val step logging (#7351)
* Fix val step logging

* Add a type

* Fix

* Update CHANGELOG.md
2021-05-07 22:58:03 +00:00
ananthsub 7b45bcfedb
[2/2] Remove outputs from evaluation epoch end hooks (#7338)
* Remove outputs from on_train_epoch_end

* iterate

* Update callback_hook.py

* update

* early stop?

* fix

* Update pytorch_lightning/trainer/training_loop.py

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* Update trainer.py

* update

* Update training_loop.py

* early stop?

* fix

* Remove outputs from evaluation epoch end hooks

* update

* Update test_remove_1-5.py

* fix lints

* Update base.py

* rm-outputs

* Update evaluation_loop.py

* try-save-more-memory

* Update trainer.py

* Update trainer.py

* cache-at-start

* Update evaluation_loop.py

* Update training_loop.py

* Update training_loop.py

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
2021-05-05 19:50:58 +00:00
ananthsub 6104a6316a
[1/2] Deprecate `outputs` in `on_train_epoch_end` hooks (#7339)
* Remove outputs from on_train_epoch_end

* iterate

* Update callback_hook.py

* update

* Update training_loop.py

* Update test_training_loop.py

* early stop?

* fix

* update tests

* Update test_hooks.py

* Update pytorch_lightning/trainer/callback_hook.py

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* Update pytorch_lightning/trainer/training_loop.py

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* Update trainer.py

* Update pytorch_lightning/trainer/trainer.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-05-05 17:18:16 +02:00
thomas chaton e147127c0e
[feat] Add better support for predict + ddp 2/3 (#7215)
* wip

* update

* update

* update

* update

* update

* typo

* update on comments

* update

* update

* update

* update

* update changelog

* update

* Fix merge

* Fix merge

* move code

* resolve test

* add extra test

* add an extra test

* update on comments

* add typing

* resolve flake8

* Refactor and Docs

* Fix tests

* Fix tests

* Fix tests

* Duplicate

* Fix tests

* resolve bug

* update

* update on comments

* update

* update changelog

* update

* update

* remove tpu

* resolve flake8

* update on comments

* update on comments

* update on comment

* resolve flake8

* add a cpu test for predict

* add None test

* update

* Update CHANGELOG.md

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* resolve tests

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-04-27 08:46:45 -04:00