Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,109 Commits 60 Branches 217 Tags 359 MiB
Commit Graph

824 Commits

Author SHA1 Message Date
Adrian Wälchli 9a64e534c7
Loop Refactor 4/N - Remove Old Evaluation Loop (#8056) 2021-06-22 11:57:37 +02:00
Adrian Wälchli 4becd1cf31
rename old `Trainer.train_loop` -> `Trainer.fit_loop` (#8025) 2021-06-22 11:49:32 +02:00
Adrian Wälchli 61e6e14ae2
update changelog after 1.3.7 (#8075) 2021-06-22 15:14:52 +05:30
Carlos Mocholí d9bf9759fb
Add `LightningCLI(save_config_overwrite=False|True)` (#8059) 2021-06-21 17:58:02 +02:00
simran2905 d1efae2e47
Fix checkpointed state for lr_schedulers with step interval (#7877) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-06-21 15:08:07 +00:00
Kaushik B 2303f9ced8
Fix(Early Stopping): move best score to device (#7959) 2021-06-21 15:41:41 +05:30
thomas chaton 651a0fbdeb
[bugfix] Properly name PyTorchProfiler traces (#8009) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-18 18:16:36 +00:00
Adrian Wälchli 0d6dfd42d8
Merge pull request #7990 from PyTorchLightning/refactor/loops/loops_everywhere_eval 
Loop Refactor 3/N - Evaluation Loop
 2021-06-18 08:54:59 -04:00
Carlos Mocholí a23a69965e
Deprecate returning extras with grads (#7994) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-18 13:05:37 +01:00
Carlos Mocholí cdcc483e9b
CHANGELOG update after v1.3.6 release (#7988) 2021-06-17 15:59:40 +00:00
David Chan c6e02e481e
[feat] Allow overriding optimizer_zero_grad and/or optimizer_step when using accumulate_grad_batches (#7980) 2021-06-17 12:50:37 +02:00
Adrian Wälchli eebdc910dd
progressive restoring of trainer state (#7652) 2021-06-17 08:13:53 +00:00
thomas chaton 3fece17ffb
[feat] Add `{,load_}state_dict` to `ResultCollection` 1/n (#7948) 
* add metric reload

* add tests

* update changelog

* udpate

* remove print

* remove attribute_name

* update

* update

* resolve test

* update on comments

* bypass typing bug

* update on comments

* Update CHANGELOG

* Update tests

* Update code

* Check if TODO persists

* Remove unrelated changes

* Fixes

* Revert "Check if TODO persists"

This reverts commit 68dac4ae69.

* Do not serialize dataclasses

* Avoid recostructing meta twice

* Keep previous sync_fn

* Move to device and map_location

* Fix bug

Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-17 08:08:22 +01:00
Austin Basye 906de2a7fa
[feat] Named Parameter Groups in `LearningRateMonitor` (#7987) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-17 03:13:54 +02:00
Carlos Mocholí bc2c2db2bf
Do not override the logged epoch in `logged_metrics` (#7982) 2021-06-16 13:36:58 +00:00
Carlos Mocholí 4ffba600c9
Add predict hook test (#7973) 2021-06-16 15:09:24 +02:00
thomas chaton d2983c7c51
[fix] Enable manual optimization DeepSpeed (#7970) 
* resolve manual optimization

* resolve manual optimization

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update changelog

* Simplify message

* Move from deprecated

* Split model parallel/manual model

* Use property

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: SeanNaren <sean@grid.ai>
 2021-06-16 09:25:41 +00:00
Adrian Wälchli b093a9e66d
Support `save_hyperparameters()` in LightningModule dataclass (#7992) 
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-16 10:30:58 +02:00
Adrian Wälchli 341adad819
Loop Refactor 2/N - Remove Old Training Loop (#7985) 
Co-authored-by: Justus Schock <justus.schock@rwth-aachen.de>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
 2021-06-16 09:00:33 +01:00
Yifu Wang b71aa55b9e
Make optimizers skippable when using amp (#7975) 
Co-authored-by: Yifu Wang <yifuwang@2012@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-16 00:23:30 +00:00
Mauricio Villegas 0004216f2f
Easier configurability of callbacks that should always be present in LightningCLI (#7964) 
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-16 02:03:37 +02:00
Mauricio Villegas b2e9fa814f
Improvements related to save of config file by LightningCLI (#7963) 
* - Exclude SaveConfigCallback for fast_dev_run=True.
- SaveConfigCallback give a clearer message if config file already exists.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* - Added unit test
- Added entry in changelog
- Improved save config docstring

* Fix log line

* Fixes

* Fix changelog entry

* Update pytorch_lightning/utilities/cli.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Suggested fixed change

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-15 23:26:39 +02:00
Adrian Wälchli 971908a1aa
Loop Refactor 1/N - Training Loop (#7871) 
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Justus Schock <justus.schock@rwth-aachen.de>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
 2021-06-15 12:55:06 +00:00
Carlos Mocholí 560b1970af
Standardize positional datamodule and argument names (#7431) 
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-15 11:50:13 +00:00
Sean Naren f7459f5328
DeepSpeed Infinity Update (#7234) 
* Update configs to match latest API

* Ensure we move the entire model to device before configure optimizer is called

* Add missing param

* Expose parameters

* Update references, drop local rank as it's now infered from the environment variable

* Fix ref

* Force install deepspeed 0.3.16

* Add guard for init

* Update pytorch_lightning/plugins/training_type/deepspeed.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Revert type checking

* Install master for CI for testing purposes

* Update CI

* Fix tests

* Add check

* Update versions

* Set precision

* Fix

* See if i can force upgrade

* Attempt to fix

* Drop

* Add changelog

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-14 16:38:28 +00:00
Dan Dale 3a0ed02bd4
Properly handle parent modules w/ parameters in `BaseFinetuning` callback (#7931) 
Co-authored-by: Daniel Dale <dan@distributedinsight.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-14 16:01:07 +00:00
Vatsalya Chaubey ce93d8bcfd
Handle errors due to uninitailized parameters (#7642) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-14 15:56:03 +00:00
Seppo Enarvi 22d826615f
Seed all workers when using DDP (#7942) 
* Seed all workers when using DDP

* Fix to dataloader seeding

* Make argument name explicit

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Use f-strings when logging

* Removed a redundant log message

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-14 14:39:50 +01:00
Carlos Mocholí 436fc53c89
Improve `LightningDataModule` hook test and fix `dataloader_idx` argument (#7941) 
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-06-14 12:42:13 +00:00
Adrian Wälchli 6b7b40473b
deprecate hpc_load() and integrate it with restore() (#7955) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-14 12:20:01 +00:00
Jirka Borovec f15ea6015e
update chlog + legacy chpt (#7954) 
* update chlog

* legacy
 2021-06-13 09:42:49 +05:30
Yuanzheng Wang 59d0c65613
Add dataclass support to `apply_to_collection` (#7935) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-06-12 11:42:49 +00:00
Mauricio Villegas cdd01f32da
LightningCLI support for argument links applied on instantiation (#7895) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-12 13:13:14 +02:00
Max Ehrlich 6856ccedfd
Remove rank_zero_only on DataModule prepare_data (#7945) 
Signed-off-by: Max Ehrlich <max.ehr@gmail.com>
 2021-06-12 12:50:29 +02:00
Sean Naren 96433d03ea
IPU Integration 5/5 (#7867) 
* Initial changes

* Add broken example for now

* Fix reference

* Fix format

* Code runs

* Fixes

* Clear up files

* Add tests, helpers, fixes

* Small cleanups

* Refactors based on review

* Swap to special tests

* Add special tests

* Add source

* Cleanups

* Add logic to attach/detach model from devices

* Fixes for tests

* Fixes for tests

* Move earlier

* Cleanups

* Add check for nvcc

* Add tests, cleanups

* Fix errors

* fix

* Try condition

* Add missing annotation

* Clearer

* Clearer message

* Fix variable

* Cleanups

* Add comment

* CHANGELOG.md

* Add simple selection test

* Remove special=True to see what happens

* Fix test

* Update tests/accelerators/test_ipu.py

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* Convert ipu_cores -> ipus

* Add typing, fail earlier

* simplify precision

* Add test, add helper

* fix accum

* Update pytorch_lightning/plugins/training_type/ipu.py

Co-authored-by: thomas chaton <thomas@grid.ai>

* Use stages

* Make sure warning message returned

* thorw error

* Add more tests, use fs

* add comment

* Clean

* Address feedback, add IPU tests

* Fixes

* Fix signature

* Add types

* Remove autoround

* Add docstring

* ipu_cores -> ipus

* Add test, remove unnecessary precision set

* Add optimizer test

* Add precision back with test

* Address code review

* Change to probs

* Move some of the asserts earlier

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
 2021-06-11 15:07:04 +00:00
Carlos Mocholí ac4eb0a06a
`is_overridden` improvements (#7918) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-11 13:47:00 +02:00
Burhanuddin Rangwala 8b73869369
Deprecate the default `EarlyStopping` callback monitor value (#7907) 
* removed monitor default value and added depreceation message

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* format change

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* requested changes

* added test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* format changes

* typehint change

* Update CHANGELOG.md

* requested changes

* regex

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-10 17:33:39 -07:00
Carlos Mocholí df812398b5
Clean-up after logger connector redesign 1/2 (#7909) 2021-06-10 06:21:03 +01:00
Carlos Mocholí ec4f8856af
Enable logger connector re-design (#7891) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-09 14:24:45 +00:00
ananthsub 6fee9262ff
Deprecate `LightningDataModule` lifecycle properties (#7657) 
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-09 12:30:40 +00:00
Carlos Mocholí b214442e74
New logger connector code (#7882) 
* New logger connector code

* Update CHANGELOG

* Update requirements

* Fix import path

* Add new suffix

* Tests

* Minor changes

* Rename and reorder

* Fix import

* Formatting

* Fix with seed_everything?

* Fix test?

* Fix test?

* Minor change

* Minor changes

* Minor changes

* Force float

* Fix minimal bug

* Fix minimal bug

* Update with latest changes

* Fix import

* bad merge

* update typing

Co-authored-by: tchaton <thomas@grid.ai>
 2021-06-08 20:20:17 +00:00
Carlos Mocholí b74f8ac149
Use `apply_to_collection` in `metrics_to_scalars` (#7888) 
* Use `apply_to_collection` in `metrics_to_scalars`

* Typing

* Update CHANGELOG

* Update pytorch_lightning/utilities/metrics.py

* Whitespace
 2021-06-08 12:54:32 -04:00
Carlos Mocholí 9d315be4df
Only track dev debugger events if enabled (#7875) 2021-06-08 12:11:20 +00:00
Carlos Mocholí 8cc55ebdb0
Add `log_grad_norm` hook to `LightningModule` (#7873) 2021-06-08 12:09:06 +01:00
Luis Perez f9fccdfb39
Move `training_output` validation to after `train_step_end` (#7868) 
* move validation to after aggregation

* changelog

* add test for training_step_end

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-06-08 08:37:50 +00:00
Carlos Mocholí 3427cb728d
Stricter `FxValidator` and add hooks (#7874) 
* Stricter FxValidator and add hooks

* Update CHANGELOG
 2021-06-08 08:26:05 +01:00
Adrian Wälchli 20f37b85b6
add warning when Trainer(log_every_n_steps) not well chosen (#7734) 
* add warning

* update changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* logger check

* add docstring for test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
 2021-06-07 12:40:43 +00:00
Sean Naren 41be61c6f2
[IPU] Add hooks for IPU lifecycle 4/5 (#7864) 2021-06-07 12:06:41 +00:00
Sean Naren 6388c29e87
[IPU] Add reset dataloader hooks to training type plugin 3/n (#7861) 
* Add hooks

* Add tests for hooks

* Add changelog

* Test changes, add typing
 2021-06-07 10:37:09 +00:00
thomas chaton d1becce4c1
[bugfix] Resolve LearningRateMonitor + BackboneFinetuning (#7835) 
* add test + resolve bug

* update changelog

* resolve bug

* resolve bug

* Update pytorch_lightning/callbacks/lr_monitor.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/lr_monitor.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* update on comments

* resolve comments

* update

* Update tests/callbacks/test_lr_monitor.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update pytorch_lightning/callbacks/lr_monitor.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-07 10:17:11 +00:00