Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,453 Commits 59 Branches 217 Tags 356 MiB
Commit Graph

119 Commits

Author SHA1 Message Date
Danielle Pintz bd13d392af
Add error handling for all trainer entry points (#8819) 
* [lightning] Ensure error handling works different trainer entry points
 2021-08-18 02:04:40 +00:00
Adrian Wälchli 522df2b89b
3/n integrate new LightningDataFetcher into loop (#8953) 
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-08-17 21:42:22 +00:00
thomas chaton 19136ac847
[Feat] 2/n Add Fault Tolerant Training to LightningFetcher (#8891) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-08-17 16:32:43 +00:00
thomas chaton 89156b7039
[1/n] Add LightningFetcher (#8890) 2021-08-16 16:02:10 +00:00
ananthsub b47e3ab7ce
Remove truncated_bptt_steps from Trainer constructor (#8825) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-08-11 03:26:01 +00:00
Carlos Mocholí cb2a8ed1b8
Add `LightningCLI(run=False|True)` (#8751) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-08-10 15:01:36 +02:00
Jirka Borovec 0778ffbe2e
Legacy: simple classif training (#8535) 
* simple_classif_training
* fix test
* pt1.6
* automate

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
 2021-08-10 08:13:31 +00:00
Rio H a7d207ce0f
Add tests for functions in utilities/data.py (#8785) 
* Add tests for utilities/data.py: test_has_iterable_dataset, test_has_len, test_get_len
 2021-08-10 07:39:00 +01:00
Carlos Mocholí f1cc6e3470
Restructure parsing flow in the `LightningCLI` (#8721) 2021-08-09 17:26:53 +02:00
Adrian Wälchli 963c267646
fix recursive call for `apply_to_collection(include_none=False)` (#8719) 2021-08-04 20:31:35 +02:00
Sean Naren 560a5c3fc5
Add functions to collate deepspeed zero 3 checkpoints (#8701) 2021-08-04 09:39:02 +00:00
samlurye f90849cc95
Deprecate LightningModule.summarize() in favor of pl.utilities.model_summary.summarize() (#8513) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-08-03 22:08:51 +00:00
thomas chaton 567e905ead
update logic to inject FastForwardSampler / CaptureIterableDataset 2/n (#8366) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-08-02 20:52:06 +00:00
Jirka Borovec f67892ea96
CI: yesqa (#8564) 
* add yesqa
* fix flake8

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-08-02 16:05:56 +00:00
Rio H ba8053492f
Deprecate LightningModule.model_size (#8495) 
Co-authored-by: Caleb Robinson <calebrob6@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-07-30 13:53:40 +00:00
Carlos Mocholí a64cc37394
Replace `yapf` with `black` (#7783) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-07-26 13:37:35 +02:00
Ethan Harris 52526c20b5
Add support for functions to be parsed by the Lightning CLI in addition to Types (#8400) 
* Initial commit

* Update docstrings

* Update CHANGELOG.md

* Fix mypy

* Fixes

* Fixes

* Update to comments

* Fix

* mypy

* Update on comments

* Update

* Fix mypy

* protected

Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-07-26 10:53:48 +02:00
Carlos Mocholí f7027a8701
Remove `torch >= 1.6` checks (#8523) 2021-07-23 04:03:20 +00:00
deepsource-autofix[bot] cfc04100f9
Remove unnecessary use of comprehension (#8471) 
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
 2021-07-19 15:43:28 +02:00
ramonemiliani93 c9eb7e4433
Hash values in LightningEnum instead of name. (#8421) 
Co-authored-by: Ramon Emiliani <ramon@mbp-de-ana.lan>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
 2021-07-19 15:02:22 +02:00
thomas chaton 374fae59ef
[Feat] Add utilities for CombinedLoader state dict and dataloader state dict 1/n (#8364) 
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-07-19 09:56:57 +00:00
thomas chaton 257fabd08d
Add support for missing return obj from to function on custom batch objects (#8433) 
* resolve bug

* update

* add changelog

* Update tests/utilities/test_apply_func.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-07-19 08:47:14 +00:00
deepsource-autofix[bot] cbf71d0a14
Remove unnecessary comprehension (#8405) 2021-07-19 08:30:24 +00:00
Kaushik B 9ed6783357
Move mixin to core (#8396) 
* Move mixin to core

* Move mixin to core
 2021-07-19 10:15:59 +02:00
Luis Perez 000fbe63d3
Expose `extract_batch_size` method and add corresponding tests. (#8357) 
* expose extract_batch and make public

* first pass

* early return

* add changelog

* move to utilities/data.py

* add test_data.py

* tests are passing

* precommit hook

* address pep8 failure

Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-07-13 11:35:10 +00:00
Mauricio Villegas 7d3452a000
LightningCLI documentation improvements (#8303) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-07-08 12:35:26 +05:30
thomas chaton 7956c6bd4b
[Feat] Add FastForwardSampler 2/n - Fault Tolerant Training (#8307) 
* wip

* update

* resolve bug

* wip

* wip

* wip

* resolved tests

* update on comments

* update

* update

* Update pytorch_lightning/utilities/auto_restart.py

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* update on comments

* Update pytorch_lightning/utilities/auto_restart.py

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* resolve bug

* update

* move properties to top

* update docs for fast forward sampler

* move public attribute to top

* add missing super call

* update docs for state_dict

* fix merge conflict

* add missing super() call

* move property to top

* update on comments

* update

* resolve bug

* update

* update on comments

* activate coverage for CaptureIterableDataset

* update on comments

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-07-07 20:21:21 +00:00
Carlos Mocholí 368ac1c622
[CLI] Drop `ArgumentParser` when pickling and save before spawning (#8017) 
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-07-07 17:56:13 +00:00
Carlos Mocholí 8fead58273
Add `functools.wraps` support for `is_overridden` (#8296) 2021-07-06 10:40:54 +02:00
Kaushik B 5cef9772a4
Add tests for GCS filesystem (#7946) 2021-07-05 12:16:32 +00:00
Mauricio Villegas 3c74502919
Add support for optimizers and learning rate schedulers to LightningCLI (#8093) 
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-07-01 12:04:11 +02:00
deepsource-autofix[bot] c3065c5ce9
Iterate dictionary directly (#8155) 
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
 2021-06-27 21:55:16 +02:00
nisheethlahoti 06f8349291
Support calling fit and test scripts using "python -m" module syntax with DDP (#8073) 
Co-authored-by: Nisheeth Lahoti <nisheeth@rephrase.ai>
 2021-06-23 02:42:04 +00:00
Carlos Mocholí dd340a6598
Actually show deprecation warnings and their line level [2/2] (#8002) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-21 18:51:53 +02:00
Carlos Mocholí d9bf9759fb
Add `LightningCLI(save_config_overwrite=False|True)` (#8059) 2021-06-21 17:58:02 +02:00
Mauricio Villegas 0004216f2f
Easier configurability of callbacks that should always be present in LightningCLI (#7964) 
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-16 02:03:37 +02:00
Mauricio Villegas b2e9fa814f
Improvements related to save of config file by LightningCLI (#7963) 
* - Exclude SaveConfigCallback for fast_dev_run=True.
- SaveConfigCallback give a clearer message if config file already exists.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* - Added unit test
- Added entry in changelog
- Improved save config docstring

* Fix log line

* Fixes

* Fix changelog entry

* Update pytorch_lightning/utilities/cli.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Suggested fixed change

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-15 23:26:39 +02:00
Yuanzheng Wang 59d0c65613
Add dataclass support to `apply_to_collection` (#7935) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-06-12 11:42:49 +00:00
Mauricio Villegas cdd01f32da
LightningCLI support for argument links applied on instantiation (#7895) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-12 13:13:14 +02:00
Carlos Mocholí ac4eb0a06a
`is_overridden` improvements (#7918) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-11 13:47:00 +02:00
Luis Perez 009e05d14f
[bugfix] Minor improvements to `apply_to_collection` and type signature of `log_dict` (#7851) 
* minor fixeS

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-06-07 09:31:36 +01:00
Mauricio Villegas f34584001c
Fix support for torch Module type hints in LightningCLI (#7807) 
* Fixed support for torch Module type hints in LightningCLI

* - Fix issue with serializing values when type hint is Any.
- Run unit test only on newer torchvision versions in which the base class is Module.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor change

* Update CHANGELOG.md

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-06-04 07:43:43 +02:00
Carlos Mocholí 195b24ba51
`apply_to_collection` improvements and add `apply_to_collections` (#7769) 
* `apply_to_collection` improvements and add `apply_to_collections`

* Update CHANGELOG

* Minor fix

* Minor fix

* Remove attr

* Swap is first is None

* None test

* OrderedDict support

* flake8

* Fix docstring
 2021-06-01 12:09:20 +00:00
Leonard Lausen 597b309f2e
Fix `Trainer.plugins` type declaration (#7288) 
* Fix trainer.plugins type declaration

* Don't ClusterEnvironment(Plugin)

* fix import error, yapf formatter

* Add test

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-05-04 08:42:57 +02:00
Adrian Wälchli 3e8db4142b
add forgotten test in #7240 (#7283) 
^
 2021-05-03 23:56:30 +00:00
Carlos Mocholí 40f80230fe
Remove `trainer.fit` return value [2/n] (#7237) 
* `_fit_impl` refactor and types

* Fix return

* Remove return docstring

* Fixes

* Fixes

* Remove `trainer.fit` return value

* Update CHANGELOG

* flake8

* Undo results change

* Fix test

* Revert changes for a separate PR

* flake8
 2021-04-28 19:11:32 +01:00
Mauricio Villegas 78d45a1134
Improve `LightningCLI` documentation and tests (#7156) 
* - Added cli unit tests for help, print_config and submodules.
- Added to cli documentation use of subclass help and print_config, submodules and other minor improvements.
- Increased minimum jsonargparse version required for new documented features.

* Improvements to lightning_cli.rst

* Add check for all trainer parameters in test_lightning_cli_help

* Increased minimum jsonargparse version

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-04-28 09:34:32 +01:00
thomas chaton e147127c0e
[feat] Add better support for predict + ddp 2/3 (#7215) 
* wip

* update

* update

* update

* update

* update

* typo

* update on comments

* update

* update

* update

* update

* update changelog

* update

* Fix merge

* Fix merge

* move code

* resolve test

* add extra test

* add an extra test

* update on comments

* add typing

* resolve flake8

* Refactor and Docs

* Fix tests

* Fix tests

* Fix tests

* Duplicate

* Fix tests

* resolve bug

* update

* update on comments

* update

* update changelog

* update

* update

* remove tpu

* resolve flake8

* update on comments

* update on comments

* update on comment

* resolve flake8

* add a cpu test for predict

* add None test

* update

* Update CHANGELOG.md

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* resolve tests

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-04-27 08:46:45 -04:00
Adrian Wälchli 3b36d81c03
Fixed `num_sanity_val_steps` affecting reproducibility of training data shuffling (#7014) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-04-27 09:51:39 +00:00
Carlos Mocholí 345e9a0245
Fix argparse docs (#7148) 2021-04-22 02:13:00 +02:00