23f0e20209
Fixed `WandbLogger` `save_dir` is not set after creation ( #12748 ) ( #14326 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-09-05 10:12:43 +00:00
ed0164a3d2
Estimate stepping batches with max_steps if max_epochs is not set ( #14317 )
...
Co-authored-by: Roberto Estevão <robertode@microsoft.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-09-05 09:05:21 +00:00
4235eff712
Use a standalone test symlink for Lite ( #14502 )
2022-09-04 20:57:28 +02:00
291dc1b615
Standalone Lite CI setup ( #14451 )
...
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-09-01 22:13:12 +00:00
e0c2c3e677
Clean up fairscale imports ( #14476 )
2022-09-01 18:08:40 +02:00
28e18881a9
Mark stage argument in hooks as required ( #14064 )
...
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2022-09-01 15:47:40 +02:00
1bcb5c301d
[App][CLI] Fix lightning cli --version ( #14433 )
...
* [App][CLI] Fix lightning cli --version
2022-09-01 07:58:09 +00:00
e90ac769d6
Reset dataloaders on failure in tuner ( #14372 )
2022-08-31 21:00:18 +00:00
a7c01c4754
Introduce lightning connect ( #14452 )
...
* update
* update
* update
* update
* Review of content
* Formatting updates
* Fomatting updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Updates based on new commits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* update
* update
* update
* update
* Introduce lightning connect (#14183 )
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Felonious-Spellfire <felonious.spellfire@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
2022-08-31 15:59:03 -04:00
2e3d85af84
Remove deprecated rank zero utilities ( #14471 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-31 18:29:11 +00:00
626827c872
update rng state save/load test to also run on cuda gpu ( #14396 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-31 16:36:35 +00:00
a1dd718781
Remove deprecated support for passing the warning category positionally ( #14470 )
2022-08-31 17:34:56 +02:00
ebc650bd95
E2E fix for custom base image ( #14468 )
...
* new custom base image
* image tag
2022-08-31 09:27:21 -04:00
9372981263
[CLI] Cluster logs CLI improvements: new log labels + test coverage increasing ( #14459 )
...
* Cluster logs improvements
* Unit tests added
* Labels for processing deletion errors
2022-08-30 22:22:01 -04:00
291267c3bf
Unify rank zero messaging utilities ( #14116 )
2022-08-30 09:51:30 +00:00
18e2a8eecd
PanelFrontend and Panel Web UI Intermediate docs ( #13531 )
...
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Marc Skov Madsen <masma@orsted.com>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
Co-authored-by: Felonious-Spellfire <felonious.spellfire@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Mansy <ahmed.mansy156@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2022-08-30 00:33:21 +00:00
d0d1818d50
Update `has_len_all_ranks` to use `Strategy.root_device` ( #12144 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-29 20:23:34 +00:00
f202e84f4b
Remove the legacy `get_deprecated_arg_names` ( #14415 )
2022-08-29 14:53:57 +02:00
1a3fe39571
Removed deprecated `Trainer.num_processes` property in favour of `Trainer.num_devices` ( #14423 )
...
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-28 23:59:24 +02:00
5cbe1f48d2
Removed the deprecated `Trainer.data_parallel_device_ids` function in favour of `Trainer.device_ids` ( #14422 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-28 18:07:00 +00:00
cea9a72d9d
Removed the deprecated the `trainer.lr_schedulers` ( #14408 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-28 18:06:09 +00:00
1e04951206
Remove deprecated `TrainerCallbackHookMixin` ( #14401 )
...
* remove deprecated callback hook
* changelog
2022-08-28 10:56:37 +00:00
f3574176e2
Change `trainer.should_stop` to not stop in between an epoch and run until `min_steps/min_epochs` only ( #13890 )
2022-08-27 12:12:24 +00:00
0e30e4a5a0
[App][CI] Fix psutil requirement CI ( #14413 )
2022-08-26 23:55:22 +00:00
250c06e406
Remove deprecated HPC model hooks ( #14315 )
...
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-26 20:59:32 +00:00
3ba0f56b18
Remove support for the deprecated torchtext legacy ( #14375 )
2022-08-26 20:01:51 +00:00
8950613552
save checkpoints and profiler output to the first logger ( #14325 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-26 17:23:54 +00:00
d4bcafad7a
Remove the deprecated loop output format ( #14373 )
2022-08-26 16:56:56 +00:00
ed84d04bcf
Fix mypy errors attributed to `pytorch_lightning.core.datamodule` ( #13693 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
Co-authored-by: otaj <ota@lightning.ai>
2022-08-26 16:26:26 +00:00
fafd254678
Fix device parser logic to avoid creating CUDA context ( #14319 )
...
* let environment disable forking
* add helper function and error messages
* tests
* changelog
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-26 15:41:38 +00:00
0102d0d4d4
Fix restoring trainer after `lr_find()` ( #14113 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-26 15:19:08 +00:00
94e567e6f0
Fix mypy errors attributed to `pytorch_lightning.trainer.connectors.data_connector.py` ( #13806 )
...
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
Co-authored-by: otaj <6065855+otaj@users.noreply.github.com>
2022-08-26 13:28:27 +00:00
e2221a0b3e
Raise an error when resuming training with Apex ( #14341 )
2022-08-26 13:11:24 +00:00
6d00f31f0c
Add auto wrapping for `DDPFullyShardedNativeStrategy` ( #14252 )
2022-08-26 09:01:48 +00:00
70deac2cd4
Reset epoch progress with batch size scaler ( #13846 )
...
Co-authored-by: Christian Schell <christian.schell@uni-wuerzburg.de>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-26 14:12:00 +05:30
e67842dcba
Support sharded optimizer state dumping outside of sharded strategies ( #14208 )
2022-08-26 07:58:21 +00:00
a01e016fff
Remove mps config for test ( #14379 )
...
* Remove mps config for test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-08-26 02:47:37 -04:00
33a5ed9879
Add torch.cuda rng state to seed save/load ( #14384 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-26 05:26:00 +00:00
807435885e
Fix `LightningDataModule` hparams parsing ( #12806 )
...
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-25 18:57:48 +00:00
2182d755c4
[CLI] Adding opportunity to see basic cluster logs ( #14334 )
...
* pinning starsessions
* pinning starsessions
* adding strict back to requirements.txt
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Duplicated
* Basic implementation
* Basic implementation
* Basic implementation
* Basic implementation
* Common things moved to log helpers file
* Decomposing logs reader classes for reusing
* Setting colors for log levels
* Manifest trimming
* Changes added to CHANGELOG
* Prettifications
* Prettifications
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Logs function name change
* Logs function name change
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* attempt to fix the pydanitc import
* Tests + command name fixes
* Extending tests
* Adding limit argument
* Unmerging CI fix
* Unmerging CI fix
* Adding fields for errors
* Adding log level fixed field width
* Adding absent typing + exeptions raising
* Adding socket error logging
* Addressing comments on cluster list function return value
* Addressing comments on adding e2e tests
* Adding version range for arrow package in reqs
* New unit tests
* arrow time parsing callback modified + unit tests
* helpers updated
* helpers updated
* helpers updated
* One more test
* CMD test fix
* CMD test fix
* CMD test fix
* CMD test fix
* CMD test fix
* LightningClient mocking
* Flaky test removed
Co-authored-by: hhsecond <sherin@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-23 15:48:22 -04:00
99ba95a38e
fix imports of collections.abc for py3.10 ( #14345 )
...
fix collections.abc for py3.10
Co-authored-by: Sherin Thomas <sherin@grid.ai>
2022-08-23 11:52:58 -04:00
7a617ec90e
Add back support for logging in the gradient clipping hooks ( #14298 )
...
* Add back support for logging in the gradient clipping hooks
* Docs and CHANGELOG
* Fix tests
2022-08-22 09:19:53 -04:00
db1835a82c
Fix an issue to avoid the impact of sanity check on `reload_dataloaders_every_n_epochs` for validation ( #13964 )
2022-08-21 23:55:03 +05:30
0b1a29b5eb
Remove references to local app admin view ( #14306 )
...
The local app '/admin' route is being removed because it does
not provide any value, so we need to remove references from it
in the CLI.
2022-08-19 12:29:44 -04:00
a8c6e69b43
Fix wrong num padding for RichProgressBar ( #14296 )
2022-08-19 09:40:44 +05:30
d71dba3721
BYOC: fix default types for cluster instance types ( #14260 )
2022-08-18 14:12:59 -07:00
d9c6090170
Deprecate `on_colab_kaggle` func ( #14247 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-18 18:34:21 +00:00
326f7565b0
Forward extra keyword arguments in `LightningDataModule.from_datasets` ( #14185 )
...
Co-authored-by: otaj <ota@lightning.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-18 14:06:39 +00:00
7879628a3a
Fix access to logger attribute when multiple loggers are used ( #14234 )
...
* Fix access to logger attribute when multiple loggers are used
* add changelog
2022-08-18 08:55:08 -04:00
e949362a6b
Enable `on_before_batch_transfer` for `DPStrategy` and `IPUAccelerator` ( #14023 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-18 12:12:29 +00:00
2e59c49592
Update defaults for WandbLogger's run name and project name ( #14145 )
2022-08-17 16:31:20 +00:00
44cdbcab04
Allowed setting attributes on `DataLoader` and `BatchSampler` when instantiated inside `*_dataloader` hooks ( #14212 )
2022-08-17 11:42:54 -04:00
7c329dd3ac
Warn when http URLs are configured ( #14233 )
...
* add a warning
* add test
* add test
* add changelog
* remove todo
* clarify http won't work in cloud
* Apply suggestions from code review
Co-authored-by: Sherin Thomas <sherin@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@grid.ai>
2022-08-17 10:17:11 -04:00
562d22f0c8
Fix install latest version of app/component ( #14181 )
...
* fix install latest version of app/component
* Add changelog
* a better testcase
* Update src/lightning_app/CHANGELOG.md
Co-authored-by: mansy <mansy@lightning.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-15 09:31:16 -04:00
48c23e5716
Use fsdp module to initialize precision scalar for fsdp native ( #14092 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
2022-08-13 07:52:06 +00:00
c8e22b4572
Avoid raising the sampler warning if num_replicas=1 ( #14097 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: otaj <6065855+otaj@users.noreply.github.com>
2022-08-12 08:44:21 +00:00
807f9d8c96
Replace unwrapping logic in strategies ( #13738 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-12 08:24:04 +00:00
6789a066b5
Avoid false positive warning about using `sync_dist` when using torchmetrics ( #14143 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-12 12:52:24 +05:30
2d9e00fab6
Profile batch transfer and gradient clipping hooks ( #14069 )
2022-08-11 23:21:53 +00:00
56533368af
Remove DeepSpeed version restriction from Lite ( #13967 )
2022-08-11 16:17:56 +00:00
3b18da3eaf
Fix saving hyperparameters in a composition where parent is not a LM or LDM ( #14151 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-11 15:49:46 +00:00
98ded4524f
[CLI] change cluster creation cost savings mode default ( #14132 )
...
* [CLI] change cluster creation cost savings mode default
instead of having customers opt-into cost savings mode, we'll ask them to opt-out of cost savings mode.
2022-08-11 08:19:21 -07:00
7c8c996f6a
Feature GRID-9731: Update Lightning Cloud.py Backend to Accept Drive Specs (2/2) ( #14106 )
...
initial work adding drives to create work API from framework cloud dispatcher
2022-08-11 13:22:59 +00:00
6eed72b621
(app) Introduce configure_api and Post, Get, Delete, Put HttpMethods ( #13945 )
2022-08-11 10:35:00 +00:00
3dc08b1ef5
Fix flaky test caused by weak reference ( #14157 )
2022-08-11 09:33:19 +02:00
a7cebf2416
Fix entry point test for Python 3.10 ( #14154 )
2022-08-11 01:32:32 +02:00
4008f9cd41
Convert subprocess test to standalone test ( #14101 )
2022-08-10 17:15:12 -04:00
5396b1899f
Resolve e2es V3 ( #14153 )
...
update
2022-08-10 16:34:23 -04:00
784b60412c
(app) Add s3 drive type (1/2) ( #14002 )
...
* Add S3 protocol and optimization field to the drive object
* Add a list of drives to the work specification
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add only protocol for s3 drives, no optimization arguments, and add tests
* added trailing slash criteria
* allow slash in s3 drives
* fix
* fixed test issues
Co-authored-by: Panos Lantavos-Stratigakis <default-email@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rick Izzo <rick@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rick Izzo <rlizzo@users.noreply.github.com>
2022-08-10 20:07:23 +00:00
e226180527
(app) Remove ClickRunner ( #14147 )
2022-08-10 17:26:01 +00:00
f132d44821
Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported ( #14117 )
2022-08-10 16:09:50 +00:00
9b61b1c482
Remove duplicated test classes ( #14122 )
...
Remove duplicated classes
2022-08-10 17:21:05 +02:00
b8b8f033fd
(app) Run the flow only if the state has updated 1/2 ( #14076 )
2022-08-10 14:56:41 +02:00
2f7daac4b8
Use websockets in e2es ( #14138 )
2022-08-10 13:17:29 +02:00
ddb476d334
[App] Application logs in CLI ( #13634 )
2022-08-10 09:48:06 +00:00
dc8ff5ed26
Fix device placement when `.cuda()` called without specifying index ( #14128 )
2022-08-10 05:23:20 -04:00
975a4fc2f1
Support checkpoint save and load with Stochastic Weight Averaging ( #9938 )
...
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Kushashwa Ravi Shrimali <kushashwaravishrimali@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-09 23:18:21 +00:00
06c255c5c1
Skip ddp fork tests on windows ( #14121 )
2022-08-09 22:54:10 +00:00
619c2ff058
[CLI] fix cluster creation CLI requiring instance-type selection ( #14056 )
...
fix cluster creation CLI requiring instace-type selection
we've marked `instance_types` as `required=False`, but the CLI calls `split` on the value.
So if nothing is provided, we'll actually receive a runtime error, effectively rendering the flag as required.
Co-authored-by: thomas chaton <thomas@grid.ai>
2022-08-09 12:17:57 -07:00
d85085479d
Reset all results on epoch end ( #14061 )
2022-08-09 23:01:11 +05:30
ac369f5570
Fix incorrect `precision="mixed"` being used with `DeepSpeedStrategy` and `IPUStrategy` ( #14041 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-09 21:25:23 +05:30
c55fe7105b
Prefix seed_everything log messages with rank info ( #14031 )
...
Co-authored-by: Anton Shevtsov <aeshevtsov@avito.ru>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-08-09 15:40:30 +02:00
0cfc53d6b4
Fix regression on default value for `find_unused_parameters` ( #14095 )
2022-08-09 13:56:02 +05:30
55ae812dbf
Resolve increased time. ( #14074 )
2022-08-08 15:48:50 +02:00
d072e4451a
Fix dtype inference during gradient norm computation ( #14051 )
2022-08-08 11:35:06 +00:00
b4ade232c8
Fix: Start Lightning App on Cloud if Repo Begins With Name "Lightning" ( #14025 )
2022-08-08 11:13:25 +00:00
aaeff90254
Remove deprecated `DistributedType` and `DeviceType` enum classes ( #14045 )
2022-08-08 10:07:54 +02:00
b25275ccc2
Cast to fp16 before moving to device with deepspeed ( #14000 )
2022-08-05 22:15:15 +00:00
26d69ceada
[CLI] add support for listing apps ( #13987 )
...
* add support for listing apps
* update changelog with correct PR number
* add tests for pagination
* fix wrong mock on test_cli
* ensure all enum values are accounted for
* make AppManager and AppList protected, add limit to pagination calls
* add restarting transition /w tests
* add state transition not yet run with tests
2022-08-05 13:42:00 -07:00
91dd6a68fb
Remove meta device utilities in favor of torchdistx ( #13868 )
2022-08-05 12:20:27 +00:00
3d5c3d24f9
Remove unused auto_collect_arguments class method ( #14015 )
2022-08-05 08:49:00 +00:00
a4e4cab7a6
Deprecate `amp_level` from `Trainer` ( #13898 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-05 08:31:19 +00:00
12a061f2aa
[App] Increased DeepDiff's verbose level to properly handle dict changes ( #13960 )
2022-08-05 07:57:00 +00:00
b88b700745
Remove the deprecated DDP2 strategy ( #14026 )
2022-08-04 20:27:35 +00:00
f5bd6e6f5f
Cast only floating types with IPUs ( #13983 )
2022-08-04 19:46:07 +00:00
341c63c2b9
[CLI] add support to run app on a specific cluster ( #13894 )
...
Add `--cluster-id` flag which can be passed to `lightning run app` if the `--cloud` flag is present.
This allows you to run your Lightning AI apps on Lightning AI BYOC clusters running on your own cloud provider infrastructure.
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
2022-08-04 10:48:29 -07:00
ef0623ec64
Remove deprecated training type plugins ( #14011 )
...
* Remove deprecated training type plugins
* update changelog
* DDP2Plugin
* Update src/pytorch_lightning/CHANGELOG.md
2022-08-04 18:00:00 +02:00
b8739a0167
Deprecate sheety API ( #14004 )
...
* deprecate sheety
Co-authored-by: manskx <ahmed.mansy156@gmail.com>
2022-08-04 16:25:41 +02:00
e78bf2044b
Raise an error if batch transfer hooks are overridden with IPUAccelerator ( #13961 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-04 12:04:42 +00:00
d748dae548
Fix erroneous warning for unset `max_epochs` ( #13262 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-03 19:17:21 +00:00
Tianshu Wang