Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,304 Commits 55 Branches 212 Tags 373 MiB
Commit Graph

3264 Commits

Author SHA1 Message Date
Carlos Mocholí af59c8d1f7
Sync with master changes (#16248) 
* Rename LightningLite to Fabric (#16244)

* Rename LightningLite to Fabric

* Fix introspection test

* Fix deprecated Lite tests

* Undo accidental Horovod removal

* Fixes

* Update Lightning Lite docs (1/n) (#16250)

* fabric docs

* fix reference

* fabric

* Fix Fabric CHANGELOG (#16247)

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2023-01-04 12:57:49 -05:00
Carlos Mocholí 174c32a9f1
Sync with master changes (#16230) 
* [App] Introduce basic auth to Lightning CLI (#16105)

* Introduce basic auth to Lightning CLI for app creation

* Parsing creds added

* Adding auth field to app instance body

* Adding tests

* Adding changelog entry

* Adding more tests


* Update runtime.py

* Setting auth on update

* Fix test

* Update lightning-cloud dep

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update runtime.py

* Fix for release

* Update base.txt

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* docs: fix order of on_fit_start() hook (#16180)

Fixes https://github.com/Lightning-AI/lightning/issues/16170

* docs: updated broken links (#16191)

Co-authored-by: Shashwat <shashwat>
Fixes https://github.com/Lightning-AI/lightning/issues/16186

* Always use the local rank zero imports (#16178)

* update BYOC documentation with AWS details (#16044)

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Using internal ip + port in a load balancer instead of URL exposed (#16119)

Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove CUDA_LAUNCH_BLOCKING from Lite tests (#16177)

* simplify torch.Tensor (#16190)

* Fix inclusion of `model_parallel` document (#16197)

fix link to gpu/advanced section

* Update numpy requirement from <1.23.1,>=1.17.2 to >=1.17.2,<1.24.1 in /requirements (#16199)

Update numpy requirement in /requirements

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](https://github.com/numpy/numpy/compare/v1.17.2...v1.24.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update s3fs requirement from <2022.8.3,>=2022.5.0 to >=2022.5.0,<2022.11.1 in /requirements (#16198)

Update s3fs requirement in /requirements

Updates the requirements on [s3fs](https://github.com/fsspec/s3fs) to permit the latest version.
- [Release notes](https://github.com/fsspec/s3fs/releases)
- [Changelog](https://github.com/fsspec/s3fs/blob/main/release-procedure.md)
- [Commits](https://github.com/fsspec/s3fs/compare/2022.5.0...2022.11.0)

---
updated-dependencies:
- dependency-name: s3fs
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ci: upload only with release (#16194)

* Update endpoint access examples: added info about accessing auth-protected apps (#16145)

* Skip a failing Bagua test for manual optimization (#16225)

* Fix type error when dividing chunk size in colossalai strategy (#16212)


Co-authored-by: awaelchli <aedu.waelchli@gmail.com>

* Fix DDP on XLA (#16020)

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>

* Added support and test for custom artifact names in WandbLogger (#16173)

* Update ipython[all] requirement from <8.6.1 to <8.7.1 in /requirements (#16220)

Updates the requirements on [ipython[all]](https://github.com/ipython/ipython) to permit the latest version.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](https://github.com/ipython/ipython/compare/rel-0.8.4...8.7.0)

---
updated-dependencies:
- dependency-name: ipython[all]
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improvements to checkpoint migration (#16233)

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Support arbitrary `Optimizable`s as optimizers (#16189)

* Differences with master

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dmitry Frolov <dmitsf@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: David Gilbertson <gilbertson.david@gmail.com>
Co-authored-by: Shashwat Agrawal <72117025+ShashwatAgrawal20@users.noreply.github.com>
Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: HELSON <c2h214748@gmail.com>
Co-authored-by: Liyang90 <liyanglu@google.com>
Co-authored-by: Manan Goel <manangoel1999@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
 2023-01-04 10:58:30 -05:00
Adrian Wälchli 053d4a03f8
Remove deprecated `auto_select_gpus` Trainer argument (#16184) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-12-24 04:32:10 +00:00
Adrian Wälchli a4c73d5106
Remove the deprecated `resume_from_checkpoint` Trainer argument (#16167) 2022-12-22 21:12:44 +00:00
Adrian Wälchli 732fdf97ee
Remove legacy device arguments in Trainer (#16171) 2022-12-22 20:14:39 +00:00
Carlos Mocholí 1ad68e5796
Sync with master changes (#16169) 2022-12-22 20:24:34 +01:00
Carlos Mocholí 4f021982a8
Add `amp_scaling_state` (apex) migration (#16161) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Fixes https://github.com/Lightning-AI/lightning/pull/16149#discussion_r1054271661
 2022-12-22 14:02:17 +01:00
Carlos Mocholí accfbc177c
Remove horovod (#16150) 2022-12-21 12:22:06 +00:00
Carlos Mocholí e544676ff4
Remove nivida/apex (#16149) 2022-12-21 12:14:58 +01:00
Carlos Mocholí 6ed2472a23
Sync debug branch with master changes (#16133) 2022-12-21 02:15:28 +00:00
Lightning Forever b52a5949bf
Move Logger utilities from PL to Lite (#16144) 2022-12-20 17:25:24 +00:00
Adrian Wälchli 76a00c3fd5
Rewrite logger utility tests without Trainer dependency (#16132) 2022-12-20 16:13:25 +00:00
Adrian Wälchli 9f05f49ae3
Sync debug branch with master changes (#16115) 
* Remove the deprecated profiler imports (#16059)

* Revert "Load app before setting LIGHTNING_DISPATCHED" (#16064)

Revert "Load app before setting LIGHTNING_DISPATCHED (#16057)"

This reverts commit 8d3339a0e9.

* [App] Hot fix: Resolve detection of python debugger (#16068)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Load the app before setting `LIGHTNING_DISPATCHED` (#16071)

* fix(cloud): detect and ignore venv (#16056)

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Add function to remove checkpoint to allow override for extended classes (#16067)

* Drop FairScale sharded parity tests (#16069)

* minor fix: indent spaces in comment-out (#16076)

* ci: print existing candidates (#16077)

* [App] Fix bug where previously deleted apps cannot be re-run from the CLI (#16082)

* Better check for programmatic lightningignore (#16080)

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [App] Removing single quote (#16079)

* [App] PoC: Add support for Request (#16047)

* Have checkgroup pull the latest runs (#16033)

* Update Multinode Warning (#16091)

* [App] Serve datatypes with better client code (#16018)

* docs: add PT version (#16010)

* docs: add PT version

* stable

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* add 1.13.1 to adjust versions (#16099)

* Remove redundant `find_unused_parameters=False` in Lite (#16026)

* [App] Add display name property to the work (#16095)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>

* Fix detection of whether app is running in cloud (#16045)

* [App] Add work.delete (#16103)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>

* [App] Improve the autoscaler UI (#16063)

[App] Improve the autoscaler UI (#16063)

* Re-enable Lite CLI on Windows + PyTorch 1.13 (#15645)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* [App] Min replica=0 would break autoscaler component (#16092)

* fixing the bug where num_replica=0 would fail

* changelog

* [App] Scale out/in interval for autoscaler (#16093)

* Adding arguments for scale out/in interval

* Tests

* Set the default work start method to spawn on MacOS (#16089)

* [App] Add status endpoint, enable `ready` (#16075)

Co-authored-by: thomas chaton <thomas@grid.ai>

* Clarify `work.stop()` limitation (#16073)

* fix merge errors

* Update torchvision requirement from <=0.14.0,>=0.11.1 to >=0.11.1,<0.15.0 in /requirements (#16108)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>

* CI: settle file names (#16098)

* CI: settle file names

* rename

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix test failing on master due to bad auto-merge (#16118)

* fix merge error

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Sean Naren <snarenthiran@nvidia.com>
Co-authored-by: Qiushi Pan <17402261+qqpann@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2022-12-20 09:08:27 -05:00
lightningforever 0fc7b82b8f
LightningModule support for Lite (#16048) 2022-12-16 15:59:38 +00:00
lightningforever 33c95d9e8e
Callback support for Lite (#16074) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2022-12-16 15:09:37 +00:00
Adrian Wälchli 4c89b82c86
Sync debug branch with master changes (#16066) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Nikhil Shenoy <nikhilshenoy98@gmail.com>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-224-163.ubcsecure.wireless.ubc.ca>
Co-authored-by: Huy Đỗ <56794124+Al3xDo@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Luca Furst <rlfurst@gmail.com>
Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-12-15 16:22:12 +00:00
Adrian Wälchli 7fc4da9b8c
Sync debug branch with master changes (#16024) 2022-12-13 16:24:12 +00:00
Adrian Wälchli 787c32b27f
Rename LightningLite (7/n) (#15993) 2022-12-12 17:49:36 +00:00
Adrian Wälchli 2f7d167132
Rename LightningLite (4/n) (#15979) 2022-12-09 12:51:52 -05:00
Adrian Wälchli 7282d5c64e
Lite: Remove legacy code (#15953) 2022-12-09 17:17:28 +00:00
Adrian Wälchli deac2b5634
Sync debug branch with master (#15983) 
* Simplify enabling CPU offload in FSDP (#15832)


Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [App] Enable running with spawn context (#15923)

* Fix compiler support test (#15927)

* Enable back inference mode support with hpu & update links (#15918)

* Enable back inference mode support with hpu
* Remove unused
* Update document link and address comment

Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [App] Introduce auto scaler (#15769)

* Exlucde __pycache__ in setuptools

* Add load balancer example

* wip

* Update example

* rename

* remove prints

* _LoadBalancer -> LoadBalancer

* AutoScaler(work)

* change var name

* remove locust

* Update docs

* include autoscaler in api ref

* docs typo

* docs typo

* docs typo

* docs typo

* remove unused loadtest

* remove unused device_type

* clean up

* clean up

* clean up

* Add docstring

* type

* env vars to args

* expose an API for users to override to customise autoscaling logic

* update example

* comment

* udpate var name

* fix scale mechanism and clean up

* Update exampl

* ignore mypy

* Add test file

* .

* update impl and update tests

* Update changlog

* .

* revert docs

* update test

* update state to keep calling 'flow.run()'

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>

* Add aiohttp to base requirements

* Update docs

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* Use deserializer utility

* fake trigger

* wip: protect /system/* with basic auth

* read password at runtime

* Change env var name

* import torch as optional

* Don't overcreate works

* simplify imports

* Update example

* aiohttp

* Add work_args work_kwargs

* More docs

* remove FIXME

* Apply Jirka's suggestions

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean example device

* add comment on init threshold value

* bad merge

* nit: logging format

* {in,out}put_schema -> {in,out}put_type

* lowercase

* docs on seconds

* process_time -> processing_time

* Dont modify work state from flow

* Update tests

* worker_url -> endpoint

* fix exampl

* Fix default scale logic

* Fix default scale logic

* Fix num_pending_works

* Update num_pending_works

* Fix bug creating too many works

* Remove up/downscale_threshold args

* Update example

* Add typing

* Fix example in docstring

* Fix default scale logic

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename method

* rename locvar

* Add todo

* docs ci

* docs ci

* asdfafsdasdf pls docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* .

* doc

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"

This reverts commit 24983a0a5a.

* Revert "Update src/lightning_app/components/auto_scaler.py"

This reverts commit 56ea78b45f.

* Remove redefinition

* Remove load balancer run blocker

* raise RuntimeError

* remove has_sent

* lower the default timeout_batching from 10 to 1

* remove debug

* update the default timeout_batching

* .

* tighten condition

* fix endpoint

* typo in runtimeerror cond

* async lock update severs

* add a test

* {in,out}put_type typing

* Update examples/app_server_with_auto_scaler/app.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update .actions/setup_tools.py

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>

* ENG-627: Docs for CloudCompute Mount Argument (#15182)

fixed conflicts

* Fix LRScheduler import for PyTorch 2.0 (#15940)

* Fix LRScheduler import for PyTorch 2.0
* Add comment for posterity

* CI: fix pypi flow (#15944)

* CI: fixing pypi syntax (#15943)
* connect
* input

* [App] Remove `SingleProcessRuntime` (#15933)

* Remove SingleProcessRuntime
* Remove unused queues
* Docs

* [App] Fix bug when using structures with works (#15911)

* Fix bug when using structures with works
* Add test
* Update CHANGELOG.md

* [App] Wait for full file to be transferred in Path / Payload (#15934)

* Wait for full file to be transferred in Path / Payload
* Fixes

* [docs] Include all components in the API reference (#15805)

* Update docs

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Bump playwright from 1.27.1 to 1.28.0 in /requirements (#15903)

* Bump playwright from 1.27.1 to 1.28.0 in /requirements

Bumps [playwright](https://github.com/Microsoft/playwright-python) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/Microsoft/playwright-python/releases)
- [Commits](https://github.com/Microsoft/playwright-python/compare/v1.27.1...v1.28.0)

---
updated-dependencies:
- dependency-name: playwright
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* 1.28

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>

* [App] Add `configure_layout` method for works (#15926)

* Add `configure_layout` method for works
* Check for api access availability
* Updates from review
* Update CHANGELOG.md
* Apply suggestions from code review

Co-authored-by: Sherin Thomas <sherin@lightning.ai>

* Make gradients available for all_gather on TPU (#15003)

* Make gradients available for all_gather on TPU
* Modify switch and tests
* Apply suggestions from code review
* Modify tests
* Fix test
* Drop test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Don't try to aggregate `requirements/__pycache__/base.txt` in setuptools (#15775)

Exlucde __pycache__ in setuptools

* [App] Multiprocessing-safe work pickling (#15836)

* Upgrade to HPU release 1.7.1 (#15956)

* Upgrade to HPU release 1.7.1
Update torch version check for hpu

Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Multinode on MPS (#15748)

* Fix restarting attribute for lr finder
* update lite executor
* update trainer executor
* update spawn executor
* add multinode component tests
* add testing helpers
* add lite tests
* add trainer tests
* update changelog
* update trainer
* update workflow
* update tests
* debug
* add reason for skipif
* Apply suggestions from code review
* switch skipif

Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [App] Resolve PythonServer on M1 (#15949)


Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Lite: Fix DataLoader shuffling when using DistributedSampler (#15931)


Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [App] Temporarily disable ready (#15958)

* Fix restarting attribute for lr finder (#15620)

* [App] Improve pdb for multiprocessing (#15950)


Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [App] Improve debug triggering (#15951)

* [App] Add automatic conversion to structures (#15961)

* Make LightningModule torch.jit.script-able again (#15947)

* Make LightningModule torch.jit.script-able again
* remove skip

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* refactor: simplify Tensor import (#15959)

* Fix ImportErrors on Multinode if package not present (#15963)

* Fix typo in definition of world size in docs (#15954)

* [App] Enable running an app from the Gallery (#15941)


Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>

* Apply dynamo to training_step, validation_step, test_step, predict_step (#15957)

* Apply dynamo to training_step, validation_step, test_step, predict_step

* Add entry to CHANGELOG.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix merge conflict

* rename tpu workflow

Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: Rick Izzo <rlizzo@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: stekiri <stekiri@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2022-12-09 16:26:07 +00:00
Adrian Wälchli 9d1895778e
Rename LightningLite (3/n) (#15939) 
* rename class and brand name

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix references

* f401

* rename package

* update name

* update

* update

* update

* update

* debug tests

* revert examples source dir

* update tpu ci workflow

* update

* ci

* requirements/lite

* group-check

* ci

* fix setup

* fix import bug

* space

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-12-09 09:51:07 +01:00
Adrian Wälchli a2252a9459
Rename LightningLite (2/n) (#15938) 
* rename class and brand name
* fix references
* f401
* changelog

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-12-07 14:59:18 +00:00
Adrian Wälchli 4f8394e853
Rename LightningLite (1/n) (#15932) 2022-12-07 13:46:52 +00:00
thomas chaton 852089e056
[App] Add ready property to the flow (#15921) 2022-12-06 17:25:42 +00:00
Adrian Wälchli 05dbf48ad0
Activation checkpointing in FSDP without boilerplate (#15826) 
* initial
* input type
* checkpointing
* fsdp in pl
* all_close

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-12-06 15:45:33 +00:00
Luca Antiga 2992002beb
Direct support for compiled models (#15922) 
* Direct support for compiled models

* Update test

* Update src/pytorch_lightning/core/module.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
 2022-12-06 16:41:30 +01:00
Ethan Harris ab022ac60f
[App] Fix hanging CI (#15913) 2022-12-06 00:21:44 +00:00
Ethan Harris 32cf1faa07
[App] Support for headless apps (#15875) 
* Add `is_headless` when dispatching in the cloud

* Bump cloud version

* Add tests

* Dont open app page for headless apps locally

* Refactor

* Update CHANGELOG.md

* Support dynamic UIs at runtime

* Comments

* Fix

* Updates

* Fixes and cleanup

* Fix tests

* Dont open view page for headless apps

* Fix test, resolve URL the right way

* Remove launch

* Clean

* Cleanup tests

* Fixes

* Updates

* Add test

* Increase app cloud tests timeout

* Increase timeout

* Wait for running

* Revert timeouts

* Clean

* Dont update if it hasnt changed

* Increase timeout
 2022-12-05 16:58:22 -05:00
Rick Izzo b4d99e3cc1
Add CLI Command to Delete Lightning App (#15783) 
* initial work on deleting apps

* after PR review

* delete CLI working

* restructred to make tests easier

* revert manifest changes

* added changelog, fix mypy issue

* updates

* Update src/lightning_app/cli/cmd_apps.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Sherin Thomas <sherin@lightning.ai>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Sherin Thomas <sherin@lightning.ai>

* import typing

* adding tests

* finished adding tests

* addressed code review comments

* fix mypy error

* make mypy happy

* make mypy happy

* make mypy happy

* make mypy happy

* fix windows cli

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
 2022-12-05 22:51:32 +01:00
Raphael Randschau a82be2ffad
[CLI] drop name column from cluster list (#15721) 
* drop name column from cluster list

* change create cluster to accept id as well

* rename validator

* remove cluster name from logs

* fix merge with master

* more merge with master issues
 2022-12-02 19:27:24 -08:00
Jirka Borovec fee52f931f
unblock legacy checkpoints (#15798) 
* fixing legacy checkpoints
* Apply suggestions from code review

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
 2022-12-02 07:50:51 +01:00
Carlos Mocholí 1bc3a97e95
Remove checks for torch greater than 1.10 (#15846) 2022-12-01 21:41:22 +01:00
Luca Furst c5d3bba764
[App] Raise error when launching app on multiple clusters (#15484) 
* Error when running on multiple clusters

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert this in separate PR: keep this focused

* Improve testing

* fixup! Improve testing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pass flake8

* Update changelog

* Address PR feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused import

* Reword error message

* Error if running on cluster that doesn't exist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixup! Error if running on cluster that doesn't exist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unsued import

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-11-30 11:33:57 -05:00
Luca Furst 33e1f9345d
[App] Improve cluster creation / deletion experience (#15458) 
Cluster creation and deletion can take a long time. Instead of having these long running operations happen in the background, they should happen in the foreground. The advantage is that failures are brought to the users attention immediately, instead of the next time they decide to run `lightning list clusters`.

While the CLI waits for the cluster to run / delete, it will display cluster status changes to the user.

This PR also hides the `--enable-performance` and `--edit-before-creation` creation flags, as well as the `--force` deletion flag. They are either not frequently used (performance mode is expensive), or prone to misuse.


Co-authored-by: Neven Miculinic <neven.miculinic@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>
 2022-11-28 16:38:43 +00:00
Jirka Borovec ad4bd66df6
hotfix import torch (#15849) 
* fix import torch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* plugin

* fix

* skip

* patch require

* seed

* warn

* .

* ..

* skip True

* 0.0.3

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-11-28 14:58:29 +01:00
Raphael Randschau c786b3d862
[CLI] fix ssh listing stopped components (#15810) 
* [CLI] fix ssh listing stopped components
* update CHANGELOG
 2022-11-28 04:37:34 +01:00
Adrian Wälchli 657bfc586a
Fix device placement when setting up FSDP model in Lite (#15822) 
* fix
* debug test
* simplify

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-11-28 04:05:48 +01:00
Mauricio Villegas 3fad651f99
Deprecate description and env parameters in LightningCLI.__init__ (#15651) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
 2022-11-27 10:52:01 +00:00
Carlos Mocholí eaa71a2f1a
Do not modify MANIFEST.in on install (#15646) 2022-11-26 22:27:28 +00:00
Carlos Mocholí 2c3cc74c28
Warn when `self.log(..., logger=True)` is called without a logger (#15814) 2022-11-25 15:57:47 +01:00
Carlos Mocholí 0d98689168
Cleaner datadir management for some tests (#15791) 2022-11-25 10:41:27 +01:00
Ethan Harris f171657030
[App] Add utility to get install command for package extras (#15809) 2022-11-24 22:29:52 +00:00
Adrian Wälchli a970f090a0
Ignore `num_nodes` when running MultiNode components locally (#15806) 2022-11-24 17:21:32 +00:00
thomas chaton 8ca6dfe646
[App] Add CloudMultiProcessBackend to run an children App within the Flow in the cloud (#15800) 
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updte

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/lightning_app/CHANGELOG.md

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
 2022-11-24 16:36:37 +01:00
Luca Antiga 0a1273187a
Add code_dir argument to tracer run (#15771) 2022-11-24 14:49:06 +00:00
Carlos Mocholí 9e43604df5
Notify the user of ignored requirements (#15799) 2022-11-24 15:09:25 +01:00
thomas chaton 3a99a256d3
[App] Stop App when it has succeeded (#15801) 2022-11-24 11:27:34 +00:00
Akihiro Nitta b682d4cf0b
Remove `pytorch_lightning.profiler.{AbstractProfiler,BaseProfiler}` deprecated since v1.6 (#15637) 
* remove deprecated base profilers

* Update changelog

* remove import statement

* rip

* correct deprecation version

* update changelog

* Mark buried classes as private

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add typing

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-11-23 13:50:15 +00:00
Carlos Mocholí a9708105f7
Lazy import tensorboard (#15762) 2022-11-23 13:59:38 +01:00