Adrian Wälchli
|
9f05f49ae3
|
Sync debug branch with master changes (#16115)
* Remove the deprecated profiler imports (#16059)
* Revert "Load app before setting LIGHTNING_DISPATCHED" (#16064)
Revert "Load app before setting LIGHTNING_DISPATCHED (#16057)"
This reverts commit 8d3339a0e9.
* [App] Hot fix: Resolve detection of python debugger (#16068)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Load the app before setting `LIGHTNING_DISPATCHED` (#16071)
* fix(cloud): detect and ignore venv (#16056)
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* Add function to remove checkpoint to allow override for extended classes (#16067)
* Drop FairScale sharded parity tests (#16069)
* minor fix: indent spaces in comment-out (#16076)
* ci: print existing candidates (#16077)
* [App] Fix bug where previously deleted apps cannot be re-run from the CLI (#16082)
* Better check for programmatic lightningignore (#16080)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [App] Removing single quote (#16079)
* [App] PoC: Add support for Request (#16047)
* Have checkgroup pull the latest runs (#16033)
* Update Multinode Warning (#16091)
* [App] Serve datatypes with better client code (#16018)
* docs: add PT version (#16010)
* docs: add PT version
* stable
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* add 1.13.1 to adjust versions (#16099)
* Remove redundant `find_unused_parameters=False` in Lite (#16026)
* [App] Add display name property to the work (#16095)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
* Fix detection of whether app is running in cloud (#16045)
* [App] Add work.delete (#16103)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
* [App] Improve the autoscaler UI (#16063)
[App] Improve the autoscaler UI (#16063)
* Re-enable Lite CLI on Windows + PyTorch 1.13 (#15645)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* [App] Min replica=0 would break autoscaler component (#16092)
* fixing the bug where num_replica=0 would fail
* changelog
* [App] Scale out/in interval for autoscaler (#16093)
* Adding arguments for scale out/in interval
* Tests
* Set the default work start method to spawn on MacOS (#16089)
* [App] Add status endpoint, enable `ready` (#16075)
Co-authored-by: thomas chaton <thomas@grid.ai>
* Clarify `work.stop()` limitation (#16073)
* fix merge errors
* Update torchvision requirement from <=0.14.0,>=0.11.1 to >=0.11.1,<0.15.0 in /requirements (#16108)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
* CI: settle file names (#16098)
* CI: settle file names
* rename
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix test failing on master due to bad auto-merge (#16118)
* fix merge error
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Sean Naren <snarenthiran@nvidia.com>
Co-authored-by: Qiushi Pan <17402261+qqpann@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
|
2022-12-20 09:08:27 -05:00 |
|
Adrian Wälchli
|
deac2b5634
|
Sync debug branch with master (#15983)
* Simplify enabling CPU offload in FSDP (#15832)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [App] Enable running with spawn context (#15923)
* Fix compiler support test (#15927)
* Enable back inference mode support with hpu & update links (#15918)
* Enable back inference mode support with hpu
* Remove unused
* Update document link and address comment
Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Introduce auto scaler (#15769)
* Exlucde __pycache__ in setuptools
* Add load balancer example
* wip
* Update example
* rename
* remove prints
* _LoadBalancer -> LoadBalancer
* AutoScaler(work)
* change var name
* remove locust
* Update docs
* include autoscaler in api ref
* docs typo
* docs typo
* docs typo
* docs typo
* remove unused loadtest
* remove unused device_type
* clean up
* clean up
* clean up
* Add docstring
* type
* env vars to args
* expose an API for users to override to customise autoscaling logic
* update example
* comment
* udpate var name
* fix scale mechanism and clean up
* Update exampl
* ignore mypy
* Add test file
* .
* update impl and update tests
* Update changlog
* .
* revert docs
* update test
* update state to keep calling 'flow.run()'
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
* Add aiohttp to base requirements
* Update docs
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
* Use deserializer utility
* fake trigger
* wip: protect /system/* with basic auth
* read password at runtime
* Change env var name
* import torch as optional
* Don't overcreate works
* simplify imports
* Update example
* aiohttp
* Add work_args work_kwargs
* More docs
* remove FIXME
* Apply Jirka's suggestions
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clean example device
* add comment on init threshold value
* bad merge
* nit: logging format
* {in,out}put_schema -> {in,out}put_type
* lowercase
* docs on seconds
* process_time -> processing_time
* Dont modify work state from flow
* Update tests
* worker_url -> endpoint
* fix exampl
* Fix default scale logic
* Fix default scale logic
* Fix num_pending_works
* Update num_pending_works
* Fix bug creating too many works
* Remove up/downscale_threshold args
* Update example
* Add typing
* Fix example in docstring
* Fix default scale logic
* Update src/lightning_app/components/auto_scaler.py
Co-authored-by: Noha Alon <nohalon@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* rename method
* rename locvar
* Add todo
* docs ci
* docs ci
* asdfafsdasdf pls docs
* Apply suggestions from code review
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* .
* doc
* Update src/lightning_app/components/auto_scaler.py
Co-authored-by: Noha Alon <nohalon@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
This reverts commit 24983a0a5a.
* Revert "Update src/lightning_app/components/auto_scaler.py"
This reverts commit 56ea78b45f.
* Remove redefinition
* Remove load balancer run blocker
* raise RuntimeError
* remove has_sent
* lower the default timeout_batching from 10 to 1
* remove debug
* update the default timeout_batching
* .
* tighten condition
* fix endpoint
* typo in runtimeerror cond
* async lock update severs
* add a test
* {in,out}put_type typing
* Update examples/app_server_with_auto_scaler/app.py
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Update .actions/setup_tools.py
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>
* ENG-627: Docs for CloudCompute Mount Argument (#15182)
fixed conflicts
* Fix LRScheduler import for PyTorch 2.0 (#15940)
* Fix LRScheduler import for PyTorch 2.0
* Add comment for posterity
* CI: fix pypi flow (#15944)
* CI: fixing pypi syntax (#15943)
* connect
* input
* [App] Remove `SingleProcessRuntime` (#15933)
* Remove SingleProcessRuntime
* Remove unused queues
* Docs
* [App] Fix bug when using structures with works (#15911)
* Fix bug when using structures with works
* Add test
* Update CHANGELOG.md
* [App] Wait for full file to be transferred in Path / Payload (#15934)
* Wait for full file to be transferred in Path / Payload
* Fixes
* [docs] Include all components in the API reference (#15805)
* Update docs
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Bump playwright from 1.27.1 to 1.28.0 in /requirements (#15903)
* Bump playwright from 1.27.1 to 1.28.0 in /requirements
Bumps [playwright](https://github.com/Microsoft/playwright-python) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/Microsoft/playwright-python/releases)
- [Commits](https://github.com/Microsoft/playwright-python/compare/v1.27.1...v1.28.0)
---
updated-dependencies:
- dependency-name: playwright
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* 1.28
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
* [App] Add `configure_layout` method for works (#15926)
* Add `configure_layout` method for works
* Check for api access availability
* Updates from review
* Update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
* Make gradients available for all_gather on TPU (#15003)
* Make gradients available for all_gather on TPU
* Modify switch and tests
* Apply suggestions from code review
* Modify tests
* Fix test
* Drop test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Don't try to aggregate `requirements/__pycache__/base.txt` in setuptools (#15775)
Exlucde __pycache__ in setuptools
* [App] Multiprocessing-safe work pickling (#15836)
* Upgrade to HPU release 1.7.1 (#15956)
* Upgrade to HPU release 1.7.1
Update torch version check for hpu
Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Multinode on MPS (#15748)
* Fix restarting attribute for lr finder
* update lite executor
* update trainer executor
* update spawn executor
* add multinode component tests
* add testing helpers
* add lite tests
* add trainer tests
* update changelog
* update trainer
* update workflow
* update tests
* debug
* add reason for skipif
* Apply suggestions from code review
* switch skipif
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [App] Resolve PythonServer on M1 (#15949)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Lite: Fix DataLoader shuffling when using DistributedSampler (#15931)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Temporarily disable ready (#15958)
* Fix restarting attribute for lr finder (#15620)
* [App] Improve pdb for multiprocessing (#15950)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Improve debug triggering (#15951)
* [App] Add automatic conversion to structures (#15961)
* Make LightningModule torch.jit.script-able again (#15947)
* Make LightningModule torch.jit.script-able again
* remove skip
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* refactor: simplify Tensor import (#15959)
* Fix ImportErrors on Multinode if package not present (#15963)
* Fix typo in definition of world size in docs (#15954)
* [App] Enable running an app from the Gallery (#15941)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
* Apply dynamo to training_step, validation_step, test_step, predict_step (#15957)
* Apply dynamo to training_step, validation_step, test_step, predict_step
* Add entry to CHANGELOG.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix merge conflict
* rename tpu workflow
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: Rick Izzo <rlizzo@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: stekiri <stekiri@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2022-12-09 16:26:07 +00:00 |
|
Adrian Wälchli
|
9d1895778e
|
Rename LightningLite (3/n) (#15939)
* rename class and brand name
* update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix references
* f401
* rename package
* update name
* update
* update
* update
* update
* debug tests
* revert examples source dir
* update tpu ci workflow
* update
* ci
* requirements/lite
* group-check
* ci
* fix setup
* fix import bug
* space
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
|
2022-12-09 09:51:07 +01:00 |