Adrian Wälchli
|
deac2b5634
|
Sync debug branch with master (#15983)
* Simplify enabling CPU offload in FSDP (#15832)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [App] Enable running with spawn context (#15923)
* Fix compiler support test (#15927)
* Enable back inference mode support with hpu & update links (#15918)
* Enable back inference mode support with hpu
* Remove unused
* Update document link and address comment
Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Introduce auto scaler (#15769)
* Exlucde __pycache__ in setuptools
* Add load balancer example
* wip
* Update example
* rename
* remove prints
* _LoadBalancer -> LoadBalancer
* AutoScaler(work)
* change var name
* remove locust
* Update docs
* include autoscaler in api ref
* docs typo
* docs typo
* docs typo
* docs typo
* remove unused loadtest
* remove unused device_type
* clean up
* clean up
* clean up
* Add docstring
* type
* env vars to args
* expose an API for users to override to customise autoscaling logic
* update example
* comment
* udpate var name
* fix scale mechanism and clean up
* Update exampl
* ignore mypy
* Add test file
* .
* update impl and update tests
* Update changlog
* .
* revert docs
* update test
* update state to keep calling 'flow.run()'
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
* Add aiohttp to base requirements
* Update docs
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
* Use deserializer utility
* fake trigger
* wip: protect /system/* with basic auth
* read password at runtime
* Change env var name
* import torch as optional
* Don't overcreate works
* simplify imports
* Update example
* aiohttp
* Add work_args work_kwargs
* More docs
* remove FIXME
* Apply Jirka's suggestions
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clean example device
* add comment on init threshold value
* bad merge
* nit: logging format
* {in,out}put_schema -> {in,out}put_type
* lowercase
* docs on seconds
* process_time -> processing_time
* Dont modify work state from flow
* Update tests
* worker_url -> endpoint
* fix exampl
* Fix default scale logic
* Fix default scale logic
* Fix num_pending_works
* Update num_pending_works
* Fix bug creating too many works
* Remove up/downscale_threshold args
* Update example
* Add typing
* Fix example in docstring
* Fix default scale logic
* Update src/lightning_app/components/auto_scaler.py
Co-authored-by: Noha Alon <nohalon@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* rename method
* rename locvar
* Add todo
* docs ci
* docs ci
* asdfafsdasdf pls docs
* Apply suggestions from code review
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* .
* doc
* Update src/lightning_app/components/auto_scaler.py
Co-authored-by: Noha Alon <nohalon@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
This reverts commit 24983a0a5a.
* Revert "Update src/lightning_app/components/auto_scaler.py"
This reverts commit 56ea78b45f.
* Remove redefinition
* Remove load balancer run blocker
* raise RuntimeError
* remove has_sent
* lower the default timeout_batching from 10 to 1
* remove debug
* update the default timeout_batching
* .
* tighten condition
* fix endpoint
* typo in runtimeerror cond
* async lock update severs
* add a test
* {in,out}put_type typing
* Update examples/app_server_with_auto_scaler/app.py
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Update .actions/setup_tools.py
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>
* ENG-627: Docs for CloudCompute Mount Argument (#15182)
fixed conflicts
* Fix LRScheduler import for PyTorch 2.0 (#15940)
* Fix LRScheduler import for PyTorch 2.0
* Add comment for posterity
* CI: fix pypi flow (#15944)
* CI: fixing pypi syntax (#15943)
* connect
* input
* [App] Remove `SingleProcessRuntime` (#15933)
* Remove SingleProcessRuntime
* Remove unused queues
* Docs
* [App] Fix bug when using structures with works (#15911)
* Fix bug when using structures with works
* Add test
* Update CHANGELOG.md
* [App] Wait for full file to be transferred in Path / Payload (#15934)
* Wait for full file to be transferred in Path / Payload
* Fixes
* [docs] Include all components in the API reference (#15805)
* Update docs
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Bump playwright from 1.27.1 to 1.28.0 in /requirements (#15903)
* Bump playwright from 1.27.1 to 1.28.0 in /requirements
Bumps [playwright](https://github.com/Microsoft/playwright-python) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/Microsoft/playwright-python/releases)
- [Commits](https://github.com/Microsoft/playwright-python/compare/v1.27.1...v1.28.0)
---
updated-dependencies:
- dependency-name: playwright
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* 1.28
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
* [App] Add `configure_layout` method for works (#15926)
* Add `configure_layout` method for works
* Check for api access availability
* Updates from review
* Update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
* Make gradients available for all_gather on TPU (#15003)
* Make gradients available for all_gather on TPU
* Modify switch and tests
* Apply suggestions from code review
* Modify tests
* Fix test
* Drop test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* Don't try to aggregate `requirements/__pycache__/base.txt` in setuptools (#15775)
Exlucde __pycache__ in setuptools
* [App] Multiprocessing-safe work pickling (#15836)
* Upgrade to HPU release 1.7.1 (#15956)
* Upgrade to HPU release 1.7.1
Update torch version check for hpu
Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Multinode on MPS (#15748)
* Fix restarting attribute for lr finder
* update lite executor
* update trainer executor
* update spawn executor
* add multinode component tests
* add testing helpers
* add lite tests
* add trainer tests
* update changelog
* update trainer
* update workflow
* update tests
* debug
* add reason for skipif
* Apply suggestions from code review
* switch skipif
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
* [App] Resolve PythonServer on M1 (#15949)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Lite: Fix DataLoader shuffling when using DistributedSampler (#15931)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Temporarily disable ready (#15958)
* Fix restarting attribute for lr finder (#15620)
* [App] Improve pdb for multiprocessing (#15950)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [App] Improve debug triggering (#15951)
* [App] Add automatic conversion to structures (#15961)
* Make LightningModule torch.jit.script-able again (#15947)
* Make LightningModule torch.jit.script-able again
* remove skip
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* refactor: simplify Tensor import (#15959)
* Fix ImportErrors on Multinode if package not present (#15963)
* Fix typo in definition of world size in docs (#15954)
* [App] Enable running an app from the Gallery (#15941)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
* Apply dynamo to training_step, validation_step, test_step, predict_step (#15957)
* Apply dynamo to training_step, validation_step, test_step, predict_step
* Add entry to CHANGELOG.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix merge conflict
* rename tpu workflow
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: Rick Izzo <rlizzo@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: stekiri <stekiri@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2022-12-09 16:26:07 +00:00 |