Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,491 Commits 59 Branches 217 Tags 353 MiB
Commit Graph

207 Commits

Author SHA1 Message Date
Carlos Mocholí c18a0ec819
Remove untested NVIDIA Dali example (#16306) 2023-01-10 14:11:08 +01:00
Yi Heng Lim 6092de9d99
Fix import error in `test_pl_examples.py` (#16268) 
Fixes https://github.com/Lightning-AI/lightning/issues/16265
 2023-01-06 15:37:16 +00:00
Carlos Mocholí 0a928e8ead
Reuse code in demos.BoringModel (#16242) 2023-01-06 09:47:08 +01:00
Nikhil Shenoy 560422692e
Check broken links (#16168) 
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-225-81.ubcsecure.wireless.ubc.ca>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-227-53.ubcsecure.wireless.ubc.ca>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-226-199.ubcsecure.wireless.ubc.ca>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-227-171.ubcsecure.wireless.ubc.ca>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-230-32.ubcsecure.wireless.ubc.ca>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-224-226.ubcsecure.wireless.ubc.ca>
Co-authored-by: Nikhil Shenoy <nikhilshenoy@dhcp-128-189-229-137.ubcsecure.wireless.ubc.ca>
Fixes https://github.com/Lightning-AI/lightning/issues/8107
 2023-01-06 09:30:13 +01:00
Adrian Wälchli b0c272e8b7
Restructure Lite examples and add GAN (#16240) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2023-01-05 14:07:43 +00:00
pre-commit-ci[bot] b59941cc52
[pre-commit.ci] pre-commit suggestions (#16224) 
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/asottile/pyupgrade: v2.34.0 → v3.3.1](https://github.com/asottile/pyupgrade/compare/v2.34.0...v3.3.1)
- https://github.com/myint/docformatter → https://github.com/PyCQA/docformatter
- [github.com/PyCQA/docformatter: v1.4 → v1.5.1](https://github.com/PyCQA/docformatter/compare/v1.4...v1.5.1)
- [github.com/asottile/yesqa: v1.3.0 → v1.4.0](https://github.com/asottile/yesqa/compare/v1.3.0...v1.4.0)
- [github.com/PyCQA/isort: 5.10.1 → 5.11.4](https://github.com/PyCQA/isort/compare/5.10.1...5.11.4)
- [github.com/psf/black: 22.6.0 → 22.12.0](https://github.com/psf/black/compare/22.6.0...22.12.0)
- [github.com/executablebooks/mdformat: 0.7.14 → 0.7.16](https://github.com/executablebooks/mdformat/compare/0.7.14...0.7.16)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-01-04 18:48:35 -05:00
Carlos Mocholí 15ef52bc73
Rename LightningLite to Fabric (#16244) 
* Rename LightningLite to Fabric

* Fix introspection test

* Fix deprecated Lite tests

* Undo accidental Horovod removal

* Fixes
 2023-01-04 10:57:18 -05:00
Shashwat Agrawal 574a951601
docs: updated broken links (#16191) 
Co-authored-by: Shashwat <shashwat>
Fixes https://github.com/Lightning-AI/lightning/issues/16186
 2022-12-24 04:18:58 +01:00
Carlos Mocholí 8c265c5239
Remove the deprecated `pl.utilities.cli` module (#16116) 2022-12-19 23:24:25 +01:00
Ethan Harris 2a85d9b257
[App] Add status endpoint, enable `ready` (#16075) 
Co-authored-by: thomas chaton <thomas@grid.ai>
 2022-12-19 14:10:58 +00:00
Sherin Thomas 0fd3d54205
[App] Scale out/in interval for autoscaler (#16093) 
* Adding arguments for scale out/in interval

* Tests
 2022-12-19 13:49:00 +00:00
Akihiro Nitta 39d27f6370
[App] Improve the autoscaler UI (#16063) 
[App] Improve the autoscaler UI (#16063)
 2022-12-19 10:25:24 +00:00
thomas chaton 22b254f491
[App] Add display name property to the work (#16095) 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2022-12-18 12:54:15 +00:00
Carlos Mocholí b1ce2639f4
Better check for programmatic lightningignore (#16080) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-12-16 09:49:17 +00:00
Qiushi Pan 385e5e2a40
minor fix: indent spaces in comment-out (#16076) 2022-12-15 17:44:10 +01:00
thomas chaton e522a12d17
[App] Improve lightning connect experience (#16035) 2022-12-14 17:09:30 +00:00
Akihiro Nitta a72d268a51
Avoid using the same port number for autoscaler works (#15966) 
* dont hardcode port in python server
* add another chglog
 2022-12-09 10:56:26 +00:00
thomas chaton 482b2799f4
[App] Improve pdb for multiprocessing (#15950) 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-12-08 13:58:57 +00:00
Ethan Harris e250dfe2b3
[App] Remove `SingleProcessRuntime` (#15933) 
* Remove SingleProcessRuntime
* Remove unused queues
* Docs
 2022-12-07 23:22:40 +00:00
Rick Izzo 20419089b6
ENG-627: Docs for CloudCompute Mount Argument (#15182) 
fixed conflicts
 2022-12-07 15:11:52 +00:00
Akihiro Nitta 64b19fb16f
[App] Introduce auto scaler (#15769) 
* Exlucde __pycache__ in setuptools

* Add load balancer example

* wip

* Update example

* rename

* remove prints

* _LoadBalancer -> LoadBalancer

* AutoScaler(work)

* change var name

* remove locust

* Update docs

* include autoscaler in api ref

* docs typo

* docs typo

* docs typo

* docs typo

* remove unused loadtest

* remove unused device_type

* clean up

* clean up

* clean up

* Add docstring

* type

* env vars to args

* expose an API for users to override to customise autoscaling logic

* update example

* comment

* udpate var name

* fix scale mechanism and clean up

* Update exampl

* ignore mypy

* Add test file

* .

* update impl and update tests

* Update changlog

* .

* revert docs

* update test

* update state to keep calling 'flow.run()'

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>

* Add aiohttp to base requirements

* Update docs

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* Use deserializer utility

* fake trigger

* wip: protect /system/* with basic auth

* read password at runtime

* Change env var name

* import torch as optional

* Don't overcreate works

* simplify imports

* Update example

* aiohttp

* Add work_args work_kwargs

* More docs

* remove FIXME

* Apply Jirka's suggestions

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean example device

* add comment on init threshold value

* bad merge

* nit: logging format

* {in,out}put_schema -> {in,out}put_type

* lowercase

* docs on seconds

* process_time -> processing_time

* Dont modify work state from flow

* Update tests

* worker_url -> endpoint

* fix exampl

* Fix default scale logic

* Fix default scale logic

* Fix num_pending_works

* Update num_pending_works

* Fix bug creating too many works

* Remove up/downscale_threshold args

* Update example

* Add typing

* Fix example in docstring

* Fix default scale logic

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename method

* rename locvar

* Add todo

* docs ci

* docs ci

* asdfafsdasdf pls docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* .

* doc

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"

This reverts commit 24983a0a5a.

* Revert "Update src/lightning_app/components/auto_scaler.py"

This reverts commit 56ea78b45f.

* Remove redefinition

* Remove load balancer run blocker

* raise RuntimeError

* remove has_sent

* lower the default timeout_batching from 10 to 1

* remove debug

* update the default timeout_batching

* .

* tighten condition

* fix endpoint

* typo in runtimeerror cond

* async lock update severs

* add a test

* {in,out}put_type typing

* Update examples/app_server_with_auto_scaler/app.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update .actions/setup_tools.py

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>
 2022-12-07 14:27:44 +01:00
thomas chaton d2a8fbf6dd
[App] Enable running with spawn context (#15923) 2022-12-07 03:01:31 +00:00
thomas chaton 852089e056
[App] Add ready property to the flow (#15921) 2022-12-06 17:25:42 +00:00
Kaushik B 2b61c92ceb
Fix the `examples/app_dag` App (#14359) 
* Fix app dag example
* Add test
* Update doc
* Update tests/tests_app_examples/test_app_dag.py

Co-authored-by: Sherin Thomas <sherin@grid.ai>
 2022-11-22 09:39:44 +01:00
Ethan Harris bc797fd376
[App] Fix multi-node pytorch example CI (#15753) 2022-11-21 16:02:30 +00:00
Ethan Harris 83067977af
[App] Update multi-node examples (#15700) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2022-11-21 10:41:59 +00:00
thomas chaton 757413c8e0
[App] Accelerate Multi Node Startup Time (#15650) 2022-11-11 20:08:16 +00:00
thomas chaton 61c1f699ea
[App] Enable state broadcast with MultiNode (#15607) 2022-11-11 10:06:40 +00:00
William Falcon d5c0effcfb
Docs 4/n (#15628) 
* remove source-lit

* docs

* docs

* docs

* docs

* ic

* deploy

* deploy

* deploy

* deploy

* deploy

* deploy

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* make build run

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rick Izzo <rick@grid.ai>
 2022-11-10 14:48:48 -05:00
Sherin Thomas 136a090f99
Sample datatype for Serve Component (#15623) 
* introducing serve component

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean up tests

* clean up tests

* doctest

* mypy

* structure-fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cleanup

* cleanup

* test fix

* addition

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* requirements

* getting future url

* url for local

* sample data typeg

* changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* prediction

* updates

* updates

* manifest

* fix type error

* fixed test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rick Izzo <rick@grid.ai>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2022-11-10 14:39:36 -05:00
thomas chaton 7ec15ae43a
[App] Rename to new convention (#15621) 
* update

* update
 2022-11-10 16:19:18 +01:00
Rick Izzo bdd27991eb
Fix: App comment command execution sequencing (#15615) 
* fixed condition in which app command comments would not execute before the flow is dispatched

* remove debug print statment

* updated example code

* fix test

* updates to test

Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-11-10 07:43:04 -05:00
thomas chaton 47314eafbd
[App] Resolve multi-node cloud bug (#15619) 
update
 2022-11-10 07:42:19 -05:00
Adrian Wälchli 2076746238
Update Lightning Lite examples (#15599) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2022-11-10 04:16:46 -05:00
thomas chaton f06de83634
[App] Enable debugger with LightningApp (#15590) 2022-11-09 20:46:31 +00:00
thomas chaton 5079ea06cb
[App] Resolve Multi Node examples (#15605) 
* update

* update

* update

* update
 2022-11-09 19:00:17 +01:00
Rick Izzo 10b945bcc1
Run Command from App Comments (#15577) 
* initial work

* this seems to work well

* added example test

* updated docs & logging

* fixed errors

* fix typing error

* now using the --setup flag to decide if we should execute app comment commands or not

* updated tests

* added tests

* added test to ci

* fixed failing tests

* code review

* updates
 2022-11-09 09:45:38 -05:00
thomas chaton f9a65731cd
[App] Expose Run Work Executor (#15561) 2022-11-08 12:55:31 +00:00
thomas chaton 820233176b
[App] Fixed Multi Node and add examples (#15557) 2022-11-07 09:36:41 +00:00
thomas chaton ecc8ac07c6
[App] Introduce Multi Node Component (#15524) 2022-11-04 17:41:59 +00:00
thomas chaton 2e72a4c801
Add support for functions (#15098) 2022-10-28 15:06:45 +00:00
Ethan Harris bbf7848a5f
[App] Fix cluster logic (#15383) 2022-10-28 15:35:21 +01:00
Ethan Harris e9a6b83437
[App] Reduce import depths and add test (#15330) 
Co-authored-by: thomas chaton <thomas@grid.ai>
 2022-10-28 13:57:35 +00:00
thomas chaton df4b705768
Add JustPy Frontend (#15002) 
* update

* update

* update

* update

* changelog

* update

* update

* update

* update

* update

* update

* update

* update

* uipdate

* update

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2022-10-27 11:48:26 -04:00
thomas chaton bd658441bd
[App] testing `lightning` in `lightning-app` package (#15286) 
* hot fix

* update

* update

* update

* cmd

* pkg

* update

* Apply suggestions from code review

* update

* update

* Apply suggestions from code review

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* cleaning

* Apply suggestions from code review

* update

* update

Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2022-10-25 09:24:20 -04:00
Adrian Wälchli 82a75c80d5
Fixes to the K-fold loop example (#15225) 2022-10-22 01:13:25 +02:00
Rick Izzo b6f01ab010
Renamed Mount root_dir Argument to mount_path (#15228) 
* renamed Mount argument

* fix tests

* Apply suggestions from code review

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* updated examples as well

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
 2022-10-20 17:33:35 -04:00
thomas chaton 6a72a15a62
[App] Automate missing requirements installation for CLI (#15198) 
* update

* update

* update

* update

* update

* update

* update

* wording

Co-authored-by: Mansy <ahmed.mansy156@gmail.com>

* update

* update

* update

* update

* update

* update

* update

* update

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Mansy <ahmed.mansy156@gmail.com>
 2022-10-20 15:02:13 -04:00
Rick Izzo 7a827b09bb
Separate the concept of a Drive from that of a Mount (#15120) 
* added mount class and configured it into compute config

* added mount to the cloud runtime dispatcher

* raise error if s3 bucket is passed to a drive telling the user to utilize mounts

* added example for app

* udpated tests

* updated tests

* addressed code review comments

* fix bug

* bugfix

* updates'

* code review comments

* updates

* fixed tests after rename

* fix tests
 2022-10-19 23:24:27 -04:00
Ethan Harris 4acb10f981
Add support for command descriptions (#15193) 2022-10-19 17:34:35 +01:00