Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,594 Commits 59 Branches 217 Tags 353 MiB
Commit Graph

112 Commits

Author SHA1 Message Date
Jirka Borovec 3f73231d5c
ci/docs: enable not fetch assets (#18333) 2023-08-21 20:22:21 +02:00
Jirka Borovec cf8f8ab383
docs: fix dead links for App (#18332) 2023-08-17 17:48:45 +02:00
Jirka Borovec efa7b2f9ef
docformatter: config with black (#18064) 
* docformatter: config with black

* additional_dependencies: [tomli]

* 119

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-09 10:44:20 -04:00
Jirka Borovec 8f29bb561b
bump sphinx to 5.3 (#18204) 2023-08-08 15:32:34 +02:00
Adrian Wälchli 1db471305d
Avoid setting the multiprocessing context when importing lightning (#18177) 
* avoid import at top module

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove comment

* update docs

* changelog

* mypy

* trigger app tests

* can't import lightning on py 3.8

* Update .github/workflows/ci-tests-app.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2023-07-31 18:05:21 +02:00
Luca Antiga a7bbbcea9a
Remove cluster commands from CLI (#18151) 
* Remove cluster commands from CLI

* Update docs

* Remove index entry

* Revert accidental conversion to static assets

* Remove reference to deleted file

* Fix highlights

* Fix highlights

* Remove file

* Update CHANGELOG

* Fix dead link

* Increase upper bound on state size to reduce flakiness
 2023-07-28 00:16:21 +02:00
Jirka Borovec 6b52b84ef8
docs: fallback for restoring right scroll menu (#18108) 2023-07-18 14:06:18 +02:00
Jirka Borovec 00a0f2e8f4
docs: fetch external sources (#17941) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-07-03 20:16:45 +02:00
Jirka Borovec 76515140d8
docs: bump sphinx ver. (#17824) 
* docs: bump sphinx ver.

* myst-parser

* myst-parser

* myst-parser

* 6.2
 2023-06-15 09:37:04 -04:00
pre-commit-ci[bot] 91cb4b9b87
[pre-commit.ci] pre-commit suggestions (#17271) 
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/PyCQA/docformatter: v1.4 → v1.6.0](https://github.com/PyCQA/docformatter/compare/v1.4...v1.6.0)
- [github.com/psf/black: 22.12.0 → 23.3.0](https://github.com/psf/black/compare/22.12.0...23.3.0)
- [github.com/charliermarsh/ruff-pre-commit: v0.0.237 → v0.0.260](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.237...v0.0.260)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* apply

* fixing

* docs/lines

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2023-04-26 21:37:41 +02:00
Jirka Borovec e7ef8db57e
Replace HPU with external implementation (#17067) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2023-03-24 13:30:53 +01:00
Jirka Borovec 2f087ae30e
docs: update broken links & latest/stable (#16994) 
* docs: update links to PL latest

* also stable

* last

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* fixing

* .

* fabric

* fixing

* .

---------

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2023-03-15 20:19:41 +01:00
Jirka Borovec f25dfbe2b5
update links to Discord (#17087) 
* update links to Discord

* link

* Apply suggestions from code review

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* slack

* Update docs/source-app/levels/expert/index.rst

* Apply suggestions from code review

---------

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2023-03-15 13:33:40 +01:00
Adrian Wälchli 550fa5fec0
Organize app examples (#17045) 
* move

* init files

* reset change

* missed

* more

* relative path

* e2e tests

* update checkgroup

* reference

* update app_*
 2023-03-14 14:48:17 -04:00
Jirka Borovec 0e8ac7e1c9
docs: move fabric on its own (#16742) 
* docs: move fabric to Lai

* update imports

* links

* drop link to Trainer

* own docs

* ci

* trigger

* prune cross-links

* cleaning

* cleaning

* template

* imports

* template

* path

* links

* tensorboardX

* plugins

* label

* drop fixme

* drop copy nb + examples

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

* try again

* rev

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-03-01 12:36:14 +01:00
Jirka Borovec f697fff5db
docs: rename source-app (#16863) 
* docs: rename source-app

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ci

* group check

* trigger

* param

* fix

* cleaning

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-02-28 10:04:43 +01:00
Jirka Borovec 0be025e8b7
rename docs/source-app & adjust docs links for lightning (#16676) 
* update CI

* config / import

* lightning_app imports

* source/ dir

* html

* ci: dirs

* pr

* req dir

* on push

* rename

* drop

* cleaning
 2023-02-13 10:59:02 +01:00
Carlos Mocholí 21d8fbfb2f
Fix broken links after reverse mirror changes (#16600) 2023-02-01 20:00:44 +00:00
Adrian Wälchli c68cfd686e
Rename LiteMultiNode to FabricMultiNode (#16505) 2023-01-26 11:36:27 +00:00
Rick Izzo 0aef0375cb
Docs: Update BYOC IAM Policy Permissions (#16474) 
Added codebuild permissions to IAM policy JSON
 2023-01-25 16:27:15 +00:00
Shreya Purohit 139e2659f4
Fix minor typos in sharing_components.rst (#16468) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2023-01-25 15:15:45 +01:00
Jirka Borovec 3f365c8997
docs: move `assets_lightning` to pl-public-data (#16419) 2023-01-24 22:55:53 +00:00
Nash cc56539cd3
fixing typos reported by community user (#16457) 2023-01-21 01:21:45 +01:00
Jirka Borovec 799ced8430
ci: replace flake8 by ruff (#16433) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-01-19 11:48:28 -05:00
Akihiro Nitta fb12879fde
[docs][App] Include components in the API reference (#16414) 2023-01-18 09:06:21 +00:00
Justus Schock e61bead324
rename flow _exit (#16378) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2023-01-17 11:06:50 +00:00
Akihiro Nitta fa73327828
[App] Rename gradio.py gradio_server.py (#16201) 2023-01-14 13:00:37 +00:00
Raphael Randschau 1778c20e6f
update documentation with upcoming supported regions (#16331) 2023-01-13 14:12:17 +01:00
Akihiro Nitta 5de93975c6
Document how to run apps with a local version of Lightning on the cloud (#16163) 
Co-authored-by: Akihiro Nitta <akihiro@lightning.ai>
 2023-01-12 17:48:35 +00:00
pre-commit-ci[bot] b59941cc52
[pre-commit.ci] pre-commit suggestions (#16224) 
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/asottile/pyupgrade: v2.34.0 → v3.3.1](https://github.com/asottile/pyupgrade/compare/v2.34.0...v3.3.1)
- https://github.com/myint/docformatter → https://github.com/PyCQA/docformatter
- [github.com/PyCQA/docformatter: v1.4 → v1.5.1](https://github.com/PyCQA/docformatter/compare/v1.4...v1.5.1)
- [github.com/asottile/yesqa: v1.3.0 → v1.4.0](https://github.com/asottile/yesqa/compare/v1.3.0...v1.4.0)
- [github.com/PyCQA/isort: 5.10.1 → 5.11.4](https://github.com/PyCQA/isort/compare/5.10.1...5.11.4)
- [github.com/psf/black: 22.6.0 → 22.12.0](https://github.com/psf/black/compare/22.6.0...22.12.0)
- [github.com/executablebooks/mdformat: 0.7.14 → 0.7.16](https://github.com/executablebooks/mdformat/compare/0.7.14...0.7.16)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-01-04 18:48:35 -05:00
Raphael Randschau 5e1db28bf4
update BYOC documentation with AWS details (#16044) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2022-12-24 04:35:06 +01:00
Sherin Thomas 7cbdc68449
[App] Cold start proxy in autoscaler (#16094) 
* cold start proxy

* Update src/lightning_app/components/serve/auto_scaler.py

* changelog

* better-doc

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-12-20 09:47:10 +00:00
Akihiro Nitta 39d27f6370
[App] Improve the autoscaler UI (#16063) 
[App] Improve the autoscaler UI (#16063)
 2022-12-19 10:25:24 +00:00
Carlos Mocholí b1ce2639f4
Better check for programmatic lightningignore (#16080) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-12-16 09:49:17 +00:00
Carlos Mocholí edd2b4259a
Introduce `{Work,Flow}.lightningignore` (#15818) 2022-12-13 13:50:08 +00:00
Akihiro Nitta e6f4c8484b
[docs] Include all components in the API reference (#15805) 
* Update docs

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2022-12-08 04:38:27 +00:00
Ethan Harris e250dfe2b3
[App] Remove `SingleProcessRuntime` (#15933) 
* Remove SingleProcessRuntime
* Remove unused queues
* Docs
 2022-12-07 23:22:40 +00:00
Rick Izzo 20419089b6
ENG-627: Docs for CloudCompute Mount Argument (#15182) 
fixed conflicts
 2022-12-07 15:11:52 +00:00
Akihiro Nitta 64b19fb16f
[App] Introduce auto scaler (#15769) 
* Exlucde __pycache__ in setuptools

* Add load balancer example

* wip

* Update example

* rename

* remove prints

* _LoadBalancer -> LoadBalancer

* AutoScaler(work)

* change var name

* remove locust

* Update docs

* include autoscaler in api ref

* docs typo

* docs typo

* docs typo

* docs typo

* remove unused loadtest

* remove unused device_type

* clean up

* clean up

* clean up

* Add docstring

* type

* env vars to args

* expose an API for users to override to customise autoscaling logic

* update example

* comment

* udpate var name

* fix scale mechanism and clean up

* Update exampl

* ignore mypy

* Add test file

* .

* update impl and update tests

* Update changlog

* .

* revert docs

* update test

* update state to keep calling 'flow.run()'

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>

* Add aiohttp to base requirements

* Update docs

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* Use deserializer utility

* fake trigger

* wip: protect /system/* with basic auth

* read password at runtime

* Change env var name

* import torch as optional

* Don't overcreate works

* simplify imports

* Update example

* aiohttp

* Add work_args work_kwargs

* More docs

* remove FIXME

* Apply Jirka's suggestions

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean example device

* add comment on init threshold value

* bad merge

* nit: logging format

* {in,out}put_schema -> {in,out}put_type

* lowercase

* docs on seconds

* process_time -> processing_time

* Dont modify work state from flow

* Update tests

* worker_url -> endpoint

* fix exampl

* Fix default scale logic

* Fix default scale logic

* Fix num_pending_works

* Update num_pending_works

* Fix bug creating too many works

* Remove up/downscale_threshold args

* Update example

* Add typing

* Fix example in docstring

* Fix default scale logic

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename method

* rename locvar

* Add todo

* docs ci

* docs ci

* asdfafsdasdf pls docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* .

* doc

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"

This reverts commit 24983a0a5a.

* Revert "Update src/lightning_app/components/auto_scaler.py"

This reverts commit 56ea78b45f.

* Remove redefinition

* Remove load balancer run blocker

* raise RuntimeError

* remove has_sent

* lower the default timeout_batching from 10 to 1

* remove debug

* update the default timeout_batching

* .

* tighten condition

* fix endpoint

* typo in runtimeerror cond

* async lock update severs

* add a test

* {in,out}put_type typing

* Update examples/app_server_with_auto_scaler/app.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update .actions/setup_tools.py

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>
 2022-12-07 14:27:44 +01:00
Raphael Randschau a82be2ffad
[CLI] drop name column from cluster list (#15721) 
* drop name column from cluster list

* change create cluster to accept id as well

* rename validator

* remove cluster name from logs

* fix merge with master

* more merge with master issues
 2022-12-02 19:27:24 -08:00
Luca Furst 33e1f9345d
[App] Improve cluster creation / deletion experience (#15458) 
Cluster creation and deletion can take a long time. Instead of having these long running operations happen in the background, they should happen in the foreground. The advantage is that failures are brought to the users attention immediately, instead of the next time they decide to run `lightning list clusters`.

While the CLI waits for the cluster to run / delete, it will display cluster status changes to the user.

This PR also hides the `--enable-performance` and `--edit-before-creation` creation flags, as well as the `--force` deletion flag. They are either not frequently used (performance mode is expensive), or prone to misuse.


Co-authored-by: Neven Miculinic <neven.miculinic@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>
 2022-11-28 16:38:43 +00:00
Laverne Henderson 70126df494
Added note about custom base images (#14125) 
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
 2022-11-26 21:58:38 +00:00
Rick Izzo 317591d2e2
Fix App Docs for lightning ssh-keys command (#15773) 
fixed ssh-keys docs
 2022-11-22 15:39:20 -05:00
Kaushik B 2b61c92ceb
Fix the `examples/app_dag` App (#14359) 
* Fix app dag example
* Add test
* Update doc
* Update tests/tests_app_examples/test_app_dag.py

Co-authored-by: Sherin Thomas <sherin@grid.ai>
 2022-11-22 09:39:44 +01:00
Ethan Harris bc797fd376
[App] Fix multi-node pytorch example CI (#15753) 2022-11-21 16:02:30 +00:00
Yurij Mikhalevich 1a31d13521
feature(docs/app/lit_tabs): add works (#15731) 2022-11-21 13:46:01 +01:00
Ethan Harris 83067977af
[App] Update multi-node examples (#15700) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2022-11-21 10:41:59 +00:00
Yurij Mikhalevich befd3f6c22
fix(docs/app/lit_tabs): remove unused app_id, enable run instead (#15702) 2022-11-17 10:13:09 +01:00
William Falcon 41f1a36c5b
docs 5/n (#15669) 
* examples

* fix few examples

* Update pl_multinode.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-11-17 08:10:14 +01:00
Nash 9670fa8b18
Docs: Fix import for scikit in XGBoost template (#15693) 2022-11-16 09:19:03 +01:00