Commit Graph

73 Commits

Author SHA1 Message Date
Jirka Borovec 99fe6563ef
precommit: ruff-format (#19434)
* precommit: ruff-format

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* manual update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* manual update

* order

* mypy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* mypy

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-02-15 13:39:17 -05:00
Justus Schock 0acd5f9810
Rename Lightning App CLI (#19440) 2024-02-09 16:54:54 +01:00
thomas chaton 10c3a71dbd
Bump Lightning Cloud 0.5.64 (#19372) 2024-01-30 14:57:11 +00:00
pre-commit-ci[bot] f120c91e9f
[pre-commit.ci] pre-commit suggestions (#19229)
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0)
- [github.com/asottile/pyupgrade: v3.14.0 → v3.15.0](https://github.com/asottile/pyupgrade/compare/v3.14.0...v3.15.0)
- [github.com/astral-sh/ruff-pre-commit: v0.1.3 → v0.1.9](https://github.com/astral-sh/ruff-pre-commit/compare/v0.1.3...v0.1.9)
- [github.com/psf/black: 23.9.1 → 23.12.1](https://github.com/psf/black/compare/23.9.1...23.12.1)
- [github.com/pre-commit/mirrors-prettier: v3.0.3 → v4.0.0-alpha.8](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.3...v4.0.0-alpha.8)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update .pre-commit-config.yaml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* drop unused

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2024-01-10 13:11:48 -05:00
Ethan Harris 02780f2cb3
App: Fix dispatch return value (#18674) 2023-09-29 15:12:31 +01:00
Jirka Borovec 830a62a722
ruff: replace isort with ruff +TPU (#17684)
* ruff: replace isort with ruff

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixing & imports

* lines in warning test

* docs

* fix enum import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixing

* import

* fix lines

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* type ClusterEnvironment

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-09-26 11:54:55 -04:00
Jirka Borovec dbe7ed46a3
replace tests skip with soft xfail (#18486)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-09-12 23:11:03 +02:00
Jirka Borovec efa7b2f9ef
docformatter: config with black (#18064)
* docformatter: config with black

* additional_dependencies: [tomli]

* 119

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-09 10:44:20 -04:00
Luca Antiga c740a63667
Add launch commands (#18155)
* Add launcher to main repo

* Move launcher CLI to main CLI

* Launch CLI tests

* Fix tests

* Remove tabulate dependency

* Allow access through lightning_app package

* Avoid conditional import in CLI

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove conditional input

* Skip mypy checks for launcher

* Add return type annotation

* Fix duplicated app_metadata.py

* Add missing import

* Fix test

* Add skip test import depth

* Remove leftover conditional import

* Change expected output to reflect the new tabulate method

* Remove check on deprecated functionality

* Improve stability of test

* Skip unrelated failing test on Windows

---------

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-07-28 18:03:08 +02:00
Ethan Harris 2c3dfc0fb7
[App] Content for plugins (#17243)
Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
2023-07-07 11:05:58 +01:00
Jirka Borovec b06ad2c4f7
tests: clean too many `,` in mocks (#17776)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-06-07 10:36:25 -04:00
Jirka Borovec 384c203532
ruff: PT some more fixes (#17569) 2023-05-05 08:25:15 +02:00
Jirka Borovec f55d10f5ee
ruff: autofix PT (#17541)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-05-04 11:50:39 -04:00
Jirka Borovec 156786343b
adding check for bandit vulnerabilities 1/n (#17382)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-04-27 09:43:12 +00:00
pre-commit-ci[bot] 91cb4b9b87
[pre-commit.ci] pre-commit suggestions (#17271)
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/PyCQA/docformatter: v1.4 → v1.6.0](https://github.com/PyCQA/docformatter/compare/v1.4...v1.6.0)
- [github.com/psf/black: 22.12.0 → 23.3.0](https://github.com/psf/black/compare/22.12.0...23.3.0)
- [github.com/charliermarsh/ruff-pre-commit: v0.0.237 → v0.0.260](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.237...v0.0.260)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* apply

* fixing

* docs/lines

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2023-04-26 21:37:41 +02:00
Jirka Borovec df97141781
add & apply flake8-simplify (#17386) 2023-04-24 21:57:08 +00:00
Jirka Borovec 6705bfcc47
app/tests: skip instead of fail (#17461) 2023-04-24 22:56:04 +02:00
thomas chaton 3688b6495d
Resolve Lightning App with remote storage (#17426)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-04-23 18:08:11 -04:00
Jirka Borovec 111d1ba088
ruff: fixing flake8-comprehensions (#17385) 2023-04-21 09:07:58 +00:00
Jirka Borovec 6da4b0f490
skip some App tests (#17401)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-04-18 11:20:07 +02:00
Ethan Harris 621644d0bc
App: Fix frontends when using multiprocessing in the cloud (#17324)
* App: Fix frontends when using multiprocessing in the cloud

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update CHANGELOG.md

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-04-11 12:28:44 -04:00
Dmitry Frolov 35d15621c7
Data connection mounts for jobs running from CloudSpaces (#17006)
* Data connection mounts for jobs running from CloudSpaces

* Tests

* Tests fix

* Bump Lightning Cloud
2023-03-11 13:17:18 -05:00
Dmitry Frolov 4a531b2d5a
System customization syncing for jobs run (#16932)
* System customization syncing for jobs run

* Constants moved

* Moving files

* Update src/lightning/app/runners/cloud.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update test_cloud.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test_cloud.py

---------

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-03-09 14:35:41 -05:00
Luca Furst e57f12851f
[App] Use environment variables from args when loading apps for the CloudRuntime
## What does this PR do?

Passes environment variables to the `CloudRuntime` when loading an app from a file. Allows apps / works / flows to be properly configured when passing variables at runtime.

### Limitations

- Only implemented for the `CloudRuntime`

<details>
  <summary><b>Before submitting</b></summary>

- Was this **discussed/agreed** via a GitHub issue? (not for typos and docs)
- [ ] Did you read the [contributor guideline](https://github.com/Lightning-AI/lightning/blob/master/.github/CONTRIBUTING.md), **Pull Request** section?
- [ ] Did you make sure your **PR does only one thing**, instead of bundling different changes together?
- Did you make sure to **update the documentation** with your changes? (if necessary)
- Did you write any **new necessary tests**? (not for typos and docs)
- [ ] Did you verify new and **existing tests pass** locally with your changes?
- Did you list all the **breaking changes** introduced by this pull request?
- Did you **update the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)

<!-- In the CHANGELOG, separate each item in the unreleased section by a blank line to reduce collisions -->

</details>

## PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the [review guidelines](https://github.com/Lightning-AI/lightning/wiki/Review-guidelines). In short, see the following bullet-list:

<details>
  <summary>Reviewer checklist</summary>

- [ ] Is this pull request ready for review? (if not, please submit in draft mode)
- [ ] Check that all items from **Before submitting** are resolved
- [ ] Make sure the title is self-explanatory and the description concisely explains the PR
- [ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

</details>

<!--

Did you have fun?

Make sure you had fun coding 🙃

-->


cc @borda
2023-03-08 10:34:41 -05:00
Jirka Borovec 760612fb8a
update list of fist party packages (#16859)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-03-03 16:55:48 +00:00
Sherin Thomas da50c1341e
Maverick registration (#16913)
Maverick registration (#16913)
2023-03-01 18:18:14 +05:30
Ethan Harris beced48904
[App] Add support for plugins to return actions (#16832) 2023-02-22 17:12:04 +00:00
Yurij Mikhalevich 6950a07eaa
[App] fix `lightning open` command & better redirects (#16794)
* fix(app): URLs, create run on app run

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-02-17 18:24:23 +00:00
Ethan Harris 7f92d5c9d4
[App] Refactor plugins to be a standalone `LightningPlugin` (#16765) 2023-02-17 11:01:38 +00:00
Ethan Harris 1288e4ccc4
[App] Initial plugin server (#16523) 2023-02-07 21:38:37 +00:00
Jirka Borovec cc49e4a31f
tests: switch imports for apps (#16554) 2023-02-01 11:07:00 +00:00
Ethan Harris 3001c7c1d5
[App] Fix app name in URL (#16575) 2023-01-31 07:14:56 -05:00
Ethan Harris 0ec93f2125
[App] Update app URLs to latest format (#16568) 2023-01-30 20:52:46 +00:00
thomas chaton b8eaabe3c9
[App] Add interruptible work (#16399)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-01-25 13:48:27 +00:00
Ethan Harris 4a802e00a8
[App] Add `lightning open` command (#16482) 2023-01-24 15:58:15 +00:00
Ethan Harris 04886ed7f1
[App] Refactor cloud dispatch and update to new API (#16456)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-01-23 12:41:33 +00:00
Justus Schock e61bead324
rename flow _exit (#16378)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-01-17 11:06:50 +00:00
Dmitry Frolov f08bc67e4d
[App] Introduce basic auth to Lightning CLI (#16105)
* Introduce basic auth to Lightning CLI for app creation

* Parsing creds added

* Adding auth field to app instance body

* Adding tests

* Adding changelog entry

* Adding more tests


* Update runtime.py

* Setting auth on update

* Fix test

* Update lightning-cloud dep

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update runtime.py

* Fix for release

* Update base.txt

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-12-22 13:07:49 -05:00
Ethan Harris 965767199c
[App] Simplify messaging in cloud dispatch (#16160) 2022-12-21 13:33:41 +01:00
thomas chaton 51ec949657
[App] Resolve some bugs from the Training Studio scaling (#16114)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2022-12-19 23:12:55 +00:00
Adrian Wälchli 07b775fbe5
Fix test failing on master due to bad auto-merge (#16118) 2022-12-19 18:42:47 +00:00
Adrian Wälchli f3157f306a
Set the default work start method to spawn on MacOS (#16089) 2022-12-19 14:06:52 +00:00
Carlos Mocholí b1ce2639f4
Better check for programmatic lightningignore (#16080)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2022-12-16 09:49:17 +00:00
Ethan Harris 5f7403e99c
[App] Fix bug where previously deleted apps cannot be re-run from the CLI (#16082) 2022-12-16 04:06:05 +01:00
Yurij Mikhalevich 3b323c842d
fix(cloud): detect and ignore venv (#16056)
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
2022-12-15 14:00:32 +00:00
Ethan Harris d3a722608e
[App] Support running on multiple clusters (#16016) 2022-12-14 16:48:49 +00:00
Carlos Mocholí edd2b4259a
Introduce `{Work,Flow}.lightningignore` (#15818) 2022-12-13 13:50:08 +00:00
Ethan Harris e250dfe2b3
[App] Remove `SingleProcessRuntime` (#15933)
* Remove SingleProcessRuntime
* Remove unused queues
* Docs
2022-12-07 23:22:40 +00:00
Ethan Harris 32cf1faa07
[App] Support for headless apps (#15875)
* Add `is_headless` when dispatching in the cloud

* Bump cloud version

* Add tests

* Dont open app page for headless apps locally

* Refactor

* Update CHANGELOG.md

* Support dynamic UIs at runtime

* Comments

* Fix

* Updates

* Fixes and cleanup

* Fix tests

* Dont open view page for headless apps

* Fix test, resolve URL the right way

* Remove launch

* Clean

* Cleanup tests

* Fixes

* Updates

* Add test

* Increase app cloud tests timeout

* Increase timeout

* Wait for running

* Revert timeouts

* Clean

* Dont update if it hasnt changed

* Increase timeout
2022-12-05 16:58:22 -05:00
Luca Furst c5d3bba764
[App] Raise error when launching app on multiple clusters (#15484)
* Error when running on multiple clusters

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert this in separate PR: keep this focused

* Improve testing

* fixup! Improve testing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pass flake8

* Update changelog

* Address PR feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused import

* Reword error message

* Error if running on cluster that doesn't exist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixup! Error if running on cluster that doesn't exist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unsued import

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2022-11-30 11:33:57 -05:00