Commit Graph

1398 Commits

Author SHA1 Message Date
Jirka Borovec b73087c5c0
Releasing app 0.7 (#15197)
* Support Injecting Secrets into Apps Running in the Cloud (#14612)

Adds a new '--secret' flag to 'lightning run app':

lightning run app --cloud --secret MY_SECRET=my-secret-name app.py

When the Lightning App runs in the cloud, the 'MY_SECRET'
environment variable will be populated with the value of the
referenced Secret. The value of the Secret is encrypted in the
database, and will only be decrypted and accessible to the
Flow/Work processes in the cloud.

Co-authored-by: Sherin Thomas <sherin@grid.ai>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>

(cherry picked from commit 71719b9960)

* secrets docs (#14951)

* secrets docs

* Update docs/source-app/glossary/secrets.rst

Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update secrets.rst

* links

Co-authored-by: Yurij Mikhalevich <yurij@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
(cherry picked from commit 8715cd0346)

# Conflicts:
#	docs/source-app/glossary/secrets.rst

* Add support for command descriptions (#15193)

(cherry picked from commit 4acb10f981)

* docs: temp drop S3 from index (#15099)

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
(cherry picked from commit 05d91c8e75)

* version 0.7.0

* chlog join 0.6.3 & 0.7

Co-authored-by: Alec Merdler <alecmerdler@gmail.com>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
2022-10-20 14:42:08 +02:00
Laverne Henderson 24105966a4 Content for Lightning with iOS and Android (#14038)
* Content for Lightning with iOS and Android

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* make link clickable

* Update docs/source-app/glossary/ios_and_android.rst

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-09-19 12:26:05 -04:00
Laverne Henderson 1b15471a73 Update installation (#14732)
* Update installation

Updates to use python -m pip install -U lightning and adds troubleshooting note

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-09-19 12:26:05 -04:00
Yurij Mikhalevich 330d36dd0d Fix Google Tag Manager for the Lightning App docs (#14731)
- updates the Lightning App docs theme to the one without Pytorch Lightning docs Google Tag Manager hardcoded
- sets the GTM id in the conf.py for Lightning App docs
2022-09-19 12:26:05 -04:00
Adrian Wälchli d2459df2ff
Standalone Lite: Remaining Utilities (#14492)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
Co-authored-by: Felonious-Spellfire <felonious.spellfire@gmail.com>
2022-09-07 15:25:23 +00:00
Kushashwa Ravi Shrimali 6de2b0b528
Docs [Fix]: use bytes instead of strings while writing (#14505)
* Fix doc examples: use bytes instead of strings while writing

* Add a note (comment)

* nit

* Update any_server.rst

* Update docs/source-app/workflows/add_server/any_server.rst

* Update docs/source-app/workflows/add_server/any_server.rst

* Update docs/source-app/workflows/add_server/any_server.rst

* Apply suggestions from code review

Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
2022-09-07 09:05:58 -04:00
Dan Nissenbaum 7ada52e33c
Typo in major heading seen by newcomers (#14501)
* Typo in major heading seen by newcomers

Correct typo in one of the first major headings newcomers to Lightning see when they are considering migrating their code to use Lightning.

I know this is a trivial change in terms of the text change itself, but I really think it's valuable for one of the most important landing pages that users first investigating Lightning see - to have rock-solid, professional text without obvious typos. Here was a typo in the main heading itself. I suggest fixing it straightaway via this PR.

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-09-07 09:06:30 +00:00
Carlos Mocholí 44216fdd69
Integrate `lightning_utilities.core.imports` (#14475)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-09-06 12:56:20 +00:00
Carlos Mocholí 8a4a3b6766
Mark the lite `DeviceDtypeModuleMixin` as protected (#14548) 2022-09-06 14:17:15 +02:00
Rohit Gupta 8c6119fbce
Add auto wrapping support for `DDPFullyShardedStrategy` (#14383) 2022-09-05 19:07:26 +00:00
Laverne Henderson b19b15546b
Updated basic debugging (#14488)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-09-02 14:23:29 +05:30
Benjamin Krala cce55b6cd3
Precise description of reload_dataloaders_every_n_epochs (#14245) 2022-09-01 10:06:28 -04:00
Adrian Wälchli 28e18881a9
Mark stage argument in hooks as required (#14064)
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2022-09-01 15:47:40 +02:00
thomas chaton a7c01c4754
Introduce lightning connect (#14452)
* update

* update

* update

* update

* Review of content

* Formatting updates

* Fomatting updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updates based on new commits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update

* update

* update

* Introduce lightning connect (#14183)


Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

Co-authored-by: Felonious-Spellfire <felonious.spellfire@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
2022-08-31 15:59:03 -04:00
Carlos Mocholí 10adcd5d12
Remove the unused pyDeprecate dependency (#14472)
* Remove the unused pyDeprecate dependency
* CHANGELOG
2022-09-01 03:30:18 +09:00
Jirka Borovec 208bf6faa8
prepare space for fused docs (#14160)
* copy app conf

* ci + req.

* script symlink

* wip

* keep only App

* add also PL

* lightning

* artifact
2022-08-30 09:25:05 -04:00
Marc Skov Madsen 18e2a8eecd
PanelFrontend and Panel Web UI Intermediate docs (#13531)
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Marc Skov Madsen <masma@orsted.com>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
Co-authored-by: Felonious-Spellfire <felonious.spellfire@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Mansy <ahmed.mansy156@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2022-08-30 00:33:21 +00:00
Rohit Gupta f3574176e2
Change `trainer.should_stop` to not stop in between an epoch and run until `min_steps/min_epochs` only (#13890) 2022-08-27 12:12:24 +00:00
Adrian Wälchli 250c06e406
Remove deprecated HPC model hooks (#14315)
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-26 20:59:32 +00:00
Tianshu Wang 8950613552
save checkpoints and profiler output to the first logger (#14325)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-26 17:23:54 +00:00
Rohit Gupta 6d00f31f0c
Add auto wrapping for `DDPFullyShardedNativeStrategy` (#14252) 2022-08-26 09:01:48 +00:00
Jerome Anand 424e78d5b3
Add document to showcase scaleout on hpu (#14357)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-08-25 14:15:01 +05:30
Laverne Henderson 8ff2e01025
Update for M1 Mac installations (#14350)
* Update for M1 Mac installations

* Apply suggestions from code review

* Update PL installation

* Update based on feedback

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-23 13:31:18 -04:00
Laverne Henderson 2265bd85c9
Bring back access app state (#14258)
* Recreated the access_app_state file

* Update the site's TOC to include the file

* Update code sample file path

* Minor formatting update

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@grid.ai>
2022-08-23 13:06:46 -04:00
Carlos Mocholí 7a617ec90e
Add back support for logging in the gradient clipping hooks (#14298)
* Add back support for logging in the gradient clipping hooks

* Docs and CHANGELOG

* Fix tests
2022-08-22 09:19:53 -04:00
Rohit Gupta e949362a6b
Enable `on_before_batch_transfer` for `DPStrategy` and `IPUAccelerator` (#14023)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-18 12:12:29 +00:00
Laverne Henderson 8ab65ffade
Fixes note formatting and more (#14264)
Fixes formatting for a note and makes headings for arguments and parameters stand out
2022-08-18 13:55:49 +02:00
Laverne Henderson 1745d192f9
Remove incorrect "template" information (#13911)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-17 14:35:23 -04:00
Adam Bobowski 79fc6afa89
[App] Moved app.py to root dir for `lightning init app <app_name>` template (#13853)
* Moved app.py to main app directory

* updated docs

* updated changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-17 10:15:52 -04:00
Laverne Henderson 61132837f2
Docs BYOC content (#13976)
* BYOC content

Content for the upcoming BYOC feature

* First DRAFT of BYOC content

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update docs/source-app/index.rst

Co-authored-by: thomas chaton <thomas@grid.ai>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: thomas chaton <thomas@grid.ai>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: thomas chaton <thomas@grid.ai>

* Updates based on feedback

* Updates based on feedback

* Update external ID with note

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>

* Updates for terraform mod

Updates for terraform mod and arg pram split

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update index.rst

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update docs/source-app/workflows/byoc/index.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update content with table

Changed bullets into table based on feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Raphael Randschau <nicolai86@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-17 10:15:41 -04:00
Sherin Thomas deaadc157b
(app) Documentation fix for Work resources (#14182) 2022-08-15 09:35:10 -04:00
Rohit Gupta 48c23e5716
Use fsdp module to initialize precision scalar for fsdp native (#14092)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com>
2022-08-13 07:52:06 +00:00
Rohit Gupta 208512e6e9
Add docs for `fsdp_native` (#14108) 2022-08-12 09:14:07 +00:00
Dan Dale 355fda3702
Add Promoted CLI to API Reference Section (#14072) 2022-08-08 10:16:53 +02:00
Rohit Gupta a4e4cab7a6
Deprecate `amp_level` from `Trainer` (#13898)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-05 08:31:19 +00:00
Carlos Mocholí b88b700745
Remove the deprecated DDP2 strategy (#14026) 2022-08-04 20:27:35 +00:00
Rohit Gupta e78bf2044b
Raise an error if batch transfer hooks are overridden with IPUAccelerator (#13961)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-08-04 12:04:42 +00:00
Laverne Henderson 52f3775bb5
Doc Terminology updates (#13972)
* Doc Terminology updates

* API updates
2022-08-03 15:59:37 -04:00
Jerome Anand b3203d93d0
Added support for HPU device stats monitor (#13819)
* Added support for HPU device stats monitor

Signed-off-by: Jerome <janand@habana.ai>

* Update changelog

Signed-off-by: Jerome <janand@habana.ai>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* Update reference

Signed-off-by: Jerome <janand@habana.ai>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* fix alignment

* add descriptions

* Update hpu_intermediate.rst

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-08-02 13:31:31 +05:30
Laverne Henderson e33d25fb28
Porting latest App docs update (#13680)
* PRs 909,910,911, and 912

moves last 4 commits to the private re;po to the OS repo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix validation error

* Fixes API links and validation issues

* Update docs/source-app/examples/file_server/app.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Fix Python validation errors

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2022-08-01 10:25:40 -04:00
edenlightning 5cbea78096
Docs update (#13959) 2022-08-01 14:10:18 +00:00
thomas chaton aefb9ab43f
(app) Introduce LightningTrainingComponent (#13830) 2022-07-29 16:44:52 +02:00
Adrian Wälchli 7708ce22b2
Update GitHub links to PL repo (#13849)
* update lightning links in docs

* update links in chlog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update src/pytorch_lightning/README.md

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update src/pytorch_lightning/README.md

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* update

* painful

* badges

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update badges

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-07-28 22:08:07 +02:00
otaj 4c7b9f0b11
Disallow batch sampler with multiple IPU devices (#13854)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-07-27 15:20:43 +05:30
Rohit Gupta faf7ff57c0
Add support for async checkpointing (#13658) 2022-07-26 21:13:19 +05:30
Adrian Wälchli a8d7b4476c
Fix PyTorch spelling errors (#13774)
* Fix PyTorch spelling errors

* more
2022-07-25 12:51:16 -04:00
Ha YongWook 11f8fa2ca6
Fix typos in Checkpointing doc (#13827)
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
2022-07-25 09:32:49 -04:00
rampartrange 8d14554383
[Typo] update introduction.rst (#13791) 2022-07-23 16:38:36 +00:00
Adrian Wälchli d24978baa3
Add ddp_notebook alias for ddp_fork (#13744)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-07-23 09:06:35 -04:00
Carlos Mocholí 4f53e7132f
Promote the CLI out of utilities (#13767) 2022-07-23 12:07:29 +00:00