Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,467 Commits 56 Branches 212 Tags 373 MiB
Commit Graph

306 Commits

Author SHA1 Message Date
Luca Antiga 392f643ba0
Merge branch 'master' into bump/python_3.9+ 2024-11-14 23:53:18 +01:00
Alan Chu cd2bd3ce53
Fix configure_optimizers example (#20420) 
Co-authored-by: Alan Chu <alanchu@Alans-Air.lan>
 2024-11-14 23:41:44 +01:00
Jirka B b9920bdd78 --unsafe-fixes 2024-11-12 21:41:33 +01:00
pre-commit-ci[bot] c11c392acd [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-11-12 20:29:05 +00:00
Jirka Borovec cf24a190ce
fix(docs): remove dead link from readme (#20170) 2024-08-06 11:05:00 -04:00
pre-commit-ci[bot] a40affb953
[pre-commit.ci] pre-commit suggestions (#20035) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
 2024-07-05 12:17:15 -04:00
awaelchli 330af381de
Remove the lightning app code (#20039) 
* remove source, tests, docs, workflows

* update checkgroup

* update codeowners

* update workflows

* package setup

* config files

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove entry point

* docs

* __main__

* remove store

* leftover store removals

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-07-05 13:56:29 +02:00
liambsmith 709a2a9d3b
Updated Fabric trainer example to not call `self.trainer.model` during validation (#19993) 2024-06-21 10:43:30 -04:00
SW Yoo d3a0ada4ff
Fix dtype for MPS in reinforcement learning example (#19982) 2024-06-21 10:36:10 -04:00
liambsmith 394c42aaf6
Fix callback call in Fabric Trainer example (#19986) 2024-06-18 13:14:32 -04:00
Bhavay Malhotra a611de0c15
Removing numpy requirement from all files in examples/pytorch/domain_templates (#19947) 2024-06-06 11:02:01 -04:00
awaelchli 98005bbed0
Add Studio badge to tensor parallel docs (#19913) 2024-05-28 09:04:55 -04:00
awaelchli c09356db1e
(10/10) Support 2D Parallelism - Port Fabric docs to PL (#19899) 2024-05-23 08:55:52 -04:00
awaelchli 987c2c4093
(7/n) Support 2D Parallelism - TP Fabric Docs (#19884) 
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2024-05-22 06:20:40 -04:00
awaelchli d76feef0d6
Enable loss-parallel in example (#19882) 2024-05-20 13:19:38 +02:00
awaelchli c8059d7bfd
(6/n) Support 2D Parallelism - Trainer example (#19879) 
* Add 2D parallel example

* replace with torchtitan code
 2024-05-18 20:35:58 -04:00
awaelchli 9455871c93
(2/n) Support 2D Parallelism - Distributed Checkpoints (#19852) 
* distributed checkpoints

* use decorator

* refactor if-strict

* update example

* filter non-persistent buffers (todo, add test)

* simplify checkpoint loading for model
 2024-05-15 08:19:08 -04:00
awaelchli 0c8a193d3c
(1/n) Support 2D Parallelism (#19846) 2024-05-07 17:02:58 -04:00
Inyong Hwang a6ab0d1486
replace bolts url on example's readme (#19677) 2024-03-20 10:23:12 -04:00
Jirka Borovec fe535970a9
lint: switch `pyupgrade` with Ruff's `UP` rule (#19638) 
* ruff
* configure
* update

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-03-15 20:48:17 +01:00
awaelchli 8549a932f7
Remove orphaned docs pages (#19555) 2024-03-12 21:19:05 -04:00
awaelchli ea89133c65
Rename `fabric run model` to `fabric run` (#19527) 2024-02-27 11:36:46 -05:00
Jirka Borovec 99fe6563ef
precommit: ruff-format (#19434) 
* precommit: ruff-format

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* manual update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* manual update

* order

* mypy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* mypy

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-02-15 13:39:17 -05:00
awaelchli e950bb4828
Remove the Graphcore IPU integration (#19405) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2024-02-12 16:16:02 -05:00
pre-commit-ci[bot] f120c91e9f
[pre-commit.ci] pre-commit suggestions (#19229) 
* [pre-commit.ci] pre-commit suggestions

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0)
- [github.com/asottile/pyupgrade: v3.14.0 → v3.15.0](https://github.com/asottile/pyupgrade/compare/v3.14.0...v3.15.0)
- [github.com/astral-sh/ruff-pre-commit: v0.1.3 → v0.1.9](https://github.com/astral-sh/ruff-pre-commit/compare/v0.1.3...v0.1.9)
- [github.com/psf/black: 23.9.1 → 23.12.1](https://github.com/psf/black/compare/23.9.1...23.12.1)
- [github.com/pre-commit/mirrors-prettier: v3.0.3 → v4.0.0-alpha.8](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.3...v4.0.0-alpha.8)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update .pre-commit-config.yaml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* drop unused

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2024-01-10 13:11:48 -05:00
thomas chaton 792cb73fc6
Remove the LightningDataset relying on un-maintained torchdata (#19019) 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2023-11-16 16:08:15 -05:00
thomas chaton cb23fc2dd4
Update auto_encoder.py to accomodate torchvision breaking change (#18996) 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2023-11-13 15:32:16 -05:00
Adrian Wälchli e0ba4d46e1
Fix spelling errors (#18773) 
fix spelling errors
 2023-10-30 17:14:52 -04:00
pre-commit-ci[bot] c0ec0decec
[pre-commit.ci] pre-commit suggestions (#18697) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2023-10-03 22:07:21 +02:00
Adrian Wälchli c631726b5c
Set the generator seed in `random_split` usages in the examples (#18651) 2023-09-27 14:09:05 +02:00
Jirka Borovec 830a62a722
ruff: replace isort with ruff +TPU (#17684) 
* ruff: replace isort with ruff

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixing & imports

* lines in warning test

* docs

* fix enum import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixing

* import

* fix lines

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* type ClusterEnvironment

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-26 11:54:55 -04:00
Jirka Borovec 358336268f
enable codespell for docs & fixing +TPU (#18629) 
* precommit/codespell

* run

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* disable

* more fixing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

* more fixing

* json

* note

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-26 11:54:44 -04:00
Adrian Wälchli f83ad093e5
Utility function to check shared filesystem (#18586) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2023-09-25 15:49:52 -04:00
Jirka Borovec d579cfed57
precommit: unify formatting with prettier (#18605) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-25 14:34:41 +02:00
Jirka Borovec 3fbf1540b0
docs: 3/3 enable Sphinx nitpicky [app] (#18603) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-21 22:44:59 +02:00
Jirka Borovec b4b21e0152
docs: 2/3 enable Sphinx nitpicky [pytorch] part 1/n (#18484) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-20 19:09:34 +02:00
Adrian Wälchli 01cd89fd04
Set up dataloaders in k-fold cross validation example (#18582) 2023-09-18 11:45:20 -04:00
Jirka Borovec 5f5d99eae6
Relax app dependencies for `lightning` | breaking imports from `L` (#18386) 
* make App conditional import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ci: min req.

* data + store

* typo

* module_available

* extra

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* typo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* envs

* doctest

* more

* docs

* **

* rm

* -f

* Apply suggestions from code review

* docs

* Revert "docs"

This reverts commit 20e8e6be98.

* Apply suggestions from code review

* warnings

* breaking

* ...

* fixing source

* fixing examples

* fixing tests

* fixing docs

* fixing source

* fix mock

* fix trainer

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-28 11:28:58 -04:00
Jirka Borovec efa7b2f9ef
docformatter: config with black (#18064) 
* docformatter: config with black

* additional_dependencies: [tomli]

* 119

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-09 10:44:20 -04:00
pre-commit-ci[bot] 834bd61164
[pre-commit.ci] pre-commit suggestions (#17983) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka B <j.borovec+github@gmail.com>
 2023-08-08 16:26:06 +02:00
Nripesh Niketan 4e3087951d
Add GPU Acceleration apple silicon examples (#18127) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2023-07-27 19:52:51 -04:00
Carlos Mocholí a9269aecdb
Add nightly testing to CUDA CI (#18078) 2023-07-17 20:53:29 +02:00
Carlos Mocholí 3a55f0c0a1
Minor miscellaneous fixes (#18068) 2023-07-13 06:01:58 -04:00
jere357 28007bd076
Consistent early exit logic for validation loop in Fabric Trainer example #17962 2023-07-03 08:26:50 -04:00
jere357 6329caa876
slightly changing the behavior of val_loop specifying the limit_val_batches or reaching should_stop (#17873) 2023-06-20 00:30:34 +02:00
Noha Alon ca30fd7752
Lightning Dataset (including optimized dataloading of s3 buckets) (#17743) 
* Lightning DataLoader

* lightning dataloader

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* init

* example

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* env var

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update src/lightning/pytorch/utilities/data/__init__.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* remove unused functions

* extra reqs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update src/lightning/pytorch/utilities/data/fileio.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* imports work now! yay

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* imports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* missing import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* error handling

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update creds for local use case

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* codeowners

* recursive get index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean up get index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update imagenet example

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* example cleanup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* changelog

* reqs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* codeowners

* requirements

* expose LightningDataset too

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* expost LightningDataset at top level

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused private methods from init

* remove private imports

* upper bound on extra requirements

* review comments

* loosen req

* deps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test updating fabric base req

* remove version pin on s3fs to test

* recover missing function

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests

* update

* random

* torchdata >= 0.3.0

* update torchdata version

* remove torchdata version to test

* try rem torch version pin

* req

* update bucket in test

* req

* skips

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* import

* update structure to lightning.data

* base.txt for data reqs

* fix imports

* rename to LightningS3Dataset

* new workflow

* dont need to test warnings

* reqs

* req

* revert data folder in pytorch

* test import

* tests

* req

* req

* req

* torch version

* req

* req

* open dep

* reformatted

* pin strict

* pin strict extra

* req

* modify workflow, no cache

* try

* patch

* import

* fix

* dataset test

* update getattr

* pin everything to test

* remove torch preinstall from workflow

* workflow

* req

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow

* workflow

* req

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow

* print

* skip test for now

* update path join

* revert app dep version bump

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow updates

* app base req

* req

* window test failure

* add data req to assistant

* try

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add missing comma

* updates

* update

* typo

* requirements

* try widening req

* older torch version

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update

* update

* update

* cleanup tests

* typo again

* update

* remove unnecessary line

* Update .github/CODEOWNERS

* Discard changes to requirements/pytorch/base.txt

* Discard changes to requirements/fabric/base.txt

* Discard changes to requirements/app/base.txt

* requirements

* requirements

* one line

* app workflow pick only app reqs

* rename package

* undo

* don't use cache

* examples CI

* pytorch and fabric CI

* try remove cache

* Apply suggestions from code review

* jirka playing

* jirka playing

* jirka playing

* blah

* flatten LightningDataset

* cleans up dataset class

* jirka playing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* jirka playing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* extra

* fix dataset test

* update checkgroups

* Luca's review comments

* val error fix

* unskip test

* min

* fix precommit warning

* cpu

* docstrings

* req

* 2.0.1

* add return type

* typing errors

* req

* return types with quotations

* import for type-checking

* no botocore in cloudagnostic code

* exit args

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* backends typing

* remove oldest from data tests

* typing

* typing

* typing

* types

* type

* typing

* typing

* typing

* import fix

* Changelog

---------

Co-authored-by: Noha Alon <nohaalon@Nohas-MacBook-Air.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
 2023-06-13 11:44:41 +01:00
Adrian Wälchli 9ff7d7120b
Add `rank_zero_first` utility (#17784) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-06-12 10:32:32 +00:00
Carlos Mocholí ea44aac10f
Use PL `is_overridden` in BYOT example (#17070) 2023-05-05 12:08:18 +00:00
Jirka Borovec 4413e98e4e
ruff: enable & fixing RET (#17540) 2023-05-05 09:34:40 +00:00
thomas chaton cafdc6d308
Examples: expose learning rate (#17513) 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2023-04-28 09:54:09 +00:00