Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,334 Commits 59 Branches 217 Tags 353 MiB
Commit Graph

10334 Commits

Author SHA1 Message Date
awaelchli 4767544b35
Update checkgroup to require PyTorch 2.2 workflows (#19459) 2024-02-13 06:45:15 +01:00
awaelchli e950bb4828
Remove the Graphcore IPU integration (#19405) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2024-02-12 16:16:02 -05:00
awaelchli 8d4768f2ae
Remove the Bagua integration (#19445) 2024-02-12 20:58:52 +01:00
Justus Schock 2ed7282f7c
Rename Lightning Fabric CLI (#19442) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2024-02-12 17:22:53 +01:00
Xinyu Yang 47c8f4cba0
bugfix: skip write index.json if no data is wrote. (#19439) 2024-02-09 17:08:28 +00:00
Justus Schock 0acd5f9810
Rename Lightning App CLI (#19440) 2024-02-09 16:54:54 +01:00
Xinyu Yang 7b867c7d91
bugfix: correct node rank (#19437) 2024-02-09 15:21:28 +00:00
awaelchli 9c8cd4ce68
Update to 2.3.0dev (#19430) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2024-02-08 06:34:51 -05:00
awaelchli 5aea3b1fc3
Fix import error when running examples in fresh environment (#19431) 2024-02-08 06:31:52 -05:00
thomas chaton 4c2fc3b0cb
Add DNS optimize support (#19429) 
* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
 2024-02-08 11:14:57 +00:00
thomas chaton ac9d63f4eb
Lightning Data: Refactor files (#19424) 2024-02-08 08:02:08 +00:00
awaelchli bc56630c68
Release Lightning 2.2 (#19399) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2024-02-07 16:55:26 -05:00
Carlos Mocholí 45103516ad
Delay `Precision.convert_module` until `configure_model` has run (#19061) 2024-02-07 16:27:19 -05:00
Carlos Mocholí db2cc8a88e
Fix CLI docs typo (#19426) 2024-02-07 10:53:58 -05:00
Carlos Mocholí 78b7a39e72
Update throughput docs (#19415) 2024-02-06 16:26:10 -05:00
thomas chaton 28a80238a4
Add support for tif (#19421) 2024-02-06 15:23:40 +00:00
thomas chaton c80226d3f1
map & operator: Add lightning.ai account creation info (#19418) 2024-02-06 15:08:25 +00:00
dependabot[bot] 0d8208fb18
Bump peter-evans/create-pull-request from 5 to 6 (#19407) 
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5 to 6.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v5...v6)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-06 12:04:25 +01:00
awaelchli 130b05fe0c
Fix dead link in docs (#19387) 2024-02-06 11:54:17 +01:00
thomas chaton 285da64386
Data Processor: Give instructions to the users when the dependencies doesn't match (#19416) 2024-02-06 09:36:49 +00:00
nik777 7a56ac5182
Support shortcut name for DeepSpeed stage 1 offload (#19075) 2024-02-05 20:53:18 -05:00
awaelchli 9624aae07e
Support non-strict loading in Trainer (#19404) 2024-02-05 19:57:43 -05:00
thomas chaton 7dfc279b3f
Add support for parallelizing processing parquet files across workers and nodes. (#19400) 2024-02-05 23:21:25 +00:00
awaelchli 277869205a
Update return type of `LightningModule.configure_optimizers()` (#19408) 2024-02-05 17:59:56 -05:00
awaelchli c7c42dcef3
Update CI to use latest PyTorch 2.2 (#19401) 2024-02-05 12:50:23 -05:00
awaelchli fb0ce03a9c
Fix input validation to support passing `device_mesh` to FSDP (#19392) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2024-02-02 06:48:12 -05:00
thomas chaton 8646515cac
map: Add support for skipping installation (#19390) 2024-02-01 20:34:01 +00:00
awaelchli 89ff87def0
Reapply compile in `Fabric.setup()` by default (#19382) 2024-02-01 15:06:18 -05:00
thomas chaton af7e79a84a
Data Processing: Tiny optimization (#19389) 2024-02-01 18:21:54 +00:00
awaelchli 6296a4fe0d
Prepare 2.2 release candidate (#19376) 
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2024-02-01 10:35:39 -05:00
Adam J. Stewart 509b2ca560
Docs: fix FSDP acronym (#19384) 2024-02-01 16:02:59 +01:00
thomas chaton 8280519642
Data Processor: Add is_last argument to know when the last item for the current worker is being processed (#19383) 2024-02-01 12:09:06 +00:00
thomas chaton 5a0d2eff8c
map operator: Add support for non absolute input_dir and output_dir (#19378) 2024-02-01 08:25:47 +00:00
awaelchli 34a34a0754
Enable saving and loading stateful DataLoaders in Trainer (#19361) 2024-01-31 21:11:19 -05:00
Wouter Zwerink 5d178d07b7
Support TQDM_MINITERS env variable (#19381) 
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
 2024-01-31 20:17:35 -05:00
awaelchli c346f4d159
Compile guide for Fabric (#19330) 2024-01-31 14:57:07 -05:00
Jirka Borovec 6421dd8d4f
precommit: drop Black in favor of Ruff (#19380) 2024-01-31 17:09:39 +00:00
awaelchli 01f8531c9d
Refactor BoringFabric in tests (#19364) 2024-01-30 23:32:45 +01:00
thomas chaton 28b380610f
StreamingDataloader: Resolve typo (#19370) 2024-01-30 16:52:47 +00:00
thomas chaton 322f474978
JPEGSerializer: Fix serializer io.bytes image (#19369) 2024-01-30 16:52:25 +00:00
thomas chaton 10c3a71dbd
Bump Lightning Cloud 0.5.64 (#19372) 2024-01-30 14:57:11 +00:00
Michael Pilosov, PhD 5361acdcca
Shorten docstring (for CLI compat) (#19356) 2024-01-30 08:11:51 +01:00
awaelchli 6018b0743c
Error message to inform bitsandbytes is only supported on CUDA (#19360) 2024-01-29 19:52:28 -05:00
awaelchli bcc8de8dec
Update Trainer's ckpt_path type for pathlib Path (#19362) 2024-01-30 00:42:18 +01:00
thomas chaton b0e1ee2469
map operator: Add support for nested folders (#19366) 2024-01-29 19:17:28 +00:00
thomas chaton 37a521cad2
map operator: Add weights to evenly distributed works among workers (#19365) 2024-01-29 18:27:37 +00:00
Jirka Borovec 9d35c61f5f
ci: adding missing requirements for generating legacy ckpt (#19353) 2024-01-28 11:22:07 +01:00
awaelchli 1a59097ab2
Drop support for PyTorch 1.12 (#19300) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
 2024-01-26 11:44:24 -05:00
Jirka Borovec 3bd133b107
CI: enable testing with coming PT 2.2 (#19289) 
* ci: build dockers for PT 2.2
* py3.12
* --pre --extra-index-url
* typing-extensions
* bump jsonargparse
* install latest jsonargparse
* Add windows skips for Fabric
* convert to xfail
* add pytorch skips
* skip checkpoint consolidation test
* set max torch

---------

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-01-26 16:42:09 +01:00
thomas chaton ee9f17eb3c
Downloader: Resolve race condition (#19348) 2024-01-25 15:36:42 +00:00