Commit Graph

37 Commits

Author SHA1 Message Date
Jirka Borovec 51ad6bb8fe
docs: pin version in links to external docs [1/n] (#19584)
* docs: ignore link to private repo
* lightning-utilities >=0.11.0, <0.12.0

---------

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
2024-03-19 16:35:40 +01:00
Jirka Borovec 0b88204c6a
bump `lightning-utilities` from <0.10 to <0.11 (#19586) 2024-03-07 11:20:57 -05:00
awaelchli f2f9978377
Update mypy in CI (#19449) 2024-02-13 12:09:35 +01:00
awaelchli 1a59097ab2
Drop support for PyTorch 1.12 (#19300)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2024-01-26 11:44:24 -05:00
Jirka Borovec 3bd133b107
CI: enable testing with coming PT 2.2 (#19289)
* ci: build dockers for PT 2.2
* py3.12
* --pre --extra-index-url
* typing-extensions
* bump jsonargparse
* install latest jsonargparse
* Add windows skips for Fabric
* convert to xfail
* add pytorch skips
* skip checkpoint consolidation test
* set max torch

---------

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-01-26 16:42:09 +01:00
Adrian Wälchli 710cac4ce9
Make fsspec requirement the same across subpackages (#19085) 2023-11-29 10:36:28 -05:00
Adrian Wälchli b8a96fe5d6
Fix fsspec local file protocol checks for new fsspec version (#19023)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-18 08:51:37 +01:00
Carlos Mocholí 80fbc0ac83
Update minimum typing-extensions to 4.4 (#18916) 2023-11-02 15:51:07 +01:00
Adrian Wälchli c514f1cbea
Enable PyTorch 2.1 (#18718) 2023-10-06 07:17:03 -04:00
Adrian Wälchli d31ef1f7d3
Drop support for PyTorch 1.11 (#18691)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-10-04 20:30:44 +02:00
dependabot[bot] 74b2ff8196
Update fsspec[http] requirement from <2023.7.0,>2021.06.0 to >2021.06.0,<2023.10.0 in /requirements (#18469)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-10-03 20:52:20 +02:00
dependabot[bot] 99d28a355f
Update numpy requirement from <1.25.3,>=1.17.2 to >=1.17.2,<1.27 in /requirements (#18624)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-10-02 22:53:27 +02:00
Jirka Borovec 7834bb6377
relax some dependencies from `<=` to `<` (#18435)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-08-31 16:47:06 +02:00
dependabot[bot] 0222d421e5
Update packaging requirement from <=23.0,>=20.0 to >=20.0,<=23.1 in /requirements (#18354)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-21 14:55:10 +02:00
dependabot[bot] cee05ea029
Update fsspec[http] requirement from <2023.5.0,>2021.06.0 to >2021.06.0,<2023.7.0 in /requirements (#18248)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-08 11:50:35 +02:00
dependabot[bot] 175aaafe1e
Update numpy requirement from <1.25.2,>=1.17.2 to >=1.17.2,<1.25.3 in /requirements (#18249)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-08 11:50:18 +02:00
dependabot[bot] 10aa00faeb
Update numpy requirement from <1.25.1,>=1.17.2 to >=1.17.2,<1.25.2 in /requirements (#18035)
Update numpy requirement in /requirements

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](https://github.com/numpy/numpy/compare/v1.17.2...v1.25.1)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-01 18:21:39 +00:00
Carlos Mocholí 27cab24ca3
bump: typing-extensions >=4.0.0, <=4.7.1 (#18125)
* Update max typing extensions

* Fix some app tests too

* "trigger ci"

* xfail on 3.9
2023-07-21 16:46:18 +02:00
Carlos Mocholí 01b82e4fb1
Minor miscellaneous fixes (#18077)
* Various miscellaneous fixes

* Update

* Update

* succeeded

* Comment everywhere

* hasattr
2023-07-20 14:44:51 +02:00
dependabot[bot] efc9852354
Update numpy requirement from <1.24.4,>=1.17.2 to >=1.17.2,<1.25.1 in /requirements (#17860)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-03 21:51:40 +00:00
Jirka Borovec 37c244f94b
bump Lit-Utils to 0.9 (#17955) 2023-07-03 17:49:00 +00:00
Noha Alon ca30fd7752
Lightning Dataset (including optimized dataloading of s3 buckets) (#17743)
* Lightning DataLoader

* lightning dataloader

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* init

* example

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* env var

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update src/lightning/pytorch/utilities/data/__init__.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* remove unused functions

* extra reqs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update src/lightning/pytorch/utilities/data/fileio.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* imports work now! yay

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* imports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* missing import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* error handling

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update creds for local use case

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* codeowners

* recursive get index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean up get index

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update imagenet example

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* example cleanup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* changelog

* reqs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* codeowners

* requirements

* expose LightningDataset too

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* expost LightningDataset at top level

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused private methods from init

* remove private imports

* upper bound on extra requirements

* review comments

* loosen req

* deps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test updating fabric base req

* remove version pin on s3fs to test

* recover missing function

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests

* update

* random

* torchdata >= 0.3.0

* update torchdata version

* remove torchdata version to test

* try rem torch version pin

* req

* update bucket in test

* req

* skips

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* import

* update structure to lightning.data

* base.txt for data reqs

* fix imports

* rename to LightningS3Dataset

* new workflow

* dont need to test warnings

* reqs

* req

* revert data folder in pytorch

* test import

* tests

* req

* req

* req

* torch version

* req

* req

* open dep

* reformatted

* pin strict

* pin strict extra

* req

* modify workflow, no cache

* try

* patch

* import

* fix

* dataset test

* update getattr

* pin everything to test

* remove torch preinstall from workflow

* workflow

* req

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow

* workflow

* req

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow

* print

* skip test for now

* update path join

* revert app dep version bump

* Update .github/workflows/ci-tests-data.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* workflow updates

* app base req

* req

* window test failure

* add data req to assistant

* try

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add missing comma

* updates

* update

* typo

* requirements

* try widening req

* older torch version

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update

* update

* update

* cleanup tests

* typo again

* update

* remove unnecessary line

* Update .github/CODEOWNERS

* Discard changes to requirements/pytorch/base.txt

* Discard changes to requirements/fabric/base.txt

* Discard changes to requirements/app/base.txt

* requirements

* requirements

* one line

* app workflow pick only app reqs

* rename package

* undo

* don't use cache

* examples CI

* pytorch and fabric CI

* try remove cache

* Apply suggestions from code review

* jirka playing

* jirka playing

* jirka playing

* blah

* flatten LightningDataset

* cleans up dataset class

* jirka playing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* jirka playing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* extra

* fix dataset test

* update checkgroups

* Luca's review comments

* val error fix

* unskip test

* min

* fix precommit warning

* cpu

* docstrings

* req

* 2.0.1

* add return type

* typing errors

* req

* return types with quotations

* import for type-checking

* no botocore in cloudagnostic code

* exit args

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* backends typing

* remove oldest from data tests

* typing

* typing

* typing

* types

* type

* typing

* typing

* typing

* import fix

* Changelog

---------

Co-authored-by: Noha Alon <nohaalon@Nohas-MacBook-Air.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Justus Schock <justus.schock@posteo.de>
2023-06-13 11:44:41 +01:00
Jirka Borovec d75bfe5c38
bump `torch` requirements to 2.0.1 (#17783) 2023-06-12 13:06:45 +00:00
dependabot[bot] 294c3b8cca
Update numpy requirement from <1.24.2,>=1.17.2 to >=1.17.2,<1.24.4 in /requirements (#17538)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-02 13:48:26 +00:00
dependabot[bot] 7c6d42a6b9
Update fsspec[http] requirement from <2023.2.0,>2021.06.0 to >2021.06.0,<2023.5.0 in /requirements (#17536)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 18:14:27 +02:00
Jirka Borovec 44ce386135
req: formatting (#17335) 2023-04-24 09:19:20 +01:00
Carlos Mocholí 05b481e3ae
[TPU] Add testing matrix with PJRT (#17368)
* Replace GKE in CI with manual gcloud usage

* Fix XRT test

* Reduce timeout to 35 minutes

* [TPU] Run tests with PJRT

* runtime as part of the job name

* CHANGELOG

* Update for app too
2023-04-14 16:39:13 +02:00
Adrian Wälchli c93952cb2c
Make BYOT imports forward compatible (#16997) 2023-03-13 21:55:32 -04:00
janEbert dd02397720
Allow frozen data classes in optimizer state dict (#16656)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-03-10 15:37:18 +00:00
dependabot[bot] 28ab88c9c4
Update fsspec[http] requirement from <2022.8.0,>2021.06.0 to >2021.06.0,<2023.2.0 in /requirements (#16544)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 15:57:38 +00:00
Carlos Mocholí ef2a6088ff
Drop support for PyTorch 1.10 (#16492)
* Drop support for PyTorch 1.10

* CHANGELOG

* READMEs

* mypy

* ls

* New poplar version

* Fixed tests

* links

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* skip azure badges

* Table

* Matching dockerfiles

* Drop unnecessary channels and packages

* Push nightly

* Undo unrelated changes

* Revert "Push nightly"

This reverts commit 9618f737c4.

---------

Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-02-01 14:09:12 -05:00
Jirka Borovec 22c34658f4
relax `packaging` versions (#16508) 2023-01-26 12:42:57 +00:00
Adrian Wälchli 961fa6a0ea
Fix strict torch_xla availability check (#16476)
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2023-01-25 18:52:33 +00:00
dependabot[bot] ba3ab2f9db
Update numpy requirement from <1.24.1,>=1.17.2 to >=1.17.2,<1.24.2 in /requirements (#16473)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 03:59:05 +00:00
dependabot[bot] 6f31d68e73
Update lightning-utilities requirement from <0.5.0,>=0.4.2 to >=0.4.2,<0.6.0 in /requirements (#16371)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-17 01:35:09 +00:00
dependabot[bot] cc387bd527
Bump pytest-rerunfailures from 10.2 to 10.3 in /requirements (#15841)
Bumps [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) from 10.2 to 10.3.
- [Release notes](https://github.com/pytest-dev/pytest-rerunfailures/releases)
- [Changelog](https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst)
- [Commits](https://github.com/pytest-dev/pytest-rerunfailures/compare/10.2...10.3)

---
updated-dependencies:
- dependency-name: pytest-rerunfailures
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 15:34:10 +00:00
Carlos Mocholí 15ef52bc73
Rename LightningLite to Fabric (#16244)
* Rename LightningLite to Fabric

* Fix introspection test

* Fix deprecated Lite tests

* Undo accidental Horovod removal

* Fixes
2023-01-04 10:57:18 -05:00