Commit Graph

10098 Commits

Author SHA1 Message Date
thomas chaton 1c86011dab
Add Video/Audio support (#18977)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-09 18:37:37 +00:00
Ethan Harris ec374ab307
App: Force plugin server to use localhost (#18976) 2023-11-09 13:49:22 +00:00
Victor Prins 4af961fffe
Add `@override` for files in `src/lightning/pytorch/core` (#18953) 2023-11-09 13:38:57 +01:00
Adrian Wälchli 618e1c8061
Provide the default vocab size for the transformer demo model (#18963) 2023-11-08 18:41:31 -05:00
Victor Prins 57cc01bc0a
Add `@override` for subclasses of PyTorch `Strategy` (#18911)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-11-08 17:34:34 -05:00
Adrian Wälchli 964364b3bb
Move torchmetrics to device when using FSDP (#18954) 2023-11-08 15:29:26 -05:00
Victor Prins 07461a1def
Fix broken links on Lightning App docs (#18904)
fix broken links on App docs
2023-11-08 14:08:52 -05:00
Victor Prins 2334a8acba
Add `@override` for files in `src/lightning/pytorch/loops` (#18966) 2023-11-08 19:02:34 +01:00
thomas chaton c524c0b01b
Bump Lightning Cloud Version to 0.5.52 (#18965) 2023-11-07 15:02:55 -05:00
thomas chaton 1b3a3fbaad
Prevent downloading more chunks than needed (#18964)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-07 19:40:21 +00:00
thomas chaton 20f58f63ef
Bump Lightning Cloud to 0.5.51 (#18962)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-07 17:30:39 +00:00
Adrian Wälchli 8a5d3423a7
Cache directory per worker to avoid collisions (#18957) 2023-11-07 10:19:03 -05:00
thomas chaton 529f07f254
Add support for deleting chunks (#18959)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-11-07 09:46:13 +00:00
Carlos Mocholí d9aa833628
Add more CUDA card FLOPs (#18958)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-07 04:13:20 +01:00
Adrian Wälchli 62771f3932
Greedily select files for data processor workers based on size (#18907)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-06 19:33:50 -05:00
thomas chaton e79ac21415
Add the input_dir in the cache_dir to avoid overlapping downloads (#18960)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-11-06 19:01:37 -05:00
Adrian Wälchli 195a3bf5b5
Fix parsing v100s in `get_available_flops` (#18952)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-11-06 21:50:11 +01:00
Adrian Wälchli c4af18b2c5
Create cache dir if it doesn't exist (#18955) 2023-11-06 11:02:05 -05:00
Victor Prins b30f1a995d
Add `@override` for subclasses of PyTorch `Logger` (#18948) 2023-11-06 16:18:39 +01:00
Adrian Wälchli 3b05d833cc
Fix symlink permission error for "last" checkpoint on Windows (#18942) 2023-11-06 16:10:53 +01:00
Adrian Wälchli 54093a4187
Remove redundant Trainer state assignment (#18929) 2023-11-06 12:04:56 +01:00
Jirka Borovec 6628d8a73a
docs: switch todo to comment (#18893) 2023-11-06 11:50:38 +01:00
thomas chaton 97c730e7cb
Add GPU support for map (#18947)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-06 09:47:40 +00:00
Victor Prins ddc1ce1fda
Add `@override` for subclasses of `Accelerator` (#18934) 2023-11-05 23:06:38 +01:00
thomas chaton 3a8609755c
lightning.data: Fix some bugs with optimize (#18949)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-05 13:06:02 +00:00
Adrian Wälchli 0e7a3b0b5f
Fix oversized items not fitting into a chunk (#18938)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-05 10:28:18 +00:00
Adrian Wälchli 4af77c6bf1
Install both `lightning` and `pytorch-lightning` in docker image (#18898) 2023-11-04 10:29:37 -04:00
thomas chaton faa64c593d
Add dataset creation (#18940)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-04 00:27:42 +00:00
Jason Won 8d68607cef
Flatten dataclass hyperparameters for logging (#18906)
Co-authored-by: jaswon <jason@jwon.xyz>
2023-11-03 19:30:19 -04:00
Jirka Borovec ed7cc27d57
docs: ignore tutorial's link "what-is-a-gpu" (#18937) 2023-11-03 18:59:42 -04:00
thomas chaton f9e82c68f5
Resolve bug with the uploader (#18939)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-03 19:43:55 +00:00
thomas chaton f5f4d0a264
Improve map, optimize and StreamingDataset (#18912)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-03 11:28:55 +00:00
Victor Prins 809e952434
Add `@override` for subclasses of PyTorch `_Launcher` (#18922) 2023-11-03 02:11:39 +01:00
Carlos Mocholí 90d3d1ca7e
Add tfloat32 to throughput (#18927) 2023-11-02 20:21:49 +01:00
thomas chaton 6a0f992d53
Improve s3 client support (#18920)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/lightning/data/streaming/client.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* update

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-11-02 13:10:25 -04:00
Jirka Borovec e9dcf4840e
ci/docs: fix copy integration sub-docs (#18921) 2023-11-02 17:45:00 +01:00
Carlos Mocholí 2b6b594dab
Rename Throughput flops argument (#18924) 2023-11-02 16:06:40 +01:00
thomas chaton 37cbee42c6
Add human readable format for chunk_bytes (#18925)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-02 15:06:11 +00:00
Carlos Mocholí 80fbc0ac83
Update minimum typing-extensions to 4.4 (#18916) 2023-11-02 15:51:07 +01:00
Carlos Mocholí ad93f64d35
Bitsandbytes docs improvements (#18903) 2023-11-02 15:50:54 +01:00
Jirka Borovec 1e68c50141
ci: restrict build docs on PR (#18923)
* ci: restrict build docs on PR

* linkcheck_timeout
2023-11-02 09:03:03 -04:00
Carlos Mocholí 5f6669f6b3
Add batches argument to throughput (#18905)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-11-02 04:15:03 +01:00
Adrian Wälchli 13e192637c
Fix bitsandbytes layer conversion under `init_module` context manager (#18914) 2023-11-01 14:10:57 -04:00
Adrian Wälchli 98685c332b
Fix parsing of version in TensorBoardLogger and CSVLogger (#18897) 2023-11-01 12:48:36 -04:00
Adrian Wälchli 7a5b7f5561
Skip hanging collective test (#18908) 2023-11-01 15:45:25 +01:00
thomas chaton 85933f355a
Improve map and chunkify (#18901)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-01 09:35:35 +00:00
Adrian Wälchli 31b8777350
Update CLI tests to no longer require 3rd party logger dependencies (#18899) 2023-10-31 09:22:17 -04:00
Adrian Wälchli 018a308269
Enable RUF018 rule for walrus assignments in asserts (#18886) 2023-10-30 21:16:02 -04:00
Adrian Wälchli e0ba4d46e1
Fix spelling errors (#18773)
fix spelling errors
2023-10-30 17:14:52 -04:00
Adrian Wälchli 079544a902
Rename PrecisionPlugin -> Precision (#18840) 2023-10-30 16:53:13 -04:00