Adrian Wälchli
|
da79480054
|
PyTest random order for Fabric tests (#19040)
|
2023-11-22 16:41:49 -05:00 |
|
thomas chaton
|
7eca9c1642
|
Add numpy support for the StreamingDataset 1/2 (#19050)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-22 18:00:15 +00:00 |
|
thomas chaton
|
1073276a58
|
Add fault tolerance for the StreamingDataset 1/n (#19049)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-22 17:22:00 +00:00 |
|
thomas chaton
|
bc1658039f
|
Add direct s3 support to the streaming dataset (#19044)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-22 01:17:49 +00:00 |
|
Yasser Souri
|
67d3844818
|
Fix last checkpoint finding in filtered files with correct extension (#17072)
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
|
2023-11-21 17:12:02 -05:00 |
|
Adrian Wälchli
|
d4614d043e
|
Address test flakiness (#19022)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
|
2023-11-21 17:11:00 -05:00 |
|
Adrian Wälchli
|
e3be762538
|
Re-enable dynamo tests that were fixed in PyTorch 2.1 (#19038)
|
2023-11-21 16:30:20 -05:00 |
|
thomas chaton
|
d3df1273b6
|
Add disk usage check before downloading files (#19041)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-21 20:10:18 +00:00 |
|
Adrian Wälchli
|
49caddde6e
|
Call `configure_model()` in `LM.load_from_checkpoint()` (#19036)
|
2023-11-21 09:44:18 -05:00 |
|
Adrian Wälchli
|
f652e6c00e
|
Fix `rank_zero_only` rank not set in ddp-spawn based strategies (#19030)
|
2023-11-20 10:49:14 -05:00 |
|
Adrian Wälchli
|
4f4c890cd7
|
Improve handling the positional encoding in Transformer example (#18987)
|
2023-11-19 14:37:31 +01:00 |
|
Adrian Wälchli
|
45c2fcb341
|
Add AttributeDict container for Fabric (#18943)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
|
2023-11-18 09:25:26 -05:00 |
|
Carlos Mocholí
|
de7faf976b
|
Update evaluation logging test (#18896)
|
2023-11-18 08:56:47 +01:00 |
|
thomas chaton
|
6e517bd55b
|
Resolve Item Loader bugs (#19017)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2023-11-16 18:06:58 -05:00 |
|
Adrian Wälchli
|
3d448ac48d
|
Remember the eval mode of submodules when switching trainer stages (#18951)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
|
2023-11-16 16:32:27 -05:00 |
|
thomas chaton
|
792cb73fc6
|
Remove the LightningDataset relying on un-maintained torchdata (#19019)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-16 16:08:15 -05:00 |
|
PL Ghost
|
46ec46bdcc
|
Adding test for legacy checkpoint created with 2.1.x (#18789)
Co-authored-by: Borda <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Jirka B <j.borovec+github@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2023-11-16 20:41:45 +01:00 |
|
Bharat Ramanathan
|
008a83ed5a
|
Improve media logging support in WandbLogger (#18164)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
|
2023-11-15 00:29:34 -05:00 |
|
thomas chaton
|
7288302186
|
Add multiple uploaders to the map, optimize (#18989)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-13 14:27:50 -05:00 |
|
Adrian Wälchli
|
340961a6ec
|
Fix test interactions (#18994)
|
2023-11-13 12:35:46 -05:00 |
|
Adrian Wälchli
|
3acea8d157
|
Fix `trainer.save_checkpoint` after `trainer.test` with FSDP (#18992)
|
2023-11-13 10:11:20 -05:00 |
|
Adrian Wälchli
|
b4605b44ee
|
Fix `ModelCheckpoint.CHECKPOINT_NAME_LAST` test interaction (#18993)
|
2023-11-12 11:01:25 +01:00 |
|
Carlos Mocholí
|
466f772e3e
|
Fix precision default from environment (#18928)
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
|
2023-11-10 23:03:51 +01:00 |
|
Ethan Harris
|
9085db4dd3
|
App: Limit rate of requests to http queue (#18981)
|
2023-11-10 10:26:58 +00:00 |
|
thomas chaton
|
1c86011dab
|
Add Video/Audio support (#18977)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-09 18:37:37 +00:00 |
|
Ethan Harris
|
ec374ab307
|
App: Force plugin server to use localhost (#18976)
|
2023-11-09 13:49:22 +00:00 |
|
Adrian Wälchli
|
964364b3bb
|
Move torchmetrics to device when using FSDP (#18954)
|
2023-11-08 15:29:26 -05:00 |
|
thomas chaton
|
1b3a3fbaad
|
Prevent downloading more chunks than needed (#18964)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-07 19:40:21 +00:00 |
|
thomas chaton
|
20f58f63ef
|
Bump Lightning Cloud to 0.5.51 (#18962)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-07 17:30:39 +00:00 |
|
Adrian Wälchli
|
8a5d3423a7
|
Cache directory per worker to avoid collisions (#18957)
|
2023-11-07 10:19:03 -05:00 |
|
thomas chaton
|
529f07f254
|
Add support for deleting chunks (#18959)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
|
2023-11-07 09:46:13 +00:00 |
|
Carlos Mocholí
|
d9aa833628
|
Add more CUDA card FLOPs (#18958)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2023-11-07 04:13:20 +01:00 |
|
Adrian Wälchli
|
62771f3932
|
Greedily select files for data processor workers based on size (#18907)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-06 19:33:50 -05:00 |
|
thomas chaton
|
e79ac21415
|
Add the input_dir in the cache_dir to avoid overlapping downloads (#18960)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
|
2023-11-06 19:01:37 -05:00 |
|
Adrian Wälchli
|
195a3bf5b5
|
Fix parsing v100s in `get_available_flops` (#18952)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
|
2023-11-06 21:50:11 +01:00 |
|
Adrian Wälchli
|
c4af18b2c5
|
Create cache dir if it doesn't exist (#18955)
|
2023-11-06 11:02:05 -05:00 |
|
Adrian Wälchli
|
3b05d833cc
|
Fix symlink permission error for "last" checkpoint on Windows (#18942)
|
2023-11-06 16:10:53 +01:00 |
|
thomas chaton
|
97c730e7cb
|
Add GPU support for map (#18947)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-06 09:47:40 +00:00 |
|
thomas chaton
|
3a8609755c
|
lightning.data: Fix some bugs with optimize (#18949)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-05 13:06:02 +00:00 |
|
Adrian Wälchli
|
0e7a3b0b5f
|
Fix oversized items not fitting into a chunk (#18938)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2023-11-05 10:28:18 +00:00 |
|
thomas chaton
|
faa64c593d
|
Add dataset creation (#18940)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-04 00:27:42 +00:00 |
|
Jason Won
|
8d68607cef
|
Flatten dataclass hyperparameters for logging (#18906)
Co-authored-by: jaswon <jason@jwon.xyz>
|
2023-11-03 19:30:19 -04:00 |
|
thomas chaton
|
f9e82c68f5
|
Resolve bug with the uploader (#18939)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-03 19:43:55 +00:00 |
|
thomas chaton
|
f5f4d0a264
|
Improve map, optimize and StreamingDataset (#18912)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-03 11:28:55 +00:00 |
|
thomas chaton
|
6a0f992d53
|
Improve s3 client support (#18920)
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update src/lightning/data/streaming/client.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
|
2023-11-02 13:10:25 -04:00 |
|
Carlos Mocholí
|
2b6b594dab
|
Rename Throughput flops argument (#18924)
|
2023-11-02 16:06:40 +01:00 |
|
thomas chaton
|
37cbee42c6
|
Add human readable format for chunk_bytes (#18925)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
|
2023-11-02 15:06:11 +00:00 |
|
Carlos Mocholí
|
5f6669f6b3
|
Add batches argument to throughput (#18905)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
|
2023-11-02 04:15:03 +01:00 |
|
Adrian Wälchli
|
98685c332b
|
Fix parsing of version in TensorBoardLogger and CSVLogger (#18897)
|
2023-11-01 12:48:36 -04:00 |
|
Adrian Wälchli
|
7a5b7f5561
|
Skip hanging collective test (#18908)
|
2023-11-01 15:45:25 +01:00 |