8bf7f9cce7
1/n Move Accelerator into strategy - move batch_to_device to strategy ( #10649 )
...
* 1/n Integrate Device Specific Accelerator Logic with strategy - move batch_to_device to strategy
* add changelog
* add model is not none check
* Apply suggestions from code review
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update CHANGELOG.md
* Update test_datamodules.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test_hooks.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update dp.py
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-29 12:11:21 -08:00
753cc4dfad
Fix default logging levels for train step specific hooks ( #10756 )
2021-11-29 19:51:17 +00:00
088818fbc6
Do not sanity check on reload ( #10785 )
2021-11-29 16:23:01 +01:00
e8659bd40e
update NGC ( #10770 )
2021-11-29 14:14:37 +00:00
d3b7492bd0
[CLI] Add support for `--key.help=class` ( #10767 )
2021-11-29 14:12:53 +00:00
24fc54f07b
Fix typing in `pl.overrides.fairscale` ( #10799 )
...
* update typing in fairscale
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-29 11:28:28 +00:00
bd3fb2e66e
Fix typing in `pl.core.mixins.hparams_mixin` ( #10800 )
...
* fix typing in hparams mixin
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unused import
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-29 12:03:47 +01:00
97e52619ea
Fix typing in `pl.overrides.data_parallel` ( #10796 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-29 10:58:23 +01:00
724a92b065
Mark outputs as protected in the evaluation loops ( #10781 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-11-28 20:09:30 +00:00
e1bf54c944
Tune Conda CI timeout and other minor improvements ( #10769 )
2021-11-28 17:58:03 +00:00
3f915aaaf9
Fix reference link to `s3fs` ( #10737 )
2021-11-28 17:25:03 +00:00
4a341e5b24
Add remote filesystems to docs ( #10752 )
...
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-11-28 16:46:52 +00:00
49d09aa28b
Update changelog after 1.5.3 release ( #10744 )
2021-11-27 05:28:23 +00:00
c752060712
Consolidate state when retrieving sharded state dict in Lite ( #10746 )
...
Co-authored-by: thomas chaton <thomas@grid.ai>
2021-11-27 04:54:45 +00:00
038c151b6e
Improve typing for plugins ( #10742 )
...
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-11-26 20:14:58 +00:00
81a0a44d8f
Improve typing for Lite ( #10743 )
...
* improve typing in pytorch_lightning/lite
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* include lite again
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-26 20:14:11 +00:00
e94aff1c5b
Fault Tolerant: Add support for fault tolerant dataloader validator ( #10465 )
2021-11-26 19:33:47 +00:00
88930725dd
Add a custom `PossibleUserWarning` category ( #10675 )
2021-11-26 19:29:42 +00:00
3089dc3829
Improve typing for loops ( #10749 )
...
* Improve typing for loops
* Free memory
2021-11-26 18:39:09 +00:00
78face65e8
Improve typing for logging ( #10748 )
...
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-11-26 18:12:21 +00:00
31bb6e69ca
Avoid optional instances in Loops ( #10735 )
...
* Avoid optional instances in Loops
* More cleanup
2021-11-26 18:00:18 +00:00
ae53562c97
Remove dead code in `TrainingEpochLoop` ( #10750 )
2021-11-26 17:49:00 +00:00
152eb57def
Rename special to standalone ( #10779 )
2021-11-26 17:13:14 +00:00
6fe6e9e414
Delete TensorBoardLogger experiment before spawning the processes. ( #10777 )
2021-11-26 17:07:57 +00:00
412d507a73
Fault Tolerant: move signal to SIGTERM ( #10605 )
2021-11-26 13:37:27 +00:00
e507bc9027
Fix compare version for packages ( #10762 )
2021-11-26 09:15:22 +00:00
3d6262b7a9
Fault Tolerant Manual: Add support for DDP ( #10638 )
2021-11-25 18:31:53 +01:00
e0b4bb2ea3
Deprecate `DeviceType` in favor of `_AcceleratorType` ( #10503 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-25 16:41:03 +01:00
b57feccbff
Be explicit with mypy ignores ( #10751 )
...
* Ignore mypy only for failing files
* Comment
2021-11-25 11:27:13 +01:00
85d7c4dce4
Configure mypy to install dependencies in CI and update pyproject.toml ( #10682 )
...
* mypy install deps
* fix deps
* add examples
* fix type errors
* fix type error
* fix
* fix
* update pyproject.toml
2021-11-25 00:19:30 +00:00
f8b2d5b128
Improve error message on `TypeError` during `DataLoader` reconstruction ( #10719 )
2021-11-24 21:51:11 +00:00
0066ff0129
Fault Tolerant Manual: Enable the feature ( #10707 )
2021-11-24 17:36:08 +00:00
30ec4815cb
Support re-instantiation for custom DataLoader in Lightning ( #10680 )
...
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-11-24 15:58:51 +01:00
e51a8ee7a3
Fault Tolerant Manual: utilities cleanup ( #10703 )
...
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-11-24 15:01:55 +01:00
f36b395c4e
Update `LightningDataModule` docs ( #10678 )
2021-11-24 11:31:03 +00:00
89d0064b33
Use `PrecisionType` enum instead of checking raw values ( #10704 )
...
* use precision type
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-23 18:23:36 +01:00
b28ab34ff5
Fault Tolerant Manual: Add loading to reload the states ( #10699 )
...
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-23 17:18:36 +00:00
dca1776870
LiteDataLoader wrapper improvements ( #10297 )
2021-11-23 16:35:07 +01:00
7cf6374bd0
Fault Tolerant Manual: Add support for collecting states across processes ( #10639 )
2021-11-23 14:27:33 +00:00
ee9f7c0421
Update DeepSpeed precision handling after moving PrecisionPlugin ( #10657 )
2021-11-23 13:51:41 +00:00
1702036c14
Fault Tolerant Manual: Add stateful dataloader iter ( #10674 )
2021-11-23 12:30:50 +00:00
48cf1adfd3
Move Colab setup to ProgressBar ( #10542 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-23 06:16:31 +00:00
2036dfb5df
Fault Tolerant Manual: Add _rotate_worker_indices utility ( #10647 )
2021-11-22 19:52:04 +00:00
823bfa6f8a
Update `LightningModule` docs ( #10637 )
2021-11-23 01:02:04 +05:30
6acfef680f
Fault Tolerant Manual: Add is_obj_stateful utility ( #10646 )
2021-11-22 18:48:32 +00:00
cd7b4342f6
remove import of datasets separately since unused ( #10668 )
2021-11-22 17:54:19 +00:00
15305c459c
Update DDPShardedPlugin precision handling after moving PrecisionPlugin ( #10658 )
2021-11-22 17:45:34 +00:00
6fc7c54c3a
refactor slurm_job_id ( #10622 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
2021-11-22 17:41:08 +00:00
338f3cf636
Use `Set` operations in `Environment.detect` ( #10673 )
2021-11-22 23:03:47 +05:30
d431ce14a1
Raise an error if batch_size cannot be inferred from current batch ( #10541 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-22 16:55:19 +00:00
four4fish