Commit Graph

31 Commits

Author SHA1 Message Date
Kaushik B 2187f309c1 Support auto_select_gpus with accelerator and devices api (#12608)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-04-13 14:14:29 -04:00
Adrian Wälchli 4326f14b76 Fix `LightningLite.run` signature for arbitrary arguments (#12629) 2022-04-13 14:14:29 -04:00
four4fish 6e14209185
Rewrite accelerator_connector (#11448) 2022-02-17 23:38:39 +00:00
Rohit Gupta 25b505508d
Add process launchers (#11643)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-02-17 21:16:51 +00:00
edward-io 87bd54aedf
fix typos (#11937) 2022-02-16 17:27:51 -08:00
Kaushik B 0adcd6a048
Rename training_type_plugin file to strategy (#11239)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-23 14:01:23 +00:00
Adrian Wälchli c210e338ef
Update strategy import statements (#11231) 2021-12-23 08:26:28 +01:00
four4fish 81301dbba7
Rename `AcceleratorConnector.training_type_plugin` to `AcceleratorConnector.strategy` (#11212)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-23 01:36:23 +00:00
Kaushik B 576a5d62a0
Introduce strategies directory for Training Strategies (#11226)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-22 20:23:30 +00:00
four4fish f98cd78e9e
Renamed the `DDPSpawnPlugin` to `DDPSpawnStrategy` (#11145) 2021-12-21 23:06:14 +00:00
Aki Nitta 9da78a94bd
Rename `TPUSpawnPlugin` to `TPUSpawnStrategy` (#11190)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-21 16:36:16 +00:00
Kaushik B 283bdece0a
Rename DeepSpeedPlugin to DeepSpeedStrategy (#11194) 2021-12-21 15:18:01 +00:00
Adrian Wälchli 29eb9cccf2
Rename the `TrainingTypePlugin` base to `Strategy` (#11120)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: four4fish <88516121+four4fish@users.noreply.github.com>
2021-12-20 12:50:11 +00:00
four4fish cec2d7946b
3/n Move accelerator into Strategy (#11022)
* remove training_step() from accelerator

* remove test, val, predict step

* move

* wip

* accelerator references

* cpu training

* rename occurrences in tests

* update tests

* pull from adrian's commit

* fix changelog merge pro

* fix accelerator_connector and other updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix doc build and some mypy

* fix lite

* fix gpu setup environment

* support customized ttp and accelerator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tpu error check

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix precision_plugin initialization to recognisze cusomized plugin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update bug_report_model.py

* Update accelerator_connector.py

* update changelog

* allow shorthand typing references to pl.Accelerator

* rename helper method and add docstring

* fix typing

* Update pytorch_lightning/trainer/connectors/accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_cpu.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pre commit complaint

* update typing to long ugly path

* spacing in flow diagram

* remove todo comments

* docformatter

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

* revert test changes

* improve custom plugin examples

* remove redundant call to ttp attribute

it is no longer a property

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-12-16 04:41:34 +00:00
Adrian Wälchli 7514adf814
Remove `return_result` argument from `DDPSpawnPlugin.spawn()` (#10867) 2021-12-01 13:29:08 +00:00
Adrian Wälchli a81accb2ad
Update LiteOptimizer signature after optimizer changes in TrainingTypePlugin (#10708) 2021-11-30 15:16:59 +01:00
Adrian Wälchli fc86c301e7
Fix typing in `pl.callbacks.timer` (#10798) 2021-11-30 14:54:08 +01:00
four4fish 1d2878523a
2/n Move Precision Plugin into strategy - move optimizer related logics (#10596)
Co-authored-by: Danielle Pintz <38207072+daniellepintz@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-11-30 08:31:23 +00:00
Adrian Wälchli 81a0a44d8f
Improve typing for Lite (#10743)
* improve typing in pytorch_lightning/lite

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* include lite again

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-26 20:14:11 +00:00
Kaushik B e0b4bb2ea3
Deprecate `DeviceType` in favor of `_AcceleratorType` (#10503)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-25 16:41:03 +01:00
Adrian Wälchli 30ec4815cb
Support re-instantiation for custom DataLoader in Lightning (#10680)
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-11-24 15:58:51 +01:00
Adrian Wälchli ee9f7c0421
Update DeepSpeed precision handling after moving PrecisionPlugin (#10657) 2021-11-23 13:51:41 +00:00
Adrian Wälchli 15305c459c
Update DDPShardedPlugin precision handling after moving PrecisionPlugin (#10658) 2021-11-22 17:45:34 +00:00
Adrian Wälchli 8ea39d2c8f
LiteDataLoader code improvements and docs (#10625)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-21 02:33:13 +01:00
Adrian Wälchli 8950354fe4
Extract dataloader utilities from `TrainerDataLoadingMixin` (#10145) 2021-11-19 12:45:35 +00:00
four4fish 700521c7d3
1/n Move precision plugin into strategy - update reference (#10570)
* 1/n move precision plugin into strategy - update reference

* update precision plugin reference in tpu_spawn

* add missing reference in error message

* add back removed license line

* update references in tests

* update reference in trainer

* update return annotation for precision_plugin property on TTP

* simplify access to precision plugin reference in sharded plug

* add changelog

* remove precision property from ttp and add deprecation message

* fix make doc and update precision reference

* simplify a reference to precision

accidentally overridden Adrian's change, now add it back

* Update CHANGELOG.md

add Adrian's change back

* Update accelerator precision

Add Adrian's change back

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add none check for precision plugin

just to be safe

* Update ipu.py

* update precision_plugin param deprecation message

* Update accelerator.py

* Remove deprecated warning 

Tests will fail after 9940

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-19 00:39:01 +00:00
Kaushik B 01cf7a2ac5
Deprecate `DistributedType` in favor of `StrategyType` (#10505) 2021-11-15 17:10:08 +00:00
Carlos Mocholí 613aa09514
Revert part of #10279 (#10376) 2021-11-08 11:28:58 +00:00
Adrian Wälchli 348fc4b49f
Move device parser utility function (#10230)
* move parser function to utils

* fix types

* keep static method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-05 10:18:56 +00:00
thomas chaton facaff94b8
Add custom dataloader support with Lite (#10279) 2021-11-01 18:33:13 +00:00
Adrian Wälchli 9d136a9fc5
Lightning Lite core and tests (#10175) 2021-10-29 21:46:39 +00:00