Commit Graph

24 Commits

Author SHA1 Message Date
Kaushik B 49420548ce
Update warnings for available accelerators not being used (#11909)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-02-24 09:12:02 +01:00
Carlos Mocholí 4c4b9d540f
Remove `Trainer._device_type` (#11992) 2022-02-22 13:02:13 +00:00
four4fish 6e14209185
Rewrite accelerator_connector (#11448) 2022-02-17 23:38:39 +00:00
ananthsub 1b107c5892
Add `Accelerator.is_available()` interface requirement (#11797) 2022-02-09 15:11:27 -08:00
Adrian Wälchli c210e338ef
Update strategy import statements (#11231) 2021-12-23 08:26:28 +01:00
Kaushik B 576a5d62a0
Introduce strategies directory for Training Strategies (#11226)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-22 20:23:30 +00:00
four4fish cf5ef32f7b
Deprecate Trainer.training_type_plugin in favor of trainer.strategy (#11141)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-22 02:11:43 +00:00
Aki Nitta c3cd4d050f
Rename `SingleTPUPlugin` to `SingleTPUStrategy` (#11182) 2021-12-21 20:09:30 +00:00
Aki Nitta 9da78a94bd
Rename `TPUSpawnPlugin` to `TPUSpawnStrategy` (#11190)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-21 16:36:16 +00:00
four4fish b64dea9dc3
Rename `DDPPlugin` to `DDPStrategy` (#11142)
* Raname DDPPlugin to DDPStrategy

* Change ddp_plugin to ddp_strategy

* update changelog

* rename occurences in docs

* rename more occurrences

* fix line too long

* more fixes

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-21 08:55:51 +00:00
four4fish cec2d7946b
3/n Move accelerator into Strategy (#11022)
* remove training_step() from accelerator

* remove test, val, predict step

* move

* wip

* accelerator references

* cpu training

* rename occurrences in tests

* update tests

* pull from adrian's commit

* fix changelog merge pro

* fix accelerator_connector and other updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix doc build and some mypy

* fix lite

* fix gpu setup environment

* support customized ttp and accelerator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tpu error check

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix precision_plugin initialization to recognisze cusomized plugin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update bug_report_model.py

* Update accelerator_connector.py

* update changelog

* allow shorthand typing references to pl.Accelerator

* rename helper method and add docstring

* fix typing

* Update pytorch_lightning/trainer/connectors/accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_cpu.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pre commit complaint

* update typing to long ugly path

* spacing in flow diagram

* remove todo comments

* docformatter

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

* revert test changes

* improve custom plugin examples

* remove redundant call to ttp attribute

it is no longer a property

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-12-16 04:41:34 +00:00
four4fish 700521c7d3
1/n Move precision plugin into strategy - update reference (#10570)
* 1/n move precision plugin into strategy - update reference

* update precision plugin reference in tpu_spawn

* add missing reference in error message

* add back removed license line

* update references in tests

* update reference in trainer

* update return annotation for precision_plugin property on TTP

* simplify access to precision plugin reference in sharded plug

* add changelog

* remove precision property from ttp and add deprecation message

* fix make doc and update precision reference

* simplify a reference to precision

accidentally overridden Adrian's change, now add it back

* Update CHANGELOG.md

add Adrian's change back

* Update accelerator precision

Add Adrian's change back

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add none check for precision plugin

just to be safe

* Update ipu.py

* update precision_plugin param deprecation message

* Update accelerator.py

* Remove deprecated warning 

Tests will fail after 9940

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-19 00:39:01 +00:00
Ning f6ed0bd8ca
introduce has_len_all_ranks() to check the length of dataloader across ranks (#9827)
* introduce , udpate tests

* update CHANGELOG.md

* change staticmethod and hook attribute naming

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix typo

* remove non-essential comment

* fix merge error and comment format

* try to fix test_tpu.py failure

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update on comments

* chlog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* chlog

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* try fix

* Revert back TPUSpawn changes

* Update test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
2021-11-02 13:22:58 -04:00
Kaushik B e0f7dbdd1c
Add support for `devices='auto'` (#10264) 2021-10-30 15:05:51 +00:00
Carlos Mocholí e4eb61d812
Raise exception for `strategy=ddp_cpu|tpu_spawn` (#10185) 2021-10-29 16:15:24 +00:00
Kaushik B c33df2639f
Set `dataset` attribute to `MpDeviceLoader` used in TPU Spawn (#10151) 2021-10-27 01:23:01 +05:30
jjenniferdai 6d79184ec5
Unify checkpoint load paths [redo #9693] (#10061) 2021-10-25 19:05:31 +00:00
Kaushik B aa1540410f
Add XLACheckpointIO (#9972)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-10-21 02:39:16 +05:30
Carlos Mocholí e8beceb631
Add `TPUPrecisionPlugin` (#10020)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-10-19 17:48:57 +00:00
Carlos Mocholí 3f355d0eb7
Remove manual tracking of optimizer steps (#9957) 2021-10-18 12:43:06 +00:00
Carlos Mocholí 01b304ec57
Update accelerator connector messages after the addition of strategy (#9937) 2021-10-18 01:10:48 +00:00
Kaushik B af4a8f1950
Refactor tests for TPU Accelerator (#9718)
Co-authored-by: tchaton <thomas@grid.ai>
2021-10-14 19:45:15 +00:00
Danielle Pintz 940b910d27
[2/4] Add DeviceStatsMonitor callback (#9712)
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2021-10-13 18:29:36 +00:00
Danielle Pintz ab069876cb
[1/4] Add get_device_stats to accelerator interface (#9586) 2021-09-26 21:09:16 -07:00