475d7d0bd6
Add cpu device parser to validate cpu devices ( #12160 )
2022-03-28 20:33:04 +05:30
939d56c6d6
Drop PyTorch 1.7 support ( #12432 )
2022-03-27 21:31:20 +00:00
812c2dc3d3
Add support for Habana accelerator (HPU) ( #11808 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: four4fish <88516121+four4fish@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: jjenniferdai <89552168+jjenniferdai@users.noreply.github.com>
Co-authored-by: Kushashwa Ravi Shrimali <kushashwaravishrimali@gmail.com>
Co-authored-by: Akarsha Rao <94624926+raoakarsha@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.comk-Pro.local>
2022-03-25 10:24:52 +00:00
dcc973e019
Add `AcceleratorRegistry` ( #12180 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
2022-03-24 18:29:32 +00:00
89d37569d8
add `accelerator.is_available()` check ( #12104 )
...
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
2022-03-02 10:07:49 +00:00
a52a6ea030
Add support for pluggable Accelerators ( #12030 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2022-02-28 21:36:23 +05:30
1b107c5892
Add `Accelerator.is_available()` interface requirement ( #11797 )
2022-02-09 15:11:27 -08:00
350c88e621
Let Accelerator inherit from ABC to make sure abstractmethod takes effect ( #11521 )
2022-01-23 20:47:43 +01:00
5914fb748f
Add typing to accelerators/gpu.py ( #11333 )
2022-01-12 19:44:51 +00:00
3692eba807
Drop Python 3.6 support ( #11117 )
2021-12-21 17:06:15 +00:00
cec2d7946b
3/n Move accelerator into Strategy ( #11022 )
...
* remove training_step() from accelerator
* remove test, val, predict step
* move
* wip
* accelerator references
* cpu training
* rename occurrences in tests
* update tests
* pull from adrian's commit
* fix changelog merge pro
* fix accelerator_connector and other updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix doc build and some mypy
* fix lite
* fix gpu setup environment
* support customized ttp and accelerator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix tpu error check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix precision_plugin initialization to recognisze cusomized plugin
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update bug_report_model.py
* Update accelerator_connector.py
* update changelog
* allow shorthand typing references to pl.Accelerator
* rename helper method and add docstring
* fix typing
* Update pytorch_lightning/trainer/connectors/accelerator_connector.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update tests/accelerators/test_accelerator_connector.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update tests/accelerators/test_cpu.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix pre commit complaint
* update typing to long ugly path
* spacing in flow diagram
* remove todo comments
* docformatter
* Update pytorch_lightning/plugins/training_type/training_type_plugin.py
* revert test changes
* improve custom plugin examples
* remove redundant call to ttp attribute
it is no longer a property
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-12-16 04:41:34 +00:00
3fcfd0214c
Remove `_call_accelerator_hook` Trainer method ( #10999 )
2021-12-09 02:27:13 +01:00
63bb4ec77d
4/n Move Accelerator into strategy - remove X_step() from accelerator ( #10890 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-06 20:16:54 +00:00
2fc64e9656
2/n Move Accelerator into strategy - remove dispatch functions from Accelerator ( #10885 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-06 09:51:14 +00:00
6fe3211573
Unroll dict input before call Accelerator X_steps ( #10908 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-03 17:00:52 +00:00
e646ca1d59
Remove `setup_optimizers_in_pre_dispatch` logic ( #10906 )
2021-12-03 15:05:08 +01:00
45dd8066e7
3/n Move Accelerator into strategy - remove model_sharded_context() ( #10886 )
...
* 3/n Move Accelerator into strategy - remove model_sharded_context()
* update ttp function
* update changelog
* update changelog
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
2021-12-02 03:34:51 +00:00
44cd412e91
Remove precision_plugin pre_dispatch() method ( #10887 )
...
* Remove precision_plugin pre_dispatch() method
* update changelog
2021-12-01 18:42:17 -08:00
1d2878523a
2/n Move Precision Plugin into strategy - move optimizer related logics ( #10596 )
...
Co-authored-by: Danielle Pintz <38207072+daniellepintz@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-11-30 08:31:23 +00:00
8bf7f9cce7
1/n Move Accelerator into strategy - move batch_to_device to strategy ( #10649 )
...
* 1/n Integrate Device Specific Accelerator Logic with strategy - move batch_to_device to strategy
* add changelog
* add model is not none check
* Apply suggestions from code review
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update CHANGELOG.md
* Update test_datamodules.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test_hooks.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update dp.py
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-29 12:11:21 -08:00
ff8ac6e2e1
Make `_get_nvidia_gpu_stats` public ( #10406 )
2021-11-19 17:52:24 +01:00
700521c7d3
1/n Move precision plugin into strategy - update reference ( #10570 )
...
* 1/n move precision plugin into strategy - update reference
* update precision plugin reference in tpu_spawn
* add missing reference in error message
* add back removed license line
* update references in tests
* update reference in trainer
* update return annotation for precision_plugin property on TTP
* simplify access to precision plugin reference in sharded plug
* add changelog
* remove precision property from ttp and add deprecation message
* fix make doc and update precision reference
* simplify a reference to precision
accidentally overridden Adrian's change, now add it back
* Update CHANGELOG.md
add Adrian's change back
* Update accelerator precision
Add Adrian's change back
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add none check for precision plugin
just to be safe
* Update ipu.py
* update precision_plugin param deprecation message
* Update accelerator.py
* Remove deprecated warning
Tests will fail after 9940
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-19 00:39:01 +00:00
aad86423f7
Remove more deprecated methods from base `Accelerator` class ( #10448 )
2021-11-10 12:58:24 +05:30
f9b9cdb0d1
Remove deprecated accelerator pass through functions in Accelerator ( #10403 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-08 17:36:37 +00:00
a270a79ed9
Rename "master" methods to "main" in ClusterEnvironment plugins ( #10103 )
...
* rename occurrences of master port, master address, maser node, master process
* rename properties
* add property decorators
* occurrences in docs
* update changelog
* update changelog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add lost method
* create deprecation
* add changelog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix typo (but it was already there!!!)
* Apply suggestions from code review
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* add todo
* update more occurences
* add types
* add missing import
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-11-08 12:32:58 +00:00
9237106451
Clip before step ( #10248 )
2021-10-30 11:27:49 +01:00
cedaebfcbb
Add `auto_device_count` method to `Accelerators` ( #10222 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-10-29 22:31:32 +02:00
81d15c5986
Implement double optimizer closure for hook structure consistency ( #10167 )
2021-10-29 13:03:04 +00:00
03f01fb5ec
Fix gradient norm tracking and gradient clipping ( #9287 )
...
* WIP
* Progress
* Undo test change
* Fix plugin closure execution order
* Update CHANGELOG
* Fix manual optimization on AMP and skipping backward
* Fix for deepspeed
* Typo
* Hook test for manual closure
* Add skipping test with AMP
* You are hideous, apex
* Add deepspeed test
* Update CHANGELOG
* Fix for broken master
* Add RunIf
* FIXMEs
* Rename
* Fix grad norm
* add a simple test
* update test
* update test
* update test
* fix merge conflicts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Sea of changes
* Undo change
* Introduce TPUPrecisionPlugin
* Undo changes
* Undo changes
* Resolve FIXME
* Undo change
* Undo change
* Undo change
* Fix FIXMEs
* Fix FIXME
* Correct value
* Bad merge
* Fix circular imports
* WIP
* Fixing clipping
* Fixes
* Bad merge
* Move optimizer step and clipping into the `PrecisionPlugin`
* Fix AMP
* Update CHANGELOG
* Fix tests
* Underscore
* Progress
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove pre_optimizer_step
* Missed one
* Progress
* Progress
* Fix test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update FIXMEs
* Fix test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix test
* DeepSpeed warning. mypy
* Rename
* Finish tests
* Update CHANGELOG
* Dumb fixes
* accelerator=auto
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update on comments
* Use ClassifModule
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-10-28 15:23:27 +00:00
48b6292cf0
Move optimizer step and clipping into the `PrecisionPlugin` ( #10143 )
2021-10-26 17:26:26 +02:00
93266e2c22
Avoid deprecated warnings from accelerator and checkpoint connector #10142
2021-10-26 14:10:30 +02:00
b376799430
Minor fixes related to clipping ( #10130 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-10-25 16:40:22 +00:00
d41902883a
Update `optimizer_step` methods in accelerator and plugins ( #10023 )
...
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-10-20 21:36:27 +01:00
ef5a12212a
Isolate optimizer step logic to the `PrecisionPlugin` ( #10029 )
2021-10-20 15:43:08 +00:00
e8beceb631
Add `TPUPrecisionPlugin` ( #10020 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-10-19 17:48:57 +00:00
e5dfdf34f9
Avoid deprecation warning after #9901 ( #9951 )
2021-10-16 17:36:25 +01:00
a002f872ea
[2/n] Directly call TrainingTypePlugin APIs instead of going through the Accelerator ( #9901 )
...
Co-authored-by: tchaton <thomas@grid.ai>
2021-10-14 17:38:22 +02:00
940b910d27
[2/4] Add DeviceStatsMonitor callback ( #9712 )
...
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2021-10-13 18:29:36 +00:00
4decbc0d95
Deprecate `dataloader_idx` from `on_train_batch_start/end` ( #9816 )
...
* deprecate hooks
* dep todo
* explicit
* Apply suggestions from code review
* Apply suggestions from code review
* code review
* base
2021-10-07 10:18:11 +00:00
0ddd6a8c19
Remove `_NATIVE_AMP_AVAILABLE` checks ( #9747 )
2021-09-29 15:34:26 +02:00
9ebfbbc349
Remove unused `post_optimizer_step` ( #9746 )
2021-09-29 13:09:22 +00:00
15cd6ad45b
Call TrainingTypePlugin collective functions directly instead of going through the Accelerator ( #9677 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-09-27 14:52:57 +02:00
ab069876cb
[1/4] Add get_device_stats to accelerator interface ( #9586 )
2021-09-26 21:09:16 -07:00
41e3be197f
Remove `call_configure_sharded_model` lifecycle property ( #9612 )
2021-09-24 03:57:53 +02:00
f5608e90d6
Document exceptions in accelerators ( #9558 )
...
* Document exceptions in ipu.py
* Document exceptions in tpu.py
* Document exceptions in gpu.py
2021-09-18 15:14:08 +09:00
b1ed1db089
Keep global step update in the loop ( #8856 )
2021-09-14 19:21:39 +05:30
b294c5760e
Fix type hint for filepath ( #9434 )
2021-09-10 21:38:54 +00:00
cc2ac02dd1
Move add_to_queue/get_from_queue to DDPSpawnPlugin ( #9118 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-09-10 20:58:02 +00:00
3070a9ea6e
Fix hiddens type annotation ( #9377 )
2021-09-09 08:45:52 +01:00
6e124e7207
CI: precommit - docformatter ( #8584 )
...
* CI: precommit - docformatter
* fix deprecated
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-09-06 12:49:09 +00:00
Rohit Gupta