Commit Graph

37 Commits

Author SHA1 Message Date
Jv Kyle Eclarin b1f8b111b5
Update `tests/accelerators/*.py` to use `devices` instead of `gpus` or `ipus` (#11817) 2022-03-24 14:09:39 +00:00
DuYicong515 491fa02aa3
Remove `AccleratorConnector.num_ipus` and deprecate `Trainer.ipus` (#12386) 2022-03-23 07:00:14 +00:00
DuYicong515 ed2bcc5ab3
Deprecate `Trainer.devices` in favor of `Trainer.num_devices` and `Trainer.device_ids` (#12151) 2022-03-18 12:38:57 -07:00
Louis Taylor 73bda54e63
CI: update poplar sdk version (#12226) 2022-03-04 23:49:30 +00:00
jjenniferdai 89d37569d8
add `accelerator.is_available()` check (#12104)
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
2022-03-02 10:07:49 +00:00
Carlos Mocholí 8fd17f2edf
[IPU] Support manually instantiating the `poptorch.DataLoader` (#12116) 2022-02-28 09:36:26 +00:00
Carlos Mocholí 61dd5e4d5e
[IPU] Do not use `DistributedSampler` (#12114) 2022-02-25 14:29:51 +00:00
Kaushik B 49420548ce
Update warnings for available accelerators not being used (#11909)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-02-24 09:12:02 +01:00
Carlos Mocholí 4c4b9d540f
Remove `Trainer._device_type` (#11992) 2022-02-22 13:02:13 +00:00
four4fish 6e14209185
Rewrite accelerator_connector (#11448) 2022-02-17 23:38:39 +00:00
ananthsub 1b107c5892
Add `Accelerator.is_available()` interface requirement (#11797) 2022-02-09 15:11:27 -08:00
Adrian Wälchli c210e338ef
Update strategy import statements (#11231) 2021-12-23 08:26:28 +01:00
four4fish cf5ef32f7b
Deprecate Trainer.training_type_plugin in favor of trainer.strategy (#11141)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-12-22 02:11:43 +00:00
Kaushik B 2e947a88e0
Rename IPUPlugin to IPUStrategy (#11193) 2021-12-21 15:55:41 +00:00
Adrian Wälchli f5c2881b68
3/n Simplify spawn plugins: Merge `pre_dispatch` and `setup` logic (#11137) 2021-12-20 17:41:22 +01:00
four4fish cec2d7946b
3/n Move accelerator into Strategy (#11022)
* remove training_step() from accelerator

* remove test, val, predict step

* move

* wip

* accelerator references

* cpu training

* rename occurrences in tests

* update tests

* pull from adrian's commit

* fix changelog merge pro

* fix accelerator_connector and other updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix doc build and some mypy

* fix lite

* fix gpu setup environment

* support customized ttp and accelerator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tpu error check

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix precision_plugin initialization to recognisze cusomized plugin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update bug_report_model.py

* Update accelerator_connector.py

* update changelog

* allow shorthand typing references to pl.Accelerator

* rename helper method and add docstring

* fix typing

* Update pytorch_lightning/trainer/connectors/accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_cpu.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pre commit complaint

* update typing to long ugly path

* spacing in flow diagram

* remove todo comments

* docformatter

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

* revert test changes

* improve custom plugin examples

* remove redundant call to ttp attribute

it is no longer a property

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-12-16 04:41:34 +00:00
Rajath Bharadwaj 7914e5c157
added UserWarnings if max_epochs not set in the Trainer class (#10700) 2021-12-06 09:44:25 +00:00
four4fish e646ca1d59
Remove `setup_optimizers_in_pre_dispatch` logic (#10906) 2021-12-03 15:05:08 +01:00
Kaushik B e0b4bb2ea3
Deprecate `DeviceType` in favor of `_AcceleratorType` (#10503)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-11-25 16:41:03 +01:00
four4fish 700521c7d3
1/n Move precision plugin into strategy - update reference (#10570)
* 1/n move precision plugin into strategy - update reference

* update precision plugin reference in tpu_spawn

* add missing reference in error message

* add back removed license line

* update references in tests

* update reference in trainer

* update return annotation for precision_plugin property on TTP

* simplify access to precision plugin reference in sharded plug

* add changelog

* remove precision property from ttp and add deprecation message

* fix make doc and update precision reference

* simplify a reference to precision

accidentally overridden Adrian's change, now add it back

* Update CHANGELOG.md

add Adrian's change back

* Update accelerator precision

Add Adrian's change back

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add none check for precision plugin

just to be safe

* Update ipu.py

* update precision_plugin param deprecation message

* Update accelerator.py

* Remove deprecated warning 

Tests will fail after 9940

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-19 00:39:01 +00:00
Carlos Mocholí ba23d91320
Update recommendation on `dataloader_idx` (#10318) 2021-11-04 01:39:55 +01:00
Kaushik B e0f7dbdd1c
Add support for `devices='auto'` (#10264) 2021-10-30 15:05:51 +00:00
Rohit Gupta 85eb17cde5
initialize poptorch_models based on trainer_fn (#10149) 2021-10-28 11:59:52 +00:00
Kaushik B 56bc55db71
Update strategy flag in docs (#10000)
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2021-10-20 21:02:53 +05:30
Kaushik B 05b15e63f0
Add `strategy` argument to Trainer (#8597)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-10-13 12:34:06 +00:00
Rohit Gupta 8a8ecb8d01
Update the logic to check for accumulation steps with deepspeed (#9826)
* support_dict

* chlog

* fix test

* epochs
2021-10-06 17:50:10 +01:00
Rohit Gupta 8fcdcb598b
Fix `accumulate_grad_batches` on init (#9652)
* fix accumuate_grad_batches on init

* chlog

* update error

* move to callback connector

* add test with callback

* fix tests

* Update pytorch_lightning/trainer/connectors/callback_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* update ipu logic

* rev

* rev

* rev

* pls work

* code review

Co-authored-by: Rohit Gupta <goku@rmac.local>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-09-24 18:51:54 +00:00
Carlos Mocholí b1ed1db089
Keep global step update in the loop (#8856) 2021-09-14 19:21:39 +05:30
Jirka Borovec 6e124e7207
CI: precommit - docformatter (#8584)
* CI: precommit - docformatter
* fix deprecated

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-09-06 12:49:09 +00:00
Carlos Mocholí 93ab24d1ee
Replace DataLoader sampler once for IPUs (#8858) 2021-08-16 11:28:05 +02:00
Carlos Mocholí a64cc37394
Replace `yapf` with `black` (#7783)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-26 13:37:35 +02:00
Kaushik B 556879e5cf
Add support for devices flag to Trainer (#8440)
* Support devices flag to Trainer
2021-07-20 04:33:12 +00:00
Kaushik B 825c5dbe8c
Add support for (accelerator='cpu'|'gpu'|'tpu'|'ipu'|'auto') (#7808)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: SeanNaren <sean@grid.ai>
2021-07-09 15:28:54 +00:00
Sean Naren 6d558961e3
[IPU] Allow poptorch.Options to override Trainer (#8233)
* Add test for poptorch Options

* Hacks to get manual plugin support

* Revert changes

* Fix tests + ensure logic follow suit

* Update pytorch_lightning/plugins/training_type/ipu.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cleaner

* Cleaner

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-05 13:42:00 +00:00
Sean Naren 07b1ce227c
[IPU] Fix Custom Poptorch options to IPUPlugin (#8241)
* Fixes to ensure ipu options are respected

* Better setter

* Add test for poptorch Options

* Fix test

* fix ipu test

* Update pytorch_lightning/plugins/training_type/ipu.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-07-02 11:23:57 +00:00
Sean Naren 024cf23c67
Remove convert_to_half, suggest using `model.half` (#7974) 2021-06-14 18:48:02 +01:00
Sean Naren 96433d03ea
IPU Integration 5/5 (#7867)
* Initial changes

* Add broken example for now

* Fix reference

* Fix format

* Code runs

* Fixes

* Clear up files

* Add tests, helpers, fixes

* Small cleanups

* Refactors based on review

* Swap to special tests

* Add special tests

* Add source

* Cleanups

* Add logic to attach/detach model from devices

* Fixes for tests

* Fixes for tests

* Move earlier

* Cleanups

* Add check for nvcc

* Add tests, cleanups

* Fix errors

* fix

* Try condition

* Add missing annotation

* Clearer

* Clearer message

* Fix variable

* Cleanups

* Add comment

* CHANGELOG.md

* Add simple selection test

* Remove special=True to see what happens

* Fix test

* Update tests/accelerators/test_ipu.py

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>

* Convert ipu_cores -> ipus

* Add typing, fail earlier

* simplify precision

* Add test, add helper

* fix accum

* Update pytorch_lightning/plugins/training_type/ipu.py

Co-authored-by: thomas chaton <thomas@grid.ai>

* Use stages

* Make sure warning message returned

* thorw error

* Add more tests, use fs

* add comment

* Clean

* Address feedback, add IPU tests

* Fixes

* Fix signature

* Add types

* Remove autoround

* Add docstring

* ipu_cores -> ipus

* Add test, remove unnecessary precision set

* Add optimizer test

* Add precision back with test

* Address code review

* Change to probs

* Move some of the asserts earlier

Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
2021-06-11 15:07:04 +00:00