Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,512 Commits 59 Branches 217 Tags 353 MiB
Commit Graph

310 Commits

Author SHA1 Message Date
Chunyang Wen 350c88e621
Let Accelerator inherit from ABC to make sure abstractmethod takes effect (#11521) 2022-01-23 20:47:43 +01:00
Carlos Mocholí 5914fb748f
Add typing to accelerators/gpu.py (#11333) 2022-01-12 19:44:51 +00:00
Carlos Mocholí 3692eba807
Drop Python 3.6 support (#11117) 2021-12-21 17:06:15 +00:00
four4fish cec2d7946b
3/n Move accelerator into Strategy (#11022) 
* remove training_step() from accelerator

* remove test, val, predict step

* move

* wip

* accelerator references

* cpu training

* rename occurrences in tests

* update tests

* pull from adrian's commit

* fix changelog merge pro

* fix accelerator_connector and other updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix doc build and some mypy

* fix lite

* fix gpu setup environment

* support customized ttp and accelerator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tpu error check

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix precision_plugin initialization to recognisze cusomized plugin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update bug_report_model.py

* Update accelerator_connector.py

* update changelog

* allow shorthand typing references to pl.Accelerator

* rename helper method and add docstring

* fix typing

* Update pytorch_lightning/trainer/connectors/accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_accelerator_connector.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update tests/accelerators/test_cpu.py

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pre commit complaint

* update typing to long ugly path

* spacing in flow diagram

* remove todo comments

* docformatter

* Update pytorch_lightning/plugins/training_type/training_type_plugin.py

* revert test changes

* improve custom plugin examples

* remove redundant call to ttp attribute

it is no longer a property

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-12-16 04:41:34 +00:00
Danielle Pintz 3fcfd0214c
Remove `_call_accelerator_hook` Trainer method (#10999) 2021-12-09 02:27:13 +01:00
four4fish 63bb4ec77d
4/n Move Accelerator into strategy - remove X_step() from accelerator (#10890) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-12-06 20:16:54 +00:00
four4fish 2fc64e9656
2/n Move Accelerator into strategy - remove dispatch functions from Accelerator (#10885) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-12-06 09:51:14 +00:00
four4fish 6fe3211573
Unroll dict input before call Accelerator X_steps (#10908) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-12-03 17:00:52 +00:00
four4fish e646ca1d59
Remove `setup_optimizers_in_pre_dispatch` logic (#10906) 2021-12-03 15:05:08 +01:00
four4fish 45dd8066e7
3/n Move Accelerator into strategy - remove model_sharded_context() (#10886) 
* 3/n Move Accelerator into strategy - remove model_sharded_context()

* update ttp function

* update changelog

* update changelog

Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
 2021-12-02 03:34:51 +00:00
four4fish 44cd412e91
Remove precision_plugin pre_dispatch() method (#10887) 
* Remove precision_plugin pre_dispatch() method

* update changelog
 2021-12-01 18:42:17 -08:00
four4fish 1d2878523a
2/n Move Precision Plugin into strategy - move optimizer related logics (#10596) 
Co-authored-by: Danielle Pintz <38207072+daniellepintz@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-11-30 08:31:23 +00:00
four4fish 8bf7f9cce7
1/n Move Accelerator into strategy - move batch_to_device to strategy (#10649) 
* 1/n Integrate Device Specific Accelerator Logic with strategy - move batch_to_device to strategy

* add changelog

* add model is not none check

* Apply suggestions from code review

Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update CHANGELOG.md

* Update test_datamodules.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test_hooks.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update dp.py

Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-11-29 12:11:21 -08:00
Rohit Gupta ff8ac6e2e1
Make `_get_nvidia_gpu_stats` public (#10406) 2021-11-19 17:52:24 +01:00
four4fish 700521c7d3
1/n Move precision plugin into strategy - update reference (#10570) 
* 1/n move precision plugin into strategy - update reference

* update precision plugin reference in tpu_spawn

* add missing reference in error message

* add back removed license line

* update references in tests

* update reference in trainer

* update return annotation for precision_plugin property on TTP

* simplify access to precision plugin reference in sharded plug

* add changelog

* remove precision property from ttp and add deprecation message

* fix make doc and update precision reference

* simplify a reference to precision

accidentally overridden Adrian's change, now add it back

* Update CHANGELOG.md

add Adrian's change back

* Update accelerator precision

Add Adrian's change back

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add none check for precision plugin

just to be safe

* Update ipu.py

* update precision_plugin param deprecation message

* Update accelerator.py

* Remove deprecated warning 

Tests will fail after 9940

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-11-19 00:39:01 +00:00
ananthsub aad86423f7
Remove more deprecated methods from base `Accelerator` class (#10448) 2021-11-10 12:58:24 +05:30
puhuk f9b9cdb0d1
Remove deprecated accelerator pass through functions in Accelerator (#10403) 
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-11-08 17:36:37 +00:00
Adrian Wälchli a270a79ed9
Rename "master" methods to "main" in ClusterEnvironment plugins (#10103) 
* rename occurrences of master port, master address, maser node, master process

* rename properties

* add property decorators

* occurrences in docs

* update changelog

* update changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add lost method

* create deprecation

* add changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix typo (but it was already there!!!)

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* add todo

* update more occurences

* add types

* add missing import

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
 2021-11-08 12:32:58 +00:00
Carlos Mocholí 9237106451
Clip before step (#10248) 2021-10-30 11:27:49 +01:00
Kaushik B cedaebfcbb
Add `auto_device_count` method to `Accelerators` (#10222) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-10-29 22:31:32 +02:00
Carlos Mocholí 81d15c5986
Implement double optimizer closure for hook structure consistency (#10167) 2021-10-29 13:03:04 +00:00
Carlos Mocholí 03f01fb5ec
Fix gradient norm tracking and gradient clipping (#9287) 
* WIP

* Progress

* Undo test change

* Fix plugin closure execution order

* Update CHANGELOG

* Fix manual optimization on AMP and skipping backward

* Fix for deepspeed

* Typo

* Hook test for manual closure

* Add skipping test with AMP

* You are hideous, apex

* Add deepspeed test

* Update CHANGELOG

* Fix for broken master

* Add RunIf

* FIXMEs

* Rename

* Fix grad norm

* add a simple test

* update test

* update  test

* update test

* fix merge conflicts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Sea of changes

* Undo change

* Introduce TPUPrecisionPlugin

* Undo changes

* Undo changes

* Resolve FIXME

* Undo change

* Undo change

* Undo change

* Fix FIXMEs

* Fix FIXME

* Correct value

* Bad merge

* Fix circular imports

* WIP

* Fixing clipping

* Fixes

* Bad merge

* Move optimizer step and clipping into the `PrecisionPlugin`

* Fix AMP

* Update CHANGELOG

* Fix tests

* Underscore

* Progress

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove pre_optimizer_step

* Missed one

* Progress

* Progress

* Fix test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update FIXMEs

* Fix test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix test

* DeepSpeed warning. mypy

* Rename

* Finish tests

* Update CHANGELOG

* Dumb fixes

* accelerator=auto

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update on comments

* Use ClassifModule

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
 2021-10-28 15:23:27 +00:00
Carlos Mocholí 48b6292cf0
Move optimizer step and clipping into the `PrecisionPlugin` (#10143) 2021-10-26 17:26:26 +02:00
Rohit Gupta 93266e2c22
Avoid deprecated warnings from accelerator and checkpoint connector #10142 2021-10-26 14:10:30 +02:00
Carlos Mocholí b376799430
Minor fixes related to clipping (#10130) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
 2021-10-25 16:40:22 +00:00
Adrian Wälchli d41902883a
Update `optimizer_step` methods in accelerator and plugins (#10023) 
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-10-20 21:36:27 +01:00
Carlos Mocholí ef5a12212a
Isolate optimizer step logic to the `PrecisionPlugin` (#10029) 2021-10-20 15:43:08 +00:00
Carlos Mocholí e8beceb631
Add `TPUPrecisionPlugin` (#10020) 
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-10-19 17:48:57 +00:00
Carlos Mocholí e5dfdf34f9
Avoid deprecation warning after #9901 (#9951) 2021-10-16 17:36:25 +01:00
four4fish a002f872ea
[2/n] Directly call TrainingTypePlugin APIs instead of going through the Accelerator (#9901) 
Co-authored-by: tchaton <thomas@grid.ai>
 2021-10-14 17:38:22 +02:00
Danielle Pintz 940b910d27
[2/4] Add DeviceStatsMonitor callback (#9712) 
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-10-13 18:29:36 +00:00
Rohit Gupta 4decbc0d95
Deprecate `dataloader_idx` from `on_train_batch_start/end` (#9816) 
* deprecate hooks

* dep todo

* explicit

* Apply suggestions from code review

* Apply suggestions from code review

* code review

* base
 2021-10-07 10:18:11 +00:00
Carlos Mocholí 0ddd6a8c19
Remove `_NATIVE_AMP_AVAILABLE` checks (#9747) 2021-09-29 15:34:26 +02:00
Carlos Mocholí 9ebfbbc349
Remove unused `post_optimizer_step` (#9746) 2021-09-29 13:09:22 +00:00
four4fish 15cd6ad45b
Call TrainingTypePlugin collective functions directly instead of going through the Accelerator (#9677) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
 2021-09-27 14:52:57 +02:00
Danielle Pintz ab069876cb
[1/4] Add get_device_stats to accelerator interface (#9586) 2021-09-26 21:09:16 -07:00
ananthsub 41e3be197f
Remove `call_configure_sharded_model` lifecycle property (#9612) 2021-09-24 03:57:53 +02:00
Aki Nitta f5608e90d6
Document exceptions in accelerators (#9558) 
* Document exceptions in ipu.py

* Document exceptions in tpu.py

* Document exceptions in gpu.py
 2021-09-18 15:14:08 +09:00
Carlos Mocholí b1ed1db089
Keep global step update in the loop (#8856) 2021-09-14 19:21:39 +05:30
Kaushik B b294c5760e
Fix type hint for filepath (#9434) 2021-09-10 21:38:54 +00:00
Danielle Pintz cc2ac02dd1
Move add_to_queue/get_from_queue to DDPSpawnPlugin (#9118) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-09-10 20:58:02 +00:00
Carlos Mocholí 3070a9ea6e
Fix hiddens type annotation (#9377) 2021-09-09 08:45:52 +01:00
Jirka Borovec 6e124e7207
CI: precommit - docformatter (#8584) 
* CI: precommit - docformatter
* fix deprecated

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-09-06 12:49:09 +00:00
four4fish f01a9a6cd2
Remove `BasePlugin` (#9066) 
* Remove BasePlugin

Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-08-25 19:10:28 +00:00
Sean Naren bac8b1be81
Add support for CPU AMP autocast (#9084) 2021-08-25 12:18:00 +00:00
four4fish c912ebf889
Remove TrainingTypePlugin.on_save and Accelerator.on_save (#9023) 
* Remove TrainingTypePlugin.on_save and Accelerator.on_save
 2021-08-23 10:11:00 -07:00
ananthsub 8a931732ae
Remove unused `on_train_epoch_end` hook in accelerator (#9035) 2021-08-23 00:20:10 +05:30
four4fish 13e64e6a80
Remove deprecated functions from accelerator.py (#9019) 2021-08-22 00:25:42 +02:00
Carlos Mocholí d0efb55b0f
Delete `TrainingEpochLoop._dataloader_idx` which always equals 0 (#8911) 2021-08-16 13:34:42 +02:00
Carlos Mocholí 93ab24d1ee
Replace DataLoader sampler once for IPUs (#8858) 2021-08-16 11:28:05 +02:00