Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,627 Commits 59 Branches 217 Tags 356 MiB
Commit Graph

78 Commits

Author SHA1 Message Date
Akash Kwatra f5304897ce
Deprecate `BaseProfiler.profile_iterable` (#12102) 2022-02-25 16:26:20 +01:00
Rohit Gupta d9938da8a4
add properties to check for trainer state in pytorch profier (#12063) 2022-02-24 17:03:16 +05:30
Kaushik B 11bd176d2f
Add XLA Profiler section to docs (#11436) 
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
 2022-02-22 05:21:41 +00:00
Carlos Mocholí 08384c45ea
Support manual optimization step profiling without a trainer reference (#11883) 2022-02-21 13:43:12 +00:00
guyang3532 79c4e5de60
Refine the pytorch profiler (#11268) 2022-02-11 14:50:18 +01:00
ananthsub a64438c897
Centralize rank_zero_only utilities into their own module (#11747) 
* Centralize rank_zero_only utilities into their own module

Fixes #11746

* PossibleUserWarning

* Update test_warnings.py

* update imports

* more imports

* Update CHANGELOG.md

* Update mlflow.py

* Update cli.py

* Update api_references.rst

* Update meta.py

* add deprecation tests

* debug standalone

* fix standalone tests

* Update CHANGELOG.md
 2022-02-07 08:09:55 +00:00
Danielle Pintz c71a1d7ea2
Remove `self._log_dir` from `BaseProfiler` (#11740) 2022-02-05 04:45:48 +01:00
Rohit Gupta 3eee8f18cf
Sort simple profiler summary based on mean duration (#11671) 2022-02-02 20:44:42 +00:00
Rohit Gupta f7f835fa0e
improve simple profiler output (#11414) 2022-01-18 19:58:34 +00:00
Rohit Gupta 221091afc4
move profiler docs (#11431) 2022-01-12 05:56:16 +00:00
Danielle Pintz 7fa1aebcc9
Remove `profile("training_step_and_backward")` (#11222) 2022-01-04 11:50:11 +01:00
Carlos Mocholí 3692eba807
Drop Python 3.6 support (#11117) 2021-12-21 17:06:15 +00:00
Danielle Pintz b1baf460d9
Include hook's object name when profiling (#11026) 2021-12-20 15:18:24 +01:00
guyang3532 cc4a978bf6
Safely disable profiler (#11167) 2021-12-20 11:51:46 +00:00
Kaushik B 6599ced17d
Don't import torch_xla.debug for torch-xla<1.8 (#10836) 2021-12-06 06:31:38 +00:00
Rohit Gupta 5b9995da04
Fix schedule reset logic in pytorch profiler (#10837) 2021-12-02 14:22:49 +05:30
Carlos Mocholí d515bcac96
Remove deprecated profiler import (#10443) 2021-11-09 23:13:02 +01:00
twsl 971281d27d
Make sure file and folder exists in Profiler (#10073) 
Co-authored-by: tchaton <thomas@grid.ai>
 2021-10-26 11:13:31 +00:00
Adrian Wälchli e951cd4dd3
fix noqa appearing in docs (#10116) 2021-10-25 10:27:00 +00:00
Carlos Mocholí 788f6864d9
Fix `LightningOptimizer` step and toggling logic (#9958) 2021-10-18 00:23:51 +00:00
Jirka Borovec 6e124e7207
CI: precommit - docformatter (#8584) 
* CI: precommit - docformatter
* fix deprecated

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-09-06 12:49:09 +00:00
thomas chaton 9149b64908
[bugfix] Resolve PyTorch Profiling for Manual Optimization (#9316) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-09-06 10:45:34 +00:00
B. Kerim Tshimanga f6614b370c
scheduled removal of BaseProfiler.output_filename in favor of dirpath… (#9214) 2021-08-31 09:30:43 +00:00
B. Kerim Tshimanga f79993a705
removing legacy profiler arg (#9178) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-08-30 09:37:09 +00:00
Adrian Wälchli 522df2b89b
3/n integrate new LightningDataFetcher into loop (#8953) 
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
 2021-08-17 21:42:22 +00:00
Jirka Borovec f67892ea96
CI: yesqa (#8564) 
* add yesqa
* fix flake8

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-08-02 16:05:56 +00:00
Carlos Mocholí 93784da2c3
Fix pre-commit blacken-docs failures (#8624) 2021-07-30 12:10:15 +00:00
Santiago Castro b256d6acd3
Avoid unnecessary list creation (#8595) 2021-07-28 13:36:45 +05:30
Carlos Mocholí a64cc37394
Replace `yapf` with `black` (#7783) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-07-26 13:37:35 +02:00
Carlos Mocholí f7027a8701
Remove `torch >= 1.6` checks (#8523) 2021-07-23 04:03:20 +00:00
deepsource-autofix[bot] b2ba2e6333
Use literal syntax instead of function calls to create data structure (#8406) 2021-07-14 10:32:13 +00:00
gaoteng-git f796cb435f
Fix `PyTorchProfiler` prefix typo (#8308) 2021-07-06 17:12:07 +02:00
gaoteng-git a7e21bd5ad
move profiler.step from training_step_and_backward to optimizer_step_… (#8224) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2021-07-06 13:22:10 +05:30
Kaushik B 2f3c65e57b
XLA Profiler integration (#8014) 2021-06-29 00:58:05 +05:30
deepsource-autofix[bot] 03154eb30a
Refactor unnecessary `else` / `elif` when `if` block has a `return` statement (#8156) 
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
 2021-06-28 15:27:41 +05:30
Carlos Mocholí dd340a6598
Actually show deprecation warnings and their line level [2/2] (#8002) 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2021-06-21 18:51:53 +02:00
thomas chaton 651a0fbdeb
[bugfix] Properly name PyTorchProfiler traces (#8009) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-18 18:16:36 +00:00
Kaushik B f447839d16
Add `warning_cache.deprecation` and set warning stacklevel [1/2] (#8005) 
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-06-18 11:50:24 +00:00
thomas chaton 51d370f4c2
[doc] Move each profiler to its own file + Add missing PyTorchProfiler to the doc (#7822) 2021-06-04 21:08:29 +05:30
Aniket Maurya 0bad2186c1
Added Vulture dead code checker (#5654) 
* integrated vulture CI

* added vulture in workflows

* added vulture in workflows

* vulture logs verbose set false

* Apply suggestions from code review

* ignore name list and args to underscore naming

* add ignore names

* deadcode whitelist

* deadcode whitelist

* Apply suggestions from code review

Co-authored-by: Rahul Jha <rahul722j@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update whitelist

* Sort

* Updates

* Updates

* Apply suggestions from code review

* Updates

Co-authored-by: Aniket Maurya <aniket.maurya@gdn-commerce.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rahul Jha <rahul722j@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
 2021-06-02 16:19:10 +01:00
Carlos Mocholí b730a5a281
Do not describe when there's no summary (#6681) 2021-03-26 14:58:05 +00:00
thomas chaton 0ea8f39841
Resolve schedule step bug for PyTorch Profiler (#6674) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
 2021-03-25 17:03:06 +01:00
thomas chaton fd5cb7fcc3
Add PyTorch 1.8 Profiler 5/5 (#6618) 
* Refactor profilers

* Update PassThrough

* WIP - This is broken and will change

* Update pytorch_lightning/profiler/pytorch.py

Co-authored-by: thomas chaton <thomas@grid.ai>

* resolve tests

* resolve tests

* find output

* try something

* update

* add support for test and predict

* update

* update

* use getattr

* test

* test

* update

* tests

* update

* update

* update

* update

* update

* remove file

* update

* update

* update

* update

* update

* test

* update#

* update

* update tests

* update

* add suport for 1.8

* rename records

* add support for 1.8

* update

* resolve flake8

* resolve test

* Refactor basic profilers

* Fixes

* Unused import

* Introduce setup

* Profile on all ranks. Print to stdout on 0

* Introduce dirpath + filename

* CHANGELOG

* Add tests. Address comments

* add `on_run_stage_setup`

* add on_run_stage_setup function

* update

* add test for RegisterRecordFunction

* update lightnng flow direction

* move variable to private

* remove trace

* Undo code that should be in 3/4

* Multi-stage multi-rank

* 2/5 changes

* Pass stage in __del__

* Remove TODOs

* Describe on_evaluation_end. Add tests

* Typo

* Address comments

* deepcopy tests

* Advanced teardown

* Fix teardown test

* Fix tests

* Minor change

* Update CHANGELOG.md

* Fix test

* Quick fixes

* Fix 6522

* resolve ddp tests

* resolve tests

* resolve some tests

* update tests

* resolve tests

* update

* resolve tests

* resolve some tests

* Missed fixes from 3/5

* Fixes

* resolve some tests

* resolve test for 1.7.1

* Broken refactor

* Missed stage

* Minor changes

* resolve tests

* Update CHANGELOG

* resolve bug

* remove print

* Typo

* Cleanup

* resolve ddp test

* remove barrier

* update profiler

* update

* Smaller model

* update

* resolve tests

* update

* Minor changes. CHANGELOG

* Minimize diff

* update to 1.8.1

* RunIf. Extra code. Check segfault

* resolve tests

* Typo. Bad merge

* Fixing a bad merge

* replace for kineto

* Update pytorch_lightning/profiler/pytorch.py

Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>

* Update pytorch_lightning/profiler/pytorch.py

Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>

* Minor changes

* Bad merge

* Use lists for flexibility

* Use sets

* predict_step

* Ananth's suggestion

* update

* Docs

* Update pl_examples/basic_examples/profiler_example.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* update example

* update example

Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-03-23 20:43:21 +00:00
Carlos Mocholí 51b10f78f4
Refactor PyTorch profiler 4/5 (#6349) 
Co-authored-by: thomas chaton <thomas@grid.ai>
 2021-03-23 18:13:29 +01:00
Carlos Mocholí 36d180e532
Refactor base profilers 3/5 (#6621) 
Co-authored-by: tchaton <thomas@grid.ai>
 2021-03-23 10:07:35 +00:00
thomas chaton 2064ece582
[refactor] Add setup to profilers + _run_stage_setup to trainer 2/5 (#6633) 
* add setup

* update

* updates on comment

* Minor changes

* Extra import

* Docs

Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
 2021-03-22 14:32:31 -04:00
camruta e2e1de0fb7
Add teardown method to BaseProfiler. (#6370) 
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
 2021-03-22 11:49:06 +00:00
thomas chaton 2ec67a48b3
[bug] Fix Pytorch profiler with emit_nvtx (#6260) 
* resolve bug

* update changelog

* Update tests/trainer/test_trainer.py

* Update pytorch_lightning/profiler/profilers.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* resolve comments

* resolve flake8

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-03-05 21:12:03 +01:00
Adrian Wälchli bc577ca792
fix duplicate console logging bug v2 (#6275) 
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2021-03-02 15:17:55 +05:30
Jirka Borovec 0f9134e043
Refactor: skipif for Windows 2/n (#6268) 
* win

* isort

* flake8
 2021-03-02 09:36:01 +00:00