* Add base hook for model parallel
* fix callback signature
* Simplify hook
* Add hook logic
* add tests
* add property setter
* add logic for being called once
* Update changelog
* Fix
* fix return type
* fix lambda callback test
* Fix tests
* Apply code suggestions
* add logic for setup_optimizers_predispatch
* add common dummy model
* Swap call order
* Remove test that isn't needed anymore
* Update tests
* Add a bit more doc
* Few code review fixes
* Update pytorch_lightning/accelerators/accelerator.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Change hook name
* Fix test
* Test setup hook, refactor names
* Swap call order of callbacks and model initialization
* Change name of context manager
Co-authored-by: SeanNaren <sean@grid.ai>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Move connection setup into the setup function. Call setup hook after we set up the accelerator
* Added CHANGELOG.md
* fix setup order in callback test
* fix input arguments in test
* Mock distributed function, remove protection to turn into training type hook
* Remove import
* Add missing mock, ensure custom plugin does not create children process
* Skip test on windows
* Update deepspeed to init connection in setup
* Do not initialize distributed module
* Move DeepSpeed tests to special tests since dist communication is being set up
* Special the test to see if this fixes CI
* Delete accelerator connector test to see if its causing build to fail
* Delete deepspeed test
* Revert "Delete accelerator connector test to see if its causing build to fail"
This reverts commit edde60b8
* Revert "Delete deepspeed test"
This reverts commit 9d317429
* Reverse hook
* Reverse setup hooks to debug again
* Add todo so i know where i left off
* For single device move in pre_dispatch after setup function
* Add additional model to device hook if any additional parameters have been set
* See if we can enable deepspeed tests
* Revert "See if we can enable deepspeed tests"
This reverts commit b5450def
* See if this hook approach works
* Introduce new granular hooks
* Remove import, fix tpu spawn by moving the function to setup
* Added missing special test
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* init information retrieval metrics
* changed retrieval metrics names, expanded arguments and fixed typo
* added 'Retrieval' prefix to metrics and fixed conflict with already-present 'average_precision' file
* improved code formatting
* pep8 code compatibility
* features/implemented new Mean Average Precision metrics for Information Retrieval + doc
* fixed pep8 compatibility
* removed threshold parameter and fixed typo on types in RetrievalMAP and improved doc
* improved doc, put first class-specific args in RetrievalMetric and transformed RetrievalMetric in abstract class
* implemented tests for functional and class metric. fixed typo when input tensors are empty or when all targets are False
* fixed typos in doc and changed torch.true_divide to torch.div
* fixed typos pep8 compatibility
* fixed types in long division in ir_average_precision and example in mean_average_precision
* RetrievalMetric states are not lists and _metric method accepts predictions and targets for easier extension
* updated CHANGELOG file
* added '# noqa: F401' flag to not used imports
* added double space before '# noqa: F401' flag
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* change get_mini_groups in get_group_indexes
* added checks on target inputs
* minor refactoring for code cleanness
* split tests over exception raising in separate function && refactored test code into multiple functions
* fixed pep8 compatibility
* implemented suggestions of @SkafteNicki
* fixed imports for isort and added types annontations to functions in test_map.py
* isort on test_map and fixed typing
* isort on retrieval and on __init__.py and utils.py in metrics package
* fixed typo in pytorch_lightning/metrics/__init__.py regarding code style
* fixed yapf compatibility
* fixed yapf compatibility
* fixed typo in doc
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* cleaning SWA (#6259)
* rename
* if
* test
* chlog
* Remove opt from manual_backward in docs (#6267)
* switch agents pool (#6270)
* Allow user to disable the automatic formatting of checkpoint file names.
* Added changelog entry.
* Made flake8 happy.
* Applied review suggestion: quotes for special characters in docstring
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Fixed example in docstring.
* Fixed syntax error in docstring.
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Ensure we set the default device before initializing deepspeed
* Add CHANGELOG.md
* Update pytorch_lightning/plugins/training_type/deepspeed.py
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
* fix
* add simple test
* fix imports
* add changelog
* tighter test with on_fit_start hook closer to the dispatch call
* move class inside test f unction
* add a comment
* fixed bug where tuner would not tune lr if also tuning batch_size
* added a '+1' to computing the smoothed loss. This maintains the behavior for the smoothed loss as before the bug fix
* pep8 fix
* add changelog
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Kaushik B