lightning/pytorch_lightning/metrics/classification
Tadej Svetina 7f71ee9265
Classification metrics overhaul: stat scores (3/n) (#4839)
* Add stuff

* Change metrics documentation layout

* Add stuff

* Add stat scores

* Change testing utils

* Replace len(*.shape) with *.ndim

* More descriptive error message for input formatting

* Replace movedim with permute

* PEP 8 compliance

* WIP

* Add reduce_scores function

* Temporarily add back legacy class_reduce

* Division with float

* PEP 8 compliance

* Remove precision recall

* Replace movedim with permute

* Add back tests

* Add empty newlines

* Add empty line

* Fix permute

* Fix some issues with old versions of PyTorch

* Style changes in error messages

* More error message style improvements

* Fix typo in docs

* Add more descriptive variable names in utils

* Change internal var names

* Break down error checking for inputs into separate functions

* Remove the (N, ..., C) option in MD-MC

* Simplify select_topk

* Remove detach for inputs

* Fix typos

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update docs/source/metrics.rst

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Minor error message changes

* Update pytorch_lightning/metrics/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Reuse case from validation in formatting

* Refactor code in _input_format_classification

* Small improvements

* PEP 8

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update docs/source/metrics.rst

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Alphabetical reordering of regression metrics

* Change default value of top_k and add error checking

* Extract basic validation into separate function

* Update to new top_k default

* Update desciption of parameters in input formatting

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Check that probabilities in preds sum to 1 (for MC)

* Fix coverage

* Split accuracy and hamming loss

* Remove old redundant accuracy

* Minor changes

* Fix imports

* Improve docstring descriptions

* Fix imports

* Fix edge case and simplify testing

* Fix docs

* PEP8

* Reorder imports

* Add top_k parameter

* Update changelog

* Update docstring

* Update docstring

* Reverse formatting changes for tests

* Change parameter order

* Remove formatting changes 2/2

* Remove formatting 3/3

* .

* Improve description of top_k parameter

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Update pytorch_lightning/metrics/functional/accuracy.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Explicit checking of parameter values

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Apply suggestions from code review

* Fix top_k checking

* PEP8

* Don't check dist_sync in test

* add back check_dist_sync_on_step

* Make sure half-precision inputs are transformed (#5013)

* Fix typo

* Rename hamming loss to hamming distance

* Fix tests for half precision

* Fix docs underline length

* Fix doc undeline length

* Replace mdmc_accuracy parameter with subset_accuracy

* Update changelog

* Fix unwanted accuracy change

* Enable top_k for ML prob inputs

* Test that default threshold is 0.5

* Fix typo

* Update top_k description in helpers

* updates

* Update styling and add back tests

* Remove excess spaces

* fix torch.where for old versions

* fix linting

* Update docstring

* Fix docstring

* Apply suggestions from code review (mostly docs)

* Default threshold to None, accept only (0,1)

* Change wrong threshold message

* Improve documentation and add tests

* Add back ddp tests

* Change stat reduce method and default

* Remove DDP tests and fix doctests

* Fix doctest

* Update changelog

* Refactoring

* Fix typo

* Refactor

* Increase coverage

* Fix linting

* Consistent use of backticks

* Fix too long line in docs

* Apply suggestions from code review

* Fix deprecation test

* Fix deprecation test

* Default threshold back to 0.5

* Minor documentation fixes

* Add types to tests

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2020-12-30 20:49:50 +01:00
..
__init__.py Classification metrics overhaul: stat scores (3/n) (#4839) 2020-12-30 20:49:50 +01:00
accuracy.py Classification metrics overhaul: stat scores (3/n) (#4839) 2020-12-30 20:49:50 +01:00
average_precision.py drop duplicate metrics (#5014) 2020-12-11 18:42:53 +01:00
confusion_matrix.py [Metrics] PrecisionRecallCurve, ROC and AveragePrecision class interface (#4549) 2020-12-04 22:42:23 +01:00
f_beta.py annotat unused vars (#5017) 2020-12-19 13:53:06 +01:00
hamming_distance.py Classification metrics overhaul: stat scores (3/n) (#4839) 2020-12-30 20:49:50 +01:00
helpers.py Classification metrics overhaul: stat scores (3/n) (#4839) 2020-12-30 20:49:50 +01:00
precision_recall.py [Metrics] PrecisionRecallCurve, ROC and AveragePrecision class interface (#4549) 2020-12-04 22:42:23 +01:00
precision_recall_curve.py refactor - check E501 (#5200) 2020-12-21 14:23:09 +05:30
roc.py drop duplicate metrics (#5014) 2020-12-11 18:42:53 +01:00
stat_scores.py Classification metrics overhaul: stat scores (3/n) (#4839) 2020-12-30 20:49:50 +01:00