Tadej Svetina
|
7f71ee9265
|
Classification metrics overhaul: stat scores (3/n) (#4839)
* Add stuff
* Change metrics documentation layout
* Add stuff
* Add stat scores
* Change testing utils
* Replace len(*.shape) with *.ndim
* More descriptive error message for input formatting
* Replace movedim with permute
* PEP 8 compliance
* WIP
* Add reduce_scores function
* Temporarily add back legacy class_reduce
* Division with float
* PEP 8 compliance
* Remove precision recall
* Replace movedim with permute
* Add back tests
* Add empty newlines
* Add empty line
* Fix permute
* Fix some issues with old versions of PyTorch
* Style changes in error messages
* More error message style improvements
* Fix typo in docs
* Add more descriptive variable names in utils
* Change internal var names
* Break down error checking for inputs into separate functions
* Remove the (N, ..., C) option in MD-MC
* Simplify select_topk
* Remove detach for inputs
* Fix typos
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Minor error message changes
* Update pytorch_lightning/metrics/utils.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Reuse case from validation in formatting
* Refactor code in _input_format_classification
* Small improvements
* PEP 8
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Alphabetical reordering of regression metrics
* Change default value of top_k and add error checking
* Extract basic validation into separate function
* Update to new top_k default
* Update desciption of parameters in input formatting
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Check that probabilities in preds sum to 1 (for MC)
* Fix coverage
* Split accuracy and hamming loss
* Remove old redundant accuracy
* Minor changes
* Fix imports
* Improve docstring descriptions
* Fix imports
* Fix edge case and simplify testing
* Fix docs
* PEP8
* Reorder imports
* Add top_k parameter
* Update changelog
* Update docstring
* Update docstring
* Reverse formatting changes for tests
* Change parameter order
* Remove formatting changes 2/2
* Remove formatting 3/3
* .
* Improve description of top_k parameter
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Remove unneeded assert
* Update pytorch_lightning/metrics/functional/accuracy.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Remove unneeded assert
* Explicit checking of parameter values
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Apply suggestions from code review
* Fix top_k checking
* PEP8
* Don't check dist_sync in test
* add back check_dist_sync_on_step
* Make sure half-precision inputs are transformed (#5013)
* Fix typo
* Rename hamming loss to hamming distance
* Fix tests for half precision
* Fix docs underline length
* Fix doc undeline length
* Replace mdmc_accuracy parameter with subset_accuracy
* Update changelog
* Fix unwanted accuracy change
* Enable top_k for ML prob inputs
* Test that default threshold is 0.5
* Fix typo
* Update top_k description in helpers
* updates
* Update styling and add back tests
* Remove excess spaces
* fix torch.where for old versions
* fix linting
* Update docstring
* Fix docstring
* Apply suggestions from code review (mostly docs)
* Default threshold to None, accept only (0,1)
* Change wrong threshold message
* Improve documentation and add tests
* Add back ddp tests
* Change stat reduce method and default
* Remove DDP tests and fix doctests
* Fix doctest
* Update changelog
* Refactoring
* Fix typo
* Refactor
* Increase coverage
* Fix linting
* Consistent use of backticks
* Fix too long line in docs
* Apply suggestions from code review
* Fix deprecation test
* Fix deprecation test
* Default threshold back to 0.5
* Minor documentation fixes
* Add types to tests
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
|
2020-12-30 20:49:50 +01:00 |
|
Tadej Svetina
|
ccffc344ca
|
Classification metrics overhaul: accuracy metrics (2/n) (#4838)
* Add stuff
* Change metrics documentation layout
* Add stuff
* Change testing utils
* Replace len(*.shape) with *.ndim
* More descriptive error message for input formatting
* Replace movedim with permute
* PEP 8 compliance
* Division with float
* Style changes in error messages
* More error message style improvements
* Fix typo in docs
* Add more descriptive variable names in utils
* Change internal var names
* Break down error checking for inputs into separate functions
* Remove the (N, ..., C) option in MD-MC
* Simplify select_topk
* Remove detach for inputs
* Fix typos
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Minor error message changes
* Update pytorch_lightning/metrics/utils.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Reuse case from validation in formatting
* Refactor code in _input_format_classification
* Small improvements
* PEP 8
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/classification/utils.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Alphabetical reordering of regression metrics
* Change default value of top_k and add error checking
* Extract basic validation into separate function
* Update to new top_k default
* Update desciption of parameters in input formatting
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Check that probabilities in preds sum to 1 (for MC)
* Fix coverage
* Split accuracy and hamming loss
* Remove old redundant accuracy
* Minor changes
* Fix imports
* Improve docstring descriptions
* Fix edge case and simplify testing
* Fix docs
* PEP8
* Reorder imports
* Update changelog
* Update docstring
* Update docstring
* Reverse formatting changes for tests
* Change parameter order
* Remove formatting changes 2/2
* Remove formatting 3/3
* .
* Improve description of top_k parameter
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Remove unneeded assert
* Update pytorch_lightning/metrics/functional/accuracy.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Remove unneeded assert
* Explicit checking of parameter values
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Apply suggestions from code review
* Fix top_k checking
* PEP8
* Don't check dist_sync in test
* add back check_dist_sync_on_step
* Make sure half-precision inputs are transformed (#5013)
* Fix typo
* Rename hamming loss to hamming distance
* Fix tests for half precision
* Fix docs underline length
* Fix doc undeline length
* Replace mdmc_accuracy parameter with subset_accuracy
* Update changelog
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Suggestions from code review
* Fix number in docs
* Update pytorch_lightning/metrics/classification/accuracy.py
* Replace topk by argsort in select_topk
* Fix changelog
* Add test for wrong params
* Add Google Colab badges (#5111)
* Add colab badges to notebook
Add colab badges to notebook to notebooks 4 & 5
* Add colab badges
Co-authored-by: chaton <thomas@grid.ai>
* Fix hanging metrics tests (#5134)
* Use torch.topk again as ddp hanging tests fixed in #5134
* Fix unwanted notebooks change
* Fix too long line in hamming_distance
* Apply suggestions from code review
* Apply suggestions from code review
* protect
* Update CHANGELOG.md
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Shachar Mirkin <shacharmirkin@gmail.com>
|
2020-12-21 16:42:51 +01:00 |
|
Jirka Borovec
|
0f36525e8f
|
fix/enable - check F401 (#5201)
* refactor - check F401
* missed
* fix
|
2020-12-21 10:15:04 +01:00 |
|
Jirka Borovec
|
1e501f0f40
|
add back compatibility for deprecated metrics 2/n (#5068)
* add back compatibility for deprecated metrics
* fix
* imports
* imports
|
2020-12-11 22:56:19 +01:00 |
|
Nicki Skafte
|
bf7c28cd54
|
[Metrics] PrecisionRecallCurve, ROC and AveragePrecision class interface (#4549)
* initial changes
* remove old
* init files
* add average precision
* add precision_recall_curve
* add roc
* cleaning
* docs
* pep8
* docs
* pep8
* changelog
* examples prune duplicate roc
* format
* imports
* fix
* format
* flake8
* duplicate
* fix
* flake8
* docs
* docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
|
2020-12-04 22:42:23 +01:00 |
|
Nicki Skafte
|
6831ba9aa0
|
[Metrics] Unification of FBeta (#4656)
* implementation
* init files
* more stable reduction
* add tests
* docs
* remove old implementation
* pep8
* changelog
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
|
2020-11-23 09:44:35 +01:00 |
|
Nicki Skafte
|
e0b856c105
|
[Metrics] Confusion matrix class interface (#4348)
* docs + precision + recall + f_beta + refactor
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* rebase
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* fixes
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* added missing file
* docs
* docs
* extra import
* add confusion matrix
* add to docs
* add test
* pep8 + isort
* update tests
* move util function
* unify functional and class
* add to init
* remove old implementation
* update tests
* pep8
* add duplicate
* fix doctest
* Update pytorch_lightning/metrics/classification/confusion_matrix.py
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* changelog
* bullet point args
* bullet docs
* bullet docs
Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Roger Shieh <55400948+s-rog@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
|
2020-10-30 11:44:25 +01:00 |
|
William Falcon
|
09c2020a93
|
notices (#4118)
|
2020-10-13 07:18:07 -04:00 |
|
Ananya Harsh Jha
|
ae8772490d
|
classification metrics (#4043)
* docs + precision + recall + f_beta + refactor
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* rebase
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* fixes
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* added missing file
* docs
* docs
* extra import
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
|
2020-10-10 12:31:00 -04:00 |
|
Ananya Harsh Jha
|
f76bc5254e
|
revamp entire metrics (#3868)
* removed metric
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* added new metrics
Co-authored-by: Teddy Koker teddy.koker@gmail.com
* pep8
Co-authored-by: Teddy Koker teddy.koker@gmail.com
* pep8
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* reset in compute, cache compute
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* reduce_ops handling
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* sync -> sync_dist, type annotations
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* wip docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* mean squared error
* docstring
* added mean ___ error metrics
* added mean ___ error metrics
* seperated files
* accuracy doctest
* gpu fix
* remove unnecessary mixin
* metric and accuracy docstring
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* metric docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* pep8, changelog
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* refactor dist utils, pep8
* refactor dist utils, pep8
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
|
2020-10-06 17:03:24 -04:00 |