b1347c956a
[Metrics] AUROC error on multilabel + improved testing ( #3350 )
...
* error on multilabel
* fix tests
* fix pep8
* changelog
* update doc test
* fix doctest
* fix doctest
* update from suggestion
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update test_classification.py
* Update test_classification.py
* retrigger test
* 'pep8
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-09-21 11:46:48 +02:00
76c4afb840
Fix IoU score for classes not present in target or pred ( #3098 )
...
* Fix IoU score for classes not present in target or pred
Fixes #3097
- Allow configurable not_present_score for IoU for classes
not present in target or pred. Defaults to 1.0.
- Also allow passing `num_classes` parameter through from iou
metric class down to its underlying functional iou
call.
* Changelog: move IoU not-present score fix to [unreleased]
* IoU: avoid recomputing class presence in target and pred
Use already-computed support, true positives, and false positives to
determine if a class is not present in either target or pred.
* Test IoU against sklearn jaccard_score
Also add TODO to test our IoU's not_present_score against sklearn's
jaccard_score's zero_division when it beecomes available.
* IoU: remove_bg -> ignore_index
Fixes #2736
- Rename IoU metric argument from `remove_bg` -> `ignore_index`.
- Accept an optional int class index to ignore, instead of a bool and
instead of always assuming the background class has index 0.
- If given, ignore the class index when computing the IoU output,
regardless of reduction method.
* Improve documentation for IoU not_present_score
* Update default IoU not_present_score to 0.0
* Add note about IoU division by zero
* Rename IoU not_present_score -> absent_score
* Update IoU absent score changelog wording
* Condense IoU absent_score argument docstring
* Remove unnecessary IoU ignore_index comment
* docstrings
* isort
* flake8
* Fix test of IoU against sklearn jaccard
Use macro instead of micro averaging in sklearn's jaccard score, to
match multi-class IoU, which conventionally takes per-class scores
before averaging.
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-17 10:37:49 +02:00
a552d4a2d5
fix normalize mode at confusion matrix (replace nans with zeros) ( #3465 )
...
* replace nans to 0 at conf. matrix & update tests
* cm.isnan() -> torch.isnan(cm)
* fix row-wise division while normalize
* update tests
* pep8 fix
* Update tests/metrics/test_classification.py
add comment to test
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update tests/metrics/functional/test_classification.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update pytorch_lightning/metrics/functional/classification.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* final update
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-09-14 10:05:51 +02:00
d05d4c78e1
add num_classes argument to confusion matrix ( #3450 )
...
* add num_classes arg to confusion matrix
* update ConfusionMatrix test
* final update)
2020-09-10 18:39:04 -04:00
b66ce88f0d
[metrics] Renaming of precision recall metric ( #3308 )
...
* rename metrics
* update docs
2020-09-01 14:59:33 -04:00
17d8773106
New modular metric interface ( #2528 )
...
* new base structure
* missing packages
* updated interface
* revert some changes
* fixes
* add changelog
* fix bug
* added description
* test for pickable
* fixing test
* fixing test
* fix pickle issue
* reduceop typehints back
* remove redundant module arg
* add save/load test
* add aggregate method
* text clarification
* fix doctest
* Apply suggestions from code review
* change test to results obj
* fix docs
* formatting
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* formatting
* pep
* Update CHANGELOG.md
* suggestions
* fix tests
* fix pep8
* fix tests
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-08-26 13:01:29 +02:00
e0b7359555
[metrics] IoU Metric ( #2062 )
...
* add iou function
* update stat scores
* add iou class
* add iou tests
* chlog
* Apply suggestions from code review
* tests
* docs
* Apply suggestions from code review
* docs
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-18 09:06:31 -04:00
f1c732a77b
Metric docs fix ( #2209 )
...
* fix docs
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* Update metrics.rst
* title
* fix
* fix for num_classes
* chlog
* nb classes
* hints
* zero division
* add tests
* Update metrics.rst
* Update classification.py
* Update classification.py
* prune doctests
* docs
* Apply suggestions from code review
* Apply suggestions from code review
* flake8
* doctests
* formatting
* cleaning
* formatting
* formatting
* doctests
* flake8
* docs
* rename
* rename
* typo
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2020-06-17 07:34:39 -04:00
3436d00230
Native torch metrics ( #1488 )
...
* Create metric.py
* Create utils.py
* Create __init__.py
* Create __init__.py
* Create __init__.py
* add tests for metric utils
* add tests for metric utils
* add docstrings for metrics utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add function to recursively apply other function to collection
* add tests for this function
* add tests for this function
* add tests for this function
* update test
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* fix tests
* add metric tests
* fix to tensor conversion
* fix to tensor conversion
* fix apply to collection
* fix apply to collection
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* remove tests from init
* add missing type annotations
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* rename file and fix imports
* added parametrized test
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* forgot to push these files...
* forgot to push these files...
* add explicit check for dtype to convert to
* add explicit check for dtype to convert to
* fix ddp tests
* fix ddp tests
* fix ddp tests
* remove explicit ddp destruction
* remove explicit ddp destruction
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add function to reduce tensors (similar to reduction in torch.nn)
* add functionals of reduction metrics
* add functionals of reduction metrics
* add more metrics
* pep8 fixes
* rename
* rename
* add reduction tests
* add first classification tests
* bugfixes
* bugfixes
* add more unit tests
* fix roc score metric
* fix tests
* solve tests
* fix docs
* Update CHANGELOG.md
* remove binaries
* solve changes from rebase
* add eos
* test auc independently
* fix formatting
* docs
* docs
* chlog
* move
* function descriptions
* Add documentation to native metrics (#2144 )
* add docs
* add docs
* Apply suggestions from code review
* formatting
* add docs
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
* Rename tests/metrics/test_classification.py to tests/metrics/functional/test_classification.py
* Rename tests/metrics/test_reduction.py to tests/metrics/functional/test_reduction.py
* Add module interface for classification metrics
* add basic tests for classification metrics' module interface
* pep8
* add additional converters
* add additional base class
* change baseclass for some metrics
* update classification tests
* update converter tests
* update metric tests
* Apply suggestions from code review
* tests-params
* tests-params
* imports
* pep8
* tests-params
* formatting
* fix test_metrics
* typo
* formatting
* fix dice tests
* fix decorator order
* fix tests
* seed
* dice test
* formatting
* try freeze test
* formatting
* fix tests
* try spawn
* formatting
* fix
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Xavier Sumba <c.uent@hotmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-06-13 08:47:25 -04:00
Nicki Skafte