lightning/pytorch_lightning
Abe Botros 76c4afb840
Fix IoU score for classes not present in target or pred (#3098)
* Fix IoU score for classes not present in target or pred

Fixes #3097

- Allow configurable not_present_score for IoU for classes
  not present in target or pred. Defaults to 1.0.
- Also allow passing `num_classes` parameter through from iou
  metric class down to its underlying functional iou
  call.

* Changelog: move IoU not-present score fix to [unreleased]

* IoU: avoid recomputing class presence in target and pred

Use already-computed support, true positives, and false positives to
determine if a class is not present in either target or pred.

* Test IoU against sklearn jaccard_score

Also add TODO to test our IoU's not_present_score against sklearn's
jaccard_score's zero_division when it beecomes available.

* IoU: remove_bg -> ignore_index

Fixes #2736

- Rename IoU metric argument from `remove_bg` -> `ignore_index`.
- Accept an optional int class index to ignore, instead of a bool and
  instead of always assuming the background class has index 0.
- If given, ignore the class index when computing the IoU output,
  regardless of reduction method.

* Improve documentation for IoU not_present_score

* Update default IoU not_present_score to 0.0

* Add note about IoU division by zero

* Rename IoU not_present_score -> absent_score

* Update IoU absent score changelog wording

* Condense IoU absent_score argument docstring

* Remove unnecessary IoU ignore_index comment

* docstrings

* isort

* flake8

* Fix test of IoU against sklearn jaccard

Use macro instead of micro averaging in sklearn's jaccard score, to
match multi-class IoU, which conventionally takes per-class scores
before averaging.

Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-17 10:37:49 +02:00
..
accelerators ref: precision plugins 1/n (#3504) 2020-09-15 09:56:12 -04:00
callbacks cleaning up stale logger tests + flake8 (#3490) 2020-09-14 00:06:48 -04:00
core docs: use ref for anchor links, fix a few typo (#3486) 2020-09-13 21:04:21 -04:00
loggers Pass epoch argument to Comet Logger (#3438) 2020-09-15 14:30:42 +02:00
metrics Fix IoU score for classes not present in target or pred (#3098) 2020-09-17 10:37:49 +02:00
overrides Fix an import deprecation warning (#3110) 2020-08-24 10:59:47 +02:00
plugins ref: precision plugins 1/n (#3504) 2020-09-15 09:56:12 -04:00
profiler added copyright notices (#3062) 2020-08-19 22:03:22 -04:00
trainer fix gradient norm tracking for row_log_interval > 1 (#3489) 2020-09-15 18:41:27 +02:00
tuner cleaning up stale logger tests + flake8 (#3490) 2020-09-14 00:06:48 -04:00
utilities implement fix and test (#3459) 2020-09-11 10:55:58 -04:00
__init__.py Update __init__.py 2020-09-11 15:59:36 -04:00
py.typed make PyTorch Lightning PEP 561 Compliant (#3187) 2020-09-09 13:37:03 -04:00