fe290280be
Metric aggregation testing ( #3517 )
...
* aggregation testing
* add more tests
* mse
* more tests
* fix tests
* fix doctest
* fix codefactor
* fix import error
* fix doctest
* revert docfix
* test for model integration
* fix integration test
* added test cases
* fix rmsle
* aggregation testing
* add more tests
* mse
* more tests
* fix tests
* fix doctest
* fix codefactor
* fix import error
* fix doctest
* revert docfix
* test for model integration
* fix integration test
* fix psnr
* add warning/valueerror to embedding similarity
* fixed f scores
* disable some test
* fix tests
* fixing codefactor
* fix pep8
* changelog
* fix doctest
* cleaning test
* fix pickle error
* pickle fix
* fix pickle error
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* code cleanup + changes based on suggestions
* update based on suggestion
* update based on suggestions
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-10-01 15:37:51 +02:00
9a7d1a1876
[metrics] Accuracy num_classes error fix ( #3764 )
...
* change accuracy error to warning
* changelog
2020-10-01 13:00:42 +02:00
b1347c956a
[Metrics] AUROC error on multilabel + improved testing ( #3350 )
...
* error on multilabel
* fix tests
* fix pep8
* changelog
* update doc test
* fix doctest
* fix doctest
* update from suggestion
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update test_classification.py
* Update test_classification.py
* retrigger test
* 'pep8
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-09-21 11:46:48 +02:00
76c4afb840
Fix IoU score for classes not present in target or pred ( #3098 )
...
* Fix IoU score for classes not present in target or pred
Fixes #3097
- Allow configurable not_present_score for IoU for classes
not present in target or pred. Defaults to 1.0.
- Also allow passing `num_classes` parameter through from iou
metric class down to its underlying functional iou
call.
* Changelog: move IoU not-present score fix to [unreleased]
* IoU: avoid recomputing class presence in target and pred
Use already-computed support, true positives, and false positives to
determine if a class is not present in either target or pred.
* Test IoU against sklearn jaccard_score
Also add TODO to test our IoU's not_present_score against sklearn's
jaccard_score's zero_division when it beecomes available.
* IoU: remove_bg -> ignore_index
Fixes #2736
- Rename IoU metric argument from `remove_bg` -> `ignore_index`.
- Accept an optional int class index to ignore, instead of a bool and
instead of always assuming the background class has index 0.
- If given, ignore the class index when computing the IoU output,
regardless of reduction method.
* Improve documentation for IoU not_present_score
* Update default IoU not_present_score to 0.0
* Add note about IoU division by zero
* Rename IoU not_present_score -> absent_score
* Update IoU absent score changelog wording
* Condense IoU absent_score argument docstring
* Remove unnecessary IoU ignore_index comment
* docstrings
* isort
* flake8
* Fix test of IoU against sklearn jaccard
Use macro instead of micro averaging in sklearn's jaccard score, to
match multi-class IoU, which conventionally takes per-class scores
before averaging.
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-17 10:37:49 +02:00
28af34bc51
[Metrics] Class reduction similar to sklearn ( #3322 )
...
* new class reduce interface
* update docs
* pep8
* update_class_metrics
* fix doctest
* changelog
* fix docs
* fix codefactor
* fix codefactor
* formatting
* fix typo
* fix typo
* typo pr -> per
* update from suggestion
* fix error
* Apply suggestions from code review
* Update CHANGELOG.md
* formatting
* timeouts
* docstring formatting for reg metrics
* pep
* flake8
* revert workflow changes
* suggestions
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-15 14:36:14 +02:00
4dc4c8cfa5
Metric aggregation ( #3321 )
...
* metric aggregation
* metric aggregation
* add at_least_1d
* fix output formatting
* add metric tests
* add missing test case
* remove reduce_op frm metric classes
* fix reduce_op stuff
* start test fixing
* fix tests due to aggregation
* fix faulty import
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* remove reduce_op docstrings
* add compute
* remove import
* remove collection metric
* update base class
* update tests
* Update metric.py
* Update metric.py
* Apply suggestions from code review
* change default aggregate
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-09-14 07:23:11 -04:00
a552d4a2d5
fix normalize mode at confusion matrix (replace nans with zeros) ( #3465 )
...
* replace nans to 0 at conf. matrix & update tests
* cm.isnan() -> torch.isnan(cm)
* fix row-wise division while normalize
* update tests
* pep8 fix
* Update tests/metrics/test_classification.py
add comment to test
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update tests/metrics/functional/test_classification.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update pytorch_lightning/metrics/functional/classification.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* final update
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-09-14 10:05:51 +02:00
93cf6d0054
[Metrics] class based embedding similarity + tests ( #3358 )
...
* embedding similarity class + test
* fix tests
* fix pep8
* add docs
* noindex
* Update docs/source/metrics.rst
* Update pytorch_lightning/metrics/self_supervised.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/self_supervised.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* suggestions
* changes to init
* move __all__
* fix imports
* Apply suggestions from code review
* assert typo
* change import
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-09-11 12:11:50 +02:00
d05d4c78e1
add num_classes argument to confusion matrix ( #3450 )
...
* add num_classes arg to confusion matrix
* update ConfusionMatrix test
* final update)
2020-09-10 18:39:04 -04:00
d521c1b178
Fix: gather_all_tensors cross GPUs in DDP ( #3319 )
...
* Fix: gather_all_tensors cross GPUs in metrics
* add a test case for gather_all_tensors_ddp in #3253
2020-09-03 12:27:32 +02:00
b66ce88f0d
[metrics] Renaming of precision recall metric ( #3308 )
...
* rename metrics
* update docs
2020-09-01 14:59:33 -04:00
888340d17e
Fix RMSLE metric ( #3188 )
...
* fix rmsle
* Updated test to match rmsle fix
* Updated RMSLE example result to match functional
* chlog
* add randomized test
* fix pep8
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-08-26 08:02:53 -04:00
17d8773106
New modular metric interface ( #2528 )
...
* new base structure
* missing packages
* updated interface
* revert some changes
* fixes
* add changelog
* fix bug
* added description
* test for pickable
* fixing test
* fixing test
* fix pickle issue
* reduceop typehints back
* remove redundant module arg
* add save/load test
* add aggregate method
* text clarification
* fix doctest
* Apply suggestions from code review
* change test to results obj
* fix docs
* formatting
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* formatting
* pep
* Update CHANGELOG.md
* suggestions
* fix tests
* fix pep8
* fix tests
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-08-26 13:01:29 +02:00
983c030326
fix reduction docstring and clean tests ( #2885 )
...
* fix reduction docstring
* Update docstring and some cleanup
* miss
* suggestion from code review
Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
2020-08-09 06:03:24 -04:00
ed3ee982b3
clean tests imports ( #2834 )
2020-08-06 16:58:51 +02:00
ac4a215071
Faster Accuracy metric ( #2775 )
...
* Faster classfication stats
* Faster accuracy metric
* minor change on cls metric
* Add out-of-bound class clamping
* Add more tests and minor fixes
* Resolve code style warning
* Update for #2781
* hotfix
* Update pytorch_lightning/metrics/functional/classification.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update about conversation
* Add docstring on stat_scores_multiple_classes
Co-authored-by: Younghun Roh <yhunroh@mindslab.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-08-06 11:40:35 +02:00
5bbcb8db1f
Improve SSIM ( #2833 )
...
* make ssim fast
* remove padding
* pep8
* add comments for readability
* plus -> coef
2020-08-05 13:40:11 -04:00
e3732789d7
Add remaning sklearn metrics ( #2562 )
...
* added balanced accuracy
* added dcg score
* added mean absolute error
* added mean squared error
* fix
* added mean squared log error
* add median absolute error and r2 score
* switch arguments
* added mean poisson deviance
* add mean gamma deviance and mean tweedie deviance
* fix styling
* added explained variance score
* added cohen kappa score
* added hamming, hinge, jaccard
* fix styling
* update sklearn requirement to newer version
* update requirement
* fix doctest
* fix tests
* added balanced accuracy
* added dcg score
* added mean absolute error
* added mean squared error
* fix
* added mean squared log error
* add median absolute error and r2 score
* switch arguments
* added mean poisson deviance
* add mean gamma deviance and mean tweedie deviance
* fix styling
* added explained variance score
* added cohen kappa score
* added hamming, hinge, jaccard
* fix styling
* update sklearn requirement to newer version
* fix doctest
* fix tests
* fix doctest
* fix failing docs
* fix test
* trying to fix errors
* Apply suggestions from code review
* format
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-08-05 11:32:53 +02:00
ad0f1194aa
Support Mean in DDP Sync ( #2568 )
...
* Update converters.py
* Update test_converters.py
* pep8
* pep8 tests
* Update test_datamodules.py
* Update test_converters.py
* Update converters.py
* Update test_datamodules.py
* Update test_converters.py
* Update test_converters.py
* fix tests
* fix ddp tests on windows
* chlog
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-08-04 18:32:20 +02:00
bda7cf1653
metrics: add SSIM ( #2671 )
...
* metrics: add SSIM
* Update CHANGELOG.md
fix codefactor issue
fix doctest
fix doctest
fix test
* added test for raise Error
2020-07-23 12:13:52 -04:00
62ce00f96c
EvalResult support for val loop (PR 3/5) ( #2651 )
...
* add EvalResult to support to val/test loops
2020-07-22 13:53:10 -04:00
0a65826462
metrics: add BLEU ( #2535 )
...
* metrics: added bleu score and test bleu
* metrics: fixed type hints in bleu
* bleu score moved to metrics/functional/nlp.py
* refactor with torch.Tensor
* Update test_sequence.py
* refactor as Borda requests and nltk==3.2
* locked nltk==3.3
* nltk>=3.3, parametrized smooth argument for test
* fix bleu_score example
* added class BLEUScore metrics and test
* added class BLEUScore metrics and test
* update CHANGELOG
* refactor with torchtext
* torchtext changed to optional import
* fix E501 line too long
* add else: in optional import
* remove pragma: no-cover
* constants changed to CAPITALS
* remove class in tests
* List -> Sequence, conda -> pip, cast with tensor
* add torchtext in test.txt
* remove torchtext from test.txt
* bump torchtext to 0.5.0
* bump torchtext to 0.5.0
* Apply suggestions from code review
* ignore bleu score in doctest, renamed to nlp.py
* back to implementation with torch
* remove --ignore in CI test, proper reference format
* apply justus comment
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-22 09:58:24 -04:00
458bbad550
Avoid zeros in dice and iou ( #2567 )
...
* nones
* fix
* fix
* test
* test
* test
* fix
* eps
* tpu
* eps
* type
* test tpu
* Update __init__.py
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-09 20:40:10 -04:00
6f4a488bae
Add functional regression metrics ( #2492 )
...
* Add functional regression metrics
* add functional tests
* add docs
* changelog
* init
* pep8
* docs
* docs
* setup docs
* docs
* Apply suggestions from code review
* Apply suggestions from code review
* typo
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-09 17:54:38 +02:00
69cbb62774
Finish #2549 ( #2557 )
...
* removed spawns for test_converters and verified tests
Co-authored-by: Ananya Harsh Jha <ahj265@nyu.edu>
Co-authored-by: zcain <zcain@google.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-08 20:33:48 -04:00
d3f5717e81
Fix parameters and docs in metrics ( #2473 )
...
* Fix parameters and docs in metrics
* doc improvements
* whitespace
* doc indentation
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* zero
* drop defaults
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-08 14:11:40 +02:00
1dc724239a
PSNR metric ( #2483 )
...
* Add stub PSNR metric
* Fix linter
* Add data range as parameter
* Add tests
* Add scikit-image
* Add PSNR to regression metrics and add functional
* Refactor to functional
* Fix linter
* Fix linter, again
* Fix linter, again
* Fix typo in test
* Fix typo in another test
* Add scikit-image to conda
* Lift numpy requirement
* Add random tests
* Update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-08 10:26:11 +02:00
f1c96930b1
repair CI for Win ( #2358 )
...
* no cov
* no cov
* ReduceOp
* group
* reduce_op.sum
* Update sklearns.py
* formatting
* horovod
* Apply suggestions from code review
* horovod
* horovod
* horovod
* horovod
* ci
* print
* ci
* timeout
* timeout
* time
* fix
* distributed cpu
* pipes
* time
* cpu
* spawn
* spawn
* spawn
* tp
* separate
* os
* os
* npm
* Fix load_from_checkpoint() not working with URL on Windows
* Update CHANGELOG
* Update CHANGELOG.md
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* Apply suggestions from code review
* fix
* fix meta tags creating empty lines
* pyright
* node
* fix httpserver address
* drop tutils.default_trainer_options
* imports
* Better fix for load_from_checkpoint() not working with absolute path on Windows (#2294 )
* Fix load_from_checkpoint() not working with URL on Windows
* Update CHANGELOG
* Update CHANGELOG.md
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* drop duplicate
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: airium <airium@outlook.com>
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: AIRIUM <38249940+airium@users.noreply.github.com>
2020-06-26 21:38:25 -04:00
29179dbfcc
Fix ROC metric for CUDA tensors ( #2304 )
...
* Fix ROC metric for CUDA tensors
Previously roc metric (and auroc) errors when passed in CUDA tensors,
due to torch.tensor construction without specifying device.
This fixes the error by using F.pad instead.
* Update test_classification.py
* Update test_classification.py
* chlog
* Update test_classification.py
* Update test_classification.py
* Update tests/metrics/functional/test_classification.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update test_classification.py
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-23 15:19:16 +02:00
92f122e0df
Fix average_precision metric ( #2319 )
...
* Fixed average_precision metric, parenthesis were missing. Added test test that failed with the old implementation
* Modified CHANGELOG.md
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-23 13:21:00 +02:00
e0b7359555
[metrics] IoU Metric ( #2062 )
...
* add iou function
* update stat scores
* add iou class
* add iou tests
* chlog
* Apply suggestions from code review
* tests
* docs
* Apply suggestions from code review
* docs
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-18 09:06:31 -04:00
ead874b17d
Regression metrics ( #2221 )
...
* add regression metrics
* solve tests
* add docs
2020-06-17 13:44:06 -04:00
f1c732a77b
Metric docs fix ( #2209 )
...
* fix docs
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* Update metrics.rst
* title
* fix
* fix for num_classes
* chlog
* nb classes
* hints
* zero division
* add tests
* Update metrics.rst
* Update classification.py
* Update classification.py
* prune doctests
* docs
* Apply suggestions from code review
* Apply suggestions from code review
* flake8
* doctests
* formatting
* cleaning
* formatting
* formatting
* doctests
* flake8
* docs
* rename
* rename
* typo
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2020-06-17 07:34:39 -04:00
55fbcc00f6
Metrics docs ( #2184 )
...
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* add workers fix
* add workers fix
* doctests
* add workers fix
* add workers fix
* fixes
* fix docs
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* doctests
* add workers fix
* fix docs
* fixes
* fixes
* fix doctests
* Apply suggestions from code review
* fix doctests
* fix examples
* bug
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* fixes
* fixes
* fixes
* fixes
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-06-16 07:42:56 -04:00
db7bb4c348
cleaning tests ( #2201 )
2020-06-15 22:03:40 -04:00
02262d0a93
Fix for accuracy calculation ( #2183 )
...
* accuracy_fix
* fix line length
* Apply suggestions from code review
* Update test_classification.py
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-06-14 18:14:29 -04:00
3436d00230
Native torch metrics ( #1488 )
...
* Create metric.py
* Create utils.py
* Create __init__.py
* Create __init__.py
* Create __init__.py
* add tests for metric utils
* add tests for metric utils
* add docstrings for metrics utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add function to recursively apply other function to collection
* add tests for this function
* add tests for this function
* add tests for this function
* update test
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* fix tests
* add metric tests
* fix to tensor conversion
* fix to tensor conversion
* fix apply to collection
* fix apply to collection
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* remove tests from init
* add missing type annotations
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* rename file and fix imports
* added parametrized test
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* forgot to push these files...
* forgot to push these files...
* add explicit check for dtype to convert to
* add explicit check for dtype to convert to
* fix ddp tests
* fix ddp tests
* fix ddp tests
* remove explicit ddp destruction
* remove explicit ddp destruction
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add function to reduce tensors (similar to reduction in torch.nn)
* add functionals of reduction metrics
* add functionals of reduction metrics
* add more metrics
* pep8 fixes
* rename
* rename
* add reduction tests
* add first classification tests
* bugfixes
* bugfixes
* add more unit tests
* fix roc score metric
* fix tests
* solve tests
* fix docs
* Update CHANGELOG.md
* remove binaries
* solve changes from rebase
* add eos
* test auc independently
* fix formatting
* docs
* docs
* chlog
* move
* function descriptions
* Add documentation to native metrics (#2144 )
* add docs
* add docs
* Apply suggestions from code review
* formatting
* add docs
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
* Rename tests/metrics/test_classification.py to tests/metrics/functional/test_classification.py
* Rename tests/metrics/test_reduction.py to tests/metrics/functional/test_reduction.py
* Add module interface for classification metrics
* add basic tests for classification metrics' module interface
* pep8
* add additional converters
* add additional base class
* change baseclass for some metrics
* update classification tests
* update converter tests
* update metric tests
* Apply suggestions from code review
* tests-params
* tests-params
* imports
* pep8
* tests-params
* formatting
* fix test_metrics
* typo
* formatting
* fix dice tests
* fix decorator order
* fix tests
* seed
* dice test
* formatting
* try freeze test
* formatting
* fix tests
* try spawn
* formatting
* fix
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Xavier Sumba <c.uent@hotmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-06-13 08:47:25 -04:00
bd49b07fbb
Rework of Sklearn Metrics ( #1327 )
...
* Create utils.py
* Create __init__.py
* redo sklearn metrics
* add some more metrics
* add sklearn metrics
* Create __init__.py
* redo sklearn metrics
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add sklearn metrics
* start adding sklearn tests
* fix typo
* return x and y only for curves
* fix typo
* add missing tests for sklearn funcs
* imports
* __all__
* imports
* fix sklearn arguments
* fix imports
* update requirements
* Update CHANGELOG.md
* Update test_sklearn_metrics.py
* formatting
* formatting
* format
* fix all warnings and formatting problems
* Update environment.yml
* Update requirements-extra.txt
* Update environment.yml
* Update requirements-extra.txt
* fix all warnings and formatting problems
* Update CHANGELOG.md
* docs
* inherit
* docs inherit.
* docs
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* docs
* req
* min
* Apply suggestions from code review
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
2020-06-10 15:43:12 +02:00
9b629637b8
New metric classes ( #1326 ) ( #1877 )
...
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* move dtype device mixin to more general place
* refactor to general device dtype mixin
* add initial metric package description
* change default to none for mac os
* pep8
* fix import
* Update index.rst
* Update ci-testing.yml
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/converters.py
* readme
* Update metric.py
* Update pytorch_lightning/metrics/converters.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-05-19 11:05:07 -04:00
Nicki Skafte