f76bc5254e
revamp entire metrics ( #3868 )
...
* removed metric
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* added new metrics
Co-authored-by: Teddy Koker teddy.koker@gmail.com
* pep8
Co-authored-by: Teddy Koker teddy.koker@gmail.com
* pep8
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* win ddp tests skip
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* reset in compute, cache compute
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* reduce_ops handling
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* sync -> sync_dist, type annotations
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* wip docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* mean squared error
* docstring
* added mean ___ error metrics
* added mean ___ error metrics
* seperated files
* accuracy doctest
* gpu fix
* remove unnecessary mixin
* metric and accuracy docstring
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* metric docs
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* pep8, changelog
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
* refactor dist utils, pep8
* refactor dist utils, pep8
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-06 17:03:24 -04:00
93cf6d0054
[Metrics] class based embedding similarity + tests ( #3358 )
...
* embedding similarity class + test
* fix tests
* fix pep8
* add docs
* noindex
* Update docs/source/metrics.rst
* Update pytorch_lightning/metrics/self_supervised.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/metrics/self_supervised.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* suggestions
* changes to init
* move __all__
* fix imports
* Apply suggestions from code review
* assert typo
* change import
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-09-11 12:11:50 +02:00
b66ce88f0d
[metrics] Renaming of precision recall metric ( #3308 )
...
* rename metrics
* update docs
2020-09-01 14:59:33 -04:00
b7d72706c3
clean imports ( #2867 )
...
* clean imports
* miss
2020-08-08 00:33:51 +02:00
bda7cf1653
metrics: add SSIM ( #2671 )
...
* metrics: add SSIM
* Update CHANGELOG.md
fix codefactor issue
fix doctest
fix doctest
fix test
* added test for raise Error
2020-07-23 12:13:52 -04:00
0a65826462
metrics: add BLEU ( #2535 )
...
* metrics: added bleu score and test bleu
* metrics: fixed type hints in bleu
* bleu score moved to metrics/functional/nlp.py
* refactor with torch.Tensor
* Update test_sequence.py
* refactor as Borda requests and nltk==3.2
* locked nltk==3.3
* nltk>=3.3, parametrized smooth argument for test
* fix bleu_score example
* added class BLEUScore metrics and test
* added class BLEUScore metrics and test
* update CHANGELOG
* refactor with torchtext
* torchtext changed to optional import
* fix E501 line too long
* add else: in optional import
* remove pragma: no-cover
* constants changed to CAPITALS
* remove class in tests
* List -> Sequence, conda -> pip, cast with tensor
* add torchtext in test.txt
* remove torchtext from test.txt
* bump torchtext to 0.5.0
* bump torchtext to 0.5.0
* Apply suggestions from code review
* ignore bleu score in doctest, renamed to nlp.py
* back to implementation with torch
* remove --ignore in CI test, proper reference format
* apply justus comment
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-22 09:58:24 -04:00
6f4a488bae
Add functional regression metrics ( #2492 )
...
* Add functional regression metrics
* add functional tests
* add docs
* changelog
* init
* pep8
* docs
* docs
* setup docs
* docs
* Apply suggestions from code review
* Apply suggestions from code review
* typo
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-09 17:54:38 +02:00
1dc724239a
PSNR metric ( #2483 )
...
* Add stub PSNR metric
* Fix linter
* Add data range as parameter
* Add tests
* Add scikit-image
* Add PSNR to regression metrics and add functional
* Refactor to functional
* Fix linter
* Fix linter, again
* Fix linter, again
* Fix typo in test
* Fix typo in another test
* Add scikit-image to conda
* Lift numpy requirement
* Add random tests
* Update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-08 10:26:11 +02:00
e0b7359555
[metrics] IoU Metric ( #2062 )
...
* add iou function
* update stat scores
* add iou class
* add iou tests
* chlog
* Apply suggestions from code review
* tests
* docs
* Apply suggestions from code review
* docs
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-18 09:06:31 -04:00
1635ba1bb3
allow regression metrics to import ( #2225 )
...
* allow regression metrics to import
* allow regression metrics to import
* docs
* Apply suggestions from code review
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-17 18:44:29 -04:00
f1c732a77b
Metric docs fix ( #2209 )
...
* fix docs
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* Update metrics.rst
* title
* fix
* fix for num_classes
* chlog
* nb classes
* hints
* zero division
* add tests
* Update metrics.rst
* Update classification.py
* Update classification.py
* prune doctests
* docs
* Apply suggestions from code review
* Apply suggestions from code review
* flake8
* doctests
* formatting
* cleaning
* formatting
* formatting
* doctests
* flake8
* docs
* rename
* rename
* typo
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2020-06-17 07:34:39 -04:00
55fbcc00f6
Metrics docs ( #2184 )
...
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* add workers fix
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* add workers fix
* add workers fix
* doctests
* add workers fix
* add workers fix
* fixes
* fix docs
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* add workers fix
* Update docs/source/metrics.rst
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* doctests
* add workers fix
* fix docs
* fixes
* fixes
* fix doctests
* Apply suggestions from code review
* fix doctests
* fix examples
* bug
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* fixes
* fixes
* fixes
* fixes
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-06-16 07:42:56 -04:00
3436d00230
Native torch metrics ( #1488 )
...
* Create metric.py
* Create utils.py
* Create __init__.py
* Create __init__.py
* Create __init__.py
* add tests for metric utils
* add tests for metric utils
* add docstrings for metrics utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add function to recursively apply other function to collection
* add tests for this function
* add tests for this function
* add tests for this function
* update test
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* fix tests
* add metric tests
* fix to tensor conversion
* fix to tensor conversion
* fix apply to collection
* fix apply to collection
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* remove tests from init
* add missing type annotations
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* rename utils to convertors
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* rename file and fix imports
* added parametrized test
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* forgot to push these files...
* forgot to push these files...
* add explicit check for dtype to convert to
* add explicit check for dtype to convert to
* fix ddp tests
* fix ddp tests
* fix ddp tests
* remove explicit ddp destruction
* remove explicit ddp destruction
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add function to reduce tensors (similar to reduction in torch.nn)
* add functionals of reduction metrics
* add functionals of reduction metrics
* add more metrics
* pep8 fixes
* rename
* rename
* add reduction tests
* add first classification tests
* bugfixes
* bugfixes
* add more unit tests
* fix roc score metric
* fix tests
* solve tests
* fix docs
* Update CHANGELOG.md
* remove binaries
* solve changes from rebase
* add eos
* test auc independently
* fix formatting
* docs
* docs
* chlog
* move
* function descriptions
* Add documentation to native metrics (#2144 )
* add docs
* add docs
* Apply suggestions from code review
* formatting
* add docs
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
* Rename tests/metrics/test_classification.py to tests/metrics/functional/test_classification.py
* Rename tests/metrics/test_reduction.py to tests/metrics/functional/test_reduction.py
* Add module interface for classification metrics
* add basic tests for classification metrics' module interface
* pep8
* add additional converters
* add additional base class
* change baseclass for some metrics
* update classification tests
* update converter tests
* update metric tests
* Apply suggestions from code review
* tests-params
* tests-params
* imports
* pep8
* tests-params
* formatting
* fix test_metrics
* typo
* formatting
* fix dice tests
* fix decorator order
* fix tests
* seed
* dice test
* formatting
* try freeze test
* formatting
* fix tests
* try spawn
* formatting
* fix
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Xavier Sumba <c.uent@hotmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-06-13 08:47:25 -04:00
bd49b07fbb
Rework of Sklearn Metrics ( #1327 )
...
* Create utils.py
* Create __init__.py
* redo sklearn metrics
* add some more metrics
* add sklearn metrics
* Create __init__.py
* redo sklearn metrics
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* add sklearn metrics
* start adding sklearn tests
* fix typo
* return x and y only for curves
* fix typo
* add missing tests for sklearn funcs
* imports
* __all__
* imports
* fix sklearn arguments
* fix imports
* update requirements
* Update CHANGELOG.md
* Update test_sklearn_metrics.py
* formatting
* formatting
* format
* fix all warnings and formatting problems
* Update environment.yml
* Update requirements-extra.txt
* Update environment.yml
* Update requirements-extra.txt
* fix all warnings and formatting problems
* Update CHANGELOG.md
* docs
* inherit
* docs inherit.
* docs
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* docs
* req
* min
* Apply suggestions from code review
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Tullie Murrell <tulliemurrell@gmail.com>
2020-06-10 15:43:12 +02:00
9b629637b8
New metric classes ( #1326 ) ( #1877 )
...
* New metric classes (#1326 )
* Create metrics package
* Create metric.py
* Create utils.py
* Create __init__.py
* add tests for metric utils
* add docstrings for metrics utils
* add function to recursively apply other function to collection
* add tests for this function
* update test
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* update metric name
* remove example docs
* fix tests
* add metric tests
* fix to tensor conversion
* fix apply to collection
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* remove tests from init
* add missing type annotations
* rename utils to convertors
* Create metrics.rst
* Update index.rst
* Update index.rst
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/metrics/metric.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/utilities/test_apply_to_collection.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/metrics/convertors.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* add doctest example
* rename file and fix imports
* added parametrized test
* replace lambda with inlined function
* rename apply_to_collection to apply_func
* Separated class description from init args
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* adjust random values
* suppress output when seeding
* remove gpu from doctest
* Add requested changes and add ellipsis for doctest
* forgot to push these files...
* add explicit check for dtype to convert to
* fix ddp tests
* remove explicit ddp destruction
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* move dtype device mixin to more general place
* refactor to general device dtype mixin
* add initial metric package description
* change default to none for mac os
* pep8
* fix import
* Update index.rst
* Update ci-testing.yml
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update CHANGELOG.md
* Update pytorch_lightning/metrics/converters.py
* readme
* Update metric.py
* Update pytorch_lightning/metrics/converters.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-05-19 11:05:07 -04:00
Ananya Harsh Jha