Commit Graph

64 Commits

Author SHA1 Message Date
Nicki Skafte 51097669b9
[metrics] change default behaviour of state dict ()
* fix state dict

* Update docs/source/metrics.rst

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* changelog

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-16 12:33:45 +00:00
Nicki Skafte 465ec752f8
Metric ddp bugfix ()
* changes

* fix spelling

* small note

* trying to fix ddp test

* fix ddp

* fix for test

* suggestion

* CHANGELOG

* Update pytorch_lightning/metrics/metric.py

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Sean Naren <sean@grid.ai>
2020-11-10 09:16:31 +01:00
ananthsub 5d08559c03
Avoid torchscript export for Metric forward ()
* Update metric.py

* add test

* Update CHANGELOG.md

* Update test_metric_lightning.py

* Update test_metric_lightning.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-11-03 23:02:02 +01:00
Dusan Drevicky 38bb4e2da0
[Metrics] Add multiclass auroc ()
* Add functional multiclass AUROC metric

* Add multiclass_auroc tests

* fixup! Add functional multiclass AUROC metric

* fixup! fixup! Add functional multiclass AUROC metric

* Add multiclass_auroc doc reference

* Update CHANGELOG

* formatting

* Shorter error message regex match in tests

* Set num classes as pytest parameter

* formatting

* Update CHANGELOG

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-10-30 19:56:13 +01:00
Jeff Yang 0f584faa6b
PyTorch 1.7 Stable support ()
* prepare for 1.7 support [ci skip]

* tpu [ci skip]

* test run 1.7

* all 1.7, needs to fix tests

* couple with torchvision

* windows try

* remove windows

* 1.7 is here

* on purpose fail [ci skip]

* return [ci skip]

* 1.7 docker

* back to normal [ci skip]

* change to some_val [ci skip]

* add seed [ci skip]

* 4 places [ci skip]

* fail on purpose [ci skip]

* verbose=True [ci skip]

* use filename to track

* use filename to track

* monitor epoch + changelog

* Update tests/checkpointing/test_model_checkpoint.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-10-30 15:42:14 +00:00
Nicki Skafte e0b856c105
[Metrics] Confusion matrix class interface ()
* docs + precision + recall + f_beta + refactor

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* rebase

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* fixes

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* added missing file

* docs

* docs

* extra import

* add confusion matrix

* add to docs

* add test

* pep8 + isort

* update tests

* move util function

* unify functional and class

* add to init

* remove old implementation

* update tests

* pep8

* add duplicate

* fix doctest

* Update pytorch_lightning/metrics/classification/confusion_matrix.py

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* changelog

* bullet point args

* bullet docs

* bullet docs

Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Roger Shieh <55400948+s-rog@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-10-30 11:44:25 +01:00
Nicki Skafte a937394312
[Metrics] Unification of regression ()
* moved to utility

* add files

* unify

* add desc

* update

* end of line

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* add back functional test in new interface

* pep8

* doctest fix

* test name fix

* unify psnr + add class psnr, TODO: psnr test refactor ala mean squared error

* unify psnr

* rm unused code

* pep8

* docs

* unify ssim

* lower tolerance for ssim

* fix import

* pep8

* docs

* flake8

* test smaller images

* trying to fix test

* no ddp test for ssim

* pep8

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-21 18:05:59 -04:00
Dusan Drevicky 96785b99df
Feature/4244 iou input expectations ()
* =Add iou input checks

* =Add test for iou input checks

* =Update docstring for iou pred and target
2020-10-21 09:01:24 -04:00
Abhinav Gupta 5d1583d7f2
Corrected f_beta computation ()
* Update f_beta.py

Added METRIC_EPS in the denominator to avoid nan values in f_beta score.

* Update f_beta.py

Made changes flake8 compliant

* Update f_beta.py

Makes use of class_reduce for macro f_beta computation to avoid nans

* Update f_beta.py

Made flake8 compliant

* Corrected F beta computation

* Removed offset to make the computation precise
2020-10-21 10:52:04 +02:00
Jirka Borovec f37444fa3e
CI: add flake8 () 2020-10-19 21:20:17 +01:00
Teddy Koker 827a557269
Add persistent flag to Metric.add_state ()
* add persistant flag to add_state in metrics

* wrap register_buffer with try catch

* pep8

* use loose version

* test

* pep8
2020-10-16 14:36:03 -04:00
Jirka Borovec 4290c9e1e8
Tests: clean metrics ()
* namme inputs

* sk rename

* imports
2020-10-15 07:50:08 +02:00
Nicki Skafte edcb6e49b9
Speedup of metric tests ()
* speedup

* something working

* update the rest

* more desc

* recurse tests/metrics again

* pep8

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-14 13:51:58 -04:00
Jirka Borovec 2c3b512c50
reverted "temporary drop metrics tests while speeding them up" and SKIP ()
* Revert "temporary drop metrics tests while speeding them up ()"

This reverts commit 86c70622fb.

* skip metrics tests

* skipping
2020-10-14 19:01:43 +02:00
William Falcon 86c70622fb
temporary drop metrics tests while speeding them up () 2020-10-11 07:38:58 -04:00
Ananya Harsh Jha ae8772490d
classification metrics ()
* docs + precision + recall + f_beta + refactor

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* rebase

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* fixes

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* added missing file

* docs

* docs

* extra import

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-10 12:31:00 -04:00
Nicki Skafte 8a3c800641
Update explained variance metric ()
* update metrics

* pep8

* Update pytorch_lightning/metrics/regression/explained_variance.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* add typing for testing utils

* change from assert to raise exception

* add test for raised shape error

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-10-09 19:23:12 -04:00
Teddy Koker b961e12f50
Implement Explained Variance Metric + metric fix ()
* metric fix, explained variance

* one more test

* pep8

* remove comment

* fix add_state condition

Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
2020-10-08 22:58:33 -04:00
Teddy Koker 7db26a913b
Add Metric <-> Lightning Module integration tests ()
* lightning module metric tests

* whitespace

* pep8
2020-10-08 22:58:17 -04:00
Ananya Harsh Jha 6f1a2ce517
integrate metrics API with self.log ()
* metrics integration into self.log

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* ddp and regualr test for self.log + metrics

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* pep8

* fix log tests

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-07 22:54:32 -04:00
Jirka Borovec 616fe82c7d
latest restore func metrics ()
* latest restore

* latest restore
2020-10-07 14:26:30 -04:00
Jirka Borovec b3d25262e9
restore functional metrics ()
* restore functional metrics

* clean

* fix
2020-10-07 12:48:58 -04:00
William Falcon 9c415d2c71
moves configure ddp to each backend ()
* moves configure ddp to each backend

* moves configure ddp to each backend

* moves configure ddp to each backend

* added torch manual seed in test_mean_error

* test for complicated batch structure

* test for complicated batch structure

* test for complicated batch structure

Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
2020-10-07 00:50:16 -04:00
Ananya Harsh Jha 4cd14c4237
fixes metrics pickle issue ()
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-06 20:33:57 -04:00
Ananya Harsh Jha f76bc5254e
revamp entire metrics ()
* removed metric

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* added new metrics

Co-authored-by: Teddy Koker teddy.koker@gmail.com

* pep8

Co-authored-by: Teddy Koker teddy.koker@gmail.com

* pep8

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* win ddp tests skip

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* win ddp tests skip

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* win ddp tests skip

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* win ddp tests skip

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* reset in compute, cache compute

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* reduce_ops handling

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* sync -> sync_dist, type annotations

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* wip docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* mean squared error

* docstring

* added mean ___ error metrics

* added mean ___ error metrics

* seperated files

* accuracy doctest

* gpu fix

* remove unnecessary mixin

* metric and accuracy docstring

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* metric docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* pep8, changelog

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* refactor dist utils, pep8

* refactor dist utils, pep8

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2020-10-06 17:03:24 -04:00
Nicki Skafte fe290280be
Metric aggregation testing ()
* aggregation testing

* add more tests

* mse

* more tests

* fix tests

* fix doctest

* fix codefactor

* fix import error

* fix doctest

* revert docfix

* test for model integration

* fix integration test

* added test cases

* fix rmsle

* aggregation testing

* add more tests

* mse

* more tests

* fix tests

* fix doctest

* fix codefactor

* fix import error

* fix doctest

* revert docfix

* test for model integration

* fix integration test

* fix psnr

* add warning/valueerror to embedding similarity

* fixed f scores

* disable some test

* fix tests

* fixing codefactor

* fix pep8

* changelog

* fix doctest

* cleaning test

* fix pickle error

* pickle fix

* fix pickle error

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* code cleanup + changes based on suggestions

* update based on suggestion

* update based on suggestions

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-10-01 15:37:51 +02:00
Nicki Skafte 9a7d1a1876
[metrics] Accuracy num_classes error fix ()
* change accuracy error to warning

* changelog
2020-10-01 13:00:42 +02:00
Nicki Skafte b1347c956a
[Metrics] AUROC error on multilabel + improved testing ()
* error on multilabel

* fix tests

* fix pep8

* changelog

* update doc test

* fix doctest

* fix doctest

* update from suggestion

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update test_classification.py

* Update test_classification.py

* retrigger test

* 'pep8

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-09-21 11:46:48 +02:00
Abe Botros 76c4afb840
Fix IoU score for classes not present in target or pred ()
* Fix IoU score for classes not present in target or pred

Fixes 

- Allow configurable not_present_score for IoU for classes
  not present in target or pred. Defaults to 1.0.
- Also allow passing `num_classes` parameter through from iou
  metric class down to its underlying functional iou
  call.

* Changelog: move IoU not-present score fix to [unreleased]

* IoU: avoid recomputing class presence in target and pred

Use already-computed support, true positives, and false positives to
determine if a class is not present in either target or pred.

* Test IoU against sklearn jaccard_score

Also add TODO to test our IoU's not_present_score against sklearn's
jaccard_score's zero_division when it beecomes available.

* IoU: remove_bg -> ignore_index

Fixes 

- Rename IoU metric argument from `remove_bg` -> `ignore_index`.
- Accept an optional int class index to ignore, instead of a bool and
  instead of always assuming the background class has index 0.
- If given, ignore the class index when computing the IoU output,
  regardless of reduction method.

* Improve documentation for IoU not_present_score

* Update default IoU not_present_score to 0.0

* Add note about IoU division by zero

* Rename IoU not_present_score -> absent_score

* Update IoU absent score changelog wording

* Condense IoU absent_score argument docstring

* Remove unnecessary IoU ignore_index comment

* docstrings

* isort

* flake8

* Fix test of IoU against sklearn jaccard

Use macro instead of micro averaging in sklearn's jaccard score, to
match multi-class IoU, which conventionally takes per-class scores
before averaging.

Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-17 10:37:49 +02:00
Nicki Skafte 28af34bc51
[Metrics] Class reduction similar to sklearn ()
* new class reduce interface

* update docs

* pep8

* update_class_metrics

* fix doctest

* changelog

* fix docs

* fix codefactor

* fix codefactor

* formatting

* fix typo

* fix typo

* typo pr -> per

* update from suggestion

* fix error

* Apply suggestions from code review

* Update CHANGELOG.md

* formatting

* timeouts

* docstring formatting for reg metrics

* pep

* flake8

* revert workflow changes

* suggestions

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-09-15 14:36:14 +02:00
Justus Schock 4dc4c8cfa5
Metric aggregation ()
* metric aggregation

* metric aggregation

* add at_least_1d

* fix output formatting

* add metric tests

* add missing test case

* remove reduce_op frm metric classes

* fix reduce_op stuff

* start test fixing

* fix tests due to aggregation

* fix faulty import

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* remove reduce_op docstrings

* add compute

* remove import

* remove collection metric

* update base class

* update tests

* Update metric.py

* Update metric.py

* Apply suggestions from code review

* change default aggregate

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-09-14 07:23:11 -04:00
Cookie_thief a552d4a2d5
fix normalize mode at confusion matrix (replace nans with zeros) ()
* replace nans to 0 at conf. matrix & update tests

* cm.isnan() -> torch.isnan(cm)

* fix row-wise division while normalize

* update tests

* pep8 fix

* Update tests/metrics/test_classification.py

add comment to test

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Update tests/metrics/functional/test_classification.py

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Update pytorch_lightning/metrics/functional/classification.py

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* final update

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-09-14 10:05:51 +02:00
Nicki Skafte 93cf6d0054
[Metrics] class based embedding similarity + tests ()
* embedding similarity class + test

* fix tests

* fix pep8

* add docs

* noindex

* Update docs/source/metrics.rst

* Update pytorch_lightning/metrics/self_supervised.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/self_supervised.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* suggestions

* changes to init

* move __all__

* fix imports

* Apply suggestions from code review

* assert typo

* change import

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-09-11 12:11:50 +02:00
Cookie_thief d05d4c78e1
add num_classes argument to confusion matrix ()
* add num_classes arg to confusion matrix

* update ConfusionMatrix test

* final update)
2020-09-10 18:39:04 -04:00
HT Liu d521c1b178
Fix: gather_all_tensors cross GPUs in DDP ()
* Fix: gather_all_tensors cross GPUs in metrics

* add a test case for gather_all_tensors_ddp in 
2020-09-03 12:27:32 +02:00
Nicki Skafte b66ce88f0d
[metrics] Renaming of precision recall metric ()
* rename metrics

* update docs
2020-09-01 14:59:33 -04:00
Sordie 888340d17e
Fix RMSLE metric ()
* fix rmsle

* Updated test to match rmsle fix

* Updated RMSLE example result to match functional

* chlog

* add randomized test

* fix pep8

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-08-26 08:02:53 -04:00
Nicki Skafte 17d8773106
New modular metric interface ()
* new base structure

* missing packages

* updated interface

* revert some changes

* fixes

* add changelog

* fix bug

* added description

* test for pickable

* fixing test

* fixing test

* fix pickle issue

* reduceop typehints back

* remove redundant module arg

* add save/load test

* add aggregate method

* text clarification

* fix doctest

* Apply suggestions from code review

* change test to results obj

* fix docs

* formatting

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* formatting

* pep

* Update CHANGELOG.md

* suggestions

* fix tests

* fix pep8

* fix tests

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-08-26 13:01:29 +02:00
Rohit Gupta 983c030326
fix reduction docstring and clean tests ()
* fix reduction docstring

* Update docstring and some cleanup

* miss

* suggestion from code review

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
2020-08-09 06:03:24 -04:00
Jirka Borovec ed3ee982b3
clean tests imports () 2020-08-06 16:58:51 +02:00
Younghun Roh ac4a215071
Faster Accuracy metric ()
* Faster classfication stats

* Faster accuracy metric

* minor change on cls metric

* Add out-of-bound class clamping

* Add more tests and minor fixes

* Resolve code style warning

* Update for 

* hotfix

* Update pytorch_lightning/metrics/functional/classification.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update about conversation

* Add docstring on stat_scores_multiple_classes

Co-authored-by: Younghun Roh <yhunroh@mindslab.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-08-06 11:40:35 +02:00
Jeff Yang 5bbcb8db1f
Improve SSIM ()
* make ssim fast

* remove padding

* pep8

* add comments for readability

* plus -> coef
2020-08-05 13:40:11 -04:00
Nicki Skafte e3732789d7
Add remaning sklearn metrics ()
* added balanced accuracy

* added dcg score

* added mean absolute error

* added mean squared error

* fix

* added mean squared log error

* add median absolute error and r2 score

* switch arguments

* added mean poisson deviance

* add mean gamma deviance and mean tweedie deviance

* fix styling

* added explained variance score

* added cohen kappa score

* added hamming, hinge, jaccard

* fix styling

* update sklearn requirement to newer version

* update requirement

* fix doctest

* fix tests

* added balanced accuracy

* added dcg score

* added mean absolute error

* added mean squared error

* fix

* added mean squared log error

* add median absolute error and r2 score

* switch arguments

* added mean poisson deviance

* add mean gamma deviance and mean tweedie deviance

* fix styling

* added explained variance score

* added cohen kappa score

* added hamming, hinge, jaccard

* fix styling

* update sklearn requirement to newer version

* fix doctest

* fix tests

* fix doctest

* fix failing docs

* fix test

* trying to fix errors

* Apply suggestions from code review

* format

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-08-05 11:32:53 +02:00
Justus Schock ad0f1194aa
Support Mean in DDP Sync ()
* Update converters.py

* Update test_converters.py

* pep8

* pep8 tests

* Update test_datamodules.py

* Update test_converters.py

* Update converters.py

* Update test_datamodules.py

* Update test_converters.py

* Update test_converters.py

* fix tests

* fix ddp tests on windows

* chlog

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
2020-08-04 18:32:20 +02:00
Jeff Yang bda7cf1653
metrics: add SSIM ()
* metrics: add SSIM

* Update CHANGELOG.md

fix codefactor issue

fix doctest

fix doctest

fix test

* added test for raise Error
2020-07-23 12:13:52 -04:00
William Falcon 62ce00f96c
EvalResult support for val loop (PR 3/5) ()
* add EvalResult to support to val/test loops
2020-07-22 13:53:10 -04:00
Jeff Yang 0a65826462
metrics: add BLEU ()
* metrics: added bleu score and test bleu

* metrics: fixed type hints in bleu

* bleu score moved to metrics/functional/nlp.py

* refactor with torch.Tensor

* Update test_sequence.py

* refactor as Borda requests and nltk==3.2

* locked nltk==3.3

* nltk>=3.3, parametrized smooth argument for test

* fix bleu_score example

* added class BLEUScore metrics and test

* added class BLEUScore metrics and test

* update CHANGELOG

* refactor with torchtext

* torchtext changed to optional import

* fix E501 line too long

* add else: in optional import

* remove pragma: no-cover

* constants changed to CAPITALS

* remove class in tests

* List -> Sequence, conda -> pip, cast with tensor

* add torchtext in test.txt

* remove torchtext from test.txt

* bump torchtext to 0.5.0

* bump torchtext to 0.5.0

* Apply suggestions from code review

* ignore bleu score in doctest, renamed to nlp.py

* back to implementation with torch

* remove --ignore in CI test, proper reference format

* apply justus comment

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-22 09:58:24 -04:00
Jirka Borovec 458bbad550
Avoid zeros in dice and iou ()
* nones

* fix

* fix

* test

* test

* test

* fix

* eps

* tpu

* eps

* type

* test tpu

* Update __init__.py

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-09 20:40:10 -04:00
Rohit Gupta 6f4a488bae
Add functional regression metrics ()
* Add functional regression metrics

* add functional tests

* add docs

* changelog

* init

* pep8

* docs

* docs

* setup docs

* docs

* Apply suggestions from code review

* Apply suggestions from code review

* typo

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-09 17:54:38 +02:00
William Falcon 69cbb62774
Finish ()
* removed spawns for test_converters and verified tests

Co-authored-by: Ananya Harsh Jha <ahj265@nyu.edu>
Co-authored-by: zcain <zcain@google.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-08 20:33:48 -04:00