Early stopping doc (#3193)

* Updated explanation to enabling early stopping via boolean flag.

Now also includes case of returning Result objects.

* Improved API documentation for checkpoint_on and early_stp_on in results.

* Apply suggestions from code review

* Fix terminology.

* Fix wrong documentation. Strict checking is disabled when using structured results.

* element typo

* update remaining edits from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

docs/source/early_stopping.rst

@@ -33,9 +33,17 @@ callback can be used to monitor a validation metric and stop the training when n
 There are two ways to enable the EarlyStopping callback:
 
 -   Set `early_stop_callback=True`.
-    The callback will look for 'val_loss' in the dict returned by 
-    :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_epoch_end`
-    and raise an error if `val_loss` is not present.
+    If a dict is returned by
+    :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_epoch_end`,
+    the callback will look for `val_loss` in the dict
+    and display a warning if `val_loss` is not present.
+    Otherwise, if a :class:`~pytorch_lightning.core.step_result.Result` is returned by
+    :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_epoch_end`,
+    :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_step` or
+    :meth:`~pytorch_lightning.core.lightning.LightningModule.training_step`,
+    the `early_stop_on` metric, specified in the initialization of the
+    :class:`~pytorch_lightning.core.step_result.Result` object is used
+    and display a warning if it was not specified.
 
     .. testcode::
 

pytorch_lightning/core/step_result.py

@@ -501,7 +501,16 @@ class TrainResult(Result):
         Args:
             minimize: Metric currently being minimized.
             early_stop_on: Metric to early stop on.
+                Should be a one element tensor if combined with default
+                :class:`~pytorch_lightning.callbacks.early_stopping.EarlyStopping`.
+                If this result is returned by
+                :meth:`~pytorch_lightning.core.lightning.LightningModule.training_step`,
+                the specified value will be averaged across all steps.
             checkpoint_on: Metric to checkpoint on.
+                Should be a one element tensor if combined with default checkpoint callback.
+                If this result is returned by
+                :meth:`~pytorch_lightning.core.lightning.LightningModule.training_step`,
+                the specified value will be averaged across all steps.
             hiddens:
         """
 
@@ -656,7 +665,16 @@ class EvalResult(Result):
 
         Args:
             early_stop_on: Metric to early stop on.
+                Should be a one element tensor if combined with default
+                :class:`~pytorch_lightning.callbacks.early_stopping.EarlyStopping`.
+                If this result is returned by
+                :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_step`,
+                the specified value will be averaged across all steps.
             checkpoint_on: Metric to checkpoint on.
+                Should be a one element tensor if combined with default checkpoint callback.
+                If this result is returned by
+                :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_step`,
+                the specified value will be averaged across all steps.
             hiddens:
         """
 

pytorch_lightning/trainer/__init__.py

@@ -411,10 +411,13 @@ early_stop_callback
 Callback for early stopping.
 early_stop_callback (:class:`pytorch_lightning.callbacks.EarlyStopping`)
 
-- ``True``: A default callback monitoring ``'val_loss'`` is created.
-   Will raise an error if ``'val_loss'`` is not found.
+- ``True``: A default callback monitoring ``'val_loss'`` (if dict is returned in validation loop) or
+  ``early_stopping_on`` (if :class:`~pytorch_lightning.core.step_result.Result` is returned) is created.
+  Will raise an error if a dictionary is returned and ``'val_loss'`` is not found.
+  Will raise an error if a :class:`~pytorch_lightning.core.step_result.Result` is returned
+  and ``early_stopping_on`` was not specified.
 - ``False``: Early stopping will be disabled.
-- ``None``: The default callback monitoring ``'val_loss'`` is created.
+- ``None``: Same as, if ``True`` is specified.
 - Default: ``None``.
 
 .. testcode::