Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
lightning/tests/base/datamodules.py

31 lines
1.1 KiB
Python
Raw Normal View History

Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
from torch.utils.data import random_split, DataLoader
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
from pytorch_lightning.core.datamodule import LightningDataModule
from tests.base.datasets import TrialMNIST
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
class TrialMNISTDataModule(LightningDataModule):
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
def __init__(self, data_dir: str = './'):
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
super().__init__()
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
self.data_dir = data_dir
def prepare_data(self):
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
TrialMNIST(self.data_dir, train=True, download=True)
TrialMNIST(self.data_dir, train=False, download=True)
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
def setup(self):
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
mnist_full = TrialMNIST(root=self.data_dir, train=True, num_samples=64, download=True)
self.mnist_train, self.mnist_val = random_split(mnist_full, [128, 64])
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
self.dims = tuple(self.mnist_train[0][0].shape)
Enable val/test loop disabling + datamodule tests (#2692) * :art: warn instead of error out on loaders * :bug: test misconfiguration should still fail * :construction: . * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj * updated docs with new result obj Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-25 16:57:40 +00:00
self.mnist_test = TrialMNIST(root=self.data_dir, train=False, num_samples=32, download=True)
Datamodule (#2668) * :sparkles: Add copy of pl_bolts datamodule to lightning * :sparkles: add datamodule to necessary init files * :construction: add datamodule property to LightningModule * :construction: . * :art: Let DataModule do its own thing * :construction: add back setup and run both hooks implicitly * :construction: . * :bug: fix add_argparse_args * :lipstick: apply black formatting and isort * :pencil: docstrings * :pencil: . * :pencil: . * :bug: overwrite cls prepare_data instead of instance * :pencil: . * :white_check_mark: add some tests * Update datamodule.py * Update datamodule.py * Update datamodule.py Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-07-24 15:42:15 +00:00
def train_dataloader(self):
return DataLoader(self.mnist_train, batch_size=32)
def val_dataloader(self):
return DataLoader(self.mnist_val, batch_size=32)
def test_dataloader(self):
return DataLoader(self.mnist_test, batch_size=32)