Restructure Fabric docs (2/n) (#17126)
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com> Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
This commit is contained in:
Lightning Forever
GitHub
parent
7bc2a65212
commit
c886317c0c
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 7.7 KiB After Width: | Height: | Size: 19 KiB |
|
|
@ -0,0 +1,22 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
#############################
|
||||
lightning.fabric.accelerators
|
||||
#############################
|
||||
|
||||
|
||||
Accelerators
|
||||
^^^^^^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.accelerators
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Accelerator
|
||||
CPUAccelerator
|
||||
CUDAAccelerator
|
||||
MPSAccelerator
|
||||
TPUAccelerator
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
####################################
|
||||
lightning.fabric.plugins.collectives
|
||||
####################################
|
||||
|
||||
.. warning::
|
||||
This is an `experimental <https://lightning.ai/docs/pytorch/latest/versioning.html>`__ feature.
|
||||
|
||||
|
||||
Collectives
|
||||
^^^^^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.collectives
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Collective
|
||||
TorchCollective
|
||||
SingleDeviceCollective
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
#####################################
|
||||
lightning.fabric.plugins.environments
|
||||
#####################################
|
||||
|
||||
|
||||
Environments
|
||||
^^^^^^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.environments
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate_noindex.rst
|
||||
|
||||
~cluster_environment.ClusterEnvironment
|
||||
~kubeflow.KubeflowEnvironment
|
||||
~lightning.LightningEnvironment
|
||||
~lsf.LSFEnvironment
|
||||
~mpi.MPIEnvironment
|
||||
~slurm.SLURMEnvironment
|
||||
~torchelastic.TorchElasticEnvironment
|
||||
~xla.XLAEnvironment
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
#######################
|
||||
lightning.fabric.Fabric
|
||||
#######################
|
||||
|
||||
|
||||
Fabric
|
||||
^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.fabric
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Fabric
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
###########################
|
||||
lightning.fabric.plugins.io
|
||||
###########################
|
||||
|
||||
|
||||
.. warning::
|
||||
This is an `experimental <https://lightning.ai/docs/pytorch/latest/versioning.html>`__ feature.
|
||||
|
||||
|
||||
IO
|
||||
^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.io
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~checkpoint_io.CheckpointIO
|
||||
~torch_io.TorchCheckpointIO
|
||||
~xla.XLACheckpointIO
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
########################
|
||||
lightning.fabric.loggers
|
||||
########################
|
||||
|
||||
|
||||
Loggers
|
||||
^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.loggers
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Logger
|
||||
CSVLogger
|
||||
TensorBoardLogger
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
##################################
|
||||
lightning.fabric.plugins.precision
|
||||
##################################
|
||||
|
||||
|
||||
Precision
|
||||
^^^^^^^^^
|
||||
|
||||
.. TODO(fabric): include DeepSpeedPrecision
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.precision
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Precision
|
||||
DoublePrecision
|
||||
MixedPrecision
|
||||
TPUPrecision
|
||||
TPUBf16Precision
|
||||
FSDPPrecision
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
.. include:: ../links.rst
|
||||
|
||||
###########################
|
||||
lightning.fabric.strategies
|
||||
###########################
|
||||
|
||||
|
||||
Strategies
|
||||
^^^^^^^^^^
|
||||
|
||||
.. TODO(fabric): include DeepSpeedStrategy, XLAStrategy
|
||||
|
||||
.. currentmodule:: lightning.fabric.strategies
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ./generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Strategy
|
||||
DDPStrategy
|
||||
DataParallelStrategy
|
||||
FSDPStrategy
|
||||
ParallelStrategy
|
||||
SingleDeviceStrategy
|
||||
SingleTPUStrategy
|
||||
|
|
@ -1,88 +0,0 @@
|
|||
################
|
||||
Fabric Utilities
|
||||
################
|
||||
|
||||
|
||||
seed_everything
|
||||
===============
|
||||
|
||||
This function sets the random seed in important libraries.
|
||||
In a single line of code, you can seed PyTorch, NumPy, and Python:
|
||||
|
||||
.. code-block:: diff
|
||||
|
||||
+ from lightning.fabric import seed_everything
|
||||
|
||||
seed = 42
|
||||
- random.seed(seed)
|
||||
- numpy.random.seed(seed)
|
||||
- torch.manual_seed(seed)
|
||||
- torch.cuda.manual_seed(seed)
|
||||
|
||||
+ seed_everything(seed)
|
||||
|
||||
The same is also available as a method on the Fabric object if you don't want to import it separately:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from lightning.fabric import Fabric
|
||||
|
||||
fabric.Fabric()
|
||||
fabric.seed_everything(42)
|
||||
|
||||
|
||||
In distributed settings, you may need to set a different seed per process, depending on the application.
|
||||
For example, when generating noise or data augmentations. This is very straightforward:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
fabric = Fabric(...)
|
||||
fabric.seed_everything(seed + fabric.global_rank)
|
||||
|
||||
|
||||
By default, :meth:`~lightning.fabric.fabric.Fabric.seed_everything` also handles the initialization of the seed in :class:`~torch.utils.data.DataLoader` worker processes:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
fabric = Fabric(...)
|
||||
|
||||
# By default, we handle DataLoader workers too:
|
||||
fabric.seed_everything(..., workers=True)
|
||||
|
||||
# Can be turned off:
|
||||
fabric.seed_everything(..., workers=False)
|
||||
|
||||
|
||||
----
|
||||
|
||||
|
||||
print
|
||||
=====
|
||||
|
||||
Avoid duplicated print statements in the logs in distributed training by using Fabric's :meth:`~lightning.fabric.fabric.Fabric.print` method:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
print("This message gets printed in every process. That's a bit messy!")
|
||||
|
||||
fabric = Fabric(...)
|
||||
fabric.print("This message gets printed only in the main process. Much cleaner!")
|
||||
|
||||
|
||||
----
|
||||
|
||||
|
||||
is_wrapped
|
||||
==========
|
||||
|
||||
You can test whether an object (:class:`~torch.nn.Module`, :class:`~torch.optim.Optimizer`, :class:`~torch.utils.data.DataLoader`) was already set up once by Fabric:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from lightning.fabric import is_wrapped
|
||||
|
||||
if not is_wrapped(model):
|
||||
model = fabric.setup(model)
|
||||
|
||||
if not is_wrapped(train_dataloader):
|
||||
train_dataloader = fabric.setup_dataloaders(train_dataloader)
|
||||
|
|
@ -1,140 +0,0 @@
|
|||
.. include:: links.rst
|
||||
|
||||
#############
|
||||
API Reference
|
||||
#############
|
||||
|
||||
|
||||
Fabric
|
||||
^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.fabric
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Fabric
|
||||
|
||||
|
||||
Accelerators
|
||||
^^^^^^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.accelerators
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Accelerator
|
||||
CPUAccelerator
|
||||
CUDAAccelerator
|
||||
MPSAccelerator
|
||||
TPUAccelerator
|
||||
|
||||
|
||||
Loggers
|
||||
^^^^^^^
|
||||
|
||||
.. currentmodule:: lightning.fabric.loggers
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Logger
|
||||
CSVLogger
|
||||
TensorBoardLogger
|
||||
|
||||
Precision
|
||||
"""""""""
|
||||
|
||||
.. TODO(fabric): include DeepSpeedPrecision
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.precision
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Precision
|
||||
DoublePrecision
|
||||
MixedPrecision
|
||||
TPUPrecision
|
||||
TPUBf16Precision
|
||||
FSDPPrecision
|
||||
|
||||
|
||||
Environments
|
||||
""""""""""""
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.environments
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate_noindex.rst
|
||||
|
||||
~cluster_environment.ClusterEnvironment
|
||||
~kubeflow.KubeflowEnvironment
|
||||
~lightning.LightningEnvironment
|
||||
~lsf.LSFEnvironment
|
||||
~mpi.MPIEnvironment
|
||||
~slurm.SLURMEnvironment
|
||||
~torchelastic.TorchElasticEnvironment
|
||||
~xla.XLAEnvironment
|
||||
|
||||
|
||||
IO
|
||||
""
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.io
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~checkpoint_io.CheckpointIO
|
||||
~torch_io.TorchCheckpointIO
|
||||
~xla.XLACheckpointIO
|
||||
|
||||
|
||||
Collectives
|
||||
"""""""""""
|
||||
|
||||
.. currentmodule:: lightning.fabric.plugins.collectives
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Collective
|
||||
TorchCollective
|
||||
SingleDeviceCollective
|
||||
|
||||
|
||||
Strategies
|
||||
^^^^^^^^^^
|
||||
|
||||
.. TODO(fabric): include DeepSpeedStrategy, XLAStrategy
|
||||
|
||||
.. currentmodule:: lightning.fabric.strategies
|
||||
|
||||
.. autosummary::
|
||||
:toctree: api/generated
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Strategy
|
||||
DDPStrategy
|
||||
DataParallelStrategy
|
||||
FSDPStrategy
|
||||
ParallelStrategy
|
||||
SingleDeviceStrategy
|
||||
SingleTPUStrategy
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
########
|
||||
Glossary
|
||||
########
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div class="display-card-container">
|
||||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: Accelerator
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Apple Silicon
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Autocast
|
||||
:button_link: ../fundamentals/precision.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Barrier
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Broadcast
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Callback
|
||||
:button_link: ../guide/callbacks.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Checkpoint
|
||||
:button_link: ../guide/checkpoints.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: CLI
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Cloud
|
||||
:button_link: ../guide/multi_node/cloud.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Collective
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: CUDA
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Gather
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Gradient Accumulation
|
||||
:button_link: ../advanced/gradient_accumulation.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: GPU
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Jypyter
|
||||
:button_link: ../launch/notebook.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Launch
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: LightningModule
|
||||
:button_link: ../guide/lightning_module.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Logger
|
||||
:button_link: ../guide/logging.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Mixed Precision
|
||||
:button_link: ../fundamentals/precision.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: MPI
|
||||
:button_link: ../guide/multi_node/other.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: MPS
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Multi-GPU
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Multi-Node
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Notebook
|
||||
:button_link: ../launch/notebook.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Optimizers
|
||||
:button_link: ../advanced/multiple_setup.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Precision
|
||||
:button_link: ../fundamentals/precision.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Reduce
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: SLURM
|
||||
:button_link: ../guide/multi_node/slurm.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: TensorBoard
|
||||
:button_link: ../guide/logging.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: TorchElastic
|
||||
:button_link: ../guide/multi_node/barebones.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: TorchRun
|
||||
:button_link: ../guide/multi_node/barebones.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: TPU
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
||||
.. displayitem::
|
||||
:header: Trainer
|
||||
:button_link: ../guide/trainer_template.html
|
||||
:col_css: col-md-4
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -0,0 +1,163 @@
|
|||
#############
|
||||
How-to Guides
|
||||
#############
|
||||
|
||||
|
||||
******
|
||||
Basics
|
||||
******
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div class="display-card-container">
|
||||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: Convert to Fabric in 5 minutes
|
||||
:description: Learn how to add Fabric to your PyTorch code
|
||||
:button_link: ../fundamentals/convert.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Scale your model with Accelerators
|
||||
:description: Take advantage of your hardware with a switch of a flag
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Structure your Fabric code
|
||||
:description: Best practices for setting up your training script with Fabric
|
||||
:button_link: ../fundamentals/code_structure.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Launch distributed training
|
||||
:description: Launch a Python script on multiple devices and machines
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Launch Fabric in a notebook
|
||||
:description: Launch on multiple devices from within a Jupyter notebook
|
||||
:button_link: ../fundamentals/notebooks.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Improve performance with Mixed-Precision training
|
||||
:description: Save memory and speed up training using mixed precision
|
||||
:button_link: ../fundamentals/precision.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: basic
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
**********************
|
||||
Build your own Trainer
|
||||
**********************
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div class="display-card-container">
|
||||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: Organize your model code with with LightningModule
|
||||
:description: Organize your code in a LightningModule and use it with Fabric
|
||||
:button_link: lightning_module.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Encapsulate code into Callbacks
|
||||
:description: Make use of the Callback system in Fabric
|
||||
:button_link: callbacks.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Track and visualize experiments
|
||||
:description: Learn how Fabric helps you remove boilerplate code for tracking metrics with a logger
|
||||
:button_link: logging.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Save and load model progress
|
||||
:description: Efficient saving and loading of model weights, training state, hyperparameters and more.
|
||||
:button_link: checkpoint.html
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Build your own Trainer
|
||||
:description: Take our Fabric Trainer template and customize it for your needs
|
||||
:button_link: https://github.com/Lightning-AI/lightning/tree/master/examples/fabric/build_your_own_trainer
|
||||
:col_css: col-md-4
|
||||
:height: 150
|
||||
:tag: intermediate
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
***************
|
||||
Advanced Topics
|
||||
***************
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div class="display-card-container">
|
||||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: Use efficient gradient accumulation
|
||||
:description: Learn how to perform efficient gradient accumulation in distributed settings
|
||||
:button_link: ../advanced/gradient_accumulation.html
|
||||
:col_css: col-md-4
|
||||
:height: 160
|
||||
:tag: advanced
|
||||
|
||||
.. displayitem::
|
||||
:header: Distribute communication
|
||||
:description: Learn all about communication primitives for distributed operation. Gather, reduce, broadcast, etc.
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
:height: 160
|
||||
:tag: advanced
|
||||
|
||||
.. displayitem::
|
||||
:header: Use multiple models and optimizers
|
||||
:description: See how flexible Fabric is to work with multiple models and optimizers!
|
||||
:button_link: ../advanced/multiple_setup.html
|
||||
:col_css: col-md-4
|
||||
:height: 160
|
||||
:tag: advanced
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -118,23 +118,34 @@ For alternative ways to install, read the :doc:`installation guide <fundamentals
|
|||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Get started in steps
|
||||
:maxdepth: 1
|
||||
:caption: Get started in steps
|
||||
|
||||
Basic <levels/basic>
|
||||
Intermediate <levels/intermediate>
|
||||
Advanced <levels/advanced>
|
||||
Basic skills <levels/basic>
|
||||
Intermediate skills <levels/intermediate>
|
||||
Advanced skills <levels/advanced>
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:name: api
|
||||
:caption: Core API Reference
|
||||
|
||||
Fabric Arguments <api/fabric_args>
|
||||
Fabric Methods <api/fabric_methods>
|
||||
Utilities <api/utilities>
|
||||
Full API Reference <api_reference>
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Full API Reference
|
||||
|
||||
Accelerators <api/accelerators>
|
||||
Collectives <api/collectives>
|
||||
Environments <api/environments>
|
||||
Fabric <api/fabric>
|
||||
IO <api/io>
|
||||
Loggers <api/loggers>
|
||||
Precision <api/precision>
|
||||
Strategies <api/strategies>
|
||||
|
||||
|
||||
.. toctree::
|
||||
|
|
@ -143,6 +154,9 @@ For alternative ways to install, read the :doc:`installation guide <fundamentals
|
|||
:caption: More
|
||||
|
||||
Examples <examples/index>
|
||||
Glossary <glossary/index>
|
||||
How-tos <guide/index>
|
||||
Style Guide <fundamentals/code_structure>
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ Advanced skills
|
|||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: Efficient Gradient Accumulation
|
||||
:header: Use efficient gradient accumulation
|
||||
:description: Learn how to perform efficient gradient accumulation in distributed settings
|
||||
:button_link: ../advanced/gradient_accumulation.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -25,7 +25,7 @@ Advanced skills
|
|||
:tag: advanced
|
||||
|
||||
.. displayitem::
|
||||
:header: Distributed Communication
|
||||
:header: Distribute communication
|
||||
:description: Learn all about communication primitives for distributed operation. Gather, reduce, broadcast, etc.
|
||||
:button_link: ../advanced/distributed_communication.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -33,7 +33,7 @@ Advanced skills
|
|||
:tag: advanced
|
||||
|
||||
.. displayitem::
|
||||
:header: Multiple Models and Optimizers
|
||||
:header: Use multiple models and optimizers
|
||||
:description: See how flexible Fabric is to work with multiple models and optimizers!
|
||||
:button_link: ../advanced/multiple_setup.html
|
||||
:col_css: col-md-4
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ Basic skills
|
|||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Accelerators
|
||||
:header: Scale your model with Accelerators
|
||||
:description: Take advantage of your hardware with a switch of a flag
|
||||
:button_link: ../fundamentals/accelerators.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -37,7 +37,7 @@ Basic skills
|
|||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Code Structure
|
||||
:header: Structure your Fabric code
|
||||
:description: Best practices for setting up your training script with Fabric
|
||||
:button_link: ../fundamentals/code_structure.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -45,7 +45,7 @@ Basic skills
|
|||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Launch Distributed Training
|
||||
:header: Launch distributed training
|
||||
:description: Launch a Python script on multiple devices and machines
|
||||
:button_link: ../fundamentals/launch.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -53,7 +53,7 @@ Basic skills
|
|||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Fabric in Notebooks
|
||||
:header: Launch Fabric in a notebook
|
||||
:description: Launch on multiple devices from within a Jupyter notebook
|
||||
:button_link: ../fundamentals/notebooks.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -61,7 +61,7 @@ Basic skills
|
|||
:tag: basic
|
||||
|
||||
.. displayitem::
|
||||
:header: Mixed Precision Training
|
||||
:header: Improve performance with Mixed-Precision training
|
||||
:description: Save memory and speed up training using mixed precision
|
||||
:button_link: ../fundamentals/precision.html
|
||||
:col_css: col-md-4
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Intermediate skills
|
|||
<div class="row">
|
||||
|
||||
.. displayitem::
|
||||
:header: The LightningModule
|
||||
:header: Organize your model code with with LightningModule
|
||||
:description: Organize your code in a LightningModule and use it with Fabric
|
||||
:button_link: ../guide/lightning_module.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -27,7 +27,7 @@ Intermediate skills
|
|||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Callbacks
|
||||
:header: Encapsulate code into Callbacks
|
||||
:description: Make use of the Callback system in Fabric
|
||||
:button_link: ../guide/callbacks.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -35,7 +35,7 @@ Intermediate skills
|
|||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Logging
|
||||
:header: Track and visualize experiments
|
||||
:description: Learn how Fabric helps you remove boilerplate code for tracking metrics with a logger
|
||||
:button_link: ../guide/logging.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -43,7 +43,7 @@ Intermediate skills
|
|||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Checkpoints
|
||||
:header: Save and load model progress
|
||||
:description: Efficient saving and loading of model weights, training state, hyperparameters and more.
|
||||
:button_link: ../guide/checkpoint.html
|
||||
:col_css: col-md-4
|
||||
|
|
@ -51,7 +51,7 @@ Intermediate skills
|
|||
:tag: intermediate
|
||||
|
||||
.. displayitem::
|
||||
:header: Trainer Template
|
||||
:header: Build your own Trainer
|
||||
:description: Take our Fabric Trainer template and customize it for your needs
|
||||
:button_link: https://github.com/Lightning-AI/lightning/tree/master/examples/fabric/build_your_own_trainer
|
||||
:col_css: col-md-4
|
||||
|
|
|
|||
Loading…
Reference in New Issue