lightning/docs/source-pytorch/levels/advanced_level_21.rst

:orphan:

##################################################
Level 20: Train models with billions of parameters
##################################################

Scale to billions of parameters with multiple distributed strategies.

----

.. raw:: html

    <div class="display-card-container">
        <div class="row">

.. Add callout items below this line

.. displayitem::
   :header: Scale with distributed strategies
   :description: Learn about different distributed strategies to reach bigger model parameter sizes.
   :col_css: col-md-6
   :button_link: ../accelerators/gpu_intermediate.html
   :height: 150
   :tag: intermediate

.. displayitem::
   :header: Train models with billions of parameters
   :description: Scale to billions of params on GPUs with FSDP or Deepspeed.
   :col_css: col-md-6
   :button_link: ../advanced/model_parallel.html
   :height: 150
   :tag: advanced

.. raw:: html

        </div>
    </div>