diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1930f99cd1..ada11278f6 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -72,6 +72,7 @@ repos: hooks: - id: isort name: Format imports + exclude: docs/source-app - repo: https://github.com/psf/black rev: 22.6.0 @@ -97,11 +98,11 @@ repos: #- mdformat-black - mdformat_frontmatter exclude: | - (?x)^( - src/pytorch_lightning/CHANGELOG.md| - src/lightning_app/CHANGELOG.md| - src/lightning_lite/CHANGELOG.md - )$ + (?x)^( + src/pytorch_lightning/CHANGELOG.md| + src/lightning_app/CHANGELOG.md| + src/lightning_lite/CHANGELOG.md + )$ - repo: https://github.com/PyCQA/flake8 rev: 4.0.1 diff --git a/docs/source-app/get_started/add_an_interactive_demo.rst b/docs/source-app/get_started/add_an_interactive_demo.rst index 98e6c3fd8a..655d86f17f 100644 --- a/docs/source-app/get_started/add_an_interactive_demo.rst +++ b/docs/source-app/get_started/add_an_interactive_demo.rst @@ -6,7 +6,7 @@ Add an Interactive Demo .. _add_an_interactive_Demo: -**Required background:** Basic Python familiarity and complete the :ref:`install` guide. +**Required background:** Basic Python familiarity and complete the install guide. **Goal:** We'll walk you through the 4 key steps to run a Lightning App that trains and demos a model. diff --git a/docs/source-app/get_started/build_model.rst b/docs/source-app/get_started/build_model.rst index ed2a90fbe3..40a40738e9 100644 --- a/docs/source-app/get_started/build_model.rst +++ b/docs/source-app/get_started/build_model.rst @@ -6,7 +6,7 @@ Build and Train a Model ####################### -**Required background:** Basic Python familiarity and complete the :ref:`install` guide. +**Required background:** Basic Python familiarity and complete the guide. **Goal:** We'll walk you through the creation of a model using PyTorch Lightning. diff --git a/docs/source-app/get_started/go_beyond_training.rst b/docs/source-app/get_started/go_beyond_training.rst index eade993e59..8d5a842362 100644 --- a/docs/source-app/get_started/go_beyond_training.rst +++ b/docs/source-app/get_started/go_beyond_training.rst @@ -6,7 +6,7 @@ Start from an ML system template .. _go_beyond_training: -**Required background:** Basic Python familiarity and complete the :ref:`install` guide. +**Required background:** Basic Python familiarity and complete the install guide. **Goal:** We'll walk you through the 4 key steps to run a Lightning App that trains and demos a model. diff --git a/docs/source-app/get_started/go_beyond_training_content.rst b/docs/source-app/get_started/go_beyond_training_content.rst index 5ee47eb219..f86b634370 100644 --- a/docs/source-app/get_started/go_beyond_training_content.rst +++ b/docs/source-app/get_started/go_beyond_training_content.rst @@ -25,7 +25,7 @@ Here are `the entire App's code `, don't forget to activate it before running commands. +If you are using a virtual env, don't forget to activate it before running commands. You must do so in every new shell. .. tip:: We highly recommend using virtual environments. diff --git a/docs/source-app/index.rst b/docs/source-app/index.rst index aa67dff1b4..b38bc46255 100644 --- a/docs/source-app/index.rst +++ b/docs/source-app/index.rst @@ -11,7 +11,7 @@ Build models and full stack AI apps ⚡ *Lightning fast*. .. join_slack:: :align: left -**Featured examples of what you can build with Lightning:** +**Featured examples of what you can do with Lightning:** | @@ -22,7 +22,7 @@ Build models and full stack AI apps ⚡ *Lightning fast*. .. app_card:: :title: Develop and Train - :description: Train an LLM (512 GPUs) + :description: Train an LLM (64 GPUs) :width: 280 :image: https://lightning-ai-docs.s3.amazonaws.com/develop_n_train_v1.jpg :preview: https://lightning.ai @@ -34,10 +34,10 @@ Build models and full stack AI apps ⚡ *Lightning fast*. :title: Serve and deploy :description: Production-ready stable diffusion server (<2s latency) :width: 280 + :app_id: HvUwbEG90E :image: https://lightning-ai-docs.s3.amazonaws.com/serve_n_deploy_v1.jpg - :preview: https://lightning.ai :deploy: https://lightning.ai - :target: https://apple.com + :target: https://01gbx4m78rbkpczdf5cpz2hpbh.litng-ai-03.litng.ai/root.api_component/ :tags: App .. app_card:: @@ -56,9 +56,12 @@ Build models and full stack AI apps ⚡ *Lightning fast*. ---- -************************ -Build modular components -************************ +**************************************** +Build self-contained, modular components +**************************************** +Build modular, self-contained components and plug them into your existing workflows. +A Lightning component organizes arbitrary code so it can run on the cloud. A component can train a model, deploy, or even host a web UI. +The component manages its own infrastructure, cloud costs, networking and more, so you can focus on application logic and not engineering. .. raw:: html @@ -66,27 +69,19 @@ Build modular components -.. raw:: html +| -
-
-

- Build modular, self-contained components that can train and deploy models, host a web UI or run arbitrary code on the cloud. - A Lightning Component manages its own infrastructure, cloud costs, networking and more, so you can focus on application logic and not engineering. +| -

- Combine your components and prebuilt ones from our gallery - to create full-stack AI apps, ⚡ Lightning fast. -

-
-
+**Run an example component on the cloud**: .. lit_tabs:: - :code_files: landing_app.py; - :highlights: 6 + :titles: Hello CPU world; Hello GPU (accelerated) world; Train PyTorch on a cloud GPU; Train PyTorch ⚡ on cloud GPUs; Deploy a model on cloud GPUs; Run a model script; XGBoost; XGBoost (GPU accelerated); Build a streamlit demo + :code_files: ./levels/basic/hello_components/hello_world.py; ./levels/basic/hello_components/hello_world_gpu.py; ./levels/basic/hello_components/train_pytorch.py; ./levels/basic/hello_components/train_ptl.py; ./levels/basic/hello_components/deploy_model.py; ./levels/basic/hello_components/run_script.py; ./levels/basic/hello_components/xgboost.py; ./levels/basic/hello_components/xgboost_gpu.py; ./levels/basic/hello_components/build_demo.py + :highlights: 7; 10, 11; 3, 6;3;4;5; 6, 9; 15, 20, 21; 10, 14, 29 :app_id: abc123 - :height: 250px - :code_only: True + :tab_rows: 4 + :height: 550px .. lit_tabs:: :code_files: landing_app_run.bash @@ -94,12 +89,7 @@ Build modular components :height: 150px :code_only: True -| - -.. raw:: html - -
-
+Explore pre-built community components in `our gallery `_. | @@ -112,7 +102,7 @@ Build modular components .. displayitem:: :header: Get started - :description: Learn to go from Lightning components to full stack AI apps step-by-step. + :description: Learn to build Lightning components step-by-step. :col_css: col-md-12 :button_link: levels/basic/index.html :height: 160 diff --git a/docs/source-app/install/installation.rst b/docs/source-app/install/installation.rst index ed7dbc67af..5ade49619e 100644 --- a/docs/source-app/install/installation.rst +++ b/docs/source-app/install/installation.rst @@ -6,7 +6,7 @@ Installation ############ -**Prerequisites**: Use Python 3.8.x or later (3.8.x, 3.9.x, 3.10.x). We also recommend you install in a virtual environment (`learn how `_). +**Prerequisites**: Use Python 3.8.x or later (3.8.x, 3.9.x, 3.10.x). We also recommend you install in a virtual environment (learn how). .. lit_tabs:: :descriptions: Pip; Macs, Apple Silicon (M1/M2/M3); Windows diff --git a/docs/source-app/install/mac.bash b/docs/source-app/install/mac.bash index c8877f948b..22825bb246 100644 --- a/docs/source-app/install/mac.bash +++ b/docs/source-app/install/mac.bash @@ -2,4 +2,4 @@ export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 -python -m pip install -U lightning +pip install lightning diff --git a/docs/source-app/install/pip.bash b/docs/source-app/install/pip.bash index 1fe569674a..f6d38b76b8 100644 --- a/docs/source-app/install/pip.bash +++ b/docs/source-app/install/pip.bash @@ -1 +1 @@ -python -m pip install -U lightning +pip install lightning diff --git a/docs/source-app/levels/basic/build_a_dag.rst b/docs/source-app/levels/basic/build_a_dag.rst index d4eeae8715..dec9d4050c 100644 --- a/docs/source-app/levels/basic/build_a_dag.rst +++ b/docs/source-app/levels/basic/build_a_dag.rst @@ -1,3 +1,5 @@ +:orphan: + ########################### Example: Deploy a model API ########################### diff --git a/docs/source-app/levels/basic/build_a_lightning_component.rst b/docs/source-app/levels/basic/build_a_lightning_component.rst index 82bebdccf5..fa633c9cd3 100644 --- a/docs/source-app/levels/basic/build_a_lightning_component.rst +++ b/docs/source-app/levels/basic/build_a_lightning_component.rst @@ -1,6 +1,6 @@ -########################################## -Package your code in a lightning component -########################################## +############################################## +Level 1: Package code in a lightning component +############################################## **Prereqs:** You know *basic* Python. @@ -14,10 +14,13 @@ Package your code in a lightning component ********************************* Why you need Lightning components ********************************* -ML workflows and full stack AI apps require many pieces working such as training, deploying, data annotation. However, this tight coupling -can lead to monoliths that are hard to scale or many microservices that are hard to monitor, coordinate -and scale. A Lightning component is a self-contained piece of code (ie: a microservice) that executes your code -and manages its own infrastructure, auto-scaling and costs for you. +A Lightning component organizes a piece of code into a self-contained, modular component that +can be integrated into your existing workflows or assembled to form a Lightning app. +A Lightning component manages its own infrastructure, auto-scaling, cost management, and more, so you +can focus on the program logic and not the cloud engineering. + +Components run on the cloud or your laptop without code changes 🤯🤯. Connect components using your current workflow management tools or use +Lightning apps to build powerful sequential AND reactive workflows. .. raw:: html @@ -27,9 +30,7 @@ and manages its own infrastructure, auto-scaling and costs for you. | -By using Lightning components you can focus on application logic without dealing with cloud operations -or even how components speak to each other. Together, Lightning components form a Lightning App that -gives you these benefits: +Organizing your code into Lightning components offers these benefits: .. collapse:: Build systems not scripts @@ -38,7 +39,6 @@ gives you these benefits: The Lightning structure forces best practices so you don't have to be an expert production engineer. Although it feels like you're writing a script, you are actually building a production-ready system. - .. collapse:: Cost control | @@ -54,7 +54,6 @@ gives you these benefits: coding experience. You can write code like you normally do, and the Lightning structure ensures your code is implicitely production ready... even if you're just doing research. - .. collapse:: For experts: Scale with full control | @@ -63,6 +62,14 @@ gives you these benefits: scaling logic, fault-tolerance and even pre-provisioning, all from Python. We even give you full flexibility to use tools like `terraform <../../cloud/customize_a_lightning_cluster.html>`_ to optimize cloud clusters for your Lightning apps. +.. collapse:: Integrate into your current workflow tools + + | + + Lightning components are self-contained pieces of funcionality. Add them to your current workflow + tools to quickly fill in gaps in your ML workflow such as monitoring drift, training LLMs and more. + You can (optionally) use the Lightning App to integrate components into a cohesive workflow. + .. collapse:: Packaged code | @@ -91,45 +98,25 @@ Install Lightning ***************** First, install Lightning. -.. code:: bash - - python -m pip install -U lightning - -.. collapse:: Mac M1/M2/M3 and Windows users - - | - - **Mac** - - To install on Mac, set these 2 environment variables - - .. code-block:: bash - - # needed for M1/M2/M3 - export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 - export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 - - python -m pip install -U lightning - - **Windows users** - - To install on Windows: - - - setup an alias for Python: python=python3 - - Add the root folder of Lightning to the Environment Variables to PATH +.. lit_tabs:: + :descriptions: Pip; Macs, Apple Silicon (M1/M2/M3); Windows + :code_files: /install/pip.bash; /install/mac.bash; /install/windows.bash + :tab_rows: 4 + :height: 180px ---- ************************** Build your first component ************************** -A Lightning component organizes Python code so it can run on the cloud and be connected with other components to form a Lightning App. -Pick one of these components to run: +A Lightning component organizes Python code into a self-contained module so it can run on the cloud. + +**Run one of these components!** .. lit_tabs:: - :titles: Hello CPU world; Hello GPU (accelerated) world; Train PyTorch on a cloud GPU; Train PyTorch ⚡ on cloud GPUs; Deploy a model on cloud GPUs; Run a model script; Build a model web UI - :code_files: ./hello_components/hello_world.py; ./hello_components/hello_world_gpu.py; ./hello_components/train_pytorch.py; ./hello_components/train_ptl.py; ./hello_components/deploy_model.py; ./hello_components/run_script.py; ./hello_components/build_demo.py - :highlights: 7; 10, 11; 3, 7-23;3;4;5;6 + :titles: Hello CPU world; Hello GPU (accelerated) world; Train PyTorch (cloud GPU); Train PyTorch with Lightning Trainer (cloud GPUs); Deploy a model on cloud GPUs; Run a model script; XGBoost; XGBoost (GPU accelerated); Build a streamlit demo + :code_files: ./hello_components/hello_world.py; ./hello_components/hello_world_gpu.py; ./hello_components/train_pytorch.py; ./hello_components/train_ptl.py; ./hello_components/deploy_model.py; ./hello_components/run_script.py; ./hello_components/xgboost.py; ./hello_components/xgboost_gpu.py; ./hello_components/build_demo.py + :highlights: 7; 10, 11; 3, 6;3;4;5; 6, 9; 15, 20, 21; 10, 14, 29 :app_id: abc123 :tab_rows: 4 :height: 550px @@ -142,6 +129,7 @@ Components run the same on the cloud and locally on your choice of hardware. :titles: Lightning Cloud (fully-managed); Your AWS account; Your own hardware :code_files: ./hello_components/code_run_cloud.bash; ./hello_components/code_run_cloud_yours.bash; ./hello_components/code_run_local.bash :tab_rows: 4 + :highlights: ; 5; 0 :height: 195px ---- @@ -155,17 +143,17 @@ powerful Lightning app. Here are a few key features available to super-charge yo .. lit_tabs:: :titles: 15+ accelerators; Auto-stop idle machines; Auto-timeout submitted work; Use spot machines (~70% discount); Work with massive datasets; Mount cloud storage; Use a custom container :code_files: ./key_features/accelerators.py; ./key_features/idle_machine.py; ./key_features/auto_timeout.py; ./key_features/spot.py; ./key_features/massive_dataset.py; ./key_features/mount_data.py; ./key_features/custom_container.py; - :highlights: 10;10;10;10;10;2,6,9, 10; 7 + :highlights: 11;11;11;11;11;2,7,10, 11; 11 :app_id: abc123 :tab_rows: 3 :height: 430px ---- -*************************** -Use the community ecosystem -*************************** -Lightning has a vibrant collection of community-built components you can use as templates or to inspire you. +****************************** +Use community-built components +****************************** +Lightning has a vibrant collection of `community-built components `_ you can use as templates or to inspire you. ---- diff --git a/docs/source-app/levels/basic/connect_lightning_components.rst b/docs/source-app/levels/basic/connect_lightning_components.rst index d4c1f9d63b..caf8359bec 100644 --- a/docs/source-app/levels/basic/connect_lightning_components.rst +++ b/docs/source-app/levels/basic/connect_lightning_components.rst @@ -11,10 +11,13 @@ Level 2: Connect components into a full stack AI app **************************** What is a full stack AI app? **************************** -A full stack AI app coordinates 2 or more `Lightning components `_ together. -We call this system of components interacting with each other a Lightning App. +In the ML world, workflows coordinate multiple pieces of code working together. In Lightning, +when we coordinate 2 or more `Lightning components `_ working together, +we instead call it a Lightning App. The difference will become more obvious when we introduce reactive +workflows in the advanced section. -In this guide, we'll coordinate 2 components together and explain how it works. +For the time being, we'll go over how to coordinate 2 components together in a traditional workflow setting +and explain how it works. .. note:: If you've used workflow tools for Python, this page describes conventional DAGs. In `level 5 <../intermediate/run_lightning_work_in_parallel.html>`_, we introduce reactive workflows that generalize beyond DAGs @@ -33,10 +36,10 @@ on a separate CPU machine. We save money by stopping the GPU machine when the wo :titles: Import Lightning; Define Component 1; Define Component 2; Orchestrator; Connect component 1; Connect component 2; Implement run; Train; Analyze; Define app placeholder :descriptions: First, import Lightning; This component trains a model on a GPU machine; This component analyzes a model on a CPU machine; Define the LightningFlow that orchestrates components; Component 1 will run on a CPU machine; Component 2 will run on an accelerated GPU machine; Describe the workflow in the run method; Training runs first and completes; Analyze runs after training completes; This allows the app to be runnable :code_files: ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py - :highlights: 2; 4-6; 8-10; 12; 15; 16; 18; 19; 20; 22 + :highlights: 2; 5-7; 9-11; 13; 16; 17; 19; 20; 21; 23 :app_id: abc123 :tab_rows: 4 - :height: 450px + :height: 460px | @@ -64,10 +67,10 @@ Without going out of your way, you're now doing the following: (Hint: Click **vi .. lit_tabs:: :titles: Orchestration; Distributed cloud computing; Multi-machine communication; Multi-machine communication; Multi-cloud; - :descriptions: Define orchestration in Python with full control-flow; The two pieces of independent Python code ran on separate machines 🤯🤯; The text "GPU machine 1" was sent from the flow machine to the machine running the TrainComponent; The text "CPU machine 2" was sent from the flow machine to the machine running the AnalyzeComponent; The full Lightning app can move across clusters and clouds + :descriptions: Define orchestration in Python with full control-flow; The two pieces of independent Python code ran on separate machines 🤯🤯; The text "CPU machine 1" was sent from the flow machine to the machine running the TrainComponent; The text "GPU machine 2" was sent from the flow machine to the machine running the AnalyzeComponent; The full Lightning app can move across clusters and clouds :code_files: ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./level_2_scripts/hello_app.py; ./hello_components/multi_cloud.bash :tab_rows: 4 - :highlights: 18-20; 15-16; 19; 20; 2 + :highlights: 19-21; 16-17; 20; 21; 2, 6, 10 :images: | | | | :height: 450px @@ -81,13 +84,30 @@ Although we've abstracted the infrastructure, you still have full control when y .. lit_tabs:: :titles: Scheduler; Crontab syntax; Auto-scaling; Organized Python; Full terraform control; :descriptions: Although you can use Python timers, we have a scheduler short-hand; You can also use full cron syntax; Code your own auto-scaling syntax (Lightning plays well with Kubernetes); *Remember* components organize ANY Python code which can even call external non-python scripts such as optimized C++ model servers ;Experts have the option to use terraform to configure Lightning clusters - :code_files: ./level_2_scripts/hello_app_scheduler.py; ./level_2_scripts/hello_app_cron.py; ./level_2_scripts/hello_app_auto_scale.py; ./level_2_scripts/organized_app_python.py; ./hello_components/terraform_example.bash; + :code_files: ./level_2_scripts/hello_app_scheduler.py; ./level_2_scripts/hello_app_cron.py; ./level_2_scripts/hello_app_auto_scale.py; ./level_2_scripts/organized_app_python.py; ./level_2_scripts/tr.bash :tab_rows: 4 - :highlights: 22, 23; 22, 23; 20, 23, 26, 27; 8-11, 15-17; 2 + :highlights: 24; 24; 21, 24, 27, 28; 9, 16, 17; 5 :height: 680px ---- +**************************************** +Use your own orchestrator (experts only) +**************************************** +If you have your own orchestrator, feel free to coordinate individual Lightning components with +those existing tools. + +The Lightning orchestrator described below is optimized for advanced patterns using regular python +control-flow and an embedded state system that is discussed in the advanced sections. + +To run the app from a CI/CD job, CLI or other orchestrators, run it without opening the UI (unless you need the UI) + +.. code:: bash + + lightning run app app.py --open-ui=false + +---- + *************************** Next step: Build a real app *************************** diff --git a/docs/source-app/levels/basic/create_a_model_demo.rst b/docs/source-app/levels/basic/create_a_model_demo.rst index 27460574b6..da6f28ca10 100644 --- a/docs/source-app/levels/basic/create_a_model_demo.rst +++ b/docs/source-app/levels/basic/create_a_model_demo.rst @@ -1,3 +1,5 @@ +:orphan: + ############################ Example: Create a model demo ############################ diff --git a/docs/source-app/levels/basic/deploy_ai_model_api.rst b/docs/source-app/levels/basic/deploy_ai_model_api.rst index d4eeae8715..dec9d4050c 100644 --- a/docs/source-app/levels/basic/deploy_ai_model_api.rst +++ b/docs/source-app/levels/basic/deploy_ai_model_api.rst @@ -1,3 +1,5 @@ +:orphan: + ########################### Example: Deploy a model API ########################### diff --git a/docs/source-app/levels/basic/hello_components/build_demo.py b/docs/source-app/levels/basic/hello_components/build_demo.py index 75c65bad90..62fbf467f9 100644 --- a/docs/source-app/levels/basic/hello_components/build_demo.py +++ b/docs/source-app/levels/basic/hello_components/build_demo.py @@ -1,15 +1,41 @@ -# A hello world component +# pip install streamlit omegaconf scipy +# pip install torch # app.py + import lightning as L +import torch +from io import BytesIO +from functools import partial +from scipy.io.wavfile import write +import streamlit as st -class YourComponent(L.LightningWork): - def run(self): - print('RUN ANY PYTHON CODE HERE') +class LitStreamlit(L.app.components.ServeStreamlit): + def build_model(self): + sample_rate = 48000 + model, _ = torch.hub.load( + repo_or_dir='snakers4/silero-models', + model='silero_tts', + speaker="v3_en", + ) + return partial( + model.apply_tts, + sample_rate=sample_rate, + speaker="en_0", + ), sample_rate -# run on a cloud machine -compute = L.CloudCompute("cpu") -worker = YourComponent(cloud_compute=compute) -app = L.LightningApp(worker) + def render(self): + st.title("Text To Speech") + text = st.text_input("Text:", "Lightning Apps are the best!") + + if text: + model, sample_rate = self.model + audio_numpy = model(text).numpy() + audio = BytesIO() + write(audio, sample_rate, audio_numpy) + audio.seek(0) + st.audio(audio) + +app = L.LightningApp(LitStreamlit()) diff --git a/docs/source-app/levels/basic/hello_components/train_pytorch.py b/docs/source-app/levels/basic/hello_components/train_pytorch.py index ff322d4dcc..fe33bd77d7 100644 --- a/docs/source-app/levels/basic/hello_components/train_pytorch.py +++ b/docs/source-app/levels/basic/hello_components/train_pytorch.py @@ -1,8 +1,6 @@ # app.py -import torch - import lightning as L - +import torch class PyTorchComponent(L.LightningWork): def run(self): diff --git a/docs/source-app/levels/basic/hello_components/xgboost.py b/docs/source-app/levels/basic/hello_components/xgboost.py new file mode 100644 index 0000000000..c91d9e1ed2 --- /dev/null +++ b/docs/source-app/levels/basic/hello_components/xgboost.py @@ -0,0 +1,22 @@ +# pip install sklearn xgboost +# app.py + +import lightning as L +from sklearn import datasets +from sklearn.model_selection import train_test_split +from xgboost import XGBClassifier + +class XGBoostComponent(L.LightningWork): + def run(self): + iris = datasets.load_iris() + X, y = iris.data, iris.target + + X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) + + bst = XGBClassifier(verbosity=3) + bst.fit(X_train, y_train) + preds = bst.predict(X_test) + print(f'preds: {preds}') + + +app = L.LightningApp(XGBoostComponent()) diff --git a/docs/source-app/levels/basic/hello_components/xgboost_gpu.py b/docs/source-app/levels/basic/hello_components/xgboost_gpu.py new file mode 100644 index 0000000000..e356d21a8d --- /dev/null +++ b/docs/source-app/levels/basic/hello_components/xgboost_gpu.py @@ -0,0 +1,22 @@ +# pip install sklearn xgboost +# conda install py-xgboost-gpu +# app.py +import lightning as L +from sklearn import datasets +from sklearn.model_selection import train_test_split +from xgboost import XGBClassifier + +class XGBoostComponent(L.LightningWork): + def run(self): + iris = datasets.load_iris() + X, y = iris.data, iris.target + + X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) + + bst = XGBClassifier(tree_method='gpu_hist', gpu_id=0, verbosity=3) + bst.fit(X_train, y_train) + preds = bst.predict(X_test) + print(f'preds: {preds}') + +compute = L.CloudCompute('gpu') +app = L.LightningApp(XGBoostComponent(cloud_compute=compute)) diff --git a/docs/source-app/levels/basic/index.rst b/docs/source-app/levels/basic/index.rst index dee1df7238..3ae0399b29 100644 --- a/docs/source-app/levels/basic/index.rst +++ b/docs/source-app/levels/basic/index.rst @@ -6,11 +6,6 @@ build_a_lightning_component connect_lightning_components - train_pytorch_on_the_cloud - deploy_ai_model_api - run_jupyter_notebook_on_the_cloud - create_a_model_demo - build_a_dag ############ Basic skills @@ -33,7 +28,7 @@ Every new user should start here. .. displayitem:: :header: Level 1: Package code in a Lightning component - :description: Learn to build a Lightning component which is the core building block of a full stack AI app (Lightning App). + :description: Learn to build a Lightning component which you can use in existing workflows or turn into a Lightning app. :button_link: build_a_lightning_component.html :col_css: col-md-6 :height: 170 diff --git a/docs/source-app/levels/basic/key_features/mount_data.py b/docs/source-app/levels/basic/key_features/mount_data.py index c376d104ae..1141983e52 100644 --- a/docs/source-app/levels/basic/key_features/mount_data.py +++ b/docs/source-app/levels/basic/key_features/mount_data.py @@ -1,6 +1,5 @@ -import os - import lightning as L +import os class YourComponent(L.LightningWork): diff --git a/docs/source-app/levels/basic/level_2_scripts/hello_app.py b/docs/source-app/levels/basic/level_2_scripts/hello_app.py index bb5e2d668f..d6ce3ea222 100644 --- a/docs/source-app/levels/basic/level_2_scripts/hello_app.py +++ b/docs/source-app/levels/basic/level_2_scripts/hello_app.py @@ -17,7 +17,7 @@ class WorkflowOrchestrator(L.LightningFlow): self.analyze = AnalyzeComponent(cloud_compute=L.CloudCompute('gpu')) def run(self): - self.train.run("GPU machine 1") - self.analyze.run("CPU machine 2") + self.train.run("CPU machine 1") + self.analyze.run("GPU machine 2") app = L.LightningApp(WorkflowOrchestrator()) diff --git a/docs/source-app/levels/basic/level_2_scripts/organized_app_python.py b/docs/source-app/levels/basic/level_2_scripts/organized_app_python.py index c34a5deb55..8b174aded9 100644 --- a/docs/source-app/levels/basic/level_2_scripts/organized_app_python.py +++ b/docs/source-app/levels/basic/level_2_scripts/organized_app_python.py @@ -1,6 +1,5 @@ # app.py import subprocess - import lightning as L diff --git a/docs/source-app/levels/basic/level_2_scripts/tr.bash b/docs/source-app/levels/basic/level_2_scripts/tr.bash new file mode 100644 index 0000000000..30c8d19407 --- /dev/null +++ b/docs/source-app/levels/basic/level_2_scripts/tr.bash @@ -0,0 +1,5 @@ +# custom control for optimized clusters with tools like terraform +# are only supported on the enterprise tier (support@lightning.ai) + +# once the cluster is created you can run any app on it +lightning run app app.py --cloud my-custom-optimized-cluster diff --git a/docs/source-app/levels/basic/run_jupyter_notebook_on_the_cloud.rst b/docs/source-app/levels/basic/run_jupyter_notebook_on_the_cloud.rst index 825175516d..b39dfa107d 100644 --- a/docs/source-app/levels/basic/run_jupyter_notebook_on_the_cloud.rst +++ b/docs/source-app/levels/basic/run_jupyter_notebook_on_the_cloud.rst @@ -1,3 +1,5 @@ +:orphan: + ############################################# Example: Develop a Jupyter Notebook component ############################################# diff --git a/docs/source-app/levels/basic/train_pytorch_on_the_cloud.rst b/docs/source-app/levels/basic/train_pytorch_on_the_cloud.rst index ea257eb9b0..c3ef841a91 100644 --- a/docs/source-app/levels/basic/train_pytorch_on_the_cloud.rst +++ b/docs/source-app/levels/basic/train_pytorch_on_the_cloud.rst @@ -1,3 +1,5 @@ +:orphan: + ################################### Example: Train PyTorch on the cloud ################################### diff --git a/docs/source-app/quickstart.rst b/docs/source-app/quickstart.rst index e94dbb3636..96c9eeee79 100644 --- a/docs/source-app/quickstart.rst +++ b/docs/source-app/quickstart.rst @@ -16,7 +16,7 @@ and uses `Gradio `_ to serve it. Step 1 - Installation ********************** -First, you'll need to install Lightning. You can find the complete guide here: :ref:`install`. +First, you'll need to install Lightning. You can find the complete guide here. Then, you'll need to install the `Lightning Quick Start package `_.