2.5 KiB
Build a Lightning Hyperparameter Optimization (HPO) App
A bit of background
Traditionally, developing machine learning (ML) products requires choosing among a large space of hyperparameters while creating and training the ML models. Hyperparameter optimization (HPO) aims to find a well-performing hyperparameter configuration for a given ML model on a dataset at hand, including the ML model, its hyperparameters, and other data processing steps.
HPOs free the human expert from a tedious and error-prone, manual hyperparameter tuning process.
As an example, in the famous scikit-learn library,
hyperparameters are passed as arguments to the constructor of
the estimator classes such as C kernel for
Support Vector Classifier, etc.
It is possible and recommended to search the hyperparameter space for the best validation score.
An HPO search consists of:
- an objective method
- a defined parameter space
- a method for searching or sampling candidates
A naive method for sampling candidates is grid search, which exhaustively considers all hyperparameter combinations from a user-specified grid.
Fortunately, HPO is an active area of research, and many methods have been developed to optimize the time required to get strong candidates.
In the following tutorial, you will learn how to use Lightning together with Optuna.
Optuna is an open source HPO framework to automate hyperparameter search. Out-of-the-box, it provides efficient algorithms to search large spaces and prune unpromising trials for faster results.
First, you will learn about the best practices on how to implement HPO without the Lightning Framework. Secondly, we will dive into a working HPO application with Lightning, and finally create a neat HiPlot UI for our application.
Getting started
Step 1: Download the data
python download_data.py
Step 2: Run the HPO Lightning App without an UI
lightning run app app_wo_ui.py
Step 3: Run the HPO Lightning App with HiPlot UI in Streamlit.
lightning run app app_wi_ui.py
Learn More
In the documentation, search for Build a Sweep App.