How do leading researchers ask the right questions during experiment design?

Luis Bermudez
AI at Scale, Artificial Intelligence, Bayesian Optimization, Classification, Deep Learning, Experiment Management, Graph Neural Networks, Hyperparameter Optimization, Intelligent Experimentation, Machine Learning, Natural Language, Prediction, Time Series, Vision

SigOpt is hosting our first user conference, the SigOpt AI & HPC Summit, on Tuesday, November 16, 2021. It will showcase the great work of our customers from a variety of industries with a diverse set of use cases. It’s virtual and free to attend, so sign up today at To give you a sense of the Summit, we will publish a series of blog posts in advance of the event. The prior post focused on the overview of the upcoming SigOpt Summit. This post focuses on the best ways to design experiments with SigOpt.

Experimentation is critical to modeling, but can be messy and tough to get satisfying results. We believe that the right combination of tools and techniques can empower modelers with a more intelligent approach to experimentation. And with intelligent experimentation, modelers are empowered to ask the right questions during experiment design, explore their modeling problems more deeply, and optimize their models to meet their particular objective metrics within their constraints.

Of these components – design, explore, optimize – that are critical to intelligent experimentation, this post will focus on Design. Designing an experiment involves applying a scientific process. The first step in a scientific process is having an initial question that you want to solve, such as “When is the best time of day to buy or sell?” From there, SigOpt’s Intelligent Experimentation platform helps researchers form the right questions to design experiments with their needs in mind. For instance, SigOpt helps researchers better understand what optimization targets and constraints are important to them. Researchers then ask themselves what the different knobs and parameters are that they’re willing to tune within their experiment design. To aid in this process, SigOpt offers tools to help researchers consider their preferred range or step size in their parameter space. At this point, the platform asks them how many runs they’re willing to compute. From there, researchers can ask themselves how they would like to weigh the importance of experiment speed versus accuracy. SigOpt tooling helps researchers answer questions about speed with the ability to parallelize experiments. These are the types of questions that SigOpt helps researchers consider when they’re designing their experiments. This process enables them to ask the right questions for their research problems right from the start.

Experiment design will be a theme at our upcoming Summit. At the beginning of the summit, Scott Clark (SigOpt’s General Manager) will be discussing how to design experiments along with the help of frameworks and design tools. Later in the afternoon, Michael McCourt (SigOpt’s Head of Engineering) will be hosting a panel discussion on how experiment design plays a crucial part in intelligent experimentation for industry leaders and academics alike. The panel includes one of our speakers from the University of Pittsburgh, who will be discussing how they used SigOpt to formulate intelligent questions for their nanostructured surface research experiments.

If you want to get a better sense of how SigOpt could impact your workflow than by simply reading about use cases, sign up in seconds at If you want to learn from our customers, sign up for the SigOpt Summit for free at Attendees will be able to join the talks and panels, meet with speakers in breakout rooms for deeper discussions and network with each other. Look for future posts that focus more deeply on the themes that will cut across the talks and panels at the Summit. We look forward to seeing you there!

Luis Bermudez AI Developer Advocate