ICYMI Recap: Warm Start Tuning with Prior Beliefs

Barrett Williams and Michael McCourt
Advanced Optimization Techniques, Applied, Augmented ML Workflow, Focus Area, Methodology, Modeling Best Practices

SigOpt has expanded to include a number of advanced features to help you save time while increasing model performance through experimentation. But what about past experience with similar models? What are the benefits of using previous information to arrive at a better model faster, versus the risks of biasing your tuning process to inadvertently avoid a globally optimum parameter set? SigOpt Research Engineer Michael McCourt walks through the following:

  • How to convey prior beliefs about parameter performance to SigOpt
  • What are the benefits and risks of introducing prior beliefs to your experiment

And here is a more specific summary of the webinar. Click through to view any segment you missed:

  • Agenda Overview (2:12)
  • Your models and data stay private (5:18)
  • What if users want to provide additional information about a past model? (6:29)
  • Users can now provide data from past successes, for example learning rate (7:20)
  • The goal of prior beliefs is to convey information about these successes to SigOpt (8:41)
  • SigOpt API provides a very specific structure, so Prior Beliefs needs one too: a probability density function (9:48)
  • Prior Beliefs are intentionally less relevant as more information about the current experiment becomes available through experimentation (11:02)
  • SigOpt allows Normal and Beta distributions (LaPlace to come) (12:11)
  • SigOpt offers an experimentation tool to try different parameters and visualize the prior (13:39)
  • Prior Beliefs enables user-assisted optimization for the first time (15:43)
  • Providing prior beliefs can have a very good or very bad impact on your modeling outcome (16:19)
  • An example of “catastrophically” bad prior beliefs (17:43)
  • Good prior beliefs accelerates optimization up front, while bad prior beliefs can drastically delay the discovery of an optimal solution (18:52)
  • Freedom comes with responsibility (20:21)

For more information, please check out our documentation, where you can find our experimentation section, and our blog post announcing the feature. If you joined us live, thank you for taking the time to learn why we implemented prior beliefs, and how it can accelerate your experimentation process. If you’d like to watch the recording, you can find it here. If you’re interested in learning more, follow our blog or try our product. 

Barrett-Williams
Barrett Williams Product Marketing Lead
MichaelMcCourt
Michael McCourt Research Engineer

Want more content from SigOpt? Sign up now.