Last week we touched briefly on what the Hyperopt framework is and some reasons you may choose to experiment with it as an optimizer. This week, we are excited to  announce a new SigOpt integration that allows you to combine Hyperopt’s HPO flexibility with SigOpt’s experiment management platform. For those of you already familiar with the Hyperopt framework, the sigopt.hyperopt  integration provides a seamless way to automatically log a Hyperopt workflow using the SigOpt API .  

Below, we will explain how to  leverage the SigOpt platform to run a standard Hyperopt workflow. 

How does sigopt.hyperopt work?  

Optimization with HyperOpt works by calling the hyperopt.fmin function, where users specify the optimization task. You then pass an object called hyperopt.trials to track the results of parameter configurations and its scores as measured by the objective function. The sigopt.hyperopt integration provides several utilities that make it easy to log all trials from an optimization task as a Run in a Project: 

• in a batch: a single API request that logs each trial   
• in real time: an API request is made at each call to the Hyperopt objective function   
• to wrap and report results from hyperopt trials run in parallel using MongoDB or Spark   

Take a look at the examples below! 

Batch Upload Trials 

import time   
from hyperopt import fmin, tpe, hp, STATUS_OK, Trials   
from sigopt.hyperopt import upload_trials   
def objective(args):   
  x, = args   
  return {   
    "loss": x ** 2,   
    "status": STATUS_OK,   
    "eval_time": time.time(),   
    "other_stuff": {"type": None, "value": [0, 1, 2]},   
  }   
trials = Trials()   
space = (hp.uniform("x", -10, 10),)   
best = fmin(objective, space=space, algo=tpe.suggest, max_evals=1, trials=trials)   
upload_trials(project="project-name", trials=trials)

Realtime Upload Trials 

import time  
from hyperopt import fmin, tpe, hp, STATUS_OK, Trials  
from sigopt.hyperopt import SigOptTrials  
def objective(args):  
  x, = args  
  return {  
    "loss": x ** 2,  
    "status": STATUS_OK,  
    "eval_time": time.time(),  
    "other_stuff": {"type": None, "value": [0, 1, 2]},  
  }  
space = (hp.uniform("x", -10, 10),)  
trials = SigOptTrials(project="project-name")  
best = fmin(objective, space=space, algo=tpe.suggest, max_evals=1, trials=trials)

Parallel Trials 

from math import sin  
from hyperopt import fmin, tpe, hp  
from hyperopt.mongoexp import MongoTrials  
trials = MongoTrials('mongo://localhost:1234/foo_db/jobs', exp_key='exp1')  
space= hp.uniform('x', -2, 2)  
best = fmin(sin, space, trials=trials, algo=tpe.suggest, max_evals=10)   
trials = SigOptTrials(project="project-name", trials=trials)

 Check out our official documentation here for further details! 

Looking forward 

Here at SigOpt we’re always looking to widen our optimization potential and improve your experimentation experience. In the process of  creating a more complete platform for your customization needs, we plan to  continue pursuing integrations with other popular tools and optimization methods in the future.  We always enjoy hearing your thoughts, so if you have feedback or suggestions, please voice them on our community page or shoot us a message at [email protected].   

Don’t have a SigOpt account yet? Click here to sign up for free!  

Happy modeling!