VSA Partners is a full-service advertising and design agency founded over 30 years ago. Working with some of the most recognizable brands in the world such as Kleenex, Alibaba and GE, their mission is to “deliver a truly unified brand experience across touch-points”.
Being an innovative design firm, VSA understood that having a data-driven approach would give them an edge in competing for business from the world’s top advertisers. Their pitch to one such advertiser was to develop a Bayesian Network that would approximate revenue generated, based allocating spend across different channels such as print, television, and online at varying degrees of specificity. Having a trustworthy simulation of revenue based on how to allocate spend was an optimization problem that VSA was able to handle on their own.
However, according to Dr. Charles Hammerslough, Director of Data Science at VSA Partners, “Optimizing revenue was a different story. When there are dozens of combinations of channel, offer, and product across which to allocate an advertising budget, we knew that this multidimensional optimization problem was going to be too complicated for conventional approaches.” Given
n spending channels, and a budget
B, the goal is to find spending allocations for each channel that sum to less than
B. VSA’s model, or collection of models, that simulate revenue based on this spending allocation. That is, their model returns revenue as the output of a complex Bayesian Network. Since the parameter space is extremely large, an exhaustive grid search is not feasible.
After investing time and money into developing an in-house genetic algorithm that failed to reliably converge to an optimal value, VSA started seeking solutions to their optimization problem – which led them to SigOpt. After a thorough evaluation, VSA “chose SigOpt because it is a state-of-the-art high powered OaaS solution that was readily available to our various programming toolkits: it integrated seamlessly into our application” said Dr. Hammerslough.
VSA provides SigOpt with information about the configuration parameters in their system. SigOpt then suggests specific configurations for VSA to evaluate, within their AWS infrastructure, and VSA reports back how well the configurations performed for their objective metric (expected revenue). SigOpt learns from this information to suggest the next best configurations to attempt. This process is repeated as SigOpt converges to the best configurations that maximize the objective metric of the system.
VSA quickly found value with SigOpt. Integration was a fairly trivial coding exercise using SigOpt’s open source Java client library. Because SigOpt automates the optimization problem, this frees up VSA’s data science team to focus on their core business: developing and providing innovative solutions for their clients.
Some additional advantages of SigOpt that VSA noted were that because SigOpt is a black box optimization service it doesn’t need to access the underlying data, model, or validation strategy to perform this optimization. SigOpt’s optimization engine needs only the configuration parameters, and how well they have performed. This protects both the PII in VSA’s data, and the IP of the models they have developed. As well, because SigOpt supports features such as the ability to pre-define constraints on parameters, VSA could ensure that SigOpt was only considering configurations that would stay under budget, while also having the freedom explore a large range of possible values for each channel.
SigOpt relies on a number of AWS services to make it possible to support our customers, like VSA. SigOpt uses EC2 compute for our customers’ experiments, VPCs to secure any sensitive customer data, and RDS for long term storage of results so that customers can analyze results and maintain the history of our optimization experiments. Perhaps most importantly, AWS provides SigOpt the ability to scale these resources up and down as our customer base grows, and customer usage changes, allowing SigOpt to maximize the efficiency of our infrastructure spend.
VSA has been getting value from SigOpt since early 2017. They optimize the advertising campaigns for their customers and are able to save time in finding optimal spending allocations across complete advertising campaigns.