Deep Learning

SigOpt Use Cases in Deep Learning

SigOpt’s hyperparameter optimization solution is used by leading companies to tune any deep learning model. Model sensitivity to hyperparameter selection and the complexity of networks both contribute to the hyperparameter optimization problem in deep learning.

Most of our customers have rather sensitive use cases for deep learning and, to protect their privacy, we do not share these examples publicly. But to demonstrate the potential of our product in deep learning, we do work with our partners to publish technical use cases. Below is a subset of these use cases in deep learning. We hope you enjoy. Visit our Try It page to find time to meet with our team to learn more.

 

Image Classification on a Budget with Hyperparameter Optimization

Learn more: https://bit.ly/img-class 

Image processing and classification tasks have now become critical to most modeling-driven companies, regardless of their industry. But the time intensive nature of optimizing complex models with particularly expensive image datasets can still be a barrier to these teams. Some teams overcome this barrier by avoiding optimizing altogether, choosing to use transfer learning with relatively complex networks like ResNet 50. Others choose the opposite route by fully optimizing cheaper models like ResNet 18. In this post, we explore and evaluate these tradeoffs in the context of an image classification task for the Stanford Cars dataset.

 

Effective and Efficient Optimization of Memory Networks

Learn more: https://bit.ly/2NifgGE

Question answering (QA) systems are increasingly important for natural language processing (NLP) tasks that largely determine the performance of voice assistants and other increasingly ubiquitous products. Model accuracy is paramount to the success of these products in the market and deep learning is often applied to improve these models. In this research, applying hyperparameter tuning to End-to-End Memory Networks on the bAbI dataset leads to meaningful improvement over standard methods in model accuracy across all twenty benchmark tasks. Importantly, we learn that advanced tuning methods like multitask optimization more cost-efficiently drive even greater performance gains.

 

Balancing Speed and Accuracy in Sequence Classification

Learn more: https://bit.ly/amplifytuning

Applying deep learning to novel real-world use cases often entails making trade-offs between multiple metrics rather than optimizing for a single one. This research uses convolutional neural networks (CNNs) in two different tasks: sequence classification and sentiment analysis. In these cases, this research shows that using novel multimetric optimization strategies can empower the researcher to efficiently trade off speed and accuracy to uncover the right optimal configuration of the CNN for their particular use case. Importantly, the use of advanced Bayesian optimization algorithms leads to uncovering optimal solutions in a fraction of the model training runs that would be required if using more traditional methods like random search.

 

400x Acceleration for Deep Learning Optimization

Learn more: https://amzn.to/2PgutIl

For particularly expensive models that have lengthy training cycles, it is critical to accelerate the hyperparameter optimization process. In this case, SigOpt worked with its partners Nvidia and Amazon to demonstrate that combining best-in-class hardware and software can lead to multiplicative time savings. This research applies a neural network to sentiment tasks using the Rotten Tomatoes dataset of labeled movie reviews. When compared to more traditional compute, GPUs drive a 40x speed up. And when compared to more traditional tuning methods, SigOpt’s proprietary optimization algorithms drive a 10x speed-up. Together, this results in a 400x acceleration in the model optimization process, while also producing a higher-performing configuration of the model than standard methods for this particular task.

 


 

If you’re interested in learning more or exploring the use of our enterprise solution, contact our team by filling out the form below.

  • How important is it to have the best version of a model?
  • How many models do you have in production today, and/or expect to have in production in 6 months?
  • How many people and/or teams in your org. are working on model development?
  • What kinds of models/libraries are your team(s) working with?
  • On what cadence do you generally develop new, or update existing, production models?
  • On what timescale does your data change?
  • How often do you optimize your models?
  • How do you currently optimize your models?
  • When evaluating new, or comparing new to old, models, how important is it to have a consistent baseline, and confidence in that baseline?
  • Which of these pains are you feeling due to your current optimization method?
  • What kind of evaluation infrastructure do you have?