At SigOpt, we are thrilled to collaborate with the outstanding community of experts from around the world. In this post, we invited Professor Dennis Barbour, MD, Ph.D., from Washington University in St. Louis to share some perspective of his work on advancing medical diagnostics for disorders of brain and behavior. His most recent work, “Accelerating Psychometric Screening Tests With Bayesian Active Differential Selection”, is a collaboration with Trevor J. Larsen and our research engineer Gustavo Malkomes.

Ancient Greek physicians on the island of Knidos introduced a very modern concept for evaluating patients: the diagnosis. Until that point, physicians tended to treat symptoms (“features”) of disease rather than their mechanistic origins (“labels”). The process of labeling a cluster of symptoms into a disease category before deciding on a treatment plan has been the cornerstone of medicine ever since.

Medical diagnosis can therefore be thought of as a classification problem. The difficult part of this process is assigning a disease class to a patient, after which the best treatment for past patients falling into the same class would be applied to the current patient. Machine learning approaches are continually improving state of the art under this framework.

A key limitation here comes by trying to define “similar patient.” The curse of dimensionality guarantees that as more symptom features are included in our models, defining similar patients becomes more challenging. This problem is exacerbated for rare diseases where few labels (i.e., diagnosed patients) are available for training.

A paper we recently presented at the NeurIPS conference demonstrates how to overcome this barrier by applying Bayesian optimization in a novel way. We initialize the diagnostic workup for an individual patient with probabilistic models reflecting canonical disease states. These models become priors for an active model search procedure either 1) to answer the question of which existing model best matches a current patient (active model selection), or 2) to update an existing model for better predictions of yet-to-be-observed symptoms in the patient at hand (active model estimation).

The resulting procedure is a form of active transfer learning, and by explicitly referencing data from previous patients and/or previous data from the current patient, we can dramatically reduce the time required to realize a model for the current patient. We call this type of procedure active differential inference (ADI).

Figure 1: Illustration (or diagram) of different types of Inference reasoning. In active differential inference we propose to actively refine our set of hypothesis as we gather more observations.

Our first test case of ADI involves estimating behavioral models, which are costly to construct because patients can only be asked one question at a time (e.g., Did you see that flash of light? Or hear that burst of sound?). A hearing test called the audiogram would require on the order of 1000 test tone queries to build a high-resolution model of an individual patient’s hearing profile with conventional methods, which cannot incorporate prior beliefs in a meaningful way.

We first used a database of over 2 million audiograms to construct 7 canonical probabilistic models encompassing the full range of hearing loss.  These profiles are displayed in the figure below.

Figure 2: 7 classifications of hearing capability: the x-axis measures the frequency of the sound, the y-axis measures the inverse intensity.  The yellow/black regions denote test conditions where patients are likely/unlikely to recognize a test sound.

Then we modeled the ground truth of simulated patients in a similar way. To perform the diagnosis, new tone queries were actively selected to best determine whether a simulated patient (“new”) matched a given diagnostic model (“old”). This decision could be made at a Bayes factor of 100 with fewer than 10 tone queries on average in most cases (see the table below).

Table 1: Number of iterations to achieve Bayes Factor equal or above 100 using ADI. For each pair of old and new conditions, we performed 10 experiments: average and standard deviation are reported.

Further refinement of an individual patient’s audiogram at that point could proceed by applying the selected diagnostic model as a prior and actively learning which tone queries were best for refining a probabilistic model specific to that patient.  Learn more about active learning here, here, here or here.

ADI represents a natural extension of current and emerging medical diagnostic procedures. By developing probabilistic models of symptoms, we anticipate making informed predictions about disease progress as well as remission stemming from treatment. Because ADI does not fully rely on identifying other similar patients—other than to establish priors useful for shortening the search procedure—it should be applicable in cases where disease mechanism is poorly understood or not enough patients exist to pursue a standard Big Data approach.

Use SigOpt free. Sign up today.