Editor's Note: Joseph Britto, MD, is CEO and Co-founder of Isabel Healthcare Inc. Isabel, a clinical decision support system, was founded in 1999 by Britto and Jason and Charlotte Maude, whose daughter Isabel was harmed by a medical error. The company has been profiled in the Wall Street Journal, and the system has undergone several validation studies. We asked Dr. Britto to talk with us about eradicating diagnosis errors through diagnosis decision support systems.

Dr. Robert Wachter, Editor, AHRQ WebM&M: After some early hope that it would be pretty easy to enhance physician diagnostic decision-making through information technology in the 1970s and 80s, these efforts appeared to hit a wall. What gave you the feeling that the time was right to approach this again?

Dr. Joseph Britto: The essential innovation was natural language processing, which is basically the ability of software to "make sense" or "read" documents. Natural language software uses statistical inference to look at the frequency, occurrence, and relationship between words in a document. In a sense, natural language processing is able to create a "kernel" of knowledge—exactly what you and I would do if we were to read a document. In 1999, natural language processing software had just entered the scene and had great promise. The challenge was to try to apply it to medicine. That was one difference. Another difference was that the prior systems, like DXplain, QMR, and Iliad, were "expert systems" and produced a rank-ordered list of diagnoses—1, 2, 3. The real world of medicine doesn't fall neatly into lists like that. Previous generation systems were rules-based systems and therefore were not very easy to use, often taking 20–40 minutes for data to be inputted. I tell colleagues that Isabel is a diagnosis reminder system and a knowledge mobilizing system. It's meant to augment, complement, and enhance the physician's own knowledge and cognitive skills. The essential difference is that it treats the physician as the "learned intermediary" (to borrow a phrase from the British informaticist and physician Jeremy Wyatt).

RW: What is happening under the hood of a system like this? What actually goes on when I enter the common signs and symptoms?

JB: Isabel uses natural language processing software to search thousands of pages of established, widely read textbooks and journals that sit on our shelves and that we read. There isn't a corpus of medical knowledge or rules that I, or anyone on the team, has actually sat down and written. Isabel does in a split second and at the point-of-care what we frequently do when faced with a diagnostic conundrum—we head to our offices, flip through our textbooks to jog our memory, and create a checklist of likely diagnoses to consider.

RW: So I enter "chest pain radiating to the back, shortness of breath, and fever." How many books is it searching? Is it trying to see which articles or which chapters in books have the most frequent occurrence of all of those words, or is there some weighting of words? How is that actually playing out?

JB: We had to first create a simple hierarchical taxonomy. For example, toxic shock syndrome would be a diagnosis node under streptococcal or staphylococcal infections, which would be a diagnosis node under gram-positive infections, that being under bacterial infections. So a simple, hierarchical taxonomy, but one that now contains 10,000 diseases or conditions and another 4,000 drugs. We've "tutored" this taxonomy using documents, and therefore this node on toxic shock syndrome is aware of the clinical features, diagnosis, investigation, and treatment. So when you enter a set of symptoms or signs, our search engine works off a very comprehensive database that has now close to 100,000 documents hanging off a taxonomy. We also had to add two more relatively minor components. One was a synonym and exact phrase file, so that fever, pyrexia, and febrile are searched as the same concept. We also had to put in heuristics that take age, gender, pregnancy, and geographical region into account when producing a differential diagnosis. For example, in North America, "fever and rash" should bring up Rocky Mountain spotted fever, toxic shock syndrome, and Kawasaki disease. In the UK, meningococcal disease would be much higher up on the differential diagnosis list and, in India, measles.

To get back to your question, we take your inputted clinical features and search in a split second across 100,000 documents. This kind of software—also known as pattern recognition software—searches all the documents looking to match the terms in the query: chest pain, shortness of breath, and fever. And we return the likely diagnoses, which you can click on and then read about—diagnosis-specific knowledge on how does one investigate, how does one treat, what are the lessons learned and recent advances.

The purpose of Isabel is not just to give a checklist of likely diagnoses to consider, but also to help corroborate your decision-making. For example, I have a patient with Kawasaki disease and I want to know: Does hyperalbuminemia occur in Kawasaki disease? Wyatt, again, showed how three questions arise in almost every clinical encounter—and sadly, two out of these three remain unanswered, sometimes to the detriment of our patients. The challenge is to develop efficient, useful ways to provide clinicians with corroborative knowledge at the point of decision-making to help improve the quality of care.

RW: Now when you were talking about what was different in 1999 than 1980, most of your answer emphasized the improvements in technology. What about physicians? Are they different—perhaps more predisposed to accept this sort of support?

JB: The short answer is yes. Physicians are more amenable to diagnosis assistance, not just because they're aware of error, but because expectations have changed. Not just has the paradigm shifted in terms of us becoming more cognizant that we make mistakes, but that these mistakes are not usually (as Mark Graber says) knowledge-based mistakes, but often involve cognitive and metacognitive factors. The planets are aligned also because the technology has changed. We now have tremendous computational power. We now have natural language processing. And of course there is increasing pressure on the system to get it right, to not make so many mistakes.

Ultimately, the biggest change that will push this paradigm even further is electronic medical records, the digitalization of health care. Because of the digitalization of patient records, it's becoming much easier, much more intuitive to interface or embed diagnosis decision support systems. Physicians in the near future will not have to type in "chest pain, fever, shortness of breath," but these data will be automatically extracted from the electronic medical record. From the physician's perspective, diagnosis decision support will sit on their dashboard, 24/7, 365. You know, I made my way over here today using a GPS system. I find that's a useful analogy when talking of diagnosis decision support systems, because the GPS system doesn't drive the car. You, as the driver, still have full control. Diagnosis decision support systems will never replace good, conscientious clinicians at the bedside. At best, they will augment, enhance, and complement our decision-making. We still have complete control at the bedside.

RW: How good are physicians at identifying their own limitations, and from your experience, if a system like yours is easily available, how often will a doctor take the time and trouble to go and use it? What factors predict whether people take advantage of this kind of support?

JB: Chuck Friedman found that physicians, medical students, and residents were confident about a diagnosis in a significant number of patients in whom they were actually wrong. Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36%, and students in 25%. For me, the message from Friedman's work is that if you're going to rely on the confidence of a physician in a particular diagnosis to go and use a diagnosis decision support system, then we won't be getting it right as often as we should. What we should do is decrease the threshold for physicians to use systems like this. If you have a standalone system that you have to go in and actually type in fever, headache, vomiting, neck stiffness, it's much less likely to be used. Whereas, if you stick it on someone's dashboard or interface it with EMR, then it's much more likely to be used. My hypothesis is that one way to manage the transition to electronic medical records is to incentivize physicians and say, help us move to electronic patient records, and in return we can help you in your cognitive processes by giving you diagnosis decision support systems and prescription support systems. Diagnosis decision support actually returns something back to the physician. Because as you know, a diagnosis is the starting point. It's crucial. It predicates what tests you order, what investigations you perform, what treatments you start. Diagnosis decision-making has been the sacrosanct, protected area of a physician, and if we can help physicians come up with the right differential diagnosis, we can bring about change much more easily.

RW: You're very careful to make clear that this is a support for physicians. It doesn't strip away clinical judgment. Physician autonomy is preserved. Yet the pressures in the market around pay for performance and transparency may conspire to begin stripping away physician autonomy for much of what we do. Some people have looked at physician autonomy and said, "Why should autonomy be preserved when we get it wrong so much of the time?" How do you see that balance playing out? Can you envision a day when these systems actually do begin to replace physicians because they'll be better than physicians working without such systems?

JB: I personally don't think that diagnosis decision support systems will ever replace physicians. Simply because the first stage in any diagnosis process is taking a good history. Yes, we might use a set of digital questionnaires to help us ask all the right questions, but finally we have to be switched on, thorough, and conscientious and take a good history—the fact that the patient has traveled back from Mexico, that there's a cat in the house. We also have to perform thorough, conscientious examinations. So the first step of taking good histories and making thorough examinations is something physicians will have to continue doing and doing well. Remember with any system, if you put rubbish into it—for example, if you put the wrong numbers into a calculator—your answers are going to be wrong. Diagnosis decision support systems actually call upon us to be better physicians. And if you are a good, thorough, conscientious physician, then systems like this will help you become an even better physician. It's that second stage, when we say—radiating pains from the neck, sweats, fever, and leukocytosis, where have I seen this before? It's that stage where I think systems like ours can alleviate the burden of memory, if you like. Where we can very quickly search through books and give physicians a list of likely suspects to consider. That's where technology will have a role. But I don't think a system like this will ever replace physicians. Just as much as very clever, sophisticated calculators haven't replaced mathematicians, or very sophisticated word processing software hasn't made all of us into expert authors.

I think if you asked residents and young attendings what their view is—they're much more digitally native than we are. I hope you don't mind me saying that I think both of us [middle-aged physicians] are digital immigrants. We grew up with books on our shelves. Residents now are not so obsessed about which textbook or publisher it is. If a digital resource answers questions more quickly in their workflow, they're much more interested in that. If you ask them a question on their ward rounds tomorrow, they're much more likely to go first to the Internet and then perhaps to books. Part of the response of physicians to technology and decision support systems is predicated by which generation one comes from.

RW: So I go onto Amazon and it remembers me, and it says, customers like you seem to like the following books and it pushes them in my direction. I guess I can envision as we all get wired, somehow in the future, I put in a certain set of signs and symptoms and lab studies and it's not looking through textbook chapters to figure out where those things have occurred, but it's actually looking at huge databases of patients who have had or have not had those signs and symptoms and lab abnormalities and it actually knows what diagnosis they ultimately ended up having. Is that the direction you think this is going? Outcomes-based connections rather than knowledge- and textbook-based connections?

JB: That is an incredible question and in fact it's around the corner. We've shown that you can apply natural language processing software to textbooks and journals. But you can also apply it to medical records. And here is one of the huge—no longer putative, but real—advantages of electronic medical records, in that you'll be able to use software to perform what you described as real-time epidemiology. Rather than looking at the prior probability of conditions within the pages of a textbook, it would actually look at the electronic medical records of an organization such as yours at UCSF, look back over the last 1–3 years and actually modify the results of the diagnosis support system taking cognizance of local epidemiology. That's the beauty of digitalizing everything in that you will be able to skew the output based upon the local epidemiology. And in a sense it can learn. For example, if you use GPS to go from point A to point B and then you find that there is a better route, you can actually skew your system next time to use that as a favored route. Similarly, when you have a natural language processing system hovering behind the scenes, 24/7, 365, of the electronic medical records of all of San Francisco, for example, then you start getting very interesting, useful data that can then skew the results and make our jobs much easier. To try to answer the crucial question that we have when we use a diagnosis decision support system—what does my patient have? Don't just give me a list of likely suspects, or a checklist, but what is the most likely diagnosis in my patient? And we can then use real-time epidemiology to help answer that question.