Improving Diagnostic Safety and Quality
Al-Khafaji J, Lee M, Mossburg S. Improving Diagnostic Safety and Quality . PSNet [internet]. Rockville (MD): Agency for Healthcare Research and Quality, US Department of Health and Human Services. 2023.
Al-Khafaji J, Lee M, Mossburg S. Improving Diagnostic Safety and Quality . PSNet [internet]. Rockville (MD): Agency for Healthcare Research and Quality, US Department of Health and Human Services. 2023.
Introduction
During an annual editorial review of featured articles in the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Network (PSNet) collection, diagnostic safety was noted as a frequent topic for 2022. In consultation with the PSNet Technical Expert Panel, this topic was selected for a Year in Review perspective. This perspective includes the contribution of Dr. Jawad Al-Khafaji, who is a clinical assistant professor at the University of Michigan and a subject matter expert in diagnostic safety. Perspective authors reviewed articles related to clinical misdiagnosis and patient safety added to the AHRQ PSNet Collection in 2022. Key findings and themes are highlighted below.
Across clinical settings and medical specialties, the diagnostic process requires a complex set of activities to gather, integrate, and interpret information, and as a result, could lead to a missed, delayed or incorrect diagnosis. Correct diagnosis may depend on a disparate range of clinical skills: communicative aptitude in obtaining patient histories, observational skill in physical examination, and data interpretation in diagnostic testing, among others.1 Diagnostic error, as defined in the National Academy of Sciences Engineering and Medicine’s (NASEM’s) seminal report on improving diagnostic safety, is “the failure to (a) establish an accurate and timely explanation of the patient’s health problem(s) or (b) communicate that explanation to the patient.”2 While the NASEM definition describes diagnostic error, it does not address multifactorial causes of diagnostic error, which include individual patient and clinician, organizational, and system factors. Recognition of the multifactorial contributors to diagnostic safety spans back almost 20 years with Dr. Mark L. Graber’s pioneering work on diagnostic error in 2005, which includes an identification of systemic problems with policy, procedures, and inefficient processes.3 The incidence of diagnostic error is not well established. Expert opinion suggests that misdiagnosis may occur in 10–15% of all diagnoses, but literature reviews show varying rates depending on disease and symptom presentation, from 2.2% for myocardial infarction to 62.1% for spinal abscess.4 In the Neonatal Intensive Care Unit, a critical care setting in which error may lead more frequently to mortality, misdiagnosis has been estimated at 6.2% within the first seven days of admission. More broadly, in pediatric hospitals misdiagnosis rates vary but there is significant heterogeneity in data gathering as well as variation depending on hospital settings. Recently, researchers and clinicians have debated methods for calculating error rates. A comparative effectiveness review by one of AHRQ’s Evidence-Based Practice Centers on diagnostic errors in emergency departments has prompted a discussion as to the accuracy of diagnostic error and harms rates generated from the limited data available.5
Factors Contributing to Diagnostic Errors
Research in diagnostic safety in 2022 showed that diagnostic errors include individual, organizational, and system-wide factors and that the synergy of these factors was associated with diagnostic error. Other research highlighted the variability in the extent to which these factors, or the interaction of these factors, play a role in diagnostic error. For example, research in pediatric settings has indicated that the contribution of individual clinician factors to diagnostic errors ranges from 20% to 60%. The wide variation in these estimates results from a lack of standardized data on diagnostic error which is mostly from neonatal or pediatric intensive-care settings. However, these numbers are broadly consistent with adult misdiagnosis rates. It is important to note that factors contributing to misdiagnosis are often broadly or inadequately defined and are largely classified based on clinician surveys or experts’ definitions. Studies may refer to cognitive, individual, system, or organizational factors that contribute to misdiagnosis, but often without consistency in these factors from study to study. Furthermore, the causal chain for misdiagnosis may span multiple factors. For example, an individual-level factor such as being rushed or taking shortcuts in making a diagnosis can cascade from being in a distracted environment, high time-pressure, or being understaffed. Given the complexity of the diagnostic process, it is unsurprising that contributors to diagnostic error are multifaceted beyond individuals to include organization- and system-level factors which are challenging to define and isolate.
Beyond challenges in identifying the causes of errors, challenges also exist with the type of data available to understand factors contributing to and harms from diagnostic errors. A study on misdiagnosis in pediatric settings emphasize that additional data are needed to further identify underlying causes of diagnostic error and offer examples of how clinical settings and patient populations can impact diagnosis. Authors of the study note that current data on misdiagnosis over relies on autopsy studies, which tend to overestimate the rates of harm from misdiagnosis for both adults and children. Although factors affecting pediatric and adult diagnostic errors may differ due to differences in frequency and types of diseases and the addition of parents/caregivers, among both children and adults researchers are limited by the data available to identify underlying causes.
Individual-Level Factors
Acknowledging the contribution of system-level failures, diagnostic error resulting from human error continues to be studied. In one study, researchers analyzed serious adverse event reports identified using the Safer Dx tool and found that more than half of errors occurred during patient assessment. Researchers noted that specific training and education to bridge knowledge gaps may reduce mistakes or knowledge-based errors but do not address the whole picture or acknowledge that these errors may arise from work- or time-related pressures. Additional data collection in serious adverse event reports is needed to drive initiatives that will prevent misdiagnosis, including identifying strategies to improve clinical decision making in diagnosis.
While a full discussion of clinical decision making is beyond the scope of this essay, it is a factor in diagnostic error. Therefore, we touch on a few issues addressed about this topic in the research this year. Clinical decision making is a complex process that includes a variety of factors. Clinicians confront a fair amount of uncertainty. In making a diagnosis and choosing a treatment plan, clinicians may need to balance individual versus population risks and benefits as exemplified by research about misdiagnosis of infection. In this study, researchers suggested that the higher rates of misdiagnosis may reflect clinician bias toward treatment, when faced with diagnostic uncertainty, because risks to the population from not treating (i.e., the risk of antimicrobial resistance) outweighed the risk of treating the individual unnecessarily in these situations (i.e., prescribing antibiotics when they are not needed). Reducing misdiagnosis in these situations may have broader, population-level effects on patient safety.
Biases in the Diagnostic Process
Unconscious thinking in clinical decision making may contribute to misdiagnosis in the form of cognitive biases. The 2015 NASEM report that established the definition of diagnostic error connected clinical decision making to dual process theory, which is a concept from psychological literature that describes two types of decision making, one that is nonanalytical, intuitive, and quick-thinking (type 1 thinking); and one that is analytical and deliberative (type 2 thinking).1 Cognitive biases and heuristics (mental shortcuts that aid decision making) are examples of type 1 decision making during diagnosis. A recent study examined heuristics related to type 1 processing in a primary care setting, noting that symptom presentation is often poorly defined in primary care, leaving patients more vulnerable to misdiagnosis. Use of heuristics was common. In particular, physicians seemed to frequently rely on (1) anchoring bias, in which a doctor is biased toward their first impression, and (2) availability bias, in which the likelihood of a diagnosis is erroneously linked to how easy it is to think of examples. Although researchers found that unconscious thinking often played a role in diagnosis, it was not always related to misdiagnosis. About 10% of cases of misdiagnosis examined did not have a correlation with heuristics. Further, telling clinicians to slow down, ostensibly to switch from type 1 to type 2 thinking, may not effectively reduce error. The extent to which cognitive shortcuts and unconscious thinking impact medical decisions has not been definitively described, and the literature is mixed on how often unconscious thinking results in misdiagnosis.
This uncertainty is further complicated by the use of hypothetical vignettes in literature, so that published research on the extent of misdiagnosis may not reflect real-world conditions.6 The conditions under which diagnoses are rendered do play a role in misdiagnosis, as does clinical decision making. As these underlying causes are elucidated, interventions are being developed and proposed to improve the diagnostic process.
Communication as a Factor in Misdiagnosis
Clinician-centered misdiagnosis is only a part of overall diagnostic safety; the NASEM definition also includes failure to communicate an explanation of the patient’s health problem to the patient as a diagnostic error. Although clinicians and researchers endorse the importance of patient-centeredness in assessing misdiagnosis, the extent to which failure to accurately communicate diagnosis constitutes diagnostic error is controversial. In patient-centered work to categorize diagnostic error, patients emphasized that communication breakdowns may play a large role in misdiagnosis, impacting reporting of symptoms. Interpersonal communication skills may therefore be an individual level factor that can mitigate or contribute to misdiagnosis, apart from clinician knowledge or decision making.
System-Level Factors
Diagnostic safety is complex and sensitive to context. To manage this complexity, research has tended to focus on single types of contributing factors, such as problems resulting from cognitive errors made by clinicians. But as research in diagnostic safety has acknowledged this complexity, more studies have added analyses that describe the role of multiple noncognitive factors and systems issues. One such study recently proposed a conceptual model connecting work conditions, such as time pressure and volume-based patient loads, to clinician stress and misdiagnosis, drawing together findings from studies on educational theory, clinician burnout, as well as diagnostic safety. This type of holistic analysis is the logical effect of acknowledging that misdiagnosis is complex and that explanations of misdiagnosis may explore everything from clinician-patient interactions to billing structure.
An example of the interplay between factors was evident in research showing patients with limited English-language health literacy or disadvantaged socioeconomic position were more likely to report unique contributing factors to their experience of diagnostic errors, such as a lack of interpreter services. The interaction between systemic issues, i.e., lack of interpreter services, and interpersonal patient factors, i.e., limited English-language health literacy, likely diminished diagnostic safety for these patients.
Methods to Improve Diagnostic Safety
Because misdiagnosis arises from complex factors, reducing misdiagnosis requires interventions that span contexts, from individual clinician training to organizational strategies to capture and analyze diagnostic error. At the individual practitioner level, diagnostic mnemonics have been developed to prompt clinicians with questions intended to reduce cognitive error, but these have shown mixed results.7,8 AHRQ has developed the Calibrate Dx guide, which provides actionable guidance that scales to particular needs at the level of individual practitioner whose scope of work includes diagnosis. However, reducing misdiagnoses may require changing a healthcare organization’s institutional culture or deploying tools that critically assess particular encounters.9 At the healthcare organization level, in 2018 the Health Research and Educational Trust released a toolkit to identify situations in which diagnostic error might arise called “Improving Diagnosis in Medicine.”10 More recently, to facilitate learning and improvement, AHRQ developed the Measure DX resource to help healthcare organizations identify and analyze data to overcome obstacles to implementing diagnostic safety programs. The Measure Dx resource has tools to engage organizational personnel in reducing misdiagnosis, assessment tools to understand readiness of an organization to implement changes to reduce diagnostic error, as well as data-driven solutions to reduce misdiagnosis
Beyond AHRQ-generated resources, in 2022 several authors published data on novel approaches to mitigate misdiagnosis and improve diagnostic safety in medicine. A systematic review of diagnostic safety checklists exposed gaps in the way these checklists are used. Although used extensively, the review observes that few checklists incorporate human factors principles and instead tend to emphasize cognitive factors involved in the diagnostic process. Because misdiagnosis is multifactorial, using behavioral and physical subcomponents could improve efficacy of these checklists. Few studies examining cognitive checklists found improved diagnosis; task-based checklists were more often associated with improved diagnostic safety. Also, the use of a diagnostic time out, an intentional pause to consider alternate diagnoses, was tested in a small pilot study over 12 months in a pediatric hospital. This study found that over half of the cases selected for a diagnostic time out did not confirm the initial diagnosis, and that implementing the pause was not found to be burdensome.
Another potentially promising, but possibly underutilized, strategy to mitigate human error is the use of clinical decision support (CDS) tools. One example is a CDS tool for pulmonary embolism that was created by incorporating a human factors approach through work system analysis, iterative design, and usability testing. This tool was launched in an academic medical center, but despite high ratings of usability and acknowledgment that diagnosing pulmonary embolism was difficult, the tool was not broadly used, due to a lack of workflow integration in the real clinical setting. Challenges to implement CDS supports to improve diagnosis in real-world settings are consistent with other studies; and CDS tools may be most useful to younger, less experienced clinicians. The study authors note that the lack of use of CDS tools appears to result from systemic, organizational problems, pointing to the need for comprehensive all-level approaches to achieve greater safety in diagnosis. As research continues to elucidate factors that contribute to diagnostic error, interventions that seek to mitigate error are being developed and tested.
Future Directions
Additional elements of clinical decision making may yet be described, which could impact diagnosis in the future. For instance, insight, or the ability to come to the correct diagnosis after initial uncertainty, does not neatly fit into the prevailing dual process theory but may play a significant role in improved diagnostic safety. Insight cannot be trained or taught but could be cultivated through both individual education and intentional systemic design, as well as by using existing diagnostic safety strategies, such as cognitive forcing functions, like having clinicians ask themselves “what else does this look like,” or mnemonics that might similarly prompt a clinician to consider clinical contexts that may not first spring to mind. Insight may be stimulated through greater patient and physician communication; if engaged in offering diagnostic suggestion, a patient’s comments may be the catalyst to a clinician’s epiphany, that is, their moment of insight that could avoid diagnostic error.
Beyond this, a key step to strengthening diagnostic safety is the widespread use of a comprehensive framework for how to define and identify factors contributing to misdiagnosis. Current efforts to study diagnostic safety are fragmented and stem primarily from surveys and expert opinions. With a comprehensive framework, diagnostic safety can be bolstered at multiple levels. However, barriers exist to changing current care practices, even if such practices are vulnerable to diagnostic error. These barriers may be difficult to change, such as reimbursement models based on patient volume, or they could be more amenable to intervention, such as institutional uncertainty about how to begin efforts, which can be addressed via the Measure Dx tool. It is important to raise awareness about diagnostic safety by emphasizing it (a) as a focused area for quality improvement at the healthcare organization level, (b) as a research need especially for real-life studies not based on simulation, and (c) as an area for continued clinician training and education.
Despite research that connects human action with system-level factors that influence diagnostic safety, misdiagnosis is still often thought of as a problem of individual practitioners, which can lead to fear of blame and hesitation among clinicians to discuss errors. The idea that misdiagnosis is primarily a problem of individual practitioners also discourages broader investment in diagnostic safety. Causes of misdiagnosis can often stem from a complex web of factors ranging from individual to system and organizational levels. Using a more holistic approach for identifying and defining factors that lead to misdiagnosis, such as the Systems Engineering Initiative for Patient Safety (SEIPS 2.0), which draws from human factors research, may help better understand the root causes of misdiagnosis and can unify researchers and healthcare institutions’ efforts in addressing them. Many organizations have committed to continuous quality improvement with peer input and the use of technology; these efforts should be focused on preventing diagnostic error. Application of the Measure Dx tool is a concrete step organizations can take to shift away from framing diagnostic safety as a problem of individual practitioners to a systems-based approach focused on opportunities to improve diagnostic safety programs.
References
1. Burke HB. Quantifying diagnostic excellence [published online ahead of print, 2022 Nov 23]. Diagnosis (Berl). 2022;10.1515/dx-2022-0104. doi:10.1515/dx-2022-0104
2. Committee on Diagnostic Error in Health Care, National Academies of Science, Engineering, and Medicine. Improving diagnosis in healthcare. National Academies Press; 2015.
3. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165(13):1493-1499.
4. Newman-Toker DE, Wang Z, Zhu Y, et al. Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the “Big Three.”Diagnosis (Berl). 2020;8(1):67-84.
5. Grantham J, Dark C. Report on ED diagnostic errors sparks controversy. ACEPNow. Accessed February 13, 2023. https://www.acepnow.com/article/report-on-ed-diagnostic-errors-sparks-controversy/
6. Blumenthal-Barby JS, Krieger H. Cognitive biases and heuristics in medical decision making: a critical review using a systematic search strategy. Med Decis Making. 2015;35(4):539-557.
7. Chew KS, Durning SJ, van Merriënboer JJ. Teaching metacognition in clinical decision-making using a novel mnemonic checklist: an exploratory study.Singapore Med J. 2016;57(12):694-700.
8. O'Sullivan ED, Schofield SJ. A cognitive forcing tool to mitigate cognitive bias-a randomised control trial. BMC Med Educ. 2019;19(1):12.
9. Redmond S, Barwise A, Zornes S, et al. Contributors to Diagnostic Error or Delay in the Acute Care Setting: A Survey of Clinical Stakeholders. Health Serv Insights. 2022;15:11786329221123540. Published 2022 Sep 13.
10. Health Research & Educational Trust. Improving Diagnosis in Medicine Change Package. Health Research & Educational Trust; 2018. Accessed February 13, 2023. https://www.improvediagnosis.org/improving-diagnosis-in-medicine-change-package/#:~:text=The%20Improving%20Diagnosis%20in%20Medicine,of%20patients%20and%20their%20families