Email Updates
Measurement of Patient Safety
Background
The 1999 Institute of Medicine (IOM) To Err Is Human report drew national attention to the problem of preventable harm in medicine and led to the creation of the modern patient safety field. Yet little of the material in To Err Is Human was new—the key studies on which the report (and its headline-grabbing estimate of up to 98,000 deaths every year due to preventable harm) was based were performed many years before the report was issued.
The impact of To Err Is Human demonstrates the importance of rigorously measuring the incidence and prevalence of preventable harm. However, measurement of patient safety is complex, and, while several different methods may be used, there is no single validated method for measuring the overall safety of care provided in a given health care setting. One commentator compared safety measurement to the fable in which five blind men describe an elephant in widely varying terms (as a wall, fan, spear, snake, or tree), depending on which part of the animal they touched. In this analogy, an institution's view of its safety issues inevitably depends on the method being used to measure safety, and a comprehensive picture can only be obtained by integrating multiple methods.
This primer will review methods of measuring patient safety. Related Patient Safety primers discuss Detection of Safety Hazards, Strategies and Approaches for Tracking Improvements in Patient Safety, Patient Safety Indicators, and National Patient Safety Goals.
A Framework for Measuring Patient Safety
In the 1960s, health services researcher Avedis Donabedian defined a taxonomy for measuring the quality of health care. The "Donabedian triad," which is still widely used today, defines three lenses through which quality may be viewed:
- Structures—how care is organized
- Processes—what is done to the patient
- Outcomes—what ultimately happens to the patient
A structural measure of patient safety might assess whether a hospital has key resources in place to improve safety, such as an electronic health record (EHR) system or a mechanism to rapidly under take root cause analysis (RCA) after a patient safety event has occurred. Process measures assess adherence to safety standards, such as the proportion of surgical patients for whom a postoperative checklist is completed or the proportion of patients in a hospital receiving appropriate prophylaxis for venous thromboembolism. Outcome measures assess the incidence or prevalence of adverse events or preventable death. A related Patient Safety Primer on Adverse Events, Near Misses, and Errors discusses the definition and types of adverse events in more detail.
There is no one-size-fits-all approach to measurement—the choice of metric varies depending on the purpose of measurement. Measurement is used for a variety of purposes: to evaluate the effectiveness of safety interventions, identify new or emerging safety threats, compare safety across hospitals and clinics, or to determine whether patient safety is improving over time. For example, studies of missed nursing care assess process measures —the frequency with which required care elements are not completed. Studies of medication reconciliation may measure processes (such as the proportion of patients for whom a best possible medication history (BPMH) was documented at hospital admission) or outcomes (such as the preventable adverse drug events). The Leapfrog Hospital Survey evaluates hospitals based on structural metrics—the use of specific patient safety practices, such as computerized provider order entry (CPOE).
Methods for Measuring Patient Safety
Several methods that can be used for measuring patient safety events are described in the Table below.
Table. Examples of Safety Measurement Strategies
| Measurement Strategies | Advantages | Disadvantages |
|---|---|---|
| Retrospective Chart Review | Considered the "gold standard," contains rich and detailed clinical information | Costly, labor-intensive, data quality variable due to incomplete clinical information, retrospective review only. Efficiency improved by focusing chart reviews on cases identified by a reliable trigger tool or software tool |
| Voluntary Error Reporting Systems | Useful for internal quality improvement and case-finding, highlights adverse events that providers perceive as important | Capture a non-representative fraction of adverse events (in hospitals, most reports are submitted by nurses; relatively few by doctors), retrospective review only based on provider self-reports |
| Automated Surveillance | Can be used retrospectively or prospectively, helpful in screening patients who may be at high risk for adverse events using standardized protocols | Need electronic data to run automated surveillance, high proportion of triggered cases are false positives |
| Administrative/Claims Data (e.g., AHRQ Patient Safety Indicators) | Low-cost, readily available data, useful for tracking events over time across large populations, can identify potential adverse events | Lack detailed clinical data, concerns over variability and inaccuracy of ICD-9-CM and ICD-10-CM codes across and within systems, may detect high proportion of false positives and false negatives |
| Electronic Health Record Data | Real-time assessment of quality of care being provided to patients. Enables prevention of errors through prospective rather than retrospective reviews when used with a trigger tool | Variability in electronic health record data structure and lack of structured fields across and within systems |
| Patient Reports | Can capture errors not easily recognized by other methods (i.e., errors related to communication between providers) | Measurement tools are still in development |
(Adapted from Wachter RM. Understanding Patient Safety, Second Edition. New York, NY: McGraw-Hill Professional; 2012. ISBN: 9780071765787.)
Problems and Controversies in Measuring Patient Safety
Despite the importance of accurately measuring adverse events, existing tools all have limitations, and controversy continues to plague efforts to measure safety and compare safety between organizations. Retrospective chart review using trigger tools or well-defined specific adverse events is often used in research studies, but it is so labor-intensive that most hospitals do not routinely monitor safety performance in this fashion. Certain types of errors—such as diagnostic errors—still lack standardized and reliable measurement strategies, and studies have shown that variation in how medication errors are defined can result in widely varying estimates of error prevalence. As a result, evaluating the effectiveness of safety programs remains a challenge for most organizations.
Even when adverse events can be measured, there is an additional layer of controversy regarding the definition of preventable harm. In the seminal studies that formed the basis of a To Err Is Human, experienced clinicians often disagreed on whether an error was preventable. Differences in definitions of errors account for some of the wide variation in estimates of the patients who experience preventable harm.
Current Context
Accurate and reliable measurement of errors and adverse events remains a major challenge for the patient safety field. The 2015 Free From Harm report by the National Patient Safety Foundation called for the creation of a "common set of safety metrics that reflect meaningful outcomes" as one of eight recommendations for advancing patient safety. Their specific recommendations included establishing a standardized set of process and outcome measures for use on a national basis (as well as retiring outdated measures), creating measures of patient safety for settings outside the hospital, improving the quality of safety reporting systems, and developing ways of measuring safety in real time (as opposed to retrospective measurement). The AHRQ Common Formats were developed in order to "help providers uniformly report patient safety events and to improve health care providers' efforts to eliminate harm." The Common Formats represent an important step toward achieving the goal of developing a universal framework for measurement of safety processes and outcomes.
Federal agencies such as the Centers for Medicare & Medicaid Services (CMS) and AHRQ continue to prioritize patient safety measurement. The CMS National Quality Strategy emphasizes the ongoing transition to digital quality measures (such as electronic clinical quality measures, or eCQMs, based on information from the electronic health record) and advances in interoperability and information sharing to facility improved patient safety measurement. The 2020 National Action Plan to Advance Patient Safety recommends the use of comprehensive patient safety measurement across the continuum of patient care, including ambulatory care and pre-hospital settings.
