Background

Over the past 25 years, policymakers and providers, payers, and purchasers of health care have increasingly focused attention on patient safety. As described in Patient Safety 101, “safety” can be defined as “avoiding harm to patients from care that is intended to help them. It involves the prevention and mitigation of harm caused by errors of omission or commission in healthcare, and the establishment of operational systems and processes that minimize the likelihood of errors and maximize the likelihood of intercepting them when they occur.” The Measurement of Patient Safety primer describes the framework for measuring patient safety based on Donabedian’s structure-process-outcome taxonomy as well as measurement strategies, such as retrospective medical record review, voluntary error reporting systems, automated surveillance or trigger tools, patient reports, and administrative or claims data.

Patient safety indicators can be viewed as a subset of tools that are designed to assess the frequency, severity, and impact of measurable harm in health care, both within health care organizations and at the health care system, regional, and national levels. Patient safety indicators may differ from other measurement tools (such as voluntary error reporting systems, sentinel events, never events and trigger tools) in that they have numerator and denominator specifications and often incorporate risk-adjustment to allow users to track observed and adjusted rates over time. In this way, users can monitor the effectiveness of their safety improvement efforts and compare the effectiveness of those efforts across organizations.

As outcome measures, patient safety indicators may not provide detailed and actionable information on how to improve quality of care,¹ but they inform health care consumers (patients), caregivers, payers, and purchasers of care. These stakeholders are more interested in how health care affects health outcomes and less interested in the structures and processes used to produce health care. Patient safety indicators also inform healthcare providers in efforts to develop systems and processes to prevent harm to patients, including the failure to provide needed care.

The original efforts to assess harm in health care applied the concept of “medical error,” which Brennan and colleagues defined in the Harvard Medical Practice Study in New York State as “an injury that was caused by medical management (rather than the underlying disease) and that prolonged the hospitalization, produced a disability at the time of discharge, or both.” This study, which was subsequently replicated and updated in Colorado and Utah, provided the basis for the 1999 Institute of Medicine (IOM) report that estimated 44,000-98,000 preventable deaths due to medical errors each year. In response to these findings and others, the Agency for Healthcare Research and Quality (AHRQ) developed Patient Safety Indicators (PSIs) to capture potentially preventable events that compromise patient safety in acute care settings, using routinely collected administrative data.² Twenty PSIs were released in 2003 to aid hospitals in identifying these events, which represent opportunities for improvement in the delivery of inpatient care. AHRQ’s PSIs have since been updated and refined annually to stay current with coding updates (such as the transition from ICD-9-CM to ICD-10-CM), evidence from validation studies, and suggestions from users.

Applications

Acute care

The AHRQ PSIs were designed as indicators of patient safety in the acute care setting based on all-payer administrative data sets. They were adopted by many organizations for internal monitoring and by state and local health agencies for comparative reporting, including California, New York, and New Jersey. The Centers for Medicare & Medicaid Services (CMS) adopted PSI 90 (Patient Safety and Adverse Events Composite) and PSI 04 (Death Rate Among Surgical Inpatients with Treatable Complications) to describe the safety of hospital care for adult patients enrolled in fee-for-service Medicare, in support of CMS programs that include:

• Hospital-Acquired Condition (HAC) Reduction Program (PSI 90 only)
• Hospital Inpatient Quality Reporting (IQR) Program

The Leapfrog Hospital Safety Grade uses more than 30 national performance measures from CMS, the Leapfrog Hospital Survey, and information from other supplemental data sources to produce a single letter grade representing a hospital’s overall performance in keeping patients safe from preventable harm and medical errors. Outcome measures of patient safety, including hospital acquired infections, hospital acquired conditions, PSI 04 and PSI 90, account for half the overall score. Healthgrades, a commercial vendor that rates hospitals using publicly available data, uses 14 AHRQ PSIs in its Patient Safety Excellence Award.

Several medical and surgical specialty societies have developed patient safety indicators specific to their specialties. For example, the National Surgical Quality Improvement Program (NSQIP) of the American College of Surgeons provides risk-adjusted surgical outcome measures for participating hospitals, including overall mortality and safety-related outcomes such as cardiac complications, postoperative pneumonia, intubations required within 48 hours after surgery, unplanned intubation, pulmonary embolism and venous thrombosis, renal dysfunction, and surgical site infections. As another example, the American Society for Clinical Laboratory Science has proposed a patient safety indicator focused on delayed test results in emergency departments due to specimen rejection. The Society of Thoracic Surgeons offers patient safety indicators for adult cardiac surgery, such as deep sternal wound infection.

Ambulatory Care

In recent years, many aspects of healthcare delivery have transitioned into the outpatient or ambulatory setting. This transition has prompted a body of research on evaluating, measuring, and preventing patient safety events in ambulatory care. However, patient safety indicators are not currently used in ambulatory care. A 2016 systematic review commissioned by the World Health Organization (WHO) identified missed and delayed diagnoses and medication errors as the chief safety priorities in ambulatory care. It also highlighted the need to develop clear and consistent definitions for patient safety incidents in primary care.

Post-Acute Care Settings

A 2018 report by the Office of the Inspector General (OIG) concluded that 33% of Medicare beneficiaries in skilled nursing facilities and 29% of Medicare beneficiaries in rehabilitation hospitals experienced harm. The report identified that over half of these adverse events and temporary harm events (54%) were clearly or likely preventable; preventable harm events were often related to substandard care (58%) and medical errors (34%). CMS responded to this report and other findings and concerns by developing a set of indicators to measure patient safety for post-acute care facilities, including outcome measures focusing on mobility (falls), skin integrity (pressure injuries) and hospital acquired infections (CLABSI, CAUTI).

International Perspective

Following AHRQ’s lead, numerous other countries and international organizations have developed or adapted their own patient safety indicators. For example, the Canadian Institute for Health Information developed a Hospital Harm measure, which captures 31 types of adverse events in acute care hospitals “that could have been potentially prevented by implementing evidence-informed practice,” including health care and medication associated conditions, healthcare associated infections, patient accidents, and procedure-associated conditions. For international comparative reporting, the Organization for Economic Cooperation and Development has adapted 6 of AHRQ’s PSIs, including measures of retained surgical items, postoperative thromboses, postoperative sepsis, surgical wound dehiscence, and obstetric trauma. The World Health Organization’s Global Patient Safety Action Plan 2021–2030 calls on governments to develop sets of “indicators for patient safety aligned with global patient safety targets,” health care facilities to “implement patient safety indicators and use (them) to track progress and monitor trends, and the WHO Secretariat to “create a (global) repository of patient safety indicators.” Their report recommends “advanced indicators” of patient safety outcomes such as avoidable deaths due to health care-associated venous thromboembolism, sepsis, or patient falls; in-hospital pressure ulcers; and ventilator-associated pneumonia incidents.

Current Context

Based on various types of data analyses using multiple approaches, there has been a significant downward trend in patient safety indicators starting in 2000. More recently, a study of 2010-2019 data based on medical chart review demonstrated a significant decline in the rates of adverse events among hospitalized patients with acute myocardial infarction, heart failure, pneumonia, and major surgical procedures (and for patients with all other conditions between 2012 and 2019). CMS’ 2021 National Impact Assessment Report (which examined data between 2013 and 2018) demonstrated that 92% of the analyzed patient safety measures had improved or had stable performance. PSI trends appeared to precede payment policies targeting patient safety events, suggesting that financial penalties may have less impact than the ongoing public and professional interest in improving patient safety. However, as demonstrated in a 2022 OIG report, which compared the frequency of patient safety incidents to a 2010 OIG report, many hospitalized patients continue to experience adverse events and temporary harm. These findings highlight the need to continue enhancing patient safety indicators to strengthen their validity, effectiveness, and ease of use. To this end, the OIG has charged national health agencies to reassess and update efforts related to the National Strategy for Quality Improvement in Health Care, optimizing patient harm surveillance tools and providing better guidance to surveyors who assess hospital compliance with these tools.

As part of its National Quality Strategy, CMS is leveraging its public reporting and value-based payment programs to include new metrics focused on diagnostic accuracy and hospital harms, captured as electronic clinical quality measures. CMS is committed to zero preventable harm events and partnering with AHRQ and the Institute for Healthcare Improvement (IHI) on the National Action Plan to Advance Patient Safety, among other initiatives. CMS’ Meaningful Measures 2.0 initiative aims to streamline the measurement process by focusing on measures related to person-centered care, equity, safety, affordability and efficiency, chronic conditions, wellness and prevention, seamless care coordination, and behavioral health. In 2022, CMS published the Digital Quality Measures Strategic Roadmap, which outlines its transition strategy.

AHRQ’s Patient Safety Indicators and related measures have been criticized because they initially failed to focus only on hospital-acquired conditions and because some indicators had relatively high false positive rates. As these concerns were resolved through measure refinements and better data quality (e.g., nearly universal adoption of the “present on admission” indicator), some critiques highlighted the phenomenon known as complexity bias, whereby healthcare organizations attempt to address metrics such as PSIs, by creating separate quality improvement teams that do not effectively coordinate their efforts in a unified model of safe hospital care. Other critics have focused on the limited reliability of patient safety indicators at the hospital level, due to the relative infrequency of these events. This problem has been partially mitigated by creating a patient safety composite measure (Patient Safety Indicator [PSI] 90) using weights that reflect both the frequency of component PSIs and their downstream consequences, based on patients' preferences for outcome-related health states.

Despite challenges due to varying definitions and data quality across health providers and health systems, patient safety indicators have continually improved and contributed to increased patient safety over time. A new generation of patient safety indicators, currently in development or testing, will leverage structured fields in electronic health records, longitudinally linked administrative data, and new taxonomies such as AHRQ’s Common Formats and WHO’s 11^th Revision of the International Classification of Diseases (ICD-11).

Irina Tokareva, RN, BSN, MAS, CPHQ
Quality Measurement Clinician Researcher
Center for Health Care Policy and Research
University of California, Davis
itokareva@ucdavis.edu

Patrick S. Romano, MD MPH
Co-Editor-in-Chief, PSNet
Professor of Internal Medicine and Pediatrics
UC Davis Health
psromano@ucdavis.edu

References

1. Salampessy BH, Portrait FRM, van der Hijden E, et al. On the correlation between outcome indicators and the structure and process indicators used to proxy them in public health care reporting. Eur J Health Econ. 2021;22(8):1239-1251 [Available at]
2. Miller MR, Elixhauser A, Zhan C, et al. Patient Safety Indicators: using administrative data to identify potential patient safety concerns. Health Serv Res. 2001;36(6 Pt 2):110-132. [Free full text]