Rating the raters: the inconsistent quality of health care performance measurement.

David M. Shahian; Sharon-Lise T. Normand; Mark W. Friedberg; Matthew M. Hutter; Peter Pronovost

Commentary

Rating the raters: the inconsistent quality of health care performance measurement.

March 2, 2016

Shahian DM, Normand S-LT, Friedberg MW, et al. Ann Surg. 2016;264(1):36-8.

View more articles from the same authors.

Public ratings of hospital quality and safety data may not always provide the best information for patients and clinicians. This commentary discusses problems with the existing set of patient safety metrics and suggests that measurement approaches need to be improved to enhance transparency and decision making.

PubMed citation

Available at

Free full text

Related Resources From the Same Author(s)

Safety culture and mortality after acute myocardial infarction: a study of Medicare beneficiaries at 171 hospitals.

October 25, 2017

Variability in the measurement of hospital-wide mortality rates.

March 19, 2014

Impact of adverse events on prescribing warfarin in patients with atrial fibrillation: matched pair analysis.

January 31, 2006

Relationship between in-hospital adverse events and hospital performance on 30-day all-cause mortality and readmission for patients with heart failure.

August 2, 2023

Hospital-readmission risk--isolating hospital effects from patient effects.

September 20, 2017

Analysis of hospital-level readmission rates and variation in adverse events among patients with pneumonia in the United States.

June 22, 2022

Association of changing hospital readmission rates with mortality rates after hospital discharge.

August 9, 2017

Association between hospital performance on patient safety and 30-day mortality and unplanned readmission for Medicare fee-for-service patients with acute myocardial infarction.

August 3, 2016

Conflict of interest, Dr Charles Denham and the Journal of Patient Safety.

December 3, 2014

I-PASS handover system: a decade of evidence demands action.

May 12, 2021

Time for transparent standards in quality reporting by health care organizations.

August 16, 2017

A controlled trial of a rapid response system in an academic medical center.

June 25, 2008

"Never events" and the quest to reduce preventable harm.

June 3, 2015

Residents report on adverse events and their causes.

January 4, 2006

The impact of the 80-hour resident workweek on surgical residents and attending surgeons.

July 12, 2006

Fostering transparency in outcomes, quality, safety, and costs.

October 12, 2016

Health system leaders' role in addressing racism: time to prioritize eliminating health care disparities.

February 10, 2021

Integrating CUSP and TRIP to improve patient safety.

November 24, 2010

Improving healthcare quality through organisational peer-to-peer assessment: lessons from the nuclear power industry.

June 6, 2012

Developing a measure of value in health care.

June 1, 2016

The role of dynamic trade-offs in creating safety—a qualitative study of handover across care boundaries in emergency care.

October 28, 2015

A practical framework for patient care teams to prospectively identify and mitigate clinical hazards.

February 11, 2009

Preventing patient harms through systems of care.

August 22, 2012

Diagnostic errors--The next frontier for patient safety.

March 18, 2009

Time of day and the decision to prescribe antibiotics.

October 22, 2014

Large-scale implementation of the I-PASS handover system at an academic medical centre.

March 22, 2017

Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary.

May 6, 2009

Comparative, cross-sectional study of the format, content and timing of medication safety letters issued in Canada, the USA and the UK.

November 7, 2018

Direct reporting of laboratory test results to patients by mail to enhance patient safety.

July 26, 2006

Managing competing organizational priorities in clinical handover across organizational boundaries.

January 21, 2015

The checklist--a tool for error management and performance improvement.

November 29, 2006

Surgical never events in the United States.

January 9, 2013

A framework for encouraging patient engagement in medical decision making.

August 29, 2012

From heroism to safe design: leveraging technology.

February 12, 2014

Ethical and practical aspects of disclosing adverse events in the emergency department.

October 4, 2006

Association of a clinical knowledge support system with improved patient safety, reduced complications and shorter length of stay among Medicare beneficiaries in acute care hospitals in the United States.

July 16, 2008

Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial.

February 24, 2016

The National Quality Forum safe practice standard for computerized physician order entry: updating a critical patient safety practice.

January 10, 2007

Planning and implementing a systems-based patient safety curriculum in medical education.

August 13, 2008

The occurrence of potential patient safety events among trauma patients: are they random?

March 5, 2008

A framework for health care organizations to develop and evaluate a safety scorecard.

November 7, 2007

Long-term sustainability and adaptation of I-PASS handovers.

September 13, 2023

Handoffs causing patient harm: a survey of medical and surgical house staff.

October 1, 2008

Massive open online course (MOOC) learning builds capacity and improves competence for patient safety among global learners: a prospective cohort study.

September 8, 2021

Learning accountability for patient outcomes.

July 21, 2010

We need leaders: the 48th Annual Rovenstine Lecture.

May 19, 2010

Enhancing physicians' use of clinical guidelines.

January 8, 2014

An interview with Peter Pronovost

November 2, 2005

Harmful medication errors involving unfractionated and low-molecular-weight heparin in three patient safety reporting programs.

April 28, 2010

A method to identify pediatric high-risk diagnoses missed in the emergency department.

August 8, 2018

Prescription opioid analgesics commonly unused after surgery: a systematic review.

August 9, 2017

Patient-assisted incident reporting: including the patient in patient safety.

June 1, 2011

Disclosing medical errors to patients: it's not what you say, it's what they hear.

July 22, 2009

Aiming higher to enhance professionalism: beyond accreditation and certification.

May 27, 2015

How will we know patients are safer? An organization-wide approach to measuring and improving safety.

June 7, 2006

Underdiagnosis of hypertension in children and adolescents.

September 5, 2007

Diagnostic errors in the pediatric and neonatal ICU: a systematic review.

March 4, 2015

Intensive care unit safety incidents for medical versus surgical patients: a prospective multicenter study.

October 3, 2007

A system factors analysis of "line, tube, and drain" incidents in the intensive care unit.

August 24, 2005

Using a logic model to design and evaluate quality and patient safety improvement programs.

July 25, 2012

Learning from defects to enhance morbidity and mortality conferences.

March 25, 2009

Patient safety reporting: a qualitative study of thoughts and perceptions of experts 15 years after 'To Err is Human.'

August 12, 2015

The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care.

August 24, 2005

Effect of reducing interns' work hours on serious medical errors in intensive care units.

March 27, 2005

A novel process for introducing a new intraoperative program: a multidisciplinary paradigm for mitigating hazards and improving patient safety.

January 21, 2009

Monitoring and reducing central line-associated bloodstream infections: a national survey of state hospital associations.

June 30, 2010

Human factors engineering in patient safety.

February 19, 2014

The safety of inpatient health care.

January 25, 2023

View the world through a different lens: shadowing another provider.

October 8, 2008

The Johns Hopkins Hospital: identifying and addressing risks and safety issues.

March 6, 2005

The safe day call: reducing silos in health care through frontline risk assessment.

September 24, 2014

A morning briefing: setting the stage for a clinically and operationally good day.

August 10, 2005

The role of South--North partnerships in promoting shared learning and knowledge transfer.

September 6, 2017

Using incident reporting to improve patient safety: a conceptual model.

March 28, 2007

Balancing "no blame" with accountability in patient safety.

October 7, 2009

Rethinking rapid response teams.

September 29, 2010

Building a highway to quality health care.

September 7, 2016

A piece of my mind. From shame to guilt to love.

January 6, 2016

Progress in patient safety: a glass fuller than it seems.

August 28, 2013

Measuring preventable harm: helping science keep pace with policy.

April 1, 2009

Diagnostic errors, health disparities, and artificial intelligence: a combination for health or harm.

September 1, 2021

Forum: The 100,000 Lives Campaign: a scientific and policy review [with IHI response].

November 1, 2006

Shepherding change: how the market, healthcare providers, and public policy can deliver quality care for the 21st century.

March 1, 2006

The tension between promoting mobility and preventing falls in the hospital.

May 10, 2017

An objective methodology for task analysis and workload assessment in anesthesia providers.

December 7, 2005

Expert consensus on currently accepted measures of harm.

September 9, 2020

Close calls in patient safety: should we be paying closer attention?

May 15, 2013

Sustaining reliability on accountability measures at the Johns Hopkins Hospital.

January 27, 2016

Demonstrating high reliability on accountability measures at The Johns Hopkins Hospital.

December 4, 2013

The potential of Twitter as a data source for patient safety.

January 27, 2016

Wrong-site sinus surgery in otolaryngology.

August 11, 2010

Risk factors in patient safety: minimally invasive surgery versus conventional surgery.

March 7, 2012

Simulation study of rested versus sleep-deprived anesthesiologists.

January 9, 2005

Reducing medical errors and adverse events.

February 8, 2012

How safe is my intensive care unit? Methods for monitoring and measurement.

November 21, 2007

Making soft intelligence hard: a multi-site qualitative study of challenges relating to voice about safety concerns.

March 14, 2018

Needlestick injuries among surgeons in training.

July 4, 2007

Association of note quality and quality of care: a cross-sectional study.

April 23, 2014

Medication errors associated with code situations in U.S. hospitals: direct and collateral damage.

January 16, 2008

A trigger tool fails to identify serious errors and adverse events in pediatric otolaryngology.

October 27, 2010

Related Resources

Annual Perspective

Equity in Patient Safety

March 27, 2024

MHA and MHA Keystone Center Annual Reports.

October 20, 2023

Annual Perspective

Improving Diagnostic Safety and Quality

April 26, 2023

Patient Safety Innovations

The I-READI Quality and Safety Framework: Strong Communications Channels and Effective Practices to Rapidly Update and Implement Clinical Protocols During a Time of Crisis

March 15, 2023

Veterans Health Administration response to the COVID-19 crisis: surveillance to action.

October 26, 2022

Qualitative content analysis: a framework for the substantive review of hospital incident reports.

March 23, 2022

Each Baby Counts.

April 1, 2021

Creating a framework to integrate residency program and medical center approaches to quality improvement and patient safety training

January 13, 2021

Closing the loop with ambulatory staff on safety reports.

December 4, 2019

IV Push Gap Analysis Tool (GAT) helps uncover national priorities for safe injection practices.

September 11, 2019

A mixed-methods study of challenges experienced by clinical teams in measuring improvement.

September 11, 2019

Patient Safety Primers

Patient Safety 101

September 7, 2019

Interview

In Conversation With… Shantanu Agrawal, MD, MPhil

September 1, 2019

Managing risk in hazardous conditions: improvisation is not enough.

July 24, 2019

A decade of preventing harm.

June 19, 2019

What we know about designing an effective improvement intervention (but too often fail to put into practice).

June 28, 2017

Is excessive resource utilization an adverse event?

March 8, 2017

Addressing physician burnout: the way forward.

February 22, 2017

What have we learnt after 15 years of research into the 'weekend effect'?

January 25, 2017

America's Hospitals: Improving Quality and Safety—The Joint Commission's Annual Report 2016.

November 23, 2016

Headline-grabbing study brings attention back to medical errors.

August 24, 2016

Toward a safer health care system: the critical need to improve measurement.

April 27, 2016

Automatic errors: a case series on the errors inherent in electronic prescribing.

April 13, 2016

Scalia's death and the value of autopsy: a teachable moment.

April 6, 2016

Patient safety and the problem of many hands.

March 30, 2016

Understanding why quality initiatives succeed or fail: a sociotechnical systems perspective.

March 23, 2016

Systematically Identified Failure Is the Route to a Successful Health System.

March 9, 2016

From To Err Is Human to Improving Diagnosis in Health Care: the risk management perspective.

March 2, 2016

Walking the tightrope: communicating overdiagnosis in modern healthcare.

February 24, 2016

The Safer Delivery of Surgical Services Program (S3): explaining its differential effectiveness and exploring implications for improving quality in complex systems.

February 24, 2016

See More About The Topic

Hospitals

Health Care Executives and Administrators

Medicine

Epidemiology of Errors and Adverse Events

Quality Improvement Strategies

Rating the raters: the inconsistent quality of health care performance measurement.

The I-READI Quality and Safety Framework: Strong Communications Channels and Effective Practices to Rapidly Update and Implement Clinical Protocols During a Time of Crisis

Patient Safety 101

Connect With Us

Sign up for Email Updates

Agency for Healthcare Research and Quality

Rating the raters: the inconsistent quality of health care performance measurement.

The I-READI Quality and Safety Framework: Strong Communications Channels and Effective Practices to Rapidly Update and Implement Clinical Protocols During a Time of Crisis

Patient Safety 101

Connect With Us

Sign up for Email Updates

Agency for Healthcare Research and Quality

Submit Your Innovations

Continue as a Guest

Continue Logged In

Submit Your Innovations

Continue as a Guest

Continue logged in

New Users to the PSNet site