Clinical reasoning of a generative artificial intelligence model compared with physicians.

Stephanie Cabral; Daniel Restrepo; Zahir Kanjee; Philip Wilson; Byron Crowe; Raja-Elie Abdulnour; Adam Rodman

Study

Clinical reasoning of a generative artificial intelligence model compared with physicians.

April 17, 2024

Cabral S, Restrepo D, Kanjee Z, et al. JAMA Intern Med. 2024;Epub Apr 1.

View more articles from the same authors.

Research into how large language models (LLM) such as ChatGPT can be used in healthcare is growing rapidly. This study compared ChatGPT and physicians' clinical reasoning in a simulated case study. Using the validated Revised-IDEA (R-IDEA) framework, the LLM performed better than physicians did in processing medical data and clinical reasoning; there were no differences observed using other frameworks.

PubMed citation

Available at

Related Resources From the Same Author(s)

Assessment of adverse drug events among patients in a tertiary care medical center.

November 29, 2006

Effect of restriction of the number of concurrently open records in an electronic health record on wrong-patient order errors: a randomized clinical trial.

May 29, 2019

Changes in medical errors after implementation of a handoff program.

November 12, 2014

Preventing home medication administration errors.

March 14, 2022

Annals Clinical Decision Making: avoiding cognitive errors in clinical decision making.

May 20, 2020

Inattentional blindness in medicine.

April 17, 2024

The practice of respect in the ICU.

August 1, 2018

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics.

November 3, 2021

Transparency, public reporting, and a culture of change to quality and safety in cardiac surgery.

November 9, 2022

Priorities for pediatric patient safety research.

February 13, 2019

Errors of diagnosis in pediatric practice: a multisite survey.

July 7, 2010

Safety culture in the operating room: variability among perioperative healthcare workers.

September 13, 2017

Learning from errors: analysis of medication order voiding in CPOE systems.

March 15, 2017

Patient safety in inpatient mental health settings: a systematic review.

February 5, 2020

Barriers and facilitators to incident reporting in mental healthcare settings: a qualitative study.

November 13, 2019

Use of a surgical safety checklist to improve team communication.

September 21, 2016

A toolkit to disseminate best practices in inpatient medication reconciliation: Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS).

July 31, 2013

Prevalence of adverse events in the hospitals of five Latin American countries: results of the 'Iberoamerican Study of Adverse Events' (IBEAS).

August 10, 2011

National surveillance of emergency department visits for outpatient adverse drug events in children and adolescents.

March 5, 2008

Implementation of the I-PASS handoff program in diverse clinical environments: a multicenter prospective effectiveness implementation study.

November 16, 2022

Defining a high-quality and effective morbidity and mortality conference: a systematic review.

November 1, 2023

A scoping review of distributed cognition in acute care clinical decision-making.

June 7, 2023

Stopping the error cascade: a report on ameliorators from the ASIPS collaborative.

March 7, 2007

Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011.

May 25, 2016

A contemporary analysis of closed claims related to wrong site surgery.

March 29, 2023

Overriding drug-drug interaction alerts in clinical decision support systems: a scoping review.

September 7, 2022

Physician impairment and rehabilitation: reintegration into medical practice while ensuring patient safety: a position paper from the American College of Physicians.

June 19, 2019

Quantifying and characterizing adverse events in dermatologic surgery.

May 15, 2013

The AHRQ Report on Diagnostic Errors in the Emergency Department: the wrong answer to the wrong question.

June 28, 2023

The impact of post-fall huddles on repeat fall rates and perceptions of safety culture: a quasi-experimental evaluation of a patient safety demonstration project

October 16, 2019

Diagnostic errors in interpretation of pediatric musculoskeletal radiographs at common injury sites.

June 25, 2014

Pediatric prehospital medication dosing errors: a national survey of paramedics.

March 29, 2017

Bad behavior in healthcare: an insidious threat to patients, staff, and organizations.

April 17, 2024

Incidence of adverse drug events and medication errors in intensive care units: a prospective multicenter study.

February 18, 2009

Sequential implementation of the EQUIPPED geriatric medication safety program as a learning health system.

September 9, 2020

Early prescribing outcomes after exporting the EQUIPPED medication safety improvement programme.

January 19, 2022

Effect of pharmacist counseling intervention on health care utilization following hospital discharge: a randomized control trial.

June 8, 2016

Repeat medication errors in nursing homes: contributing factors and their association with patient harm.

July 28, 2010

Surgical training, duty-hour restrictions, and implications for meeting the Accreditation Council for Graduate Medical Education core competencies: views of surgical interns compared with program directors.

July 11, 2012

Effect of patient- and medication-related factors on inpatient medication reconciliation errors.

March 7, 2012

Novel use of electronic whiteboard in the operating room increases surgical team compliance with pre-incision safety practices.

March 14, 2012

Speaking up about care concerns in the ICU: patient and family experiences, attitudes and perceived barriers.

August 8, 2018

Fundamental Use of Surgical Energy (FUSE): an essential educational program for operating room safety.

April 5, 2017

Families as partners in hospital error and adverse event surveillance.

March 8, 2017

Wide variation and overprescription of opioids after elective surgery.

October 11, 2017

Association of household opioid availability and prescription opioid initiation among household members.

January 10, 2018

Simulated laparoscopic operating room crisis: an approach to enhance the surgical team performance.

July 2, 2008

Supplemental perioperative oxygen and the risk of surgical wound infection: a randomized controlled trial.

November 23, 2005

All CLEAR? Preparing for IT downtime.

September 28, 2016

Question answering systems for health professionals at the point of care - a systematic review.

March 20, 2024

The road to zero preventable birth injuries.

April 9, 2008

Variability in diagnostic error rates of 10 MRI centers performing lumbar spine MRI examinations on the same patient within a 3-week period.

November 17, 2016

To expand the evidence base about harms from tests and treatments.

September 15, 2021

Improving the specificity of drug-drug interaction alerts: can it be done?

April 6, 2022

Problem list completeness in electronic health records: a multi-site study and assessment of success factors.

August 26, 2015

Graphical display of diagnostic test results in electronic health records: a comparison of 8 systems.

April 8, 2015

Preventable mortality after common urological surgery: failing to rescue?

September 3, 2014

Knowledge retention after simulated crisis: importance of independent practice and simulated mortality.

June 5, 2019

Increasing adoption of computerized provider order entry, and persistent regional disparities, in US emergency departments.

August 31, 2011

Antibiotic prescribing in ambulatory pediatrics in the United States.

January 11, 2012

Circumstances involved in unsupervised solid dose medication exposures among young children.

April 22, 2020

Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room.

November 23, 2016

Operating room traffic as a modifiable risk factor for surgical site infection.

January 11, 2017

The Iatroref study: medical errors are associated with symptoms of depression in ICU staff but not burnout or safety culture.

February 4, 2015

Selected medical errors in the intensive care unit: results of the IATROREF study: parts I and II.

February 10, 2010

Guideline for opioid therapy and chronic noncancer pain.

May 31, 2017

Centers for Disease Control and Prevention Guideline for the Prevention of Surgical Site Infection, 2017.

May 10, 2017

Posttraumatic growth and second victim distress resulting from medical mishaps among physicians and nurses.

January 11, 2023

Impact of adverse events on outcomes in intensive care unit patients.

July 2, 2008

The MedSafer study-electronic decision support for deprescribing in hospitalized older adults: a cluster randomized clinical trial.

February 2, 2022

Effects of a refined evidence-based toolkit and mentored implementation on medication reconciliation at 18 hospitals: results of the MARQUIS2 study.

May 19, 2021

What works in medication reconciliation: an on-treatment and site analysis of the MARQUIS2 study.

April 12, 2023

Wrong-site and wrong-patient procedures in the Universal Protocol era: analysis of a prospective database of physician self-reported occurrences.

October 27, 2010

Disclosure and reporting of surgical complications: a double-edged sword?

August 18, 2010

The 5th anniversary of the "Universal Protocol": pitfalls and pearls revisited.

July 29, 2009

Internally-developed online adverse drug reaction and medication error reporting systems.

June 7, 2006

Effects of a multifaceted medication reconciliation quality improvement intervention on patient safety: final results of the MARQUIS study.

September 12, 2018

Impact of resident workload and handoff training on patient outcomes.

April 4, 2012

Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a patient-centered fall prevention toolkit.

August 9, 2017

Association of display of patient photographs in the electronic health record with wrong-patient order entry errors.

December 2, 2020

Impact of professional interpreters on outcomes for hospitalized children from migrant and refugee families with limited English proficiency: a systematic review.

September 16, 2020

Ethical considerations and patient safety concerns for cancelling non-urgent surgeries during the COVID-19 pandemic: a review.

May 12, 2021

Identification of common themes from never events data published by NHS England.

April 14, 2021

Common general surgical never events: analysis of NHS England never event data.

April 7, 2021

Registration errors among patients receiving blood transfusions: a national analysis from 2008 to 2017.

December 16, 2020

The effects of three consecutive 12-hour shifts on cognition, sleepiness, and domains of nursing performance in day and night shift nurses: a quasi-experimental study.

October 20, 2021

Role of nursing home quality on COVID-19 cases and deaths: evidence from Florida nursing homes.

September 22, 2021

Repurposing clinical decision support system data to measure dosing errors and clinician-level quality of care.

November 11, 2020

A blinded, prospective study of error detection during physician chart rounds in radiation oncology.

October 14, 2020

We're all in this together: how COVID-19 revealed the co-construction of mindful organising and organisational reliability.

February 9, 2022

Quality and safety in hospital pediatrics during COVID-19: a national qualitative study.

January 12, 2022

The role of feedback in emergency ambulance services: a qualitative interview study.

April 6, 2022

Toward zero harm: Mackenzie Health's journey toward becoming a high reliability organization and eliminating avoidable harm.

November 16, 2022

The impact of meaningful use and electronic health records on hospital patient safety.

November 2, 2022

Development of the Leapfrog Group's bar code medication administration standard to address hospital inpatient medication safety.

September 21, 2022

Investigating racial and ethnic disparities in maternal care at the system level using patient safety incident reports.

August 9, 2023

Types and effects of feedback for emergency ambulance staff: a systematic mixed studies review and meta-analysis.

July 12, 2023

Understanding medication safety involving patient transfer from intensive care to hospital ward: a qualitative sociotechnical factor study.

June 7, 2023

From principles to practice: embedding clinical reasoning as a longitudinal curriculum theme in a medical school programme.

June 15, 2022

A checklist to address implicit bias in healthcare settings during the COVID-19 pandemic: The PLACE Strategy.

June 8, 2022

Related Resources

Question answering systems for health professionals at the point of care - a systematic review.

March 20, 2024

Assessing diagnostic performance.

February 14, 2024

A text mining approach to categorize patient safety event reports by medication error type.

February 14, 2024

Current State of Diagnostic Safety: Implications for Research, Practice, and Policy.

February 7, 2024

Embracing the future-is artificial intelligence already better? A comparative study of artificial intelligence performance in diagnostic accuracy and decision-making.

February 7, 2024

Effectiveness of ChatGPT in clinical pharmacy and the role of artificial intelligence in medication therapy management.

February 7, 2024

Diagnostic accuracy of a large language model in pediatric case studies.

January 17, 2024

Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study.

January 10, 2024

"ChatGPT, can you help me save my child's life?" - Diagnostic accuracy and supportive capabilities to lay rescuers by ChatGPT in prehospital basic life support and paediatric advanced life support cases - an in-silico analysis.

December 6, 2023

Advancing Diagnostic Excellence for Maternal Health Care: Proceedings of a Workshop–in Brief.

November 15, 2023

Assessing biases in medical decisions via clinician and AI chatbot responses to patient vignettes.

November 1, 2023

Can AI help doctors come up with better diagnoses?

October 4, 2023

Understanding And Addressing Pre-Hospital Diagnostic Delays.

September 20, 2023

Patient and family contributions to improve the diagnostic process through the OurDX electronic health record tool: a mixed method analysis.

September 13, 2023

Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study.

September 13, 2023

Pediatric Diagnostic Safety: State of the Science and Future Directions.

September 13, 2023

A natural language processing approach to categorise contributing factors from patient safety event reports.

August 30, 2023

Reimagining Healthcare Teams: Leveraging the Patient-Clinician-AI Triad To Improve Diagnostic Safety.

August 16, 2023

Experimental evidence for structured information-sharing networks reducing medical errors.

August 9, 2023

Redesign of health care systems to reduce diagnostic errors: leveraging human experience and artificial intelligence.

August 9, 2023

Improving safety outcomes through medical error reduction via virtual reality-based clinical skills training.

June 21, 2023

AI may be on its way to your doctor’s office, but it’s not ready to see patients.

May 24, 2023

Optimizing measurement of misdiagnosis-related harms using symptom-disease pair analysis of diagnostic error (SPADE): comparison groups to maximize SPADE validity.

May 3, 2023

More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA.

May 3, 2023

Understanding complexity in a safety critical setting: a systems approach to medication administration.

April 26, 2023

I’m an ER doctor: here’s what I found when I asked ChatGPT to diagnose my patients.

March 22, 2023

Using A.I. to detect breast cancer that doctors miss.

March 15, 2023

Cost of inpatient falls and cost-benefit analysis of implementation of an evidence-based fall prevention program.

February 1, 2023

How providers can optimize effective and safe scribe use: a qualitative study.

February 1, 2023

Using community detection techniques to identify themes in COVID-19-related patient safety event reports.

December 7, 2022

See More About The Topic

Medicine

Clinical Misdiagnosis

Technologic Approaches

Clinical reasoning of a generative artificial intelligence model compared with physicians.

Connect With Us

Sign up for Email Updates

Agency for Healthcare Research and Quality

Clinical reasoning of a generative artificial intelligence model compared with physicians.

Connect With Us

Sign up for Email Updates

Agency for Healthcare Research and Quality

Submit Your Innovations

Continue as a Guest

Continue Logged In

Submit Your Innovations

Continue as a Guest

Continue logged in

New Users to the PSNet site