Combined Proactive Risk Assessment (CPRA) – 4-Step Technique Innovation Summary

Innovation Snapshot

Risk assessment is an important tool a healthcare organization may use to prevent patient harm. Researchers have found that analysis of multiple adverse events is more effective at preventing harm and revealing system vulnerabilities than investigating individual patient safety events.¹ The Veteran Health Administration (VHA) National Center for Patient Safety (NCPS) created the Combined Proactive Risk Assessment (CPRA) technique to analyze numerous patient safety reports, aggregate proactive risk assessments (e.g., Failure Mode and Effects Analysis), and combine them with reactive risk assessments (e.g., Root Cause Analysis) on a single topic across multiple facilities.² The objective of CPRA is to use available proactive risk assessments (PRA) and reactive risk assessments (RRA)s to proactively monitor risk, guide clinical practice, and share quality improvement plans across multiple facilities at the system level.² Results of a CPRA at VHA facilities yielded a unified process map diagram with main process steps and sub-steps depicting how the flow of outpatient blood draws and revealing failure modes within the process.²

Description of Innovation

The NCPS developed CPRA to evaluate patient safety reports and combine proactive and reactive risk assessments on a specific care area, in this case outpatient blood draws, across VHA facilities. The objective of CPRA is to utilize institutional databases to analyze patient safety data, proactively monitor risks, guide clinical practice, and inform quality improvement plans at the system level across multiple facilities. John Bender and Steve Kulju, two members of the NCPS team, report that NCPS piloted CPRA in 2020 using Failure Mode and Effects Analysis assessments from 3 VHA facilities to review patient safety data on outpatient blood draws from October 1, 2017, to September 30, 2019.² The technique was informed by a literature review and designed to combine proactive and reactive risk assessment techniques to form the basis of a more robust and comprehensive alternative method for locating system breakdowns in clinical practice across the VHA.² The impetus for this new technique was two-fold:1) to make patient safety information more approachable, and 2) to view a dashboard of patient safety data at the system level.

Bender and Kulju note that CPRA was designed to be an approachable, scalable, and generalizable technique applicable in various health settings without requiring sophisticated software.² CPRA was designed to build on traditional proactive risk assessment techniques to capitalize on data from the VHA data repositories and analyze trends.² The innovators share that the goal was to improve upon proactive risk assessments by using the Failure Mode and Effects Analysis process and mapping all events, reactive risk assessments, and adverse events to every process bucket to bring the data to life.

CPRA involves a 4-step process. In step 1, CPRA combines proactive risk assessments on a specific health topic (i.e., outpatient blood draws) to create a single process flow diagram alongside all possible potential failure modes.² The technique allows for the simultaneous comparison of proactive risk assessments and process flows (also known as clinical workflows) across facilities.² The resulting diagram is a depiction of the system clinical process across facilities. The flow diagram summarizes the process and reveals a list of all failure modes. If possible, in this first step, it is useful to observe the process and seek consultation from subject matter experts if available.²

In step 2, the team engages in a detailed, collaborative process to create concept sheets. Depending on the resources and skills available to search and create concept sheets, CPRA can be conducted using a simple spreadsheet, a SQL Server to combine fields into one searchable field, or natural language processing software. Concept sheets define all key terms and note which items should be included or excluded.² A unique concept sheet is created for every process step, sub-step, and failure mode of the model to sort the data from the proactive risk assessments and patient safety report data.² The innovators note that the keyword search technique, which involves queries, searching, and tabulating terms from open text fields, reduces cost and technology needs by avoiding the use of coding or structured language processing software. One coder designs the concept sheets, and then the team compares results to ensure they interpret the open text fields the same way. The team calculated that open text data fields were translated into concept sheets with 85% accuracy or better.² A final audit of subsamples allows for terms to be refined if needed.²

In step 3, the final diagram from step 1 is studied with patient safety data and key terms defined to reveal system breakdowns and categorize harm.² This allows for further failure modes to be noted and topic and safety trends to be studied at the system level.²The team laid a subset of the patient safety data over each process step until all model steps were captured. The team tabulated counts of patient safety reports that fell into each step to quantify where most issues arose.²

Finally, step 4 involves creating summary charts to investigate which process steps have the most patient safety reports, which failure modes are most frequent, and any changes in the percentages of patient safety reports over time.² Year-to-year charts allow for data to be captured, sorted, and sifted to see what patient safety improvements have happened, which processes have deteriorated, and what new processes have taken hold over time.² All results are shared with additional team members and leadership to inspire continuous quality improvement dialogues.²

Published Results

NCPS analyzed 1,226 patient safety reports focused on outpatient blood draws across 3 facilities using the CPRA technique.² NCPS created a process flow diagram aggregating the 3 proactive risk assessments on outpatient blood draws.  There were 7 process steps, 35 subprocess steps with some variation between sub-steps, and some unique processes reported from each facility. The process flow diagram clearly shows the number of patient safety reports tracked in each step according to the database.  Approximately one-third of failure modes (n=14) were identified by at least one facility, and about three-quarters of failure modes (n=32) were not analyzed by any of the 3 facilities in the proactive risk assessments but were listed in process steps captured from patient safety reports. NCPS concluded that when all 3 proactive risk assessments were analyzed together, there were a total of 32 unique failure modes.² Finally, the team analyzed the safety report events overlaid onto the diagram, and found that 85.8% of events were from just 3 of the 7 process steps. These specific results informed tailored continuous quality improvement efforts.

Planning and Development

The keys to success of any patient safety innovation or intervention include leadership and staff buy-in, staff involvement in planning and development, and a establishing a data infrastructure to collect process and/or outcomes measures to track performance and drive improvement. Specific to this innovation, the following planning and development activities should occur prior to implementation:

• Establish a data repository with proactive risk assessment and patient safety reports.
• Design a project management structure with the following steps: 
  (1) pick a topic
  (2) choose your team*
  (3) plan the project steps
  (4) conduct the analysis
  (5) create a final report with action plans
  (6) monitor success.;

*It is recommended to form a small, dedicated team of 3 members with expertise in either data analysis, clinical processes, patient safety, or project management.

The resources and skills used to support implementation of this innovation include:

• Adequate number of project staff, allotted time and resources to conduct the CPRA analysis.
• High quality data submitted by clinical experts with structured data fields shared across the facility
• Data in the form of free text or open text data fields to be queried to create concept sheets
• Qualitative and quantitative data skills for creating concept sheets and charts
• Data analysis skills to conduct cross-tabulation to see results by failure modes
• Software for spreadsheets, natural language processing, and diagram creation
• Team meetings among coders to write queries for the concept sheets
• Consultation with subject matter experts (optional)
• Input from patient advocates and/or patient safety committee (optional)

Implementation Costs And External Funding Support

John Bender and Steve Kulju report that no additional funding was needed to conduct the CPRA.

Sustaining the Innovation

Sustaining an innovation requires tracking and monitoring innovation specific data and using that data to facilitate continuous quality improvement efforts. It also requires updating the workflows as needed to better integrate the innovation into the standard activities and processes. CPRA is a repeatable technique; it can be used with an indefinite type of datasets spanning across a number of patient safety issues based on organizational needs and priorities. Sustaining the changes made because of CPRA could include an ongoing quarterly review of data by different topics.

References

1. Alexander M, Bagian T, Williams LC. Approaching proactivity armed with evidence from reactivity. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care. 2014; 3(1):92-98). [Available at]
2. Bender JA, Kulju, S, Soncrant C. Combined proactive risk assessment: unifying proactive and reactive risk assessment techniques in health care. Jt Comm J Qual Patient Saf, 2022;48(6):326-334. [Available at]