Problem Addressed

The purpose of the innovation is twofold: (1) to construct an ASN to prevent missed and delayed diagnoses of colon cancer and lung cancer, and (2) to build a system from the point of test-ordering to the completion of guideline-based recommended follow-up care. The colon cancer safety net was constructed based on prior colonoscopy procedures. The lung cancer safety net was constructed based on radiologist recommendation for follow-up imaging when an incidental lung nodule is discovered by a radiologist during chest x-rays or CT scan interpretation.

Description of the Innovative Activity

An ASN is a collection of tools, reports, registries, and workflows built into an outpatient setting wherein the results lead to better test result management and a more reliable system of communicating key health information.¹ The goal of the ASN tools is to track abnormal test results and ensure medically necessary follow-up. Key components of the ASN innovation include working groups to steer the process, design, and deployment of reports and registries that combine electronic and manual entry tools, patient tracking, and outreach.  

Working groups must be convened to establish a meeting schedule, design agendas, and conduct the work of building the safety net. The focus of working group meetings is subject to change, and it may be expanded or altered based on preliminary findings and implementation needs. Working groups meet periodically; it may take several years to build tools that garner results. Quality improvement is a crucial aspect of ASN. The BWH Patient Safety Collaborative used a Plan-Do-Study-Act (PDSA) model of quality improvement to identify missing elements and illuminate key gaps strategically and systematically to continuously revise and improve the implementation.

Reports and registries must be designed by administrative teams and created by following an iterative process. The process typically starts by identifying key variables in electronic health records, and it determines what data needs to be collected. Then, it establishes reliable data collection. After all data components are captured reliably, the electronic health records team can design refreshable registries and reports that capture the data for review. The creation and validation of registries and reports can take six months with input from various clinical and administrative stakeholders. Furthermore, the BWH Patient Safety Collaborative team found that often manual chart review was necessary to validate report results and took close to six months to complete. This extended timeframe was caused by several factors, such as the nuanced evaluation required to examine how incidental lung nodules were mentioned in radiology reports, changing guidelines, and the necessary input from data and information technology teams to identify target patient populations. A combination of electronic and manual tools was necessary, including a natural language processing tool to build a report of patients with incidental lung nodules who required follow-up imaging or evaluation.

Patient tracking and outreach should include all stakeholders in the design process and should be refined to pursue results and clinical impact. After the innovation team developed colonoscopy registries, a chart review was conducted to confirm the correct roster of patients was identified. Patient outreach occurred in waves with two attempts. Hard-copy letters were mailed, or notices were sent electronically via patient portal account. A third attempt was made via phone call to the patient by a patient navigator. In the case of lung CT scan results, registries were developed, and then patient navigators reached out to the patient’s primary care provider and practice leadership via secure email (with up to three monthly reminders) to assess patient history and results to determine whether a follow-up chest CT was clinically necessary.

Context of the Innovation

Across the field of radiology, there is concern about clinically necessary but unscheduled radiologic exam orders.¹⁸ Completing unscheduled exams is integral to reducing diagnosis errors and avoiding a delay in diagnosis and treatment. The results of the preliminary ASN study illuminated several additional considerations and needs in the area of radiology that may have contributed to missed or delayed cancer diagnoses. For example, the following variations led to challenges in determining the need for follow-up imaging in the radiology report, including for lung nodule test results^19,20:

• A wide variation in the rates of recommendations for additional imaging among radiologists²¹
• Ambiguity in language radiologists may use to convey their recommendations¹⁷
• Variations in how radiologists convey diagnostic certainty in radiology reports
• A lack of a harmonized radiology report²²

Variations in radiologist follow-up recommendation rates may be due to lack of consensus on best practices even when evaluating published guidelines.²³ New work from the BWH Patient Safety Collaborative team has proposed a framework for optimal radiologist recommendations that requires three key aspects to ensure capture of appropriate information at time of assessment and support relevant follow-up.^17,23 The design requires radiologists to make an explicit recommendation that must include three key pieces of information: the modality and body part, time frame, and rationale for follow-up.^16,23,24

Results

Study authors tested the effectiveness of the ASN innovation by conducting chart reviews and using electronic health record data. The two outcomes of interest were (1) the proportion of patients who were scheduled for or completed a colonoscopy following safety net team outreach to the patient and (2) the proportion of patients for whom the safety net was able to identify and implement appropriate follow-up, as defined by scheduled or completed chest CT.¹

Adding in a follow-up letter and phone call outreach to patients increased the volume of scheduled and completed colonoscopies. In January 2018, sending letters to patients yielded 12 scheduled appointments and seven completed colonoscopies (58.3% completion rate).¹ As efforts were ramped up and a phone call to the patient was added, in August of 2018 scheduled appointments jumped to 43, to include 22 completed colonoscopies (51.2% completion rate). By March 2019, using both outreach letters and calls resulted in 113 scheduled appointments and 84 completed colonoscopies (74.3% completion rate).

Overall, the effectiveness of the colon cancer safety net was 44% and the lung cancer safety net was 56.9% when accounting for the proportion of patients who were effectively caught by the safety net by either being scheduled for a follow-up or successfully completing a follow-up on account of safety net intervention.¹ Additional results of this innovation include the creation of patient safety registries, the fostering of a fruitful and generative collaboration among an interdisciplinary team of clinicians and administrative staff, and the creation of new workflows that included patient tracking and outreach that improved patient follow-up. Qualitative structured interviews conducted with five physicians and three administrative directors indicated the ASN adoption process was “thoughtful” and “collaborative.”¹

Planning and Development Process

Key steps in planning and implementing the innovation are as follows:

• Bring together a strategic coalition of doctors, medical staff, and other professional administrative staff.
• Identify leadership roles by naming a medical director and a project coordinator for each specialty area.
• Seek support of frontline clinicians and leadership including physicians, administrative staff, and nurse leaders.
• Seek project management support.
• Create an implementation plan.
• Create patient tracking and outreach through registry development, chart review, and customized patient outreach via a clinic-determined method (hard copy letters, phone calls to client, or outreach to patient’s primary care provider).
• Use quality improvement process models to identify missing elements and/or to illuminate key gaps strategically and systematically (revise your plan and continue the cycle as needed).

Resources Used and Skills Needed

• Leadership buy-in and support for ambulatory patient safety
• Buy-in from a multidisciplinary stakeholder team
• An equity-informed, highly reliable organizational framework
• Subject matter expertise interpreting imaging exams, such as x-rays or CT scans, and performing colonoscopies
• Workflow redesign
• Information technology registries, tools, and analytic capabilities to track each clinically necessary recommendation to resolution and to evaluate progress, impact, and demonstrate value to sustain the innovation

Funding Sources

Preliminary research funding came from CRICO, which is the medical professional liability insurer for BWH. Then, the ASN was funded through a grant program. Due to successful implementation and results, the innovator recognized value in the ASN innovation and subsequent institutional support after grant funding came to an end. For instance, duties conducted by grant staff were transitioned to operational teams. In fact, the success of the ASN has instilled a culture of innovation within the local hospital system upon which the author team has been able to continue to innovate and build even more effective tools to support patient safety. Once they were able to demonstrate value through clinical outcomes, they have been able to apply for additional grant funding. A recent grant from AHRQ (Diagnostic Excellence Center on Diagnostic Errors, DECODE, 2022-2026) is enabling CEBI investigators to assess the impact of disseminating the radiology safety net program across the Mass General Brigham system on diagnostic errors and patient harm.

Getting Started with This Innovation

Deploying a small pilot program to test minor changes prior to widespread scaling and implementation across the various primary care settings was essential to test their theory of change and to ensure they captured all key steps in the process and gained participation of all key stakeholders. Originally, the team believed the project would be straightforward. They felt it could be built in a registry and sufficiently supported by a human safety net team. However, when they applied technology to build the registry, the information did not exist in an actionable way. Without input from a primary care provider, they did not know other patient risk factors. Without knowing all the risk factors, the team could not determine clinical necessity for follow-up testing. Furthermore, if the radiologist did not indicate a timeframe for imaging follow-up, it was especially challenging to measure whether follow-up happened in a timely manner. The team quickly realized that definitions of timely and clinical necessity must be explicitly confirmed between a radiologist and the referring provider, what they called a collaborative care plan for the patient.^16,17,24 So, in starting with this innovation or any similar innovation, it is important to be prepared to engage in continuous quality improvement initiatives (like PDSA cycles) to track down all of the relevant data, to analyze it, and to generate an actionable plan based on quality data in order to close any gaps.

Sustaining This Innovation

Keys to sustaining this innovation are as follows:

• Seek and obtain support of hospital leadership.
• Build a coalition of professionals with medical, technological, and administrative expertise and leadership.
• Establish a just culture framework and ensure buy-in from all partners.
• Identify drivers of change to ensure patient safety and to minimize patient harm.
• Engage just culture principles and transform thinking from a root-cause analysis framework to a systems-focused framework to prevent future harms.
• Collect ongoing data to demonstrate value to leadership and potential funders.
• Find means to institutionalize the innovation through regular operating roles and responsibilities.
• Realize you may locate gaps that create a need to redesign current models of care.
• Utilize the strengths of human capital, clinical expertise, and technology to build and customize a patient safety innovation that works for your health issue and setting.
• Seek additional grant funding for further needed work and research.

References/Related Articles

Desai SP, Jajoo K, Taber K, et al. A quality improvement intervention leveraging a safety net model for surveillance colonoscopy completion. Am J Med Qual. 2022;37(1):55-64. 

Desai S, Kapoor N, Hammer MM, et al. RADAR: a closed-loop quality improvement initiative leveraging a safety net model for incidental pulmonary nodule management. Jt Comm J Qual Patient Saf. 2021;47(5):275-281.

Emani S, Sequist TD, Lacson R, et al. Ambulatory safety nets to reduce missed and delayed diagnoses of cancer. Jt Comm J Qual Patient Saf. 2019;45(8):552-557.

Kapoor N, Lacson R, Cochon LR, Boland GW, Khorasani R. Radiologists’ self-assessment versus peer assessment of perceived probability of recommending additional imaging. J Am Coll Radiol. 2020;17(4):504-510.

Lacson R, Odigie E, Wang A, et al. Multivariate analysis of radiologists’ usage of phrases that convey diagnostic certainty. Acad Radiol. 2019;26(9):1229-1234.

Lacson R, Healey MJ, Cochon LR, et al. Unscheduled radiologic examination orders in the electronic health record: a novel resource for targeting ambulatory diagnostic errors in radiology. J Am Coll Radiol. 2020;17(6):765-772.

Footnotes

1. Emani S, Sequist TD, Lacson R, et al. Ambulatory safety nets to reduce missed and delayed diagnoses of cancer. Jt Comm J Qual Patient Saf. 2019;45(8):552-557.
2. Giardina TD, King BJ, Ignaczak AP, et al. Root cause analysis reports help identify common factors in delayed diagnosis and treatment of outpatients. Health Aff (Millwood). 2013;32(8):1368-1375.
3. Ellis PM, Vandermeer R. Delays in the diagnosis of lung cancer. J Thor Dis. 2011;3(3):183-188.
4. Hanna TP, King WD, Thibodeau S, et al. Mortality due to cancer treatment delay: systematic review and meta-analysis. BMJ. 2020;371:m4087.
5. Gildea TR, DaCosta Byfield S, Hogarth DK, Wilson DS, Quinn CC. A retrospective analysis of delays in the diagnosis of lung cancer and associated costs. Clinicoecon Outcomes Res. 2017;9:261-269.
6. Morris AM, Rhoads KF, Stain SC, Birkmeyer JD. Understanding racial disparities in cancer treatment and outcomes. J Am Coll Surg. 2010;211(1):105-113.
7. Watson RA. Understanding racial disparities in cancer treatment and outcomes. J Am Coll Surg. 2011;212(1):131-132.
8. O’Keefe EB, Meltzer JP, Bethea TN. Health disparities and cancer: racial disparities in cancer mortality in the United States, 2000–2010. Front Public Health. 2015;3:51.
9. Abraham P, Bishay AE, Farah I, Williams E, Tamayo-Murillo D, Newton IG. (2021). Reducing health disparities in radiology through social determinants of health: lessons from the COVID-19 pandemic. Acad Radiol. 2021;28(7):903-910.
10. Bradley CJ, Given CW, Roberts C. Disparities in cancer diagnosis and survival. Cancer. 2001;91(1):178-188.
11. Danforth KN, Smith AE, Loo RK, et al. Electronic clinical surveillance to improve outpatient care: diverse applications within an integrated delivery system. EGEMS (Wash DC). 2014;2(1):article 9.
12. Jones D, Dunn L, Watt I, Macleod U. Safety netting for primary care: evidence from a literature review. Br J Gen Pract. 2019;69(678):e70-e79.
13. Heinrich A. What is ambulatory care? Learning more about the future of healthcare. Rasmussen University. Accessed June 1, 2023. https://www.rasmussen.edu/degrees/nursing/blog/what-is-ambulatory-care/#:~:text=By%20definition%2C%20ambulatory%20care%20is%20any%20same-day%20medical,in%20a%20hospital%20or%20facility%20that%20requires%20admission
14. Lacson R, Khorasani R, Fiumara K, et al. Collaborative case review: a systems-based approach to patient safety event investigation and analysis. J Patient Saf. 2022;18(2):e522-527.
15. Kapoor N, Lynch EA, Lacson R, et al. Predictors of completion of clinically necessary radiologist-recommended follow-up imaging: assessment using an automated closed-loop communication and tracking tool. Am J Roentgenol. 2022;220(3):429-440.
16. Guenette JP, Kapoor N, Lacson R, et al. Development and assessment of an information technology intervention to improve the clarity of radiologist follow-up recommendations. JAMA Netw Open. 2023;6(3):e236178.
17. Lacson R, Healey MJ, Cochon LR, et al. Unscheduled radiologic examination orders in the electronic health record: a novel resource for targeting ambulatory diagnostic errors in radiology. J Am Coll Radiol. 2020;17(6):765-772.
18. Lacson R, Odigie E, Wang A, et al. Multivariate analysis of radiologists’ usage of phrases that convey diagnostic certainty. Acad Radiol. 2019;26(9):1229-1234.
19. Trinh TW, Shinagare AB, Glazer DI, et al. Radiology report template optimization at an academic medical center. Am J Roentgenol. 2019;213(5):1008-1014.
20. Cochon LR, Kapoor N, Carrodeguas E, et al. Variation in follow-up imaging recommendations in radiology reports: patient, modality, and radiologist predictors. Radiology. 2019;291(3):700-707.
21. Shinagare AB, Lacson R, Boland GW, et al. Radiologist preferences, agreement, and variability in phrases used to convey diagnostic certainty in radiology reports. J Am Coll Radiol. 2019;16(4, Part A):458-464.
22. Kapoor N, Lacson R, Cochon LR, Boland GW, Khorasani R. Radiologists’ self-assessment versus peer assessment of perceived probability of recommending additional imaging. J Am Coll Radiol. 2020;17(4):504-510.
23. Hammer MM, Kapoor N, Desai SP, et al. Adoption of a closed-loop communication tool to establish and execute a collaborative follow-up plan for incidental pulmonary nodules. Am J Roentgenol. 2019;19:1-5.
24. Abbasi N, Lacson R, Kapoor N, et al. Development and external validation of an artificial intelligence model for identifying radiology reports containing recommendations for additional imaging. Am J Roentgenol. 2023.