Veterans Health Administration Stratification Tool for Opioid Risk Mitigation (STORM) Shows Promise for Targeting Prevention Interventions to Reduce Mortality in Patients Who Are Prescribed Opioids

January 7, 2022

Innovation
Contact Innovator

Summary

The Veterans Health Administration (VHA) Stratification Tool for Opioid Risk Mitigation (STORM) decision support system and targeted prevention program were designed to help mitigate risk factors for overdose and suicide among veterans who are prescribed opioids and/or with opioid use disorder (OUD) and are served by the VHA.¹ Veterans, particularly those prescribed opioids, experience overdose and suicide events at roughly twice the rate of the general population.^1,2

The STORM decision support system uses data extracted from VHA electronic medical records and predictive analytics to facilitate the identification of patients at high risk of experiencing overdose and suicide events. The STORM decision support system can also review risk factors for patients who are being considered for prescription opioid therapy. STORM prioritizes patients for monitoring and intervention according to their modeled risk and aids clinicians by displaying a patient’s risk factors and associated evidence-based risk mitigation interventions. Note that the target population does not include patients with OUD in medication-assisted treatment (MAT).

Many patients with OUD and/or in prescription opioid therapy have complex medical and psychosocial needs (e.g., painful conditions, mental health challenges), resulting in interactions with multiple care providers. To address the complexity of a patient’s case, STORM aims to provide a holistic intervention that includes multiple care providers and accounts for multiple parts of the patient’s history and medical profile.³ Under the STORM-based targeted prevention program, an interdisciplinary team of clinicians, including those with expertise in pain and behavioral health, conduct case reviews for patients identified to be at the highest risk of overdose and/or suicide and implement treatment changes or share recommendations with the patients‘ providers.

The STORM predictive model validation found that the risk model, informed by a multitude of types of complex clinical data, identified nearly twice as many patients who would go on to have an adverse event as did the previously used approaches.

The VHA completed a three-year randomized program evaluation of the implementation of the national STORM-based targeted prevention program. Preliminary results indicate that mandating that very high-risk patients receive an interdisciplinary review was associated with a decrease in all-cause mortality among identified patients in the 127 days after identification by the decision support system.⁴

The STORM decision support system and targeted prevention program were developed and implemented in the context of relatively high rates of opioid prescribing to veterans and overall rising opioid-involved overdose mortality in the U.S. population. In the last 10 years, overdose deaths have more than doubled in the United States.⁵ As one response to the problem, the 2016 Comprehensive Addiction and Recovery Act requires the VHA to improve opioid therapy strategies and to ensure responsible prescribing practices. STORM is one of several VHA overdose prevention initiatives that include the distribution of naloxone, efforts to reduce opioid prescribing, and introduction of pain management clinical review and support teams.⁵

Innovation Patient Safety Focus

This innovation focuses on mitigating overdose and suicide risks associated with opioid prescribing to veterans by providing clinicians information on individual patient risk factors and mitigation strategies that target these risk factors. Additionally, the most at-risk patients are elevated to a more intensive level of review, and their cases are assessed by an interdisciplinary team.

Resources Used and Skills Needed

VHA support for implementation includes training, technical assistance, and academic detailing for clinicians/prescribers. The STORM team (a group of Veterans Affairs staff who developed the initiative under the VHA Office of Mental Health and Suicide Prevention) emphasizes that successful implementation of STORM requires having a dedicated informatics team to manage all data analysis and reporting components of the STORM system. The team should be available for the life of the program. While much of the system can be automated, throughout the initiative there is occasional need for updates, such as when medical coding systems change, when new clinical practice guidelines need to be incorporated, or the model is recalibrated with new data.

Additionally, the initiative requires support from site leadership, access to robust and varied data (e.g., demographic, diagnostic, pharmacy, and healthcare utilization data), dedicated time for the staff to conduct the initiative (e.g., adequate patient encounter time), and “local champions” who help coordinate the initiative at each implementation site.

Effective implementation of the initiative also requires the participation of multidisciplinary teams. Teams must include a facility lead and representation from the facility’s substance abuse disorder (SUD) program or a mental health provider who can facilitate rapid engagement in care if appropriate. Additional subject matter experts may be asked to participate in the risk review process when indicated by patient care needs. These providers might be from disciplines such as pain management, pharmacy, primary care, suicide prevention, or patient advocacy. These teams work together in clinical case reviews to coordinate and improve the patient’s care.

Use By Other Organizations

Clinical decision support systems that integrate predictive modeling are not widely adopted. However, results from STORM indicate that previously identified and novel risk indicators that are readily available in electronic medical records (EMRs), can be used to identify patients at risk for overdose or suicide-related events and provide clinicians with comprehensive, actionable information to mitigate risk.

Efforts are under way to implement the STORM decision support platform for active-duty patients under the Defense Health Agency (DHA) opioid registry, and to pass information about risk factors from the U.S. Department of Defense (DOD) to VHA when service members transition out of DOD. This will be used internally in DHA to help guide safe pain care for service members on active duty and to improve early identification of risk for overdose and suicide in new veterans as they transition from the DHA to the VHA.

Date First Implemented

2017

Problem Addressed

In the United States, opioid-involved overdose deaths rose from 21,088 in 2010 to 49,860 in 2019.⁹ The problem is compounded in the veteran population due to high rates of post-traumatic stress disorder, chronic pain and other risk factors for SUDs, overdose, and suicide.¹⁰ For example, the suicide rate in veterans has been increasing in the last 10 years, rising 35.9% from 2001 to 2019, and is nearly twice that of the general population (30.4 per 100,000 compared to 17.0 per 100,000 among nonveterans in the same age bracket).¹¹ Researchers have looked at the possible link between suicide and prescription pain medications. In a 2017 U.S. Department of Veterans Affairs (VA) study of nearly 124,000 veterans, those receiving the highest doses of opioid pain relievers were more than twice as likely to die by suicide, compared with those receiving the lowest doses.¹² Overdose is another growing problem that disproportionately impacts veterans. Historically, veterans treated in the VHA were almost twice as likely to die by overdose as the general population.¹³

The Stratification Tool for Opioid Risk Mitigation (STORM) includes a predictive model that mines data from VHA medical records to estimate patient risk of overdose and suicide, based on similarities to patients who experienced overdose or suicide attempts in the past. Unlike previous risk identification strategies, the STORM predictive model includes chronic health conditions, adverse events, and healthcare events (e.g., emergency department visits that are available in a patient’s health record), in addition to widely accepted risk factors (e.g., mental health conditions, SUDs). The STORM decision support system incorporates predictive analytics and electronic health record (EHR) data into an informatics tool that has real-time results and pairs risk information with tailored recommendations for evidence-based mitigation strategies (e.g., suicide safety planning) applicable to each specific patient.

Description of the Innovative Activity

STORM helps clinicians/prescribers to identify patients most at risk for overdose and suicide and provides mitigation strategies to address risk factors for overdose and suicide and improve the effectiveness of pain care. The population management component of the STORM decision support system targets the population of VHA patients with an active opioid prescription or who have been treated for an opioid use disorder diagnosis in the last year. Key features include the following:

Data that populates a clinical decision support tool that is updated nightly
Estimates of an individual patient’s risk for overdose/suicide-related adverse events or death, based on a predictive model
A suite of dashboards to assist clinicians with opioid risk evaluation and mitigation
Promotion of patient-centered opioid risk mitigation strategies from the most current VA/DoD clinical practice guidelines. Example strategies include:
- - Review of risk factors (e.g., benzodiazepine prescriptions, previous adverse events, mental health and medical diagnoses, opioid dose)
  - Individualized risk mitigation strategies, including non-pharmacological treatment options employed and/or to be considered
  - Urine drug screening
  - Suicide prevention safety planning
  - Medications for opioid use disorders
  - Opioid overdose education and naloxone distribution
  - Review of patients’ upcoming appointments and current providers to facilitate care coordination
Comprehensive Addiction and Recovery Act (CARA)-mandated pre-opioid initiation reviews including case review for patients considering initiating opioid therapy
Mandatory clinical case review by a multidisciplinary team for the 5% of patients with the highest risk scores
The model is recalibrated periodically and incorporates mortality data which is available on a two-year lag.

Case reviews are conducted by an interdisciplinary team and include follow-up to ensure a treatment plan is optimized as needed. The providers who initiate the risk review vary by site. CARA-mandated point-of-care reviews are intended to be done by the prescriber or someone closely involved in the patient’s care. Prescribing protocols follow the A/DOD Clinical Practice Guideline for Opioid Therapy for Chronic Pain.¹⁴ VHA monitors dispensing practices system-wide and coordinates pain management and provides patient and provider education, opioid tapering programs, and non-pharmaceutical interventions like acupuncture and behavior therapy.

While the study population for the randomized program evaluation of the STORM targeted prevention program was composed primarily of veterans, STORM may be relevant for any population using opioids, and the tools are being expanded for use outside of the VHA.¹⁵

Context of the Innovation

STORM was developed and implemented in accordance with federal law and in the context of years of relatively high rates of opioid prescribing to veterans and overall rising opioid-involved overdose mortality in the United States, where deaths have more than doubled in the last 10 years.¹⁶ In response to high rates of overdose and opioid-related harm, in 2016 Congress passed CARA, which requires providers to improve opioid therapy strategies and to ensure responsible prescribing practices. STORM is one of many national opioid safety initiatives implemented in the last 5 to 10 years. In 2013, the VHA launched the Opioid Safety Initiative to promote the safe and effective use of opioid analgesics. In addition to STORM, under the Opioid Safety Initiative, the VHA has worked to distribute naloxone, reduce opioid prescribing, and implement use of pain management teams.¹⁷

There were other screening tools to identify high-risk patients in use at the time of the implementation of the STORM predictive model; however, the STORM team sought to improve the usability and accuracy of these tools. With these aims, the STORM team incorporated data from the VHA EHR and more sophisticated and rapid data processing into the calculation of risk scores. Additionally, STORM was designed for a specific population (i.e., veterans) with mitigation strategies that are intended to be actionable and easily implemented into routine clinical processes.¹⁸

Results

Pre-Implementation: To develop the predictive model the STORM team used 2010 and 2011 VHA electronic medical record data to identify factors (e.g., history of overdose, receiving mental health treatment) found to be associated with overdose or self-harm events. Using this information, the team estimated that targeting the top 100,000 patients with the highest risk scores (8.8%) would identify 50.2% of the patients who would have an overdose or suicide event, while incorrectly identifying only 7.9% of that risk cohort.

Targeted Prevention Program Implementation: The implementation of the STORM targeted prevention program used a stepped-wedge design to stage implementation and evaluation including 64,783 patients. Implementation of the case review mandate initially focused on the top 1% of patients based on risk scores at all facilities. At 9 months, a randomly selected half of facilities expanded the case review mandate to cover patients in the top 5% based on risk score. At 15 months, the other half of facilities expanded the case review mandate to cover patients in the top 5%. Published results are forthcoming but preliminary results indicate that the mandate for interdisciplinary reviews in the expanded population (i.e., those between the top 1% and 5% of estimated risk) was associated with a decrease in all-cause mortality within 127 days in the targeted patients. Compared to patients in the top 1% to 5% risk estimates who were not mandated to receive a clinical review, the odds ratio from adjusted logistic regression models of serious adverse events was 0.78 (confidence interval [CI]: 0.65 to 0.93) for mortality, and 0.99 (CI: 0.65 to 0.93) for any documented severe adverse event per the VHA medical record. The odds ratio for case review was 5.13 (CI: 3.64 to 7.23), meaning that during the study period, targeted patients mandated for a case review were approximately 5 times more likely to receive a case review than those with similar risk when the mandate did not apply. The size expansion was a challenge for sites as they adjusted to the additional volume of reviews and coordinating between patients’ care providers. An implementation study found that the level of national oversight for implementing the initiative did not impact the sites’ implementation strategies.

Expansion: VA and the DOD have collaborated to validate a matching predictive model on the active-duty population. Efforts are under way to implement the STORM predictive model and decision support elements for active-duty patients in the DHA opioid registry. DHA is also providing VA with information about risk factors when service members transition out of DOD, and VA is incorporating this into the STORM risk estimates and decision support. This is being used internally in DHA to help guide safe pain care for service members on active duty and to improve early identification of risk for overdose and suicide in new veterans as they transition from the DHA to the VHA.

Planning and Development Process

For the pre-implementation planning phase, the STORM team recommends that interested groups focus on the following:

Plans for academic detailing
Goal setting (e.g., expanding interdisciplinary opioid risk review teams to reach a broader population of high-risk patients)
Formation of interdisciplinary clinical review teams
Processes to review and improve care for the patients estimated to be at the very highest risk of overdose or suicide-related events or death
Processes for data-based case review prior to initiating an opioid prescription
Identification of contact people/local champions to help lead the initiative and work with the VHA STORM support team
Processes for case reviews of patients with new opioid prescriptions in clinical settings that commonly initiate opioids (e.g., emergency department, surgery, dental)

Additionally, sites should ensure that they have access to the appropriate technology and data resources. While the VHA STORM program uses several data sources (demographic, diagnostic, pharmacy, and healthcare utilization data), it can be modified to incorporate the data available within a given health system. Securing the dedicated time of an information technology/data analysis/informatics team is important to get the program started.

Resources Used and Skills Needed

Funding Sources

The development, maintenance, and national implementation of STORM were financially supported by the VHA Office of Mental Health and Suicide Prevention. The randomized program evaluation was supported by the U.S. Department of Veterans Affairs’ VHA Office of Research and Development (HSR&D SDR 16-196; QUERI PEC 16-001). The VHA’s randomized program evaluation of STORM included four different VHA operational and research partners:

Office of Mental Health and Suicide Prevention (OMHSP)
Partnered Evidence-based Policy Resource Center (PEPReC)
Center for Health Equity Research and Promotion (CHERP)
New England Veterans Engineering Resource Center (VERC)

Getting Started with This Innovation

The following are first steps in the implementation of risk identification and risk mitigation tools as well as preparing for multidisciplinary clinical case reviews.

Send out notice to participating sites explaining the intervention and providing resources.¹⁹
Conduct training.
Identify local champions and interdisciplinary review team members.
Establish and clarify protocols for care coordination across services.
Develop, establish, and track goals.
Refine the tool based on user feedback (e.g., try to fix any “bugs” promptly).
Track and measure successes.

Sustaining This Innovation

To sustain the program, the STORM team cites the following:

Having “champions” at every site
Giving feedback and sharing best practices, metrics, and successes
Maintaining general commitment to the issue, including:
- Staff who are dedicated to the initiative, locally and nationally
- Use of a shared value statement
- Adequate funding

Some additional tips for successful sustainment of the intervention include keeping the patient at the center of all development, ensuring treatment planning is collaborative and includes the range of the patient’s care providers, focusing on the patient’s treatment plan, using risk mitigation interventions, considering additional strategies, and ensuring staff are adequately trained and have support for working with complex patients.

References/Related Articles

Chinman M, Gellad WF, McCarthy S, et al. Protocol for evaluating the nationwide implementation of the VA Stratification Tool for Opioid Risk Management (STORM). Implement Sci. 2019;14(1):5. Published 2019 Jan 18. doi:10.1186/s13012-019-0852-z

Minegishi T, Frakt AB, Garrido MM, et al. Randomized program evaluation of the Veterans Health Administration Stratification Tool for Opioid Risk Mitigation (STORM): a research and clinical operations partnership to examine effectiveness. Subst Abus. 2019;40(1):14-19. doi:10.1080/08897077.2018.1540376

Minegishi T, Garrido MM, Pizer SD, Frakt AB. Effectiveness of policy and risk targeting for opioid-related risk mitigation: a randomised programme evaluation with stepped-wedge design. BMJ Open. 2018;8(6):e020097. Published 2018 Jun 27. doi:10.1136/bmjopen-2017-020097

Oliva EM, Bowe T, Tavakoli S, et al. Development and applications of the Veterans Health Administration's Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv. 2017;14(1):34-49. doi:10.1037/ser0000099

Rogal SS, Chinman M, Gellad WF, et al. Tracking implementation strategies in the randomized rollout of a Veterans Affairs national opioid risk management initiative. Implement Sci. 2020;15(1):48. Published 2020 Jun 23. doi:10.1186/s13012-020-01005-y

U.S. Department of Veterans Affairs, Veterans Health Administration, Pharmacy Benefits Management Academic Detailing Service. Updated February 2019. Accessed September 30, 2021. VA National Academic Detailing Service: Patient Education Materials. https://www.pbm.va.gov/PBM/AcademicDetailingService/Documents/Academic_Detailing_Educational_Material_Catalog/AcademicDetailingSTOPPAINMaterials.pdf

U.S. Department of Veterans Affairs (VA), U.S. Department of Defense (DoD). VA/DoD Clinical Practice Guideline for Opioid Therapy for Chronic Pain, Version 3.0. 2017. Accessed September 30, 2021. https://www.va.gov/HOMELESS/nchav/resources/docs/mental-health/substance-abuse/VA_DoD-CLINICAL-PRACTICE-GUIDELINE-FOR-OPIOID-THERAPY-FOR-CHRONIC-PAIN-508.pdf

Footnotes

Bohnert AS, Ilgen MA, Galea S, McCarthy JF, Blow FC. Accidental poisoning mortality among patients in the Department of Veterans Affairs Health System. Med Care. 2011;49(4):393-396. doi:10.1097/MLR.0b013e318202aa27
2021 National Veteran Suicide Prevention Annual Report. Office of Mental Health and Suicide Prevention, US Dept of Veterans Affairs; 2021. Accessed November 19, 2021. https://www.mentalhealth.va.gov/docs/data-sheets/2021/2021-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-9-8-21.pdf
Seal KH, Shi Y, Cohen G, et al. Association of mental health disorders with prescription opioids and high-risk opioid use in US veterans of Iraq and Afghanistan. JAMA. 2012;307(9):940-947. doi:10.1001/jama.2012.234
Strombotne K, Legler A, Minegishi T, Trafton J, Oliva E, Lewis E, Sohoni P, Garrido M, Pizer S, Frakt A, Rogal S. VHA Stratification Tool for Opioid Risk Mitigation (STORM) randomized program evaluation: interdisciplinary reviews associated with decrease in all-cause mortality and other lessons learned. Poster presentation.
National Institute on Drug Abuse. Trends and statistics: overdose death rates. January 29, 2021. Accessed September 30, 2021. https://www.drugabuse.gov/drug-topics/trends-statistics/overdose-death-rates
Sandbrink F, Oliva EM, McMullen TL, et al. Opioid prescribing and opioid risk mitigation strategies in the Veterans Health Administration. J Gen Intern Med. 2020 Dec;35(Suppl 3):927-934. doi:10.1007/s11606-020-06258-3
Oliva EM, Bowe T, Tavakoli S, et al. Development and applications of the Veterans Health Administration's Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv. 2017;14(1):34-49. doi:10.1037/ser0000099
Strombotne K, Legler A, Minegishi T, Trafton J, Oliva E, Lewis E, Sohoni P, Garrido M, Pizer S, Frakt A, Rogal S. VHA Stratification Tool for Opioid Risk Mitigation (STORM) randomized program evaluation: interdisciplinary reviews associated with decrease in all-cause mortality and other lessons learned. Poster presentation.
National Institute on Drug Abuse. Trends and statistics: overdose death rates. January 29, 2021. Accessed September 30, 2021. https://www.drugabuse.gov/drug-topics/trends-statistics/overdose-death-rates
Liberto JG. The opioid crisis: treating our nation’s veterans. University of Maryland School of Nursing. Accessed September 30, 2021. https://www.nursing.umaryland.edu/media/son/academics/professional-education/The-Opioid-Crisis-Treating-Our-Nations-Veterans-pdf.pdf
2021 National Veteran Suicide Prevention Annual Report. Office of Mental Health and Suicide Prevention, US Dept of Veterans Affairs; 2021. Accessed November 19, 2021. https://www.mentalhealth.va.gov/docs/data-sheets/2021/2021-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-9-8-21.pdf
Ilgen MA, Bohnert ASB, Ganoczy D, Bair MJ, McCarthy JF, Blow FC. Opioid dose and risk of suicide. Pain. 2016;157(5):1079-1084. doi:10.1097/j.pain.0000000000000484
Bohnert AS, Ilgen MA, Galea S, McCarthy JF, Blow FC. Accidental poisoning mortality among patients in the Department of Veterans Affairs Health System. Med Care. 2011;49(4):393-396. doi:10.1097/MLR.0b013e318202aa27
U.S. Department of Veterans Affairs (VA), U.S. Department of Defense (DoD). VA/DoD Clinical Practice Guideline for Opioid Therapy for Chronic Pain, Version 3.0. 2017. Accessed September 30, 2021. https://www.va.gov/HOMELESS/nchav/resources/docs/mental-health/substance-abuse/VA_DoD-CLINICAL-PRACTICE-GUIDELINE-FOR-OPIOID-THERAPY-FOR-CHRONIC-PAIN-508.pdf
Comprehensive Addiction and Recovery Act of 2016, Pub L No. 114-198, 130 Stat 695 (2016). https://www.congress.gov/114/plaws/publ198/PLAW-114publ198.pdf The section mandating the pre-opioid initiation reviews is Section 911(a)(2).
National Institute on Drug Abuse. Trends and statistics: overdose death rates. January 29, 2021. Accessed September 30, 2021. https://www.drugabuse.gov/drug-topics/trends-statistics/overdose-death-rates
Sandbrink F, Oliva EM, McMullen TL, et al. Opioid prescribing and opioid risk mitigation strategies in the Veterans Health Administration. J Gen Intern Med. 2020;35(Suppl 3):927-934. doi:10.1007/s11606-020-06258-3
Oliva EM, Bowe T, Tavakoli S, et al. Development and applications of the Veterans Health Administration's Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv. 2017;14(1):34-49. doi:10.1037/ser0000099
Oliva E, Tucker D, Kazanis W, Sim A. Leveraging Big Data to improve opioid risk mitigation across three electronic health records. Presentation at: Rx Drug Abuse & Heroin Summit.

Date Verified by Innovator

Date Verified by Innovator indicates the most recent date the innovator provided feedback during the review process.

December 8, 2021

FYI: You may notice that PSNet Innovations Exchange has recently been updated (June 2022) to remove the evidence rating section. For more information or questions, please email psnetsupport@ahrq.hhs.gov.

The inclusion of an innovation in PSNet does not constitute or imply an endorsement by the U.S. Department of Health and Human Services, the Agency for Healthcare Research and Quality, or of the submitter or developer of the innovation.