In Conversation with David W. Bates about Are We Safer Today?

Editor’s note: David W. Bates, MD, MSc, is the Medical Director for Clinical and Quality Analysis for MGB Healthcare and co-directs the Center for Artificial Intelligence and Bioinformatics in the Learning Healthcare System. He is also a Professor of Medicine at Harvard Medical School and a Professor of Health Policy and Management at the Harvard School of Public Health. We spoke to him about the state of patient safety, especially in light of past, landmark reports on patient safety that established the field.

Sarah Mossburg: Welcome, Dr. Bates. Please tell us about yourself and your current role. 

David Bates: I am a general internist and have done a lot of work on patient safety, preventable harm, and improving safety and quality. After 25 years, I recently stepped down as Chief of General Medicine at Brigham and Women’s Hospital. I am currently the Medical Director for Clinical and Quality Analysis for MGB Healthcare. I also co-direct the Center for Artificial Intelligence and Bioinformatics in the Learning Healthcare System or CAIBILS.

Sarah Mossburg: You recently published an article, "The Safety of Inpatient Health Care: A Long-Term Follow-Up for the Harvard Medical Practice Study," and I would like to get into the details of that. But a question on everyone's mind that follows from that is the big picture: Are we safer today? 

David Bates: I am not completely sure whether we are safer today. What we found in our recent study was that one in four patients suffers an injury during a hospitalization. That rate is too high, and there are many things we could do to improve it. We also did an outpatient study and found that about one in 15 patients suffers some type of harm over the course of a year. The latter study is the first big study that looked at patient harm prospectively in outpatient settings in some time.

In the Harvard Medical Practice Study, the harm rate was 3.7% in a random sample of hospitals from the state of New York in 1984. But many things are different between now and then. One is that medical charts are now electronic records, so it is much easier to find things that happened that may have harmed someone. The other big change is that we are now using triggers to find adverse events, which were not used in the Harvard Medical Practice Study. Because triggers are a more efficient way of finding adverse events, researchers can find a lot more.

Sarah Mossburg: Could you say more about your recent work to quantify the rates and kinds of adverse events that occur in medical care?

David Bates: We looked at not just the overall frequency but also the main causes of harm. Adverse drug events are still at the top of the list, both inside and outside the hospital. Other important factors are nursing-sensitive events, which include things like pressure injuries and falls. Those are particularly likely to be preventable among surgical patients. Hospital-acquired infections are still an issue, but they have decreased somewhat in frequency.

Sarah Mossburg: Interesting. You mention the trigger tool as a methodological change for identifying those adverse events. Could you expand on that?

David Bates: Trigger tools have been a major advance. They look for specific things that suggest someone may have had an adverse event. An example would be an unexpected transfer to an intensive care unit or the use of a drug that is an antidote. If a patient is prescribed an antidote to something, it is very likely that there was an adverse event. Another example is a positive Clostridium difficile test. C. difficile is usually a complication, so having a positive test strongly suggests that the person suffered an adverse event. Dr. David Classen of the University of Utah has done a series of studies that showed that with triggers, you can find adverse events much more efficiently than previously. As a result, trigger tools have been widely used.

Sarah Mossburg: So, is it fair to say it is now easier to find and document the adverse events, potentially confounding any comparisons we are making to older paper-based retrospective reviews?

David Bates: That is certainly the case to a degree. However, it is still interesting to look at what the absolute rates are and ask, “Are these the same or different, and how do I feel about the current rate?”

The one-in-four chance of having an adverse event is really too high. If I were getting on an airplane and was told that the chances of something bad happening before I got off were one in four, I would be very nervous about getting on the airplane. Health care needs to be better. It can do better. There are a variety of things that we already know that, if done, reduce the frequency of harm. Yet those are not always done routinely today, and we should do better with that.

Sarah Mossburg: Have there been any other methodological changes that have made recent work more or less accurate at assessing adverse events other than the use of trigger tools to identify them?

David Bates: People always ask how reliably you can assess whether an individual event was or was not an adverse event. We can do that now with a pretty high degree of reliability, with kappa’s of 0.5-0.8 typically, which is a very good level of reliability. Sometimes, people are concerned about whether safety studies are overestimating the frequency of events. But I would say, in many of these, it is pretty clear. If you fall out of bed and break a bone, that is an adverse event. It is a little more difficult to decide whether that was preventable or not, and that would depend on the read of the reviewer. But assessment of the presence of harm is reasonably reliable now. 

Sarah Mossburg: So, does it make more sense for us to prevent the much more common adverse events?

David Bates: Yes, definitely. Clearly, we should try to prevent fatal events too, but those are a more difficult target.

Sarah Mossburg: You mentioned that assessing preventability is more challenging because it depends upon the “read” of the reviewer. Could you say more about that?

David Bates: Assessment of preventability relies on the reviewer looking at an individual case and seeing whether something was done that was a clear error or was not done that should have been done. An example would be if a patient had an indication for deep venous thrombosis prophylaxis but did not get it. That would be considered a preventable event. Sometimes, for example, during surgery the scalpel slips, and something gets cut that should not have been cut. It is very hard to say whether that was preventable, especially by a non-surgeon. We typically call those unpreventable, but it is possible that with new approaches or the use of a different type of surgical technique, these ostensibly unpreventable events could be prevented.

Sarah Mossburg: I imagine it is challenging, for example, to determine preventability for some of the nursing-sensitive indicators, such as pressure injuries and falls, based solely on the documentation in the chart. 

David Bates: Yes, that is certainly true. That being said, I think that by using some new approaches, we might be able to prevent many of those injuries going forward. For example, with pressure injuries, wetness in the bed dramatically increases the risk that somebody will develop a dangerous pressure injury. There are now tools that allow you to put sensors in the bed so that you can detect when there is fluid or wetness and then intervene and change things.

Some of these sensors are expensive, but pressure injuries are extremely expensive. So, in the right situation, I am confident that it will be justifiable to use these approaches. You have to be selective in how you use them, of course.

Sarah Mossburg: Given changes in health care and our ability to measure adverse events, what data would you point to that suggests patient safety initiatives have improved patient safety?

David Bates: We have national data suggesting that some types of hospital-acquired infections have decreased, notably catheter-related bloodstream infections and ventilator-associated pneumonia. Catheter-associated UTIs (urinary tract infections), or CAUTIs, have been a more difficult target. So, things have not gotten better for all types of infections. But it seems like infections, where we have bundles that have been demonstrated to reduce risk, have worked especially well. The safe surgery checklist is another example of an intervention that has clearly reduced risk, although there is still a lot of risk associated with surgery.

Sarah Mossburg: So, bundles and checklists are effective tools to improve safety, based on the indicators?

David Bates: Yes, but they must be routinely applied, too. An example of that not being the case as much as it should be is deep venous thrombosis. We have very good guidelines about what to do in terms of prevention, yet the rates are still higher than they should be. That is because the prevention recommendations are not always applied consistently.

Every checklist or set of tools like that takes some time. It is hard for healthcare providers to remember all the various checklists, and they are not necessarily getting help to remember when a specific one should be done to follow them routinely. We should be reinforcing the use of safety tools with electronic health records, making specific suggestions that people follow certain approaches in specific settings. The issue now is that we have so many checklists that nobody can remember them all. One good example is the Ottawa ankle rules, which specify when you should get an ankle radiograph in the emergency room, a very specific setting. Most of the things that come into the emergency room are not ankle injuries. But if you are seeing a patient with an ankle injury, it should be underscored that there is a rule so that it is easy to use it. But that does not happen now at most institutions or even for the best checklists.

Sarah Mossburg: It seems like we are in a place where we have a lot of evidence, information, and tools to help guide safer care, but at the same time, disparities exist in how we are able to access that information at the moment that tools are needed. Is that an accurate statement?

David Bates: Yes, and healthcare providers are very busy. Safety scientists like Rene Amalberti find that healthcare providers are very short on time and are continually moving towards trying to do things faster and more efficiently, which pushes toward an unsafe space. If you do things too quickly, you make more mistakes, including forgetting to do some checks.

Sarah Mossburg: Some hospitals have reported zero harm for some types of adverse events, such as hospital-acquired infections. Do those findings suggest that patients are meaningfully safer, or are there other concerns or needs that those reports raise? 

David Bates: When I see a rate of zero harm, I am suspicious. There are times when that might be achieved, for example, for CLABSIs (catheter-related bloodstream infections). But my belief is that institutions reporting zero harm for certain things probably are not measuring the frequency of harm accurately (or measuring it at all). 

The way that organizations have traditionally measured adverse drug event rates is to use self-reported data. In studies that we and others have done, we’ve found about one in 20 of the adverse drug events was identified through self-report. So, you are underestimating the magnitude of the problem in your organization by a factor of 20. It is not surprising that people feel their medication safety practices are relatively safe when they are just not getting most of the signal. 

Sarah Mossburg: I see. So, you are saying that the way they are looking for adverse medication events misses the vast majority of them.

David Bates: Yes, exactly. That is not the case for falls. Hospitals do find out about nearly every fall. It is an obvious thing, although the definition can be tricky. For example, some hospitals don’t consider that a fall occurred if the patient doesn’t reach the ground. When a fall occurs, the nurse has to do a report. Pressure injuries are better assessed, although those assessments are very difficult to do, especially regarding staging. I think we will get better with new tools, for example, taking photographs of injuries and then using AI to figure out how severe the injury has been. There is not yet robust data showing that AI assessment of injuries makes a difference, but we and others are interested in it. It is likely to be helpful.

Sarah Mossburg: Based on your data, adverse drug events are one of the most common events and yet one of the most challenging for organizations to identify. 

David Bates: And yet there are commercial tools that measure adverse drug events very accurately. I think organizations should be required to put in measurement technology in this area. The evidence is that when we do that, things get better. For example, because it’s required, hospitals do a reasonably good job of detecting hospital acquired infections (HAIs), and rates of HAIs have fallen over time.

Sarah Mossburg: Evidence broadly suggests that adverse events continue to occur at high rates. What are some hopeful findings that point to increased safety or that demonstrate better safety is achievable? You just mentioned that we are on the cusp of being able to access information much more readily via the electronic health record. What other thoughts do you have on that?

David Bates: The sensing technologies are going to be helpful. I mentioned detecting wetness in beds. But a very big area where I think we will be able to see substantial improvement is detection of decompensation in patients. We and others have done trials that show if you take the data from the electronic record, you can find individuals who are decompensating and find them sooner than you otherwise would. If you intervene earlier, you can reduce mortality. The Kaiser team showed in a very large study that they were able to do that. That was published in the New England Journal a few years ago.¹ There are a number of other studies along those lines. So, I am really excited about detection of decompensation.

Another frontier is diagnostic error. There is a lot of evidence that diagnostic errors result in harm both inside and outside the hospital. Much of the work so far has been done outside the hospital, but recent work suggests there are lots of problems inside the hospital, too. We do not have as clear an idea about how exactly to prevent that yet, but there are a variety of strategies being evaluated that will be helpful in terms of decreasing the likelihood of diagnostic error. The first thing you have to figure out is if the diagnosis is wrong, and I think artificial intelligence will help us with that. Then clinicians will be able to identify those situations and work on figuring out exactly what is going on when there are cases like that.

Let me give you one other good example. In the outpatient setting, one of the biggest causes of harm is that somebody has a diagnostic test that suggests they may have cancer, and sometimes it takes longer than it should to follow up those tests. Sometimes people are even lost in follow-up. Tools that ensure very high degrees of follow-up are helpful. Again, Kaiser has done some great work. They have put those tools in place for almost fifty conditions in the outpatient setting, and they and others have shown that an approach of putting in “ambulatory safety nets” really improves care. Another issue is X-rays that suggest there are some worrisome findings. Dr. Ramin Khorasani at our institution has developed a set of tools that enable you to find radiographs that require follow-up and then track them to ensure (1) that the ordering physician is aware and (2) that the appropriate follow-up actually occurs.² Now we are approaching 100% in terms of how many things like worrisome pulmonary nodules get followed up appropriately.

Sarah Mossburg: Those examples lead to my next question. You co-wrote a 2018 Health Affairs article, "Two Decades Since To Err Is Human: An Assessment of Progress and Emerging Priorities in Patient Safety." You noted progress in some areas but very little in others. Could you talk about what those areas are and the differences between them?

David Bates: Infection is the poster child for an area where we have made improvements. The keys there were developing checklists or bundles for specific types of important infections. The catheter-related bloodstream work, which was led by Dr. Peter Pronovost, showed that not only could you do well in preventing these infections, but you could also spread progress to hospital systems and, eventually, to all of Michigan. For hospital-acquired infections, people developed bundles. They are relatively simple and used in very specific situations. People then developed change packages which enabled other organizations to adopt the same practices.

But adverse drug events, for example, differ from infection prevention in that there are lots of different drugs that are used. This area is not amenable to a single checklist. The average academic center uses roughly 5,000 drugs, which makes things really complicated. Most of the adverse events are not caused by two or three drugs; they are caused by lots of different drugs. 

Decompensation is also an area that has not really been addressed until recently. That requires you to be able to get data from the electronic health record in real-time, process it, and look to see if things are changing from where the patient was. That is computationally difficult and it is hard even with today’s electronic health records. It is just a more complex problem than hospital-acquired infections.

Sarah Mossburg: You mentioned a change package. Could you talk briefly about what that is?

David Bates: Change packages are sets of tools that enable an institution to take up a specific practice and implement it in their setting. They are critical in terms of enabling spread, which is very important. We have lots of instances in which one or two institutions have made a lot of progress in one area but not improved safety performance overall because those practices have not spread sufficiently. Robust change packages are critical.

Sarah Mossburg: One of the things you have noted is that variation in how well hospitals prevent falls or pressure injuries suggests that failures in implementation may have a role in why some of those harms continue to occur at higher rates in some hospitals. What would you say is needed to improve implementation and dissemination of best practices?

David Bates: A couple of things would be helpful. The first would be some sort of requirement that all organizations use their electronic record to find adverse events and then put them into a dashboard so they can see how they are doing and how much harm they are creating. If organizations have that data, I think they will respond appropriately. For example, if you know you have a high rate of pressure injuries resulting in substantial harm, you will take a look at your pressure injuries practices and take steps to improve those. Getting to the point that all institutions use their EHR data to find infections is the single most important change that I think would make a difference. 

Dashboards are very helpful. We have looked at organizational dashboards, and no two are the same. There is a lot of variation. They are useful because if you have a dashboard, you can see where you are having a big problem right now. You probably cannot afford to make major efforts in seven or eight safety areas all at the same time. If you have a dashboard, you can pick where you are not doing so well and implement something to make things better.

CMS (Centers for Medicare and Medicaid Services) could be helpful in this regard by standardizing the definitions of safety metrics. That could make things apples to apples. We found in the study we just did that even between two large hospitals in one system, Massachusetts General Hospital and the Brigham and Women’s Hospital, we often were not using the same metrics, and we are in the same network.³ It is not surprising that people are using different things in different places.

Sarah Mossburg: Could you expand a little bit about that, on standardization and the difference between what you all were using?

David Bates: There are a few types of harm that result in most of the harm in hospitals in this country—adverse drug events, hospital-acquired infections, surgical harms, deep venous thromboses and pulmonary emboli, falls, and pressure injuries. I think if CMS standardized metrics for those measures of harm and then required organizations to report on them, ideally publicly, that would drive improvement. Diagnostic errors also result in a lot of harm, but we don’t yet have solid metrics for that.

Sarah Mossburg: And I guess the pushback that many folks would give on that is that they are already reporting on so many quality outcomes, metrics, et cetera. What are your thoughts on how to address that?

David Bates: I would ask them to report on less. If you are going to ask organizations to do something extra, I agree with you, you should be taking something away. When I was the Chief Quality Officer for our organization, we had to report to something like 30 different entities, which just does not make sense. I would rather see us focus on a smaller number of metrics and report to fewer organizations. We spent virtually all our time and resources on meeting those requirements.

CMS is very focused on improving safety, and they are adding additional electronic metrics. However, the coverage of the metrics they have so far is quite narrow. They have identified multiple metrics that are going to be useful additions, but they cover a very small fraction of the overall safety problem and the rate of injuries that occur in hospitals. They should focus on getting broader coverage with their metrics. For example, an electronic metric on low glucose abnormalities is going to be released soon, and that will be useful, but it is just a small fraction of all the types of harm that occur in a hospital.

Sarah Mossburg: What do you see as some of the future priorities for advancing patient safety?

David Bates: The biggest priority, I think, is starting routinely to leverage the electronic health record to identify safety problems and enable organizations to readily measure patient safety. It will also be useful if there is public reporting on those rates, and CMS is by far in the best position to push that. We also need to work on doing better with decompensation and diagnostic errors and looking more at safety in the outpatient setting. We are moving more and more care outside the hospital, and that has been an important issue.

Sarah Mossburg: Well, thank you for chatting with us. It was a really interesting conversation. You had a lot of really good insights.

References

1. Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020;383(20):1951-1960. [Free full text]
2. Kapoor N, Khorasani R. Beyond the AJR: the need for high-reliability systems to create and track actionable follow-up recommendations in radiology reports. AJR Am J Roentgenol. 2023;220(6):905. [Free full text]
3. Myers LC, Salmasian H, Iannaccone C, et al. A description of the variation in quality and patient safety structures within a health system. Jt Comm J Qual Patient Saf. 2023;49(5):285-287. [Available at]