Accreditation and Regulation: Can They Help Improve Patient Safety?

Rebecca N. Warburton

Accreditation and Regulation: Can They Help Improve Patient Safety?

Rebecca N. Warburton, PhD | April 1, 2009

View more articles from the same authors.

Perspective

My grandfather, an eminent Houston internist, never lived to practice in our modern era of managed care and evidence-based medicine. He used to joke that the reason he never had malpractice insurance in the early days of his career was that he wasn't doing any malpractice then. I know that he resented the growing intrusion of lawyers and legislators into medicine over his years of practice. He was zealous in his pursuit of quality care for his patients, but I think he would have resisted efforts by non-medically-trained managers and regulators to enforce any kind of control over physicians.

I'm not sure what my grandfather would have thought of the modern patient safety movement, with its reliance on a systems approach to reducing harm, and use of methods and insights from nonmedical industries. I suspect that he, like many or perhaps most physicians of his generation, would have felt that nothing was needed beyond heeding the admonition to "First, do no harm." He never knew that I would go on to become a health economist, but I feel safe in assuming that he would not have thought there should be such a thing, and certainly would not have believed that his health economist granddaughter had any business working or writing in the field of patient safety. I don't believe he would have thought there was much value or legitimacy to allowing regulators and accreditors to force changes in medical practice.

I'm sure some readers are, at this point, thinking what a wise man my grandfather must have been. I believe he was an excellent physician, and he was, in many ways, very wise. And, it's human nature to prefer not to be told what to do—a sentiment that tends to be stronger among proud, highly trained professionals. It is not surprising, therefore, to find a certain tension between the culture of medicine and the demands of health care accreditation and regulatory bodies. But I hope to convince readers that he would have been wrong to resist a role for accreditation and regulation in improving the quality and safety of health care.

Before discussing how regulation and accreditation can (and cannot) help improve patient safety, some definitions and distinctions.

Regulation and accreditation are not identical. Regulation involves rules that must be followed, while accreditation is a seal of approval (from some independent accrediting body) certifying that an organization or individual has met specific standards. In practice, in health care, accreditation is frequently so essential that many accreditation requirements have the same power as regulations.(1) In this paper, regulation and accreditation will be discussed together and contrasted with other strategies for improving quality and safety.

Quality and safety are sometimes used interchangeably in health care, but they too should be distinguished. If we think of quality of care as a distribution, and graph the amount of care delivered at various quality levels, we might get something like Figure 1. We hope the lower (left-hand) tail is small, and we hope quality is pretty good at the high point of the curve. Some care is truly excellent, but how much is unknown.

High-quality care causes fewer errors and less harm than low-quality care. Quality improvement (QI) aims to shift the entire quality curve to the right and increase the height of the right-hand tail (more truly excellent care). Patient safety improvement is quality improvement focused on lowering or truncating the lower tail of the quality distribution. Both kinds of improvement are needed, and both improve safety for patients, but their focus is different.

Does this distinction have any relevance to the role of accreditation, regulation, and other methods of improving patient safety? I say, yes: Regulation and accreditation are particularly legitimate and useful for patient safety improvements, while they are less useful in ensuring more general QI. It is hard to argue that a clearly unsafe practice should be permitted, and it is much more reasonable to assert, as we do in other areas of public safety (2), that minimum standards be met by a hospital or other health care agency or provider if they wish to continue providing care. It is also important that rules be clear, compliance be observable, and performance be fully in control of the hospital or agency being regulated. Prescriptive solutions therefore work best for less complex problems where one-size-fits-all solutions are sensible, such as "sign-your-site" rules for surgery, requiring hospital staff to wash hands before and after touching patients, and banning known high-risk abbreviations.(1)

For complex problems, such as the need to improve the user-friendliness and reliability of device alarms, a coordinated approach involving many parties is essential; no simple rule would improve safety.(3)

It is also less legitimate, and less practical, to use rule-making to lift care from acceptable to excellent. It is impossible for all care to be above average; there will always be a distribution of quality. What is reasonable is to ensure that the minimum acceptable standard is sufficiently good and safe to protect the public. In practice, a distribution like Figure 2 would be much better than what we have now. The right-hand tail would represent care delivered at a higher-than-required quality level.

Deciding on the minimum level of quality, the best programs and procedures to be used to achieve it, how to monitor success or failure, and how quickly to raise standards as overall quality and safety increase are all easier said than done. Getting those requirements right is the critical work of accreditation agencies.

Accreditation has been observed to be more effective in promoting good safety practices than state-required error reporting or public awareness (4), and in most hospitals, accreditation requirements are the primary driver of safety efforts.(5) In others, however, particularly those that are more oriented to safety improvement and excellence in general, accreditation requirements are viewed as a floor; staff at the Veterans Health Administration explicitly set safety goals that exceed accreditation requirements, and many hospitals have voluntarily implemented rapid response teams and other optional enhancements to care.(6)

In other words, accreditation and regulation are likely to be quite effective in causing a shift in the quality distribution from Figure 1 to Figure 2, but their main effect will likely be in truncating the lower tail of the distribution.

Increasing the amount of excellent care (right tail) requires organizations to exceed required quality levels. That is likely to happen through a combination of other policy tools, such as pay-for-performance and voluntary improvement work (such as the Institute for Healthcare Improvement campaigns). Boards can play an important role if they begin to hold managers and clinicians accountable for achieving quality targets and continually improving quality and safety goals (7), but board leadership works best by recruiting clinical leaders from all disciplines for bottom-up change.(8)

Two points still seem underappreciated in the literature and by accreditors and regulators. First is the need to consider both costs and effects of potential safety and QI interventions. The reason is simple: no agency can adopt all potential improvements, and if cost-effectiveness is ignored, then less quality/safety improvement will be obtained for the resources dedicated to improvement.(9) Accreditors and regulators need to be sure that they require the most cost-effective changes first, for the same reason.

Second, changes must be evaluated for effectiveness, ideally both before (ex ante evaluation) and after implementation. As required by its own regulations, the U.S. Food and Drug Administration (FDA) carefully studied the evidence on likely costs and effects before announcing its bar code rule of 2004 (10), but it is one of the few agencies with such a mandate.

In ex post evaluation, it is particularly important to detect and assess unforeseen consequences. When the Joint Commission issued its 1998 Sentinel Alert about keeping concentrated potassium chloride only in hospital pharmacies and specialized units such as intensive care units (ICUs), the details of implementation were left to hospitals, and variations in practice occurred. At least one hospital (11) found that delays in receiving dilute solutions from the pharmacy turned ICUs into de facto pharmacies and led general-ward staff to create illicit stashes of the concentrate—potentially creating higher risks than before the change. The Joint Commission had to follow up the initial recommendation with examples of safe and unsafe practices.(12) Absent ongoing monitoring, though, it is impossible to know whether this change was actually beneficial. Similarly, the 2002 attempt to improve care for hospital patients with community-acquired pneumonia (through creating a performance measure monitoring how quickly they were given antibiotics) was shown to have encouraged unnecessary use of antibiotics and not to have reduced mortality, as had been hoped, primarily because of the difficulty of definitive diagnosis in actual clinical settings. The rule was amended in 2006 and again in 2007, but it remains far from clear that it is creating net benefits for patients.(13)

The modern evidence-based medicine movement was started by doctors (14), but even its strongest proponents would not claim to have had the success they hoped for in getting all physicians to adopt their methods. Even when standards for care are clear, they are often not followed; a 2003 study found that patients received only 55% of recommended care, with wide variation between rates for different medical conditions.(15) Evidence of differences in medical practice that do not reflect patient characteristics or preferences, and that worsen patient outcomes, abounds in all countries with advanced health care systems. Evidence of harm caused by care is equally widespread (16), yet growing awareness has not automatically or instantaneously ended the problem. If "first, do no harm" were sufficient, there would be no need for a patient safety movement.

Wider adoption of evidence-based medicine by clinicians may actually be the most promising route for retaining professional autonomy and avoiding having the quality/safety agenda driven by nonclinicians. Interestingly, an ethnographic study at a UK hospital provides some evidence that clinicians facing external pressures to improve performance have become adept at developing their own, clinician-driven safety and quality improvement procedures.(17) This is consistent with the pattern in other safe industries; outside pressure can help promote the cultural and behavioral change needed to create safer systems.

I'll conclude with some evidence-based recommendations to improve the quality and safety benefits from regulation and accreditation. First, regulators and accreditors need to become more aware of the costs and effects of their actions. Changes need to produce net benefits, and the most cost-effective changes should be adopted first. Second, they need to improve their use of monitoring and evaluation of both intended and unintended consequences, so that mistakes can be swiftly corrected. Third, they need to do a better job of meaningfully involving the actual clinicians who will be affected by new rules, guidelines, and measures in their development; this would provide a double benefit, both generating better standards and increasing practitioners' appreciation of the beneficial role of accreditation and regulation. Fourth, they should consider piloting changes (and fine-tuning them based on pilot results) before making them mandatory system-wide. Health care is a very complex system, and it is difficult and dangerous to assume that well-intentioned changes will always have the predicted real-world effects.

If my grandfather were still practicing medicine today, I know that he would support efforts to improve quality and safety, because he always wanted to do what was best for his patients. I am equally sure that he would believe physicians and other practitioners should be the leaders. He would probably be skeptical of the role of accreditation and regulation to do anything beyond setting minimum standards, and no doubt he would have preferred payment-for-performance and other voluntary improvement strategies. I hope readers will be convinced, though, that regulation and accreditation are an important part of the solution to our quality and safety problems.

Rebecca N. Warburton, PhDAssociate ProfessorSchool of Public Administration University of Victoria Victoria, BC, Canada

References

Back to Top

1. Wachter RM. Understanding Patient Safety. New York, NY: McGraw-Hill; 2007. ISBN: 978-0071482776.

2. Leape LL, Berwick DM. Safe health care: are we up to it? BMJ. 2000;320:725-726. [go to PubMed]

3. Clinical Alarms Task Force. Impact of clinical alarms on patient safety: a report from the American College of Clinical Engineering Healthcare Technology Foundation. J Clin Engineering. 2007;32:22-33. [Available at]

4. Hosford SB. Hospital progress in reducing error: the impact of external interventions. Hosp Top. 2008;86:9-19. [go to PubMed]

5. Devers KJ, Pham HH, Liu G. What is driving hospitals' patient-safety efforts? Health Aff (Millwood). 2004;23:103-115. [go to PubMed]

6. McCarthy D, Blumenthal D. Stories from the sharp end: case studies in safety improvement. Milbank Q. 2006;84:165-200. [go to PubMed]

7. Haughom JL. The PeaceHealth governance journey in support of quality and safety [Perspective]. AHRQ WebM&M [serial online]. July 2007. [Available at]

8. Larson L. Physician autonomy vs. accountability: making quality standards and medical style mesh. Trustee. 2007;60:14-16, 21, 1. [go to PubMed]

9. Warburton RN. Patient safety--how much is enough? Health Policy. 2005;71:223-232. [go to PubMed]

10. Bar code label requirement for human drug products and blood; proposed rule. Federal Register. March 14, 2003; 68:12499-12534. [Available at]

11. Shojania KG, Duncan BW, McDonald KM, Wachter RM. Making Health Care Safer: A Critical Analysis of Patient Safety Practices; Evidence Report/Technology Assessment No. 43. Rockville, MD: Agency for Healthcare Research and Quality; 2001. AHRQ Publication No. 01-E058. [Available at]

12. Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Samples of alternative approaches to the recommendations. Oakbrook Terrace, IL; 2003.

13. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med. 2008;149:29-32. [go to PubMed]

14. Sackett DL, Richardson WS, Rosenberg WMC, Haynes RB. Evidence-based Medicine: How to Practice and Teach EBM. New York, NY: Churchill Livingstone; 1997. ISBN: 978-0443062407.

15. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635-2645. [go to PubMed]

16. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000. ISBN: 0585212732.

17. Waring J. Adaptive regulation or governmentality: patient safety and the changing regulation of medicine. Sociol Health Illn. 2007;29:163-179. [go to PubMed]

Figures

Back to Top

Figure 1. Model of current distribution of quality of care.(Go to figure citation in commentary)

Figure 2. Model of potential distribution of quality of care.(Go to figure citation in commentary)

In Conversation with...Mark Chassin, MD, MPP, MPH

April 1, 2009

Editor's note: Mark R. Chassin, MD, MPP, MPH, is president of The Joint Commission, the preeminent standard setting and accrediting organization in health care in the United States and, increasingly, the world. Over the course of his notable career, Dr. Chassin, an emergency medicine physician, has held a variety of key positions, including New York State Health Commissioner and chair of the department of health policy at Mt. Sinai. He has published several seminal papers and was a member of the team that authored the IOM report, "To Err Is Human." We asked him to speak with us about his role at The Joint Commission, as well as future directions for the organization.

Dr. Robert Wachter, Editor, AHRQ WebM&M: What has surprised you the most about your position and about The Joint Commission?

Dr. Mark Chassin: The most surprising and gratifying thing that I discovered is that, despite the longevity of many of the senior staff, the appetite and enthusiasm for new ideas are really extraordinary. As I thought more about that in context, it became clear that this organization has really been in continuous change for at least 10 to 15 years, at an accelerating pace. To find that fertile ground for new initiatives and new ideas was surprising and gratifying.

RW: On one hand, there is tremendous pressure to fix the problems of medical errors and poor quality; on the other hand, there are often major gaps in what we really understand about how to make them better. How do you come down on that philosophical divide?

MC: I think we have to deal with two separate but related challenges. One is adverse events. The other is routine safety processes, such as hand hygiene, communication, and medication reconciliation, which frequently break down. In the safety process domain, we do have some models that could be used much more effectively than they have been. I'm talking about the tools that have been used in business with great success, such as Lean Six Sigma, the Toyota Production System, and change management methodologies. We are in the planning stages of an initiative that would bring these tools to bear much more effectively in health care organizations to solve the safety and quality problems we are all struggling with. With respect to adverse events, I think the story is different.

RW: In which way?

MC: The problem is that we need to generate new knowledge. Adverse events are different from routine process breakdowns because they represent unique sequences of errors that will never happen again the same way. We have imperfect tools to assess exactly what happened in the course of an adverse event. If you consider Reason's Swiss cheese model and think about all the defenses that have to be breached before harm can be done, we have imperfect ways to figure out exactly which defenses failed in which way. Then, once we understand where the weak defenses are, we have imperfect analytic tools to tell us which one to fix first. Which one is going to be lethal tomorrow? Which one can we safely put on the back burner while the other high-priority project is underway? We lack a systematic way to learn across adverse events, to build knowledge about institutional vulnerability that benefits from seeing how the defenses fail in more than one adverse event. Our current tools do not encourage us to do that. They encourage us to do what we call root cause analysis, which I think is a misnomer since there is never a single cause. There aren't usually even two or three. Five, eight, or ten defenses fail in many of these adverse events, especially the more complicated ones. But knowing what happened in one is not enough information to understand what the vulnerabilities are in an institution. The analytic methods we have are from a different generation. We need to develop the next generation of those tools to achieve the goal of high reliability.

RW: As a philosophy for you and for The Joint Commission, one way of proceeding would be to focus mostly on measuring and reporting the outcomes and then assuming the organizations will figure out how to improve. Another way, which has been more traditional for The Joint Commission, is to focus on the structures and the processes. What is your philosophy on that?

MC: It will inevitably be a blend. Either process or outcome measures can be valid measures of quality—valid in the sense that they really reflect the nature of quality. I go back to the 1999 Institute of Medicine definition (which is also The Joint Commission definition): Quality is the extent to which health services increase the likelihood of desired health outcomes. So if you're going to use an outcome as a valid measure of quality, you must have a proven relationship to processes that you can change to affect the outcome. It might be useful to know about that outcome for other reasons, but if you don't know how to improve it, then it is not really a valid measure of quality.

The reverse is true about process. You must have a proven relationship of a process to an outcome in order for that process to be a valid measure of quality. I don't think we should be taking up the scarce resources of health care organizations by focusing them on anything other than valid measures of quality. The Joint Commission has been one of the most powerful forces that has focused health care organizations—particularly hospitals—on solving critical safety and quality problems through National Patient Safety Goals, core measures, and accreditation standards. These programs spotlight for organizations exactly where they should deploy their very scarce quality improvement resources to make themselves better. We, and others with the same focusing power, have an obligation to make sure that we have the highest confidence that health outcomes will improve directly as a result of that spotlighted activity. If we do not have that high confidence, then we have to seriously question why we are asking organizations to expend effort on activities that may be off the mark.

RW: Do you have any sympathy for the hospital that says, "We just can't afford to continue to do this," especially as the bar gets raised?

MC: I have tremendous sympathy for all organizations with regard to the resources available to do quality improvement. Whether you are small, medium, or large, you have scarce resources to devote to quality improvement. I am making the argument to health care organizations that they need to find more ways to devote more resources to improvement. Ultimately, that is what it is going to take if we are going to meet the expectations of the public, and our own expectations to transform health care into a high-reliability industry with rates of adverse events and rates of breakdowns in safety processes that are comparable to other industries that have achieved high reliability, such as commercial air travel and nuclear power.

The other point I made is even more important to hammer home. To make the argument credibly that these resources are producing better quality and safer care, we must spotlight programs that really produce results that are measurable, documentable, and sustainable. That is another component to the current state of the environment that I think is problematic. There is no question that the industry has put enormous effort into improvement, but when public stakeholders say that adverse events keep happening that should not happen and safety processes keep breaking down, what do you have to show for all that effort? We do have some things to show for it. If you look at The Joint Commission's publicly reported core measures over the past 5 to 8 years, there has been steady and substantial improvement. But that is a small slice of the action for what our public stakeholders rightly expect us to be able to document. We need more resources for improvement. We need more proven interventions. We need to be able to document with good measures that improvement is producing real results in terms of health outcomes.

RW: One of the things The Joint Commission has not done, at least to my knowledge, is accredit around information technology (IT). Do you see a future in which The Joint Commission might mandate technology such as CPOE [computerized provider order entry], barcoding, and smart pumps?

MC: There are standards around information management in the manual. In general, the approach on the accreditation standard side is to say, these are the processes you need to get working right. How you choose to do that—you have a lot of leeway. As you know, we accredit lots of different places with lots of different access to capital and investment for bells and whistles. I do not see that fundamental approach changing much.

I will make two other points about IT. One is that The Joint Commission has taken a strong interest in pushing the IT agenda. We have been pushing outside the routine accreditation and certification processes for the more rapid development of IT and its application in health care. On the other hand, I do not view IT in health care as a panacea. One has to be very careful about how you employ it, especially if you have not really worked out the defects and pitfalls in the process you are trying to automate. We have seen this happen with CPOE implementation and other kinds of IT implementation. If you do not get the process working right before you automate it, bad things can happen to patients very quickly. CPOE is a perfect example of that. The aphorism that I find continually very scary about IT is that computers do not make us less stupid; they make us stupid faster.

RW: One of the things I hear from hospitals is that they understand they are going to be measured, accredited, and regulated. But they wonder whether these organizations can clean it up so that it is coming in a single voice with a single set of measures, rather than something coming from The Joint Commission, something else from CMS [Centers for Medicare & Medicaid Services], or something else from NQF [National Quality Forum ]. How do you respond to that?

MC: It is a terrible problem and The Joint Commission has been working with other national organizations and alliances—the Hospital Quality Alliance, the Ambulatory Quality Alliance, the Quality Alliance Steering Committee—and lots of other organizations, including NQF, to try to wrestle this problem to the ground. The term of art in this arena is called harmonization. The measures should be identical when they are measuring the same thing. One of the problems is that all of the organizations have to agree on exactly the same way to manage whatever the problem is. For the first 10 years or so, there was really good agreement between the two organizations that matter the most to hospitals—Medicare and The Joint Commission. So far that has worked really well. But if last year's proposals from CMS, particularly outpatient and new inpatient measures, are harbingers of the future, the threat is that the agreement will be disrupted. Medicare is considering adding a large number of measures that are derived from their claims databases—measures that are not, in my view, as highly valid from a quality standpoint as most of the clinically based core measures. But those require a lot of effort on the part of hospitals to collect from medical record data. Medicare is under pressure from other stakeholders to get more measures out there and to more quickly cover more territory. It is getting more and more difficult to achieve a productive consensus.

RW: I want to ask you about some of the more controversial new directions, which are medical staff standards and the assessment of individual physician competence for credentialing and re-credentialing. Do you feel like the science and the data sources are up to the task?

MC: This is also an inevitable direction for quality measurement. The physician quality reporting initiative is out there. They have developed hundreds of measures. There is a movement now, largely in the private sector, to develop physician practice and individual physician measures on a community-wide basis. Whether it is good, bad, or indifferent in terms of measurement is in the eye of the beholder. I have already told you my bias that claims data do not produce highly valid quality measures. But they are already out there on individual physicians as well as on organizations. The challenge for hospitals is to develop ways to credential physicians that meet not just The Joint Commission standards but the common sense standards of physicians. Hospitals must have confidence that physicians are qualified to do the procedures and offer the services they have privileges to do. We certainly see examples—fortunately they are uncommon—of physicians undertaking procedures that they are not qualified to do. This is an ongoing challenge. When I was Commissioner of Health in New York, I created the first program publishing data on risk-adjusted mortality for hospitals and surgeons following coronary bypass surgery. That program has grown and flourished, and it was a long battle to get the right mix of clinically valid risk adjustment data with the right time series; physician data are published on a rolling 3-year time period to make sure that adequate volumes are represented. This is tricky business, but it is clearly a direction that cannot be reversed.

RW: One thing that strikes me about the position of accreditor or regulator is that they may not get a lot of feedback from those they work with because people are scared to give them feedback. Do you think that's real and, if so, do you have any strategies for mitigating that?

MC: The Joint Commission is not a regulator in the governmental sense of having the authority to either say you can or cannot do business here. When I was Commissioner of Health in New York, I did have that authority. This is very different. This is a private organization that relies on organizations voluntarily subscribing to our service; they pay us to assess and educate to help them improve. That is a very different posture than a pure governmental regulator. That said, there is no shortage of feedback. The Joint Commission accredits more than 4500 hospitals, but the total number of organizations and programs that we accredit or certify topped out last year at about 16,000 including long-term care, behavioral health, laboratory, home health, ambulatory care, and disease-specific programs. We are primarily known for the hospital program because it covers so much of the marketplace. We continuously reach out to all of our customers and other stakeholders to understand how we can do our job better. I see that process only intensifying as we create a more effective internal improvement process.

RW: My last question is: are you having fun?

MC: I am having a ball. This is a great organization. The people are some of the best I have ever worked with in terms of how enthusiastic and mission driven they are. If you think back just 10 years ago, when hospitals were not measuring anything on any sort of national or consistent basis, the revolution that has taken place is extraordinary. What we hear in the field is no longer, "There are problems, but they're not in my place." Organizations know they have quality problems. What they are asking us, with increasing decibel levels, is: "Don't keep telling us what to fix—tell us how to fix it." So it is a great time to be here.

This project was funded under contract number 75Q80119C00004 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services. The authors are solely responsible for this report’s contents, findings, and conclusions, which do not necessarily represent the views of AHRQ. Readers should not interpret any statement in this report as an official position of AHRQ or of the U.S. Department of Health and Human Services. None of the authors has any affiliation or financial involvement that conflicts with the material presented in this report. View AHRQ Disclaimers