Editor's note: Leah Binder is President and CEO of The Leapfrog Group, a national nonprofit representing employers and other purchasers of health benefits advancing safety and quality in American hospitals. We spoke with her about the development of the Hospital Safety Score and Leapfrog Hospital Survey, and Leapfrog's work to address key health policy issues, including early elective deliveries and hospital infections.

Dr. Robert Wachter, Editor, AHRQ WebM&M: What was the intent when Leapfrog was formed?

Leah Binder: Leapfrog was formed so that purchasers of health benefits could join together to better leverage their investment in health care to get better quality of care for their employees. They also came to Leapfrog thinking that, in their industries, such as the automobile industry, products are subject to enormous scrutiny in the public sphere. Vehicles are analyzed and compared publicly in a variety of formats, and pricing is completely transparent. So they wondered why that's not the case for hospitals. Why can't their employees look at the quality and cost effectiveness of hospitals when deciding where to seek care? Hospital care is very important to people, so they felt their employees deserve the kind of transparency and competitiveness for quality that their industries experience for their own products.

RW: In the early years of Leapfrog, the most visible thing that the organization did was endorse certain kinds of safety practices: intensivists, high-volume centers for surgery, and computerized order entry. First of all, how did that happen, and second, how did you think it worked?

LB: Well, Leapfrog was formed in 2000 out of the IOM report To Err Is Human, which suggested upwards of 98,000 people were dying every year from errors and accidents in hospitals. Our members were astounded by this number and really wanted to do something. In 2000 there were very few measures of performance of hospitals. So the story is that Robert Galvin of General Electric said, "I want to find some measures of performance that my mother understands." One of them they started with, "My mother understands that you need computerized methods for managing prescription orders in a hospital." It turns out the experts and the studies confirm Mother's wisdom: medication errors drop by as much as 75% of hospitals with CPOE. Thus, we came to computerized physician order entry.

For each of the major leaps, we looked at the literature and then talked about what makes sense to purchasers and the public as things you would look for in a hospital and feel safe with your family members going to. It was really a combination of common sense and whatever literature and expertise they could get their hands on. Measurement science has advanced quite a bit in the 14 years since Leapfrog was founded. Our survey has become more complex and more nuanced than it was with those first three major leaps. But we still stand by those three leaps as fundamental to a high-performing hospital.

RW: How has the organization balanced the science? The inevitable tension in health care is that these things are often nuanced and the science is imperfect and there are unanticipated consequences?

LB: Well that's the battlefield of measurement right now. To what extent do we expect measures to be fair and reliable as indicators of the performance of a hospital? What is the level of perfection that we demand in order to have public reporting? Understandably, for many providers and hospital associations, their standards for the reliability of measures are as high as you can get, perfection or close to it. Purchaser and consumer organizations like ours also have a high threshold for measurement integrity, but we also believe that the public's need to know is more important than measurement perfection. We aren't going to wait until our grandchildren are old people before we are able to report something that we know about how well hospitals are doing. So it is a political as well as a scientific argument that plays out here every day. Leapfrog has always felt strongly that the science has to be foremost, but the need for more transparency is also critical.

RW: Leapfrog comes out of industry and businesses. How has that influenced how it has done this work?

LB: I think that the influence of business has been to bring a fresh perspective to health care wonkery. Our staff is in the weeds every day on issues around patient safety and quality issues and measurement. My staff can recite a lot of measures and a lot of the details about the latest proposal by CMS. But our members don't want to hear about that. They just want to know if it's common sense or not. I cannot go to my board and talk to them about the details of a new measure on maternity care. Their eyes will glaze over. But I can talk to them about why we think it's important to have this maternity measure. What are its implications? What are the pros and cons and issues behind it? Why does it matter not only for them in their purchasing but also for the American public to drive a market for improved maternity care? We have to be able to talk in a language that the business community understands. That brings a freshness to our approach and our own advocacy within the health care world.

One major element missing in policy debates is our ability to speak English about health care. We talk in our own language, acronyms, technical jargon, and the public doesn't understand it. So you have this disconnect politically when we try to make changes in health policy; the public doesn't understand why and what we're doing. We are so used to talking to each other we don't know how to articulate what we really mean and why it's important. The value of having purchasers governing Leapfrog is that it forces the discipline for us to communicate and really advocate effectively for the kinds of changes that we all want to see.

RW: How do you decide how edgy to be? You're coming at this from a different angle and what you're pushing the system to do is inevitably going to lead to push back and some controversy. You have to try to figure out how much controversy you are willing to engender to move the system forward.

LB: That's a good question. We start from the premise that we want to move the system forward. Then we decide how we can best do that. That requires some edginess. If it didn't, we probably wouldn't be able to make the change we want to see, because complacency is a major part of the problem we have in health care. It's easier to not move than it is to change. At the same time we want to be credible and sitting at the tables where decisions are made. If you are too out there on the fringe, you aren't necessarily going to have that influence.

If we weren't controversial at Leapfrog, I don't think my board would think we're doing the right things. Because they know you cannot make change unless you're disruptive in some way. It is a balancing act for us because we do engage with a lot of stakeholders, and we rely heavily on experts in the health care field to advise us. We see ourselves as part of a larger movement to make change, and we want to stay that way. We don't want to be on the fringe; we want to be in the center of it all. At the same time, we do see ourselves as disrupters and we recognize and gladly accept that that will also make us a lightning rod.

RW: When Leapfrog was founded, you weren't alone in the world of organizations trying to move the system forward, but it was a much less crowded space. Now you're operating in a busy ecosystem; a good number of entities are now rating hospitals, like US News, Consumer Reports, Healthgrades, etc. How has that changed your work?

LB: I think we have been a catalyst for some of the work that those organizations have done. Leapfrog really pushed the envelope, and a lot of others came forward to do things that we pioneered. I wasn't here at the time, but my predecessors pioneered some work that everyone said couldn't be done. Now everyone's out there doing it, and that's great. Leapfrog still has at its core one element that is different and makes us indispensable to the movement—we are able to collect hospital data at the national level that is otherwise unavailable. We're able to do that because of our purchaser members pressuring hospitals and asking them to report. That gives us a different space in the world of advocacy around patient safety, simply because we're able to get important data that's otherwise unavailable. We're able to use that particular leverage of having hundreds of purchasers asking hospitals for data to push a little bit harder and maybe plow a trail for others who also want to use data to make change.

RW: Do you have any sympathy for the argument that you periodically hear from hospitals: "I'm being asked for data from so many different organizations that if everybody could just get their act together and ask for a uniform dataset, life would be simpler and in some ways better"?

LB: We have total sympathy for that, and we've actually pushed particularly around stakeholders that we have influence on, one of which is health plans, to try and at least come up with common sets of measures that they use. We've had marginal success and that's a lot better than it was about a decade ago. We keep the list short of what we look for in the Leapfrog survey simply in recognition of the burden of reporting for hospitals. But at the same time, there's a long list of things that purchasers would like to get their hands on.

Having said that, though, hospitals do spend time collecting and reporting data, and the inconsistencies among measures is problematic, very little of what they collect is ever made public. So when hospitals say that the 40 hours of staff time it takes to report to Leapfrog is burdensome, we remind them of the several staff people assigned full time year round to reporting to registries and other entities that never make the data available to their communities. Forty hours is not a great deal of time to respond to the public you serve.

RW: Much of the organization's history has focused its attention on hospitals. And the world seems to be focusing its attention increasingly out of hospitals in the ambulatory environment and on population health. Is that correct that Leapfrog has focused on hospitals largely and is that being rethought for the future?

LB: Leapfrog has traditionally focused only on hospitals, and every year we have a strategic planning discussion at the board level about whether we ought to move outside of hospitals. Right now the focus remains on hospitals because hospitals still exist. Even though we are seeing the evolution of new models for delivering care, still, in the center of most of those models are hospitals. We don't think that now is a good time to take our eyes off that prize. We don't want to be distracted by these evolving new models and lose sight of what's going on in hospitals. For instance, we're still seeing persistent problems with patient safety. We have not seen the kind of progress in patient safety that we would have expected to see in 10 years and the data is frightening. That alone is a reason for us to make sure, at least from Leapfrog's spot, that there's a continued laser focus and pressure put on hospitals to see improvements in that area. We still see a real role for Leapfrog and an important role in leading change in the hospital environment, even while other changes are happening in the larger system.

RW: Do you or your members have a theory as to why things have not improved as much as one would have liked?

LB: Our board met for a major strategic planning event at their 10-year anniversary in 2010. There were a number of reports coming out trying to assess whether there had been progress on patient safety since our formation because it was also the 10-year anniversary of To Err Is Human. Most of those reports concluded there had not been any progress, or just extremely incremental change. So we asked exactly that question. Has Leapfrog failed? Is this model for change—transparency, reporting on performance, and driving a market for quality—a failure? Our conclusion was that the model is still good, but we didn't get the level of transparency that we need to make it work. The reason we don't is because the Leapfrog Hospital Survey is voluntary. That means higher performing hospitals will be more likely to respond, but we are not getting all of the hospitals responding and therefore we are not getting the competition we want to see. We decided at that meeting to explore ways to focus our efforts around transparency and improve and expand on transparency of health information nationally. What came out of that about a couple of years later was the Hospital Safety Score.

RW: Tell us about the safety score, whether it's met its goals. The decision to report in a grade-like fashion that obviously has engendered some controversy. How did that happen?

LB: The board became quite committed to engaging consumers around making decisions about hospital care by improving transparency. We wanted to do something that would be specifically tailored to engage consumers in making decisions about their own care. So we began by doing some research on what kinds of rating systems in any industry are most effective at encouraging consumers to make change. We found several examples, but one that was most interesting to the board was what had happened in Los Angeles and New York City with restaurant inspections. In both of those cities, starting with LA, the Public Health Department inspects a restaurant and assigns it a letter grade, which the restaurant is required to post in its front window. Within a year or two of implementing this program, virtually all the restaurants became "A" restaurants. If they couldn't make the grade they were out. Because consumers really were using that grade. Within a year of implementing the same program in New York City, a poll of New Yorkers asking if they consult the letter grade before going to a restaurant and 72% said yes. That's an enormous percentage for a year-old program. Clearly restaurant inspections have always been published, but most New Yorkers weren't looking at restaurant inspection reports prior to the letter grade, and it made a difference.

Our board was really interested in that and wondered, "Why can't we do that for hospitals in rating their safety?" We put together some concepts and ideas of how it might be done and took about a year. We appointed a volunteer, Blue Ribbon Expert Panel of foremost experts in patient safety; they convened and voted and researched the available measures, weighting them, measuring them, and then advising Leapfrog on a methodology and a protocol. We launched the Hospital Safety Score in June 2012. We graded every hospital for which there's adequate data on errors, accidents, and injuries. We looked at only publicly available data at the national level and gave hospitals their grade based on how they did on more than 26 measures. Most of the data is from CMS through Medicare, and we also use data from The Leapfrog Hospital Survey for those hospitals that report. For hospitals that decline to report, Leapfrog data is not used in calculating their score and treated as not applicable. At the advice of our Blue Ribbon Expert Panel, we revised our methodology over time after some very good constructive suggestions from experts, hospitals, and different constituents who are interested in the score.

We designed the score for consumers, and the key thing has been that we are trying to drive change with this. What has been somewhat of a welcome surprise for us is how constructively hospitals have used this score. Although hospitals that don't get an A are quite vocal in the press sometimes talking about all the reasons they think the scores cannot be right, behind the scenes we have built relationships with these hospitals. Our phones ring off the hook with hospitals that really want to understand their data and set goals. There are hospitals that tell us now they have provisions in their executive pay contracts that relate to the score of that hospital. So we're encouraged by that.

RW: It sounds like there's a lot of engagement on the part of hospitals that see their scores and react to it. Are you seeing improvement? What are you going to monitor over time to figure out if this is working?

LB: We've seen incremental change. One challenge: Because we are dependent on CMS data, we are also dependent on the fact that CMS data is not up to the minute. When we started the score, most of the data was covered periods a year or two in the past. The most recent score that we put out had data as late as 2012, with a little bit from early 2013. Over the next year, we will start to see periods of time covered in the safety score that weren't covered before the safety score. So that will enable us to start to see if there were trends of change in the wake of our score.

RW: I think the general experience with consumerism in health care and transparency has been in some ways reflective of yours. The organizations take it up to a surprising degree, but consumers have not in the same way that they shop for a car or a washing machine. Are you feeling like that's beginning to change? Are people looking at these scores and making a choice to go to a given hospital because of its score?

LB: Well, I can only judge that anecdotally. We get phone calls from consumers who are surprised that their hospital got a C, for example. They will recount stories like, "I went to my doctor and I said, 'Why are you sending me to this hospital? They got a C. What's wrong with them?'" This is how consumers perceive the score. And the physicians respond in a whole variety of ways. But the consumer then calls us to say, "I don't understand, what does this mean?" We show them what it means, and generally they either choose a different hospital or they go back to their doctor and push the point a little more. Both of those responses are extremely important to getting the change we want to see. Even if physicians think that Leapfrog is nuts, even if they just dismiss us entirely, they will pay attention to the problem of safety if they hear it from their patients. Then hopefully they will go to the hospital and put some pressure on them to elevate the priority of safety initiatives within the hospital.

So we see lots of levers, and we hear from consumers every day that seem to be pulling on those levers. Most consumers have not been aware of either the variation among hospitals in safety or some of the things that can go wrong in a hospital. Most people don't know much about what actually happens in a hospital. I was talking about some of our scores with my mother. I said something like we were looking at objects left in after surgery, and my mother just stared at me and she said, "Wait a minute, that can happen?" She was stunned. And she's a very well read and intellectual person, but she did not know and hadn't thought of it before. So sometimes consumers read our list of measures and they are stunned just by the list, let alone how hospitals do on the list.

RW: You sit at a unique interface between leaders and very accomplished people in one line of work, business leaders, and very accomplished people in health care. Can you tell us one thing about the other world that they don't understand, or misconceptions that health care people have about business people and misconceptions that business people have about health care people?

LB: I think health care people have a misconception that purchasers really only care about the money. To some extent, plans convey that at the negotiating table. But my experience with purchasers is the opposite. Although they do care about how much they're spending on health care, they are absolutely laser focused on quality. Every purchaser I've ever talked to wants to know how do I figure out how to get better care for my employee? How do I protect them from the errors and injuries that are rampant? What can I do better?

From a purely business point of view, purchasers want their employees to be happy with their health benefits. That makes their company competitive for recruiting and retaining a great workforce. They aren't going to be competitive if they cut health costs to the bone without any impact on the quality of care employees get.

Purchasers don't understand how hospitals think. They're smart and savvy, but they find the language of health care bewildering. That's because the health policy community has its own jargon and its own language, and many purchasers just kind of give up. They just want to delegate their purchasing decisions to health plans. And I think that's been a mistake. Leapfrog was partly founded just so purchasers could find a way to communicate effectively and directly with the health care community.

There's one opportunity for purchasers and hospitals in particular to work together. I've seen it pioneered by Virginia Mason and some other hospitals where they use techniques of efficient production from certain industries. They apply it into the hospital setting to improve on safety and performance. Where I see those partnerships occurring, I've seen just incredible innovation and effectiveness. I hope that as we grow this movement of purchasers becoming more involved directly with the health care system that we can see more of those kinds of productive partnerships.

RW: Where do you see Leapfrog going in the future?

LB: Leapfrog is going to look at the issue of overuse. This is a huge priority for purchasers and our membership. We are exploring a variety of avenues for purchasers to get involved in trying to ensure that their employees are getting appropriate services. That is the next horizon for us. But we will continue to try to expand our footprint among Americans on safety and quality of care.