Ms. Benita Kaimowitz wrote to AnnArbor.com recently asking a very reasonable question: Why would the University of Michigan Hospitals and Health Centers, always ranked highly by US News and World Report’s hospital ranking system, have received a low ranking in its new patient safety segment?

This is a question that deserves an answer given U-M’s express goal to be the safest hospital in the nation.

One might imagine that safety rankings are based on direct bedside observation involving a process that directly reflects the carefulness of the health system in question. What is actually measured within national ranking systems comes instead from computer tapes generated during the hospital billing process. This information is readily available but was never intended to be used to gauge patient safety.

The Agency for Health Care Research and Quality has defined 27 "patient safety indicators" derived from hospital billing information. USNWR has chosen to use five PSIs to assess patient safety, which is where the confusion begins.

Different reputable ranking systems, all pointed toward patient safety, use different PSIs in their calculations. Thus, when the University Health System Consortium recently ranked its member hospitals using a different set of PSIs, U-M emerged as one of the safest hospitals in the nation.

U-M was in good company in the USNWR rankings: Johns Hopkins, Massachusetts General, Duke and Mayo Clinic all received identical low safety scores. These are all excellent hospitals, leaders in the field of patient safety. But this type of scoring system does not address the complexity of patients. Sicker patients receive more complicated care that predisposes them to complications. Incidentally, USNWR develops one patient safety score from each hospital’s aggregate data then applies it to every department in the hospital, which accounts for the consistency from specialty to specialty.

Presently, PSIs lack the careful definition needed to gauge safety, which adds to the confusion. One PSI used by USNWR identifies cases with "postoperative hemorrhage" from billing data, which refers to bleeding after surgery. Currently, each hospital is left to guess what amount of bleeding qualifies for inclusion in this PSI. For example, should a bruise around the incision be included as post operative hemorrhage, or perhaps a small amount of bloody drainage? No surgeon would think so. Perhaps cases requiring post operative blood transfusions should qualify, which might seem more reasonable, but some bleeding is expected after major surgery and difficult cases might require a few transfusions.

Where should the line be drawn that separates safe care from unsafe care?

Because AHRQ provides little definition and there is no auditing function, reporting consistency is lost from hospital to hospital; a true apples-to-apples comparison of hospitals becomes impossible.

Certainly the public has a right to information about safety, but bad safety information is no help at all. Likewise, hospitals need good, comparative information about safety to improve. A reliable national registry to assess safety has yet to be developed.

What is more productive is to assess the “safety culture” of a hospital - how care givers feel about the safety of the care they deliver, and how they feel about the hospital’s response to safety problems that are identified. When the environment is right, caregivers feel enabled to speak up about safety problems, regardless of their position in the pecking order. In this way, problems can be avoided and safety is improved.

At U-M, we use a standardized AHRQ safety culture survey. Our scores have increased progressively for the past six years.

While AHRQ safety culture scores are not available to the public, individual hospitals can use these data to judge if they are going in the right direction, and to make mid-course corrections if they are not. No data regarding safety culture are included in the USNWR rankings.

We use a local “hands-on” approach to assess patient safety at U-M by conducting “patient safety rounds.” Hospital leadership at the very highest levels show up, unannounced, to randomly selected patient care units. These rounds ferret out safety problems directly from the caregivers who actually work at the bedside. We ask questions like, “What worries you most when you come to work?” We get rich information about potential problems that need to be solved and solved quickly.

This information is far more useful to a hospital than an imperfect national ranking system. But it will only be provided if the care givers feel they work in an environment in which thanks, and not retribution, is the response when identifying a potential error.

So, to Ms. Kaimowitz, we are grateful that you brought the subject up. It must weigh on the mind of potential patients or patient family members. The USNWR safety ranking is new, and cannot yet provide the type of information you want and deserve.

Given the priority the health care environment places on safety, we can anticipate a more reliable ranking methodology in coming years. Until then, be assured that U-M places the safety of its patients its highest priority, that the internal efforts to identify and fix problems are more important than national rankings, and that we are aiming to be the “safest hospital in the nation.” We believe we are not far off. Come and visit us, and see for yourself.

Darrell A. Campbell Jr., MD, FACS, is the chief medical officer at the University of Michigan Hospitals and Health Centers. Vinita Bahl, DMD, MPP, is the director of Clinical Information and Decision Support Services, UMHHC.