Measurement of Hard Outcomes Trumps Process Measurement (Again)

Researchers this week cast doubt on the wisdom of looking first at processes and operations to drive quality improvement and urged Medicare, which provides health insurance to people 65 and older, to focus on “hard” outcomes such as rates of surgical deaths and serious complications. In a study published this Monday in Archives of Surgery, a medical journal, and reported this Tuesday by the Wall Street Journal (Thomas M. Burton, “Medicare Faulted on Surgery Evaluation,” October 19, 2010, A6), researchers at the University of Michigan said a better way to lower rates of death and serious complications is to focus on actual outcomes like death rates and to publicize those  rates for all hospitals. This focus on actual outcomes is something my colleagues and I have been advocating for courts since the promulgation of the Trial Court Performance Standards twenty years ago and continue to do so with the CourTools and the Appellate CourTools. (See one of the first few postings here, "A Preference for Outcome Measures," Made2Measure, September 28, 2005).


The University of Michigan researchers looked at over 2,000 hospitals and 325,052 surgeries with high risks of death or serious complications. They found little evidence of a consistent relationship – and this is the important part of their findings – between the rates of surgical deaths and serious complications 30 days after surgery and how well the hospitals complied with the processes prescribed by Medicare, such as giving hospital patients aspirin at discharge after heart attack treatment and doctors giving pneumonia patients antibiotics within six hours of arriving in the hospitals. In other words, there was no evidence linking what Medicare might have thought are “best practices” to improved outcomes for patients.


“I definitely think some of the money we’re spending measuring process compliance doesn’t promote quality,” said Lauren H. Nicholas, the lead University of Michigan researcher said in an interview with the Wall Street Journal. “We feel [Medicare] should be moving toward greater use of measures of outcomes.”

Popular posts from this blog

Top 10 Reasons for Performance Measurement

There is a growing demand of “show me the data” from citizens and taxpayers, legislators, executive agencies, oversight boards and communities – all of whom exchange money and other support (e.g., trust and confidence) for services -- that courts can ill afford to ignore. Courts need to measure their performance to meet the demand for public accountability. But the benefits of court performance extend beyond accountability including improved prediction, better understanding and control, influence, a sharpened focus on what is important, and discovery of better practices. Objective performance data is central to good management practices like strategic planning, performance-based budgeting, Balanced Scorecards, Total Quality Management (TQM), and Six Sigma. Performance measures allow court leaders and managers, policymakers, legislators and the public to evaluate court programs’ inputs (the resources allocated), outputs (direct results of program or service activities) and outcomes (bro...
Read more

Q & A: Outcome vs. Measure vs. Target vs. Standard

Q: Those of who write and speak about court performance measurement rely on their share of jargon -- and it's often confusing. What are the differences among the terms “outcome,” “measure,” “target” and “standard”? A: Socrates said that the beginning of wisdom is definition. We’ll see. Outcomes are the benefits or changes for the intended beneficiaries of a court’s programs and services. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition or status of the program participants or recipients of services. Examples include litigants' satisfaction with a court's courtesy and responsiveness, success of probation, time to case disposition, clarity of orders, integrity of case files, percent of mediation agreements, enforcement of orders, and perceived fairness of proceedings, as well as percent of expected expenditures and percent of revenues received. Outcomes should not be confused with the activities of those who run the court. (See the October 1...
Read more

Taming “Wild Problems”: Measure Everything That Matters

Every problem is not solvable, but every problem is measurable. This assertion is the key to tackling problems in public services including courts, business, and life in general. Surprising to me is that even astute scholars take exception to this, among them  is Russ Roberts, the John and Jean De Nault Research Fellow at Stanford University’s Hoover Institution, the president of Shalem College in Jerusalem, and host of the award-winning podcast EconTalk: Conversation for the Curious. The belief that certain things, dubbed “wild problems” by Professor Roberts, cannot be measured does a disservice to courts and other public and private organizations. There are many benefits to measuring everything that matters. It allows organizations to   focus on and to operationalize the most important problems and issues facing them,   to track progress over time and identify areas of improvement,   to compare their performance to that of other organizations and learn from the d...
Read more