Comparing Apples and Oranges … And Learning from It

In his latest Public Management Report, Bob Behn argues that we have more to learn from dissimilar organizations than from ones that are very much like ours. He presents a powerful counterargument to a reason often raised for not doing comparative performance measurement – i.e., efforts that focus on a court’s performance in relation to that of other courts. The reason goes something like this: What we do, and the environment in which we operate, are so unique that we can’t be compared with other courts. And, because no two courts and no two jurisdictions are the same, any comparisons would be like comparing apples and oranges. We can only learn from courts that are exactly like us. End of discussion.

This is a “brilliantly convenient excuse,” says Behn. Managers believe that nothing that they are not already doing works; and if it does, it must mean that the circumstances in which it works must be “weird.” So why bother even trying to learn from “them.” Whatever they might be doing would never work for us because we’re unique and they are different.

Behn turns this apples vs. oranges argument against comparative performance measurement on its head. He acknowledges that no two organizational circumstances are alike. Therefore, he writes, all learning must come from dissimilar situations. This learning is not easy. It requires objective analysis, “the ability to look beyond the specifics of a situation to understand the underlying principles.” To do the hard work of learning, you have to separate the real innovation, the core idea, from the clutter of “weird” organizational factors, deviant personalities, and strange politics.

For court managers, Behn’s lesson is important. Courts shouldn’t hide behind their differences. The reasons for poor performance may, in fact, lie in those differences. Those differences should be confronted and exploited to identify strategies for improvement. Moreover, comparisons should not stop at courts but extend to other public agencies and even private companies. For example, courts are not the only organizations required to manage front-counter services for large numbers of “customers.” If a court can effectively benchmark its customer service performance against that of the state department of motor vehicles, or even Target, why worry about how different the court is from these organizations?

See also:

“How Do we Stack Up Against Other Courts? The Challenges of Comparative Performance Measurement,” The Court Manager, Vol. 19, No.4 (Winter 2004-2005)

Eight Reasons Not To Measure Court Performance, Made2Measure, April 5, 2006

For the latest posts and archives of Made2Measure click here.

Copyright CourtMetrics 2006. All rights reserved.

Comments

Post a Comment

Popular posts from this blog

Top 10 Reasons for Performance Measurement

There is a growing demand of “show me the data” from citizens and taxpayers, legislators, executive agencies, oversight boards and communities – all of whom exchange money and other support (e.g., trust and confidence) for services -- that courts can ill afford to ignore. Courts need to measure their performance to meet the demand for public accountability. But the benefits of court performance extend beyond accountability including improved prediction, better understanding and control, influence, a sharpened focus on what is important, and discovery of better practices. Objective performance data is central to good management practices like strategic planning, performance-based budgeting, Balanced Scorecards, Total Quality Management (TQM), and Six Sigma. Performance measures allow court leaders and managers, policymakers, legislators and the public to evaluate court programs’ inputs (the resources allocated), outputs (direct results of program or service activities) and outcomes (bro
Read more

Q & A: Outcome vs. Measure vs. Target vs. Standard

Q: Those of who write and speak about court performance measurement rely on their share of jargon -- and it's often confusing. What are the differences among the terms “outcome,” “measure,” “target” and “standard”? A: Socrates said that the beginning of wisdom is definition. We’ll see. Outcomes are the benefits or changes for the intended beneficiaries of a court’s programs and services. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition or status of the program participants or recipients of services. Examples include litigants' satisfaction with a court's courtesy and responsiveness, success of probation, time to case disposition, clarity of orders, integrity of case files, percent of mediation agreements, enforcement of orders, and perceived fairness of proceedings, as well as percent of expected expenditures and percent of revenues received. Outcomes should not be confused with the activities of those who run the court. (See the October 1
Read more

A Logic Model of Performance Inputs, Outputs and Outcomes

Increasingly, the courts' stakeholders -- justice system partners, the general public, litigants, jurors, witnesses, and even the courts’ own court employees -- demand clear evidence that the resources courts expend actually produce benefits for people. Looking for this evidence starts early in the development of a court performance measurement system (CPMS) (see Step 2, Identifying and Defining the Performance Measures Needed and Desired to Help Achieve Goals, October 15, 2005 posting, “Six-Step Process for Building an Effective Court Performance Measurement System”). For every strategic initiative, program or specific strategy, court managers should develop a simple logic model that describes how the initiative, program or strategy theoretically works to achieve the desired results from program inputs through end outcomes. Critical thinking and discussion of the logic of how a proposed program or strategy may bring about results that benefit citizens and the community can reap g
Read more