Made2Measure

While experts in performance measurement might differ in their views about how many is best, they agree that users can handle only a critical few performance measures at any one time and in one view . A court may have only one core measure of community safety, security and well being like recidivism, for example, but it may use numerous subordinate measures of recidivism at a strategic level (e.g., average recidivism by case categoreis and types), tactical level (e.g., recidivism for certain units like drug courts and probation departments) and operational levels (e.g., recidivism of probationers who particpated in a particular rehabilitation program). In addition, a court or its justice partners may usefully monitor, analyze and manage a host of meaures related to community safety, security and well being like the change in police contacts with families with cases in the jurisdiction of a family court. The point is that the users of these recidivism measures have selective attent...

A: Why not simply adopt the set of 10 performance measures prescribed by the CourTools or by other authoritative sources and dispense with the six steps for building a court performance measurement system (See October 15, 2005, posting, Six-Step Process for Building an Effective Court Performance System (CPMS) )? After all, the CourTools are endorsed by the Conference of Chief Justices (CCJ) and the Conference of State Court Administrators (COSCA) (see Resolution 14 ). Why reinvent the wheel? Q: This is an important question, especially in view of the significant investment of time and resources required for building a court performance measurement system (CPMS). Unquestionably, models such as the CourTools are extremely valuable. No individual court, no court system, and no justice system (that includes courts) considering performance measurement should – and none of the dozen or so my colleagues have assisted in the last year did -- proceed without first studying the CourTools ....

In his 1996 book, Keeping Score: Using the Right Metrics to Drive World-Class Performance (Quality Resources, a Division of Kraus Productivity Organization Ltd.), Mark Graham Brown delivered a practical common-sense guide on how to develop and use performance measures as tools for world-class performance. In his latest book, Get It, Set It, Move It, Prove It: 60 Ways to Get Results in Your Organization (Productivity Press, 2004), he puts a sharper point on some the issues he raised in his earlier book. Forget “statistically significant” when assessing organizational performance, he writes in Chapter 49. Distinguishing between organizational performance measurement and science, he states that the concept of statistical significance is critically important for science to rule out that the differences between the experimental and control groups are due to the independent variable and not to chance. For example, scientific researchers may be focused on whether the differences in lowered ...

A previous post ( Implementing Performance Measurement , November 12, 2005), explored what it takes to implement a court performance measurement system (CPMS). It reached the conclusion that even a well-conceived, well-designed CPMS will not necessarily get used unless it is woven into the fabric of a court’s management practices and processes. While such integration with strategic planning, performance-based budgeting and other formal management processes may be more demanding (more on this will follow in future postings), some relatively simple things can be done with powerful effects. Two simple techniques that facilitate the implementation of a CPMS are: (1) making the review of the court's core performance results a permanent agenda item on the court’s executive meetings and (2) assigning ownership of core measures to key managers. Consider the following ideal scenario. Your court has successfully built a CPMS that includes a set of eight core performance measures aligned with...

Q: In recent postings you wrote about getting the right people on the bus (November 23, 2005) and about creating the right attitude (December 4, 2005). But how do you handle those in the court who are not at all convinced by Peter Drucker’s admonition, “You can’t manage what you can’t measure”? They are smart and thoughtful people who are philosophically resistant to performance measurement. “We don't work on an assembly line making widgets,” they might say. “We don’t work with machines. We deal with unique cases involving human beings who have complex problems that require individual attention. You can’t just send in the bean counters and number crunchers and crank out a bunch of performance indicators to capture what we do on a spreadsheet.” A: You describe a common attitude throughout government, not just in the courts. Underlying this negative attitude toward performance measurement -- what Bob Behn , a lecturer at Harvard University’s John F. Kennedy School of Governm...

Q: My court wants to start using the CourTools and the Six-Step Design Process (see October 15, 2005 Made2Measure post) for building a court performance measurement system (CPMS). Beyond getting the right people on the bus (November 23, 2005 Made2Measure post), what do we need to get started? A: You need the right attitude. The real value of performance measurement is not the performance measures themselves but rather the fundamental questions they force you to ask about your court’s programs and services. How is your court performing today ? Where is it heading in the future ? What are your major challenges? What are you going to do about them? It’s all about the kind of serious self-inquiry, self-analysis, and learning that is at the heart of good management. Performance measurement is not an end, but a means to an end. Kevin Baum, a long-time consultant and practitioner of performance measurement, wrote in a recent issue of Perform that performance measurement and management “i...

Q: Step 6 – Building Performance Measurement Displays – of the Six-Step Design Process for an effective court performance measurement system (CPMS) requires the design, development, and implementation of performance measurement displays that provide relevant and meaningful information that can be quickly accessed and easily understood by the intended users (see the Made2Measure October 15, 2005 post). The step suggests that courts begin by reviewing the functionality of commercial computer software referred to as performance management or “business intelligence” (BI) solutions that are offered by an increasing number of companies. Courts can then decide to buy or to build their own computer-based performance display systems. But what about courts, especially small courts, that do not have the money to buy, or the technology resources to build, sophisticated computer-based performance display systems? Are such systems a necessary requirement for success? management? A: Not necessari...

Q: My court wants to start using the CourTools and the Six-Step Design Process (see October 15, 2005, Made2Measure Posting) for building a court performance measurement system (CPMS). How do we get started? Who does it? How do we organize ourselves? A: To start, get the right people on the bus. Sustaining the effort of planning, developing and implementing a court performance measurement system (CPMS) requires a guiding coalition of individuals who command both the resources and the respect of the court, and who can maintain the energy and momentum to keep the initiative going through its four phases (see below). The court should start the initiative by assembling a 9 to 15-member steering committee to direct and oversee the work. The steering committee, as part of its early planning, in turn, identifies a 6 to 12-member design team to take the steps of building a court performance measurement system (CPMS) in the second phase of the initiative. These two groups form the guidin...

In the 1989 film Field of Dreams , an Iowa corn farmer (Kevin Costner) hears voices that tell him, “If you build it, they will come.” He interprets this message as a command to build a baseball field on his farm. He does and they -- Shoeless Joe Jackson and the other seven Chicago White Sox players banned from the game for throwing the 1919 World Series – come. This works in the movies but it does not work for court performance measurement. It is one thing to build a court performance measurement system (CPMS) and quite another to get the CPMS to be used effectively. A Pile of Stethoscopes Kevin Baum, a performance management consultant who works in government outside of the courts, warns us in a recent edition of Perform (Special Edition, Government, no date) that we make a fatal mistake when we declare victory too soon, that is, immediately after we have built a CPMS (see the October 15, 2005, Posting, “Six-Step Process for Building an Effective Court Performance Measurement Sys...

Q: Here’s a concern I hear often about performance measurement: Serious initiatives to build and implement court performance measurement systems (CPMS) will raise false expectations among court staff and other stakeholders that the court will actually make improvements in response to performance results. The concern is based in the fear that court leaders will be caught flat-footed responding to performance results (e.g., low public satisfaction with court services, sub-par on-time performance in case processing, and rates of compliance with court orders that fall short of benchmarks). A: While a poor performance showing is rarely a cause for rejoicing, the concern that performance measurement itself will raise false expectations that something will be done is misplaced. The concern misses the power of performance measurement to engage individuals at all levels, not just top managers, in improvement strategies. It also makes the questionable assumption that all change for the better ...

Q: “What is a public accountability framework ?” This question came this week from Alice Phalan, Planning and Evaluation Manager of the Oregon Judicial Department, after she saw the term as a topic in an upcoming meeting in her state, and after she “googled” it and came up empty. Q: The term public accountability framework , or PAF , was introduced earlier this year by my colleagues and me at the National Center for State Courts and CourtMetrics in the context of building organizational performance measurement systems for courts and justice systems in Arizona and British Columbia. In the argot of management, we’ll see if it has any legs! PAF in Mesa, Arizona In August this year the Mesa (Arizona) Municipal Court launched the “Good to Great” Public Accountability Framework (PAF) Project, an ambitious six-month project of building its performance measurement capacity, process improvement, courthouse design and planning, technology development, and strategic planning. The central task o...

Increasingly, the courts' stakeholders -- justice system partners, the general public, litigants, jurors, witnesses, and even the courts’ own court employees -- demand clear evidence that the resources courts expend actually produce benefits for people. Looking for this evidence starts early in the development of a court performance measurement system (CPMS) (see Step 2, Identifying and Defining the Performance Measures Needed and Desired to Help Achieve Goals, October 15, 2005 posting, “Six-Step Process for Building an Effective Court Performance Measurement System”). For every strategic initiative, program or specific strategy, court managers should develop a simple logic model that describes how the initiative, program or strategy theoretically works to achieve the desired results from program inputs through end outcomes. Critical thinking and discussion of the logic of how a proposed program or strategy may bring about results that benefit citizens and the community can reap g...

The key to the successful use of performance measurement is to identify and develop the right measures – measures that are aligned with vision, goals and success factors -- and then making sure that the measures get into the hands of those that can make good use of them at the right time. This requires a Court Performance Measurement System (CPMS). A CPMS is the routine collection, analysis, synthesis, delivery and display of performance measures and other information about the work and accomplishment of a court as an organization. It is an ordered and comprehensive assembly of parts -- interrelated data, principles, standards, methods, processes and procedures -- forming a unitary whole. All the elements of a CPMS -- the performance measures, data collection methods, analysis and interpretation, and information distribution and display -- operate toward the common purpose of running a court effectively and efficiently. Over the last five years, by benchmarking the design processes use...

A note of caution for court managers: There is today a consensus that a court’s performance measures should provide critical information for developing and justifying budgets. The need and rationale for performance-based budgeting are not in dispute. However, there are serious concerns about how it can be done. Need and Rationale Are Indisputable Budgeting is a rational and systematic way of estimating and allocating resources. Performance based budgeting or results-based budgeting, and related approaches like activity-based budgeting (ABB) and activity-based costing (ABC), are methods of budgeting that link resource estimates and appropriations to outputs and outcomes (see September 28, 2005 Post, “A Preference for Outcome Measures”). For many court managers, making their budget process more performance-based is the primary motivation for developing court performance measurement systems (CPMSs). The major elements of performance-based budgeting are defining goals and objectives, deve...

It is common for large and medium-size courts to have research units or departments that employ researchers with titles like “analyst,” “statistician,” or “planner,” if not “researcher.” No courts today -- to my knowledge -- have units or departments devoted solely to performance measurement. Only a handful of court-employed researchers consider their job to be performance measurement. This situation is likely to change as courts become more familiar with the functions and uses of performance measurement, and as the differences between performance measurement and research become more apparent. “Performance measurement” is the process of measuring an organization’s accomplishments (outcomes), work and service levels (output), and its resources (inputs). It is any effort undertaken to meet the need for evidence of an organization’s performance on a regular and continuous basis. Behavioral and social “research” (including evaluation research and program evaluation), on the other hand, is ...