Step 2 – Identifying Desired Performance Measures (Sub-Steps 2.1 and 2.2)

Sixth posting in a multi-part series exploring the Six-Step Process for Building an Effective Court Performance Measurement System (CPMS) first summarized in Made2Measure (M2M) in October 2005.

Step 2 is the most conceptual of the six steps. It demands thoughtful consideration of the purpose of performance measures – mission, strategic goals, direction, clarity and strategy -- not just their mechanics. The amount of time and effort the design team devotes to the four sub-steps of Step 2, and how deliberately the team takes all four, will vary depending on how much prior work a court has done to define its mission, direction and strategic goals. This prior work may have taken the form of a key management process, such as strategic planning, or a more targeted effort, such as performance-based budgeting or process mapping.

Step 2.1 - Identifying Key Success Factors

The first sub-step entails clarifying the court's key success factors or performance areas. These factors or areas have different labels including “performance measurement frameworks,” “critical success factors,” “perspectives,” “domains,” “performance criteria,” “key results,” and “key outcomes.”

The validity of your indicators depends on their relationship to the outcomes you seek to achieve and the ability of different people to calculate their value consistently to obtain comparable results over time. While the process of developing indicators, therefore, inevitably shifts back and forth between assessing the reliability of available data and clarifying the desired outcomes, it is crucial to start precisely defining the outcomes you aim to achieve.

Vera Institute of Justice, Global Guide to Performance Indicators (2003)

Performance measures must be aligned with the unique mission, strategic goals, values and other guiding ideas of a court. The requirement of a linkage of performance measures to a court’s mission is critical. Figuratively and literally, performance does not count unless it is related to the things that really matter to the court. Court leaders and managers must determine what performance information is needed to lead, manage and run the court. In his very practical introductory text, The Basics of Performance Measurement (Portland, OR: Productivity Press, 1997), Jerry Harbor suggests that completing the statement, "I need performance-related information in order to..." will help in identifying the key performance areas in which to collect performance information. What is the mission of the court? What are its programs and services intended to achieve? What are the things that you are committed to accomplishing? Effective performance measures are based on the answers to these fundamental questions.

The result of Step 2.1 is the identification of factors critical to the success of the court. Here are 16 examples of success factors identified by courts:

  1. access to justice;
  2. expedition and timeliness;
  3. fairness, equality and integrity;
  4. independence and accountability;
  5. public trust and confidence;
  6. finance;
  7. client-customer satisfaction;
  8. internal processes;
  9. innovation and learning;
  10. leadership;
  11. strategic planning;
  12. customer and market;
  13. information and analysis;
  14. process management;
  15. human resources; and,
  16. business results.

The first five examples are the fundamental court performance areas defined in the Court Performance Standards. They are also the success factors upon which the CourTools are based. They represent outcomes the public wants from the courts. That is, citizens want ready access to the justice delivered by the courts; they want that access to be safe, relatively convenient, and affordable. Once they have gained access, they want their business with the courts dealt with expeditiously and fairly, according to the facts and according to established rules. They want their disputes to get individual attention and to be dealt with fairly. And they want their courts to behave independently of other branches of government and other agencies to assure that decisions and actions are based solely on legally relevant factors. Ultimately, they seek trust and confidence in the courts. The five performance areas identify these civic ideals.

Examples 6 through 9 are the four "perspectives" of the Balanced Scorecard – a measurement-based management system first advanced by Robert S. Kaplan and David P. Norton in the early 1990s. Examples 10 through 16 are the Malcolm Baldridge Quality Award Program criteria. The Balanced Scorecard identifies and integrates four different ways of looking at performance (financial, customer, internal business, and innovation and learning perspectives). For example, the customer perspective of the Balanced Scorecard (How do customers see us?) focuses on the concerns of the users of a court including timeliness, quality, performance and service, and cost. Example 9, the innovation and learning perspective (Can we continue to improve and create value?), focuses on a court's ability to innovate, improve and learn. The Fourth District Court in Hennepin County, Minnesota, which has used the Malcolm Baldridge Quality Award Program criteria as part of its improvement efforts, defines leadership (Example 10) as the manner in which the court's senior leaders address values, direction and performance expectations, as well as their focus on customers, stakeholders, empowerment, innovation and learning.

Of course, none of these 16 examples should dictate the success factors for your court. They serve as models or starting points. Few would take issue with the need for a court to formulate its own unique guiding ideas -- the values, purpose, and vision of its future that define its organizational "character.” Just as a court's fundamental values cannot be pulled off-the-shelf, a court's unique key success factors, which are tied to its character, must be fashioned by the court. At their best, the examples can give you a good starting point for identifying where you might begin to look for performance measures and how to group and balance them after you have identified them. The five performance areas of the Court Performance Standards -- which are aligned with guiding principles, goals and performance measures -- are the most well-articulated and are perhaps the best model. They were developed for state courts by court experts and practitioners and have the endorsement of the courts community. Reliance on the perspectives of the Balanced Scorecard and the Baldridge Quality Award Program criteria may add the cachet that private sector innovations and seem to hold for courts.

There is no need to certify or endorse the identification of key success factors before proceeding to the next sub-step of determining the types of measures needed. A set of key success factors or performance areas acceptable to the court may not reveal itself until late in the development of a CPMS, especially for courts that have not engaged in prior strategic planning or goal setting. Therefore, courts should not get stuck in Step 2.1 but instead make a preliminary identification of four to seven success factors and quickly move on to considerations of desired performance measures. Invariably, these considerations will trigger a reexamination of the tentative list of key success factors.

Step 2.2 - Determining Types of Measures

It is better to have a few meaningful indicators than multiple poor ones, and it is preferable to concentrate on outcome indicators rather than resorting to easy-to-develop workload or input indicators.

Morley, Bryant and Hatry, Comparative Performance Measurement (Urban Institute, 2001)

The next sub-step entails the consideration of the types of measures to be included in the CPMS. Are the desired measures likely to be focused on inputs, outputs or outcomes? Are they broad court-wide measures (e.g., public trust and confidence in the court) or narrowly focused measures (e.g., percent of witnesses that appear in court) that may serve as proxies for broader measures? From what data sources are they likely to be drawn – administrative records of cases or finances, surveys, or third-party databases? Do the sources of data exist or must they be created? What are the likely costs of taking the measures?

It is preferable to concentrate on outcome measures -- indicators of events, occurrences and conditions important to court users and stakeholders -- rather than measures of resources (inputs) and activities and processes (outputs) used to produce the outcomes. Outcome measures reflect the court's accomplishments not merely the work it performs. The best outcome measures provide easy-to-understand information on events, occurrences, conditions, and changes in the attitudes and behaviors in the recipients of the court's services and participants in its programs. Outcome measures provide the best incentives and the tools for improvements in court policy and practice.

The result of Step 1 -- the assessment of the completeness and balance of the court’s current performance measures across factors that drive the court’s success – should facilitate this sub-step. How many of the current measures focus on the highest policy levels that indicate progress toward strategic goals and purpose (i.e., outcomes)? Are most of the court’s measures focused on case processing and little else? How many of the catalogued measures focus on such key success factors as access to justice, court user satisfaction, procedural fairness, independence and accountability, and public trust and confidence?

The result of Step 2.2 is a tentative determination of the general types of desired measures aligned with the identified key success factor. For example, the CPMS design team of a municipal court may conclude with Step 2.2 having concluded that it wants to measure public trust and confidence -- one of six success factors identified in Step 2.1 -- by a broad-based outcome measure of court user satisfaction determined by survey and a more narrow output measure of speed of service delivery at the court’s public counters.

Next in this series: Step 2.3, Identifying Core Measures, and Step 2.4, Defining Measures in Operational Terms.

Previous postings in this series:

  1. Introduction to the Six-Step Process for the Design of an Effective Performance Measurement System (Part 1), Made2Measure, June 6, 2006
  2. Introduction to the Six-Step Process (Part 2), Made2Measure, June 12, 2006
  3. Step 1. Assessing Currently Used Performance Measures, Made2Measure, June 17, 2006
  4. Q & A: Can Step 1 Be Taken At the State-Level, Made2Measure, June 23, 2006
  5. Step 2. Identifying Desired Performance Measures (Part 1), Made2Measure, July 2, 2006

For the latest posts and archives of Made2Measure click here.

Copyright CourtMetrics 2006. All rights reserved.

Comments

Post a Comment

Popular posts from this blog

Top 10 Reasons for Performance Measurement

There is a growing demand of “show me the data” from citizens and taxpayers, legislators, executive agencies, oversight boards and communities – all of whom exchange money and other support (e.g., trust and confidence) for services -- that courts can ill afford to ignore. Courts need to measure their performance to meet the demand for public accountability. But the benefits of court performance extend beyond accountability including improved prediction, better understanding and control, influence, a sharpened focus on what is important, and discovery of better practices. Objective performance data is central to good management practices like strategic planning, performance-based budgeting, Balanced Scorecards, Total Quality Management (TQM), and Six Sigma. Performance measures allow court leaders and managers, policymakers, legislators and the public to evaluate court programs’ inputs (the resources allocated), outputs (direct results of program or service activities) and outcomes (bro
Read more

The “What Ifs” Along the Road in the Quest of a Justice Index

What if we could draw an indicator of the “health” of the justice systems throughout the United States – a Justice Index - from readily available public data that is impartial, reliable, objective and comparable, let’s say the amount of time that criminal defendants spend in custody without a trial, or even without any access to legal or justice services? Or the bail amounts imposed on misdemeanor defendants?   This kind of a measure would not be perfect – no measure is – but it would be something, better than nothing. It would address questions of fairness and efficient use of public resources. That is, a criminal defendant languishing in jail awaiting trial suffers, at the least, a delay in justice. He or she is also occupying a jail bed and spending taxpayers’ money. What if we then join this measure with another meaningful indicator of the health of our justice systems in counties and states, let’s say the degree to which jurisdictions prosecute domestic violence cases.
Read more

Q & A: Outcome vs. Measure vs. Target vs. Standard

Q: Those of who write and speak about court performance measurement rely on their share of jargon -- and it's often confusing. What are the differences among the terms “outcome,” “measure,” “target” and “standard”? A: Socrates said that the beginning of wisdom is definition. We’ll see. Outcomes are the benefits or changes for the intended beneficiaries of a court’s programs and services. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition or status of the program participants or recipients of services. Examples include litigants' satisfaction with a court's courtesy and responsiveness, success of probation, time to case disposition, clarity of orders, integrity of case files, percent of mediation agreements, enforcement of orders, and perceived fairness of proceedings, as well as percent of expected expenditures and percent of revenues received. Outcomes should not be confused with the activities of those who run the court. (See the October 1
Read more