Six-Step Process for Building an Effective Court Performance Measurement System (CPMS)
The key to the successful use of performance measurement is to identify and develop the right measures – measures that are aligned with vision, goals and success factors -- and then making sure that the measures get into the hands of those that can make good use of them at the right time. This requires a Court Performance Measurement System (CPMS). A CPMS is the routine collection, analysis, synthesis, delivery and display of performance measures and other information about the work and accomplishment of a court as an organization. It is an ordered and comprehensive assembly of parts -- interrelated data, principles, standards, methods, processes and procedures -- forming a unitary whole. All the elements of a CPMS -- the performance measures, data collection methods, analysis and interpretation, and information distribution and display -- operate toward the common purpose of running a court effectively and efficiently.
Over the last five years, by benchmarking the design processes used successfully in private sector and public organizations, and by extensive field study of courts throughout the world, CourtMetrics and colleagues have developed a best practice design process of six overlapping steps and sub-steps (tasks). This design process introduces discipline, conceptual clarity and method to the myriad conceptual, methodological, analytic issues associated with the building a CPMS.
Step 1. Assessing the performance measures currently used.
Step 2. Identifying and defining the performance measures needed to help achieve goals.
Step 3. Developing hierarchies or families of measures.
Step 4. Testing the measures.
Step 5. Creating data collection and distribution methods that ensure timeliness and utility.
Step 6. Building useful performance measurement displays.
Although the steps are laid out in a linear sequence, taking these steps is iterative in practice. The design team may rethink the results of steps several times before reaching a consensus and final decision.
Each of the essential building steps addresses critical questions: How does the court currently measure its performance? How are the current measures distributed across inputs, outputs, outcomes, key success areas, perspectives, and core performance areas? What required or desired performance information currently is not available to the court? What specific performance measures would provide that information? How does the court select a vital few, instead of a trivial many, performance measures? How should the court go about developing the desired performance measures? To what degree should the selected measures be tried and demonstrated before implementation? When and how should the performance data be collected? To whom and how often should it be distributed? In what format and on what schedule should the performance information be conveyed and displayed?
Step 1. Assessing Current Performance Measures
Purpose: What performance measures are currently used in your court and by your court’s justice system partners? Are they sufficient in terms of number, type, and balance of perspectives? Which of the measures seem more important than others? By what methods are the measures taken and by whom? Taking a critical look at the performance measures your court and justice system partners are using today is a practical starting point for getting to the fundamental question of what are the right measures for your court. Rather than trying to tackle this daunting question with a blank slate, this first step draws on knowledge already available to the court. It produces results in a relatively short time. Even if a court postpones or abandons the building of a CPMS, both the process and the results of this first step can serve as valuable references for planning and development of various court improvement projects.
Tasks: The products of this first step are an inventory of input, output and outcome measures currently used by the court and its divisions, departments and units, and an evaluation of the adequacy of the measures as a basis for building a CPMS. Three tasks are required to complete this step: (1) cataloguing the court’s performance measures and indicators, as well those of the court’s justice system partners; (2) categorizing the measures by whether they are input (resources), output (activities) or outcome (impact) measures; and (3) assessing their completeness and balance.
Step 2 – Developing Desired Performance Measures
Purpose: The second step places a premium on conceptual clarity and the operational definition of values, principles, and fundamental success factors that are at the heart of a court’s purpose –access and fairness, efficiency and effectiveness, public trust and confidence. It requires the court to identify its key performance areas, determine the types of performance measures to be included, select a limited number core measures, and define the selected measures operationally. People must sign on to the purpose of a performance measure, the key outcome it indicates, not just the metric. Although there is no ideal number of measures that should be identified, it is better to have a few meaningful performance measures than many poor ones. It is also preferable to select measures that indicate the desired outcomes of the court's programs and services rather than measures of the resources (input) or completed activities (outputs) used to produce those outcomes.
Tasks: The four tasks of this second step proceed from the general to the specific, from broad conceptualization of performance areas to the operational definition of desired measures. The step produces a synopsis of seven to twelve core performance measures, operationally defined and linked to key court performance areas.
Step 3 – Creating Measurement Hierarchies
Purpose: Using the results of the previous two steps –an inventory all the performance measures currently used at all levels of the court a set of high-level core performance measures – the third step of building a CPMS requires the creation of hierarchies (families) of related performance measures. The value of measurement hierarchies is that they define the connection between high-level strategic goals and performance measures with lower-level departmental or divisional objectives and measures. Measurement hierarchies in which lower-level subordinate measures cascade down from core measures help align the overall goals of the court with the goals and objectives of its divisions, units, and programs. They help make it clear to all court employees precisely how their actions help fulfill the court’s mission and strategic goals.
Tasks: Step 3 requires three overlapping tasks: (1) breaking out (disaggregating) core measures; (2) identifying and defining other subordinate measures aligned with core measures that are not mathematical breakouts of core measures; and (3) constructing a hierarchy of performance measures for each of the core measure.
Step 4 – Testing, Demonstrating and Developing Measures
Purpose: Once steps have been taken to identify and define the desired performance measures, and to construct hierarchical relationships among them, simply issuing an edict to "go forth and measure" is likely to invite failure. It is one thing to identify and define performance measures, and quite another to demonstrate that a court can actually take the measures given its current operating structures, systems and procedures. Court staff should be provided detailed directions, as well as encouragement, to take the prescribed measures. Procedures for planning and preparations for the measures, data sources, data collection methods, analysis, and distribution and use of the measures should be carefully prescribed. The measures should then be field tested, demonstrated, and modified as necessary.
Tasks: This step requires three tasks. The first task is to describe the measures in sufficient detail to allow the testing and demonstration of the measures, which is the second task. The third task is to develop and refine the measures based on their test and demonstration.
Step 5 – Developing Data Collection and Reporting Timeframes
Purpose: Once you have taken the first four steps of building a CPMS -- identified the desired core performance measures based on assessment of your courts current performance measures, created measurement hierarchies of related measures, tested and demonstrated the measures -- it is time to make sure that the right performance data gets into the hands of the right people at the right time. How frequently should the data for each measure be collected and displayed? Who needs the performance data provided by the core measures and subordinate measures in the allied hierarchies? How will they use that data? When do they need it and how often will they use it?
Tasks: This step requires two discrete tasks. The first is to determine the ideal timeframe for availability (e.g., data may already available in real-time in automated databases) and use of the performance data for each core performance measure. The second task is to consider feasibility and costs of data collection and display and to adjust the ideal timeframes accordingly. The result of the step is a timeframe for collection and distribution of each core and subordinate measure in all of the measurement hierarchies.
Step 6 – Building Performance Measurement Displays
Purpose: Courts today are drowning in data and will succeed to the extent that they are able to harness performance information to make better, more informed and quicker decisions. Collecting and assembling critical performance data are useless if the data are not delivered to the right people at the right time. Although standardized or special reports are useful for many users, they can not meet the needs of users who need more rapid and flexible data access. An effective display of a CPMS is one that users can readily access, that is easily read and understood, that is organized for easy navigation among core and subordinate measures, and one that provides a “line of sight” that conveys to everyone what the drivers of success are and provides them with the concrete knowledge of how they contribute to that success.
Tasks: The sixth and final step of building a CPMS is the design, development, and implementation of performance measurement displays – presentations that allow end users to access, to view and to use the performance data. The first task is to decide how the display should look and function. A good place to start the design of a display system is with a study of the functionality of computer software products collectively referred to as performance management solutions or “business intelligence” (BI) offered by an increasing number of companies. The task is complete with a decision to buy or to build a computer-based performance display system. The second task of this step includes not only the actual writing of computer code but also the preparation of requirements and objectives, the design of what is to be coded, as well as testing and confirmation that what is developed has met the objectives. It should proceed through successive phases – from analysis of software requirements through system integration and testing --that are familiar to computer software engineers.
-----------------
For the latest Made2measure postings and archives click here
-----------------
Copyright CourtMetrics 2005. All rights reserved.
Over the last five years, by benchmarking the design processes used successfully in private sector and public organizations, and by extensive field study of courts throughout the world, CourtMetrics and colleagues have developed a best practice design process of six overlapping steps and sub-steps (tasks). This design process introduces discipline, conceptual clarity and method to the myriad conceptual, methodological, analytic issues associated with the building a CPMS.
Step 1. Assessing the performance measures currently used.
Step 2. Identifying and defining the performance measures needed to help achieve goals.
Step 3. Developing hierarchies or families of measures.
Step 4. Testing the measures.
Step 5. Creating data collection and distribution methods that ensure timeliness and utility.
Step 6. Building useful performance measurement displays.
Although the steps are laid out in a linear sequence, taking these steps is iterative in practice. The design team may rethink the results of steps several times before reaching a consensus and final decision.
Each of the essential building steps addresses critical questions: How does the court currently measure its performance? How are the current measures distributed across inputs, outputs, outcomes, key success areas, perspectives, and core performance areas? What required or desired performance information currently is not available to the court? What specific performance measures would provide that information? How does the court select a vital few, instead of a trivial many, performance measures? How should the court go about developing the desired performance measures? To what degree should the selected measures be tried and demonstrated before implementation? When and how should the performance data be collected? To whom and how often should it be distributed? In what format and on what schedule should the performance information be conveyed and displayed?
Step 1. Assessing Current Performance Measures
Purpose: What performance measures are currently used in your court and by your court’s justice system partners? Are they sufficient in terms of number, type, and balance of perspectives? Which of the measures seem more important than others? By what methods are the measures taken and by whom? Taking a critical look at the performance measures your court and justice system partners are using today is a practical starting point for getting to the fundamental question of what are the right measures for your court. Rather than trying to tackle this daunting question with a blank slate, this first step draws on knowledge already available to the court. It produces results in a relatively short time. Even if a court postpones or abandons the building of a CPMS, both the process and the results of this first step can serve as valuable references for planning and development of various court improvement projects.
Tasks: The products of this first step are an inventory of input, output and outcome measures currently used by the court and its divisions, departments and units, and an evaluation of the adequacy of the measures as a basis for building a CPMS. Three tasks are required to complete this step: (1) cataloguing the court’s performance measures and indicators, as well those of the court’s justice system partners; (2) categorizing the measures by whether they are input (resources), output (activities) or outcome (impact) measures; and (3) assessing their completeness and balance.
Step 2 – Developing Desired Performance Measures
Purpose: The second step places a premium on conceptual clarity and the operational definition of values, principles, and fundamental success factors that are at the heart of a court’s purpose –access and fairness, efficiency and effectiveness, public trust and confidence. It requires the court to identify its key performance areas, determine the types of performance measures to be included, select a limited number core measures, and define the selected measures operationally. People must sign on to the purpose of a performance measure, the key outcome it indicates, not just the metric. Although there is no ideal number of measures that should be identified, it is better to have a few meaningful performance measures than many poor ones. It is also preferable to select measures that indicate the desired outcomes of the court's programs and services rather than measures of the resources (input) or completed activities (outputs) used to produce those outcomes.
Tasks: The four tasks of this second step proceed from the general to the specific, from broad conceptualization of performance areas to the operational definition of desired measures. The step produces a synopsis of seven to twelve core performance measures, operationally defined and linked to key court performance areas.
Step 3 – Creating Measurement Hierarchies
Purpose: Using the results of the previous two steps –an inventory all the performance measures currently used at all levels of the court a set of high-level core performance measures – the third step of building a CPMS requires the creation of hierarchies (families) of related performance measures. The value of measurement hierarchies is that they define the connection between high-level strategic goals and performance measures with lower-level departmental or divisional objectives and measures. Measurement hierarchies in which lower-level subordinate measures cascade down from core measures help align the overall goals of the court with the goals and objectives of its divisions, units, and programs. They help make it clear to all court employees precisely how their actions help fulfill the court’s mission and strategic goals.
Tasks: Step 3 requires three overlapping tasks: (1) breaking out (disaggregating) core measures; (2) identifying and defining other subordinate measures aligned with core measures that are not mathematical breakouts of core measures; and (3) constructing a hierarchy of performance measures for each of the core measure.
Step 4 – Testing, Demonstrating and Developing Measures
Purpose: Once steps have been taken to identify and define the desired performance measures, and to construct hierarchical relationships among them, simply issuing an edict to "go forth and measure" is likely to invite failure. It is one thing to identify and define performance measures, and quite another to demonstrate that a court can actually take the measures given its current operating structures, systems and procedures. Court staff should be provided detailed directions, as well as encouragement, to take the prescribed measures. Procedures for planning and preparations for the measures, data sources, data collection methods, analysis, and distribution and use of the measures should be carefully prescribed. The measures should then be field tested, demonstrated, and modified as necessary.
Tasks: This step requires three tasks. The first task is to describe the measures in sufficient detail to allow the testing and demonstration of the measures, which is the second task. The third task is to develop and refine the measures based on their test and demonstration.
Step 5 – Developing Data Collection and Reporting Timeframes
Purpose: Once you have taken the first four steps of building a CPMS -- identified the desired core performance measures based on assessment of your courts current performance measures, created measurement hierarchies of related measures, tested and demonstrated the measures -- it is time to make sure that the right performance data gets into the hands of the right people at the right time. How frequently should the data for each measure be collected and displayed? Who needs the performance data provided by the core measures and subordinate measures in the allied hierarchies? How will they use that data? When do they need it and how often will they use it?
Tasks: This step requires two discrete tasks. The first is to determine the ideal timeframe for availability (e.g., data may already available in real-time in automated databases) and use of the performance data for each core performance measure. The second task is to consider feasibility and costs of data collection and display and to adjust the ideal timeframes accordingly. The result of the step is a timeframe for collection and distribution of each core and subordinate measure in all of the measurement hierarchies.
Step 6 – Building Performance Measurement Displays
Purpose: Courts today are drowning in data and will succeed to the extent that they are able to harness performance information to make better, more informed and quicker decisions. Collecting and assembling critical performance data are useless if the data are not delivered to the right people at the right time. Although standardized or special reports are useful for many users, they can not meet the needs of users who need more rapid and flexible data access. An effective display of a CPMS is one that users can readily access, that is easily read and understood, that is organized for easy navigation among core and subordinate measures, and one that provides a “line of sight” that conveys to everyone what the drivers of success are and provides them with the concrete knowledge of how they contribute to that success.
Tasks: The sixth and final step of building a CPMS is the design, development, and implementation of performance measurement displays – presentations that allow end users to access, to view and to use the performance data. The first task is to decide how the display should look and function. A good place to start the design of a display system is with a study of the functionality of computer software products collectively referred to as performance management solutions or “business intelligence” (BI) offered by an increasing number of companies. The task is complete with a decision to buy or to build a computer-based performance display system. The second task of this step includes not only the actual writing of computer code but also the preparation of requirements and objectives, the design of what is to be coded, as well as testing and confirmation that what is developed has met the objectives. It should proceed through successive phases – from analysis of software requirements through system integration and testing --that are familiar to computer software engineers.
-----------------
For the latest Made2measure postings and archives click here
-----------------
Copyright CourtMetrics 2005. All rights reserved.
Comments
Post a Comment