Core Measures and Measurement Hierarchies
While experts in performance measurement might differ in their views about how many is best, they agree that users can handle only a critical few performance measures at any one time and in one view.
A court may have only one core measure of community safety, security and well being like recidivism, for example, but it may use numerous subordinate measures of recidivism at a strategic level (e.g., average recidivism by case categoreis and types), tactical level (e.g., recidivism for certain units like drug courts and probation departments) and operational levels (e.g., recidivism of probationers who particpated in a particular rehabilitation program). In addition, a court or its justice partners may usefully monitor, analyze and manage a host of meaures related to community safety, security and well being like the change in police contacts with families with cases in the jurisdiction of a family court.
The point is that the users of these recidivism measures have selective attention and different needs for information at the strategic, tactical and operational levels and, therefore, can and will only pay attention to a limited number of them at any one time.
Core Measures
Like most complex organizations, courts may have hundreds of indicators and measures of their work and performance. However, few court managers can (or want to) keep track of more than a handful of measures and indicators. There’s nothing wrong with a court having many measures and indicators of their inputs, outputs and outcomes in their databases -- it’s just that any individual can only keep track of a vital few.
On a car dashboard, we have a few gauges that we look at all the time to help us as drive we our car, a few that need to be monitored regularly but not all the time, several that need to be looked at less frequently, maybe by a mechanic, and some that need to be attended to only when we are given a warning signal or when the car breaks down.
A “core” performance measure is a primary indicator – akin to a gauge on your car’s dashboard -- of an important area of the court performance. A core performance measure has most of the following attributes:
· Linkage to Mission – It is aligned with one or more of a court's key performance areas or success factors (e.g., access to justice). It is this linkage with key performance areas that limits the number of core measures to a vital few.
· Aggregation – It is a combination, an index, or a conjunction of a number of measures, variables or aspects of court performance that may be identified with subordinate measures.
· Outcome Focus – It emphasizes the condition or status of the recipients of court services or the participants in court programs (outcomes) over that of internal aspects of court processes, programs and activities (inputs and outputs) -- that is, they indicate results rather than resources and level of effort.
· Consistency Across Entire Court – It is consistent from the top to the bottom of the organization of the court. Each may sit at the top of a “hierarchy” of related measures and indicators (see below).
· Driver of Success -- It is both an incentive and a tool for improvement. The key to collecting data for court performance measurement is identifying those performance measures that will actually help to achieve the desired results (i.e., measures that are drivers of success).
· Emblem or Symbol -- the court and its stakeholders easily understand its meaning and significance.
The great benefit of using a vital few core performance measures – the National Center for State Courts’ CourTools include 10 core measures -- that are aligned with a court’s mission and highest-level strategic goals is the ease and efficiency with which they can be used. At best, they are consistent court-wide, highly aggregated, outcome-focused, and emblematic of a key performance area or success factor. They serve as triggers for more in-depth examination of problems or successes in key performance areas and what should be done about them.
It is, of course, of great benefit to busy court leaders and managers if they need only pay attention to a few easily accessible core measures that indicate the performances of many court programs and services, rather than the many numbers that are represented in measurement hierarchies. They quickly see the big picture. When the numbers change, they know they need to act. The downside is that important information may remain hidden when they rely only on the values of handful of core measures. (Have you heard of the tragic story of the man who drowned in a lake with an average depth of six inches?)
Measurement Hierarchies
The third step of building a court performance measurement system (CPMS) is the construction of hierarchies (some call these “families” or “cascades”) that include a court’s core performance measures and related subordinate measures. (See October 15, 2005, posting, Six-Step Process for Building an Effective Court Performance System). The step involves three overlapping tasks: (1) breaking out (disaggregating) a court’s core measures; (2) identifying and defining subordinate measures that are related to core measures but are not necessarily mathematical breakouts of core measures; and (3) creating a hierarchy of performance measures for each of the core measure.
Creating measurement hierarchies for each core measures ensures that important information at the level of court divisions, units, and programs is not masked by exclusive reliance on aggregated core measures. Measurement hierarchies identify opportunities for teamwork and collaboration. They put the court’s top management in much greater contact with every level of staff by defining the connection between high-level strategic goals and performance measures with lower-level departmental or divisional objectives and measures. Driving the CPMS to every level of the court signals to everyone what the drivers of success are and provides them with the concrete knowledge of how they contribute to that success.
Measurement hierarchies in which lower-level subordinate measures “cascade” down from core measures help align the overall goals of the court with the goals and objectives of its divisions, units, and programs. They help make it clear to all court employees precisely how their actions help fulfill the court’s mission and strategic goals. For example, a clerk in a court’s jury commissioner’s office, will recognize that improvement in the percent of undeliverable mail – a lower-level subordinate performance measure -- drives juror qualification yield, which in turn drives the overall juror yield, which drives juror representativeness, a high-level core measure of fairness and equality.
Or, for example, a core measure of cost per case (Measure 10 of the CourTools) aggregated across all case types may show no change over time. This is possible if cost per case of general civil cases rises sharply while that of all other cases drops slightly. Those responsible for improving case processing will want to see the breakouts (disaggregations) of cost per case for all the major case types rather than just the aggregate. The sharp rise in the cost of general civil cases may pinpoint problems or issues requiring immediate attention.
The advent of electronic databases and performance management and business intelligence software has all but eliminated the need for court managers to decide whether or not the problem of obscuring important information outweighs the value of using a vital few core performance measures. They need not choose between broad overview and rich detail. By organizing other important performance measures related to the core measure in a hierarchical format, court managers quickly will be able to drill down (descend) to lower levels of the hierarchy and access increasingly more detailed performance information. At the same time, court staff at lower levels will be able to ascend the hierarchy and align their day-to-day results with the court’s high-level core performance measures and broad strategic goals.
In summary, hierarchies of measures keep the number of core performance measures to a vital few at the top of the hierarchy while at the same time allowing the measurement of different perspectives, levels or aspects of the core measures from top to bottom of the court. Developing measurement hierarchies helps ensure that the right levels of measures are collected, distributed, and used at the right level of the court. While a court manager and other court leaders may focus on the high-level core measure of on time case processing aggregated across case types and case processing events, department or unit managers may focus their attentions on lower levels of hierarchy of this core measure such as on-time case processing of various case types, the elapsed time from filing to judge assignment, assignment to first hearing, disposition to post-disposition event, and so forth. Effective measurement hierarchies ensure that everyone in a court can gain a solid understanding of the subordinate measures and their objectives that make up the high-level core measures.
For the latest posts and archives of Made2Measure click here.
© Copyright CourtMetrics 2005 - 2008. All rights reserved
A court may have only one core measure of community safety, security and well being like recidivism, for example, but it may use numerous subordinate measures of recidivism at a strategic level (e.g., average recidivism by case categoreis and types), tactical level (e.g., recidivism for certain units like drug courts and probation departments) and operational levels (e.g., recidivism of probationers who particpated in a particular rehabilitation program). In addition, a court or its justice partners may usefully monitor, analyze and manage a host of meaures related to community safety, security and well being like the change in police contacts with families with cases in the jurisdiction of a family court.
The point is that the users of these recidivism measures have selective attention and different needs for information at the strategic, tactical and operational levels and, therefore, can and will only pay attention to a limited number of them at any one time.
Core Measures
Like most complex organizations, courts may have hundreds of indicators and measures of their work and performance. However, few court managers can (or want to) keep track of more than a handful of measures and indicators. There’s nothing wrong with a court having many measures and indicators of their inputs, outputs and outcomes in their databases -- it’s just that any individual can only keep track of a vital few.
On a car dashboard, we have a few gauges that we look at all the time to help us as drive we our car, a few that need to be monitored regularly but not all the time, several that need to be looked at less frequently, maybe by a mechanic, and some that need to be attended to only when we are given a warning signal or when the car breaks down.
A “core” performance measure is a primary indicator – akin to a gauge on your car’s dashboard -- of an important area of the court performance. A core performance measure has most of the following attributes:
· Linkage to Mission – It is aligned with one or more of a court's key performance areas or success factors (e.g., access to justice). It is this linkage with key performance areas that limits the number of core measures to a vital few.
· Aggregation – It is a combination, an index, or a conjunction of a number of measures, variables or aspects of court performance that may be identified with subordinate measures.
· Outcome Focus – It emphasizes the condition or status of the recipients of court services or the participants in court programs (outcomes) over that of internal aspects of court processes, programs and activities (inputs and outputs) -- that is, they indicate results rather than resources and level of effort.
· Consistency Across Entire Court – It is consistent from the top to the bottom of the organization of the court. Each may sit at the top of a “hierarchy” of related measures and indicators (see below).
· Driver of Success -- It is both an incentive and a tool for improvement. The key to collecting data for court performance measurement is identifying those performance measures that will actually help to achieve the desired results (i.e., measures that are drivers of success).
· Emblem or Symbol -- the court and its stakeholders easily understand its meaning and significance.
The great benefit of using a vital few core performance measures – the National Center for State Courts’ CourTools include 10 core measures -- that are aligned with a court’s mission and highest-level strategic goals is the ease and efficiency with which they can be used. At best, they are consistent court-wide, highly aggregated, outcome-focused, and emblematic of a key performance area or success factor. They serve as triggers for more in-depth examination of problems or successes in key performance areas and what should be done about them.
It is, of course, of great benefit to busy court leaders and managers if they need only pay attention to a few easily accessible core measures that indicate the performances of many court programs and services, rather than the many numbers that are represented in measurement hierarchies. They quickly see the big picture. When the numbers change, they know they need to act. The downside is that important information may remain hidden when they rely only on the values of handful of core measures. (Have you heard of the tragic story of the man who drowned in a lake with an average depth of six inches?)
Measurement Hierarchies
The third step of building a court performance measurement system (CPMS) is the construction of hierarchies (some call these “families” or “cascades”) that include a court’s core performance measures and related subordinate measures. (See October 15, 2005, posting, Six-Step Process for Building an Effective Court Performance System). The step involves three overlapping tasks: (1) breaking out (disaggregating) a court’s core measures; (2) identifying and defining subordinate measures that are related to core measures but are not necessarily mathematical breakouts of core measures; and (3) creating a hierarchy of performance measures for each of the core measure.
Creating measurement hierarchies for each core measures ensures that important information at the level of court divisions, units, and programs is not masked by exclusive reliance on aggregated core measures. Measurement hierarchies identify opportunities for teamwork and collaboration. They put the court’s top management in much greater contact with every level of staff by defining the connection between high-level strategic goals and performance measures with lower-level departmental or divisional objectives and measures. Driving the CPMS to every level of the court signals to everyone what the drivers of success are and provides them with the concrete knowledge of how they contribute to that success.
Measurement hierarchies in which lower-level subordinate measures “cascade” down from core measures help align the overall goals of the court with the goals and objectives of its divisions, units, and programs. They help make it clear to all court employees precisely how their actions help fulfill the court’s mission and strategic goals. For example, a clerk in a court’s jury commissioner’s office, will recognize that improvement in the percent of undeliverable mail – a lower-level subordinate performance measure -- drives juror qualification yield, which in turn drives the overall juror yield, which drives juror representativeness, a high-level core measure of fairness and equality.
Or, for example, a core measure of cost per case (Measure 10 of the CourTools) aggregated across all case types may show no change over time. This is possible if cost per case of general civil cases rises sharply while that of all other cases drops slightly. Those responsible for improving case processing will want to see the breakouts (disaggregations) of cost per case for all the major case types rather than just the aggregate. The sharp rise in the cost of general civil cases may pinpoint problems or issues requiring immediate attention.
The advent of electronic databases and performance management and business intelligence software has all but eliminated the need for court managers to decide whether or not the problem of obscuring important information outweighs the value of using a vital few core performance measures. They need not choose between broad overview and rich detail. By organizing other important performance measures related to the core measure in a hierarchical format, court managers quickly will be able to drill down (descend) to lower levels of the hierarchy and access increasingly more detailed performance information. At the same time, court staff at lower levels will be able to ascend the hierarchy and align their day-to-day results with the court’s high-level core performance measures and broad strategic goals.
In summary, hierarchies of measures keep the number of core performance measures to a vital few at the top of the hierarchy while at the same time allowing the measurement of different perspectives, levels or aspects of the core measures from top to bottom of the court. Developing measurement hierarchies helps ensure that the right levels of measures are collected, distributed, and used at the right level of the court. While a court manager and other court leaders may focus on the high-level core measure of on time case processing aggregated across case types and case processing events, department or unit managers may focus their attentions on lower levels of hierarchy of this core measure such as on-time case processing of various case types, the elapsed time from filing to judge assignment, assignment to first hearing, disposition to post-disposition event, and so forth. Effective measurement hierarchies ensure that everyone in a court can gain a solid understanding of the subordinate measures and their objectives that make up the high-level core measures.
For the latest posts and archives of Made2Measure click here.
© Copyright CourtMetrics 2005 - 2008. All rights reserved
Comments
Post a Comment