Q & A: Automated Performance Data Display Systems

Q: Step 6 – Building Performance Measurement Displays – of the Six-Step Design Process for an effective court performance measurement system (CPMS) requires the design, development, and implementation of performance measurement displays that provide relevant and meaningful information that can be quickly accessed and easily understood by the intended users (see the Made2Measure October 15, 2005 post). The step suggests that courts begin by reviewing the functionality of commercial computer software referred to as performance management or “business intelligence” (BI) solutions that are offered by an increasing number of companies. Courts can then decide to buy or to build their own computer-based performance display systems.

But what about courts, especially small courts, that do not have the money to buy, or the technology resources to build, sophisticated computer-based performance display systems? Are such systems a necessary requirement for success? management?

A: Not necessarily. The recommendation of Step 6 that courts begin their consideration of performance data displays with a study of commercially available computer software applications (or the early efforts of courts like the Maricopa County Trial Courts that are building their own display) simply recognizes that computers can be invaluable for collecting, assembling, and delivering critical performance data. There is no single best way to display and to communicate performance data. Performance software is simply an efficient way of doing it that avoids unreadable written reports and spreadsheets.

A common reason that courts fail to detect performance problems is that the methods used to report and communicate performance data are poorly designed – or not designed at all. Potential users of the performance information get bogged down in too much data or too many reports. A typical format for performance reports (usually limited to case data) is spreadsheets with many columns and rows of mind-numbing figures identified by words, phrases and acronyms that are not easily understood. A midsize court I recently consulted with prepares a “transmittal sheet” for its monthly statistical report to the state administrative office with a note that the presiding judge “makes no representation that these figures … are completely accurate.” It’s little wonder that important performance data get missed.

An effective presentation of a set of performance measures is one that users can readily access, that is easily read and understood, that is organized for easy navigation among core and subordinate measures, and one that provides a “line of sight” which conveys to everyone what the drivers of success are and one that provides them with the concrete knowledge of how they contribute to that success. Few courts today operate without spreadsheet programs like Excel to organize, analyze and present data for budget and case processing reports. Computer software that sets up a simple, easy-to-read and easy-to-update system to track performance is merely an extension of such spreadsheet programs. Step 6 of building an effective CPMS should not be read as a necessary requirement to buy or to develop sophisticated performance management computer software, but instead as a strong recommendation to look at increasingly available technology to reach farther, faster, wider and deeper than paper reports can.

Thomas L. Friedman writes in his insightful bestselling book, The World Is Flat: A Brief History of the Twenty-First Century (Farrar, Strauss and Giroux, 2005):

Rule #2: And the small shall act big … One way small companies flourish inthe flat world is by learning to act really big. And the key to beingsmall and acting big is being able to quickly take advantage of all the newtools for collaboration to reach farther, faster, wider and cheaper.

Substitute “small courts” for “small companies” and Friedman’s rule still holds.
---------------------
For the latest posts and archives of Made2Measure click here
---------------------
Copyright CourtMetrics 2005. All rights reserved.

Q & A: Getting Started with Performance Measurement -- The Right People on the Bus

Q: My court wants to start using the CourTools and the Six-Step Design Process (see October 15, 2005, Made2Measure Posting) for building a court performance measurement system (CPMS). How do we get started? Who does it? How do we organize ourselves?

A: To start, get the right people on the bus. Sustaining the effort of planning, developing and implementing a court performance measurement system (CPMS) requires a guiding coalition of individuals who command both the resources and the respect of the court, and who can maintain the energy and momentum to keep the initiative going through its four phases (see below). The court should start the initiative by assembling a 9 to 15-member steering committee to direct and oversee the work. The steering committee, as part of its early planning, in turn, identifies a 6 to 12-member design team to take the steps of building a court performance measurement system (CPMS) in the second phase of the initiative. These two groups form the guiding coalition of the initiative.

Once assembled and oriented, the steering committee decides where to start, develops a timeline, and distributes the game plan to key participants and stakeholders. The steering committee should include both senior-level "cheerleaders" who demonstrate in words and actions that they are committed to the effort and technicians (e.g., the head of a court's research unit) who are able to invest a substantial amount of time. (In its manual, Measuring Program Outcomes (1996), the United Way of America estimates that the manager of what it calls the "outcome measurement work group" will be required to devote an average of five to 10 hours per week for 12 months to the effort.)

The size, makeup, and organization of the guiding coalition – including the steering committee and the design team -- will vary from court to court. For example, the Family Court of the State of Delaware, a court with 15 judges, 15 commissioners, a staff of approximately 300, and a total of 55,000 statewide case filings in 2001, began building its performance measurement system under the banner of the Quality Counts Project in 2001. Its guiding coalition included a steering committee -- a decision making and policy making body -- and five subcommittees under the control of the steering committee. Two senior-level staff co-managed the project. The steering committee, dubbed the Quality Counts Leadership Committee, included the chief judge and several other judges, the court administrator, two state legislators, a representative of the state's auditor's office, and the two project co-managers. The five subcommittees, each responsible for identifying and developing performance measures in one of the performance areas of the Court Performance Standards were headed by a senior managers and included between 10 to 20 staff members.

Outside consultants and facilitators are helpful but not essential to the success of the initiative. In the roles of technical advisor, project manager, change agent and cheerleader they can help keep the initiative from going stale Their value lies in their abilities to explain performance measurement at the beginnings and many points along the way; to help the guiding coalition and its members stay focused on what’s important; to tailor the phases and steps of the initiative to the needs of the court; to focus on signs of loss of momentum and to challenge formal and informal institutional rules that favor undesirable inertia; to convey enthusiasm; to help get people get unstuck; to press the steering committee and design team toward taking action and assigning responsibilities; and to congratulate people whenever possible.

4 Phases of Establishing a Court Performance Measurement System
(CPMS)

Phase 1. Initiating and Planning. Organization and project management, proof of concept, education, readiness and feasibility assessments, identification of design team, planning and scheduling.
Phase 2. Design and Development. Six-step process of building a court performance measurement system (CPMS).
Phase 3. Implementation and Training. Integration of performance measurement and CPMS with key management practices and processes including strategic and operational planning, budgeting, resource management, communication, information technology, and quality improvement.
Phase 4. Institutionalization. Automation, organizational infrastructure, and maintenance.

Copyright CourtMetrics 2005. All rights reserved.

Implementing Performance Measurement

In the 1989 film Field of Dreams, an Iowa corn farmer (Kevin Costner) hears voices that tell him, “If you build it, they will come.” He interprets this message as a command to build a baseball field on his farm. He does and they -- Shoeless Joe Jackson and the other seven Chicago White Sox players banned from the game for throwing the 1919 World Series – come. This works in the movies but it does not work for court performance measurement. It is one thing to build a court performance measurement system (CPMS) and quite another to get the CPMS to be used effectively.

A Pile of Stethoscopes

Kevin Baum, a performance management consultant who works in government outside of the courts, warns us in a recent edition of Perform (Special Edition, Government, no date) that we make a fatal mistake when we declare victory too soon, that is, immediately after we have built a CPMS (see the October 15, 2005, Posting, “Six-Step Process for Building an Effective Court Performance Measurement System”). Baum makes his point with this anecdote:

You’re in a board room waiting for the executive staff meeting to begin when in walks the Director. Under his arm you see 20 stethoscopes, and you think, “What’s up with that?” The stethoscopes aren’t all the same though – they are different sizes, shapes, colors and brands – but yes they are still stethoscopes. The Director, with a broad grin and a tad of flair, tosses all the scopes on the boardroom table and declares in a proud and booming voice, “Look how well we are managed. We are truly a performance-informed organization and I’d like to thank all of you for your efforts.

Baum’s point is that a CPMS, like a stethoscope, is only a tool. Nothing more, nothing less. Courts should not declare victory once the CPMS has been built. Like a pile of stethoscopes, a CPMS is essentially meaningless until we get it into the hands of people who can put it to use, understand what it is telling us, and apply what we are learning.

Performance Measures Drive Success

The ability to measure performance is a critical enabler for getting results and achieving goals. Knowing what and how to measure makes a complicated world less so. Because they are unambiguous and actionable, performance measures drive success. Effectively used, they serve both as incentives and as practical tools for justice system improvement. An effective court performance measurement system (CPMS) enables court leaders and managers to:

-- Translate vision, mission and broad goals into clear performance targets
-- Communicate progress and success succinctly in the language of performance measures and indicators
-- Respond to legislative and executive branch representatives’ and the public’s demand for accountability
-- Formulate and justify budget requests
-- Respond quickly to performance downturns (corrections) and upturns (celebrations) in performance
--Provide incentives and motivate court staff to make improvements in programs and services
-- Make resource allocation decisions
-- Set future performance expectations based on past and current performance levels
-- Insulate the court from inappropriate performance audits and appraisals imposed by external agencies or groups

Around the globe, the use performance measurement has spread dramatically in recent years at all levels of government, as well as in private and nonprofit organizations. In the courts community, performance measurement increasingly is seen not only as the best way to improve the quality of programs and services but also to drive major policy reform and organizational transformation.

Performance measurement can fundamentally change the way courts do business. For this to happen, however, a CPMS has to be integrated with a court’s key business processes and day-to-day management. Until the measures are actually used, they will never begin to work for us as tools to improve performance. Worse, warns Kevin Baum, an idle CPMS will alienate the court’s workforce by burdening it with yet another management initiative that suffers from no apparent follow-through.

Implementing a CPMS

Even before a CPMS has been fully built and developed, courts should begin to consider the following two general strategies:

(1) Train court managers and staff on the performance measures of the CPMS. The value of a performance measure lies not in the measure itself but rather in the questions it forces us to ask and how we learn and grow as a result. What is the current or initial performance level? What are the changes over time? What are the acceptable upper and lower boundaries of the particular measure? What are the problems identified by the measure? Given what we know about the measurement, what performance expectations should we have in the future? Managers and court staff need to be thoroughly familiar with the functions of the performance measures that these questions highlight – baselining and benchmarking, control, trend spotting, problem diagnosis, and operational and strategic planning – for the court as a whole and, importantly, for their area of responsibility.

(2) Integrate performance measurement with the court’s key operations and management processes. Yes, performance measurement can fundamentally change the way a court does business, but it will not happen by itself. They will not come into the field of dreams simply because the CPMS is rolled out. If a court’s leadership and management are to become truly performance-based, performance measurement has to become hard-wired into the very DNA of the court’s organizational culture.

Some of this is simple and straightforward. For example, automated performance measurement displays can identify specific court staff as “owners” of the performance measures who can be queried (“Why was there a downturn in trial certainty this month even though we tightened our continuation policies?”) with an email function linked to the measure on the display. Court leaders can decide to make the results of core performance measures a standing item on executive meeting agendas. Only those measures that require action – i.e., those that fall outside of the lower and upper control boundaries -- are discussed. Falling below the lower controls will stimulate improvement actions and exceeding upper controls (goals) will be cause for recognition and, perhaps, celebration.

Aligning performance measurement with other key management processes like strategic planning, budgeting, quality improvement, and human resource management may be more demanding but not necessarily difficult or complex. For example, a court’s ability to develop measurable performance objectives is critical to success of its strategic planning process. A strategic goal like maintain a high-performance workplace is made useful only if it is translated into a measurable objective such as workforce strength, commitment and engagement exceeds 80% as measured by a quarterly survey of court employees. A CPMS that includes a measure of workforce strength not only facilitates development of a strategic plan by giving definition to goals and objectives but also establishes the mechanism by which the strategic plan is put into action.

Future postings will explore the specific steps for training and aligning a CPMS with key court operations and management practices.

Copyright CourtMetrics 2005. All rights reserved.

Q & A: Raising the Expectations of Performance Measurement

Q: Here’s a concern I hear often about performance measurement: Serious initiatives to build and implement court performance measurement systems (CPMS) will raise false expectations among court staff and other stakeholders that the court will actually make improvements in response to performance results. The concern is based in the fear that court leaders will be caught flat-footed responding to performance results (e.g., low public satisfaction with court services, sub-par on-time performance in case processing, and rates of compliance with court orders that fall short of benchmarks).

A: While a poor performance showing is rarely a cause for rejoicing, the concern that performance measurement itself will raise false expectations that something will be done is misplaced. The concern misses the power of performance measurement to engage individuals at all levels, not just top managers, in improvement strategies. It also makes the questionable assumption that all change for the better must be delivered by top management. (The concern may be due in part to a misunderstanding of the differences between performance measurement and research. See October 7, 2000, posting, “The Differences Between Performance Measurement and Research”).

Innovations Are Uploaded, Not Downloaded

If responding to performance results is considered the exclusive domain of court leaders and top management there well may be cause for concern. But finding solutions and formulating strategies is less often a top-down process. (Think about the in the way the Internet has changed users from spectators to active contributors in a participatory democracy that was thought impossible just a few years ago – this blog is an example.) Court leaders and top managers typically do not have the detailed knowledge and direct control of processes to tell people what to do differently. At best they have remote control. Their job is not to tell people what to do, but rather what they want to achieve, what their expectations are. And that’s where performance measures come into play.

From a leadership and management perspective, performance measures are valuable because they are unambiguous and actionable. They facilitate focus, and clarity, and help action. Rather than relying on vague exhortations about mission (“We need to build public confidence.” “Our court will become the best limited jurisdiction court in the country.”), court leaders can be absolutely clear about their high expectations, about what they would like to achieve – an 8% increase in public satisfaction with court services from 67% to 75% the next quarter, an increase of 4% in overall on-time case processing across all case types in six months, and an increase in collection of monetary penalties of 11% over the next year.

Bringing People Together for Joint Action

In my view, the fear of raising false expectations with performance measurement is less about high expectations for action per se – most managers like high expectations for their motivational qualities – but rather more about who will do the heavy lifting. If performance measurement points the finger at top management to do it all by themselves, there’s not much incentive for court managers to engage in it. But that’s not what performance measurement does. Because good performance measures can help court leaders and managers be clear and focused about what needs to be achieved, performance results can serve as invitations for wide participation in needed improvements.

By being focused and clear about the desired outcome in terms of performance measures (e.g., a clearance ratio of 105 percent in the next quarter) court staff are free to devise creative means to achieve the desired outcome. Performance measures encourage delegation and discourage “micro-management.” General George Patton said that when you don’t tell people what to do, but rather where you want to be, they will surprise you with their ingenuity and diligence.

Rather than shying away from high expectations, court leaders and managers instead should use performance measures to encourage and shape them. They should use performance measurement to bring people together for joint action.

Copyright CourtMetrics 2005. All rights reserved.