Ensuring the Right Use of Performance Data: A Cautionary Tale from Health Care

Health care has provided lessons for court administration in the past (see Monitoring “Never Events in Court Administration, Made2Measure, September 23, 2008; Pursuing Perfection – A Lesson from Health Care, Made2Measure, November 1, 2006. This time it provides a cautionary tale, specifically for performance measurement and performance management.

The Story of PICS
As reported in the Economist (“From petrol to prescriptions,” June 16, 2012. p. 65), the Queen Elizabeth hospital in Birmingham in the U.K. pioneered a new approach to patient care learned from the carmaker BMW’s engine assembly factory where over 99.9% of tasks were completed flawlessly. Big health systems in the U.K. and the U.S., in contrast, tolerate many more errors.
Emulating BMW’s approach to manufacturing, the Birmingham hospital system installed a quality-control regime called PICS, which stands for “Prescribing, Information and Communication System.” PICS features a performance dashboard system on a computer screen in every in-patient ward displaying such performance measures as hospital-borne infection rates following surgery and the number of falls by frail patients. Efficiencies that are benchmarked against comparable wards and past performance are continually monitored by staff and management in real time. No doubt, the Birmingham hospitals had the right measures and the right delivery system in PICS to get the right measures to the right people in the right way and at the right time.
The Birmingham experiment was welcomed by Britain’s health-care establishment as Queen Elizabeth’s mortality rates fell and errors that lead to poor patient care decreased. Other hospitals with poor performance contracted out patient care to Queen Elizabeth.
But this is where the good story ends. The Birmingham experiment has stalled. It seems that Britain’s National Health Service (NHS) is not into bold thinking as it is preoccupied with austerity. It emphasizes the universality of care provided rather than its outcomes. Hospitals do not compete with each other. Successful innovations like PICS do not spread because NHS has no mechanism for ensuring that they do.
The lesson of this cautionary tale is that it’s not sufficient to identify the right performance measures and even to build the right delivery mechanisms like PICS, a modern performance dashboard system. A performance measurement and management system is not self-executing. Users of performance data need to act on the problems or opportunities that the data reveals on a regular and continuous basis and have the incentive to do so.
For courts, the right use of performance measurement will not happen unless courts implant performance measurement and performance management into the very DNA of their governance, management practices, and daily operations. It will not happen until the courts make innovations such as assigning new responsibilities, instituting specific policies, creating governance structures, and starting processes, procedures and practices to ensure adoption, implementation and institutionalization of a performance measurement and management system.
Right Measures, Right Delivery and Right Use
Identifying the right performance measures for an individual court or court system, and making sure that they are used effectively, can be translated operationally into three key requirements and corresponding phases of development: Right Measures - identifying and developing the right performance measures; Right Delivery and Distribution. ensuring that the right measures are delivered to the right people, at the right time in an easy to understand way (increasingly, this is done by computer-based performance scorecards or “dashboards” that let users view critical performance information at a glance, and move easily through successive layers of strategic, tactical and operational information on a self-help, on-demand basis; and Right Use. adopting, implementing, and integrating the measures performance, as well as the delivery system and distribution system (e.g., performance scorecard or dashboard), with key management processes and operations, including budgeting and finance, resource and workload allocation, strategic planning, organizational management, and staff development.

You can’t just throw an innovative performance measurement and management system “over the wall” to executives, managers and staff to adopt and to implement. Even well-conceived, well-designed systems are unlikely to get implemented unless they are woven into the very fabric of a court’s management practices and processes.

© Copyright CourtMetrics and the National Center for State Courts 2012. All rights reserved.

Popular posts from this blog

Top 10 Reasons for Performance Measurement

There is a growing demand of “show me the data” from citizens and taxpayers, legislators, executive agencies, oversight boards and communities – all of whom exchange money and other support (e.g., trust and confidence) for services -- that courts can ill afford to ignore. Courts need to measure their performance to meet the demand for public accountability. But the benefits of court performance extend beyond accountability including improved prediction, better understanding and control, influence, a sharpened focus on what is important, and discovery of better practices. Objective performance data is central to good management practices like strategic planning, performance-based budgeting, Balanced Scorecards, Total Quality Management (TQM), and Six Sigma. Performance measures allow court leaders and managers, policymakers, legislators and the public to evaluate court programs’ inputs (the resources allocated), outputs (direct results of program or service activities) and outcomes (bro
Read more

The “What Ifs” Along the Road in the Quest of a Justice Index

What if we could draw an indicator of the “health” of the justice systems throughout the United States – a Justice Index - from readily available public data that is impartial, reliable, objective and comparable, let’s say the amount of time that criminal defendants spend in custody without a trial, or even without any access to legal or justice services? Or the bail amounts imposed on misdemeanor defendants?   This kind of a measure would not be perfect – no measure is – but it would be something, better than nothing. It would address questions of fairness and efficient use of public resources. That is, a criminal defendant languishing in jail awaiting trial suffers, at the least, a delay in justice. He or she is also occupying a jail bed and spending taxpayers’ money. What if we then join this measure with another meaningful indicator of the health of our justice systems in counties and states, let’s say the degree to which jurisdictions prosecute domestic violence cases.
Read more

Q & A: Outcome vs. Measure vs. Target vs. Standard

Q: Those of who write and speak about court performance measurement rely on their share of jargon -- and it's often confusing. What are the differences among the terms “outcome,” “measure,” “target” and “standard”? A: Socrates said that the beginning of wisdom is definition. We’ll see. Outcomes are the benefits or changes for the intended beneficiaries of a court’s programs and services. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition or status of the program participants or recipients of services. Examples include litigants' satisfaction with a court's courtesy and responsiveness, success of probation, time to case disposition, clarity of orders, integrity of case files, percent of mediation agreements, enforcement of orders, and perceived fairness of proceedings, as well as percent of expected expenditures and percent of revenues received. Outcomes should not be confused with the activities of those who run the court. (See the October 1
Read more