Made2Measure

My last blog noted three promising international models of justice system performance measurement and management: (1) the EU Justice Scoreboard , (2) the Global Measures of Court Performance , and (3) the CourTools .   All three, more or less, aim for harmonization and consistent use of a common set of justice sector performance measures. There are, of course, differences among them, but it is their commonality that is potentially transformative for justice systems around the globe. What distinguishes these three models from international global governance initiatives like the World Justice Project’s WJP Rule of Law Index™ and the American Bar Association’s Judicial Reform Index, as well as myriad program evaluations of justice and rule of law projects, is that they promote an approach to performance measurement and management that: is essentially a bottom-up instead of a top-down strategy grounded in the local ambitions of justice institutions and justice systems exercisin...

Three models of justice system performance measurement and management that have gained significant currency internationally over the last few years are: ·          the recently launched European Commission’s EU Justice Scoreboard based on the extensive work of the Council of Europe Commission for the Evaluation of the Efficiency of Justice ( CEPEJ ); ·          the Global Measures for Court Excellence developed by the International Consortium for Court Excellence as part of its International Framework for Court Excellence; and, ·            the National Center for State Courts' CourTools . All three of these models aim for consistent use by all countries and states or, in the case of the EU Justice Scoreboard , all EU member states; all three elevate to the status of core measures of performance a limited number of indicators (e.g., case clearance ra...

Q : Adopting government-wide or justice sector-wide performance measurement and management   to make public service or justice service more efficient and effective is politically attractive, even if elected officials are drawn mostly by the symbolic values of the key success factors with which performance measures are aligned (e.g., legitimacy, fairness, and public trust and confidence in institutions).   Successful leaders and managers are drawn to performance measurement because it informs uncertain decisions. Beyond such models as the European Commission’s EU Justice Scoreboard ,   the Global Measures for Court Excellence   developed by the International Consortium for Court Excellence as part of its International Framework for Court Excellence, and the National Center for State Courts' CourTools , and their attendant exhortations to adopt performance measurement and management, are there any compelling success stories of effective use? A : Yes. Almost five ...

What if we could draw an indicator of the “health” of the justice systems throughout the United States – a Justice Index - from readily available public data that is impartial, reliable, objective and comparable, let’s say the amount of time that criminal defendants spend in custody without a trial, or even without any access to legal or justice services? Or the bail amounts imposed on misdemeanor defendants?   This kind of a measure would not be perfect – no measure is – but it would be something, better than nothing. It would address questions of fairness and efficient use of public resources. That is, a criminal defendant languishing in jail awaiting trial suffers, at the least, a delay in justice. He or she is also occupying a jail bed and spending taxpayers’ money. What if we then join this measure with another meaningful indicator of the health of our justice systems in counties and states, let’s say the degree to which jurisdictions prosecute domestic violence ca...

Bill Gates: My Plan To Fix the World’s Biggest Problems** Measure Them!   This is the headline of the lead article in the Review section of the weekend edition of the Wall Street Journal   written by Bill Gates, co-chair of the Bill & Melinda Gates Foundation and co-founder of Microsoft.      The article was accompanied by a large picture of transparent globe wrapped by a tape measure covering most of the first page of the section.   Its message is the theme of Made2Measure. Because it comes from Bill Gates, we tend to listen. “In the past year,” Gates writes, “I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal .. .This may seem basic, but it is amazing how often it is not done and how hard it is to get right.” He writes specifically about foreign aid among other areas like education, health ca...

It is not uncommon for court leaders and justice stakeholders to raise the access to justice “card” to subvert the closing or consolidation of courts.   If you close this court -- the argument goes -- citizens will be denied access to justice.   This is a strong argument because it typically evokes an emotional response irrespective of the information’s truth value, especially if the counter-arguments are couched in bland terms like efficiency and cost-savings. Trouble is that such arguments are heavy on rhetoric and light on hard evidence; and they often produce factionalism and political stalemate (see Court Consolidation in Mahoning County ).      In a 2005 survey of the public and of attorneys in California, respondents who were asked about eleven reasons that might keep someone from “going to court,” cited “travel distance to court from home” less often than eight of the other reasons including fees, cost of hiring an attorney, the time it takes to r...

Close to 2.5 quintillion bytes of new data are created daily. ....Organizations know this data is rich in information and potential insight, but it’s as if they receive a fresh-minted library daily with answers to all their questions, except the books are written in languages they cannot understand.   The above words appeared in a full-page ad by IBM in yesterday’s Wall Street Journal under the headline “Watson Goes To Work.” The ad touted IBM’s Watson, the supercomputer Jeopardy! champion over two human challengers in a pop culture event in February 2011.   Watson’s performance improved as the game progressed as it built on each bit of information acquired from each new clue and answer.   Watson demonstrated the advanced form of computing IBM calls a cognitive system, “a system that is not simply programmed but is trained to learn based on interactions and outcomes.”    The ad brought to mind a “what if” question my National Center for State colleague J...

We have failed to breathe life into the prescriptions for the performance measures that we exhort courts to use on a regular and continuous basis. The connection between well known health indicators like blood pressure, cholesterol level, and blood glucose and our health and wellbeing is self-evident to most informed people.   We know that these measures mean something vital and something very important to us. In my experience, the same cannot be said about the connection between such relatively common measures of court performance as time to disposition , trial date certainty , and collection of monetary penalties and fundamental justice and fairness (see Ten Reasons Not to Measure Court Performance , Made2Measure , November 19, 2008).   Many judges and court managers see these measures as indicators of the relatively narrow concerns of those for whom it’s all about the numbers -- the number crunchers, the spreadsheet guys, and the IT folks who manage the case manageme...

The rationale for measuring citizen/court user satisfaction has rested mostly on the argument that we should improve courts on the basis of what we learn from those served (or not so well served) by the courts, rather those who run the courts.   Two new developments make this argument more compelling: the emergence of the theory of procedural justice (or procedural fairness) and the finding that our desire for being heard, for having a voice in matters, is hard-wired into our brains. Procedural Justice Here is what Kevin Burke, District Court Judge in Minnesota, current President of the American Judges Association, and a long-time proponent of court performance measurement, recently had to say about the importance of procedural fairness ( Judicature, Viewpoint, May/June, Volume 95, Number 6, 251 -254): Better performance is the key to building public support for the judiciary… A court or a judiciary that is as good as its promise is known not just for speed or efficienc...

Health care has provided lessons for court administration in the past (see Monitoring “Never Events in Court Administration , Made2Measure , September 23, 2008; Pursuing Perfection – A Lesson from Health Care , Made2Measure , November 1, 2006 . This time it provides a cautionary tale, specifically for performance measurement and performance management. The Story of PICS As reported in the Economist (“From petrol to prescriptions,” June 16, 2012. p. 65), the Queen Elizabeth hospital in Birmingham in the U.K. pioneered a new approach to patient care learned from the carmaker BMW’s engine assembly factory where over 99.9% of tasks were completed flawlessly. Big health systems in the U.K. and the U.S., in contrast, tolerate many more errors. Emulating BMW’s approach to manufacturing, the Birmingham hospital system installed a quality-control regime called PICS, which stands for “Prescribing, Information and Communication System.” PICS features a performance dashboard system on a...

In previous posts I’ve called for courts and justice systems to be good stewards of performance data.   And, as an advocate of radical transparency , I’ve pushed for self-assessment and accountability by means of regular and continuous performance measurement and management instead of reliance on periodic third-party program evaluation and research. Now comes another good reason for courts and other justice institutions to go that route.   As reported by Carl Bialik, Census Gets Questions on Mandatory Queries , the Wall Street Journal’s “Numbers Guy,” members of Congress are challenging the Census Bureau’s massive data-collection effort known as the American Community Survey.   The survey uses dozens of detailed questions – including the use of carpools, flush toilets, and wood fuel – to help determine the need for and distribution of funds of various government programs. The survey, which reaches 3.5 million households, is mandatory, a fact that gets close to 100% o...

In his 1993 book The Ecological Vision: Reflections on the American Condition, the social scientist and renowned management thinker Peter Drucker argued that there are no underdeveloped countries. The same thing can be said about a country’s justice system including courts and other justice institutions.   There are only “undermanaged” countries and justice systems. Management creates social and economic development.   Wherever there are only capital and other resources (inputs), there can be no development.  Management is the process of getting people together for joint behavior to accomplish desired goals using available resources efficiently and effectively to produce desirable outputs and outcomes. Management is the driver; development is the consequence.   Economic and social development is a matter of human energies, Drucker writes, and the inspiration and mobilization of human energies is the job of management.    The last link in the causal c...

The question How Are We Performing? lies at the heart of self-governance and effective leadership of courts. The capacity and political will to address this self-directed question regularly and continuously using the tools of performance measurement is the hallmark of a successful court organization. More and more court leaders and managers are turning to performance measurement to drive success. However, the trend is on a slow march impeded by two entrenched ways the judicial sector has tended to measure its success: (1) reliance on third-party monitoring and evaluations of court performance; and adherence to a research paradigm for assessing court performance. Self-governance, transparency, and accountability of courts will depend on the degree to which these two impediments are attenuated. Court leaders will need to champion and support the self-assessment of performance by courts, instead of relying on third party evaluations. They will need to seek the replacement of the metho...

Hospitals across the country are being pushed and dragged into outcome-based performance measurement systems with increasing incentives for providing quality care and better outcomes. Proponents argue that such systems discourage unnecessary treatments, keep people healthier, and reduce overall costs. Janet Adams reports in today’s Wall Street Journal ("Wellpoint Shakes Up Hospital Payments," May 16, 2011, B1) that Wellpoint Inc. is raising the stakes for 1,500 hospitals in 14 states serving 34 million people on its Blue Cross Blue Shield plans by cutting off annual payment increases to the hospitals if they fail to deliver quality patient care. Under the new system, Wellpoint will pay increases only to those hospitals that earned them by scoring high enough on outcome measures of treatment quality including how satisfied patients were with the care they received. The change is part of a broader trend in the health care industry toward a compensation approach for health ...

My colleagues and I at the National Center for State Courts and elsewhere have been thinking long and hard about a “justice index” here in the United States, as well as a similar index, the Global Court Performance Index (GCPI), applicable at an international scale. Though not unschooled in ways and means of performance measurement and management in justice systems, we’ve been daunted by the challenges that the construction of such indexes present. What should the conceptual framework of the index be – the rule of law, justice systems institutions, legal frameworks, the experiences of citizens with the justice system, and so forth? Should the indicators that comprise the index be drawn from actual performance data available from justices systems, such as case clearance rates and median time in criminal defendants spend in custody before trial, or should the data be drawn from secondary sources, or should new measures be identified and defined? How much weighting should be given to...