Experience Counts for the Advancement of Performance Measurement and Management

Made2Measure returns today to regular postings after a long hiatus (September 9, 2013 was the last post) during which it was suspended to avoid potential conflicts of interests while its principal, Ingo Keilitz, was seconded to the World Bank and the National Center for State Courts.  

Performance measurement and management (PMM) is the (self) discipline of monitoring, analyzing, and using organizational performance data on a regular and continuous basis (in real or near-real time) for the purpose of improvements in efficiency and effectiveness, transparency and accountability, and increased public trust and confidence in government institutions. PMM is both a way of understanding the justice sector, as well as a discipline and a promising approach to solving serious global problems such as the high rate and length of incarceration, especially pre-trial detention.

Relatively Small Space in the Toolbox of International Development

Compared to two other disciplines and technologies of knowledge production and governance – program impact evaluations conducted by international donors such as the World Bank, and global indicators such as World Justice Project Rule of Law Index™ -- PMM occupies a relatively small space in the toolbox of international development. This despite increasing evidence that countries and their justice institutions who measure and manage their own performance are likely to enjoy more success and gain more legitimacy, trust and confidence in the eyes of those they serve.  Why is this happening?

Experience Development Counts

Over the last fifteen years or so, my colleagues and I at the International Consortium for Court Excellence, the National Center for State Courts, and of late at the College of William and Mary’s Institute for the Theory and Practice of International Relations (ITPIR), have spent much time on the design of PMM developing the “right” metrics, the “right” delivery of performance data (i.e., getting it into hands of the right people, at the right time, and in the right way), and the right actual use (i.e., injecting PMM into the very DNA of an institution’s business processes and operations). But good design alone has not, in my view, created more space in the tool box of international development for PMM.  Experience counts as much as design.

In today’s world of international development, well-designed approaches and products are not enough for potential users of those approaches (including donors) who value experience.  Before adoption or adaptation, they want to know what developing country or institution has built its capacity and/or actually used a particular performance measure such as duration of pre-trial custody, a measure that is part of the International Consortium’s International Framework for Court Excellence? For the most part, the answers to such questions are anecdotal and speculative.

Several things need to happen before PMM can emerge with a bigger role in international development.  First and foremost, the PMM that is taking place in countries and justice systems throughout the world needs to be well documented and known in terms of actual experiences, which it has not.  This impediment to a greater role of locally-owned or locally-directed PMM by host countries and institutions in international development is in large part a lack of effective incentives for PMM knowledge production and dissemination. Quite simply, much more is known about program impact evaluation and global indicators because their producers are in the business of publishing and disseminating their results, thereby burnishing their reputations. Many international development organization, multilateral development banks, bilateral aid agencies, private foundations, think-tanks, international activist groups, and consultancies publish books and articles, newsletters and blogs, touting the results of their program impact evaluations and global indicators, often through their own publishing arms.

The countries and their justice institutions actively using PMM, not so much. They lack the orientation, incentive, and the capacity. Promulgation of PMM and dissemination of results are inward directed to drive improvements in the organization's performance. And as pointed out in a thoughtful paper by Wade Channell ten years ago in the Carnegie [Endowment for International Peace] Papers Rule of Law Series, even when donors and projects do get involved with host institutions they have high incentives for guarding their information and lessons learned are unlikely to be shared widely.

The Justice Measurement Visibility Project

Last month my ITPIR colleagues Kate Conners, Maya Ravindran, Jonah Scharf, and I launched the Justice Measurement Visibility (JMV) Project, a project that aims to identify successful PMM in developing countries throughout the world focused on the eleven specific measure of the Global Measures of Court Performance, which is part of the International Framework for Court Excellence developed by the International Consortium.  For each of the eleven measures, we hope to be able to give a definitive answer to the question of what countries and their justice institutions have adopted or adapted it and what has been their experience.

Stay tuned here for more on the JMV Project.

 © Copyright CourtMetrics 2015. All rights reserved.

Popular posts from this blog

A Logic Model of Performance Inputs, Outputs and Outcomes

Q & A: Outcome vs. Measure vs. Target vs. Standard

Top 10 Reasons for Performance Measurement