Make It Official: The Case for In-Country Performance Measurement and Management
This is the third in a series of posts exploring the three international models of justice system performance measurement and management: (1) the EU Justice Scoreboard, (2) the Global Measures of Court Performance, and (3) the CourTools.
Law and justice scholars lament the spotty evidence linking rule of law and justice programs with development outcomes like economic growth, human rights, sound governance, and poverty reduction. A key cause of the evidence gap is the lack of emphasis on building in-country or domestic performance measurement and management in favor of third-party evaluations. All three international models, more or less, promote increased attention and investment in performance measurement and management – the regular and continuous monitoring, analysis, and use of performance information -- by justice officials and their institutions and justice systems themselves, not by third parties. Capacity for performance measurement is the ability of countries to meet user needs for good quality statistics on performance – usually those statistics that are consider to be “official” (i.e., those statistics produced by governments as a public good). Granted the three models are promulgated by organizations with considerable heft – the European Commission, the International Consortium for Court Excellence, and U.S.-based National Center for State Courts -- whose interests can be seen reflected in choice of metrics and the values, goals, and key success factors upon which they are built, all three emphasize in-country performance measurement.
Performance measurement and management differs from program evaluation in terms of goals and purposes, definitional style, sponsorship, organization, audience, functions, timing, and data interpretation rules. The differences are critical for justice reform. For example, not the least of the differences is that performance measurement and management is designed to achieve the goals of the justice institutions, systems and the countries, not necessarily those of donors or other third parties, though harmonizing performance measurement across levels is a worthy aspiration (i.e., within an individual institution, across a justice system or country as a whole, and at the level of global governance). In contrast, program evaluation (and its variants including “monitoring and evaluation,” “impact evaluation,” and “evaluation research) may reflect mixed motives for justice reform and definitions of success that are aligned with the value sets and business goals of third parties.
Performance measurement is not yet the norm around the world, though it has a strong foothold in the in the European Union, Australia, the United States, large parts of the developed world, and in some developing countries where investments have been made in building domestic capacities (e.g., Moldova). Most assessments of justice programs, processes, and reform initiatives are done instead by monitoring and evaluations instigated and conducted by third-parties, including donors, aid providers, and their agents (hordes of researchers, analysts, and consultants). The abiding concern of these third parties is return on their investments, or something akin to it, and this concern does not necessarily align with the expressed purposes and fundamental responsibilities of the justice institutions, systems, or countries that are the “subjects” of their assessments. (It is tempting to include the crowded field of international indices of law and justice in this type of third-party assessment, but that is a topic to be dealt with in future posts.)
In a 2011 paper titled Problems of Power in the Design of Indicators of Safety and Justice in the Global South, Christopher E. Stone, President of the Open Society Foundations and formerly at Harvard Kennedy School’s Program in Criminal Justice Policy and Management, urges those of us working in the domain of justice and safety, to emphasize what he calls “country-led indicator development.” He calls for the building of indicators “from the bottom up, supporting local ambitions and building on the legitimate authority close to the operations they seek to influence, rather than starting with ambitions and power at the global level.” He advocates for the design and development of “active indicators” that are distinguished from ones designed without the participation local authorities and are designed for use by officials of local institutions as management tools. He argues that such active indicators and a bottoms-up approach “is not only possible and practical, but has the potential to engage citizens and domestic leaders enthusiastically in a creative and democratic construction of justice.”
When it comes to answering the question “How are we doing?” justice institutions and systems do not like to be the mere “subjects” of the program evaluations and research of third parties, be they international donors, associations, or domestic stakeholders. It only seems reasonable and logical that they should like to see themselves as the proper and legitimate authority for issuing the “official” version of the truth. The official authorized view of a justice system’s performance must not only be owned by the justice system but be seen to be owned by the justice system. When this is not the case, as it appears in many places around the globe, the legitimacy of and the public’s trust and confidence in the justice system suffer.
©Copyright CourtMetrics 2013. All rights reserved.