How To Be Heard by Policymakers


The design of international development is ill-suited for our fast-paced world. It is not unusual for aid programs to take five or more years from blueprint to start-up and another five years for results to be reported, and even more time for the results to be “translated” into policy. 

How Scientists Should Act

Writing in the February 10, 2017 issue of Science, Erik Stockstad summarizes the message of Paul Cairney, a political scientist at the University of Stirling in the UK, author of the book, The Politics of Evidence-Based Policy Making. Cairney’s message is for those scientists who want their findings to find their way into policy:

Data does not speak for itself.  Scientists should be “sifters, synthesizers, and analyzers” to make the evidence “speak.” Cairney repeats the common refrain of policy-makers: “I don’t have the time to consider all the information. How do I decide?”

Policymaking is disorderly. Scientists need to dispense with the notion that policymaking is an orderly process. It is anything but. This should not be a justification for scientist to avoid getting involved.

Publishing the results is not enough. I have written in these pages and elsewhere about the three development phases and requirements of effective performance measurement and management (PMM): the “right” performance measures to yield the relevant data; the “right” distribution system for the getting the results to the right people in the right way, and at the right time (preferably in real-time or near real-time); and processes that encourage the “right” use of the performance data. Similarly, Cairney says, scientists who want their evidence to influence policy must be persistent, find the right networks, and the “right moment” to get their results to policymakers.

Pick Your Battles. Scientists should realize that results that matter are likely to be controversial and disputed. PMM, like most scientific endeavors, is not just a diagnostic exercise but, more or less, an exercise of power and control.  Cairney counsels scientists to avoid areas where emotions are high, think of other ways to engage using techniques of presenting technical information in accessible and persuasive language that recognizes “an audience’s pre-existing concerns, values, and biases.”

Be patient. I have been frustrated by the slow pace in which justice systems around the world have embraced PMM, despite what I view as sound principles and strong evidence of its merits. Cairney says that we should have patience, and lots of it. It takes two or three decades for profound changes to be made, even in areas like smoking and cancer where strong evidence of cause and effect have been developed.

New Techniques and Tools

Cairney’s advice is certainly wise in terms of how scientists should behave and position themselves to influence policymaking. But what about technocratic changes and new scientific tools that scientists might use in the design and dissemination of their findings to influence policymaking?  

In a talk last week on February 16 at William & Mary, Caroly Shumway, Director of Center for Development Research at the United States Agency for International Development (USAID), Chief Scientist for the Global Development Lab at USAID, and the Senior Science Advisor to the Administrator of USAID, discussed how USAID is using science and technology to transform international development efforts. She mentioned one such tool: rapid feedback. According to the USAID website, Rapid Feedback MERL (the acronym referring to monitoring, evaluation, research and learning) is a “collaborative approach to learning and adapting. Improved data capture and compressed feedback loops provide decision-makers with timely, actionable evidence. Design and implementation decisions can be optimized to maximize chances of impact and improve prospects for long-term success.”

Why not require scientists to hew to standards of real-time or near real-time for “compressed feedback loops,”  standards that are de rigueur in business and much of the private sector (think of the DOW, sports reporting and emerging in PMM in the public sector)? Why should scientists who profess an interest in shaping policy adhere to rigid standards for the timing of reporting that are defined by scientific designs divorced from the demands of policymaking?  Should those designs not be flexible enough to meet those standards? Why should even researchers relying on randomized controlled trials be precluded from providing rapid feedback in real-time? 

Consider, for example, a researcher who is required to report in real-time his or her evidence that his or her research is producing data that might not be reproducible for various reasons. Would not such rapid feedback benefit both policymaking and good science?

© Copyright CourtMetrics 2017. All rights reserved.

Popular posts from this blog

Top 10 Reasons for Performance Measurement

There is a growing demand of “show me the data” from citizens and taxpayers, legislators, executive agencies, oversight boards and communities – all of whom exchange money and other support (e.g., trust and confidence) for services -- that courts can ill afford to ignore. Courts need to measure their performance to meet the demand for public accountability. But the benefits of court performance extend beyond accountability including improved prediction, better understanding and control, influence, a sharpened focus on what is important, and discovery of better practices. Objective performance data is central to good management practices like strategic planning, performance-based budgeting, Balanced Scorecards, Total Quality Management (TQM), and Six Sigma. Performance measures allow court leaders and managers, policymakers, legislators and the public to evaluate court programs’ inputs (the resources allocated), outputs (direct results of program or service activities) and outcomes (bro...
Read more

Q & A: Outcome vs. Measure vs. Target vs. Standard

Q: Those of who write and speak about court performance measurement rely on their share of jargon -- and it's often confusing. What are the differences among the terms “outcome,” “measure,” “target” and “standard”? A: Socrates said that the beginning of wisdom is definition. We’ll see. Outcomes are the benefits or changes for the intended beneficiaries of a court’s programs and services. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition or status of the program participants or recipients of services. Examples include litigants' satisfaction with a court's courtesy and responsiveness, success of probation, time to case disposition, clarity of orders, integrity of case files, percent of mediation agreements, enforcement of orders, and perceived fairness of proceedings, as well as percent of expected expenditures and percent of revenues received. Outcomes should not be confused with the activities of those who run the court. (See the October 1...
Read more

Taming “Wild Problems”: Measure Everything That Matters

Every problem is not solvable, but every problem is measurable. This assertion is the key to tackling problems in public services including courts, business, and life in general. Surprising to me is that even astute scholars take exception to this, among them  is Russ Roberts, the John and Jean De Nault Research Fellow at Stanford University’s Hoover Institution, the president of Shalem College in Jerusalem, and host of the award-winning podcast EconTalk: Conversation for the Curious. The belief that certain things, dubbed “wild problems” by Professor Roberts, cannot be measured does a disservice to courts and other public and private organizations. There are many benefits to measuring everything that matters. It allows organizations to   focus on and to operationalize the most important problems and issues facing them,   to track progress over time and identify areas of improvement,   to compare their performance to that of other organizations and learn from the d...
Read more