The Differences Between Performance Measurement and Research

It is common for large and medium-size courts to have research units or departments that employ researchers with titles like “analyst,” “statistician,” or “planner,” if not “researcher.” No courts today -- to my knowledge -- have units or departments devoted solely to performance measurement. Only a handful of court-employed researchers consider their job to be performance measurement. This situation is likely to change as courts become more familiar with the functions and uses of performance measurement, and as the differences between performance measurement and research become more apparent.

“Performance measurement” is the process of measuring an organization’s accomplishments (outcomes), work and service levels (output), and its resources (inputs). It is any effort undertaken to meet the need for evidence of an organization’s performance on a regular and continuous basis. Behavioral and social “research” (including evaluation research and program evaluation), on the other hand, is the scientific study to discover factual truth, to test models and to develop theories to increase our knowledge and understanding about human behavior and social phenomena. Performance measurement and research share an adherence to scientific methods and processes. Both use quantitative and qualitative methods including surveys and questionnaires, interviews, direct observation, recording, descriptive methods, tests and assessments, and statistical analysis.

Performance measurement and research, however, are vastly different in their purposes, functions, uses, and the way they are funded and structured. To suggest how, think of the differences between the process you use to monitor your “driving” performance as you check the gauges on your car dashboard and that of your state department of motor vehicles (DMV) commissioning a study of cell phone use and driving patterns throughout your state.

Timing. Performance measurement is conducted on a continuous and regular basis. Research is episodic. It is done when time and funds permit. Consider the absurdity of getting a print-out of your car’s speed from your state’s DVM on a monthly basis instead of your car’s speedometer in real-time.

Functions. The functions of performance measurement are specific and targeted: establishing a baseline for current performance, setting organizational goals and assessing whether performance is within determined boundaries or tolerances (controls), identifying and diagnosing problems, determining trends, and planning. Performance measurement is done for the utilitarian and practical purposes of making improvements in court programs, services and policies. Research, on the other hand, seeks truth and is intended to add to our general knowledge and understanding.

Uses. Court leaders and managers use performance information to make improvements in programs and services. Specifically, they might use performance measurement to:

- Translate vision, mission and broad goals into clear performance targets
- Communicate progress and success succinctly in the language of performance measures and indicators
- Respond to legislative and executive branch representatives’ and the public’s demand for accountability
- Formulate and justify budget requests
- Respond quickly to performance downturns (corrections) and upturns (celebrations) in performance
- Provide incentives and motivate court staff to make improvements in programs and services
- Make resource allocation decisions
- Set future performance expectations based on past and current performance levels
- Insulate the court from inappropriate performance audits and appraisals imposed by external agencies or groups
- Communicate better with the public to build confidence and trust

Performance measurement can give clues to why outcomes are as bad or good as they are but it cannot go the full distance of determining why things are as they are. Performance measurement can help to identify variations in performance and to isolate where and when those variations occur (e.g., an upward trend in the public’s rating of the courts is largely due to attorneys’ satisfaction with case processing timeliness after the court initiated electronic filing); it does not determine causes. This is the domain of research which helps us to understand why something has occurred. An additional use of performance measurement is to trigger in-depth program evaluations or research.

Sponsorship and Audience. Performance measurement and research generally differ in their sponsorship and audience – who is doing it for (or “to”) whom. Performance measurement is done by the court, for the court. Results are made known, first and foremost, to court leaders and managers. Distribution to the public and other stakeholders is done at the discretion of the court. Research is more often than not sponsored or instigated by third parties (e.g., administrative offices of the courts or funding agencies) and courts are mere “subjects” of the research. Results are first made known to the sponsors of the research and made known to the court at the discretion of the sponsors.

Data Interpretation Rules. Performance measurement is focused on the performance of individual courts with an eye toward accountability. Researchers are interested in generalizability of findings to all courts. Both research and performance measurement must adhere to the requirements of the scientific method but those requirements may be applied differently for research and performance measurement. For example, sample sizes may be smaller and levels of confidence lower in performance measurement, and much more can be made of discrete variable, ordinal-level measurement in court user or employee satisfaction ratings. These are largely differences of degree not kind.

Organizational Infrastructure. Where are the resources to support court performance measurement systems (CPMSs)? Where will the CPMS be maintained? Researchers employed by courts today should be thrilled at the movement toward performance measurement in the courts. In 1997, Kathryn E. Newcomer, professor and chair in the Department of Public Administration at George Washington University, noted in Using Performance Measurement to Improve Public and Nonprofit Programs (Josses-Bass), that when the federal Government Performance and Results Act was initially passed it was dubbed the “full employment act for program evaluators.” She went on to suggest that if government agencies had reinvigorated evaluation and research units to help design and maintain performance measurement systems “all parties would have benefited.”

Future postings will explore the recommendation that the major functions of court research divisions, departments or units should be the design, building and maintenance of court performance measurement systems (CPMSs).

Copyright CourtMetrics. All rights reserved.