Made2Measure

The benefits of an effective court performance measurement and management system are the same as those of strategic planning – accountability, consensus building, focus, coordination, control, learning, communication, hope and inspiration. To identify the right performance measures, a court must address the same fundamental questions about guiding ideals, values, mission, goals and broad strategies as it must address in strategic planning. We count what counts and measure what matters. And what we measure determines what will be considered relevant. Measurement uses numbers but it is ultimately not about numbers. It is about perception, understanding and insight. [1] It is not the measure itself that is important but rather the questions it compels us to confront. How are we doing? How is the court performing? Where are we now (performance level, baseline)? What is the current performance level compared to established upper and lower “controls” (e.g., performance targets, objectives, ...

For court performance measurement initiatives to succeed they must marry proven methods of assessments of the health of a court with a disciplined process for improving it. This may have to be a shotgun marriage. Without it – even in the very early stages of development -- the initiatives will fall prey to the “gotcha” game and be used to undermine the effort and to discredit it proponents. The Gotcha Game In this political season, we are all familiar with the ploy in which one candidate (or the media) seeks to catch another in a misstep or flub -- no matter how untruthful or inconsequential the accusation might be. The whole point is to discredit, place blame, embarrass, or otherwise put things in the worst possible light. When this “gotcha” game is played with measurement data it can have disastrous effects on court performance initiatives. For example, relatively straightforward efforts to monitor, analyze, and assess court citizen encounter using a survey such as that for Measure 1...

“What if I were to tell you,” IBM’s Dean R. Spitzer asks in the first sentence of his 2007 book, Transforming Performance Measurement – Rethinking the Way We Measure and Drive Organizational Success (New York: American Management Association), “that one of the most important keys to your organization’s success can be found in a very unlikely place – a place many of you may consider to be complicated, inaccessible, and perhaps even boring? … The key to success is MEASUREMENT,” he tells us (emphasis in the original). Effective performance measurement and management can transform your organization, writes Spitzer. It shows you where you are and gets your organization where it wants to be. I, of course, agree with Spitzer. But I would go even further. Performance Measurement No Longer Optional How is your court or court system performing in meeting its fundamental obligations to those it serves -- access to justice, fairness and equality, efficiency and effectiveness, professionalism, hon...

Where We Are Today The fundamental goal of court performance dashboards is to empower all stakeholders with the right information, at the right time, using the right technology to make better decisions across all court functions. An increasing number of individual courts and court systems are beginning to look at performance dashboards, and supporting business analytics and intelligence, not only as reporting and accountability tools but as the means to make improvements and to drive success. They deploy these tools to discover and to explore information, to describe historical trends and to predict future trends, to devise improvement strategy, and to share information with stakeholders. That’s the basic message my colleagues and I delivered at the Super Session, Performance Dashboards: Measuring, Monitoring, and Managing Your Courts, at the Tenth Court Technology Conference (CTC10) in Tampa, Florida, last month. I concluded at the close of the session attended by about 300 – 400 cou...

The theoretical foundations and methodology of the disciplines of research and performance measurement overlap, but they are very different in important ways: sponsorship, organization, audience, functions, timing, and data interpretation rules. (See The Differences Between Performance Measurement and Research , Made2Measure , October 7, 2005; and Forget “Statistically Significant ,” Made2Measure , December 17, 2005.) Replication in Performance Measurement and in Research A critical difference between performance measurement and research that I did not mention previously has to do with replication. Basically, this means repeating the research to corroborate the results and to safeguard against overgeneralizations and other false claims. Repeated measurements on a regular and continuous basis are part of the required methodology in performance measurement. Analyzing trends beyond initial baseline measurement requires replication of the same data collection and analysis on a monthl...

Having proposed or otherwise advocated for various performance measures that have not seen the light of a court day, I should be accustomed to the low use of measures that I happen to believe have high value. But, alas, I continue to puzzle over why one such measure in particular – jury representativeness -- is not used more by courts. Jury representativeness – as I defined it in a two-part Made2Measure posting on April 12 and April 22 last year – is the comparative parity (i.e., the absence of disparity) -- expressed as a percentage -- between the representation of minority groups in the population and the representation of the same groups in the final juror pool or venire. How well juries mirror the community from which they are drawn is widely considered a reflection of the equality, fairness and integrity of our justice system. Arguably, identifying a combination of demographic characteristics as the source referent -- including gender, age, income level, and education -- may be b...

If you do nothing else, be clear, says Marcus Buckingham , who has spent a lot of his time studying leadership. You probably thought that your job is to analyze the complexity and chaos of your court’s operating environment and reflect it back to the court’s stakeholders. You’d be wrong. (When you’re asked what time it is, don’t respond with instructions for building a clock!) Tell people what it is the court should achieve. But tell them succinctly. Point them in the right direction and give them a good handle on how to get there. Today’s court executives and managers need more than a strong message and charisma to lead effectively. They need a performance measurement system that focuses and magnifies what is most important. They need ready access to clear and actionable measures that allow them to explore the court’s performance from multiple perspectives and to steer the court in the right direction. Performance measures – like the percent of court users who are satisfied with the w...

Not even close, at least in the business sector. In the last posting, I suggested that court managers listen actively and patiently to what people have to say no matter what truth-finding ways and means they’ve used including: (1) the truth we feel (including “truthiness”); (2) the truth that is given to us; (3) the truth we ferret out ourselves by reason and logic; (4) and the truth we perceive through our senses and empirical data, including performance information. No one method is foolproof, I argued. Each can complement and correct the mistakes of the others. Better to use all four methods, though we may favor one. The common ground is where the truth is more likely to reside. I advised – sensibly I think -- that we not we not discredit those who prefer to reach the truth in ways different from ours. Apparently, business does not see it that way. “We’ve had management by objective and total quality management. Now it’s time for the latest trend in business methodology: management ...

“The truth-quest is always the same: the unwavering search for signs to match reality.” -- Felipe Fernandez-Armesto Stephen Colbert, the satirist and host of The Colbert Report, coined the term “truthiness” to refer to the things people know to be true “from the gut,” as opposed to from the head and from dry data that comes from the laborious (and boring) process of science. Many court managers, judges and clerks rely on truthiness to answer the question “How is the court performing?” Quite predictably, some of us who are “made to measure” and feel smug about using the scientific method to get at the truth disparage them. That disparagement may be misguided, if not arrogant or dead wrong. Getting at the Truth But how do we know what is true? How do we know the Way Things Really Are? How can we tell falsehood from truth? In his deft little book, Truth: A History and a Guide for the Perplexed (St. Martin's Press, 2000), Felipe Fernandez-Armesto, a member of the modern history facult...

What is a court’s business? Peter Drucker, probably the most revered management thinker, suggests that this fundamental question may seem simple and obvious. It is in fact difficult and anything but obvious (see his Management: Tasks, Responsibilities, Practices. HarperCollins, 1993, pp. 77 – 79). I know it may sound heretical, but a court’s business is not necessarily determined and defined by law. Instead, it is defined by the wants and needs of its “customers,” suggests Drucker. “To satisfy the customer is the mission and purpose of every business.” By business, he means any enterprise: private, non-profit, and public. For example, an appellate court may consider its primary customers to be the members of the appellate bar and trial bench who are the major consumers of the appellate court’s decisions and opinions. Because a few pro se litigants are likely to be the only members of the public who have direct contact with the appellate court, and because most appellant litigants’ co...

Transparency is the hallmark of good government, an antidote to corruption and a facilitator of openness, communication, and accountability. It is the metaphorical extension of the meaning used in the physical sciences – a “transparent” object is one that can be seen through. In his new book, The Art and Practice of Court Administration , Alexander B. Aikman urges courts to be more transparent. He suggests that “leadership courts” are those that open their administrative decision-making to public input, scrutiny, and participation, and argues that the tradition of silence regarding judicial decisions need not be carried over to administrative decision-making. So much for theory. Transparency in practice is another matter, especially when it comes to courts’ performance. How Much Performance Information Should We Share? This is a question that causes much hand-wringing among court leaders and managers. Much of the resistance to performance measurement is based in fears that baring the...

What is fairness? Why do people cooperate with authorities? Why do they obey the law? Why is public trust in our courts so important? To his already impressive body of research addressing these questions, Tom R. Tyler , Professor of Psychology at New York University, continues to add to our understanding of the interplay of fairness and trust and how both effect cooperation with authorities. Writing in the May issue of the Journal of Applied Psychology (Vol. 92, No. 3, 639 – 649), Tyler and his colleague David De Cremer, who is in the Department of Economic and Social Psychology, Tilburg University, Netherlands, report the results of two experimental studies and two field studies of the effects of procedural fairness and trust on people’s willingness to cooperate with authorities. What Tyler found should be of interest to court leaders and managers: procedural fairness leads to cooperation and compliance only when trust in authority is high. It seems you can’t have one without the oth...

An employee of a large urban court recently confessed to me her opinion about her court’s performance measurement initiative. (She did so reluctantly because she knew that I’m a strong advocate of court performance measurement.) She told me this: The results of the court’s access and fairness survey of court users holds about as much interest for me as the details of the presiding judge’s travel schedule – interesting, but not something that is relevant to what I do everyday. Hers was not an isolated viewpoint in the court. Even though the survey results were made readily available in the court newsletter and other printed materials distributed by the court, most mid-level managers and line staff paid little attention. Some viewed the survey data as yet another brainchild of upper management that would translate into additional work, and nothing more, if they were not careful. This is a problem that needs fixing. You can’t advance an idea in an unreceptive environment. Getting With the...

For some folks in court administration, the subject of performance measures is an anathema – the revenge of the number crunchers and spreadsheet guys -- for others, it’s their lifeblood. Apparently, the financial world sits up and takes notice when someone messes with its numbers. Today’s Wall Street Journal reports that accounting-rule makers around the world, in the coming months, may eliminate the bedrock measure of business health and value: net income or net profit, the bottom-line figure showing what is left after expenses have been met. Net profit is the measure of business performance millions of investors use every day to buy or sell stocks and bonds. It is used to determine executive bonuses and other forms of compensation. The single net profit number is the most commonly used measure to evaluate a company’s health, especially when it’s compared to the price of the company’s stock, what’s called the price-to-earnings ratio, or simply P/E. For example, the P/E ratio of comp...

Much of today’s hype about performance dashboards – a new book seems to coming out every few days -- misses an important point. Stephen Few, in his book, Information Dashboard Design: The Effective Visual Communication of Data (O’Reilly, 2006), for example, writes that dashboards have emerged in response to the “tsunami of data that rolls over and flattens us in its wake.” To be sure, performance dashboards help those who are drowning in data -- the executive, manager or analyst who is overwhelmed with too much data. A well-designed performance dashboard lets her view performance measures at a glance, and then move easily through successive levels of strategic, tactical and operational performance information to get the insight she needs to solve problems and to improve program and services. The focus is on those who already have access to performance data – albeit in a form that prohibits or at least discourages its use. But what about the people who do not yet have any access to re...

The key to self-renewal and survival for an organization is a performance measurement and management system integrated with key management processes and daily operations. Courts are conservative institutions. Unlike private business, which society will let disappear, courts are here to stay. They are steeped in the traditions of law that favor stability and continuity (precedence) over innovation and change. Like most public inctitutions, courts were originally created to prevent, or at least to slow down, change. Therefore, courts need to work harder than other public and private organizations to counter rigidity and decay that threaten their continuity. Revolution every so often, as Thomas Jefferson recommended, is a radical response to senile decay and a failure of self-renewal. Obviously, such destabilization to the point of destruction is no way to run a court. As Peter Drucker has pointed out in several of his writings, to maintain continuity – to survive – organizations must fi...

Gainsharing – it’s like profit-sharing in the private sector -- is a system whereby units of government (e.g., a collections division or a jury management unit of a court) share in the gains teams of employees make in their bottom line or that of the state, county or city, without losses in quality of services and programs. Employees receive bonuses or payments based upon the improved productivity or efficiency as reflected in "gains" in cost savings or revenue increases. Gainsharing is based on widely accepted management principles that encourage employee initiative in continuous improvement of program and services to meet the needs of customers and citizens. In my January 21, 2007 posting I discussed the prospects of gainsharing in courts . The promise for courts, I argued, is that gainsharing may help courts achieve sustained increases in productivity and efficiency, that employees may become more involved in the gains made by the court as they share in the benefits of emp...

Variation in treatment of court users and court employees, in the time and cost of processing cases, in the reliability and integrity of case files, in the compliance with court orders, and in other key court performance areas is inevitable but generally not desirable. If you had your choice of between processes that produced predictable and consistent results and ones that produced good results one day and bad the next, poor quality under some circumstances and good quality in others, which ones would you choose? Both court managers and the public recognize the benefits of stable processes and consistent and predictable results. Understanding and controlling variation (e.g., knowing whether particular performance falls outside established upper and lower “control limits”) are at the heart of quality improvement methods such as Total Quality Management (TQM) and Six Sigma. More than 25 years ago, W. Edward Deming and Joseph Juran noted that variability on core measures of performance i...