Made2Measure

I recently was invited to participate in a new pilot program offered by my electricity provider, Virginia Dominion Power. It’s a part of a national trend to help consumers control their electricity and, at the same time, help utilities cut their operating costs. Virginia Dominion Power, following the lead of utilities in California, Texas and other states, is rolling out the program to install “smart” electric meters in homes in the belief that they will help cut electricity consumption and reduce the need for new power plants. The smart meters will allow consumers to see, at a glance, how much electricity they are consuming at what cost. The Critical Peak Pricing Pilot Program is based on two joined approaches increasingly used in the private sector. I predict that these two approaches will soon become standard “best practices” for most monitoring, analysis and management of all kinds of performance, including that of courts. (1) Bring everyone – customers and employees alike – not ...

Q: How many performance measures should my court be considering? My colleagues have reviewed the ten performance measures of the CourTools developed by the National Center for State Courts, as well as other measures – like treatment court recidivism -- that courts are considering. They all seem compelling but we are concerned that the whole enterprise will just be too much for us. What advice do you have? A: It is better to do more with less than less with more. First and foremost, how many performance measures your court should develop depends on what matters to you, what you consider important – the court’s key success factors. If these include, for example, access and fairness, citizen satisfaction, expedition and timeliness, community welfare, and employee engagement, you’re likely to need more measures than if you’re only interested in efficient case processing. Second, there’s no sense in developing measures that will not be used. Generally speaking, it is much better to do mo...

Along with a number of client-partners, I’ve been rethinking the way courts gauge the satisfaction of court users and the strength and engagement of their work force And, yes, how we can improve the court user – court employee encounter. Quite a few court leaders and managers still regard these lines of performance measurement and management as “touchy feely,” “soft” or “subjective,” despite the overwhelming evidence to the contrary from some hard-nosed researchers and practitioners. But that’s a subject for another time. Human Sigma In their book, Human Sigma: Managing the Employee – Customer Encounter (Gallup Press, 2007) – a much expanded version of their groundbreaking article “Manage Your Human Sigma” that appeared in the July/August 2006 issue of Harvard Business Review – Gallup researchers John H. Fleming and Jim Asplund rewrite the rules for how we should measure and improve employee - customer encounters. Though based on extensive research of employees and customers of privat...

Performance measures are often proxy variables. That is, they are not necessarily themselves of any great interest but they can tell us a lot about a particular thing or outcome. To varying degrees, performance measures are such proxies -- removed and highly simplified version of the outcome of interest. We use proxy measures in order to make it possible to measure things easily, routinely and at a reasonable cost. The value of a proxy measure is that it is expected to correlate with the desired outcome. Not perfectly, but good enough. In other words, while some performance indicators may or may not jibe with our common understanding or mental images of the concept or construct under consideration, they indicate its meaning. Cholesterol level is not health. Tree ring widths and ice core layering are not temperature records. Case clearance is not exactly court productivity or efficiency. Recidivism is not community well being. And – here it goes -- a win by the Patriots over the Giants ...

How is your court or court system performing? How should it perform? Answering these two questions requires different methods: the first, performance measurement; and the second, the establishment of standards, benchmarks or goals to serve as norms or models for others. A court should strive to answer both questions, but it should not delay answering the first because it is uncertain about the answer to the second. Unfortunately, this is not how things have happened. Points of Reference Versus Standards Difficulties arise when certain points of reference required to answer the first question are unnecessarily burdened with the weight of serving as performance benchmarks , standards and goals . For example, the Oregon Court of Appeals measures the timeliness of its processing of land use cases against the point of reference of 91 days. The Court defines the metric of on-time case processing of land use cases as the percent of cases disposed or otherwise resolved within 91 days. Similar...

Measures without meetings are meaningless Performance data that are not used – by definition – are useless. Yet, an alarming number of courts and court systems ignore this simple lesson. Many individual courts and state court systems have made vast investments in securing the right sponsorships and resources for performance management initiatives, creating the right metrics and standardizing their meaning, and even building pricey performance dashboards and business intelligence technology to deliver the performance data. But, after all that, they fail to ensure that the performance data is actually used to drive strategy and success. This is a shame because many solutions, like a simple performance review meeting can help to integrate performance measurement into the very fabric of a court’s leadership, management and everyday operations. All it takes the decision to do it. By making performance monitoring, analysis, and management a standing item on the agenda of regular executive me...

The most important thing a court manager can do is to communicate what the court should accomplish, what results are desired, and why that’s important – the theory of the court’s “business.” Performance management with effective performance measure and indicators – like those of the National Center for State Court’s CourTools -- that are understandable, broadly applicable, uniformly interpreted, and practical to apply, are a court managers best tool for doing this. Recently, a court manager described his job to me as putting out fires. He said it consumes most of his time and energies. He said it with pride. This would be a good thing, but only if he were running a fire department instead of a court system. That would demonstrate clarity and strategic focus. After all, no one would question that putting out fires is “mission critical” for fire departments. Court leaders and managers must focus – and be seen to focus – on the court’s mission and its fundamental obligations. Saying that...

The benefits of an effective court performance measurement and management system are the same as those of strategic planning – accountability, consensus building, focus, coordination, control, learning, communication, hope and inspiration. To identify the right performance measures, a court must address the same fundamental questions about guiding ideals, values, mission, goals and broad strategies as it must address in strategic planning. We count what counts and measure what matters. And what we measure determines what will be considered relevant. Measurement uses numbers but it is ultimately not about numbers. It is about perception, understanding and insight. [1] It is not the measure itself that is important but rather the questions it compels us to confront. How are we doing? How is the court performing? Where are we now (performance level, baseline)? What is the current performance level compared to established upper and lower “controls” (e.g., performance targets, objectives, ...

For court performance measurement initiatives to succeed they must marry proven methods of assessments of the health of a court with a disciplined process for improving it. This may have to be a shotgun marriage. Without it – even in the very early stages of development -- the initiatives will fall prey to the “gotcha” game and be used to undermine the effort and to discredit it proponents. The Gotcha Game In this political season, we are all familiar with the ploy in which one candidate (or the media) seeks to catch another in a misstep or flub -- no matter how untruthful or inconsequential the accusation might be. The whole point is to discredit, place blame, embarrass, or otherwise put things in the worst possible light. When this “gotcha” game is played with measurement data it can have disastrous effects on court performance initiatives. For example, relatively straightforward efforts to monitor, analyze, and assess court citizen encounter using a survey such as that for Measure 1...

“What if I were to tell you,” IBM’s Dean R. Spitzer asks in the first sentence of his 2007 book, Transforming Performance Measurement – Rethinking the Way We Measure and Drive Organizational Success (New York: American Management Association), “that one of the most important keys to your organization’s success can be found in a very unlikely place – a place many of you may consider to be complicated, inaccessible, and perhaps even boring? … The key to success is MEASUREMENT,” he tells us (emphasis in the original). Effective performance measurement and management can transform your organization, writes Spitzer. It shows you where you are and gets your organization where it wants to be. I, of course, agree with Spitzer. But I would go even further. Performance Measurement No Longer Optional How is your court or court system performing in meeting its fundamental obligations to those it serves -- access to justice, fairness and equality, efficiency and effectiveness, professionalism, hon...

Where We Are Today The fundamental goal of court performance dashboards is to empower all stakeholders with the right information, at the right time, using the right technology to make better decisions across all court functions. An increasing number of individual courts and court systems are beginning to look at performance dashboards, and supporting business analytics and intelligence, not only as reporting and accountability tools but as the means to make improvements and to drive success. They deploy these tools to discover and to explore information, to describe historical trends and to predict future trends, to devise improvement strategy, and to share information with stakeholders. That’s the basic message my colleagues and I delivered at the Super Session, Performance Dashboards: Measuring, Monitoring, and Managing Your Courts, at the Tenth Court Technology Conference (CTC10) in Tampa, Florida, last month. I concluded at the close of the session attended by about 300 – 400 cou...

The theoretical foundations and methodology of the disciplines of research and performance measurement overlap, but they are very different in important ways: sponsorship, organization, audience, functions, timing, and data interpretation rules. (See The Differences Between Performance Measurement and Research , Made2Measure , October 7, 2005; and Forget “Statistically Significant ,” Made2Measure , December 17, 2005.) Replication in Performance Measurement and in Research A critical difference between performance measurement and research that I did not mention previously has to do with replication. Basically, this means repeating the research to corroborate the results and to safeguard against overgeneralizations and other false claims. Repeated measurements on a regular and continuous basis are part of the required methodology in performance measurement. Analyzing trends beyond initial baseline measurement requires replication of the same data collection and analysis on a monthl...

Having proposed or otherwise advocated for various performance measures that have not seen the light of a court day, I should be accustomed to the low use of measures that I happen to believe have high value. But, alas, I continue to puzzle over why one such measure in particular – jury representativeness -- is not used more by courts. Jury representativeness – as I defined it in a two-part Made2Measure posting on April 12 and April 22 last year – is the comparative parity (i.e., the absence of disparity) -- expressed as a percentage -- between the representation of minority groups in the population and the representation of the same groups in the final juror pool or venire. How well juries mirror the community from which they are drawn is widely considered a reflection of the equality, fairness and integrity of our justice system. Arguably, identifying a combination of demographic characteristics as the source referent -- including gender, age, income level, and education -- may be b...

If you do nothing else, be clear, says Marcus Buckingham , who has spent a lot of his time studying leadership. You probably thought that your job is to analyze the complexity and chaos of your court’s operating environment and reflect it back to the court’s stakeholders. You’d be wrong. (When you’re asked what time it is, don’t respond with instructions for building a clock!) Tell people what it is the court should achieve. But tell them succinctly. Point them in the right direction and give them a good handle on how to get there. Today’s court executives and managers need more than a strong message and charisma to lead effectively. They need a performance measurement system that focuses and magnifies what is most important. They need ready access to clear and actionable measures that allow them to explore the court’s performance from multiple perspectives and to steer the court in the right direction. Performance measures – like the percent of court users who are satisfied with the w...

Not even close, at least in the business sector. In the last posting, I suggested that court managers listen actively and patiently to what people have to say no matter what truth-finding ways and means they’ve used including: (1) the truth we feel (including “truthiness”); (2) the truth that is given to us; (3) the truth we ferret out ourselves by reason and logic; (4) and the truth we perceive through our senses and empirical data, including performance information. No one method is foolproof, I argued. Each can complement and correct the mistakes of the others. Better to use all four methods, though we may favor one. The common ground is where the truth is more likely to reside. I advised – sensibly I think -- that we not we not discredit those who prefer to reach the truth in ways different from ours. Apparently, business does not see it that way. “We’ve had management by objective and total quality management. Now it’s time for the latest trend in business methodology: management ...

“The truth-quest is always the same: the unwavering search for signs to match reality.” -- Felipe Fernandez-Armesto Stephen Colbert, the satirist and host of The Colbert Report, coined the term “truthiness” to refer to the things people know to be true “from the gut,” as opposed to from the head and from dry data that comes from the laborious (and boring) process of science. Many court managers, judges and clerks rely on truthiness to answer the question “How is the court performing?” Quite predictably, some of us who are “made to measure” and feel smug about using the scientific method to get at the truth disparage them. That disparagement may be misguided, if not arrogant or dead wrong. Getting at the Truth But how do we know what is true? How do we know the Way Things Really Are? How can we tell falsehood from truth? In his deft little book, Truth: A History and a Guide for the Perplexed (St. Martin's Press, 2000), Felipe Fernandez-Armesto, a member of the modern history facult...

What is a court’s business? Peter Drucker, probably the most revered management thinker, suggests that this fundamental question may seem simple and obvious. It is in fact difficult and anything but obvious (see his Management: Tasks, Responsibilities, Practices. HarperCollins, 1993, pp. 77 – 79). I know it may sound heretical, but a court’s business is not necessarily determined and defined by law. Instead, it is defined by the wants and needs of its “customers,” suggests Drucker. “To satisfy the customer is the mission and purpose of every business.” By business, he means any enterprise: private, non-profit, and public. For example, an appellate court may consider its primary customers to be the members of the appellate bar and trial bench who are the major consumers of the appellate court’s decisions and opinions. Because a few pro se litigants are likely to be the only members of the public who have direct contact with the appellate court, and because most appellant litigants’ co...

Transparency is the hallmark of good government, an antidote to corruption and a facilitator of openness, communication, and accountability. It is the metaphorical extension of the meaning used in the physical sciences – a “transparent” object is one that can be seen through. In his new book, The Art and Practice of Court Administration , Alexander B. Aikman urges courts to be more transparent. He suggests that “leadership courts” are those that open their administrative decision-making to public input, scrutiny, and participation, and argues that the tradition of silence regarding judicial decisions need not be carried over to administrative decision-making. So much for theory. Transparency in practice is another matter, especially when it comes to courts’ performance. How Much Performance Information Should We Share? This is a question that causes much hand-wringing among court leaders and managers. Much of the resistance to performance measurement is based in fears that baring the...

What is fairness? Why do people cooperate with authorities? Why do they obey the law? Why is public trust in our courts so important? To his already impressive body of research addressing these questions, Tom R. Tyler , Professor of Psychology at New York University, continues to add to our understanding of the interplay of fairness and trust and how both effect cooperation with authorities. Writing in the May issue of the Journal of Applied Psychology (Vol. 92, No. 3, 639 – 649), Tyler and his colleague David De Cremer, who is in the Department of Economic and Social Psychology, Tilburg University, Netherlands, report the results of two experimental studies and two field studies of the effects of procedural fairness and trust on people’s willingness to cooperate with authorities. What Tyler found should be of interest to court leaders and managers: procedural fairness leads to cooperation and compliance only when trust in authority is high. It seems you can’t have one without the oth...