Henry Mintzberg Misses the Mark on Performance Measurement Data

We’ve all been in situations where we get agitated because someone we admire, and with whom we generally agree, goes too far in pushing his or her agenda. This situation occurred to me as I read Henry Mintzberg’s new book, Managing (Berrett-Koehler, 2009), which updates his thinking in his first book, The Nature of Management (Harper & Row; reprinted by Prentice-Hall, 1973) based on his doctoral dissertation more than 35 years ago.

Mintzberg all but dismisses the value of using performance outcome data in favor of an “information diet” of gossip, hearsay, and speculation.” Such “informal information” he writes, “can be much richer, even if less reliable.”

I could not believe what I was reading. It goes counter to what I’ve been advocating to court managers for years, i.e., effective performance measurement and management can transform your court; it shows you where you are and gets you to where you want to be. Performance monitoring, analysis and management are no longer an option for successful court managers. (See, for example, "Principles of Effective Performance Measurement," Made2Measure, November 12, 2007; and "Court Intelligence – A Matter of Survival," Made2Measure, December 11, 2007)

While Mintzberg’s observations and insights about what managers actually do – the actual practice of management – are invaluable contributions to the literature of management and leadership, he badly misses the mark when he disparages management based on what he calls “formalized management information systems.”

I know that many of today’s court managers rely heavily on performance data and are wary of spouting anecdotal evidence when asked “How is the court doing?” They know that their courts’ stakeholders increasingly want them to show them the data, and nothing else will do.

The Elevation of Anecdote, Gossip and Speculation

Today’s managers ignore evidence-based performance data and rely instead on a steady information diet of gossip, hearsay and speculation at their peril.

Consider this: If your court or court system has developed the right measures (e.g., court user satisfaction, case clearance, on-time case processing, and employee engagement), all on a balance scorecard of performance that is available to you on demand on a self-service basis in real-time, why would you risk basing your decisions on gossip, hearsay and speculation? Excuse the metaphors, but why would you not rely on your car dashboard right in front of your eyes to give you a reading of your car’s speed and, instead, ask the passengers to speculate how fast you’re going? (Such speculation won’t carry the day with a trooper that has stopped you for speeding?) Or why would you depend on speculation about your health when you could be reading your blood pressure and cholesterol levels?

A Blind Spot

But let me give Mintzberg his proper due. He’s the Cleghorn professor of management studies at McGill University and one the most original management thinkers writing today. As I said, I generally admire his writings.

Many years ago, my colleague and friend Bob Wessels -- the manager of the Harris County (Texas) Criminal Courts, and a innovate manager in his own right who has invested heavily in performance measurement and business intelligence -- introduced me to Mintzberg’s writings and, like Wessels and others in court administration, I have read his writings steadily since.

My copy of his 1994 book, The Rise and Fall of Strategic Planning: Reconceiving Roles for Planning, Plans and Planners (Free Press) is dog-eared and filled with my marginal notes. I found his recent article in the July – August 2009 issue of Harvard Business Review, “Rebuilding Companies as Communities,” truly inspiring.

Mintzberg has championed study of the status quo – what managers actually do rather than what management theorist conjure up about the job of management and leadership. He is among only a few scholars who have bothered to study mangers and to make some sense of the work they actually do in organizations. For this alone, he is worth reading.

Mintzberg, of course, is also right that much of management is not necessarily information-driven.”To manage through information means to sit two steps removed from the ultimate purpose of managing: information is processed by the manager to encourage other people to take the necessary action. In other words,” he writes, “on this plane the focus is neither on people nor on actions directly, but on information as an indirect way to make things happen.”

So far so good, but why go to the extreme of dismissing performance data almost altogether and instead elevate gossip, hearsay and speculation as staples for managers? One step too far, I say, you’ve lost me in what appears to be a big blind spot.

I am particularly disturbed and disappointed about the message Mintzberg sends to court professionals about modern performance measurement and management systems, i.e., the monitoring, analysis, and management of results (outcomes or accomplishments) on a regular basis for the purposes of transparency, accountability and continuous improvement. The significant position that these modern information guidance systems occupy in today’s management practice seems to have been missed or ignored by Mintzberg.

Maybe because Mintzberg has focused much of his work on what managers actually did years ago, he shows little patience with what managers may have to do, what they should do, and what they want to do today and in the future. All of the latter, he suggests, is part of the popular but unreal image of the manager as symphony conductor carefully planning, organizing, coordinating, and controlling.

But if the managers Mintzberg studied really favored a steady diet of anecdotes, gossip and speculation, I would go so far as to suggest that they are bad managers, or at least not as good as they could (and should) be. (See "This Just In: Performance Measurement Works," Made2Measure, September 24, 2008)

Mintzberg blind spot may also extend to what today’s managers actually do in the area of performance measurement and management. For example, many managers, including court managers, are increasingly turning to performance dashboards, an aspect of business intelligence, to monitor, analyze and manage their organizations’ performance. (See "The Real Promise of Performance Dashboards," Made2Measure, May 9, 2007)

The information on a performance dashboard is consolidated and arranged in a combination of text and graphics on a computer screen that busy executives, managers and staff can easily monitor and analyze. The dashboard display is dynamic, allowing users to navigate rapidly across and through layers of strategic, tactical and operational performance data as they choose, whenever they choose, as data is updated on a real-time basis (the performance data available on the modern dashboards, like the one completed in September 2009 by my colleagues and me for the Kosovo Judiciary updated every day).

Performance dashboards help court managers to identify problems or opportunities quickly as they happen, to identify trends and patterns, to discover ways to improve programs and services, and to help guide them toward effective decisions. It provides a “line of sight” between core strategic indicators and subordinate tactical and operational indicators that convey to them what the drivers of successful court performance might be, and gives them concrete knowledge about how they contribute to that success.

Sure beats gossip, hearsay and speculation!

A Straw Man

Mintzberg writes that managers favor oral communications because the information they get from formal management information systems is not timely, is outdated once it gets to them, and typically is not easy to understand. What busy mangers have time for that, he asks.

This is a straw man argument and it surely goes too far. No one is suggesting that busy managers should wait around for someone to deliver untrustworthy performance data long after is it of any use. Has Mintzberg bothered to sample the burgeoning literature on performance dashboards and business intelligence technology? Did he happen to catch the special July 2009 issue of Wired focused on the “personal metrics movement” with the cover title “Living by Numbers – Track your data? Analyze your results. Optimize your life”?

The trouble is that Mintzberg has what seems to be an outdated perspective on modern management information systems and the use that managers make of them today. He rails against email – he refers to it as “this new medium” --in ways that I think readers will find archaic, if not uninformed.

I confess that as I read Managing I could not help think of Mintzberg as a technophobe who takes stances against modern information technology in order to preserve his ideologies.

© Copyright CourtMetrics 2009. All rights reserved.

Court Executives (Should) Have Their Heads in the Clouds

The launch this month of Microsoft’s new operating system, Windows 7, marks the end of one era of information technology and the start of another, says the Economist (“Briefing Cloud Computing,” October 17, 2009). Windows is not going to disappear but it will be much less important in the future.

Cloud Computing

Much of the computing we do today on our computers in our homes and offices will soon be – so to speak - in the “clouds,” and not on our personal computers, where Windows resides today. Instead, desktop computing on personal computers – featuring full-featured database and spreadsheet capabilities – is being replaced by IT architectures that call for the heavy lifting to be performed by external data centers accessible to us over the Internet.

Cloud computing is attracting an enormous amount of attention. The term “cloud computing” is a metaphor that originated with IT architects who routinely used cloud shapes to depict the flow of data from unknown external sources instead of the squares, rectangles and cylinders representing internal servers, storage disks, and databases they used to represent internal sources.

We are already familiar with cloud computing technology, though we may not have grasped its revolutionary significance. Widely used services like Twitter, YouTube, Skype, and Google’s Gmail, Docs, and Spreadsheets are all built on cloud computing technology. They deliver content and services to our computers, smart phones, and other devices from external sources

An Attractive Option for Cash Strapped Courts

Chief information technology officers see in cloud computing the promise of lower costs, less complexity and work for their IT departments. Cloud computing enables a court to tap into computing, storage systems, and software applications like performance dashboards and other business intelligence over the Internet.

At least in theory, cloud computing technology all but eliminates the burden of software installation, maintenance, ongoing operation, and support. Coming up with new software versions or changing requirements is taken care of the by the service provider. Costs are a continuous expense, rather than a single expense at time of purchase.

Cloud computing lets courts avoid building their own performance dashboards and associated technology architecture. They pay only for the services they need, when they need them. Both big and small corporate computer and software vendors that have a presence in the court community will soon be offering performance dashboards based in cloud computing technology.

One company, Threshold CS, based in Clearwater, Florida, makes its court performance dashboard, Justice PM (Performance Measurement) available both as a traditional client site installation and as a cloud computing service (Threshold calls it by its antecedent name, Software as Service or SaaS). For a monthly subscription fee, after a brief data linkage to the court’s source (legacy) systems, a court’s performance data is automatically uploaded to Threshold’s data center and the performance dashboard is accessible to court users via a secure Web interface.

© Copyright CourtMetrics 2009. All rights reserved.

Rankings Based on Outcomes

Today's Wall Street Journal printed my letter to the Editor on the value of rankings focused on outcomes, a topic that has occupied this space often:

Most “number guys” can criticize performance measures, especially rankings, in their sleep because most measures are imperfect. Carl Bialik (“Ill-Conceived Ranking Makes for Unhealthy Debate,” The Numbers Guy, Oct. 21) takes the easy route for a cheap shot at the dated and flawed World Health Care low 37th ranking of the U.S. in the world in health care. He suggests that the “unhealthy debate” caused by the U.S. ranking would be cured by more methodological rigor and that, in any event, we shouldn’t rank everything, especially health care. I wonder if Mr. Bialik thinks that our debate about health care would be healthier if the U.S. ranked let’s say 12th in the world using methods that pass muster with the scientific community. I suspect that most of us would like to see the U.S. in first place and that even 12th place would not change what we think of health care today. Americans love rankings, the top-ten of whatever including cities, high schools and universities, hospitals and toasters. I think rankings of health care are here to stay and that’s a good thing. As flawed as they may be, they focus attention of the debate where it should be, on outcomes.

Ingo Keilitz
Williamsburg, Virginia

Decentralized Innovation and Improvement

Court systems have concentrated too much authority for continuous improvement at the top where there are good intentions, but relatively few resources and little capacity. Court performance data are delivered too little and too late, if at all.

When performance is presented to staff, it is often done so in endless documents stuffed with indecipherable figures and statistics, to make much of a difference. Performance data is a virtual temple secret that only the priests (designated top-management and analysts) can read and interpret.

Courts should seek to give all court employees all the performance measurement information they need to make improvements themselves. Courts should tap into the capacities of all court employees to track and analyze performance data and to devise solutions to problems.

A similar strategy, referred to as “radical transparency,” was advocated as a road map for economic recovery in a “manifesto” written by Daniel Roth in Wired March 2009: “Instead of assigning oversight responsibility to a finite group of bureaucrats,” writes Roth, “we should enable every investor to act as a citizen-regulator. We should tap into the massive processing power of people around the world by giving everyone the tools to track, analyze, and publicize financial machinations. The result would be a wave of decentralized innovation.”

Decentralized innovation and improvement are central to "open book management," a management approach created by Jack Stack and popularized by John Case in his 1995 book, Open-Book Management: The Coming Business Revolution (New York: Harper Business). Its aim is to give employees all relevant information (the focus is on financial information) about their company so they can make better decisions as workers.

The concept behind open book management is the same as that supporting "line of sight" in performance measurement and management. That is, in order for performance measures to be practical tools and to serve as incentives for improvement -- for measures to be motivational -- there must be a line of sight between the measure and actions that can be taken by employees at various levels of the court system. It conveys to everyone what the drivers of success are and provides them with the concrete knowledge of how they contribute to that success.Here are the basic rules of open book management (extended beyond financial information) to all performance data available to court managers and staff as part of a comprehensive court performance measurement system:

-- Give all employees access to all performance data on a self-help basis. Line of sight can be created by computer based graphic displays in performance dashboards.

-- Give employees a simple performance scorecard. Because employees may not be naturally drawn to mind-numbing sets of numbers, open book management relies on the “critical number” and a “scorecard” that brings all the critical numbers together.

-- Give employees training to understand and to use the performance information available to them including an intensive system of performance review meetings to keep employees informed about the status of the court in terms of its performance measures.

-- Give employees ownership and responsibility for the numbers under their control. If the performance measure rises above the control level, the “owner” of the measure is held accountable.

-- Give employees a stake in how the court performs.

One proponent of open book management, Ed Dorian, the president and chief operation officer of a White Plains, N.Y., company, was quoted as saying in a recent Wall Street Journal article, “I saw open book as an opportunity … to send a message to our staff that said, ‘We trust you. And we not only trust your intentions, but we trust your ability to help us.’” (Laura Lorber, “An Open Book: When Companies Share Their Financial Data With Their Employees, the Results Cab Be Dramatic,” Wall Street Journal, February 23, 2009, R8).

For the latest posts and archives of Made2Measure click here.

© Copyright CourtMetrics 2009. All rights reserved.

Annals of Backlog and Congestion: New Delhi, India

State court leaders and managers take heart! Things could be worse. Much worse!

In a widely circulated story last week, Sam Dolnick of the Associated Press (AP) reported that, according to Chief Justice A.P.Shah, the High Court in New Delhi is so behind that it could take up to 466 years (not days or even months, years) to clear its backlog of cases. In a vast understatement, retired Supreme Court Justice J.S. Verma, who is critic of the system, is quoted as saying “I don’t think you would have to wait four centuries to have a case decided.”

Reasons cited for the backlog in India include the usual suspects: lack of accountability for results, corruption, inefficiency, and an uneven application of the rule of law favoring the wealthy and well-connected. Another is that India does not have enough sitting judges. "It’s a completely collapsed system,” Prashant Brushan, a well known lawyer in New Delhi, is quoted as saying. “This country only lives under the illusion that there is a judicial system.”

What is particularly noteworthy and instructive about this story is the New Delhi High Court’s use of its jaw-dropping congestion rate as the measure of efficiency (or in this story, inefficiency) to stimulate reform and buttress its appeal for an injection of more judicial resources. Congestion rate is the pending caseload divided by the number of resolved cases. It estimates the amount of time it would take a court to dispose of its pending caseload given its current clearance rate.

As the New Delhi report suggests, congestion rate is a more meaningful and persuasive efficiency measure than backlog (percent of pending cases “older” than an established threshold) and clearance rate (the number of outgoing cases as a percentage of incoming cases), and even other measures such as the number of judges per capita. In a 1999 technical paper, Court Performance Around the World: A Comparative Perspective, Maria Dakolias of the World Bank reported that it would take Ecuador a little more than ten years to dispose of its pending caseloads, the poorest congestion rate among 12 countries she compared.

I don’t know what’s good or bad, but I immediately know that 466 years of congestion is really bad. I also have a sense of the scope, if not the nature, of the solutions. It will take more than a band aid to fix the New Delhi High Court’s backlog and congestion.

The lesson is that if we want our performance measurement initiatives to be noticed and used – what gets measured gets attention and gets done – we need to identify and develop measures that translate abstract concepts like efficiency and effectiveness into terms that are clear, unambiguous, and actionable and that capture the attentions of the public and other stakeholders.

For the latest posts and archives of Made2Measure click here.

© Copyright CourtMetrics 2009. All rights reserved

Measuring What Really Matters in Hard Times

State courts are facing severe budget cuts in the current economic crisis. According to the Center on Budget and Policy Priorities, at least 44 states are facing shortfalls in their FY 2009 and/or FY 2010 budgets. By most accounts, the situation is likely to get worse before it gets better.

Some of my more optimistic colleagues (who -- I might add -- are fortunate enough to have solid jobs) subscribe to the “necessity is the mother of invention” school of thought on the deepening recession. They have a point.

While they do not wish ill toward their court friends on the receiving end of drastic budget cuts, they see a bright spot in the months and years ahead. They welcome the sense of urgency. They're hoping it will give birth to clarity of focus and innovation.

They see courts and state court systems today forced to confront issues and questions that they believe should be asked all the time, not just now: What are our fundamental obligations? What is expected of us? Which programs and services are mandated and which are not? What should we stop, continue and start? On what basis?

My sense is that as local courts and state court systems deal with budgets that are hemorrhaging, the internal and external scrutiny of courts will intensify and focus on what matters most. In good times, the operative question is “Are you doing things right?” In other words, are the programs or services doing what they are designed to do?

In bad times, the question shifts more toward fundamental worth and merit, toward the impacts of programs and services, to return-on-investments, and to cost-benefit: “Are you doing the right things?” In other words, is the program or service you are providing meeting the court’s fundamental governmental obligations. Are they contributing to a positive change in the status or condition of people they are designed to serve? If not, then you need to give up the resources or move them to programs and services that have positive impacts.

For court performance measurement and management this means “counting what counts,” i.e., a preference for outcome measures over input and output measures. (See “A Preference for Outcome Measures,” Made2Measure, September 28, 2005).

Jack Roedema, court administrator of the 17th Circuit Court in Kent County, Michigan, believes that he has witnessed first-hand this shift toward a preference for outcome measures. I know that he and Chief Judge Paul Sullivan have done a masterful job in the past using performance data to develop, maintain and strengthen the court’s relationship with county officials who fund the court. The court uses close to 100 performance measures on a regular basis.

In a recent email, Mr. Roedema said that he is seeing county officials looking at the court from a different perspective than they did in the past. They still greatly appreciate assurances backed by solid data showing that the court programs and services are doing what they’re supposed to be doing, but they are beginning to ask “so what.” With emphasis on mandated versus non-mandated services and significant budget restraints in his county, he says that it “seems like in our court we need to go a step further with [performance measurement].” In the area of alternative dispute resolution (ADR), he is beginning to report the results of a “sort of cost-benefit analysis.” If as a result of mediated settlements the ADR program prevents 60 cases from reaching a judge, then “we are saving about 120 hours of judicial time,” Roedema said.

This type of information now seems to strike more of a chord with the county officials, Mr. Roedema believes. “Of course, you can't really get to the ‘so what’ information unless you have good performance measures in place,” he said.

I told him that I agreed that he was heading in the right direction by looking at cost and time savings of programs and services, but I recommended that he set his sights higher toward even more outcome-oriented performance measures. For example, I suggested that he think of the purpose of juvenile diversion, another program he cited, not only in terms of savings in court resources, but also in terms of community safety, welfare and protection. How do you measure success in that area? Recidivism? Payment of restitution? Direct contributions to the community by work in lieu of monetary penalties? What is the court’s fundamental obligation in these programs?

If funders are made aware by trustworthy performance data that a program or service not only runs the way it was designed, but also contributes significantly to the protection and well-being of the community, they are much less likely to give it the ax. For courts and state court systems, counting what counts and measuring what really matters may be a sound strategy not only for accountability, but also for survival.

For the latest posts and archives of Made2Measure click here.

© Copyright CourtMetrics 2009. All rights reserved.

Ranking High Schools and Courts on Their Performance

The 2009 U.S. News & World Report second annual rankings of America’s best public high schools came out this week. The rankings were done by School Evaluation Services, a K – 12 education data research firm run by Standard & Poor’s, based on an analysis of the performances of 21,069 public high schools in the 2006 -2007 school year (see www.usnews.com/highschools).

The annual rankings of high schools hold two important lessons for judges and court managers, especially those who bristle at the idea of comparative performance measurement. (See “Ten Reasons Not to Measure Court Performance,” Made2Measure, November 19, 2008)

The first lesson is that performance matters to citizens. The U.S. News & World Report rankings are based on the key principle that a great high school must be able to produce measurable academic outcomes to show that it successfully educates all of its students across a range – a balanced scorecard – of performance indicators. Little else matters to parents as much.

The hallmarks of a high-performing court are the political will and the capacity to produce measureable outcomes that matter to the public and other stakeholders. The best courts ask themselves “How are we doing and how could we improve?” They measure their performance on a regular and continuous basis to gain knowledge and insight into new realities, opportunities and necessities and to convert that knowledge and insight into effective strategies for improvement.

The second lesson to be learned from the U.S. News & World Report rankings is that similar comparisons and rankings of courts are likely to occur with increasing frequency. Court leaders are well advised to be prepared with their own court performance data.

Just last week, the Michigan State Court Administrative Office took great pains to inform a state-wide gathering in Lansing of 100 chief trial court judges and court administrators about “current misinformation” about the cost-per-case and clearance rates of Michigan trial courts compiled and presented to state and national groups by a Michigan State University academic researcher.

The fact is that the best high school, colleges and universities, cities, public parks and, yes, America’s courts are of great interest to citizens. Courts not producing performance data to satisfy this interest are going to be asked why they’re not (see “How Do we Stack Up Against Other Courts? The Challenges of Comparative Performance Measurement,” The Court Manager, Vol. 19, No.4 (Winter 2004-2005).

See also “Toward the 50 ‘Best in Class’ Courts,” Made2Measure, April 23, 2008

For the latest posts and archives of Made2Measure click here.

© Copyright CourtMetrics 2008. All rights reserved.