Made2Measure

In previous posts I’ve called for courts and justice systems to be good stewards of performance data.   And, as an advocate of radical transparency , I’ve pushed for self-assessment and accountability by means of regular and continuous performance measurement and management instead of reliance on periodic third-party program evaluation and research. Now comes another good reason for courts and other justice institutions to go that route.   As reported by Carl Bialik, Census Gets Questions on Mandatory Queries , the Wall Street Journal’s “Numbers Guy,” members of Congress are challenging the Census Bureau’s massive data-collection effort known as the American Community Survey.   The survey uses dozens of detailed questions – including the use of carpools, flush toilets, and wood fuel – to help determine the need for and distribution of funds of various government programs. The survey, which reaches 3.5 million households, is mandatory, a fact that gets close to 100% o...

Add this big idea to the concepts that underlie modern performance measurement and management: We are seeing the end of traditional research and scientific method – hypotheses testing and experimentation -- as we know it. And performance measurement is there at the right time and place. From Kilobytes to Petabytes Today we’re able to capture, store, and make sense of massive amounts of data. We’ve gone from kilobytes to megabytes, from terabytes to the Petabyte Age. A petabyte is a measure of memory or storage capacity that is 2 to the 50th power bytes, or the equivalent of 20 million four-drawer filing cabinets full of text. Infinite storage or almost no storage necessary. Unlimited processing capacity. Early-warning sensors and measures everywhere sending automatic alerts from intelligent agents. No structured databases necessary. Just petabytes of information flowing through pattern-matching and trend-watching algorithms. It seems we really don’t need hypotheses and the...

The theoretical foundations and methodology of the disciplines of research and performance measurement overlap, but they are very different in important ways: sponsorship, organization, audience, functions, timing, and data interpretation rules. (See The Differences Between Performance Measurement and Research , Made2Measure , October 7, 2005; and Forget “Statistically Significant ,” Made2Measure , December 17, 2005.) Replication in Performance Measurement and in Research A critical difference between performance measurement and research that I did not mention previously has to do with replication. Basically, this means repeating the research to corroborate the results and to safeguard against overgeneralizations and other false claims. Repeated measurements on a regular and continuous basis are part of the required methodology in performance measurement. Analyzing trends beyond initial baseline measurement requires replication of the same data collection and analysis on a monthl...