Read the full report (PDF)
Read "The CitiStat Model: How Data-Driven Government Can Increase Efficiency and Effectiveness" (PDF)
In 1966, fatalities from automobile accidents topped 50,000 after decades of steady increases. That same year, Congress enacted the Highway Safety Act and directed the creation of an information system “to determine the probable causes of accidents, injuries and deaths”—making data collection, analysis and dissemination a central component of auto-safety efforts. This action laid the groundwork for smarter decisionmaking.
In carrying out Congress’s directive, the National Highway Traffic Safety Administration, NHTSA, soon developed a rich inventory of data, allowing the agency to isolate causes of accidents that result in fatalities and injuries and compare the performance of policy approaches from state to state. These data are now regularly organized into quantitative tables—so that problems are easy to spot—and posted to the agency’s Website. These tables invite states to learn from each other and help NHTSA target areas for safety improvements.
Under this approach, the trend of ever-increasing traffic fatalities has been reversed. The accident fatality rate has steadily declined over the last three decades, and today it stands at an all-time low. This is data-driven policymaking at work. Its key features include:
- Collection and analysis of data to spotlight problem areas and potential solutions;
- Development of quantifiable measures to assess policy performance and draw comparisons across similar circumstances or peer groups so that “best practices” can be identified and expanded; and
- Public dissemination of data and metrics on policy results, so those outside government can hold public officials accountable for their performance.
Unfortunately, across most federal policy areas, this model cannot be fully and confidently applied because of significant gaps in data. In education, for example, we lack basic classroom data that could be used to deploy highly effective teachers where they are needed most. In health care, we are unable to systematically draw comparisons across providers to identify and expand the most effective treatments. And in the environmental arena, basic data on air and water pollution as well as chemical exposures are often unavailable, impairing our ability to prevent public harm.
In a paper-based world, this sort of information was virtually impossible to generate. The costs and administrative burden associated with data collection and analysis were simply too steep. Today, however, these costs are falling dramatically due to advances in information technologies. Data are now far easier and cheaper to gather, store, analyze and disseminate. Moving information from one place to another is instantaneous and virtually free. These advances make possible a whole series of monitoring opportunities, data exchanges, comparisons, and analytical inquiries that would have been impossible even a few years ago.
Leading corporations such as General Electric Co., Marriott International Inc., and Capital One Services Inc. have seized on new technologies to transform the way they make decisions. Data and metrics are used to manage inventories, assess and improve product quality, measure the success of marketing strategies, set optimal prices, and identify priorities for capital allocation.
A similar revolution in government is waiting to be unleashed. Indeed, a number of pioneering local and state governments are showing the way. The city of Charlotte, N.C., for example, has developed metrics to identify and target fragile neighborhoods for revitalization. In Baltimore, the mayor’s office continuously monitors performance data from city departments to improve service delivery and achieve budget savings. At the state level, Washington has developed a data-driven system for priority setting and results-focused budgeting, while Virginia and Iowa set ambitious, quantifiable goals to hold state officials accountable for results.
The missing ingredients at the federal level are political commitment, funding, and a coherent strategy for moving forward. There are three broad areas that must be addressed to build a more data-driven and empirical approach to governance.
First, we need to close critical gaps in our knowledge by harnessing new technology and investing in data collection, analysis and dissemination. In the absence of robust, high-quality data, life and death problems may go undetected, cause and effect correlations may be missed, and comparisons from place to place may be misleading.
Second, we need to focus on results by setting quantitative, outcome-focused goals, rigorously measuring policy performance, and comparing results among peers, in particular state and local governments. As it currently stands, goal-setting is frequently focused on tasks rather than results, while performance measurement suffers from political manipulation and a lack of meaningful data, impairing comparative analysis.
Third, we need to develop systems to ensure that data are used to guide policy priorities and solutions. Even if we had all the necessary data, we would still lack the expertise, decisionmaking processes, and commitment from top leadership (including the president and Congress) to adopt data-driven decisionmaking.
Taking these steps will require significant effort and investment, but the payoff is potentially enormous. Debates over policy frequently get hung up on problem assessment. If we are unsure of the facts, we don’t move on to solutions. In the meantime, the public is left to suffer the consequences—children stranded in failing schools; communities exposed to cancer-causing chemicals; patients denied life-saving treatments.
Robust data collection and analysis promises to illuminate problems and reduce uncertainty by revealing severity, geographic concentration, trends, and causation. This knowledge, publicly disseminated, can sharpen debate over policy choices, facilitate political consensus, and provide leverage over entrenched special interests that may stand in the way of addressing a particular problem.
Having a clear picture of our problems also enhances our policy options. Policymaking can become more nimble, able to quickly adjust to changing circumstances, more tailored, so that responses fit divergent needs, and more experimental, testing how problems respond to different strategies.
These benefits can only be realized, however, if we recognize and avoid the potential downsides of data-driven decisionmaking. Wrong conclusions and policy decisions may emerge in the absence of thoughtful analysis—especially when critical issues or determinants of results are inescapably difficult to measure quantitatively. Analysis will always be necessary to interpret available data, take account of factors that may not be reflected in the numbers, and clarify underlying assumptions.
In addition, performance measurement, if oversimplified or misdirected, can create warped perceptions and distorted incentives. Without proper “checks and balances,” those being measured can “game” the numbers in ways that undermine policy objectives. Hospitals evaluated solely on death rates, for example, may choose not to treat the sickest patients with the greatest chance of dying. Metrics need to be carefully selected to ensure that they present an accurate picture of key issues and promote desired outcomes.
Finally, data might be used in ways that intrude on personal information, such as medical records, or to support inappropriate policies, such as racial profiling. Protections are therefore needed to ensure data are not misused.
This paper provides a starting point for thinking about data-driven decisionmaking as a new approach to governing. It describes the major elements that need to be implemented at the federal level, explains the potential advantages of this approach, and points out possible downsides that must be overcome. We give particular attention to education, health care and the environment for illustrative purposes. However, data-driven decisionmaking can be applied to meet the full range of challenges facing the country, from homeland security to food safety to energy alternatives to financial fraud. At its heart, this proposal is about building an effective, efficient government that is responsive to the needs of its people.
For more information: