Whether it’s a mom-and-pop coffee shop, a Fortune 500 firm, or a health care nonprofit, well-run organizations employ technology as a way to improve their performance. These businesses and organizations think of digital technology as part of larger efforts to boost productivity and improve outcomes. For American companies, leveraging digital solutions has long been a way of doing business, and over the past sixty years, the approach has resulted in average worker productivity climbing by more than 2 percent a year due in large measure to improvements in equipment, computers, and other high-tech solutions.
Educators, however, generally do not take this approach to technology. Far too often, school leaders fail to consider how technology might dramatically improve teaching and learning, and schools frequently acquire digital devices without discrete learning goals and ultimately use these devices in ways that fail to adequately serve students, schools, or taxpayers.
Because of a growing debate concerning spending on education technology, CAP decided to look closely at the issue of how students used technology and the return that educators were getting on their technology investment. In conducting this examination, we relied on one of the richest sources of national student survey data—the National Assessment of Educational Progress, or NAEP—and conducted an analysis of the 2009 and 2011 background surveys. Known as the Nation’s Report Card, the NAEP assessments are administered every two years by the U.S. Department of Education’s National Center for Education Statistics, and the exams serve as a way to benchmark student performance. In addition, we conducted a state-by-state survey of the websites of state departments of education during the first two weeks of February 2013 to see if states had conducted any evaluation of the return on their school-technology investment.
Among our findings are the following:
- Students often use technology for basic skills.We found, for instance, that more than a third of middle school math students regularly used a computer for drill and practice. In contrast, only 24 percent of middle school students regularly used spreadsheets—a computer application for data analysis—for their math assignments, and just 17 percent regularly used statistical programs in math class. These data varied widely across the nation. In Louisiana almost 50 percent of middle school math students said that they regularly used a computer for drill and practice. In Oregon that figure was just 25 percent. (See appendix for detailed state-by-state figures.)In high schools we found a similar trend of students using technology for lower-order thinking skills and knowledge acquisition. Our analysis showed that 73 percent of students, for example, reported regularly watching a movie or video in science class. By contrast, far fewer students used computers in their science classes—just 66 percent of students reported regularly using a computer in science class.At the same time, high school students were not getting the hands-on STEM—science, technology, engineering, and mathematics—experiences that they need to succeed. Just 39 percent of high school students indicated that they had hands-on experience with simple machines in their science classes over the past year. And just a third of high school students said that they did hands-on projects with electricity over the past year.
- States are not looking at what sort of outcomes they are getting for their technology spending. In 2008, the Center for American Progress, the U.S. Chamber of Commerce, and Frederick M. Hess of American Enterprise Institute did a systematic survey of state department of education websites of all 50 states and the District of Columbia, and in that report titled “Leaders and Laggards,” we found no evidence that any state had conducted a large-scale technology return-on-investment, or ROI, study. This year we re-conducted that study, and again we found that no state collected data on technology ROI. It appears that states instead collect data only on the presence of technology such as the number of schools with high-speed Internet access.
- Students from disadvantaged backgrounds are less likely to have access to more rigorous STEM-learning opportunities.We found that students from high-poverty backgrounds were far less likely to have rigorous learning opportunities when it comes to technology. Forty-one percent of eighth-grade math students from high-poverty backgrounds, for instance, regularly used computers for drill and practice. In contrast, just 29 percent of middle school students from wealthier backgrounds used the computers for the same purpose. We also found that black students were more than 20 percentage points more likely to use computers for drill and practice than white students.We found similar issues at the high school level here as well. We further noted racial disparities when it comes to computer use. Sixty-eight percent of white students regularly used computers for science class, compared to sixty percent of Hispanic students. Students of color were also less likely to have access to hands-on science projects, and just 37 percent of black students had experienced hands-on activities with simple machines in their science class over the past year. In contrast, 40 percent of white students and 45 percent of Asian students reported having such experiences.
Technology’s potential
There is no question that technology holds significant potential when it comes to improving classroom practices and encouraging more effective learning. Computers, tablets, and other devices can help boost the reach of highly effective teachers, allowing more students to study with the best math and reading teachers, for instance. Several schools have successfully experimented with such reforms, and in various forms, the schools will allow highly effective teachers to focus less on administrative duties and more on teaching. Under this approach, schools will often use support staff to take over noninstructional activities for highly effective teachers such as their lunch and recess duties, while more effective teachers take on responsibility for more students.
Technology can also create greater personalization of educational material. We know, of course, that students vary as learners. But for the most part, schools basically treat all students the same, and far too few teachers personalize their teaching to individual students and their particular needs and skills. But well-built computer programs easily do this through personalized interaction. Differentiation is, in fact, built into almost every video game.
Technology can also improve testing by making it less expensive, as well as more adaptive. Moreover, the tools of the digital age also have the potential to dramatically improve the quality and scale of back-end services such as student-record management. In the end, it is clear that when strategically implemented, school technology could become what Harvard Business School Professor Clay Christensen calls a “disruptive technology”—a simple concept that creates widespread innovation. Such innovations can create new markets and opportunities.
For the most part, policymakers—and the private sector—have recognized the potential of school technology. One recent analysis found that venture capital has been flowing into K-12 education due in part to technological advances. GSV Advisors, a Chicago consulting firm, estimates that investment money to both public and private schools jumped more than 150 percent from 2010 to 2012 and now stands at around $334 million. States and the federal government have also been very focused on putting computers and other technologies into schools, and today federal agencies spend more than $3 trillion on STEM investments annually.
In recent years, however, there has been a growing concern around technology spending with a chorus of experts arguing that school technology is not helping students achieve learning goals in the most effective ways. Education observer Rick Hess has written about this issue thoughtfully, arguing that absent in schools is “serious thought about how technology might help cut costs or modernize educational delivery.” In light of these concerns and our previous work in this area for the Leaders and Laggards report, we decided to take a closer look at the issue of technology use and other related issues. In this analysis, it quickly became clear to us that many schools and districts have not taken full advantage of the ways that technology can be used to dramatically improve education-delivery systems.
Across the nation, we found that many schools were using technology in the same way that they have always used technology; students are using drill and practice programs to hone basic skills. Students are passively watching videos and DVDs. Too many students do not have access to hands-on science projects. In short, there is little indication that technology has revolutionized our nation’s school system.
In many ways, this is an old problem. As education scholar Larry Cuban has noted, schools have long been deeply resistant to change. Part of the problem is that schools and districts often see technology as something to add to their current approach rather than something that might change their current approach. In other words, schools are not using technology to do things differently.
This is partly an issue of education culture where most schools do not operate in a performance-based environment and lack the incentives and support necessary to try new things. As we’ve noted in previous reports on productivity, this problem manifests itself in both big and small ways. There are the examples of straightforward waste such as overpaying for technology services by not considering outside vendors. The bigger issue, however, is that educators do not have the tools and incentives needed to connect spending to outcomes and reorganize programs in ways that take full advantage of school technology. Many states, for instance, use seat-time requirements for classes—time spent sitting at a desk listening to a teacher—to determine whether a student is ready to graduate from high school. While such a requirement may have made sense at one time, it divorces inputs from outcomes and prevents educators from trying more productive, technology-facilitated ways of ensuring that all students are college and career ready.
Local school leaders also have little flexibility to spend education dollars in ways that they believe will best utilize school technology. In many areas, the district superintendent largely manages each school’s budget and building principals do not have the final say on spending decisions. What’s more, teacher salaries are also often set at the state level, leaving little room for educators at the local level to experiment with new ways to spend their money and deliver education through online services or other technology-enhanced programs.
Capacity is another issue and school leaders have to realize that education technology needs a great deal of support if it is going to be done well. When technology expert Lee Wilson recently conducted an analysis of the cost of deploying iPads, he found that a school might spend 552 percent more to implement iPad textbooks. Wilson is no Luddite and, as a matter of fact, used to work for Apple and Pearson, one of the world’s largest education publishers, before starting his own firm.
For Wilson the crucial takeaway from his study is that it is easy to underestimate the additional necessary costs of implementing new technologies in ways that will ultimately improve student achievement. As he writes, “[w]e are likely to hear lots of bleating about engagement and how much the kids love to work with these devices. To which educators should respond with—‘great – where is the objective data on improved outcomes?’”
Technology’s tough questions
There are no easy answers when it comes to how best to deploy technology within schools, especially when financial matters are added to the mix. Consider, for instance, the mastering of basic algebra. Drill and repetition is certainly key, and as experts such as psychologist Daniel Willingham have made clear, students need to have mastered the basics in order to engage in higher-level math thinking. The question then is: What is the most effective way to help students accomplish this? Is it a paper-and-pencil worksheet? Does it take the latest Macbook Air computer? Or is an iPhone with an appropriate app sufficient?
There are no simple answers. Coming to the right conclusions involves understanding the costs and the outcomes of the educational process, as well as having a sense of how we can deliver the same product better. There also has to be an awareness of the ways in which computers are changing the employment landscape and increasing demand for jobs that computers cannot execute. Economist Richard Murnane argues, for instance, that schools need to do a better job of providing students with “expert thinking”—the ability to solve new problems that cannot be solved by rules.
Still, we were deeply surprised to find that no state is looking at technology return on investment, or ROI, given the hundreds of millions of dollars that are currently being spent on technical devices across the nation. There were a few bright spots, to be sure, and some states have at least looked at whether or not technology has improved student learning. In Kansas, for instance, the state sought to determine if its technology-rich classroom environment made a difference in student engagement and teacher pedagogy. The report found some evidence that the use of technology in classrooms helps provide students with more opportunities to work together on projects. The study also found that technology may promote higher-level thinking. But the report in Kansas—and other states—failed to look at the costs associated with those learning outcomes.
The growing digital divide
What is also clear from our analysis is that our schools do not do nearly enough to support students from disadvantaged backgrounds, particularly when it comes to technology use—a disturbing trend that is widening the digital divide between poor students and their financially better-off peers. According to our findings, for instance, low-income students and those of color were far more likely to be using drill and practice computer programs.
In a way these findings are not surprising. We know that students of color and students in high-poverty schools are allocated less money per student, and they are far less likely to be taught by effective teachers. These factors all contribute to the nation’s large achievement gap where, on average, black and Latino students are academically about two years behind white students of the same age.
We are certainly not arguing for the nation to stop or slow funding for education technology. It is imperative that students graduate from high school knowing how to effectively use technology. At minimum, high school graduates should have the skills to create a spreadsheet and calculate simple formulas such as averages and percentages. Equally crucial is the need to increase access to technology for all students, particularly ones from disadvantaged backgrounds. The federal and state governments should work to ensure access to broadband and other technologies for all communities, particularly poor communities and those of color.
Fulfilling the promise
Technology is clearly fulfilling some of its promises. Virtual schools, for example, are offering students more course and curriculum options than conventional schools. Many virtual schools also appear to serve students relatively well. When the U.S. Department of Education conducted a detailed review of virtual education studies of both K-12 and higher education efforts, they found that students in online education actually performed slightly better than students who received face-to-face education. As the Department of Education report concluded, “[t]he meta-analysis found that, on average, students in online-learning conditions performed modestly better than those receiving face-to-face instruction.” But the report also cautioned that the increased achievement that is “associated with blended learning should not be attributed to the media, per se,” because of methodological issues.
But at the same time, it is clear that we are not approaching technology with an eye toward improving educational delivery. In too many schools computers appear to be an add-on rather than a true lever for change. As policymakers and other stakeholders invest billions of dollars in school technology each year, we should be asking ourselves: Are these investments the best use of our limited dollars? Is technology allowing us to do things that we do not—or cannot—already do? How are we ensuring that students have the skills that they need to succeed?
We do not necessarily have the answers to these questions, and in many ways they might be best addressed at the local level. But what is clear is that this is part of a larger problem in which our education system does not do enough to value outcomes, and for too long our nation’s school system has failed to ensure that education funding consistently promotes strong student achievement. This explains, for instance, why after adjusting for inflation, education spending per student has nearly tripled over the past four decades. But while some states and districts have spent their additional dollars wisely—and thus shown significant increases in student outcomes—overall student achievement has largely remained flat.
In the end, we almost certainly will not solve these technological issues through broad mandates alone. We instead believe that education policymakers should create performance-focused management systems that are flexible on inputs and strict on results. Successful organizations reward success, encourage innovation, and ensure the efficient use of funds. With a sharper focus on inputs such as technology and on outputs such as student achievement, we believe that schools will find the best bang for their education buck.
Recommendations
The questions posed above and the accompanying analysis lead to the following recommendations:
- Policymakers must do more to make sure that technology promotes key learning goals. Education technology should do more than simply replace low-cost alternatives. It should instead give teachers and schools new ways of reaching students and delivering education. This starts with a management environment that rewards new approaches. But at the same time, we need to ensure that schools have the capacity to put digital tools in the classroom in ways that raise the bar for all students regardless of their background.
- Schools must address the digital divide. The digital divide used to be between the students who had access to computers and those who did not. But times have changed, and while access remains a problem in many schools, technology use should be a far bigger concern to reformers. The data here are clear. In many schools around the country, students from disadvantaged backgrounds are being given the least engaging, least promising technology-facilitated learning opportunities.
- Advocates must push for studies of the cost-effectiveness of technology. Are taxpayers getting their money’s worth when it comes to technology in schools? We simply do not know the answer to this basic question right now. Study after study shows that technology in education can raise student outcomes under certain conditions. The question now is how we can bring those outcomes to scale and at what cost. Education leaders could be doing far more in this area, including close and careful studies of technology’s return on investment.
Conclusion
It is easy to forget that technology is a tool. It is a way of accomplishing something by using a technical process or knowledge. As a consequence technology has tradeoffs. Given the task, there might be a better tool. Given the circumstance, there might be a better option. This notion sounds so basic and obvious that it seems sort of silly. But it is easy for this concept to become obscured among the glitz and high-tech bells and whistles of today’s information age. In a world filled with technology, we need to continually ask ourselves: What is the goal? Are we using a screwdriver when a hammer would work better? Is the technology working for us, or are we working for our technologies?
This problem looms particularly large in education, and we have not done enough to consider new models of educational delivery nor have we thought enough about ways to create a performance-based culture. This needs to change. Technology can kickstart the process of leveraging new reforms and learning strategies, and we hope that this report serves as a much-needed wake-up call.
Acknowledgements
The analysis in this report grew out of “Leaders and Laggards: A State-by-State Report Card on Educational Innovation” a joint project of the Center for American Progress, the U.S. Chamber of Commerce, and Frederick M. Hess of the American Enterprise Institute. The author also wishes to acknowledge the invaluable research assistance of Scott O’Halloran. Cindy Brown, Raegen Miller, Lindsay Rosenthal also all provided helpful feedback and assistance.
Ulrich Boser is a Senior Fellow at the Center for American Progress.