Introduction and summary
For as long as homework has been a part of school life in the United States, so too has the debate over its value. In 1900, a prominent magazine published an article on the evils of homework titled, “A National Crime at the Feet of Parents.”1 The author, Edward Bok, believed that homework or too much school learning outside the classroom deprived children of critical time to play or participate in other activities at home. The very next year, California, influenced by those concerns, enacted a statewide prohibition on homework for students under the age of 15.2 In 1917, the state lifted the ban, which has often been the case as districts have continually swung back and forth on the issue.3
More than 100 years later, homework remains a contentious issue, and the debate over its value rages on, with scholars coming down on both sides of the argument. Homework skeptic Alfie Kohn has questioned the benefit of homework, arguing that its positive effects are mythical, and in fact, it can disrupt the family dynamic.4 He questions why teachers continue to assign homework given its mixed research base. Taking the opposite view, researchers Robert Marzano and Debra Pickering have voiced their support for purposeful homework that reinforces learning outside of school hours but still leaves time for other activities.5
In 1989, prominent homework scholar Harris Cooper published a meta-analysis of more than 100 studies on homework in a survey that found a correlation between homework and performance on standardized tests, but only for certain grade levels. According to Cooper’s research, for students in late-elementary grades through high-school, there was a link between homework and improved standardized test performance. However, there was no evidence of the same correlation for younger students.6 Even without a connection to academic achievement, Cooper still recommended assigning homework to younger students because it helps “develop good study habits, foster positive attitudes toward school, and communicate to students the idea that learning takes work at home as well as school.”7
Far from academia, parents—not surprisingly—are some of homework’s most ardent supporters and, also, its most vocal critics. For better or worse, many parents help or are involved in their child’s homework in some way. As a result, homework can shape family dynamics and weeknight schedules. If a child receives too much homework, or only busywork, it can cause stress within families and resentment among parents.8 Some parents report spending hours each night helping their children. For instance, a 2013 article in The Atlantic detailed a writer’s attempt to complete his 13-year-old daughter’s homework for a week. The headline simply read: “My Daughter’s Homework Is Killing Me.”9 The father reported falling asleep trying to thoughtfully complete homework, which took around three hours per night.10 On the other hand, some parents appreciate the glimpse into their child’s daily instruction and value homework’s ability to build positive learning habits.
It is no surprise that the debate over homework often spills onto the pages of newspapers and magazines, with calls to abolish homework regularly appearing in the headlines. In 2017, the superintendent of Marion County Public Schools in Florida joined districts in Massachusetts and Vermont in announcing a homework ban. To justify his decision, he used research from the University of Tennessee that showed that homework does not improve student achievement.11 Most recently, in December 2018, The Wall Street Journal published a piece that argued that districts were “Down With Homework”—banning it, placing time caps or limiting it to certain days, or no longer grading it—in order to give students more time to sleep, read, and spend time with family.12
Given the controversy long surrounding the issue of homework, in late spring 2018, the Center for American Progress conducted an online survey investigating the quality of students’ homework. The survey sought to better understand the nature of homework as well as whether the homework assigned was aligned to rigorous academic standards. Based on the best knowledge of the authors, the CAP survey and this report represent the first-ever national study of homework rigor and alignment to the Common Core State Standards—rigorous academic standards developed in a state-led process in 2010, which are currently in place in 41 states and Washington, D.C. The CAP study adds to existing research on homework by focusing on the quality of assignments rather than the overall value of homework of any type. There are previous studies that considered parental involvement and the potential stress on parents related to homework, but the authors believe that this report represents the first national study of parent attitudes toward homework.13
For the CAP study, the authors used the Amazon Mechanical Turk (MTurk) online survey tool to collect from parents their child’s actual homework assignments. Specifically, as part of the survey, the authors asked parents to submit a sample of their child’s most recent math or language arts homework assignment and have the child complete questions to gauge if the assignment was challenging, as well as how long it took to complete the assignment. In all, 372 parents responded to the survey, with CAP analyzing 187 homework assignments.
Admittedly, the methodological approach has limitations. For one, it’s a convenience sample, which means people were not selected randomly; and broadly speaking, the population on the MTurk site is younger and whiter than the U.S. population as a whole. However, research has shown that MTurk yields high-quality, nationally representative results, with data that are at least as reliable as those obtained via traditional methods.14
In addition, the homework sample is not from a single classroom or school over the course of a year; rather, it is a snapshot of homework across many classrooms during the span of a few weeks in May 2018. The assumption is that looking at assignments from many classrooms over a short period of time helps to construct a composite picture of mathematics and language arts homework.
Moreover, the design of the CAP study has clear advantages. Many of the previous existing studies evaluated homework in a single district, whereas the CAP study draws from a national sample, and despite its limitations, the authors believe that the findings are robust and contribute significantly to the existing research on homework.
Three key findings from the CAP survey:
- Homework is largely aligned to the Common Core standards. The authors found that the homework submitted is mostly aligned to Common Core standards content. The alignment index that the authors used evaluated both topic and skill. As previously noted, the analysis is a snapshot of homework and, therefore, does not allow the authors to determine if homework over the course of a year covered all the topics represented in the standards.
- Homework is often focused on low-level skills in the Common Core standards, particularly in the earlier grades. While the authors’ analysis shows that there was significant alignment between Common Core and the topics represented in the homework studied, most of the assignments were fairly rote and often did not require students to demonstrate the full depth of knowledge required of the content standards. There was clear emphasis on procedural knowledge, and an even stronger emphasis on memorization and recall in language arts. Common Core content standards, on the other hand, require students to demonstrate deeper knowledge skills, such as the ability to analyze, conceptualize, or generate.15
- Homework frequently fails to challenge students. Nearly half of the parents who responded to the CAP survey reported that homework is too easy for their child. In particular, parents of primary-grade children were most likely to agree or strongly agree that the homework assignment they submitted was too easy for their child.
Based on these key findings, CAP recommends that states, districts, and schools improve the quality of homework and increase opportunities for students to practice rigorous grade-level content at home. Specifically, the authors—drawing from this survey and other existing research on homework—recommend the following actions to improve the role of homework in education:
- Schools and districts should develop homework policies that emphasize strategic, rigorous homework. In many cases, the homework debate is limited and short-sighted. Currently, many arguments focus on whether or not students should have homework at all, and there are entire school districts that have simply banned homework. Instead of debating the merits of banning homework, reformers and practitioners should focus on improving the rigor and effectiveness of all instructional materials, including
Districts, schools, and teachers should ensure that the total amount of homework students receive does not exceed the 10-minute rule—that is to say, no more than 10 minutes of homework multiplied by the student’s grade level.16 According to research, any more than that can be counterproductive.17 Also, too much homework may be an unnecessary burden on families and parents. Homework should be engaging and aligned to Common Core standards, which allow students to develop deeper-level learning skills—such as analysis or conceptualization—that help them increase retention of content.
- Districts and schools should periodically audit homework to make sure it is challenging and aligned to standards. Rather than implementing homework bans, district policymakers and school principals should regularly review examples of homework assignments to ensure that it is aligned to grade-level standards and requires students to demonstrate conceptual learning. In instances where the district or school finds that homework assignments are not aligned or take too much or too little time to complete, they should help teachers improve homework assignments by recommending instructional materials that may make it easier for them to identify appropriate, grade-level homework assignments.
- Schools and districts should provide access to technology and other supports that can make it easier for students to complete rigorous schoolwork at home. Technology can also provide additional support or scaffolding at home, allowing more students to complete homework without help from adults or older siblings. For instance, programs such as the Khan Academy can give students rigorous homework that’s aligned to Common Core standards.18 Unfortunately, many households across the nation still do not have adequate access to devices or internet at home. Schools and districts should consider options to ensure that all students can benefit from technology and broadband. Greater access to technology can help more students benefit from continual innovation and new tools. While most of these technologies are not yet research-based, and the use of devices may not be appropriate for younger children, incorporating new tools into homework may be a low-cost method to improve the quality of student learning.
- Curriculum reform and instructional redesign should focus on homework. There are many states and districts that are reforming curriculum or adopting different approaches to instruction, including personalized learning. Curriculum reform and personalized learning are tied to greater academic outcomes and an increase in motivation. Homework should also be a focus of these and other efforts; states and districts should consider how textbooks or other instructional materials can provide resources or examples to help teachers assign meaningful homework that will complement regular instruction.
The findings and recommendations of this study are discussed in detail below.
Homework must be rigorous and aligned to content standards
All homework is not created equal. The CAP study sought to evaluate homework quality—specifically, if homework is aligned to rigorous content standards. The authors believe that access to grade-level content at home will increase the positive impact of adopting more rigorous content standards, and they sought to examine if homework is aligned to the topics and skill level in the content standards.
The 10-minute rule
According to Harris Cooper, homework is a valuable tool, but there is such a thing as too much. In 2006, Cooper and his colleagues argued that spending a lot of time on homework can be counterproductive. He believes that research supports the 10-minute rule—that students should be able to complete their homework in no more than 10 minutes multiplied by their grade. For example, this would amount to 20 minutes for a second-grade student, 50 minutes for a fifth-grade student, and so on.19
The Common Core, developed by the National Governors Association and the Council of Chief State School Officers, established a set of benchmarks for “what students should know and be able to do” in math and language arts by the end of the academic year in kindergarten through high school.20 The math standards focus on fewer concepts but in more depth and ask students to develop different approaches to solve similar problems. In language arts, the standards moved students away from narrative-based assignments, instead concentrating on using evidence to build arguments and reading more nonfiction.
The Common Core is not silent in the cognitive demand needed to demonstrate mastery for each standard.21 For example, a second grade math standard is “[s]olve word problems involving dollar bills, quarters, dimes, nickels, and pennies, using $ and ¢ symbols appropriately.” For this standard, a second-grader has not mastered the standard if they are only able to identifying the name and value of every.
Remember, apply, integrate: Levels of cognitive demand or depth of knowledge
There are numerous frameworks to describe levels of cognitive skills. One of the most prominent of these models, Bloom’s taxonomy, identifies six categories of cognition. The original levels and terms were knowledge, comprehension, application, analysis, synthesis, and evaluation; however, these terms have changed slightly over time.22 Learning does not necessarily follow a linear process, and certainly, all levels of cognitive demand are important. Yet these categories require individuals to demonstrate a different level of working knowledge of a topic. With the advent of standards-based reform, the role of cognitive skill—particularly in the area of assessment—has become a much more explicit component of curriculum materials.
Over the past two decades, cognitive science has shown that individuals of any age retain information longer when they demonstrate deeper learning and make their own meaning with the content—using skills such as the abilities to conjecture, generalize, prove, and more—as opposed to only committing ideas to memory or performing rote procedures, using skills such as the ability to memorize or recall.
In essence, Common Core created rigorous expectations to guide the instruction of students in all states that chose to adopt its standards. These standards aimed to increase college preparedness and make students more competitive in the workforce. Policymakers, advocates, and practitioners hoped that Common Core would create greater consistency in academic rigor across states. In addition, with the classroom and homework aligned to these standards, many anticipated that students would graduate from high school prepared for college or career. As of 2017, 41 states and the District of Columbia have adopted and are working to implement the standards, although many of these states have modified them slightly.23
In this study, the authors evaluated homework to determine if it was aligned to Common Core standards in two ways: First, does it reflect grade-level content standards; second, does it require students to use skills similar to those required to demonstrate proficiency in a content area. This multitiered approach is critical to evaluating alignment between standards and instruction—in this case homework. Instruction must teach content and help students develop necessary levels of cognitive skill. Curricula for each grade should include instructional materials that are sequenced and rigorous, thus enabling students to develop an understanding of all content standards.
The study
In spring 2018, the Center for American Progress used Amazon Mechanical Turk (MTurk) to administer a survey. MTurk is a crowdsourcing marketplace managed by Amazon; it allows organizations to virtually administer surveys for a diverse sample.24 The CAP survey asked parents to submit a sample of their child’s most recent math or language arts homework assignment and complete a few questions to gauge if the assignment was challenging, as well as how long it took for the student to complete the assignment. A total of 372 parents responded to the survey, and CAP analyzed 187 homework assignments.
Of the 372 parents who participated in the survey, 202, or about 54 percent of respondents, submitted samples of their child’s homework assignment. The researchers dropped a total of 15 homework submissions from analysis either because the subject matter was not math or language arts—but rather, science, music, or social studies—or because the authors could not examine the specific content, for example, in cases where parents only provided a copy of the cover of a textbook. Of the remaining homework samples submitted, 72 percent (134 samples) focused on mathematics content, while the remaining 28 percent (53 samples) represented language arts content.
Of the 372 responding parents, 234—or 63 percent—were female, and 126—or 37 percent—were male. Forty-eight percent of parents responding to the survey were under the age of 34, while almost 90 percent of respondents were under the age of 45. There was an unequal distribution of parents representing elementary and secondary grade levels. Seventy-one percent of the total sample were parents with students in primary (K-2) and elementary (3-5) grades. (See Methodology section below)
Findings
Based on the analysis, the authors’ drew the following conclusions:
Homework is largely aligned to Common Core standards, especially the topics in the standards
The authors found that the submitted homework, for the most part, was aligned to Common Core standards content or within the so-called “good” range based on content expert evaluations. As described in the Methodology, the authors used an alignment index that does not require a homework assignment to exactly mirror the content standards—both topic and skill level—for evaluators to note that it is within a good range. For context, the study’s alignment index has a range of 0.00 to 1.00, where 0.00 indicates no content in common whatsoever between the two descriptions—perfect misalignment—and 1.00 indicates complete agreement between the two descriptions—perfect alignment. Generally speaking, what one might call “good” alignment for instruction tends to range on the alignment index between 0.4 and 0.6, with a measure of 0.5 serving as a median indicator of good alignment.
The analysis is a snapshot of homework and, therefore, does not allow the authors to determine if homework over the course of a year covered all required standards. In other words, it is difficult to say how many of the standards for a given grade are covered across a full school year, simply because of the limited sample of assignments.
The alignment index evaluates both topic and skill, but there was particular alignment in topic areas. For instance, there was a strong emphasis in the topic areas of number sense and operations for primary math homework. When combined with the third-most emphasized topic, measurement, these three areas accounted for more than 90 percent of primary mathematics homework content. The actual math content standards for the primary grades also placed heavy emphasis on the topic areas of number sense, operations, and measurement—though they accounted for only about 80 percent of primary math content.
In general, across all age groups, math homework was more closely aligned to content standards—both topic and skill level—than language arts. The alignment results for middle school math were particularly strong, at 0.56, based on 27 homework samples. The stronger alignment among math homework samples may be in part due to the fact that there were more math assignments in the sample than language arts assignments. Larger samples offer more opportunities to show alignment. As a result, smaller samples may underestimate alignment.
The table below presents the alignment indices, which were calculated using the homework samples collected for each grade band.
Homework is often focused on low-level skills in the standards, particularly in younger grades
While the authors’ analysis shows that there was significant alignment in the topic of standards and homework assignments, most of the homework did not require students to demonstrate the full depth of knowledge required of content standards. The analysis uncovered an emphasis on procedural knowledge, with an even stronger emphasis on memorization and recall in language arts. Content standards, on the other hand, require students to demonstrate deeper-knowledge skills, such as the ability to analyze, conceptualize, or generate.
Of five performance expectation categories across math and language arts that the authors used to measure alignment between standards and homework, there was a disproportionate emphasis on skills that require a lower level of knowledge or understanding. In grades K-2, for instance, the content standards emphasize the performance expectations of “procedures,” or computation, and “demonstrate,” or understanding, but the homework samples submitted primarily emphasized the procedures level of performance expectation. Similarly, homework for grades three through five focused almost entirely on the performance expectation of procedures, rather than standards that emphasized both procedures and demonstrate.25
As seen with the middle school grades, high school math standards—despite a continued emphasis on procedures—show increased emphasis on the more challenging performance expectations of “demonstrate understanding” and “conjecture, generalize, prove.” Interestingly, this shift toward more challenging performance expectations is most visible for the topic areas of geometric concepts and functions, in both the standards and the homework samples submitted by parents of high school students.
Parents report that homework frequently does not challenge students
Nearly half of parents that participated in the survey reported that homework does not challenge their child. In particular, parents of primary-grade children were most likely to agree or strongly agree that the homework assignment they submitted was too easy for their child—58 percent for language arts and 55 percent for math.
Parents’ opinions about homework difficulty varied between mathematics and language arts assignments. Forty-eight percent of parents who submitted a mathematics assignment and 44 percent of parents who submitted a language arts assignment reported that it was too easy for their child. There was some variance across grade spans as well. As noted above, parents of primary-grade children were most likely to find the homework assignments too easy for their child. Meanwhile, parents that submitted high school math homework were also more likely to agree or strongly agree that the assignments were too easy, with 50 percent agreeing or strongly agreeing and only 33 percent disagreeing or strongly disagreeing with the statement. While there were clear trends in parent opinions, it is important to acknowledge that the sample size for each subset was small.
The comments of surveyed parents echoed this finding. One parent noted that “most homework that they are assigned seems like nothing more than busy work.” Another parent said: “The homework is not strong enough to build conceptual knowledge. It assumes that the child already has that knowledge.” Meanwhile, another parent commented: “Homework is way oversimplified and they don’t seem to spend much time on it. It’s a bit sad that English and math don’t seem to require what they used to. I remember much longer and harder worksheets to complete when I was a child.”26
Weak homework samples
Within the sample of homework assignments, there were some that fell short of rigorous. For instance, one assignment listed 24 pairs of numbers—three and nine, 24 and 21, and so on—and asked the student to circle the smaller number in order to build numbers sense. While homework can be critical when establishing foundational knowledge, repetitive activities such as this often fail to engage students and, instead, overemphasize rote learning. Asking a student to list or name a number of a lesser value, for instance, would make this assignment more interactive.
A second example from kindergarten asked a student to create an uppercase and lowercase letter “f” by filling in dots with paint. The parent who submitted it highlighted the limited utility of the assignment, emphasizing that it does not hold students to high expectations. What’s more, the homework only gave the student two opportunities to practice writing the letter, both in a nonauthentic way. Indeed, the assignment focused more on filling in circles than it did constructing letters. While this task might help build a kindergartener’s hand-eye coordination, it does little to support language arts.
Exemplary homework samples
While many of the assignments submitted focused on procedures and, for math, computation, it is worth acknowledging some of the more exemplary types of homework included in the samples. These offer examples of how homework can challenge students, engage rigorous cognitive processes, and demonstrate that content standards at all levels—not just middle and high school—can support challenging homework that pushes students to think critically.
For example, one math homework assignment asked a student to identify which individuals possessed each of four groups of shapes based on the following description:
Ally, Bob, Carl, and Dana each have a set of shapes.
- Bob has no triangles.
- The number of rectangles that Dana has is the same as the number of triangles that Carl has.
This example is interesting on two counts. First, the assignment goes beyond procedure, requiring the student to analyze the various sets of shapes in order to determine which set belongs to which individual. It is also interesting insofar as it demonstrates a common real-world situation: There is usually more than one way to solve a problem, and sometimes, there is more than one correct answer.
Similarly, another example asked a student to determine actions that would help students beautify the school. The header of the assignment read, “Make a Decision: Keep Our School Beautiful!” The assignment had various boxes, each with a question above, such as, “Should we recycle?” or “Should we make art?” The assignment asked the student to “(1) think about each choice, (2) consider how each choice would affect them and others in the school community, (3) write their ideas in boxes below.” In doing so, it required primary students to analyze and generate ideas—both of which are skills that promote deeper learning.
Recommendations
Homework offers a valuable window into the curricula, assessment practices, and instructional preferences of teachers. It provides insight into classroom learning as well as the types of knowledge and skills the teacher believes will reinforce that instruction at home.
This analysis shows that the content and value of homework varies. While most homework within the sample was aligned to content standards, there is still a significant need to increase the rigor of homework and create opportunities for students to use higher-order skills.
Overall, schools and districts should pay more attention to homework as a reform lever. A growing body of research shows that homework is connected to learning outcomes, and as a result, schools and districts should ensure that policies help teachers provide meaningful assignments.27 Based on this survey and the existing research on homework quality, the authors identified recommendations that can help increase the quality of homework:
Schools and districts should develop homework policies that emphasize strategic, rigorous homework
In many cases, the current debate over homework is short-sighted. Many arguments focus on whether or not students should have homework. There are entire school districts that have simply banned homework altogether. However, the debate should move beyond the merit of homework. Research shows that homework is linked to better performance on standardized assessments, especially in higher grades.28 Many homework scholars also believe that a reasonable homework load can help develop important work habits.29 Therefore, instead of eliminating homework outright, schools, districts, and advocates should focus on improving its rigor and effectiveness. As discussed throughout, homework should be an extension of instruction during the school day. Accordingly, policymakers and schools must make changes to homework that are in concert with curriculum reform.
Like all instruction, homework should be aligned to states’ rigorous content standards and should engage students in order to promote deeper learning and retention. To do this, homework should ask students to use higher-order skills, such as the ability to analyze or evaluate.
However, schools and districts, rather than simply assigning longer, more complicated assignments to make homework seem more challenging, should make strategic shifts. Homework assignments should be thought-provoking. But there is a such thing as too much homework. Districts and schools should ensure that teachers follow the research-supported 10-minute rule.30 Also, teachers, schools, and districts should consider resources to set all students up for success when faced with more rigorous home assignments; homework should never be a burden or source of stress for families and parents.
Districts and schools should audit homework to make sure it is challenging and aligned to standards
Rather than implementing homework bans, district policymakers and schools should regularly review homework samples to ensure that they are aligned to grade-level standards, are engaging, require students to demonstrate higher-order skills, and adhere to the 10-minute rule. The audit should review multiple homework assignments from each classroom and consider how much time children are receiving from all subject areas, when appropriate. The district or school should ask for ongoing feedback from students, parents, and guardians in order to collect a comprehensive representation of the learning experience at home.
In instances where the district or school principal finds that homework assignments are not aligned to grade-level standards or take too much or too little time to complete, they should help the school or teachers improve them by recommending instructional materials that may make it easier for teachers to identify appropriate, grade-level homework assignments. In addition, if parents or students identify challenges to complete assignments at home, the district or school should identify solutions to ensure that all students have access to the resources and support they need to complete homework.
Schools and districts should provide access to technology and other supports that make it easier for students to complete homework
Technology can go a long way to improve homework and provide additional support or scaffolding at home. For instance, programs such as the Khan Academy—which provides short lessons through YouTube videos and practice exercises—can give students rigorous homework that is aligned to the Common Core standards. Unfortunately, many households across the nation still do not have adequate access to devices or internet at home. A 2017 ACT survey found that 14 percent of students only have access to one technology device at home.31 Moreover, federal data from 2013 found that about 40 percent of households with school-age children do not have access to broadband.32 It is likely that the percentage has decreased with time, but internet access remains a significant problem.
Schools and districts should adopt programs to ensure that all students can benefit from technology and broadband. For instance, Salton City, California, installed a Wi-Fi router in a school bus. Every night, the bus parks near a neighborhood with low internet connectivity, serving as a hot spot for students.33
Moreover, greater access to technology can help more students benefit from new innovative resources. While most of these technologies are not yet research-based, and the use of devices may not be appropriate for younger children, incorporating new tools into homework may be a low-cost option to improve the quality of student learning. For instance, ASSISTments is a free web-based tool that provides immediate feedback as students complete homework or classwork. It has been proven to raise student outcomes.34 Other online resources can complement classroom learning as well. There are various organizations that offer students free lessons in the form of YouTube videos, while also providing supplementary practice exercises and materials for educators. LearnZillion, for example, provides its users with high-quality lessons that are aligned to the Common Core standards.35
Curriculum reform and instruction design should focus on homework
There are many states and districts that are engaging in curriculum reform. Many of these recent reform efforts show promise. In an analysis of the curricula and instructional materials used by the nation’s 30 largest school districts, the Center for American Progress found that approximately one-third of materials adopted or recommended by these districts were highly rated and met expectations for alignment.36
Homework should be a focus of curriculum reform, and states and districts should consider how textbooks or other instructional materials can provide resources or examples to help teachers assign meaningful homework that will complement regular classroom instruction.
Personalized learning—which tailors instruction and learning environments to meet each student’s individual interests and needs—is also gaining traction as a way to increase declining engagement in schools and increase student motivation.37 These ideas are also relevant to homework quality. A 2010 study found that when students were offered a choice of homework assignment, they were more motivated to do the work, reported greater competence in the assignments, and performed better on unit tests, compared with peers that did not have choice in homework.38 The study also suggested that offering students a choice improved the rate of completion of assignments.39 Districts and schools should help implement more student-centered approaches to all instruction—in the classroom and at home.
Conclusion
When it comes to change management, experts often advise to look for low-hanging fruit—the simplest and easiest fixes.40 In education, homework reform is low-hanging fruit. Research shows that quality homework and increasing student achievement are positively correlated; and yet, the authors’ analysis shows that some schools may not be taking advantage of a valuable opportunity to support student achievement. Instead of mirroring the cognitive demand in rigorous content standards, homework assigned to students is often weak or rote. But it does not have to be this way. More rigorous, insightful homework is out there. Policymakers and schools need to move beyond the debate of whether or not to assign work outside of school hours and do their own due diligence—or, put another way, their own homework—before assigning homework to students in this nation’s schools.
Methodology
As mentioned above, the authors used the Amazon Mechanical Turk (MTurk) online survey tool to collect from parents their child’s actual homework assignments. Specifically, as part of the survey, the authors asked parents to submit a sample of their child’s most recent math or language arts homework assignment and have the child complete questions to gauge if the assignment was challenging, as well as how long it took to complete the assignment. In all, 372 parents responded to the survey, with CAP analyzing 187 homework assignments. The submissions of samples were analyzed by a group of analysts under the supervision of John L. Smithson, researcher emeritus at the University of Wisconsin-Madison.
Measuring alignment
The homework samples were reviewed by two teams of content analysts—one for mathematics and one for language arts—who were asked to describe the academic content represented by the submitted homework, as well as the performance expectation. Each team consisted of three analysts who possessed the relevant content expertise and experience in methodology used to gather the descriptive data.
The teams used a taxonomy-based methodology that was developed by education researchers Andrew Porter and John Smithson during Porter’s tenure as director of the Wisconsin Center for Education Research at the University of Wisconsin-Madison.41 Researchers both nationally and internationally have subsequently used this approach to content description for decades in order to examine issues of alignment as well as to support program evaluation and inform school improvement efforts.
The U.S. Department of Education also recognizes the validity of this approach. Specifically, the Education Department completes a peer review of states’ annual assessment program’s alignment to state academic content standards.42 The Porter/Smithson approach is one of a handful of alignment methodologies that has been determined to meet these federal requirements.43
The Porter/Smithson approach is unique because it defines instructional content as a two-dimensional construct consisting of topic and cognitive demand, or skill. This approach to describing cognitive skill is similar to Bloom’s, which the authors have described above. It has five categories: recall, process, analyze, integrate, and conceptual understanding. The Porter/Smithson approach is the most stringent of alignment indicators, as it looks at both topic and cognitive demand; it is also possibly the most challenging to interpret because the final alignment score considers two dimensions.
The alignment index has a range of 0.00 to 1.00, where 0.00 indicates no content in common whatsoever between the two descriptions—perfect misalignment—and 1.00 indicates complete agreement between the two descriptions—perfect alignment. A measure of 1.00 is exceedingly unlikely, requiring perfect agreement across every cell that makes up the content description. In practice, this is only seen when comparing a document to itself. For instance, very high alignment measures—more than 0.70—have been noted when comparing different test forms used for a particular grade-level state assessment; but those are instances where high alignment is desired. In terms of instructional alignment—in other words, how well instruction is aligned to the standards—a measure of 1.00 is not the goal. For this reason, the authors did not expect any analysis of homework alignment, no matter how well designed, to have a measure of or close to 1.00.
Generally speaking, what one might call “good” alignment for instruction tends to range between 0.4 and 0.6 on the alignment index, with a measure of 0.5 serving as a median indicator of good alignment. The description of the content standards represents the goal of instructional practice—the destination, not the journey. As such, it does not indicate the best path for achieving those goals. The 0.5 indicator measure represents a middle road where teachers are balancing the expectations of the content standards with the immediate learning needs of their students.
Limitations
The authors acknowledge that the analysis has shortcomings. The sample was relatively small and does not directly mirror the national population of parents of elementary and secondary school students. As such, the sample does not necessarily reflect the views or homework experiences of the larger U.S. population.
Limited sample size
The current study analyzes a snapshot of homework across many classrooms, rather than homework from a single classroom or school. The assumption is that looking at individual homework assignments across many classrooms will help to construct a composite picture of mathematics and language arts homework that will be somewhat reflective of the picture one would get from following many classrooms for many days. If the sample is large enough with a wide enough geographical spread, that assumption serves researchers well enough.
For the current study, however, the number of homework samples available for each grade band were, in some cases, quite small—as low as five assignments each for middle and high school language arts. The largest sample sizes were for primary and elementary math, with 47 and 41 homework assignments collected, respectively. However, even 47 is a fairly small sample size for drawing inferences about a full year of homework.
Selection bias
The respondents that participated in this study were a reasonably diverse group in terms of age, gender, and ethnicity, but there are notable differences between the makeup of the parents represented in the study and the makeup of parents of school-age children more generally. Respondents were predominantly female, with women making up almost two-thirds—63 percent—of the sample. They also tended to be parents of younger school-age children, with 71 percent of the respondents reporting on children from the bottom half of the K-12 system—grades K-5. Finally, in terms of race and ethnicity, the sample overrepresented Asian American families and underrepresented African American families. These groups comprised 14 percent and 8 percent of respondents, respectively, compared with national averages of 6 percent and 12 percent.
Because the sample does not well reflect the population of parents of elementary and secondary students, the authors considered possible selection biases that may help to explain the differences in sample and overall population and that may have affected certain members of the population more than others.
For instance, the authors administered the survey using MTurk, which may have skewed the sample. In general, the population on the site is younger and whiter than the U.S. population as a whole. However, research has shown that MTurk yields high-quality, nationally representative results, with data that are at least as reliable as those obtained via traditional methods.44 The researchers also targeted California and Texas in order to increase the diversity of the sample.
In addition, accessibility could have led to selection bias. Despite broad internet access in 2018, there remain families in low-income locales where internet access is not readily available for parents. It is also possible that older parents are less likely to be as active on the internet as younger parents, further contributing to selection bias.
About the authors
Ulrich Boser is a senior fellow at the Center for American Progress. He is also the founder and CEO of The Learning Agency.
Meg Benner is a senior consultant at the Center.
John Smithson is the researcher emeritus at the University of Wisconsin-Madison.
Acknowledgements
The authors would like to thank Sarah Shapiro, a former research assistant at the Center for American Progress, for her support developing the survey. They also appreciate the valuable feedback of Catherine Brown, senior fellow for Education Policy at the Center for American Progress; Tom Loveless, senior fellow at the Brookings Institution; Lisette Partelow, director of K-12 Special Initiatives at the Center; and Scott Sargrad, vice president of K-12 Education Policy at the Center.
Conflicts of interest
The author, Ulrich Boser, has a financial relationship with the creators of the online homework tool ASSISTments.