Download this report (pdf)
Download the introduction and summary (pdf)
Read the full report in your web browser (Scribd)
See also: Teaching Children Well by Robert C. Pianta
A central part of education reform today is the wide-ranging and unprecedented effort to either revamp existing teacher evaluation systems or develop and implement entirely new systems. In the past three years, for example, 32 states and the District of Columbia have made some change to their state teacher evaluation policy, and 23 states currently require that teacher evaluations include objective evidence of student learning, up from only four states in 2009. The success of this work will in large part be judged by the extent to which the resulting systems can evaluate teachers with rigor, objectivity, and in ways that differentiate teachers’ abilities to promote student learning.
Meeting this high bar in our nation’s high schools poses especially difficult challenges, and yet the stakes for doing so are enormous, a point brought home by the extant research. One particular strand of research focuses attention to the importance of identifying and addressing teacher effectiveness within schools, where the bulk of the variation in teacher effectiveness resides. At the same time, research indicates a clear and urgent need to accomplish this task in our nation’s high schools.
The argument for focusing attention at the high school level is three-pronged. First, the performance of high school students lags behind that of demographically similar students in the elementary and middle grades, which suggests that, relative to the earlier years, there is a heightened need for improving the quality of instruction in high school. Second, dropout decisions are made by students in their high school years, which means improving average teacher quality in high school is one potential avenue for addressing the stubbornly persistent dropout rate. The research-based linkage is that student engagement is related to dropping out and teachers’ behaviors and practices are, in turn, related to student engagement.
Third, high school is our last line of defense for preparing students for college and the world of work—and teachers are an obviously critical component of the quality of that preparation. Students entering college lacking a solid high school education often have to spend time in remedial college courses, a sidetrack associated with fewer earned credits and a lower likelihood of graduating with a degree. In terms of labor market consequences, young people who enter today’s labor force without basic academic skills, the ability to think critically and creatively, and who are deficient in so-called “noncognitive” skills are at a competitive disadvantage in the global, information-age economy.
High-quality teacher evaluation systems are seen as one lever for improving the teacher workforce and hence the outcomes of students, including high school students. The current degree of consensus around efforts to improve teacher evaluation is striking for the world of kindergarten-through-12th-grade education. From traditionally conservative education observers and activists to teachers’ union leaders, from professional education organizations to philanthropic foundations, and from the U.S. Department of Education to local education agencies, a wide array of individuals, groups, and organizations are involved and often cooperating in efforts to support the design, testing, and implementation of the next generation of teacher evaluation systems.
On the public side the U.S. Department of Education made teacher evaluation an integral part of the Obama administration’s $4.3 billion Race to the Top competitive grant initiative designed to encourage and reward states that are creating the conditions for education innovation and reform. Meanwhile, in the nonprofit arena some of the nation’s most prominent foundations are reallocating grant money toward teacher evaluation initiatives, one example of which is the Bill & Melinda Gates Foundation investment of $290 million in four “intensive partnership” sites to support teacher effectiveness initiatives that include teacher evaluation and another $45 million in the Measures of Effective Teaching, or MET, project, a two-year effort to develop methods and tools for identifying and developing good teaching.
As the various teacher evaluation initiatives move forward over the coming years, how they play out will likely be shaped by a simple but important contextual reality. While there is wide agreement about the need for new and better ways to evaluate teachers, different stakeholders place different emphases on what they ultimately want from evaluation systems. Some see teacher evaluation as a way to identify and remove low-performing teachers. Others view teacher evaluation as the cornerstone of new performance-based teacher compensation systems. Still others think that the emphasis should be on evaluation as a mechanism for improving teaching practice, a way to help teachers get better. At the end of the day, however, the extent to which any of this can happen rests on evaluation that can consistently determine who are the more and less effective teachers in our classrooms.
Information to accomplish this comes from two sources. First, we can use teacher-related inputs to the education process, such as classroom teaching observations or classroom artifacts such as lesson plans and teacher-designed student assessments. From these practice-based measures, we make inferences about a teacher’s ability to promote student learning. Second, we can measure outputs from the teaching-learning process—actual student performance—and, based on these measures, make inferences about the teacher’s contribution to that output. In each case doing this well for high school teachers is a challenge.
In terms of practice-based measures of effectiveness, the many content areas that are covered in a typical comprehensive high school make it impossible for all teachers to be observed and evaluated by individuals who have training in the teacher’s content area. This not only potentially compromises the validity and reliability of the evaluation results; it decreases the likelihood that teachers will buy into and support the evaluation system.
Likewise, the difficulties in using student performance data to evaluate high school teachers begins with the fact that these teachers rarely teach in grades or subjects where students have had comparable pre- and post-tests that can be used to construct prototypical value-added measures for the teacher. Another issue in using value-added measures at the high school level is that, unlike the case for elementary students, we have to worry about the fact that students in, say, an 11thgrade English course took different paths to get to that course.
If these different paths affect their outcomes, then value-added models that do not account for this “path dependence” may not accurately estimate the teacher of record’s contribution to student learning. A similar problem is present if teachers affect learning across courses in a given year. Failure to account for this kind of “cross-fertilization” would again call into question value-added measures of teacher effectiveness.
Thus, there are clear challenges to effectively evaluating high school teachers. Nevertheless, states and school districts across the nation are confronting these challenges and in the process solutions are emerging. A preview of the potential solutions that the analysis in this paper suggests may be employed in building optimum evaluation systems for high school teachers includes:
- Developing new and enhancing existing assessments that test high school teachers’ content-based pedagogical knowledge
- Exploring, developing, and testing the increased use of technology such as classroom video recording as a means for generating efficiency and productivity gains in practice-based evaluation
- Conducting more research on the properties and use of Student Learning Objectives, or SLOs, as a measure of effective teaching based on student performance
- Continuing investigations into how value-added measures can be effectively used at the high school level
- Finding the best ways to incorporate all available information from both practice-based measures and student performance data into the ultimate evaluation of teachers
This paper examines the challenges and potential solutions to evaluating high school teachers, looking first at practice-based evaluation and then turning to student performance as the basis for evaluation. In each case the stage is first set with a brief discussion of the overarching, across-grade issues that accompany each method.
In reviewing the issues at hand, it is important to keep in mind that these two models of evaluation, practice-based and student-performance-based evaluation, make inferences based on different points in the education process—input versus output, and they rely on different kinds of data—qualitative and more subjective versus quantitative and objective. And they are at different stages of developmental evolution—well-established for many years (though evolving) for practice-based evaluation versus rapid developments over the last 10 years in using student performance data for evaluation. Nevertheless, the early evidence is that most new evaluation systems will be characterized by some combination of both of these methods to evaluate teachers, including high school teachers.
John H. Tyler is associate professor of education, economics, and public policy at Brown University. He is also a research associate at the National Bureau of Economic Research.
Download this report (pdf)
Download the introduction and summary (pdf)
Read the full report in your web browser (Scribd)
See also: