SOURCE: report cover
Interactive | Report | Findings | Recommendations | State Profiles | Methodology & Data | FAQs
Download the FAQs (pdf)
Why did you not create an overall grade for each state?
Evaluating educational innovation in each state is a complex task, and giving a summative grade would have masked wide variation between states.
Why did you not grade all of the categories in the same way? Or use pluses and minuses for each state’s grade?
All the categories except for one were graded against a set standard. We graded the category Staffing: Removing Ineffective Teachers on a curve because the data were not collected in a way that allowed us to grade the states against a standard. If we had graded the states against a standard in that category, we would have awarded only Ds and Fs. We gave states whole grades in each category because we wanted to be clear about each state’s standing.
Why does my state rank lower than other states with the same grade?
States with the same grade typically have different numerical scores, and the states were ranked from highest to lowest depending on how well they performed in that category. If states had the same numerical score, they were listed alphabetically.
How can a state receive a high grade in your evaluation but still get low scores on state and national achievement assessments?
Student achievement is the primary indicator of academic success, and in our previous Leaders and Laggards report, we closely examined student achievement. But states control the programs and conditions that influence student achievement, and in this report card, we graded the states on the policies that we believe are central to creating a flexible, innovative system of schooling that can solve persistent educational problems. Still, if a state earns a high grade in a single category, it does not necessarily mean that the state will show improvement in student achievement this year or next. State education systems are immensely complex, and we approached this project understanding that there was no school reform silver bullet.
Why did you award gold stars?
We awarded the states gold stars for programs and policies that we thought deserved special recognition for being particularly promising or forward-thinking but did not seem appropriate to include in our grades. These include expanded learning time, charter school accountability, and student-based funding policies.
What can states do to improve?
Based on our extensive research in the area of educational innovation, we developed a detailed list of recommendations. In a series of sidebars, we also highlighted state and local reform initiatives that potentially offer more effective ways to deliver quality schooling.
About the Categories
School Management
How did you decide which states have better academic standards than others?
To evaluate state academic standards, we relied on the work of the Thomas B. Fordham Institute, a Washington-based think tank, which evaluated the quality and rigor of each state’s science, math, and English standards.
How did you decide which states have expanded learning time policies?
To create this indicator, we relied on data supplied by the National Center on Time & Learning (NCTL). For a state to receive credit from NCTL, the program had to meet a number of criteria, including that the initiative focused on redesigning the school day or school year, instead of simply tacking on extra hours.
Why did only a few states get credit for charter school accountability? My state has shut down charter schools.
We awarded a state a gold star if it met two criteria: (1) It had more than 250 charter schools, and (2) more than 15% of its charter schools had been shut down. We did this because we wanted to highlight the states that have both created the conditions for a vibrant charter sector as well as have taken care to shut down substantial numbers of schools hampered by performance or management issues. To be sure, not all of the charter schools were shut down by authorizers for poor academic performance. Many were closed, or closed by themselves, for fiscal or management issues. But we believe it to be appropriate to highlight the states that have both created the conditions for a vibrant charter sector and taken care to create an environment in which failing or mismanaged schools shut their doors.
Why did you not include school vouchers or tax credits in your evaluation?
While the coauthors of this report firmly agree on the importance of public school choice, they have a good-faith disagreement about the merits of other forms of school choice, such as school vouchers and tuition tax credits. Proponents of such measures, such as co-author Frederick M. Hess, regard them as potentially powerful ways to promote operational flexibility, inject competition into the sector, and create more room for effective providers.
Skeptics, such as co-author the Center for American Progress, believe that these reforms undermine the virtues of public schooling and divert attention from promising areas of reform where there is consensus and are unconvinced that such measures improve educational equity and student achievement. In light of such disagreement, we have opted not to incorporate these policies into the grading criteria.
What is the Schools and Staffing Survey?
The Schools and Staffing Survey, or SASS, is a nationally representative survey of teachers and principals administered every four years by the U.S. Department of Education. For the project, we obtained a special license from the department to analyze the SASS data.
Finance
What does it mean that districts have full authority over teacher pay?
Many states have minimum teacher salary schedules, which compensates educators solely on the basis of their degrees and years of experience. This limits local autonomy and prevents schools and districts from spending money in potentially more effective ways. Relying on data from the National Council on Teacher Quality (NCTQ), a Washington-based research and policy organization, we graded the states on whether they gave districts the power to decide what teachers should be paid.
How did you evaluate online accessibility of state finance data?
To create this indicator, we surveyed each state department of education’s Web site to see whether it made finance data easily available. Specifically, we searched for total expenditure per district, total revenue per district, average teacher salary at the district level, and per pupil expenditure at the district level. To simulate the experience of a parent or taxpayer, we limited ourselves to 10 minutes on the Web site.
How did you evaluate the simplicity of a state’s funding mechanism?
Based on our research and the work of others, we believe that line items serve as a reliable proxy for the number of funding programs in a state. Therefore, we evaluated the budgets of all 50 states and counted up the number of line item expenditures. States with the most line items received the lowest grades; states with the fewest line items received the highest grades.
What is a student-based funding policy?
In almost every state, education dollars do not follow students to the schools they attend according to their needs. Instead, funds are distributed based on factors that have little to do with students, such as the number of teachers in a school or the types of educational programs. Such financial practices make it nearly impossible for principals to allocate resources in new and innovative ways.
We believe that states could benefit from a system of student-based funding under which a student’s school would receive a certain amount of dollars based on the student’s particular needs and could spend those funds flexibly. Currently, Hawaii is the only state that has a student-based funding policy in which education dollars truly follow the student.
Staffing: Hiring & Evaluation
Why did you not include measures of teacher certification?
We rejected the premise that credentialing is a reliable measure of teacher quality. To raise the capacity of the teaching force, we believe that states need to open up additional pathways into the profession while maintaining checks on quality.
How did you measure how much flexibility principals have over teacher hiring?
We relied on data from the Schools and Staff Survey, a nationally representative survey of teachers and principals administered every four years by the National Center for Education Statistics. The data came from the principal questionnaire, which asked, “How much actual influence do you think each group or person has on decisions concerning the following activities … hiring new full-time teachers?” Principals ranked their own influence on a scale: minor influence, moderate influence, major influence, and no influence. We looked only at principals who reported a major degree of influence.
Staffing: Removing Ineffective Teachers
Why are the data on removing ineffective teachers important?
Many states have an archaic system of policy and procedures that make it extremely difficult for school leaders to remove ineffective educators from the classroom. But without the ability to remove poor-performing teachers, school leaders can’t build a cohesive school culture, create an environment of accountability, and ensure that all students can learn.
Did you look at the process for the removal of teachers from the classroom? What protections and procedures are in place to make sure that the removal of the teacher is fair?
We relied on data from Schools and Staffing Survey, a nationally representative survey of teachers and principals administered every four years by the National Center for Education Statistics. The survey did not look at how or why teachers were considered ineffective. The questionnaire only asked principals if they believed that certain issues were a barrier to the “dismissal of poorperforming or incompetent teachers.”
Data
What is a teacher-identifier system?
A teacher-identifier system allows state officials to match teachers to their students’ achievement and is a key tool in understanding teacher effectiveness and student performance.
What is a unique statewide student identifier?
A unique statewide student identifier is a single number that is assigned to a student and stays with that student as he or she progresses through each grade level. While providing for student privacy, the number allows state officials to follow students as they move through school and helps educators track dropout and graduation rates as well as student achievement for individual students.
Technology
What is a high-speed Internet-connected computer?
For this indicator, we relied on data collected by the research firm Market Data Retrieval and published by Editorial Projects in Education. The firm defined a high-speed Internet-connect computer as one connected to the Internet by a T1, T3, or cable modem.
What is a virtual school?
We relied on data collected by Editorial Projects in Education for this indicator, and the organization defined a virtual school as an education institution where instruction is delivered over the Internet.
Pipeline to Postsecondary
What are the skills necessary to make a student both college and workplace ready?
The business community has been clear about what students should know and be able to do. Achieve Inc.’s American Diploma Project has, for example, established detailed benchmarks that define the knowledge needed for a student to graduate high school. In short, students must possess a combination of academic and applied skills, including the ability to read, write, communicate (in English and another language), calculate, problem solve, and work in groups.
How was the Advanced Placement (AP) indicator calculated?
We relied on data calculated by the College Board for this indicator. The New York-based research organization, which oversees the AP Exams, calculated the figure by dividing the number of public school students who earned a 3 or higher at any point during his or her high school career on any AP exam by the number of students in the class of 2008.
What does it mean for a state to require a college- and career-ready diploma?
For this indicator, we relied on data collected by the Washington-based research group, Achieve, Inc., which looked at whether students in each state needed to complete a college- and career-ready curriculum in order to graduate. For the most part, states have raised their course requirements using one of two approaches. Some have required students to automatically enroll in a college-ready curriculum but allowed them to opt out if their parents sign a waiver. Others have set mandatory course requirements without any opt-out provisions. Since both approaches aim to expand access to rigorous academics, we gave credit to states that had taken either approach.
What high school exams qualify as gauging college- and career-readiness?
Achieve Inc. collected this data, and the organization gave credit to states such as New York that have developed their own college readiness exams as well as states such as Maine that have incorporated national college admissions exams like the SAT into their assessment systems.
State Reform Environment
Why did you not grade the states in this category?
While we would argue that a state’s reform environment is perhaps the most important category in our report, it
Interactive | Report | Findings | Recommendations | State Profiles | Methodology & Data | FAQs
Download the FAQs (pdf)