A team of faculty and students at the University of Washington was tasked with assessing how Seattle’s 2014 minimum wage ordinance, which is gradually raising the city’s minimum wage to $15 per hour, is affecting low-wage workers. This week, the group released a working paper—without peer review—that looks at the ordinance’s first two phases, under which the minimum wage for most workers increased from $9.47 to $11 per hour in 2015 and then to $13 per hour in 2016.
Methodological flaws plague the group’s approach, causing them to draw conclusions wildly out of step with dozens of studies of similarly sized wage increases cited by both critics and proponents of higher minimum wages. The vast majority of rigorous, credible studies conclude that higher minimum wages have appreciably boosted workers’ earnings with little or no effects on employment. By contrast, the University of Washington researchers conclude that higher minimum wages not only reduced employment and hours worked in Seattle, but that the costs of the wage hike outweigh the benefits for the average low-wage worker—a finding at odds with the conclusions of even the most skeptical mainstream researchers. At the same time, the study’s results suggest—implausibly and largely inexplicably—that the wage hike to $13 per hour caused substantial growth in jobs paying more than $19 per hour in the restaurant industry. That’s just one of several questionable results that should give readers serious pause.
The authors’ flawed approach—as well as the limitations of their data set, which will automatically bias them toward finding employment losses—drew immediate, detailed criticism from economists and minimum wage experts. Their findings are also dramatically at odds with a separate study on Seattle’s wage hike published one week beforehand, whose co-authors include two well-known minimum wage scholars. These scholars, studying the same two phases of the Seattle wage ordinance, found effects in line with existing research—specifically, that a 10 percent increase in Seattle’s minimum wage caused wages in the predominately low-wage food services industry to rise by about 1 percent, on average, without discernibly affecting employment.
Key methodological issues
Here are five ways the University of Washington study fails a credibility test—and why its conclusions should be treated with great skepticism.
- The study excludes 48 percent of workers in Seattle who earned less than $13 in mid-2016—including workers most likely to earn close to minimum wage. The authors’ data set is limited to workers at single-site employers. This means many of the heaviest users of minimum wage labor—multisite employers such as chain restaurants and retail stores—are excluded from their analysis, making their data highly unrepresentative of the low-wage labor market. Additionally worrying is that the excluded multisite employers are much more prevalent in Seattle—by about 10 percentage points—than in the authors’ control group, which is drawn from other areas of Washington state. This, as well as other concerns raised by researchers at the University of California, Berkeley, calls sharply into question whether the study has a valid control group against which to analyze the effects of Seattle’s minimum wage change.
- The study finds that Seattle’s minimum wage caused the greatest effects in the least plausible place—high-wage jobs. In an attempt to confine their analysis to low-wage workers, the authors cut off their sample of workers at a wage of $19 per hour, explaining that this threshold is “roughly twice the initial value of the minimum wage, a level beyond which cascading effects are less likely to occur” based on prior research. Yet as the Economic Policy Institute (EPI) uncovered, the study’s own methods actually find that in the restaurant industry, the strongest causal effects of Seattle’s minimum wage were on jobs paying more than $19 per hour.
In other words, EPI’s clever analysis of the University of Washington team’s results is not simply showing that growth was stronger in higher-wage jobs during the study period—although that is also true—but rather that the team’s analysis finds that Seattle’s minimum wage directly caused much higher-paying jobs to grow—and to grow nearly twice as much as it caused low-wage jobs to shrink. There are no two ways about it: If one is not convinced that raising the minimum wage has larger consequences for workers making more than $19 per hour than for low-wage workers, then neither should one be convinced by the study’s claim that employment and hours dropped among low-wage workers. This highly questionable result is strong evidence that the University of Washington study mistakenly attributes to the minimum wage labor market changes that are instead being driven by other factors in Seattle’s booming economy.
- The study’s data limitations bias it toward overestimating the downside—but missing the upside—in employment. If, as some research suggests, minimum wage changes cause some workers to shift from single-site employers to types of work outside the authors’ data set—such as jobs at multisite employers, independent contracting, or informal employment—the study’s approach will record this as the loss of a job and a reduction in hours worked. On the flip side, insofar as higher minimum wages cause additional lower-skilled workers to enter the workforce—perhaps because they can now afford child care or transportation to and from work—the authors will fail to capture these job gains and hours increases if they occur at large employers, chains, or other multisite businesses.
- The findings are utterly out of step with hundreds of rigorous studies, spanning multiple decades of research. According to the authors’ conclusions, a 10 percent increase in the minimum wage will reduce low-wage employment by an astounding 30 percent. As researchers at the University of California, Berkeley, point out, this is “ten times higher than the average in 942 published minimum wage and nonminimum wage estimates,” as well as more than triple that of even strident minimum wage critic David Neumark’s largest estimates.
Figure 1, from a meta-analysis of 64 rigorous minimum wage studies, illustrates just how much of an outlier the study’s results are. Each dot in the figure represents a credible study, plotted by its employment elasticity on the horizontal axis and its statistical precision on the vertical axis. More precise estimates are higher up. The dots are clustered around zero—and are also most precisely estimated there—indicating that most studies find little or employment effect. Out of nearly 1,500 estimates, only 15—drawn from just four studies—represent employment elasticities more negative than the University of Washington team’s estimate of -3.
Furthermore, the authors’ findings conflict with a significant body of related causal research, including studies showing that higher minimum wages increase family incomes, reduce poverty, and lower spending on taxpayer-financed public assistance programs such as food stamps and Medicaid. If higher minimum wages truly made workers worse off, as the University of Washington team concludes, one would expect such studies to reach the opposite conclusions.
- The results hinge on a geographic sample size of the loneliest number—one—subjected to an untested, unconventional approach. In any single local labor market, hundreds of factors contribute to changes in wages, earnings, and employment. Isolating the effects of minimum wage policy from those other factors is very challenging and requires great care. This will be particularly true in the Seattle labor market, which may have more than its share of other factors affecting wages and employment: As the study’s authors admit, prior to the minimum wage ordinance Seattle had been experiencing—and continues to experience—a boom of rapid job growth and expansion.
To tease out the effects of minimum wage policy from these confounding factors, most studies combine information from hundreds of different geographic locations, time periods, and wage-hike sizes—or narrow their focus on a sector known to have many affected workers such as the restaurant industry. By contrast, the University of Washington researchers have only one single “treatment” location and policy—Seattle in 2015 through 2016. Moreover, they attempt to tackle all jobs paying less than $19 per hour across industries, and they do so by applying an untested statistical technique to a nonrepresentative sample as noted in the first point above. Given the rapid changes in Seattle’s labor market, this is akin to conducting a clinical trial of a muscle-enhancing drug on a single subject if that subject were, say, LeBron James—who is light-years more physically fit than most humans. As analysis from EPI demonstrates, the authors’ statistical techniques don’t appear to be powerful enough to separate out the effects of the minimum wage change from the many other factors affecting wages and employment in Seattle, thus contaminating their results.
Conclusion
The University of Washington group’s findings contrast starkly with those of their first report, released in July 2016, which found significant earnings gains alongside modest and statistically insignificant employment effects—much more in step with existing research. Paired with concerns about methodological flaws and findings of unprecedented magnitude, this reversal suggests that the University of Washington’s research should be held to a high standard of review before it can be taken seriously.
Unfortunately, however, other researchers cannot currently replicate the group’s analysis or investigate why the authors’ results are so far outside the range of previous research, because the authors rely on proprietary data to which only they have access. Until the University of Washington team can satisfactorily address the problems with its methodology, Americans watching the wave of state and local minimum wage increases will find a more reliable guide in both the newly released study from the University of California, Berkeley, and the rich research based on hundreds of past minimum wage changes.
Ultimately, the dismal picture the University of Washington team paints of widespread job loss among low-wage workers is at sharp odds with the reality of Seattle’s thriving economy. The city’s unemployment rate of 2.6 percent—its lowest in nearly a decade—is a full 1.5 percentage points below the national average. As the study’s authors acknowledge, strong economic and job growth have prevailed during the very period when Seattle’s minimum wage has risen, creating a highly favorable environment for job seekers that the city forecasts will continue.
Seattle’s workers, businesses, and policymakers should demand high-quality, rigorous evaluation of their city’s minimum wage ordinance. Unfortunately, the flawed working paper from the University of Washington not only fails to reflect labor market realities or pass the credibility test—it also risks misinforming the strong majority of Americans who support raising wage for the nation’s lowest-paid workers and their families.
Rachel West is an associate director for the Poverty to Prosperity Program at the Center for American Progress. She thanks her colleague Brendan Duke, the associate director for economic policy at the Center, for helpful contributions.