Talks between the Chicago Public Schools and the Chicago Teachers Union broke down Sunday, and now the city’s teachers are on strike, just as classes were starting for the 2012-13 school year. Labor will insist that the strikes lead to contracts that attract good teachers who promote student learning in the long-run. Mayor Rahm Emanuel, meanwhile, notes that the teachers are striking over his proposed evaluation system, which he argues will help achievement going forward. Leaving that debate aside, what does the strike itself mean for students?

Nothing good, the best empirical evidence suggests. Two of the best recent studies on the effects of teacher work stoppages and strikes concern labor disputes in Ontario schools in the late ’90s and early 2000s. One, by the University of Toronto’s Michael Baker, compared how standardized test scores rose between grade 3 and grade 6 for students who lost instructional time because of the Ontario strikes, and for students who were unaffected.

Baker found that if the strike happened when a student was in grade 2 or 3, their scores rose by slightly less. But if the strike happened when the student was in grade 5 or 6, their scores rose by a whole lot less. Scores for strike-affected fifth-graders were a full 3.8 percent lower than those for fifth-graders in schools and grades not affected. If that doesn’t seem like much, it’s 29 percent of the standard deviation (or the typical amount by which students differ from their class average). Strikes, in other words, accounted for one third of why some students did better than others.

Wilfrid Laurer’s David Johnson studied the same Ontario strikes and also found that they hurt student achievement. Like Baker, he found only small effects for students for whom the strike occurred in third grade, but large effects if the student was in sixth grade. In the latter case, the percentage of students getting a passing score on math standardized tests fell by 0.21 percentage points per day, and the percentage getting a non-failing score across all tests fell by 0.10 points per day. The effects were much more dramatic in poorer and more socially disadvantaged school districts, where overall passing scores went down by 0.35 points per day. Given that strikes typically last a week or more, these results can add up. A nine-day strike, for instance, reduces passing rates 3.15 percentage points.

And it’s not just Ontario. Michele Belot and Dinand Webbink, now of the Universities of Edinburgh and Rotterdam, respectively, found that work stoppages hurt student achievement, increased the number of students repeating grades and reduced higher education attainment in Belgium. What’s more, studies dealing with teacher absences for reasons other than strikes bolster these findings.

A study by Harvard’s Raegen Miller, Richard Murnane and John Willett tracked the effects of teacher absences while controlling for teacher experience and skill level. They noted that teachers who are absent more regularly may be less motivated and skilled, and so they isolated absences due to poor weather, the idea being that even highly skilled teachers will be absent if the weather prevents them form getting to work.

The study found that absences lead to statistically significant drops in student math and reading scores. The drops are lower than those found in the Baker and Johnson studies, but then again, the students in the Harvard study received instruction from substitutes, whereas students in strikes get no instruction at all. Studies by Charles Clotfelder, Helen Ladd and Jacob Vigdor at Duke and by Mariesa Herrmann and Jonah Rockoff at Columbia found significant drops in student achievement because of absences in North Carolina and New York schools, respectively, with the latter finding that a lengthy absence had the same effect as replacing an average teacher with one at the 30th percentile.

The only recent study to find no significant results from teacher strikes was conducted by Harris Zwerling, a researcher at the Pennsylvania State Education Association, the state’s largest teacher’s union. That study compared Pennsylvania school districts that experienced strikes to those that didn’t, and found no difference in outcomes once one controls for demographics and years of teacher service; this is much the same methodology as the Ontario studies. One could argue that because the study focused on U.S. schools rather than Canadian or Belgian ones, it is more directly relevant.

But then again, Pennsylvania requires schools to make up lost time due to teacher strikes at the end of the school year, which Canadian and Belgian schools don’t. Illinois schools are required to teach 176 days a year, and the union insists that agreements to make up lost school days are traditional in bargaining agreements. But the 176-day requirement is frequently ignored, with 400,000 Chicago schoolchildren only attending school for 170 days. So there’s a real possibility that the Chicago strike will end up like the Canadian and Belgian ones, with real lost instructional time and big effects on student learning as a result, rather than like the Pennsylvania one, with no lost time and no effect on learning.

One last thing: one could protest that all these results rely on standardized testing, which may or may not correlate to real learning. That’s fair enough, but there’s a bounty of evidence, from Harvard’s Raj Chetty and Stanford’s Eric Hanushek, among others, suggesting that standardized test scores correlate with higher education achievement, lifetime earnings and more. So if the Chicago strike does end up hurting student scores, it could affect their lives for years into the future.

– – –

Matthews covers taxes, poverty, campaign finance, higher education and all things data. He has also written for The New Republic, Salon, Slate and The American Prospect.