I was enjoying the third quarter of the tight Raptors vs Wizards game on Sunday night when my coworker sent me this article and the accompanying comments on the Boston Marathon:
Oh my. This article makes me disappointed. So let’s skip Cavs/Pacers and Westworld and dig in.
Introduction
On the surface it feels like the article is going to have math to back up the claim that “men quit and women don’t.” It has *some:*
But finishing rates varied significantly by gender. For men, the dropout rate was up almost 80 percent from 2017; for women, it was up only about 12 percent. Overall, 5 percent of men dropped out, versus just 3.8 percent of women. The trend was true at the elite level, too.
And some attempt to examine more than just the 2018 race:
But at the same race in 2012, on an unusually hot 86degree day, women also finished at higher rates than men, the only other occasion between 2012 and 2018 when they did. So are women somehow better able to withstand extreme conditions?
But that’s it. No more actual math or analyses. Just some anecdotes and attempts to explain biological or psychological reasons for the difference.
Let’s ignore those reasons (controversial as they may be) and just look at the numbers.
Analysis
The metrics used are illdefined. There is mention of how the midrace dropout rate was up 50 percent overall from last year, but no split by gender. As quoted above, the finishing rates varied significantly by gender, but no numbers are given. Only the overall dropout rates are reported. What does overall dropout rate mean? I assume it is a combination of runners who dropped before the race began plus those who dropped midrace. And then the overall dropout rates are 3.8% for women and 5% for men. But the splashy number is that men dropped out 80% more than last year whereas women only dropped out 12% more. Is… is that right? I’ve already gone crosseyed. The whole thing reeks of hacking and obscures the meaning.
There are a lot of numbers here. Some are combined across genders. Some are overall rates, some are midrace. Some are differences across years.
Frustrated with the lack of numbers in the article, I went looking for the actual numbers. I found the data on the official website. I wish it had been linked in the article itself…
2018
CATEGORY 
NUMBER ENTERED 
NUMBER STARTED 
NUMBER FINISHED 

Runners 
all 
29,978 
26,948 
25,746 
95.50% 
male 
16,587 
14,885 
14,142 
95.00% 
female 
13,391 
12,063 
11,604 
96.20% 
Now we can do some proper statistics.
First, we can perform an actual two sample test and construct confidence intervals to see if there was a difference in finishing rates between genders.
For those who entered the race, the 95% confidence interval for the difference in percent finished between males and females was (0.022, 0.006).
For those who started the race, the 95% confidence interval for the difference in percent finished between males and females was (0.017, 0.007).
The difference is technically significant, but not at all interesting. And that is ignoring the fact that we shouldn’t really care about pvalues to begin with.
But the article mentions dropout rate, not finishing rate, so let’s use that metric:
Of those who started the race, about 5% of males and 3.8% of females dropped out.
For those who started the race, the 95% confidence interval for the difference in percent dropout between males and females was (0.0069, 0.0168).
So yes, there is a significant difference. But with these kinds of sample sizes, it’s not surprising or interesting to see a tiny significant difference.
But what about 2017? What about the big change from 2017 to 2018? After all the main splashy metric is the 80% increase in dropout for men.
2017 (numbers from here)
CATEGORY 
NUMBER ENTERED 
NUMBER STARTED 
NUMBER FINISHED 

Runners 
all 
30,074 
27,222 
26,400 
97.00% 
male 
16,376 
14,842 
14,431 
97.20% 
female 
13,698 
12,380 
11,969 
96.70% 
In 2017, for those who entered the race, the 95% confidence interval for the difference in percent finished between males and females was ( 0.00006, 0.01497).
And in 2017, for those who started the race, the 95% confidence interval for the difference in percent finished between males and females was (0.0013, 0.0097).
Of those who started the race in 2017, about 2.8% of males and 3.3% of females dropped out.
For those who started the race in 2017, the 95% confidence interval for the difference in percent dropout between males and females was ( 0.0097, 0.0013).
So it does look like women dropped out more in 2017 compared to 2018. But the difference is so tiny that… whatever. This isn’t interesting. But at least now there are actual statistics to back up the claim.
But really, there’s not a lot going on here.
And FINALLY, we can look at the differences from 2017 to 2018.
The dropout rate for females increased from ~3.3% to ~3.8% which (using the exact numbers) was an increase of about 14.6% (not the 12% reported in the NYT article). The dropout rate for males increased from ~2.8% to ~5.0% which (using the exact numbers) was an increase of about 80% as reported.
At least now I understand where these numbers are coming from.
I still don’t buy it. Using dropout numbers instead of finishing numbers makes ratios much larger. An 80% increase in dropout sounds a lot more impressive than a 2% drop in finishing.
And that’s all before we try to compare to other years that might have also had extreme weather. If I had more time or interest I might look at the temperature, humidity, wind speed, wind direction etc for the past 20+ marathons. And then look at differences in dropout/finishing rate for men and women while controlling for weather conditions. That sort of analysis still probably wouldn’t convince me, but it would get closer.
Conclusion
This article is really frustrating. There are just enough scraps of carefully chosen numbers to make differences seem bigger than they really are. Comparing dropout rates to finishing rates is a bit hacky, and then comparing just two years (as opposed to many) gets even hackier. There’s an interesting hypothesis buried in the article and the data. And if we were to pull data on many marathons, we might get closer to actually being able to test if dropout rates vary by gender according to conditions. But the way the data is presented in the article obscures any actual differences and invites controversy. Audiences are eager for guidance with statistics and math. Tossing around a few numbers without explaining them (or giving a link to the source…) is such poor practice.