Tuesday, July 1, 2014

Sensationalism Is Not an Acceptable Substitute for Scientific Reasoning

I read this article recently. "Female-named hurricanes kill more than male hurricanes because people don’t respect them, study finds." At first I thought it was an entertaining article. Now it kind of bothers me a bit.

I get that it is popular to find sexism in everything, but that is no excuse for publishing garbage science. Or presenting such a misleading headline. The study just found that female hurricanes kill more than male hurricanes when comparing 2 very unequal sample groups and that when asked about hypothetical hurricanes, people tended to estimate that ones with male names were more devastating. We will discuss the hypothetical storms first.

People consider hypothetical storms with female names less threatening than hypothetical storms with male names. I concede that I may just not be understanding what they did, but from the description in the article, it seems like they just asked people how threatening different storms sounded to them and how likely they would be to evacuate if Hurricane Danny (or Hurricane Debby) was on its way (I borrowed these names from the actual 6 year rotating list you can find here. I don't know if the study used these names, but they probably should have). Apparently people said the female storms were less threatening.

Okay, if this is indeed what they did, all they did was prove that people found female names less threatening (which may be sexist, but it still has nothing to do with hurricanes). They did nothing to show that people actually behaved differently as a result of these names. If you asked me how dangerous I thought an imaginary hurricane was I would have to base my answer on the limited information given me, which seems to be mostly a name; this study found that people perceive male names as more dangerous. If you asked me how dangerous I perceived a real hurricane is I would base my answer on pretty much all the information but the name. In a study room with no real threat, I may be perfectly content to base my assessment of the imaginary off of completely arbitrary things like my dislike of that kid Dean in my 3rd grade class; hey, I decide who I am rooting for in sports that way all the time. In real life, I would never base a decision about evacuating entirely on a name.

Maybe I am alone in this and other people regularly decide whether to risk their lives based on something completely unrelated to the problem at hand. Still, I don't accept that finding that people assess female names that they happen to be told are hurricanes as less threatening than male names they happen to be told are hurricanes as proof that naming a storm after a girl makes it more likely to kill people. There is just a bit too much of a leap there for me. Of course, if you could compare real male and female storms and control for all other variables (besides the name) and you found female storms were significantly more deadly, then you might have something.

So, without further ado, we move on to the study's comparison of male and female storms.

"Researchers at the University of Illinois and Arizona State University examined six decades of hurricane death rates according to gender, spanning  1950 and 2012.  Of the 47 most damaging hurricanes, the female-named hurricanes produced an average of 45 deaths compared to 23 deaths in male-named storms, or almost double the number of fatalities."

So let me get this straight: you took a period of 62 years, took only the 47 most devastating hurricanes (which is a bit of a suspiciously random number by the way. I guess the results were less interesting if they used the top 50), and then used it to determine that people are sexist when it comes to hurricanes. Of course, for the first 29 years, they only gave out female names, so any differences could just as easily be credited to advancements in life-saving technology by the time any male storms were christened. It says it discounted Katrina and Audrey as outliers, but it should have discounted everything before 1979.

Bigger samples are usually better in statistics, but this is one instance where it made the entire thing less credible. In my mind, including storms before 1979 is roughly similar to comparing men and women on eating habits when your samples consist of 25 men found in the mall food court and 75 women, 25 of whom were also found in the food court and 50 of whom were found in a women's gym down the street and then declaring women eat healthier than men. You can't get your samples from unequal places and consider your results sound, particularly when there are obvious reasons that the subjects in one place are likely to produce drastically different data than those in another.

In the latter decades, a really bad hurricane had a 50/50 chance at being named a female name because every other hurricane was given a female name. Before 1979, all hurricanes were female, hence the number difference of 75 to 25 in the analogy above; these researchers are drawing from data where female hurricanes outnumber male ones roughly 3 to 1 (depending on how many hurricanes actually occurred each year). Statistics has ways of dealing with differences in sample size in 2 compared groups, but generally it is still better to avoid large discrepancies; the methods don't eliminate the effects of the differences, they just minimize it. Still, I will let that slide for now. What I will not let slide is the differences in where the information is from.

Think about 1950 for a second. TVs were black and white and by no means something everyone had. Cell phones were the stuff of science fiction and telephones were often party lines, which meant you could only use the phone if none of your neighbors were doing so already. Radios were a common method of obtaining news. Meteorology has also gotten much better in the last 60 years. According to Wikipedia (humor me here) besides the naming of hurricanes, 1950 brought the first successful numerical weather prediction experiment; really, we were just getting started. It isn't until the 1970s that radars are really standardized and organized into networks. The National Oceanic and Atmospheric Administration ( NOAA) was also established in 1970. Doppler weather radar didn't become common until after 1980 meaning that before then we didn't really know how fast a storm was traveling; that is kind of a big deal. Now digital satellite, radar, and surface observations are analyzed to predict storm patterns. A lot has changed.

So let's line up this information regarding technological advancements with hurricane naming practices. We have 29 years of female hurricanes and then 33 years of half male and half female hurricanes. We have 20-30 years of very little radar use that aligns with the 29 years of all female hurricanes. Then, 1-10 years after male names begin to be used, we finally started widely using technology that tells us how fast a storm is going. In other words, most male names have been given out after Doppler radar and most female names (that 50 versus the 25) were given out before. Add to that the significant advancements in being able to communicate with the general public, and what we have is roughly 2/3 of the female hurricanes coming from an era where it is quite logical to assume more people would die in a hurricane than now and all of the male names coming from an era where technological advances hopefully minimize deaths.

Not convinced? How about this handy comparison of two hurricanes, Audrey in 1957 (the death toll of which can't be entirely determined but is somewhere between 300 and 700 from the figures I have seen) and Rita in 2005 (note both female hurricanes). "Rita hit the same region of coast with weaker winds (Category 3, 115 mph), but a storm surge even higher (15 feet). Rita destroyed virtually 100% of Cameron, whereas Audrey destroyed 75% of the town. Nearly two years later, Cameron is mostly just concrete slabs and trailers, thanks to Rita. However, Rita caused only one direct death in Southwest Louisiana--a drowning in Lake Charles. The answer is preparedness. Rita was a massive Category 5 hurricane several days before landfall, giving people plenty of time to receive the warnings and evacuate. Warning systems are much better now than in 1957, and Cameron was deserted when Rita hit." To be entirely fair, Audrey formed ridiculously fast, so even today the death toll would likely be higher than Rita, which as the quote points out, gave plenty of warning (though arguably some of that warning was due to better weather tracking). Also, it is important to note that Rita came just after Katrina in 2005; with that devastation fresh in the nation's mind, lack of respect for a hurricane was not really a problem. Still, the difference here was likely the ability to prepare--be it because of more time or more technology--not the lack of respect for a female name versus a male one.

Still not convinced? Okay, we will go about this a different (and simpler) way. This article addresses the falling number of fatalities due to hurricanes in the US. The sharp increase in 2000-2010 is pretty much entirely due to Katrina, by far the deadliest hurricane during these six decades, which even the original study declared an outlier. This table shows the breakdown by year, which showed me two things: first we are generally talking about not really all that many people in the grand scheme of things which means statistical analysis is greatly affected by small variations and second, we are really making progress in dealing with hurricanes.

No, none of this proves that sexism does not play a role in how seriously people take hurricanes, but I would say that it does cast reasonable doubt upon the conclusions made by a poorly designed statistical analysis. As seems to always be the case on this blog, the lesson is one of logic; while statistics are a great tool, it takes logic to use them correctly and logic to determine if another individual has used them correctly. In this case, logic seems to have been entirely missing from the analysis. It does make for an entertaining article though.