Broken Telephone

A recent post at Reason‘s Hit & Run (a blog I generally much like), says the following:

Playing StarCraft can boost problem solving and creative thinking, according to a new study by researchers at Queen Mary University of London and University College London. Researchers found that those who engaged in the real-time military strategy game improved their “cognitive flexibility,” or the ability to adjust their thinking to meet different situations.

The participant pool was composed entirely of 72 female students at the University of Texas at Austin, because researchers were unable to find male participants who played computer games for less than two hours a day.

Yes, that’s right. They could not find any college-aged men who barely played video games….

The source linked to by Hit & Run was Wall Street Journal‘s Digits blog:

Playing StarCraft can boost problem solving and creative thinking, according to a new study by researchers at Queen Mary University of London and University College London. Researchers found that those who engaged in the real-time military strategy game improved their “cognitive flexibility,” or the ability to adjust their thinking to meet different situations.

The participant pool was composed entirely of 72 female students at the University of Texas at Austin, because researchers were unable to find male participants who played computer games for less than two hours a day.

So far so good — but here’s what the underlying study actually says (emphasis added):

Potential participants were screened with a web-based questionnaire regarding their current and past video gaming habits. The principle items of the questionnaire were, “In the last year, how many hours per week do you tend to play video games?” and “Prior to the last year, think back to the period in which you most frequently played video games. How many hours per week did you tend to play during that period?” Participants who reported 2 hours or less of video game play per week qualified for inclusion in the study. A sample of 816 prospective participants (463 female) were screened for inclusion in the present study. Only nine male respondents qualified based on their self-reported gaming habits, whereas 90 females qualified.

That’s right — the researchers had a hard time finding male participants who played computer games less than two hours per week, during the period in which they most frequently played video games, and not necessarily now.[1] That morphed into two hours per day of current play in the Wall Street Journal blog post, and then was stressed as two hours per day in the Hit & Run blog post.

Mistakes happen, especially in blogging, where one generally doesn’t carefully proofread one’s posts, or spend a good deal of research time on tracking down original sources. Still, this kind of thing reminds us that, if possible, checking the original source is a good idea even for blog posts. Otherwise, the intermediate source’s errors can easily become one’s own.

[1] I’m pretty sure that the two-hours-per-week threshold was applied to the second question and not the first because only 90 of the 463 female respondents qualified, and “[f]or all female respondents, the median hours of current weekly game play was 1 (SD = 7.08) and 7 (SD = 12.6) at their most frequent.” If the two-hours-per-week threshold were applied based on current play, then well over half the women respondents would have qualified. Moreover, the study seems to discuss the eligible participants as “novice players,” which would likewise suggest that they were selected based on their never having played much.