If the news accounts are correct, the new study distinguishing the brain scans of liars from truth-tellers has a serious design flaw that goes beyond the small sample size. Indeed, it is such an obvious flaw that I wonder whether the researchers really made it, or whether instead the reporters got the story wrong:
Faro and colleagues tested 10 volunteers. Six of them were asked to shoot a toy gun and then lie and say they didn’t do it. Three others who watched told the truth about what happened. One volunteer dropped out of the study.
While giving their “testimony,” the volunteers were hooked up both to a conventional polygraph and also had their brain activity imaged using fMRI, which used a strong magnet to provide a real-time picture of brain activity.
There were clear differences between the liars and the truth-tellers, Faro’s team told a meeting in Chicago of the Radiological Society of North America.
“We found a total of seven areas of activation in the deception (group),” he said. “We found four areas of activity in the truth-telling arm.”
Overall, it seemed to take more brain effort to tell the lie than to tell the truth, Faro found.
Lying caused activity in the frontal part of the brain — the medial inferior and pre-central areas, as well as the hippocampus and middle temporal regions and the limbic areas. Some of these are involved in emotional responses, Faro said.
In experimental design it is elementary to match the experimental conditions as closely as possible. In very small studies, one should vary only one variable at a time (there are some designs that rotate several changes). Here the liars fired a gun while the truth-tellers passively watched someone else fire a gun. Then the people who shot the gun engaged more of their brains than the passive people who just watched. Not surprisingly, the more active gun-shooters engaged more active portions of their brains.
But the researchers concluded that the gun shooters engaged more of their brains, not because they fired guns, but because they lied. How can they tell? There is no basis in the news report to think that the experiment tested lying v. truth-telling as opposed to gun shooting v. passively watching. The two experiments are completely confounded.
As I said, this is such an obvious defect that I am skeptical that the news reporters have accurately reported the study.
To test how obvious the error is, I asked my daughter taking high school AP Statistics what is wrong with the study. Before finishing the reading the news article, she pointed out the sample size. When I said, “No, not that,” it took her about 15 seconds to see the problem.
I hope that the news reports are the ones that are in error or that there is some independent reason to think that watchers engage as much of their brains as more active gun shooters.
UPDATE: Aha, just as I suspected. The error appears to be in the newspaper accounts rather than in the study itself. Thanks to a tip from “JustOneMinute,” I see that there is an account online which says that they did vary who was lying and who told the truth. I would still like more details, but there is nothing obviously wrong in the research design described here:
The research group used 11 volunteers and asked six to shoot a toy gun with blank bullets. Five other participants did not shoot the gun.
In two experiments, both shooters and non-shooters were asked to alternately lie and tell the truth about their participation. Scientists then examined the individuals with fMRI, while simultaneously administering a polygraph exam. The polygraph tests measured blood pressure, respiration and changes in perspiration.
The team found that both fMRI and polygraph accurately identified cases where participants had lied about their involvement in the shooting.
Comments are closed.