A lot of stupid academic research goes on every day. Today's Washington Post magazine features one of the dumbest I have come across in some time--Project Implicit. According to the Project Implicit website its purpose is as follows:

It is well known that people don't always 'speak their minds', and it is suspected that people don't always 'know their minds'. Understanding such divergences is important to scientific psychology.

This web site presents a method that demonstrates the conscious-unconscious divergences much more convincingly than has been possible with previous methods. This new method is called the Implicit Association Test, or IAT for short.

I took the test a few times. The first asked me to identify whether I preferred "Bill Clinton" or "Hillary Clinton", the second asked me whether I preferred "Church" or "State". I told the test that I didn't really like Bill or Hillary very much, but that I had a slight preference for Bill; and the same with Church versus State. It then led me through 10 minutes of various rapid association responses at which point it told me--drumroll--that I had a slight preference for Bill over Hillary and Church over State. The questions were focused on such things as whether I liked structure, rules, and predictability in my life. From what I can tell, these implicit associations supposedly drive my conclusions about public policy issues.

Is it really plausible that my impression of Bill and Hillary is driven more by whether I have a messy desk than my personal perception that Bill Clinton is a liar and Hillary Clinton is a megalomaniac and opportunist? Or, that my perception is that Bill's ideological views are only somewhat liberal, and Hillary's are very liberal, and therefore Bill's views are closer to mine than Hillary's? From what I can tell, this is about as scientific and insightful as a horoscope or palm reading. Needless to say, the Washington Post loves the research because it purports to demonstrate that deep down inside we are all racists. This may or may not be true, but I certainly didn't find anything in Project Implicit that would shed meaningful light on that question.

And, of course, none of this says anything at all about whether it actually explains how people act, or the extent to which our overall behavior and attitudes are shaped by these factors versus learned and conscious behaviors. I have little question that Project Implicit could demonstrate that I have a preference for cheesecake versus green beans, yet I eat green beans with dinner twice a week and cheesecake once a month.

Incidentally, for all our libertarian readers out there, you will be happy to know that I had to cancel one round of tests because it insisted that the "correct" association for "anarchy" was "chaos" rather than "order" and since I refused to give in it wouldn't let me proceed.


It appears that some commentators have misunderstood my criticism of project implicit (others share my assessment). Some have said that I am denying that part of our cognition occurs at a subconscious level that we cannot control (see here and here). That is not a correct interpretation of what I said in my original post. In fact, I agree that much of our cognition goes on beneath our subconscious awareness, so I am certainly not mocking that idea. Rather, I am in fact criticizing Project Implicit as a methodology for distilling unconscious attitudes--a concern which, ironically, my critics share. Second, while again I agree with the proposition that much of our thinking goes on at a subconscious level, it is a heckuva a long way from that observation to suggesting that may be the reason why I don't dislike Bill Clinton as much as I dislike Hillary Clinton. Like evolutionary psychology (which I have endorsed as being potentially very useful, see also my comments on Lawrence Summers here), I think the study of cognition and unconscious reasoning is very useful and explains much. I just think that it is important in studying this, as with everything else, that we remain aware of the limitations of the work and, in particular, make sure that the conclusions and implications we draw are actually supported by what the experiments are actually calibrated to test. Clearly many of my beliefs and actions are motivated primarily by my subconscious, equally clearly to me many of my other beliefs and actions are motivated primarily by my conscious, and most is in-between. I recognize that my love for my family or the Pittsburgh Steelers is heavily rooted in my subconscious mind; but I also find it much more likely that my slight preference for Bill versus Hillary Clinton has a lot more to do with my conscious. Similarly, I believe that evolutionary psychology can tell us a lot about why we prefer candy to broccoli, but tells us little about why we may prefer Coke to Pepsi.

More fundamentally, I think the point here is that at some level all of us have an obligation to use our critical thinking and common sense to determine whether given research makes sense and whether it can support the conclusions that some will want to attribute to it. I have recently commented on some of the problems that behavioral law & economics has run into, for instance, by failing to meet this test. And given that funding for university research is a scarce economic good, and that we all support it with our tax dollars and tuition payments, it behooves us to direct societal resources to useful research rather than the opposite. Indeed, there are American university faculty who are still blieve in a Marxian "scientific" view of history, who will insist that all of our views are shaped at a subconscious level by class bias and that there is a scientific unfolding of history and will be happy to provide you with "evidence" of both. Prior to that, social Darwinists purported to demonstrate that Darwinian evolution described and justified a particular social order. Do you have to be an expert to reject the hypothesis that there is a Marxian science to history or social Darwinism? Do I have to be an expert in the "science" of horoscope reading in order to reject the proposition that "the stars" are in control of my life? I think not. I think that the lesson is that we should draw as citizens and taxpayers is that we all would have all been better off if we had had raised questions about these arguments and their limits from the outset, rather than tenuring these theorists and inviting them to teach their theories to generations of college undergraduates.

So the bottom line is that I encourage each of you to visit the Project Implicit web site and take the test and make up your own mind. As "In the Agora" states commenting on my original post, "My point ... is to highlight how important it is to cast a discerning eye on everything you find in the news, even if it comes from professors working under the auspices of 'academic research.'" I couldn't say it better.


I thought readers might be interested in some correspondence that has followed up on my earlier post on Project Implicit. One person wrote (edited):

Perhaps your knee-jerk reaction was not to the Implicit Association Test but to the possibility that you are not always consciously in control of your beliefs and behaviors. If that is the case, then I am sorry to inform you that decades of research in neuroscience and psychology demonstrates that you are, in fact, in control of very little of your mental life. In fact, you're not even aware of most of it, and that includes your goals, motivations, and attitudes.

Now if this an accurate statement of the importance of subconscious reasoning (other correspondents did not state the position so strongly but made the same general points), it would seem to raise some pretty thorny questions. In particular, if this is true, it seems to necessarily imply that the primary reason why a person believes in the finding of the Implicit Association Test and the overwhelming importance of subconscious reasoning is primarily because there is something in that particular person's subconscious that makes them believe this theory as opposed to other competing theories. And, it would seem to follow, the reason why I am skeptical of some of the theory's more extravagant claims about what it can explain regarding personal beliefs and behaviors (such as my preference for Bill versus Hillary Clinton) is primarily because of something in my subconscious as well. In short, if the theory itself is correct, then one's belief about the validity of the theory itself must be the result of the same subconscious reasoning processes that it purports to explain. And if it is the case that our views on the usefulness of Project Implicit are little more than a reflection of our subconscious, wouldn't it be pointless to have a conversation trying to persuade me to use my conscious mind to revise my supposed subconsciously-biased negative opinion of Project Implicit itself? The fact that the correspondent took the time to write to me (which I always appreciate, by the way, although I would prefer if you would refrain from ad hominem attacks in your emails) suggests that she (subconsciously perhaps?) recognizes the limitations of her own theory.

As I said earlier, it seems much more plausible to me that there are some of our views and opinions that are a reflection of subconscious and others that overwhelmingly reflect the influence of our conscious minds and that we don't want to try to claim too much for either. Bill versus Hillary and Church versus State (the two tests I took) seem to me to fall much more on the conscious side of the line, and one suspects that the reason those modules are in there is so that Project Implicit can try to demonstrate just how broad its claims can reach. And to try get from subconscious attitudes to an explanation of many aspects of behavior and eventually large-scale social policy recommendations (as implied by the article in the Washington Post) seems like an extraordinary stretch to me.

Perhaps the architects of the Implicit Association Test should develop a new module that would be able to predict individual's views on the usefulness of the Implicit Association Test. If they find that individual's beliefs about the usefulness and explanatory power of the test itself are primarily a reflection of individual subconscious attitudes, then that would be some excellent supportive evidence for their theory. On the other hand, I wouldn't expect to see this test being run anytime soon.


Replies to this post and my original update can be found here and here for anyone who is interested. agrees with me and concludes: "If the Implicit Project can be called science, then it is junk science. Not only is the methodology ridiculous, but the underlying premise behind each test I took was filled with biases that cannot be empirically proven." Another interesting question dealing with the falsifiability of the hypothesis is posed here.

Related Posts (on one page):