The Volokh Conspiracy - Citizens, Courts, and Confirmations: Understanding the Mass Public's Attitudes toward Law and Courts.

I greatly appreciate the opportunity Eugene has provided to chat a bit about some research in which Greg Caldeira and I have long been engaged. That work concerns the institutional legitimacy of the U.S. Supreme Court. We have conducted numerous projects over the years, currently have a survey of public opinion in the field connected to the Sotomayor nomination, and recently published a book -- Gibson, James L., and Gregory A. Caldeira. 2009. Citizens, Courts, and Confirmations: Positivity Theory and the Judgments of the American People. Princeton, NJ: Princeton University Press -- on the causes and consequences of the Alito nomination to the high bench. Especially in light of the opening of the Sotomayor hearings, I thought it might be useful to discuss some of our findings.

I'm not so much interested in who is for and against any given confirmation -- although that topic does command a chapter in our book -- but rather I prefer to focus on the institutional legitimacy of the Supreme Court. Please allow me to begin a consideration of why it is important to understand the views of the American people and especially how knowledgeable they are about things judicial. After all, if the American mass public is comprised of nothing but know-nothings then it hardly makes any sense to ask them their views about the Court as an institution and the judges who staff the Court.

Conventional wisdom holds that the American people are incredibly ignorant about the Court. Indeed, just in the last week, C-SPAN published their "Supreme Court Survey" purporting to document how dumb ordinary folks are.

I do not want to spend too much time on technical and statistical matters but I do urge readers to be wary about all survey results. The key indicator of a legitimate survey is the ability to cite a response rate. Most non-scholarly polls cannot, which indicates that deviations -- perhaps even vast deviations -- from random sampling have occurred. Without some semblance of random sampling, one cannot generalize from a poll to the larger population.

The C-SPAN survey shows that only 46 % of the respondents can name a Supreme Court justice. The question asked requires the respondent to pull from thin air a justice's name, and, by the way, that justice cannot be Sandra Day O'Connor or Sonia Sotomayor. According to their poll, Clarence Thomas is the most named justice, selected by 14 % of the respondents. Substantial gender differences exist in the ability to conjure up a name, with 54 % of the men but only 41 % of the women able to do so. As usual, young people score pathetically on this measure.

It turns out that a chapter on our book (also recently published in somewhat different form in the April 2009 issue of The Journal of Politics), also was concerned about Clarence Thomas. However, we helped our respondents by first asking them if they knew if there was an African American on the Supreme Court and then asking them to identify Thomas' name from a list of three judges. The proportion of the sample able to correctly identify Thomas was tremendously larger than that reported by C-SPAN. The reason why is that unaided recall is always different from name recognition. Many who study political knowledge believe the latter is a more valid measure than the former.

In our book we give a simple example of why it is so unimportant to be able to recall justices' names, arguing that it is difficult to envisage an actual political scenario in which it would be necessary for citizens to be able to recall a judge's name without any prompting. Imagine, for example, the following Rehnquist-era conversation among three voters.

Voter 1: The Supreme Court is out of control, what with its decision to give Bush the presidency, the threats to a woman's right to choose whether to have an abortion, etc.

Voter 2: Yes, I agree. I bet it has to do with the Chief Justice, who I think is a staunch Republican.

Voter 1: Yeah, I agree. What is his name? I can't remember.

Voter 2: Neither can I, but the guy ought to be impeached.

Voter 3: I think his name is Rehnquist, Ringgold, or something like that.

Voter1: Maybe. But whatever his name is, we need a new judge on the Supreme Court. I guess I'll have to vote for the Democrats next time so as to get a better Supreme Court.

It is difficult to see that this conversation would be any more politically meaningful were the discussants able to remember the name of the Chief Justice. And, after all, even Richard Nixon, as he was about to nominate him to the Supreme Court, had difficulty remembering the Assistant Attorney General's name, referring to him instead as "Renchburg," at least according to John Dean (2001)!

The C-SPAN poll indicates that fully 76 % of the American people (82 % of men, 70 % of women) know that the justices of the Supreme Court serve a life term (in what is something of a trick question inasmuch as it asks about "the maximum number of years a Justice can serve"). For my money, it is vastly more important for judicial politics for people to know that justices serve a life term than to know that people as obscure as Stephen Breyer, Anthony Kennedy, and John Paul Stevens (named by 1 % each in the C-SPAN poll) sit on the bench. I find the 76 % figure impressive, not depressing, in that it indicates that this information is widespread throughout American society. At the end of the day, a substantial revisionist school is developing among political scientists that argues that the traditional criteria for judging the competence of citizens in a democracy are misguided and that, were more appropriate tests applied, the American people are considerably more competent than most believe.

I am a social scientist who claims little special expertise when it comes to normative arguments. In all of my research my goal is testing hypotheses drawn from theory, and letting the data fall where they may. My objective is always to report findings as I actually see them, even when I might not personally agree with the findings and their implications.

I mention this because vast normative consequences stem from the findings on political ignorance, with those who favor a diminished role for the mass public in judicial politics seizing upon media polls like the C-SPAN polls for ammunition for their position. As one who has published extensively on public views of the judiciary in peer-reviewed outlets, I can well attest to how often reviewers complain about even asking ordinary people questions about issues that ought to belong exclusively to legal experts. Consequently, I believe that findings on public ignorance are normatively satisfying to many and therefore have become conventional wisdom far too easily, without the careful scrutiny social scientists give to virtually all important and policy-relevant empirical findings.

Let me give an example. The American National Election Study routinely and regularly measures political knowledge by asking people to identify the position held by various political leaders, national and international. The question stem reads as follows:

Now we have a set of questions concerning various public figures. We want to see how much information about them gets out to the public from television, newspapers and the like.

In the 2008, ANES asked about "Nancy Pelosi," "Dick Cheney," "Gordon Brown," and "John Roberts."

Imagine how difficult it is to be asked to identify the job of Pelosi, Cheney, and Brown, and then "John Roberts." The context established by the first three questions is explicitly political; then comes the curve-ball (at least for those respondents who do not readily associate the Supreme Court with ordinary politics): John Roberts. As it turns out, in contemporary times, the Chief Justice's name, John Roberts, is a relatively common name, easily confused, for instance, with John Roberts, the host of the CNN show "American Morning." Finally, it is not at all clear why being able to spontaneously name a Supreme Court justice is a useful skill for citizens. As simply a stimulus to measure political knowledge, perhaps "John Roberts" is useful (although the utility most likely also varies over time); as a measure of public knowledge of the U.S. Supreme Court, this variable, especially as it has been coded in the past, is practically useless as an indicator of public knowledge of the U.S. Supreme Court.

So: The first important conclusion that emerges from our research is that the American people are not ignoramuses when to come understanding judicial politics, and, because this is so, it makes sense to query ordinary people about a wide variety of issues of judicial politics.

	[James L. Gibson, guest-blogging, July 20, 2009 at 4:45am] Trackbacks Citizens, Courts, and Confirmations: Understanding the Mass Public's Attitudes toward Law and Courts. I greatly appreciate the opportunity Eugene has provided to chat a bit about some research in which Greg Caldeira and I have long been engaged. That work concerns the institutional legitimacy of the U.S. Supreme Court. We have conducted numerous projects over the years, currently have a survey of public opinion in the field connected to the Sotomayor nomination, and recently published a book -- Gibson, James L., and Gregory A. Caldeira. 2009. Citizens, Courts, and Confirmations: Positivity Theory and the Judgments of the American People. Princeton, NJ: Princeton University Press -- on the causes and consequences of the Alito nomination to the high bench. Especially in light of the opening of the Sotomayor hearings, I thought it might be useful to discuss some of our findings. I'm not so much interested in who is for and against any given confirmation -- although that topic does command a chapter in our book -- but rather I prefer to focus on the institutional legitimacy of the Supreme Court. Please allow me to begin a consideration of why it is important to understand the views of the American people and especially how knowledgeable they are about things judicial. After all, if the American mass public is comprised of nothing but know-nothings then it hardly makes any sense to ask them their views about the Court as an institution and the judges who staff the Court. Conventional wisdom holds that the American people are incredibly ignorant about the Court. Indeed, just in the last week, C-SPAN published their "Supreme Court Survey" purporting to document how dumb ordinary folks are. I do not want to spend too much time on technical and statistical matters but I do urge readers to be wary about all survey results. The key indicator of a legitimate survey is the ability to cite a response rate. Most non-scholarly polls cannot, which indicates that deviations -- perhaps even vast deviations -- from random sampling have occurred. Without some semblance of random sampling, one cannot generalize from a poll to the larger population. The C-SPAN survey shows that only 46 % of the respondents can name a Supreme Court justice. The question asked requires the respondent to pull from thin air a justice's name, and, by the way, that justice cannot be Sandra Day O'Connor or Sonia Sotomayor. According to their poll, Clarence Thomas is the most named justice, selected by 14 % of the respondents. Substantial gender differences exist in the ability to conjure up a name, with 54 % of the men but only 41 % of the women able to do so. As usual, young people score pathetically on this measure. It turns out that a chapter on our book (also recently published in somewhat different form in the April 2009 issue of The Journal of Politics), also was concerned about Clarence Thomas. However, we helped our respondents by first asking them if they knew if there was an African American on the Supreme Court and then asking them to identify Thomas' name from a list of three judges. The proportion of the sample able to correctly identify Thomas was tremendously larger than that reported by C-SPAN. The reason why is that unaided recall is always different from name recognition. Many who study political knowledge believe the latter is a more valid measure than the former. In our book we give a simple example of why it is so unimportant to be able to recall justices' names, arguing that it is difficult to envisage an actual political scenario in which it would be necessary for citizens to be able to recall a judge's name without any prompting. Imagine, for example, the following Rehnquist-era conversation among three voters. Voter 1: The Supreme Court is out of control, what with its decision to give Bush the presidency, the threats to a woman's right to choose whether to have an abortion, etc. Voter 2: Yes, I agree. I bet it has to do with the Chief Justice, who I think is a staunch Republican. Voter 1: Yeah, I agree. What is his name? I can't remember. Voter 2: Neither can I, but the guy ought to be impeached. Voter 3: I think his name is Rehnquist, Ringgold, or something like that. Voter1: Maybe. But whatever his name is, we need a new judge on the Supreme Court. I guess I'll have to vote for the Democrats next time so as to get a better Supreme Court. It is difficult to see that this conversation would be any more politically meaningful were the discussants able to remember the name of the Chief Justice. And, after all, even Richard Nixon, as he was about to nominate him to the Supreme Court, had difficulty remembering the Assistant Attorney General's name, referring to him instead as "Renchburg," at least according to John Dean (2001)! The C-SPAN poll indicates that fully 76 % of the American people (82 % of men, 70 % of women) know that the justices of the Supreme Court serve a life term (in what is something of a trick question inasmuch as it asks about "the maximum number of years a Justice can serve"). For my money, it is vastly more important for judicial politics for people to know that justices serve a life term than to know that people as obscure as Stephen Breyer, Anthony Kennedy, and John Paul Stevens (named by 1 % each in the C-SPAN poll) sit on the bench. I find the 76 % figure impressive, not depressing, in that it indicates that this information is widespread throughout American society. At the end of the day, a substantial revisionist school is developing among political scientists that argues that the traditional criteria for judging the competence of citizens in a democracy are misguided and that, were more appropriate tests applied, the American people are considerably more competent than most believe. I am a social scientist who claims little special expertise when it comes to normative arguments. In all of my research my goal is testing hypotheses drawn from theory, and letting the data fall where they may. My objective is always to report findings as I actually see them, even when I might not personally agree with the findings and their implications. I mention this because vast normative consequences stem from the findings on political ignorance, with those who favor a diminished role for the mass public in judicial politics seizing upon media polls like the C-SPAN polls for ammunition for their position. As one who has published extensively on public views of the judiciary in peer-reviewed outlets, I can well attest to how often reviewers complain about even asking ordinary people questions about issues that ought to belong exclusively to legal experts. Consequently, I believe that findings on public ignorance are normatively satisfying to many and therefore have become conventional wisdom far too easily, without the careful scrutiny social scientists give to virtually all important and policy-relevant empirical findings. Let me give an example. The American National Election Study routinely and regularly measures political knowledge by asking people to identify the position held by various political leaders, national and international. The question stem reads as follows: Now we have a set of questions concerning various public figures. We want to see how much information about them gets out to the public from television, newspapers and the like. In the 2008, ANES asked about "Nancy Pelosi," "Dick Cheney," "Gordon Brown," and "John Roberts." Imagine how difficult it is to be asked to identify the job of Pelosi, Cheney, and Brown, and then "John Roberts." The context established by the first three questions is explicitly political; then comes the curve-ball (at least for those respondents who do not readily associate the Supreme Court with ordinary politics): John Roberts. As it turns out, in contemporary times, the Chief Justice's name, John Roberts, is a relatively common name, easily confused, for instance, with John Roberts, the host of the CNN show "American Morning." Finally, it is not at all clear why being able to spontaneously name a Supreme Court justice is a useful skill for citizens. As simply a stimulus to measure political knowledge, perhaps "John Roberts" is useful (although the utility most likely also varies over time); as a measure of public knowledge of the U.S. Supreme Court, this variable, especially as it has been coded in the past, is practically useless as an indicator of public knowledge of the U.S. Supreme Court. So: The first important conclusion that emerges from our research is that the American people are not ignoramuses when to come understanding judicial politics, and, because this is so, it makes sense to query ordinary people about a wide variety of issues of judicial politics.