"Whose Eyes Are You Going to Believe," And Whose Version of the Fourth Amendment? :
Dan Kahan, Dave Hoffman, and Donald Braman have posted an interesting new article on Scott v. Harris, the high speed car chase case from last Term: Whose Eyes are You Going to Believe? An Empirical (and Normative) Assessment of Scott v. Harris. The idea behind the article is great. The goal was to test who really shared Justice Scalia's views about what a reasonable juror would conclude after watching the video in the case. Their conclusion in a nutshell: Justice Scalia was privileging a conservative white male view of the case, substituting a reasonable juror of the Justices' background for a reasonable juror more broadly.
I was intensely interested to read this article, both because I worked on the case for Scott and because I focused on similar questions of what videos really show in my posts on the Utah tasering incident. Somewhat uncomfortably, I've concluded the authors made a significant methodological error in their study. Instead of asking survey respondents to apply the Fourth Amendment test the Supreme Court used, the authors devised several tests of their own for what they thought the Fourth Amendment means (or should mean). They ended up asking the survey respondents to apply the standards the authors suggested instead of the test the Supreme Court used.
My sense is that this methodological move leads the paper to miss its mark. Although its findings are independently interesting, in the end it doesn't actually say very much about Scott v. Harris's view of reasonable jurors. And ironically, what the findings do suggest seem to hint that Justice Scalia's conclusion about reasonable jurors was very likely correct. I initially raised my concerns with the authors, and after a few e-mail exchanges on these issues I was invited to blog my concerns. So I figured I would.
I. The Case
Scott v. Harris involved a high speed car chase with Scott chasing Harris and eventually bumping in into Harris to get him off the road, and Harris being injured in the crash. The Fourth Amendment issue in the case was whether Scott's use of force to bump Harris off the road was "reasonable." According to the Court, the boiled down to how much danger Harris posed to the public. Harris claimed in his complaint that he had posed little danger to the public and had been driving safely, a view that the Eleventh Circuit accepted.
In his majority opinion, Justice Scalia concluded as a matter of law that no reasonable juror could look at the videotape and believe this. To Scalia, the videotape established that Harris was driving dangerously. According to the Scalia, that fact established that Scott's use of force was reasonable under the following Fourth Amendment rule: "A police officer's attempt to terminate a dangerous high-speed car chase that threatens the lives of innocent bystanders does not violate the Fourth Amendment." (slip op, p.13) Because no reasonable juror could conclude that this case was not a dangerous high-speed car chase that threatened the lives of innocent bystanders, the Court ruled that Scott deserved summary judgment.
II. The Study
In their new article, Kahan, Hoffman, and Braman (KHB for short) sought to determine if people from different walks of life really agreed with the majority's view of the case. They created a survey in which they gave respondents an overview of the facts of the case, showed them the video, and then asked them to answer the following questions.
III. The Results
There are lots of findings in the paper, but here's a basic summary. First, large majorities of people were inclined to agree that claims (1) and (2) are true. On a scale of 1 to 6, with 1 being "strongly disagree" and 6 being "strongly agree," the claims that Harris's driving posed a "great risk of death" registered an average of 5.06 for great risk of death to the public and 4.67 for great risk of death to the police. There was somewhat of a racial gap on this determination: for example, assessments of whether Harris's driving posed a "great risk of death" for the public ranged form a 4.56 for black respondents and 5.1 for white respondents. But for the most part respondents agreed that Harris caused a "great risk of death."
There generally less "agreement with the Court" on the other questions. Most notably, when asked Question 3, whether the chase "wasn't worth it," respondents averaged a 3.12 — generally, only "slighly disagreeing" with that statement. And there the racial gap was greatest: African American respondents averaged a 3.89, "slightly agreeing" with the claim, while caucasian respondents averaged a 3.06, "slightly agreeing."
The real kick of the study occurs later in the paper, when KHB use regression analysis to imagine four different people: Ron, a 47-year old rich white male Republican from Arizona; Linda, a black liberal woman of modest means from the northeast; Bernie, a liberal communitarian professor from the northeast; and Pat, an average American. According to the regression analysis, the differences were relatively modest for questions 1 and 2. 92% of right-wing Rons would agree or lean towards agreeing that Harris posed a great risk of death to the public; 88% of Pats would, as well; 73% of the professor Bernies and 69% of liberal Lindas would have the same reaction. These are differences, certainly, but not terribly stark ones.
On the other hand, the differences among Ron, Pat, Linda, and Bernie became very dramatic when asked to determine if the chase was worth it and to assign relative fault. When asked if the chase was worth it, 83% of Rons would say "yes"; 73% of Lindas would say "no." 93% of hypothetical Rons would see Harris as clearly more at fault than the Police; in contrast, only 34% of Lindas would have that reaction. Pretty significant differences.
IV. KHB's Conclusion
The significance of these results, according to KHB: the Justices, who are mostly "Rons", were privileging the "Ron" view of the world over the other views. In a nutshell, the were looking at the case through the cultural lens of rich white conservative men, and they didn't realize how a lot of people from different walks of life would look at the case differently.
I was intensely interested to read this article, both because I worked on the case for Scott and because I focused on similar questions of what videos really show in my posts on the Utah tasering incident. Somewhat uncomfortably, I've concluded the authors made a significant methodological error in their study. Instead of asking survey respondents to apply the Fourth Amendment test the Supreme Court used, the authors devised several tests of their own for what they thought the Fourth Amendment means (or should mean). They ended up asking the survey respondents to apply the standards the authors suggested instead of the test the Supreme Court used.
My sense is that this methodological move leads the paper to miss its mark. Although its findings are independently interesting, in the end it doesn't actually say very much about Scott v. Harris's view of reasonable jurors. And ironically, what the findings do suggest seem to hint that Justice Scalia's conclusion about reasonable jurors was very likely correct. I initially raised my concerns with the authors, and after a few e-mail exchanges on these issues I was invited to blog my concerns. So I figured I would.
I. The Case
Scott v. Harris involved a high speed car chase with Scott chasing Harris and eventually bumping in into Harris to get him off the road, and Harris being injured in the crash. The Fourth Amendment issue in the case was whether Scott's use of force to bump Harris off the road was "reasonable." According to the Court, the boiled down to how much danger Harris posed to the public. Harris claimed in his complaint that he had posed little danger to the public and had been driving safely, a view that the Eleventh Circuit accepted.
In his majority opinion, Justice Scalia concluded as a matter of law that no reasonable juror could look at the videotape and believe this. To Scalia, the videotape established that Harris was driving dangerously. According to the Scalia, that fact established that Scott's use of force was reasonable under the following Fourth Amendment rule: "A police officer's attempt to terminate a dangerous high-speed car chase that threatens the lives of innocent bystanders does not violate the Fourth Amendment." (slip op, p.13) Because no reasonable juror could conclude that this case was not a dangerous high-speed car chase that threatened the lives of innocent bystanders, the Court ruled that Scott deserved summary judgment.
II. The Study
In their new article, Kahan, Hoffman, and Braman (KHB for short) sought to determine if people from different walks of life really agreed with the majority's view of the case. They created a survey in which they gave respondents an overview of the facts of the case, showed them the video, and then asked them to answer the following questions.
1. During the pursuit, Harris drove in a manner that put members of the public at great risk of death.1,350 people took the survey and answered these questions, together with answers to questions about their race, gender, politics, ideology, and the like.
2. During the pursuit, Harris drove in a manner that put the police at great risk of death.
3. It just wasn't worth the danger to the public for the police to engage in a high-speed chase of Harris when he refused to pull over for speeding. Instead, they should have tried to find and arrest him later.
4. Please indicate how much you think the parties were at fault for the risk posed to the public by the chase: (1) the police were much more at fault than Harris; (2) the police were slightly more at fault than Harris; (3) the police and Harris were equally at fault; (4) Harris was slightly more at fault than the police; (5) Harris was much more at fault than the police.
5. The danger that Harris's driving posed to the police and the public justified Officer Scott's decision to end the chase in a way that put Harris's own life in danger.
III. The Results
There are lots of findings in the paper, but here's a basic summary. First, large majorities of people were inclined to agree that claims (1) and (2) are true. On a scale of 1 to 6, with 1 being "strongly disagree" and 6 being "strongly agree," the claims that Harris's driving posed a "great risk of death" registered an average of 5.06 for great risk of death to the public and 4.67 for great risk of death to the police. There was somewhat of a racial gap on this determination: for example, assessments of whether Harris's driving posed a "great risk of death" for the public ranged form a 4.56 for black respondents and 5.1 for white respondents. But for the most part respondents agreed that Harris caused a "great risk of death."
There generally less "agreement with the Court" on the other questions. Most notably, when asked Question 3, whether the chase "wasn't worth it," respondents averaged a 3.12 — generally, only "slighly disagreeing" with that statement. And there the racial gap was greatest: African American respondents averaged a 3.89, "slightly agreeing" with the claim, while caucasian respondents averaged a 3.06, "slightly agreeing."
The real kick of the study occurs later in the paper, when KHB use regression analysis to imagine four different people: Ron, a 47-year old rich white male Republican from Arizona; Linda, a black liberal woman of modest means from the northeast; Bernie, a liberal communitarian professor from the northeast; and Pat, an average American. According to the regression analysis, the differences were relatively modest for questions 1 and 2. 92% of right-wing Rons would agree or lean towards agreeing that Harris posed a great risk of death to the public; 88% of Pats would, as well; 73% of the professor Bernies and 69% of liberal Lindas would have the same reaction. These are differences, certainly, but not terribly stark ones.
On the other hand, the differences among Ron, Pat, Linda, and Bernie became very dramatic when asked to determine if the chase was worth it and to assign relative fault. When asked if the chase was worth it, 83% of Rons would say "yes"; 73% of Lindas would say "no." 93% of hypothetical Rons would see Harris as clearly more at fault than the Police; in contrast, only 34% of Lindas would have that reaction. Pretty significant differences.
IV. KHB's Conclusion
The significance of these results, according to KHB: the Justices, who are mostly "Rons", were privileging the "Ron" view of the world over the other views. In a nutshell, the were looking at the case through the cultural lens of rich white conservative men, and they didn't realize how a lot of people from different walks of life would look at the case differently.