“Whose Eyes Are You Going to Believe,” And Whose Version of the Fourth Amendment? :

By Orin Kerr on January 10, 2008 6:54 pm

Dan Kahan, Dave Hoffman, and Donald Braman have posted an interesting new article on Scott v. Harris, the high speed car chase case from last Term: Whose Eyes are You Going to Believe? An Empirical (and Normative) Assessment of Scott v. Harris. The idea behind the article is great. The goal was to test who really shared Justice Scalia’s views about what a reasonable juror would conclude after watching the video in the case. Their conclusion in a nutshell: Justice Scalia was privileging a conservative white male view of the case, substituting a reasonable juror of the Justices’ background for a reasonable juror more broadly.

I was intensely interested to read this article, both because I worked on the case for Scott and because I focused on similar questions of what videos really show in my posts on the Utah tasering incident. Somewhat uncomfortably, I’ve concluded the authors made a significant methodological error in their study. Instead of asking survey respondents to apply the Fourth Amendment test the Supreme Court used, the authors devised several tests of their own for what they thought the Fourth Amendment means (or should mean). They ended up asking the survey respondents to apply the standards the authors suggested instead of the test the Supreme Court used.

My sense is that this methodological move leads the paper to miss its mark. Although its findings are independently interesting, in the end it doesn’t actually say very much about Scott v. Harris‘s view of reasonable jurors. And ironically, what the findings do suggest seem to hint that Justice Scalia’s conclusion about reasonable jurors was very likely correct. I initially raised my concerns with the authors, and after a few e-mail exchanges on these issues I was invited to blog my concerns. So I figured I would.

I. The Case

Scott v. Harris involved a high speed car chase with Scott chasing Harris and eventually bumping in into Harris to get him off the road, and Harris being injured in the crash. The Fourth Amendment issue in the case was whether Scott’s use of force to bump Harris off the road was “reasonable.” According to the Court, the boiled down to how much danger Harris posed to the public. Harris claimed in his complaint that he had posed little danger to the public and had been driving safely, a view that the Eleventh Circuit accepted.

In his majority opinion, Justice Scalia concluded as a matter of law that no reasonable juror could look at the videotape and believe this. To Scalia, the videotape established that Harris was driving dangerously. According to the Scalia, that fact established that Scott’s use of force was reasonable under the following Fourth Amendment rule: “A police officer’s attempt to terminate a dangerous high-speed car chase that threatens the lives of innocent bystanders does not violate the Fourth Amendment.” (slip op, p.13) Because no reasonable juror could conclude that this case was not a dangerous high-speed car chase that threatened the lives of innocent bystanders, the Court ruled that Scott deserved summary judgment.

II. The Study

In their new article, Kahan, Hoffman, and Braman (KHB for short) sought to determine if people from different walks of life really agreed with the majority’s view of the case. They created a survey in which they gave respondents an overview of the facts of the case, showed them the video, and then asked them to answer the following questions.

1. During the pursuit, Harris drove in a manner that put members of the public at great risk of death.

2. During the pursuit, Harris drove in a manner that put the police at great risk of death.

3. It just wasn’t worth the danger to the public for the police to engage in a high-speed chase of Harris when he refused to pull over for speeding. Instead, they should have tried to find and arrest him later.

4. Please indicate how much you think the parties were at fault for the risk posed to the public by the chase: (1) the police were much more at fault than Harris; (2) the police were slightly more at fault than Harris; (3) the police and Harris were equally at fault; (4) Harris was slightly more at fault than the police; (5) Harris was much more at fault than the police.

5. The danger that Harris’s driving posed to the police and the public justified Officer Scott’s decision to end the chase in a way that put Harris’s own life in danger.

1,350 people took the survey and answered these questions, together with answers to questions about their race, gender, politics, ideology, and the like.

III. The Results

There are lots of findings in the paper, but here’s a basic summary. First, large majorities of people were inclined to agree that claims (1) and (2) are true. On a scale of 1 to 6, with 1 being “strongly disagree” and 6 being “strongly agree,” the claims that Harris’s driving posed a “great risk of death” registered an average of 5.06 for great risk of death to the public and 4.67 for great risk of death to the police. There was somewhat of a racial gap on this determination: for example, assessments of whether Harris’s driving posed a “great risk of death” for the public ranged form a 4.56 for black respondents and 5.1 for white respondents. But for the most part respondents agreed that Harris caused a “great risk of death.”

There generally less “agreement with the Court” on the other questions. Most notably, when asked Question 3, whether the chase “wasn’t worth it,” respondents averaged a 3.12 — generally, only “slighly disagreeing” with that statement. And there the racial gap was greatest: African American respondents averaged a 3.89, “slightly agreeing” with the claim, while caucasian respondents averaged a 3.06, “slightly agreeing.”

The real kick of the study occurs later in the paper, when KHB use regression analysis to imagine four different people: Ron, a 47-year old rich white male Republican from Arizona; Linda, a black liberal woman of modest means from the northeast; Bernie, a liberal communitarian professor from the northeast; and Pat, an average American. According to the regression analysis, the differences were relatively modest for questions 1 and 2. 92% of right-wing Rons would agree or lean towards agreeing that Harris posed a great risk of death to the public; 88% of Pats would, as well; 73% of the professor Bernies and 69% of liberal Lindas would have the same reaction. These are differences, certainly, but not terribly stark ones.

On the other hand, the differences among Ron, Pat, Linda, and Bernie became very dramatic when asked to determine if the chase was worth it and to assign relative fault. When asked if the chase was worth it, 83% of Rons would say “yes”; 73% of Lindas would say “no.” 93% of hypothetical Rons would see Harris as clearly more at fault than the Police; in contrast, only 34% of Lindas would have that reaction. Pretty significant differences.

IV. KHB’s Conclusion

The significance of these results, according to KHB: the Justices, who are mostly “Rons”, were privileging the “Ron” view of the world over the other views. In a nutshell, the were looking at the case through the cultural lens of rich white conservative men, and they didn’t realize how a lot of people from different walks of life would look at the case differently.

For the rest of the post, click here.

V. A Methodological Flaw in the Paper?

I think this paper was a brilliant idea. At the same time, I fear it suffers from a significant methodological flaw: The study did not ask respondents to apply the Fourth Amendment test the Court announced — that is, the test that jurors would need to apply if the case had been sent to a jury.. As I read Scott, the only question that matters is whether there was a “dangerous high-speed car chase that threatens the lives of innocent bystanders”. If there was, the use of force was reasonable as a matter of law. Thus, when deciding whether to permit the case to go to trial, the only question the jury would have had to decide was whether there was a “dangerous high-speed car chase that threaten[ed] the lives of innocent bystanders.”

Unfortunately, though, the survey didn’t ask the respondents to answer this question. The closest the survey came was in questions 1 and 2. It asked the respondents separately whether they believed Harris posted a “great risk of death” to the public or a “great risk of death” to the police. But these questions are considerably narrower than the one the Court applied: the Court’s legal test looked for a “threat” of death collectively rather than “great risk” of death to either group individually. That’s problematic, I think, because a person could quite reasonably find that there was no “great risk” to the public nor a “great risk” to the police, but that taken together, there was still a “threat to life” of the police and public together.

Even with that difficulty, the answers for 1 and 2 don’t appear to me to be inconsistent with Justice Scalia’s conclusion. If I’m ready the study correctly, the results indicate a broad agreement with the majority’s view across a wide range of respondents. If you consider that questions 1 and 2 were a more extreme version of the test the majority applied, I don’t think the findings are inconsistent with Justice Scalia’s view of a reasonable juror.

More broadly, the questions that led to the real divides — questions about relative culpability and whether the search was worth it — involved questions far removed from the Fourth Amendment test the Court applied. Under the Court’s Fourth Amendment rule, jurors would not be asked these questions in a trial, and if they relied on such matters, they would do so only improperly. Given that, the existence of divisions on these questions does not shed any light on whether the Scott Court was right to credit their view of the video tape.

Of course, one response could be that perhaps the Court’s Fourth Amendment rule is wrong. That is, the criticism could be that the Justices have views of reasonableness that are different from some members of the public, and that they are wrong to try to create a bright line rule for reasonableness that some members of the public may not share. But that would seem to be a broader criticism of constitutional decisionmaking more broadly — it’s no doubt true that the Justices often balance interests differently than some members of the public would want — rather than a criticism of relying on the video tape for the facts of Scott rather than sending the case to the jury. This argument certainly could be made, but it’s not the target KHB have in mind in their paper.

VI. Conclusion

I think KHB are on to something with their basic methodology. As you might guess from my posts on the Utah tasering incident, I think videos often can be construed in different ways. It’s really important to be aware of that; it’s too easy for us to look at a video and see what we want to see. We need to step outside of our preconceptions and be aware of how other people might construe the facts. That’s the broad point KHB want to make, and on that we are in complete agreement.

On the other hand, to the extent the paper aimed to reveal problems with Scott v. Harris specifically, I think it missed the mark. The study needed either to mirror the legal test in Scott and to criticize Scalia’s assessment of the facts, or else it needed to criticize the legal aspects of Scalia’s Fourth Amendment rule as separate from the factual question. By not mirroring the Court’s Fourth Amendment test, the paper ends up creating some independently interesting results about how different people look at facts. That shouldn’t be overlooked. But those results don’t directly map on to Scott v. Harris because the survey authors assumed a different Fourth Amendment test than the Court used.

Navigation