The John G. Roberts Umpire Watch:
Is Chief Justice John Roberts an "umpire," a "servant of the law" who merely "applies the law" rather than "makes the law" to his personal preferences? Or is he a political conservative who will always vote for the conservative cause in an ideologically-charged case? In his confirmation hearings, Roberts expressed hope that he would be an umpire. In his cases Last Term, however, Roberts tended to vote consistently for the conservative side.
In light of the uncertainty, I propose a new feature here at the Volokh Conspiracy: The John G. Roberts Umpire Watch. When a new decision comes down, we'll make a judgment call about whether it was one of the Court's ideologically-charged cases. (This will require umpire-like judgment, perhaps, but I think we can probably identify most of the cases relatively straighforwardly based on how most of the Justices voted.) If it was, then we'll classify Roberts' vote as either politically conservative or politically liberal.
Then, at the end of the Term, we'll tally up the numbers. If we see votes on both sides, it will support the case that Roberts is an "umpire." Someone truly following principle will go wherever that principle will take him, and that should led to mixed views from a political standpoint. On the other hand, if all the votes end up being conservative, that will support the notion that Roberts is voting to make the law to shape his personal preferences rather than merely following it.
I should add: Yes, of course, this kind of vote-tallying is imperfect. There are some circumstances in which a neutral umpire would always vote a particular way. I'm reminded of the pro se appellant who argued that the trial judge was obviously biased: after all, the judge had denied 100% of his motions! Still, I think that's less likely given the wide mix of cases the court hears. Even if a particular legal principle has a political valence, the mix of cases on the Court's docket is so diverse that few principles explain more than one or two votes. Given that, I think the tally should give us interesting insights into how Chief Justice Roberts approaches his job.
In light of the uncertainty, I propose a new feature here at the Volokh Conspiracy: The John G. Roberts Umpire Watch. When a new decision comes down, we'll make a judgment call about whether it was one of the Court's ideologically-charged cases. (This will require umpire-like judgment, perhaps, but I think we can probably identify most of the cases relatively straighforwardly based on how most of the Justices voted.) If it was, then we'll classify Roberts' vote as either politically conservative or politically liberal.
Then, at the end of the Term, we'll tally up the numbers. If we see votes on both sides, it will support the case that Roberts is an "umpire." Someone truly following principle will go wherever that principle will take him, and that should led to mixed views from a political standpoint. On the other hand, if all the votes end up being conservative, that will support the notion that Roberts is voting to make the law to shape his personal preferences rather than merely following it.
I should add: Yes, of course, this kind of vote-tallying is imperfect. There are some circumstances in which a neutral umpire would always vote a particular way. I'm reminded of the pro se appellant who argued that the trial judge was obviously biased: after all, the judge had denied 100% of his motions! Still, I think that's less likely given the wide mix of cases the court hears. Even if a particular legal principle has a political valence, the mix of cases on the Court's docket is so diverse that few principles explain more than one or two votes. Given that, I think the tally should give us interesting insights into how Chief Justice Roberts approaches his job.
Perhaps a better exercise would be to analyze the decision and see if it actually merely applies the law and tally up votes like that.
No, I'm not conflating the two: indeed, I devote the entire last paragraph of my post to that precise issue.
The primary difficulty with your recommendation is that you really need to spend a lot of time becoming an expert in an area to know which side best applies the law. Unfortunately I don't have time to do that for every case.
Because we have no good baseline of what is a "correct" or "neutral" decision, the baseline that we refer to by default is the median Justice, Justice Kennedy.
Of course, we might see whether Roberts is intellectually consistent. That is, if he believes in free speech in one case and not in another, there might be a problem. But any skilled lawyer can simply recharacterize the level of generality to form a neutral sounding principle. For example, a rule that "ambiguity is construed in free speech's favor unless the speaker is talking about 'bong hits for Jesus'" is a perfectly neutral rule and internally consistent from one point of view. Who is to say that Roberts is incorrect and not "umpiring' to adopt it?
Niiice.
Did Roberts vote “liberal” in any of the controversial cases last term? I can’t think of any.
What does it mean for the exercise to fail?
Wow, I hope you don't build any bridges anytime soon using that awful type of logic.
It reminds me of American Textile Manufacturers Inst. v Donovan (45 U.S. 490), where Brennan, never a source of much logic, decided that where a study commissioned by an agency grossly overestimated the costs of a modest rule change, then that study was appropriate to use as a guide for a more strict rule change. Yeah, if Brennan built a bridge, I'd recommend the ferry.
But I guess that's what suffices for logic at the highest levels of our nation, so I guess it comes honestly among lawyers.
Why? Someone following principle will go wherever that principle will take him, period. I do not see what logic supports additional belief that this will lead to mixed political views.
1. What the "correct" principle of law is.
2. What Roberts believes the correct principle of law is.
3. Whether Roberts applies what he believes to be the correct principle of law, or applies some other principle because of his personal policy preferences.
(1) is basically impossible to discern, for obvious reasons.
Now, if (2) deviates from (1), that is not probative of bad faith, but shows that Roberts would be a bad justice. But, since (1) is impossible to discern, it is likewise impossible to prove that (2) deviates (1).
What Orin is trying to discern is whether (3) deviates from (2). That is, if Roberts believes that the law is X, but decides the law is Y because it leads to a preferable outcome, then he is not being an umpire.
The problem here is that we have no way of disentangling (2) from (3). Short of Roberts writing in his diary that "I thought the correct rule was X but I decided by impose Y by fiat instead," we have no way of discerning a divide between (2) and (3). Roberts may vote very conservatively because he believes the law is very conservative.
What measures of justices' votes normally do is conflate (1)-(3). That is, we assume that justices believe the correct law to be what the author believes the correct law to be. The argument then goes that a decision that does not follow this correct law is either because the justice is incompetent (a deviation of 2 from 1), or acting in bad faith (a deviation of 3 from 2), or both. But unless we have a measure of either (1) or (2) independent of (3)--that is, unless we can objectively determine what the correct law is, or what a justice believes the correct law is, independently of how they actually vote, there is no point of reference to make a comparison.
Orin's exercise is to place the median justice as the definition of (1). That is, the political center will define what the correct law is. But that underlying premise is highly debatable.
Your exercise measures political outcomes, not proper application of the law. While too many judges and justices coflate political outcomes with proper application of the law, these are two very different things.
You are also erroneously coflating Justice Roberts giving a roughly equal number of liberal and conservative political outcomes with "neutrality" (i.e. having no political opinions) when in fact such a result is more indicative of simply having political opinions on both sides of our ideological divide.
Isn't your second paragraph a direct contradiction of your first?
I think that the 'correct' decision for most Constitutional issues would turn out to be politically 'conservative' by today's standards, with the exception of the 'war' on terror, the 'war' on drugs, and other issues involving a degree of federal authority that the Founders would have shuddered at.
Like I said, please don't build any bridges!
If you can't see how using a method that has no basis in logic or reason can't return a valid result, then you certainly belong in law, not civil engineering because only a lawyer would think that garbage in equals pearls out.
For example, I think a general conservative opinion is more or less: "people get money for ridiculous pseudo-injuries from corporations that aren't at fault; these judgments need to be further restrained." The corresponding liberal position would be something like: "corporations often try to cheat individuals with fewer resources; we should make sure the lawsuits by cheated individuals don't get thrown out just because high-paid lawyers come up with silly technicalities." (Obviously, beliefs on both sides are in fact more complicated.) Everyone would agree that there are some legitimate and some illegitimate lawsuits: the question is how many of each.
So a biased, political conservative judge might just side with the defendant every time, in order to promote his political cause (that's how elected politicians usually act), whereas a conservative "umpire" would occasionally decide "here's one of those rare cases where the plaintiff has a point."
Of course, no one's perfectly pure or utterly biased, so it'll be hard to measure, but Orin's procedure sounds like it could be useful.
You are a member of the good-ol-boys club. Nothing more.
For someone with such a commitment to logic and reason, you seem unusually willing to simply assume your conclusion. (And besides, I earned an A+ in my mechanics of solids class, so I think my bridges are probably staying up. ;-) )
How would you classify Wisconsin Right to Life? I saw most media classifying the result as "conservative," either because it was a win for a Right to Life group, or because it was portrayed as a win for "corporate" contributions to campaigns.
But wasn't free speech once considered liberal?
I know, many people say campaign finance is different, but those critics should take a look at the amicus list in WRTL. WRTL was supported not only by those crazy right-wingers at the ACLU and the AFL-CIO, but also by the Alliance for Justice! So the leading Left-side group in the confirmation wars was supporting the view ultimately adopted by several Justices whose confirmations they opposed!
I don't see how something chalked up as a "win" by the Alliance and the AFL-CIO can be classified as politically "conservative."
Same thing goes for many of the speech cases, such as Hill v. Colorado, which upehld anti-abortion speech restrictions, or Bong Hits, in which many conservative groups backed the student (because conservative students are often the ones in trouble with principals).
Also, State Farm and all the punitive damages cases are fun tests — Stevens backs corporate America against injured plaintiffs, and Scalia and Thomas are on the other side? So Roberts might get tagged with a politically conservative tally for joining Stevens over Scalia?
So how would you classify such cases?
Please explain the following:
(1) "When a new decision comes down, we'll make a judgment call about whether it was one of the Court's ideologically-charged cases. (This will require umpire-like judgment, perhaps, but I think we can probably identify most of the cases relatively straighforwardly based on how most of the Justices voted."
Please explain how this calculus will work precisely -- for example, if Scalia and Thomas vote on one side, does that make it "ideologically charged," for you? Otherwise, isn't this some sort of ad hoc after the fact conclusion that will be manipulated by personal values?
(2) "If it was, then we'll classify Roberts' vote as either politically conservative or politically liberal."
Please explain (a) how you'll classify the result; and (b) how you'll determine whether the decision should have been that way or not.
(3) "Then, at the end of the Term, we'll tally up the numbers. If we see votes on both sides, it will support the case that Roberts is an "umpire." Someone truly following principle will go wherever that principle will take him, and that should led to mixed views from a political standpoint."
Please explain how you justify assuming (a) the "proper" distribution of judgments ex ante; (b) the percentage you think an "umpire" would have on each side (liberal/conservative); and (c) what acceptable deviation, if any, you think there should be.
I'm interested in your responses, particularly to (3).
If you can't answer in a principled way, please explain what your exercise will show.
Wanna take odds that he can't give it up?
I will admit that there's a problem in how to characterize decisions, and that "liberal vs. conservative" probably isn't the best way to break things down. So maybe we won't find out anything useful if we call decisions "conservative" or "liberal."
But let's stick to the umpire analogy. There are home plate umps famous for giving pitchers the high strike, there are home plate umps know as having a tight strike zone, etc. His claim seems to be that he's not a results-oriented jurist; he just calls 'em as he sees 'em. So if we perceive a bias toward calling the low and outside pitch a strike, well, then we've learned something interesting.
Obviously, the assumption may be incorrect, in either direction. However, there is no neutral definition of "correct" decision. There is much disagreement, often characteristic of the evaluator's politics. I would say an assumption that the "correct" outcomes are equally distributed is certainly as reasonable as an assumption that the "correct" outcomes are what a given evaluator thinks correct.
However, step one should be made ex ante, not after viewing how the justices voted.
That we learn nothing useful from the data.
Orin hasn't (yet) defined his criteria as precisely as Shapiro did, but there is a general precedent for his methodology.
If Goldstein is right, this term is not a good year to perform Orin's experiment. Unless, of course, the point is to get a "misleading sense of direction."
I think you are running into some objections by people who think the results will undermine their pre-conceived opinions about Chief Justice Roberts, or who don't want to know the results of your imperfect poll because it might cause others to know this information about him. I see no harm in forging ahead with the study, and then we can see if it provides useful or inaccurate/unhelpful information. This is similar to what Sunstein did, except he focused on the Court as a whole in trying to determine who were the most "activitist" justices.
I disagree with this analysis, because it is an election year. We already know, as Orin acknowledges, that last year "Roberts tended to vote consistently for the conservative side." If that changes this year, it does not necessarily support the conclusion that Roberts has become an "umpire." It equally supports the conclusion that he has an eye on the election.
Actually I think what Prof. Kerr is running up against is the fact that this exercise says more about the evaluators than the person being evaluated. If [name your favorite conservative] does the evaluation, it makes Chief Justice Roberts look like a principled jurist. If [name your favorite liberal] does it, it makes Chief Justice Roberts look like an ardent conservative, principles be damned.
The reverse would be true for Justice Ginsburg
As an analysis of the VC readers, it might be an interesting exercise. As an analysis of CJ Roberts, it probably won't be.
Do it for everyone. I think it would be interesting. My predictions, for left to right: Stevens, Ginsburg, Breyer, Souter, Kennedy, Alito, Roberts, Thomas, Scalia.
Why use the qualifiers "tended to" and "consistently"? When did he not vote for the "conservative side"?
A. Claims a purely textualist/originalist basis?
B. Seems primarily a matter of following precedent?
C. Favors the more powerful?
D. Favors feds/executive/police?
This would be similar to evaluating whether a baseball umpire calls the top and bottom and two sides of the strike zone, for lefty and righty hitters, the same for two teams. I think this would work better, but cannot name the fancy maths that we would use.
You could do everybody, but we'd be holding some up to a test they never asked for. My poorly informed understanding is that neither Stevens, Breyer, Kennedy, nor Alito ever claimed to be baseball umps.
Yes, but the umpire quote is ambiguous at best. A neutral judge might vote 65% conservative or liberal and still be neutral. Plus, it gives us context.
Moreover, this informal study could be used to counter Sunstein's poorly designed agency intensive version.
I also propose eliminating 9-0 decisions from scrutiny. These should be viewed as so clear that there is no ideological tilt to them. It make no senese to call a 9-0 decision conservative because it rejected a bad liberal argument.
Remember, Professor Kerr is not saying that HE will decide which cases are political and which outcomes were conservative or liberal. He was stating that "we" will, presumably the VC readership.
Indeed, Orin's project speaks to what I like to call the "Great Unspoken Assumption" of U.S.
Supreme Courtjudicial confirmation politics at all levels, one that seems to be shared by liberals and conservatives alike: namely, that strictly applying the law and the Constitution, and eschewing judicial activism, tends to result in a jurisprudence that is, on the whole, more favorable to conservatives than to liberals. This Great Unspoken Assumption implies that there's really no such thing as a "neutral umpire", as being neutral in a legal sense naturally skews toward conservatism in an ideological sense.Why would conservative politicians, and the conservative constituencies who elect them, so enthusiastically favor "neutral umpire" judicial nominees, as opposed to more overtly conservative activist ones, if they didn't believe legally neutral judges and justices could be counted upon to deliver generally ideologically conservative results?
Please explain why you need explanations on those items.
I think Annoymouseducator gets the point.
How so?
My first paragraph argued that political outcomes are not the same as the proper application of the law.
My second paragraph observed that your measurement of "neutrality" (not applying your own opinions to a decision) is not the same as applying your own opinions to decisions to arrive at a roughly equal number of "liberal" and "conservative" political outcomes.
These paragraphs do not address the same issues. I do not see how they contradict one another.
You may be employing a well-established methodology for assessing jurists, but is a parallel method used to assess umpires?
If everyone else jumped in front of a truck, would you jump too?
I'm not a conservative, I don't know anything about Roberts, and I don't know of any other people using this methodology. But that others use it doesn't make it right.
You seem to imagine that I argued, "my methodology is okay because it has been used before." Read my comment again, and you will see that I do not make that argument.
At the end of a baseball season, should we determine whether or not the umpires were neutral or biased based on how close they come to calling 50/50 on balls and strikes? Or do I misunderstand your methodology?
You present two bits of information:
1) Roberts consistently stands opposed to Stevens, Ginsburg, Souter, and Breyer.
2) Roberts claimed to be an umpire, not a ideologue
From which you then conclude, "#1 implies #2 is false"
What an assumption. We ought to consider that BGSS are the four horsemen of the left. Thus #2 implies the BGSS are not moderates but actually ideologues. Thus, #1 is irrelevant.
What I'm wondering is: being a rather well connected and aware law professor why are you falling for this tripe?
Or are you being open-minded to those left leaning people within your readers who persistently nag you into abandoning reason by insulting your integrity?
I don't think Orin said anything about expecting 50/50 results. But without having an exact sense beforehand of how often conservatives are right, we can still get something from the information. If you found an umpire had called, say, 99% strikes all year, you'd probably suspect he wasn't being fair; even more so if you found he had called 99% strikes only when the Yankees were pitching.
On most cases, the court is unanimous or close to unanimous. The 7-2 and 9-0 cases sometimes have "liberal" results and sometimes have "conservative results. Some of them may be politically charged, but the answers are clear enough, and all 9 of the Justices probably come out of those looking like umpires.
In the politically charged cases, though, that present tough and relatively open questions, on the margins conservative judicial philosophies tend to correlate with conservative results and liberal judicial philosophies tend to correlate with liberal results. Criminal defendants and individual plaintiffs need judges who will ignore the text of the laws and do "justice" in light of the noble purpose that they presume is there. Large corporations get their legislation through Congress through bribes, apathy, or lobbying, and then they look to textualist judges to enforce that legislation to the letter. Judges who want to know what "equal protection" and "cruel" meant in 1868 and 1787 are going to be less likely to scale back the death penalty and recognize novel forms of discrimination than judges who want to know what they can get away with making up in light of the current fashion among elites.
There are exceptions, of course. The Apprendi/Blakely line of cases produced some interesting bedfellows, as did Kelo, and the Scalia/Stevens dissent in Hamdi was quite impressive (and perhaps umpire-like).
Given the court's docket, and given the sorts of cases you will probably label as "ideologically charged," JGR will probably come out looking like an "ideologue" under your criteria. I don't think it follows that he's unprincipled or not an "umpire" somehow.
I hate to bring in a paraphrase of the "Clinton did it" retort, but I don't suppose that Scalia reading his hateful Lawrence dissent from the bench was "melodramatic histrionics used to deceive the public into believing that Justice Kennedy is going to personally start marrying gays"?
My problem with the exercise is not with its methodology, but that it proposes to draw conclusions about CJ Roberts' jurisprudence based on a single term, which also happens to be an election year. The fact is that Roberts has been consistently conservative for two terms. If he is less conservative this year, that will not provide strong evidence that he is not a conservative. Again, I refer to Tom Goldstein's article, which based on an analysis of the cases already granted this term predicts that
I will stop short of suggesting that this exercise is being proposed in order to generate such a "surprising" result. But if Goldstein -- probably the single most prominent student of the Supreme Court in the country -- is to believed, doing this experiment this term will lead to misleading results.
There's no reason this would only be done for one year. It only occurred to me this year because last year Roberts' votes were so surprisingly predictable.
Scalia's Lawrence dissent is indeed another example of melodramatic histrionics. My point isn't not "oooh Ginsburg was deceptive". My point is that reacting to what happened last term as if it informs our knowledge of who Roberts is--is silly.
The horseman had their theatrics just like Scalia did (for the same reason).
Orin has even discussed this:
http://volokh.com/posts/1193341791.shtml#285331
Maybe they learned from each other. Either way: The strong split between Roberts and the four horsemen isn't meaningful. BGSS aren't playing fair, if they seem 'moderate' its only some kind of red-shift effect of the course moving away from them. But they aren't moderate.
Ergo, their consistent 5-4 siding against the Roberts is a sign of their ideology not of Roberts bias.
If someone would just stipulate that "umpire" is one of those things you say in order to get confirmed, we could probably dispense with the whole operation.
Sure they're moderate, and only someone with a very limited perspective of a few years or a very short memory would suggest otherwise.
I'm curious: What's your test for determining if a conservative is just voting for whatever position is conservative, as opposed to being principled? Is your view that there is no distinction, as anyone who is principled will always, 100% of the time, vote in a conservative direction? I mean really, 100% of the time, with no exceptions, ever, under any circumstances?
You win the lottery jackpot. You are then charged with having cheated, for instance with having bribed lottery officials. At the trial, the prosecutor points out that winning the lottery without cheating is extremely unlikely, and that therefore your being innocent must be comparably unlikely.
CJR rules consistently for the conservative side. He is then charged with being unprincipled. Democrats point out that consistently ruling for the conservative side while remaining principled is extremely unlikely (but not impossible); and therefore the chance of CJR being principled must be comparably unlikely.
(1) Choosing which data you're going to analyze after you've collected it and looked at it is not a "well-established methodology" anywhere outside of parapsychology research.
You have got to decide if a case is political before the ruling comes out or all you are going to measure is your own unocnscious biases.
(2) By assuming that a neutral umpire would rule for liberals as often as for conservatives you are building a tendentious conclusion into your study's methodolgy. I know that doing so is a "well-established methodology" in the social "sciences," but I still don't like it.
Neither your nor Cass &Sunstein's studies would mean anything if it were in fact true that liberals are more likely than conservatives to bend the rules to get the policy outcome they want. And how else can one explain Roe vs. Wade?
Maybe so.
So what?
We're discussing Roberts, not Scalia.
Got an example of Roberts doing it?
Maybe both sides are raising objections because such a badly designed study is a loose cannon that will only add noise and random damage to the debate.
A. Claims a purely textualist/originalist basis?
B. Seems primarily a matter of following precedent?
C. Favors the more powerful?
D. Favors feds/executive/police?
What you're trying to do is meaningless, based on nothing, to get a result that is misleading. You might as well just come to the conclusion that you want to reach now, because it's just as valid.
The fatal assumption is in assuming that "applies the law" will result either result in decisions that are politically neutral, or will result -- on average -- in just as many decisions that are politically liberal as decisions that are politically conservative. If that were the case, "applying the law" versus "making the law" would not be such a political touchstone for conservatives.
Not to in any way denigrate Justice Roberts, but his response (however the question was framed) avoided the question nicely, in a way that would sound good to anyone who was politically conservative. Saying that he would "apply the law" rather than "vote conservative" was just doublespeak for saying that he would "vote conservative." Or at least that's the way it sounded to this conservative.
Incomplete information is the norm in most fields of scholarship, and is generally better than nothing.
Skyler's objection is much more on point: the question is not whether justices are principled but whether their principles are good ones. Trying to determine observationally what those prinicples are is good idea. But you have to do it well. If you start out measuring only liberal/conservative, then the answer you get will of course be one of "liberal, moderate or conservative" and you will have learning nothing about "principle". At best you will measure the correlation of Roberts' position on the liberal/conservative scale with that of the court as a whole. How that relates to "principle" is a theoretical question whose answer less than obvious.
Kerr, please take the word of a guy who spends his days staring at reams of manufacturing data trying to figure out what parameters matter and why - this is an interesting idea, but it needs a lot of work.
As a lawyer and a long-time basketball referee (and occasional rec league umpire), I think you misunderstand how to determine if a referee/umpire is biased. The most important thing for an umpire is to be consistent in making judgment calls. To use the balls/strikes analogy, the key is for the umpire to make the strike zone as consistent as possible so that all the players know what is or is not a strike. That strike zone might favor one pitcher over another, or one team's hitters over the other's, but we wouldn't consider that to be a sign of a biased umpire. If the umpire changes his strike zone to favor or disfavor a particular team, or if the umpire ignores rules to assist a team he prefers, then we can say the umpire is biased. Simply counting up the balls and strikes for each team is irrelevant to determining bias, unless you can review each ball/strike call on its own merits (or in basketball, coaches/fans often incorrectly focus on total foul counts to show bias, when in fact you would need to review each call/no-call on its merits to detect bias; the referee could call fouls consistently in a manner that happened to favor one team's style of play, but yet was not biased).
Applying this analogy to the court, CJ Roberts could set a consistently conservative "strike zone" for his interpretation of laws or for his application of Constitutional principles, resulting in his vote being consistently "conservative"; this would not necessarily mean CJ Roberts is more or less biased than any other Justice. Actually, in some ways, a consistent conservative approach from CJ Roberts is more "umpire-like" than the "case by case" vacillation from some of the "centrist" judges (Justice Kennedy and ex-Justice O'Connor, perhaps?).
That all being said, I think the umpire analogy has a certain shallow appeal, but it really doesn't hold up too well to deeper application. After all, umpires and referees don't write the rules they are interpreting, and have to answer to supervisors for bad calls or incompetence.
If it is the case that a true umpire would NOT always vote conservative in ideologically-charged cases, then we could compare the Chief Justice's practice with that of a true umpire by taking account of whether he ever voted against his political preferences.
But even then, there are several problems. First, when the Chief Justice fails to vote "conservative," he may still be voting on personal preference: people (even Justices) are complex, and even the most "conservative" Justice may disagree with the party line on some issue. Second, as pointed out above, the Chief Justice may believe that the conservative answer IS the law. He may even be right. Thus, he could be a true umpire even voting 100% conservative in ideologically-charged cases. In fact, I think that even if any observer would posit that the Chief Justice is simply applying his own policy preferences, the Chief Justice would nonetheless believe that he was consistently applying the law.
At the end of the day, it's very hard to know what we could even mean by "applying the law" v. "making the law," since so much of the Court's job is to "make" law. If one takes seriously the idea that circuits split (at least sometimes) because equally good answers are available to a given legal question, then it's to hard give any account of what the Court does that doesn't involve policy.
The Bush Administration intervened on Medellin's side. While I realize that this case is a classic "power of the federal government/power of the states" case, can anyone elucidate which vote Roberts might cast would be ideological and which one would be principled?
Here's a more rigorous fact. The Penn Law Review compared the ability of a statistical program to predict votes (that program used just a couple of variables, most importantly justice ideology) and legal experts predicting votes. The statistics did somewhat better. Ideology, whether measured by press assessments or past ideologically consistent voting patterns is demonstrably the best predictor of Supreme Court votes.
Isn't the fallacy obvious? Someone is going to win the lottery, even though it's very unlikely that it would be any specific person. But that doesn't mean there's always going to be some Supreme Court Justice who votes for the conservative side every time.
Why exclude 9-0 decisions? Why not 8-1? And would your answer change if it were, mutatis mutandis, a death penalty case and former Justice T. Marshall was the "one"?
Can someone at least give an actual example of a case (even on the DC Circuit) where JGR has violated the "umpire" test so that we can at least start to evaluate the methodology?
Hmm. Quite a few people don't believe either of those claims.
Fair enough. There are two things I am unclear about, however. First, is the claim that an ex ante analysis is more reliable than an ex post analysis? I do not think that is the true in this context; any selection bias comes from the Court's own certiorari process, not the study design (which can include all cases decided by the Court). Second, is the claim that our "judgment call" about whether a case is "one of the Court's ideologically-charged cases" is more accurate than objective measures like voting patterns? I would question that claim. It seems to me that voting patterns are a more accurate indicator of genuine controversy than the judgment call of Volokh Conspiracy readers.
Posit a test like the one proposed by Kerr, except that instead of an ad hoc determination of "ideological" cases, we adopt a strict definition: a case is an ideological case if it is a five-four decision in which Breyer, Souter, Stevens, and Ginsburg are on one side.
Roberts is a "good umpire" if these cases came out fifty-fifty, and a "bad umpire"/"lying hypocrite" if he is consistently on the anti-BSSG side.
Anybody spot the problem with this? Does the term "selection bias" jump unbidden to your mind? Do we have any reason to think that an ad hoc "ideological case" filter will remedy the problem?
Therefore an experiment intended to prove or disprove the above hypothesis would be extremely useful.
But an experminent whose design assumes an answer is totally useless.
Kerr has implicitly assumed that the above is false (an unbiased umpire should rule both ways equally often). If he finds Roberts ruling "conservative" more than half the time he has learned nothing beyond the fact that Roberts rules conservative more than half the time. Any conclusion that Roberts has a conservative bias would come not from the data but from Kerr's implicit assumptions.
Now if Kerr took Bittern's suggestion of scoring "textualism" and "precedent" for various decisions, and measured the degree to which Roberts' preference for textualism over precedent or vice versa depends on whose ox is being gored, then we might actually learn something.
But if all you measure is "liberal" vs. "conservative" you don't have enough degrees of freedom to draw any conclusions about anything else.
Kerr initially said he was going to select what he considered "politically charged" cases, which would have to be done ex-ante to avoid Kerr injecting his own selection biases. If he does all cases, or defines an objective procedure for deciding which cases he analyzes (e.g. number and indentity of amicus briefs) he can safely do an ex post analysis.
The certiori problem is unavoidable.
Please explain why you need explanations on those items.
Orin:
With regard to (1), without your explanation, I have no way to understand whether you have any principled basis whatsoever to decide if a case is "ideologically-charged." Further, without an explanation, there is little way to know if your methodology is sound.
With regard to (2), if you can't explain how you will classify Roberts' vote, then it seems likely that your methodology is unsound.
With regard to (3), if you can't explain (a) the "proper" distribution of judgments ex ante; (b) the percentage you think an "umpire" would have on each side (liberal/conservative); and (c) what acceptable deviation, if any, you think there should be, then you have apparently conceded the futility of your exercise and its lack of a supportable methodology.
If you think I'm off, I asked you to explain what you think you your exercise will show. You didn't. I'd love to hear an explanation that justifies it despite an inability to answer the preceding queries.
But being smart doesn't make you right. Einstein believed in a god. He was very smart, but that didn't mean that his conclusion was improved by his intelligence. It was simply a belief that he held absent concrete proof. Many might agree with him that there is a god, but they're still wrong, no matter how smart.
Likewise, you may think that this methodology will yield results that have meaning. It is based on nothing but pure hope on your part, not on any basis in reality.
Here, you're assuming that errors that need to be corrected by the Supreme Court will be evenly biased either for one political spectrum or another. It doesn't take into account any potential trend of the court to correct decades of influence by the one political philosophy. You're assuming that lower courts errors will be evenly biased and that the courts will select errors to be corrected in an evenly biased manner.
Most specifically, without principled answers to the preceding queries, I do not think that your assessment
will "give us interesting insights into how Chief Justice Roberts approaches his job," as you claim. Rather, I think it will provide unimportant and uninsightful answers to queries that the Harvard Law Review already answers in each annual issue, i.e. with what judges does Roberts typically vote. That is a mundane assessment of little value in determining (beyond the obvious numbers) how Roberts approaches his job.
On the negative side, it would seem like fodder for generalizations of little value (he's a "conservative" in voting cases, a "liberal" in ERISA cases, etc). Such generalized assessments will come in time anyway from general conversation. Academic review, I thought, was supposed to generate better insight.
If your study can't answer the equivalent question, what's the point?
Orin, clearly you're smart
But he's also clearly not a statistician, and seems unwilling to consider the possibility that he's out of his depth.
Also, the question isn't whether Roberts is following principle. The question is whether, as he claims, he is simply following the law. As I've said before, his claim is absurd and misunderstands what the Supreme Court does.
You both sincerely puzzle me. Davide, your request is rather absurd. All I have done is announce that I plan to conduct a study; you respond by insisting that you must hear all of the details about the study before it conducted. How strange.
Skyler, you seem blissfully unaware that I am way ahead of you on this; you repeatedly accuse me of taking positions that I have never taken (indeed, I have expressly disavowed, repeatedly), and yet you never seem to back down.
More broadly, both of you are missing the main point here. I am not arguing that a 50/50 distribution proves neutrality. Rather, I'm suggesting that evidence that someone votes 100% in one direction 100% of the time may provide support for the view that this person is probably not being 100% neutral.
I'm having trouble making anthing meaningful of this distinction, given that the number's probably going to be somewhere in between. If you find that Roberts votes "conservative" 60% of the time, what if anything will that mean?
Given my belief that one of the parties has systematically and deliberately twisted and ignored the law over the last four decades, and that the Supreme Court should spend the next few years reversing a huge backlog of dumb decision, I do not think it's reasonable to so assume.
Therefore showing that Roberts votes in a way that favors the party I consider more honest (and that I recognize many other people don't) will supply no information to settle the difference in opinion between us.
So, anything short of 100% conservative and he's an "umpire"? This makes your study even more sensitive to ex-post selection bias than I thought.
It's not intended to settle your differences.
Huh?
Well such a person would be principled, but I wouldn't regard their positions as being internally consistent--in particular the welding of classical economics with progressive moral intervention seems strained to me.
Perhaps to satisfy your intellectual curiosity you should focus on when Roberts ends up in the minority.
While I appreciate the fact that you responded, I fear that I do not comprehend what you just wrote.
So how about it, Orin - why don't you post a list of the cases accepted so far this term and tell us (a) whether the case is "ideologically-charged" and (b) which votes are conservative or liberal?
"someone votes 100% in one direction 100% of the time may provide support for the view that this person is probably not being 100% neutral"
The problem is that you have to define "neutral" somehow, and right now you are dodging that question. Right now, you are implicitly defining "neutral" as "not 100% conservative"; but you can't justify why you think the 100% conservative platform is "incorrect" or "unprincipled" in some sense.
Huh?
Kerr said what he thinks it does or does not mean if Roberts votes exactly 50% conservative, and what he thinks it means if Roberts votes exactly 100% conservative. I asked him what it would mean if he voted somewhere in between, using 60% as an example.
Kerr replied "Ralph, if the number is in between, then that means there are votes on both sides. The original post explained the significance of that, I believe."
The only thing I could find in his original post that I think he might be referring to is the phrase "If we see votes on both sides, it will support the case that Roberts is an "umpire.""
If this is what he meant, then just one "liberal" vote is enough to 'support the case the Roberts is an "umpire."'
I hope that's not what he meant. Because if it is, then the most important determinant of his result is sample size: If Roberts votes liberal once in a very long while, then the larger your sample the more likely you are to catch him doing it and call him an "umpire." When you're designing a study, you really, really don't want to anwser to be dependent on sample size. Significance level yes (of course), but reducing the sample size shouldn't change what answer you're most likely to get.
If Kerr didn't mean the above, but instead meant 'the more the votes are evenly distributed between both sides, the better the evidence Roberts is an "umpire,"' then he is implicitly assuming a perfect umpire would vote 50/50.
If he's not saying:
(1) The probability that Roberts is a fair umpire is (100% - fraction of conservative votes)*2 [the "fair umpires break 50/50" interpretation]
nor is he saying:
(2) The probability that Roberts is a fair umpire is 100% if he's ever seen to vote against the conservative position [the "if you watch him long enough he'll look like a fair umpire" interpretation]
then just what the heck is he saying?
What does it mean if Roberts gets a 60% conservative voting record? What if it's 95% Does Kerr even know? It's best to decide what a given result would mean before the data is collected.
I think this whole thread boils down to: why does it appear to be so rare (is it really?) that Supreme Court justices are willing to uphold laws of which they disapprove, and vice-versa?
I think the point of this exercise is that Roberts with his umpire analogy implied that while other justices won't uphold laws of which they disapprove/strike down laws which they like, HE would be substantially more impartial. The question we want to answer then is, is he really different, or (it seems more likely) was this umpire stuff just a lot of hooey.
You wrote, "your request is rather absurd."
One could say that about your proposed exercise. I thought commenter on the Volokh board should be civil. In that effort, I will refrain from responding to this comment.
Next, you wrote:
"All I have done is announce that I plan to conduct a study; you respond by insisting that you must hear all of the details about the study before it conducted. How strange."
That is incorrect. I asked you some fairly basic questions about your proposed exercise, which you apparently cannot answer. They were, to repeat:
(1) how you know whether a case is "ideologically-charged" at all. This is a fundamental issue to your study, not a "detail." You first wrote that doing so "will require umpire-like judgment, perhaps, but I think we can probably identify most of the cases relatively straighforwardly based on how most of the Justices voted."
Really? I don't think you can. I asked you to explain it and you can't/won't. An inability to do this moots your entire project. Refusing to explain it surely doesn't make your effort seem more legitimate.
(2) I next asked you to explain how you'd categorize a Roberts decision as either "conservative" or "liberal." Again, this is no "detail." It's the central tenet of your project. An inability to explain it, again, seems to show your effort is not useful.
(3) Finally, I asked you if you could explain how you know the baseline distribution of decisions absent partisan tilt. Your refusal/inability to answer again does not bode well for your project.
Finally, you claim that I am "missing the main point here. I am not arguing that a 50/50 distribution proves neutrality. Rather, I'm suggesting that evidence that someone votes 100% in one direction 100% of the time may provide support for the view that this person is probably not being 100% neutral."
This comment misses my point completely.
My point is that your study lacks an articulable, principled basis. If it has such a basis, you can't or wont' provide it. My point, therefore, is that your study has little to no value and that its conclusions will not, pace your claim, "give us interesting insights into how Chief Justice Roberts approaches his job." On the contrary: it seems unlikely to do so and likely only to add uninformative adjectives to his decisions ("liberal" or "conservative").
Your point, Orin, misses mine utterly. You do not defend your study at all. Instead, you claim a trivial truth: someone who votes 100% of the time in one "direction" (whatever THAT is) isn't "100% neutral."
This point is of little to no value. You have not defined(and cannot define?) "direction"; you have not defined (and cannot define?) "neutral." Your measure is thus unsound.
your inability and/or unwillingness to answer fairly basic questions about your study (and your apparent hostility to questions about it) are, to use your word, "strange," particularly when they come from someone in an academic field.
You appear to assume that my decision to decline to answer a series of seemingly hostile questions from an anoymous blog commenter is evidence that I have no response to them. I think I get the game: you presumably know that I will not take the time to answer all of your interrogatories, so you can then announce victory when you ask more than I will answer. (Ah, New York litigators.)
In any event, the empirical study inthis essay has a good example of the rough category of inquiry I have in mind. It's on a different topic, the question of "judicial activism" in constitutional decisions rather than politics in decisions generally. Still, the basic methodology is similar. Here's what I wrote about OT2001, in response to the claim that the Rehnquist Court was a conservative activist court: