Gender and Law Reviews:
Back in 2005, we had a thread on the topic of gender and law review placements. The question was, why are so many of the placed articles in top journals authored by men? Minna Kotkin has just revived the question with a new draft article, "Of Authorship and Audacity: An Empirical Study of Gender Disparity and Privilege in the 'Top Ten' Law Reviews". Professor Kotkin's conclusion: A disproportionate number of selected articles were authored by men, and explanations other than "unconscious bias" are unsatisfactory. As a result, journals should reexamine their selection processes (among other things).
My conclusion in 2005 was that the gender disparities in placement were a pretty direct result of the gender disparities in submission numbers. When we opened up this issue for comments back then, the comments from former articles editors proved illuminating. The evidence was just anecdotal, of course, but it suggested that most articles received by the top journals are by men. Here's a selection from a few commenters:
Hat tip: Dan Markel.
My conclusion in 2005 was that the gender disparities in placement were a pretty direct result of the gender disparities in submission numbers. When we opened up this issue for comments back then, the comments from former articles editors proved illuminating. The evidence was just anecdotal, of course, but it suggested that most articles received by the top journals are by men. Here's a selection from a few commenters:
1) I was an articles editor only a few year ago, and though I could not give you exact numbers, there definitely was no gender equality in article submissions — there were significantly more submissions authored by men.Professor Kotkin recognizes this possibility at pages 50-54, where she considers the possibility that women may submit fewer articles because they are forced to take on other commitments (whether service requirements or family commitments) or because they write just as much but lack the confidence to submit their work to journals. But I tend to think that this doesn't quite grapple with the issue. If it turns out that the real gender disparity comes with who is submitting articles rather than the selection process — for whatever reason — then I'm not sure how reexamining the selection process really addresses the issue. I suppose we'll have to wait for a journal to do a study of its own submissions, formally comparing the gender ratio of its submissions with the gender ratio of the accepted articles, to have a better sense of that.
2) I sat on the Essays Committee of The Yale Law Journal this year, and the bulk of our submissions appeared to come from men — not just in con law but in all specialties.
3) Yale Law Journal, several years ago — not even close — submissions by men outnumbered submissions by women maybe 3 to 1 — just a rough estimate.
4) As an articles editor for a tech journal at a top-tier school, the vast majority of submissions were from males.
5) Just took a sample of my submissions database from our last volume (of a top-tier journal), and of 200 submissions sample (out of 1956 total), 72 appeared to be by women (at least based on names). The ratio among expedite requests looks roughly the same (33 out of a sample of 100 were women). So if that holds up, there's certainly a skew in the authorship on the order of 2:1.
6) I was an articles editor on the UCLA Law review about 4 years ago. By far the majority of 120+ submissions that I can recall doing primary review on were written by men.
Hat tip: Dan Markel.
Doesn't Kotkin have any such data? If not, how can she seriously allege discrimination?
This is obviously the most un-scientific use of numbers possible, and I just offer this as my personal experience with this issue.
One comparison the author appears not to have made (based on the abstract that Orin links to) was of the quality of what was submitted to the journals--not that it would be easy to do, or that the journals would readily give copies of what they rejected. But as quality of submissions seems to me the most logical explanation of what articles journals publish, the author's inability to show that that was not the dominant factor in the journa'sl decisions renders her study pretty worthless.
Really? It seems like it would take about 4 minutes of searching through old emails and about 27 seconds of pasting the relevant info into an Excel spreadsheet to come up with a handy compilation of all the data. I'm surprised editors don't do this as a matter of routine.
If you've offered to accept 40 manuscripts, how many does that mean you've received (ballpark)? 80? 400? 4000? The number must be reasonably low, or else surely a compiled, handy record of all submissions would already be in place...
Of course, there would be no quotas. Just goals.
Hah! Good one.
To the extent that the numbers Orin’s cites are relevant, it would still tend to indicate there is gender bias in the selection of law review articles. Only 20 percent of all law review articles at top-ten journals are authored solely by women. But in the stories gathered by Orin, none of the journal members claimed that less than 20% of articles submitted were authored solely by women. (A couple could be interpreted that way, I suppose, but everyone who gave numbers indicated that it was more than 20%.) In any event, it would be interesting to learn the gender breakdown of those who submit articles.
You are right, we do have such a record, and the record is compiled as a matter of routine, and in fact we have editors who deal almost exclusively with such matters. However, the record is at the office, and I am at home. Thus, it is not "handy" (in other words, it is not conveniently near).
You are also right that I could make this determination using the email account; however, it would take too long to count the 1000s of submissions that we receive, and then calculate how many of the authors are males and how many are females. In other words, this information is not "handy" (in this case, I suppose "handy" would refer to the fact that the information is not ready for convenient use).
I'm surprised this isn't already standard practice. It would seem to solve all sorts of potential problems.
Does it mean that when you get a chance you'll be posting some data to this thread? (Please?)
Not only does it make sense to have academic publication decisions made without the referees knowing the authors, there is a real life example of precisely that in symphony orchestra auditions.
Auditions are conducted with a screen separating the performer and the conductor (the one who makes the decision). In recent years, a carpet is laid down so that the conductor can not even tell if the performer is male or female by the sound of footsteps.
Is there any inherent value in publishing articles that are not of "the highest quality"? Does it make any sense that quality might be judged in comparison to historical submissions and as such there is less "new thought"? Or when you say quality do you simply mean grammar and coherency of thought? Does increasing the number of so-called "minority" papers artificially, adjust for this automatically? Or in these days of the "series of tubes", do these things not really matter as much?
By what definition?
My experience is that fewer women hitchhike, but they are far better at it, as measured by the dwell time between the start of waving at passing cars and the pickup :)
Really? Forced? Someone call 911 and report involuntary servitude, kidnapping, slavery, or worse ... family commitments!
Give me a break.
Let me get this straight ... Kotkin got this published where? (Please, someone tell me it wasn't a law review article.)
I'd make a comment about the correlation of bad science and poor math skill with law professors and gender politics, but every one would think I was just "trolling".
Along the same lines, this headline in today's news: Study finds minorities more likely to be paddled
Boys are more likely to get in trouble at school than girls? Now how can THAT be anything BUT sexism?
Sigh.
While the anecdotal recollections of law review editors are likely correct in broad brush, they can't be sufficiently specific. The claim in the article (as summarized at the Feminist Law Professors blog) is that even though women are a minority of law profs, they are still under-represented in top law reviews.
Specifically, according to the article, women are 31% of the tenure-track/tenured faculty, and 28.3% of the faculty at the top 15 schools, but only 20.3% of the articles accepted at the top 15 journals are by women. So the editors are probably right that only a minority of articles were from women, but the question is whether there is still an interesting statistical discrepancy regarding the number of women in legal academia and the number of articles published. I doubt that recollections of percentages of the over 1,000 articles top law reviews receive per "season" would distinguish between 20% and 31%.
Again it would be good to have the number of articles submitted. But also, just as some have hypothesized reasons why women might submit less, I would hypothesize that there is a reason why they might submit more, at least disproportionately to their total numbers in legal academia.
That reason is that women are a greater proportion of legal academia in the more junior ranks. According to the summary of the article, at the associate and assistant professor levels, the national figures are 46.8% and 53.9% respectively.
For various reasons, it's at least a plausible theory that junior folks, as a group, publish or try to publish more than more senior folks. Yes, there are many very productive senior scholars, but we all know of the "dead wood" problem -- or, put more sympathetically, we all know that earlier generations of law profs entered the profession when the expectations of publishing were less than they are now. THAT cohort is greatly disproportionately male. The cohort that is trying to get tenure and advancement and generally has internalized the "you must publish" imperative is disproportionately female.
None of this proves that women submit as many articles to top journals as men in proportion to their total numbers in legal academia. Personally, I would be somewhat surprised if they didn't (I would be especially surprised if women were submitting articles, but not to top reviews). Also, even if a significant statistical discrepancy exists between the number of submissions and number of acceptances, that doesn't prove discrimination.
But a considerable amount of social science data supports the claim that people evaluate the same work product differently if they believe it comes from men and women. So, proof of a significant statistical disparity would seem to be sufficient for people at least to try to figure out what is going on.
Finally, and much more broadly, I think this is yet another reason that law reviews should join the rest of academia and review submissions blindly (without knowing who the author is). For example, I also thought it notable that of the articles in the top 15 law reviews, 45% of authors come from US News top ten schools, 61% from the top 25, and 70% from the top 50, cumulatively. I wonder if that result would be reproduced with blind review.
Second, you note that the law reviews have a selection bias for prominent authors. But even if senior authors on balance published less, the most prominent senior authors don't publish less, and they are more likely to be taken by law reviews. And, as you note, the women are more likely to be junior, not yet having achieved this status.
Fair enough, but a couple of comments. If there is an affirmative action program for hiring women at top law schools, I doubt that the distinctions between their paper qualifications and the paper qualifications of male hires would make a significant difference in either (i) the number of law review submissions they make or (ii) a difference in quality of submissions that could be reliably detected by law student editors. Law school hiring is generally quite competitive, and law school hiring at the top schools is extraordinarily competitive. Everyone there is very highly qualified.
Second, you're right that elite law reviews will likely snap up the pieces of the "distinguished senior elite" group which is dispropotionately male. I don't know if the article adjusts for that. To me, of course, it's another reason to use blind submissions.
Blind submissions would be great (though even top peer reviewed journals of law aren't double blind). But I think the problem is people seeing law reviews as the publication outlet for law professors. There are plenty of peer reviewed outlets for law professors. The law reviews are just a supplement.
And don't exaggerate the weight top law review publications carry. In a business school, the locus of publication is crucial. In a law school it is much less so, and tenure and hiring committees are more likely to make their decision based on reading the article and forming their own opinion.
But this paper is, in its own way, a brilliant example of the problems with the law review submissions process. A paper with this kind of gaping hole in the analysis would have a much harder time getting by a panel of people who knew what they were talking about. But because it is instead being reviewed by law students, it will have no trouble getting published. (For the same reason, the hypothesis that men are getting published more because their articles are higher quality is hard to take seriously.)
These data can't be that hard to come by. All you'd need to do is email the editors of some top journals and ask them to email you the spreadsheet with their submissions list. It's not like we're dealing with closely guarded business secrets here. You'd need to hire some undergrads to do the coding of author names by sex, but that's easy.
I see no excuse for this omission, unless the author asked and the journals all refused to hand the data over.
friendscolleagues who provided valuable assistance with this article. And that's crazy-talk.As to your first point, I doubt we disagree much, but to continue to the conversation . . . Small marginal effect on what? Number of publications submitted? I think everyone agrees that's an important number to know and apparently we don't know it. Or do you mean marginal difference in quality of publications? If so, do you think it would be in a way that law students could reliably determine? Both?
I do disagree with your second point. There are relatively few peer-reviewed law journals, and at least almost all of the most "prestigious" journals -- for the purposes of tenure, promotion, and lateral movement -- are not peer reviewed, but are rather student run journals (the main law review at the top schools). Don't get me wrong, I don't approve of this state of affairs, but I think I'm describing it accurately.
Common Sense:
I agree that blind submissions isn't the only thing wrong with legal publishing. Lack of peer review is also huge. And I agree with you that it would be hard to change the system as long as reviews keep getting over a thousand submissions per season.
Having said that, I'm totally against using "hey, it's from a teacher at a tier one school, as opposed to a tier two or three school" as a screening device. I know it's done, I understand why it's done, but it's one of the reasons (along with lack of blind, peer review) that law prof publishing isn't taken all that seriously outside of law schools.
Amazing how people can assume that the way something is done in their own discipline is of course the standard way things are done everywhere.....
If you would like to try to see the Nature blog post I mentioned above, copy the following to your browser and delete the extra spaces.
http://blogs.nature.c om/peer-to-peer/2008/02/w orking_doubleblind.html
This just sounds like one more outcome based argument. Don't like the outcome, argue bias rather than facts.
I will take your word for the practices in your field. But single blind, while maybe not as good as double blind, is hugely different from "everyone knows who the author is," right?
Also, if we're talking about law publishing generally, I think the even bigger difference is lack of *peer* review, at least in the ostensibly most prestigious journals. My point about blind review is that it might help out with any real or perceived gender bias.
This is (a) blindlingly obvious, and (b) politically inconvenient for grievance-mongers.
Also, women whine more about being "oppressed."
Boo Hoo.
Do you have any data to support the claim that women law professors write less than male law professors? Or that women law professors at elite schools write less than male law professors at elite schools? Because I disagree that this is "blindlingly [sic] obvious."
As for quality of the article. That is irrelevant. Since law reviews consist almost entirely of unintelligible glub that is and will be unread by anyone other than the editors and couple of other law professors, quality is already irrelevant. Number of footnotes and conformity to Blue Book Style will be the sole criteria for selection.
And I can't see how single blind review would do anything to mitigate a gender bias. The political science journals that I review for are double blind. But even that is distorted because the editors make the final call (sometimes in contravention to reviewer recommendations) and they aren't blinded. Dig into the research on peer review, and it starts to look bad pretty quickly
I'm familiar with peer-reviewed journals. I've published (twice) in the Employment Rights and Employee Policy Journal (a fine peer-reviewed journal); I recently published a book review in Law &History Review and I've done peer article reviews for them; and I've done peer article reviews for other publications, like Law &Social Policy.
I also know that these publications, and their ilk, are generally not as prestigious in the legal academic world as the top student-edited law reviews. Maybe Law &History Review is to legal historians, but we legal historians are (alas) a small and relatively insignificant part of legal academia.
Second, I don't think the fact that these law reviews are considered more prestigious affirms their quality. That's a longer discussion, but think of who most law profs are and what they did in law school -- that might influence what they think is prestigious. I don't see how the market analogy works here.
Finally, sure, blind review and peer review do not solve all problems. Look what Alan Sokol did. But I think it's a better system than what law reviews do now. Of course there are serious problems in implementing that in legal academia. As Common Sense pointed up way upthread, if we assume the model will continue to feature over a thousand submissions on any possible law subject to every elite law review for each edition, then it's hard to know what to do. As you probably know, there are lots of blog threads devoted just to this.
In sum, to the extent the thread is about unconscious gender bias, I do think some blind review would help. I also understand that some student-edited law reviews do engage in at least some stages of blind review. But more broadly, I understand this is a big can of worms.
Ejo:
That's not the case now and, whenever you were in school, that wasn't the case then either.
From looking at some computer science submission statistics in the past, I conjecture that senior people are co-authors on more papers. One possible mechanism for this is that the authors on a paper are usually someone senior who came up with the idea plus some research students who actually did the work. So if you're in a senior position, you get your name on a lot more papers. A lower precentage of submissions from female authors relative to the percentage of female academic staff might be due to relatively few being in senior positions. In particular, the appointment of just one woman to a senior position can be enough to cause a noticeable jump in the submission statistics - when you look to see what the cause of this year's improvement is, you can find that its down to one person who just got tenure and is co-author on a whole lot of stuff done by her research students.
Of course there are feminist scholars, feminist theories of torts, and of other subjects. Your claim, however, was that "all" female faculty were "feminist scholars" writing about such topics, and that is not and has never been true. Sorry if holding you to your actual words bothers you, but that's a risk of trolling.
However, I also think the limitations of law reviews are recognized. I see people getting tenured and lateraling to top schools without a bunch of law review publications. Seems like the work is independently judged.
Frank: I think the objection is not that stuff in the top journals isn't usually good, but rather that equally or pretty much equally good stuff in "lesser" journals is unfairly/disproportionately devalued. I hope you are right about folks getting tenured and hired at top schools without a bunch of law review publications of certain kinds.
Common Sense: I agree that it's hard to see how the system could be much improved generally as long as the "every decent journal gets 1,000+ submissions every September" model stays in place.
Nevertheless, I made a concerted effort to make offers on submissions from the female professors. What I found was that the acceptance ratio for the female professors was much lower. They would inevitably go with a higher ranked school/journal. My thought was that other articles editors must be doing the same thing at higher-ranked journals, and that the professors were choosing to publish with them. As a result, published a higher ratio of male authors.
The point is that blind review helps to eliminate some potential sources of bias, even if it's an imperfect system. In the age of internet searches and sharing of information before publication, it may not eliminate all possibility of identifying an author, but at least the author's name isn't pasted across the top of the page. At least there's less danger that an author's work will be preferred simply because of the prestige of a name or an association, and not on the merits. Personally, I doubt that gender bias is a much of a concern as bias in favor of the "bigger name" author or institution. Is the quality of a journal a function of the quality of the articles or the prestige of its authors? Probably both, so I can see why this bias exists. Nevertheless, when faculty jobs depend on publishing, it would be nice to think that objective measures of quality are preferred.
I love google and Yahoo!
:)
Bye
http://ssrn.com/abstract=1140644
. The article concludes with the suggestion that editorial boards examine their selection processes for unconscious bias with regard to gender and conscious bias with regard to privilege
If the ratio of acceptances equals the ratio of submissions by female authors, would you say there is bias in the selection process? If you do not know the answer to this question, is it fair to make this suggestion in the absence of such information?
Second, I would want a control for age of publisher in the top journals. If they are mostly senior professors with well established records, you would have to control for that. There were fewer women professors back then. I don't know this to be the case, but it is a very possible explanation for the findings.
Language Log has written about this extensively, though I don't have a particular post handy.
Part of the law journal joke is that the editors are law students (save for a couple of fairly small journals not herein germane). They pick the articles that are published, edit them and publish the journals. There is very little in the way of adult supervision.
This is a major reason why the journals publish nothing of interest to the legal profession. Furthermore, the kids who edit them can check footnote styles, but they have no idea what good readable prose is. The net result is articles about subjects no one cares about, written in a language no one can understand. But the footnotes conform to the arcane and useless rule of the "Blue Book."
This whole controversy is a tempest in a teapot. If the law journals picked the articles that they published by some utterly fair method -- give each one a random string of numbers as an id, print the numbers on small uniform slips of paper, place the slips in a small spherical capsule, put the capsules in a vat, stir the vat until the contents are well mixed and have a blindfolded person reach into the vat, and pull out a capsule, it would make no difference. The articles would be of no interest and they would still be unreadable.
Tempest in a miniature teapot? No, not nearly that important.
Outreach to get more women writing would run into the reason women aren't writing more, and that would never do.
We do the AA thing and impose it on the class which imposes it on the rest of us.
What's not to like?
Now, all the learned discussions prior to this one can be found, or their equivalent, in re sports. Fat lot of good that did the athletic departments. Time for, at least, law schools to suffer. Turn about, and all that.
[Important Note to Helpful Readers: If we have confusing typos and especially ugly formatting errors, such as an unclosed underline or bold tag, we'd love to hear from you about them -- but please e-mail the author about this, rather than leaving a comment. We often won't read the comments for a while after the post, and if there's a glaring formatting error, we'd see it quickly when we revisit the post, even without the comment; and in any event the comment likely isn't going to be that helpful to your fellow comment readers. So please e-mail us directly about glitches like this. Thanks!]
Comment Policy: We'd like the posts to be civil, of course (no profanity, personal insults, and the like), but we're also hoping that people try to be as calm, reasoned, and substantive as possible. So please, also avoid rants, invective, substantial and repeated exaggeration, and radical departures from the topic of the thread. Sticking with substance -- and staying on-topic -- will make the comments more helpful to other readers, and more pleasant.
As editors, we reserve the right to delete posts, and even to kick out posters, though we hope that both of these will be exceptional events. (We also reserve the right to be busy with other things, and therefore (1) not remove all the posts that might merit removal, and (2) ignore demands such as "You should remove A's posts, because they're just as bad as B's!")
Here's a tip: Reread your post, and think of what people would think if you said this over dinner. If you think people would view you as a crank, a blowhard, or as someone who vastly overdoes it on the hyperbole, rewrite your post before hitting enter.
And if you think this is the other people's fault -- you're one of the few who sees the world clearly, but fools wrongly view you as a crank, a blowhard, or as someone who overdoes it on the hyperbole -- then you should still rewrite your post before hitting enter. After all, if you're one of the few who sees the world clearly, then surely it's especially important that you frame your arguments in a way that is persuasive and as unalienating as possible, even to fools.
Our goal is to provide an interesting and pleasant environment that can help inform readers. To do that, we'll occasionally have to exercise our editorial discretion. Think of this as an in-person discussion group, where having different voices is critical to a great conversation -- but where sometimes the leader has to deal with cranks who sour the conversation more than they enliven it.
Naturally, there's always a risk that this discretion will be used erroneously, no matter how well-intentioned the editor. But discussion groups (especially on the Internet, but also off it) generally need an editor who'll occasionally make such judgments.
And, remember, it's a big Internet. If you think we were mistaken in removing your post (or, in extreme cases, in removing you) -- or if you prefer a more free-for-all approach -- there are surely plenty of ways you can still get your views out.