I sometimes blog on the process of moderating blog comments. I realize it’s a bit “inside baseball,” as most readers don’t comment or run blogs that allow them. But I see Internet comment threads as a new and relatively important kind of online discussion, and I’m very interested in the conditions in which comment threads tend to be useful or just noise. In my view, having a really good comment thread is a terrific asset to a blog: It allows the post to be the beginning of a conversation, with the rest of the conversation carried on it the thread. The interesting and new question is, what are the conditions of helpful comment threads? What kind of comment policies and software leads to the best, most interesting comment threads, and which don’t?
In my experience, there are two basic conditions of strong comment threads. Here’s the first condition: Comments need to be relatively open and accessible to those using a pseudonym. If you make it too hard to comment, or you require real names, most will stay away. They won’t want to engage, for a range of personal and professional reasons.
And here’s the second condition: There needs to be some way to moderate threads to delete inappropriate comments or ban commenters who are out of line. For every one Internet commenter who is consistently thoughtful and interesting, there are X Internet commenters who are either inclined to be or can be coaxed into becoming abrasive and obnoxious. Consider the well-known “Greater Internet fuckwad theory” from the site Penny Arcade:
There’s a lot to that, with an important caveat: When the site is a popular blog with hundreds of commenters, some of the commenters will be “normal people” and some won’t. In any collection of that many people who can post at any time, there will be some stylistic quirks: There are the commenters who always bring up their pet topic, no matter the subject of the post; the commenters who see themselves as needing to wage constant battle with perceived ideological foes; the commenters who see criticism of their views as inherently objectionabe, etc. When any one can comment, everyone gets invited, and the quirky types join in with the rest.
This diversity of audience, combined with the Greater Internet Fuckwad Theory, means that unmoderated threads have a tendency to devolve into virtual food fights. That’s especially true if the topic is controversial and relies heavily on ideological priors, like current debates here at the VC on gay marriage or the individual mandate. Threads that devolve into food fights are entertaining for the subset of commenters who get a kick out of written sparring. But they come at a major cost: They tend to discourage readers and contributors interested in more thoughtful contributions. When the thread turns to muck, the readership drops dramatically: Few people want to wade through the accusations and hostility to find the few morsels of insight. So to maintain the quality of comments, there needs to be some sort of monitoring of threads.
These two conditions combine to produce what you might call the Effort Rule of commenting: Having consistently strong comment threads requires a significant effort moderating threads. Vibrant dialogue requires a relatively open door on the front end, and keeping it from devolving into a digital food fight requires significant attention to editing on the back end. But back-end moderation is always unpleasant, for two reasons. The first is that it’s work. It requires careful judgment as to where the line us, based on the editor’s necessarily limited exposure to the full range of comments. In a blog with thousands of comments a day, no blogger can be fully informed as to the full history (sometimes going back several years) as to exactly which commenter said what to whom. Judgments have to be made, but they necessarily have to be made based on exposure to a subset of the evidence.
Second, the combination of no front-end filter and back-end moderation invariably leads to accusations of bias and claims of censorship. Commenters are most hostile when the subject is deeply controversial, which means that posts on those subjects will trigger the most need for comment moderation. But these are precisely the contexts in which people with strong views tend to interpret the facts to be whatever reaffirms their priors. A great example is the forthcoming paper by Dan Kahan et. al., “‘They Saw a Protest’: Cognitive Illiberalism and the Speech-Conduct Distinction”, which I blogged about here: When shown a video of a protest, people evaluated whether the protest was violent based on whether they supported the cause being protested.
The same basic reasoning applies to interpreting editorial decisions on a blog. The more passionate a commenter feels about the subject, the more likely they are to interpret editing or (in extreme cases) a ban on commenters as incredibly obvious evidence of bias against them based on their viewpoints. The “Joys of Anonimus” thread from a few days ago, now at 450+ comments and counting, has a lot of examples. Anonimus’s violations of the comment policy are flagrant, and he candidly admits he ignores the comment policy and says whatever he wants, but several commenters who agree with Anonimus on the issues are deeply persuaded that the real reason he has been banned is that I disagree with the merits and I’m trying to “silence” him.
Where do these points take us? First, to the conclusion that really good Internet comment threads are rare. Good comment threads require someone with the patience to do the editing work and deal with the inevitable bias accusations, efforts to circumvent bans, etc. On a group blog, each commenter need not do that kind of work; some bloggers can free ride on the efforts of others. But there needs to be at least some amount of work put into an unpleasant task to maintain or even raise the quality of threads. That’s relatively hard to find, and that means that good comment threads will be rare.
Second, I suspect the future of Internet comment threads is a bifurcation into two sorts of threads on high-traffic sites: open and unmoderated threads, where anyone can say anything and few people read the threads; and sites with more moderation on the front end, such as requiring registration through a Facebook account. Neither of those are ideal, for the reasons stated above, but they are more stable forms of comment threads because they don’t require the same amount of work from the editor.
UPDATE: My apologies that comments were off initially; I had forgotten that the software seems to do this automatically when a post has been in draft form for more than a day or two. Comments are now open. As always, civil and relevant comments only.
shg says:
A very kind way of putting it, even if they still wear tin foil hats.
January 18, 2012, 4:02 pmuh_clem says:
I saw this post at about 1:30pm EST and then it disappeared for a couple of hours. I wondered if you had deleted your own post because of a perceived lack of civility and thought man that Orin Kerr is one serious dude.
January 18, 2012, 4:17 pmjustcurious says:
Shouldn’t it be as simple as using the sort of language and tone that would make our parents and children proud of us?
January 18, 2012, 4:19 pmBruce McCullough says:
There is a blogging software out there, somewhere, that enables a troll to see his own comments, but other readers cannot see them. This leaves the troll wondering why no one is feeding him. A person can be labelled a troll by the admin, or by votes from other readers/commenters. A few minutes of searching did not enable me to find this software, but should Volokh.com be enamored of the idea, I’m sure someone can find it.
January 18, 2012, 4:24 pmHenry Clay says:
Internet Rule 40: Everything leads to accusations of bias.
Seriously, just ban the folks you feel like banning and don’t worry about it. Moderation is part of the blog experience. I assume you don’t sit around worrying about people who claim your blog posts are biased; treat comment moderation the same way.
I completely disagree with your bifurcation theory, by the way, there’s lots of ways to do back-end moderation, for example through voting mechanisms or by assigning a large number of moderators.
January 18, 2012, 4:26 pmmartha says:
Whew! I was worried when I saw the initial “comments off” that you had given up on moderating comments. THANK YOU to all Conspirators who moderate comments. I’ve learned a lot from the comments, and more than once my views have changed. The civil debate here keeps me coming back.
January 18, 2012, 4:31 pmNessuno says:
In my experience, the commenters on this site are especially mean-spirited and are quick to use ad hominem attacks, which is why I generally avoid even reading the comments here. (Yes, I occasionally get lulled into reading or even commenting, but I almost always regret it.)
I read a lot of varied internet sites with comments–everything from professional sites to video game forums–and usually enjoy reading or participating. The best comment sections are ones where I go to get additional information related to the article (I like informationdissemination.net or dailytech.com for that). On most other sites I read comments to gauge reader reaction or to look for humorous remarks.
But in everywhere except this site, the ad hominem attack is actually quite rare and will get you effectively shouted down by other commenters. Over time, it sort of self-polices and becomes understood that it is a line of argument that just shouldn’t be done.
On Volokh.com, however, I actually expect some random and mean-spirited insult to my intelligence, education, or my presumed crypto-racist/fascist/populist agenda every time I dip my toe in the water.
It’s just not worth it.
January 18, 2012, 4:31 pmGrant Fenwick says:
I suggest limiting a commenter to two or three comments per topic. One should be able to get his or her point across within those limits. This would cut down on some of the inane “Am not”-”Are too” back-and-forth that clutters up some of the threads. It also would cut down on some of the unenlightening comments that are intended to just provoke further responses.
January 18, 2012, 4:39 pmjack burton says:
For your reading enjoyment…
Flame Warriors
perhaps you’ll see yourself listed as one of them.
January 18, 2012, 4:40 pmuh_clem says:
Your mother was a hamster and your father smelled of elderberries.
//Don’t want you to be disappointed!
January 18, 2012, 4:40 pmMark Field says:
The best comment section I’ve seen, at least on the more heavily trafficked sites, is Ta-Nehisi Coates. There’s plenty of debate, a great deal of information supplied by commenters, and a generally good tone. He’s quicker to ban than you are, but the site itself uses tools to encourage civility such as being able to “like” a comment and to report TOS violations.
January 18, 2012, 4:40 pmepluribus says:
Orin’s blogs are consistently among the best on this site. (I say this even though I don’t think I share his political views–at least as far as I know anything about them.) This thoughtful commentary on monitoring policy is further evidence of that.
January 18, 2012, 4:41 pmSteve White says:
This is a very helpful post. One point about moderation: the work required increases exponentially as the numbers of comments increase. I moderate a small blog. We typically have 5 to 15 comments (but 40 to 60 separate posts a day). Many of the commenters are regulars, but even then it takes an appreciable amount of time to scan comments and ensure that trolls, jerks, etc are removed (we do remove them).
A more popular blog has exponentially more work to do in moderation. The numbers of comments, the greater number of regular commenters, and — importantly — the greater number of semi-regular and occasional commenters makes it more difficult to moderate effectively. You can do it but it will take far, far longer, and few people will want to (unless paid, and even then it’s the equivalent of digital dentistry).
I simply cannot imagine Instapundit with comments: Prof. Reynolds would need a dozen full-time, paid moderators just to keep up.
Just as importantly — suppose Prof. Reynolds did just that? The problem then would be, it wouldn’t be his blog any more. It would belong to the moderators and commenters, and his unique voice would be lost.
The opposite is unmoderated commentary. Look at typical articles at newspaper sites (e.g., Washington Post, UK Daily Mail) that have unmoderated commentary. It’s total trash; the comments are not helpful, not worth reading, and appeal only to those with nothing better to do than post inane comments.
A small blog with few commenters can have strong commentary. But I’m convinced that a large blog really cannot. I suppose that leads to the question of whether the Volokh Conspiracy is a large or a small blog, and what your goals are in allowing comments.
January 18, 2012, 4:44 pmbillo says:
A couple of, er, comments.
I’m not sure I agree with your axioms at the start. I don’t see why “strong” comment threads require moderation beyond the rather trivial and mechanical spam filter. I suspect it’s because we disagree on *who* the comment thread is supposed to serve.
If the comment thread exists to provide interesting feedback *to the author* then, indeed, it is appropriate to limit comments to those opinions and expressions that the author enjoys. If the comment thread exists to provide an outlet to the *readers* then by definition, *all* comments serve their intended purpose.
I see this dichotomy often in mailinglist discussions. In many mailinglists, there is a relatively small cadre of people who post frequently and a much larger group of “lurkers” who read the mailinglist but do not post. In longstanding mailinglists, the cadre of frequent posters often get to know each other and engage in conversations that seem tangential or personal. At such times, lurkers will occasionally post, saying “You people are not talking about things I like in a way I like, and you should stop.” The problem is that mailinglists are kept alive by the people who *post,* not the people who *lurk.* And the people who post are posting because they enjoy it, not because they exist to entertain lurkers. Unfortunately, many lurkers seem to believe that posters exist to entertain them, even though they can’t be bothered to post themselves.
The same sort of thing can be found in comment threads. Do comment threads exist to entertain those who do not comment, or to entertain the people who *do* comment? If the latter is true, then there is no need to moderate. If the former is true, then there is.
I suppose there is a third path, in which one sees commenters as a group who should be able to impose self-censorship in order to keep the atmosphere congenial to one particular point of view or manner of expression. For instance, frequent commenters to a blog devoted to Progressive or Conservative ideas may simply not want to hear someone of the opposite persuasion opine in “their” space. One could, I suppose, have some sort of voting system in which posts and posters would be removed or banned based on a plurality of opinions from commenters who enjoy a place that enforces some sort of orthodoxy of expression.
Finally, there is nothing inherently wrong with censorship in a private environment. If you come to my house and shout expletives in front of my godchildren at dinner, I’ll kick you out. Because it’s my house. There is no “rights” issue here. It’s just a matter of the kind of function you see served.
January 18, 2012, 4:44 pmAnonimus says:
[Deleted by OK, who has already deleted about 5 or 6 comments by Anonimous in the last 30 minutes that call OK an asshole. Anonimus, your view that I am an asshole is duly noted. Now go back to one thread where you are allowed to comment, and leave us alone so we can have a discussion. That thread is open just for you, and you can comment all you want there. You just can't comment at this blog anywhere else.]
January 18, 2012, 4:46 pmFantasiaWHT says:
When I first started reading Volokh back in law school (06-09), the quality of comments on this site was superb. There were occasional trolls, but people knew who they were and didn’t feed them much.
My first job out of law school kept me way too busy to read blogs, but a couple years later I started reading again and found the comments section now to be, by and large, crap. Sadly.
Re: Billo I think your mistake is to lump “the readers” into one big group, all of whom are served by every single comment ever made. There are readers who are served by thoughtful comments and readers who are served by flame wars (and other readers with other needs/desires/wants being served by reading comments). The flamers are not serving anybody but themselves. Likewise, the flamers don’t care about the reasoned opposition, except insofar as it gives them something to spit at.
January 18, 2012, 4:50 pmOrin Kerr says:
Billo,
The audience for comment threads is the same as the audience for blog posts: thoughtful readers of varying views who are interested in engaging ideas, insights, and perspectives.
January 18, 2012, 4:54 pmuh_clem says:
I disagree with this. Some of the best threads involve thoughtful back-and-forth between/among commenters. Of course, that italicized word is important – long threads of “am not” / “are too” are a tedious waste of everyone’s time.
And, BTW, I find the comments on this blog to be of generally higher caliber than that of other politically-themed blogs.
January 18, 2012, 4:58 pmptt says:
Me, I’d look into hosting software that allowed individual users to ignore other individual users at their choosing.
January 18, 2012, 4:59 pmHouston Lawyer says:
I read this blog primarily for the comments. Some of the comment threads devolve into shouting matches, but others are interesting and informative. The majority of those who comment here at least appear to be college educated and bother to comment in proper English. For that I am thankful.
January 18, 2012, 5:00 pmCorey says:
The posting delay involved in live moderation makes cross-talk between commentators rather difficult and certainly pointless, which is probably a benefit, because it encourages focus on the original post. The high comment volume here can also be discouraging to the more sophisticated would-be participants. If I wanted people to read my response to something here I suspect I would be better off linking and responding on my own blog or facebook page.
If you are in fact interested in maintaining a constructive discourse here rather than just an evangelical one, and are willing to put editorial energy into that goal, I suggest that positive reinforcement should be part of your strategy. By responding to the best comments, either with follow-on comments or by acknowledged reincorporation of ideas into post updates, you can reward those who take time to think about what they are saying. Its one thing to screen for quality customers at the door, but you also need repeat business.
January 18, 2012, 5:05 pmshawn-non-anonymous says:
I run and moderate a few community forums and at times there are some heated discussions. I’ve often struggled with the concept of moderating people because I do not want to appear biased. I’ve had users intentionally sabotage conversations through various methods. It’s frustrating because I take the time to provide this resource with the dream of having it be valuable to others and not because I’m interested in dealing with other people’s under-medicated social issues.
I read an article a year or so ago (I swear on Wired but Google isn’t providing me a good return) that went over various methods designed to discourage the internet troll. One such method, mentioned earlier in the thread, was to repeat the comment only for the troll, causing them to think they are being successfully ignored. However, that method is easily gotten around by just using a different anonymous account or IP address. I used it briefly and the troll began to flame the site for being “buggy”, etc. It wasn’t the most helpful method. Another way is to make it obvious but leave the comment mostly readable. The article suggested pulling all the vowels out of the comment. Unlike the hidden approach, this one is more up front. I think the best approach is to use a user-generated filter based on “likes” that lets people choose a threshold, below which no comments show. So if anonymous user “X” were to get few “likes”, her comments would not show on my listing and I would be unable to respond.
In the end, the blog belongs to the blogger and it should reflect their wishes for tone and content. If a commenter is ruining that vision, they should be warned and then banned. The alternative is to shut the blog down, which is likely the troll’s primary goal. (Bad little parasite!)
January 18, 2012, 5:06 pmJohn A. Fleming says:
One technical blog I’m familiar with, often gets about 50-60 comments for every post, sometimes more. As far as I’m aware, solely moderated by the proprietor.
He uses three techniques: snips anything off-topic to the current post, snips anything on a short list of always-banned topics (no threadjacking), and doesn’t allow “hate the sinner” comments. In other words, comment on my chosen topic, or you’re outta here.
He gets vigorous discussion, and people stay on topic. He gets both pros and duffers. I don’t see many snips (he leaves in evidence that a snip occurred and the reason why).
That seems like an easy moderation task.
January 18, 2012, 5:07 pmHenry Clay says:
@billo, I think you should go back and read OK’s post again. One issue is readership drastically changes depending on the quality of the comments. Aside from personal/philosophical issues, think of it as a simple business decision: he wants to keep hits on the comment pages as high as possible, and bad comment threads have very low hit rates (the few participants hit the page repeatedly, but others leave). This isn’t a mailing list: the lurkers see the same ads as the participants here.
Consider a vibrant comment page where there are 40 posts by 30 users and 10,000 readers. Compare that to a flame fest with 400 posts by 3 users and 5 readers. It makes no sense to justify the latter by saying “that’s what the readers want”. Yes, after all the intelligent people left, the remaining people wanted trash, but the original readers wanted something better (draw your own TV news comparison here).
January 18, 2012, 5:15 pmStephen Lathrop says:
One way to ease the moderating role might be to establish some sort of explicit status for commenters who honor standards. Once approved, you get to comment freely—without moderation—as long as you don’t slip into bad habits. Others could start on moderated probation, and maybe be limited in comments per thread until they prove up. That might lighten the load and incentivize good comments at the same time.
January 18, 2012, 5:19 pmOrin Kerr says:
NOTE TO READERS: Anonimus is presently trying to flood the thread with insulting comments to me, and he’s commenting through an anonymizing Tor server that uses a different IP address every time. That means that I need to sit here and recheck comments every few minutes to keep Anonimus from taking over the thread. As it happens, a buddy of mine is in town an I’m going to meet him for a beer, so I’m going to close up comments to avoid an Anonimus take-over. I’ll plan to re-open comments later tonight if I have time to sit there and delete all of the incoming new Anonimus comments, but I can’t promise anything.
January 18, 2012, 5:20 pm