I conclude this series of web logs where I began — with that haunting graph of the asylum and prison populations in the United States during the twentieth century (rates per 100,000 adults):

Chris Uggen, chair of the sociology department at Minnesota, asks on his public criminology blog: "Shouldn't sociological criminologists be able to offer some explanation for the figure at left, showing the aggregate rate of institutionalization for prisons and mental hospitals?"

The figure does, indeed, call out for explanation. But my sense is that the graph itself and the state panel regressions do more to undermine confidence in our conventional explanations and accepted wisdom, than they do to stabilize them.

It may simply be too early to offer answers. We may need first to rethink and study afresh the notion of total institutions. So let me suggest here a few directions for further research and a question regarding the larger theoretical framework.

First, I think we need to place the demographic differences between the two populations in a richer historical context. Many readers immediately question my findings because of the demographic contrast between the asylum and prison populations. But there may be more to the picture. On the issue of racial compositions, for instance, the national counts may mask important differences at the state and regional level.

The early surveys by the Census Bureau are revealing in this respect. Aggregated to the national level, African-Americans represented a small fraction of residents in mental hospitals enumerated on January 1, 1923 — 7.6% to be exact — and had a relatively low institutionalization rate (192 persons per 100,000 African-Americans). Whites in contrast represented 92.9% of mental hospital residents and had a significantly higher ratio of 259.8 per 100,000 whites. But things look very different within and between states and regions.

The New England and Pacific regions had high rates of black institutionalization, in fact far higher than white institutionalization in those regions, and also far higher than black institutionalization in the South. The Census Bureau in 1932 explained why: "This is undoubtedly due to the lack of adequate hospitals for negroes in the South. In the parts of the country in which negro patients are admitted to State hospitals without discrimination, the rate for negroes generally exceeds that for whites. In Massachusetts, for example, the rate for resident negro patients is 644.4 and for resident white patients, 408.8." Here are some other state breakdowns (in rates "Per 100,000 of same race"):

Notice how the comparative rates differ as between states and regions. Clearly, racial demographics varied at the state level and will require more nuanced analysis.

Second, we need to explore in greater depth the relative magnitudes of the different possible effects. In the state panel data regressions, there are interesting clues about other potential explanations. I've already discussed here the issue — or non-issue — of capital punishment. But there are other explanations to investigate.

The size of the youth population seems to play an important role in my regressions, which is consistent with what many criminologists have argued (see here and here). (Some economists do not agree, see here and here). What is particularly interesting about my results is that the effect shows up with the 20 to 24 cohort in the most complete models, but not with the 15 to 19 cohort. This suggests that the actual ages chosen may have significant impacts on the results.

The race effects are also remarkable and, in all likelihood, have to do with high victimization rates in the African-American community, as Lawrence Bobo at Stanford suggests here in "A Taste for Punishment." The negative effect of urbanization is surprising, but may be an artifact of a very loose definition of urbanity. The Census Bureau defines "urbanized areas" very broadly to include areas that have a density of 500 persons per square mile. The gradual lowering of the urban threshold may account for these surprising results.

The key question, though, is how these potential explanations compare in magnitude to each other and, more importantly perhaps, as compared to sociological theories of neighborhood effects, social disorganization, social control, subcultural theories, etc. How does aggregated institutionalization compare to Robert Sampson's measures of collective efficacy and social cohesion in the size of its effect on crime or unemployment or education? Is it possible that institutionalization may actually dwarf those other effects, especially when we investigate a lengthy time period such as 1934 to 2001?

This raises a larger theoretical question about kinds of explanations. The fact is, aggregated institutionalization is about social physics. The term "social physics" may sound nineteenth-century — rightly so. It was first used by Auguste Comte to refer to what is now the discipline of sociology, though Comte abandoned the term when the statistician, Adolphe Quetelet, started using it in a more narrow statistical sense to refer to the "homme moyen" (the average man).

In reappropriating the term here, I would define social physics narrowly as social theories that are necessarily true as a result of the physical nature of our mortal existence, in contrast to theories that depend on the intermediation of human consciousness and decision-making. By way of illustration, consider six theories now central to sociological and economic criminology: (1) rational choice theory, (2) the broken-windows theory, (3) legitimacy theory, (4) incapacitation theory, (5) youth demographics, and (6) the abortion hypothesis. The first three operate through the intermediary of human consciousness. In each case, the theory depends on actors believing certain things and conforming their behavior to those beliefs. In contrast, the last three theories involve only social physics — physical restraint in the case of incapacitation or cohort size in the latter two cases.

Depending on the magnitude of the effects, it may turn out that social physics explain far more than theories of rational choice and social influence. (I discuss this idea here in a a new essay). This raises a troubling question: If social physics have far greater explanatory power, then why have we spent so much of the twentieth century developing socio-cultural and political explanations of deviance — theories of deviant subcultures, disorderliness, social disorganization, collective efficacy, anomie, social conflict, to name but a few? If the dominant factor is simply the rate of total institutionalization qua incapacitation or the size of youth cohorts, then why have we spent so much time trying to identify and shape social relations and social processes?

The answer to this puzzling question — should it arise — may lie in our schizophrenic relationship to punishment that is so glaringly reflected in the arresting figure of asylum and prison populations.