National Performance Measures Strategy

My name is Alexander Volokh of the Reason Foundation, a public policy research organization based in Los Angeles. I am also the co-author of Environmental Enforcement: In Search of Both Effectiveness and Fairness.” Thank you for giving me the opportunity to speak to you about the National Performance Measures Strategy.

I will offer some ideas for developing new performance measures, but we cannot begin to reform the old system without precisely understanding its failings. The problem with the old input-counting system was not that it gave us wrong information, but that the information it gave us did not help us to figure out whether the enforcement agency was doing a good job. The main problems are the following:

1. First, the numbers that are reported are often meaningless, because they do not make important distinctions — such as distinctions between trivial and harmful violations, accidental and intentional violations, and violations taking place in different media.

2. Second, increasing numbers of enforcement actions are interpreted as success, while in fact (even when the numbers are meaningful) this is not necessarily the case.

Let me briefly discuss two sorts of problems with the current performance measures.

1. What do the numbers mean?

Because not all violations are the same, aggregated numbers often mean less than they seem.

a. Violations have different impacts.

A one-time permit exceedence involving a low-level, low-toxicity pollutant in an uninhabited area is not the same as dumping a highly poisonous chemical into the water supply near a major city. To say that "one trivial violation + one harmful violation = 2" gives us no information on whether the EPA is solving real problems.

Since the law is at least in part supposed to be a deterrent to bad actors and a tool of moral education, we would like to concentrate our resources on intentional and reckless violations. An accidental paperwork mistake is not the same as intentionally defrauding the EPA or treating chemicals in a way that one knows has a high chance of killing people. To say that "one accidental violation + one intentional violation = 2" gives us no information on whether the moral goals of the law are being properly fulfilled.

Suppose that by transferring resources from air to water enforcement, we reduce air enforcement actions by 2 but increase water enforcement actions by 1. The numbers would indicate that total enforcement actions have decreased by 1. But counting total enforcement actions does not work here either. An air quality violation is not equivalent to a water quality violation, which is not equivalent to a taking of endangered species. In allocating its enforcement resources, the EPA is implicitly making this judgment. But there will always be a certain amount of arbitrariness in the decision. To say that“"one RCRA violation + one FIFRA violation = 2" gives us no information on whether “enforcement results,” in some absolute sense, are increasing or decreasing.

Intimately related to this last point is that environmental quality is subjective. "Environmental quality" has no rigorous definition. Some people consider biodiversity to be the most important environmental issue; other people care about human mortality rates; others care about sustainability. Even though we know that not all "paperwork" violations are trivial -- some are quite important, since the entire system depends on honest and accurate reporting -- there is no clear way to rank a paperwork violation, which represents the possibility of future harm, against, say, an actual oil spill.

2. What do increases in the numbers mean?

Even seeing the evolution of these numbers through time gives us misleading information. Here are two problems related to the interpretation of changes in enforcement quantity.

a. Any improvement in environmental compliance will make the numbers go down.

When the measure of success is the quantity of enforcement, the agency’s performance record is hurt by any improvement in the environment or in compliance. The business community has a more cooperative attitude toward enforcement today than it had in the 1970s. This makes the numbers go down and makes the EPA look as though it is doing less. If regulations suddenly become simpler, noncompliance will drop dramatically, and again, the EPA will look bad; conversely, if a new regulatory program is enacted, the numbers will suddenly increase, giving the illusion of greater effectiveness.

Every time the EPA spends money on a compliance assistance program or on an audit program, it is taking resources that could otherwise be used on traditional enforcement, and using them instead to increase compliance. To the extent that an audit or compliance assistance program succeeds, it again makes enforcement numbers decrease, and gives the impression that the EPA is doing a bad job.

We all know that environmental law is complicated and highly technical. A National Law Journal survey reported that about two-thirds of corporate environmental attorneys believed that their companies had violated some environmental law over the past year, largely because of uncertainty and complexity. They also believed that full compliance with all local, state, and federal environmental laws was impossible. This means that someone looking for violations of environmental laws is likely to find them, especially in statutory regimes characterized by elaborate permit and record-keeping systems, like the Clean Water Act. When the numbers increase, it could mean that the enforcers are doing their job, or it could mean that they have shifted to those violations that are easiest to find, which are not necessarily the most valuable ones. Decreases or increases in enforcement or failure. If the numbers increase, it doing their job properly; that is, that numbers could indicate either success could mean that the enforcers are they are trying harder and catching more people. Or it could mean that compliance has gone down -- in other words, that the enforcers are not doing their job. Or it could mean that enforcers are picking the easier cherries. Unfortunately, it is impossible to tell by looking at the numbers.

3. What should be done?

• by amount of harm -- separated into “paperwork violations, "harmless permit exceedences," "harmful emissions," and the like;
• by level of intent -- separated into "accidental violations," "negligent violations," "reckless violations," and "intentional violations";
• by medium or by regulatory regime -- separated, for instance, into "Clean Air Act violations," "Clean Water Act violations," "RCRA violations," and so on.