The Wall Street Journal, Forbes, and, most recently, the Sacramento Bee have pieces on Arik Levinson’s new NBER working paper, “California Energy Efficiency: Lessons for the Rest of the World, or Not?”   The paper makes a nice point, but I worry that it is being misinterpreted.

Levinson starts with the well-known “Rosenfeld Curve” (below), named after energy-efficiency pioneer Arthur Rosenfeld.  For the last four decades, residential electricity use per capita in California has been nearly flat, while growing 75 percent in the rest of the United States.

Residential Electricity Use per Capita 1963-2009

Fig 1

While some have attributed the difference to California’s energy-efficiency policies, Levinson argues that California’s temperate climate, changing demographics, and other factors can explain almost 90% of the gap. For example, Levinson shows that part of the explanation for the increase in other U.S. states is that more and more people are living in the Southwest, where air-conditioning is used more intensively.

Levinson’s paper is thoughtfully done and deserves to be widely read, but the results are not terribly surprising. Even energy-efficiency proponents have long understood that at least half of the gap is likely due to non-policy factors (Sudarshan and Sweeney, 2008; Rosenfeld and Poskanzer, 2009).

But the truth is that it is hard to learn much from this type of aggregate data.  The challenge with any empirical analysis is how to construct a counterfactual. What would have happened to California’s electricity use without energy-efficiency policies?  This is a deceptively difficult question because of all the ways, large and small, that California is different from the rest of the United States. These differences accumulate over the 40+ year time horizon, obscuring the causal impact of energy-efficiency policies.

And none of these comparisons capture some of the broader impacts. California has consistently pushed the national agenda on energy-efficiency.  For example, in 1976 California was the first state to introduce appliance energy-efficiency standards. Other states quickly followed, leading eventually to national appliance standards in 1988. These national “spillovers” don’t show up in these analyses because they result in decreased electricity use both in and out of California.

So the “Rosenfeld Curve” does not prove that California’s energy-efficiency policies have worked. But nor does Levinson’s analysis prove that the policies haven’t worked.  With aggregate data it is impossible to answer this question definitively, let alone to say anything about which particular types of policies are most effective, or about how differences in program design impact effectiveness.

To be fair, Levinson understands all this. But people tend to have strong views on energy-efficiency. So when Levinson pokes holes in some of the best known “evidence”, it is tempting to run to the other extreme. Let’s not.  It doesn’t make sense to have such extreme views when the evidence is so incomplete. As it often is, the truth is probably somewhere in the middle.

We need more studies using better data. This is the motivation for a new project, E2e, which brings together researchers from UC Berkeley and MIT.  Our goal is to apply more scientific rigor to the analysis of energy-efficiency to better understand how to best design policy. We are particularly focused on randomized control trials and other experimental approaches. In fact, we are organizing a one-day course on the topic August 26th in Berkeley. Click here for more information.