A new EI@Haas Working Paper by Koichiro Ito offers a fresh look at California’s well-known 20/20 program.  The paper is available here.  During the summer of 2005, California households could receive a 20% discount on their electricity bills if they reduced their summer electricity consumption by 20%. Similar state-wide programs were available in 2001 and 2002, and PG&E continues to operate a 20/20 type program in the winter for its natural gas customers (click here for details).

Programs like this have a nice intuitive appeal. They are easy to understand and a 20% discount is large enough to get peoples’ attention. As Ito’s paper points out, however, programs like this also have an enormous problem of measuring “additionality.” Year-to-year variation in weather and idiosyncratic changes in consumption mean that many households receive the subsidy for savings that would have occurred anyway. The paper shows, for example, that 14% of California households reduced their summer consumption in 2004 by more than 20%, even though the 20/20 program was not in place that year.

To crack this challenging problem, the paper uses an ingenious strategy based on the program’s eligibility rules. As illustrated in the figure below, customers were eligible for the program only if they started electricity service on or before June 5th 2004, during the summer before the program. Households who started service just after the cutoff were ineligible for the program in summer 2005 and did not receive a notice letter or other promotional materials about the program.

Cutoff Fig

Source: Ito, September 2013, EI@Haas Working Paper 244

The blue and green lines above illustrate two hypothetical households. The blue line indicates a  household that started electricity service on, for example, June 4th. This household was eligible for the 20/20 program in Summer 2005. The green line indicates a household that started electricity service on, for example, June 6th. This household was ineligible for the program in Summer 2005.

This eligibility rule creates a quasi-experiment. In particular, the ineligible households who just barely missed the cutoff date make a highly-credible control group for the households who were just barely eligible. It is almost as if this eligibility rule creates a true randomized control trial, with a random subset of households being declared eligible for the program.  The paper confirms this quasi-randomness by comparing the characteristics of customers on either side of the cutoff date and shows that they are extremely similar.

The figures below summarize the paper’s main results. Along the x-axis is the date at which each household started electricity service, normalized to zero at the cutoff. Households to the right of the red line were eligible for the 20/20 program, while households to the left were not. The y-axis is average electricity savings between 2004 and 2005, measured in percent. Averages are reported for 15-day bins. So, for example, the first green dot to the right of the red line is average electricity savings for households who started electricity service between 1 and 15 days after the eligibility cutoff.

Coastal Fig

Inland Fig

Source: Ito, September 2013, EI@Haas Working Paper 244

In coastal climate zones the program had a near zero impact. There is essentially no difference in savings between households who were eligible and ineligible. However in inland climate zones, the program had a visible positive impact. The change at the threshold is about .05, indicating that eligible households reduced their electricity consumption by about 5% more than ineligible households.

These results make sense. Coastal Californians use much less air-conditioning, so it is hard for them to reduce summer consumption by 20%. In contrast, households that rely on air-conditioning all summer have more scope for conservation. These households can change their temperature settings, operate their units fewer hours per day, or even leave the house on hot days. There are many substitutes for air conditioning use, but few substitutes for running your refrigerator or charging your cell phone.

The paper concludes by using these estimates to evaluate the cost-effectiveness of the program. Overall, the program costs are high relative to the consumption reductions, reflecting the small estimated effect in coastal climate zones. In inland areas, however, Ito finds that the program costs 2.5 cents per kilowatt hour of savings. This compares favorably with cost-effectiveness measures for other types of conservation programs (see, e.g., Arimura, Li, Newell, and Palmer, 2012), particularly considering that the program can be targeted to particular months when the external benefits from consumption reductions are high.


Tagged: electricity, energy efficiency, energy policy, Randomized Control Trials