Since my husband and I both work in the energy industry, we often exchange energy-themed gifts. He has given me energy books (my favorite was American Power by Mitch Epstein filled with beautiful art-shots of power plants), power plant trading cards, and several times a year, someone in the family jokes about coal in the stocking.
At the beginning of December, my 11-year-old daughter, who loves giving gifts but hates keeping them secret, asked me some pointed questions about Nest Thermostats. Given her track record, I was anticipating a Nest under the Christmas tree and said as much to a reporter, who included that line in her story.

Under pressure and with any secrecy out the window, my husband and I decided to think hard about whether a Nest made sense for our family. (Yes, we are both economists, so even gifts get reduced to costs and benefits…).

The first question we asked ourselves: would a Nest help us save energy?

For the uninitiated, a Nest Learning Thermostat, created by two former Apple designers, is a smart thermostat that learns about your home and your habits and sets the indoor temperature based on what it’s learned. True to its lineage, it appears easy to use, sleek and generally high in cool factor.

The company advertises the energy savings potential of its devices on their home page, exhorting you to, “teach it well and the Nest Thermostat can lower your heating and cooling bills up to 20%.”

How? According to this review, the Nest uses three types of data:

  • Three temperature sensors, designed to get a more precise measurement than a single sensor
  • Motion and light sensors that detect activity in the room at a wide 150-degree angle
  • A WiFi connection to get weather data about your area from the Internet.

The idea is that the Nest will save energy by helping you use less of it when you’re not home. Sounds like a good idea in general. In my house, the Nest would replace a perfectly good programmable thermostat that we have instructed to keep our house warm when we’re home and awake, a little less warm while we’re asleep and a cool 60 degrees between 8AM and 6PM.

So, we are already reducing our heating load quite dramatically when we’re away. Given that we live in Berkeley, the only time I remember the house actually being 60 degrees is when we returned from a vacation. If pet fish could shiver, ours were that day. But most days, our furnace is not working while we’re away.

If we got a Nest and it was really paying attention, though, its motion sensors would detect that I am often home past 8AM, usually coming back from a run (so with natural body heat) and headed out the door after I send a couple emails. On mornings when it’s cold outside and one email turns into ten, it can be chilly in the house by the time I notice.

I suspect that, if anything, the Nest would raise our heating bills. (Like most Berkeley residents, we don’t have air conditioning). It might make me a little warmer on the mornings I stay at home. Given that a Nest can be controlled remotely, we would also probably turn the heat up on the road home from vacation (so the fish would be defrosted by the time we returned home). But, even with the sleek interface, my husband and I decided that neither of those things justified the $250 price tag.

Is there a general lesson here? Probably not about the Nest. My family is idiosyncratic. My colleague, Max Auffhammer, lives 14 miles away in a hotter microclimate and swears by his Nest. A report from Lawrence Berkeley Labs suggests that most people do not actually program their programmable thermostats. So, it may turn out that the Nest can save most people energy.

This example does highlight two important messages about programs and gadgets, like the Nest, designed to make us more energy efficient: they can have both heterogeneous and unanticipated impacts.

Take heterogeneity first. It is highly unlikely that there’s a one size fits all answer with energy efficiency, but, right now, most programs are evaluated based on their average impact. If we can recognize and learn more about cross-customer heterogeneity, policymakers could target programs where they are likely to be most effective.

The possibility that energy efficiency programs will have unanticipated impacts reminds me of the Rumsfeld quote (“…known unknowns…”). We should anticipate that human beings are complicated, which makes it hard to model our behavior. We have limited attention (if I didn’t, I would manually override our thermostat on the mornings I suspected I would be home for awhile), some of us make impulse purchases, and sleek gadgets attract us.

To learn more about how households and businesses actually respond to energy efficiency, we at The E2e Project are encouraging policymakers to consider randomized controlled trials and other robust empirical techniques to measure programs’ impacts.