I recently stumbled upon a TED talk given by Alex Laskey, the founder of Opower, a company using data to affect the behavior of energy consumers. The gist of his talk is that a small behavioral change can have large effects on overall consumption. It reminded me of the debate in impact evaluation (IE) and whether IE asks central or peripheral questions. In the case of energy, very little evidence exists regarding the impact of energy on the environment and the economy. But a vibrant new strand of work is developing along the peripheral questions of how to shift behavior and change the way we look at the supply and demand of energy.
Consider that since 1945, US$ 624 billion of the overall US$ 6.6 trillion in aid flows has been allocated to energy generation and supply (source: Aiddata.org). Not surprisingly, a question that gets asked often is the economic impact of power generation– a question that makes us impact evaluators cringe. The analytical difficulty in evaluating bulky single-event investments cannot be understated, especially when there is little data around the event. I, for one, could not find many studies that address this question. The title of the often quoted paper on the impact of dams in India, “Dams”, is itself emblematic of the paucity of studies in this area. Even in that case, the study evaluates the costs and benefits on the local upstream and downstream population—not the economy-wide value of the dam in generating power or replacing “dirtier” power. The same dearth of evidence applies to transmission investments. Here Alex makes the point that 90 percent of energy produced gets lost (that's crazy!). Transmission efficiency should be on our radar screen, albeit with similar analytical constraints.
To look at the economic impact of energy, impact evaluators have turned to a more “tractable” investment: energy distribution. To identify impacts, researchers take advantage of gradual geographical expansion of the electric grid, while having to account for household selection into connections, the endogenous placement of the grid, and possible spillover effects. I would recommend van de Walle et al. (2013) for a good review of the literature and associated methodological issues. Their main conclusion is that household electrification in India brought significant gains to consumption and earnings, through changes in labor market supply. They find however that previous studies (e.g. Khandker et al. 2012) may have overstated the gains from electrification by inadequately addressing issues of endogeneity and externalities. Other papers you may want to look at are Dinkelman et al. (2011, 2013 ) who find that electrification in South Africa significantly raises female employment, and hours of work for both men and women; and point to the importance of accounting for endogenous population movements when evaluating welfare gains of spatial programs.
A whole other strand of literature is learning about the demand side—what in this sector is considered as peripheral questions. In 2006, I met Luiz Maurer, then the co-Team Lead of an energy access project in Ethiopia. He was singularly responsible for making me aware of the value of demand-side management in the energy sector. We set out to design two RCTs with EEPCO, the Ethiopia energy company. The first RCT was intended to understand the demand for household connections using lotteries to vary the discount on connection costs. The results were increased but flat uptake at any level of discount suggesting that factors other than cost may have been responsible for low connection rates. The second RCT intended to lower barriers to the adoption of CFLs (the energy efficient light bulb of choice) by amortizing their cost in EEPCO’s bills. The RCT failed when the government scaled up the distribution of CFLs nationwide to lower pressure on generation capacity. Costolanski et al. (2013), however, went ahead to estimate a 13.3MW saving energy savings from the distribution of those (350,000) light bulbs. As the results make clear, take-up and adoption of energy efficient technology are important questions that help us consider a wider range of possibilities on the supply and demand side.
First world utilities extensively exploit the use of experiments to understand and change consumer behavior but this is hardly tapped in the developing world. It is due in part to the state of the utilities’ information systems, but mostly due to the lack of know-how. The advantage of demand side interventions—from information and pricing, to monetary and non-monetary incentives, to regulation and rationing—is that they are relatively cheap to implement. Need convincing on this perspective? Take a look at the selection of studies that make this simple point. Ian Ayres (2009) documents the sway of Opower’s peer comparisons on customers’ electricity and natural gas usage in California. Mailing peer feedback reports to customers reduce their energy consumption by 1.2 percent to 2.1 percent. Exploiting exogenous variation in prices, Bastos et al. (2014) find that tariffs increases caused significant declines in gas consumption in Buenos Aires. Jessoe and Rapson’s (2012) aptly titled “Knowledge is (Less) Power”, details how households experiencing price increases reduce demand by 0 to 7 percent whereas those exposed to information feedback on price increases (via electronic displays), reduce demand by a much larger margin (8 to 22 percent). In an evaluation of Time-of-Use (TOU) rates in Ontario, Canada, Faruqi et al. (2013) show the potential for large changes in the intensive margin of energy use, with users shifting loads off-peak and mid-peak from 1-6 percent of demand. This is of great interest to all utilities struggling with peak-hour overload.
Kotchen (2010) finds that voluntary initiatives to reduce greenhouse-gas emissions increased the number of households who purchased green energy by 35 percent. In Cash for Coolers, Davis et al. (2012) find ambiguous effects of incentives for appliance replacements in Mexico. More efficient refrigerators cut energy use by seven percent and more efficient air conditioners induce increased consumption and net energy use (the rebound effect again!). Regulations have also been known to stimulate consumer response. Jacobsen and Kotchen (2009) find that the 2002 building energy codes put in place in Gainesville, Florida, is associated with a four percent decrease in electricity consumption and a six percent decrease in natural gas consumption. And finally, using utility data, Chico Costa’s analysis of the long-run effect of short-term rationing during the 2001 energy crisis in Brazil tells us that behavioral interventions can be long lasting. The nine-month compulsory rationing program had large (14 percent) and persistent effects on energy consumption reduction ten years hence (blog). In short, by learning how to leverage the large and untapped potential of demand-side interventions, we have an opportunity to significantly and positively influence national demand.
More generally, Costolanski et al. (2013) and Davis et al. (2012) also help us make a point about the need for impact evaluation in this sector. Their energy savings estimates are well below engineering estimates. This is because impact evaluation measures the combined effect of the technology and the behavioral response to its introduction, which, in this case induces a more intensive use of light bulbs and air conditioners. This “rebound effect” makes explicit why ex-ante simulations will calculate standard engineering estimates that will likely be wrong when calculating energy efficiency effects.
Consider that since 1945, US$ 624 billion of the overall US$ 6.6 trillion in aid flows has been allocated to energy generation and supply (source: Aiddata.org). Not surprisingly, a question that gets asked often is the economic impact of power generation– a question that makes us impact evaluators cringe. The analytical difficulty in evaluating bulky single-event investments cannot be understated, especially when there is little data around the event. I, for one, could not find many studies that address this question. The title of the often quoted paper on the impact of dams in India, “Dams”, is itself emblematic of the paucity of studies in this area. Even in that case, the study evaluates the costs and benefits on the local upstream and downstream population—not the economy-wide value of the dam in generating power or replacing “dirtier” power. The same dearth of evidence applies to transmission investments. Here Alex makes the point that 90 percent of energy produced gets lost (that's crazy!). Transmission efficiency should be on our radar screen, albeit with similar analytical constraints.
To look at the economic impact of energy, impact evaluators have turned to a more “tractable” investment: energy distribution. To identify impacts, researchers take advantage of gradual geographical expansion of the electric grid, while having to account for household selection into connections, the endogenous placement of the grid, and possible spillover effects. I would recommend van de Walle et al. (2013) for a good review of the literature and associated methodological issues. Their main conclusion is that household electrification in India brought significant gains to consumption and earnings, through changes in labor market supply. They find however that previous studies (e.g. Khandker et al. 2012) may have overstated the gains from electrification by inadequately addressing issues of endogeneity and externalities. Other papers you may want to look at are Dinkelman et al. (2011, 2013 ) who find that electrification in South Africa significantly raises female employment, and hours of work for both men and women; and point to the importance of accounting for endogenous population movements when evaluating welfare gains of spatial programs.
A whole other strand of literature is learning about the demand side—what in this sector is considered as peripheral questions. In 2006, I met Luiz Maurer, then the co-Team Lead of an energy access project in Ethiopia. He was singularly responsible for making me aware of the value of demand-side management in the energy sector. We set out to design two RCTs with EEPCO, the Ethiopia energy company. The first RCT was intended to understand the demand for household connections using lotteries to vary the discount on connection costs. The results were increased but flat uptake at any level of discount suggesting that factors other than cost may have been responsible for low connection rates. The second RCT intended to lower barriers to the adoption of CFLs (the energy efficient light bulb of choice) by amortizing their cost in EEPCO’s bills. The RCT failed when the government scaled up the distribution of CFLs nationwide to lower pressure on generation capacity. Costolanski et al. (2013), however, went ahead to estimate a 13.3MW saving energy savings from the distribution of those (350,000) light bulbs. As the results make clear, take-up and adoption of energy efficient technology are important questions that help us consider a wider range of possibilities on the supply and demand side.
First world utilities extensively exploit the use of experiments to understand and change consumer behavior but this is hardly tapped in the developing world. It is due in part to the state of the utilities’ information systems, but mostly due to the lack of know-how. The advantage of demand side interventions—from information and pricing, to monetary and non-monetary incentives, to regulation and rationing—is that they are relatively cheap to implement. Need convincing on this perspective? Take a look at the selection of studies that make this simple point. Ian Ayres (2009) documents the sway of Opower’s peer comparisons on customers’ electricity and natural gas usage in California. Mailing peer feedback reports to customers reduce their energy consumption by 1.2 percent to 2.1 percent. Exploiting exogenous variation in prices, Bastos et al. (2014) find that tariffs increases caused significant declines in gas consumption in Buenos Aires. Jessoe and Rapson’s (2012) aptly titled “Knowledge is (Less) Power”, details how households experiencing price increases reduce demand by 0 to 7 percent whereas those exposed to information feedback on price increases (via electronic displays), reduce demand by a much larger margin (8 to 22 percent). In an evaluation of Time-of-Use (TOU) rates in Ontario, Canada, Faruqi et al. (2013) show the potential for large changes in the intensive margin of energy use, with users shifting loads off-peak and mid-peak from 1-6 percent of demand. This is of great interest to all utilities struggling with peak-hour overload.
Kotchen (2010) finds that voluntary initiatives to reduce greenhouse-gas emissions increased the number of households who purchased green energy by 35 percent. In Cash for Coolers, Davis et al. (2012) find ambiguous effects of incentives for appliance replacements in Mexico. More efficient refrigerators cut energy use by seven percent and more efficient air conditioners induce increased consumption and net energy use (the rebound effect again!). Regulations have also been known to stimulate consumer response. Jacobsen and Kotchen (2009) find that the 2002 building energy codes put in place in Gainesville, Florida, is associated with a four percent decrease in electricity consumption and a six percent decrease in natural gas consumption. And finally, using utility data, Chico Costa’s analysis of the long-run effect of short-term rationing during the 2001 energy crisis in Brazil tells us that behavioral interventions can be long lasting. The nine-month compulsory rationing program had large (14 percent) and persistent effects on energy consumption reduction ten years hence (blog). In short, by learning how to leverage the large and untapped potential of demand-side interventions, we have an opportunity to significantly and positively influence national demand.
More generally, Costolanski et al. (2013) and Davis et al. (2012) also help us make a point about the need for impact evaluation in this sector. Their energy savings estimates are well below engineering estimates. This is because impact evaluation measures the combined effect of the technology and the behavioral response to its introduction, which, in this case induces a more intensive use of light bulbs and air conditioners. This “rebound effect” makes explicit why ex-ante simulations will calculate standard engineering estimates that will likely be wrong when calculating energy efficiency effects.
Join the Conversation