Measuring success in energy-efficiency financing
Growing momentum for energy-efficiency financing in the United States has motivated State and Local Energy Efficiency Action Network to conduct around 20 interviews with stakeholders in five states to explore what it takes to make utility-sponsored programs succeed.
The research team produced a report, "Making It Count: Understanding the Value of Energy Efficiency Programs Funded by Utility Customers (PDF)," that outlines the pitfalls and promises of a wide range of evaluation techniques.
The interviews revealed that questions about cost-effectiveness screening and incremental impacts are coming up regularly, said Chuck Goldman, staff scientist and division director of the Energy Analysis and Environmental Impacts Division at Lawrence Berkeley National Laboratory.
Interviewees from California, New York, Connecticut, Massachusetts and Maryland gave the authors feedback. The authors also reviewed regulatory and evaluation documents from these states.
"What are your policy objectives? How will you know if you’re successful? How will they judge if you’re successful? These are what we call interim metrics," Goldman said.
Financing for energy efficiency is not typically evaluated outside of larger-scale program evaluations, the researchers said.
Goldman said conversations about barriers, target audiences, success measurements and metrics can help create successful programs.
"Everybody’s doing this a different way," said Chris Kramer, senior consultant at Energy Futures Group. He said two different programs can look at the same data in different ways and get different answers.
"Take a look at the basics," said Kramer. "People [often look] at loan volume. There's nothing wrong with that. I'm guilty of that too. But evaluation can be a nice way of putting some context around the basics."
Kramer said he recommends thinking unconventionally about evaluation metrics: "Think about the other 91 percent of your customers who aren't using that product."
Taking an in-depth look at what evaluation strategies work and do not work, the researchers categorized two program approaches that predominate in the United States. While some programs use financing to complement their existing offerings, others use it as a substitute for them.
Two general recommendations that emerged, Kramer said, were that programs should consider the unique features of financing options and also that they should have robust requirements to estimate potential and actual impacts.
"You may want to consider whether your overall benefits go up or down," Kramer said. "The most cost-effective thing you can do is to invest until your last dollar is no longer worth it."
Where financing needs to gain more traction, program developers should seek inventive ways to amp it up.
Lack of in-depth data on information, education and loan performance holds back the growth of these energy-efficiency financing programs. The authors recommend that utilities leverage customer-funded credit enhancements and direct investment to help kick-start these markets.
To supplement ratepayer funds, the authors recommended that utility programs seek additional cost contributions from customers.
To simplify the bureaucratic structure around energy-efficiency programs so that they can attract private capital, some developers are constructing programs that have fewer reporting requirements than earlier ones did.
But according to Kramer, this may not always be a good idea. "The programs that operated within the traditional structure tended to have more of an incentive to apply the planning and performance tools that we talked about."
Kramer said he advises looking beyond cost-effectiveness tests. "The TRC and societal tests don't actually look at who's paying for what. They just look at what's best for the jurisdiction or society as a whole."
Examining costs in more detail, Kramer said, can yield interesting data. "What are our real costs? If you're putting it at a below-market rate, what's the cost there? How do you deal with interest cost? All of these are things you have to wrestle with."
Decision makers are thinking about how to account for measure costs, write-offs, loss reserves and opportunity costs.
Cost-allocation can present some analytical challenges as well, Kramer said. "If you allocate money toward actual loan capital and put that money out as loans, that may not be considered a cost until there are loan losses."
Theoretically, performance incentives for financing programs could be brought into the picture, but so far, this is still on the drawing board.
Questions about savings attribution can be thorny.
"Is the financing important to those who use it? This is where things get a lot trickier," Kramer said. "If you didn't have that program or strategy in place, would you have gotten those savings?"
Kramer said he recommends asking the following questions:
- Did the customer need the financing in general?
- Did our particular financing program really help us get more customers and more savings — or would the private market have filled in?
- Would the program have occurred without incentives or other project offerings?
"There are a number of states that have adopted legislative requirements to ramp up energy efficiency," Goldman said. "There is increasing interest by private-sector entities."
Some states such as California, New Jersey, Connecticut and New York are seeking to build more financing activity, Goldman said. "We wanted to make sure those states have a framework."
Climate finance may provide additional momentum to back up these efforts.
As programs ramp up their ambitions to meet higher targets, Kramer said, they need to reach toward more challenging objectives such as whole-building retrofits.
When discussing underserved markets, "the markets we typically talk about are small business, multifamily and moderate-income single-family," Kramer said.
A variety of obstacles slow down the growth of these markets, blocking progress for financiers, business owners and homeowners.
"Sometimes it’s not as easy to convince financiers that energy efficiency will be cash-flow positive," Goldman said.
Access to capital and low credit scores pose problems as well. "If access to capital is a barrier, you’d better be looking at what the other barriers are too and addressing those," Kramer said.
"Sometimes building owners are not sure how long they want to own the properties," Goldman said.
On-bill financing and property-assessed clean energy can help programs to surmount these barriers, depending on the specific challenges programs face.
"We’ve seen some programs with alternative loan underwriting criteria perform well, but you have to ask why those participants signed up," Kramer said.
The energy efficiency market may build traction the way the solar market has, depending on how financing grows, Goldman said. "We know that in photovoltaics, people have been able to finance those projects and sell those loans in secondary markets. We’re still going up that curve for efficiency. It’s not as far along."