The International Energy Agency consistently underestimates wind and solar power. Why?

By David Roberts@drvolts Oct 12, 2015, 2:30pm EDT

"This glass is clearly two-thirds empty."

The International Energy Agency was created in 1974 by countries that had just been through a bruising oil crisis (and were headed for another). Some 23 participants in the Organisation for Economic Cooperation and Development (OECD) founded the IEA to gather and share information about energy, model future energy trends, and help mitigate the adverse impacts of (or avoid) subsequent energy crises.

Since then, IEA has become a widely respected source of energy data and analysis. Its annual World Energy Outlook (WEO) is considered the gold standard in energy modeling, producing endless media coverage and shaping the assumptions of policymakers and the investment class.

It is somewhat vexing, then, that the IEA has always been, and remains, dismally pessimistic about wind and solar energy. This pessimism has led it to underestimate wind and solar again and again, a track record of failure one might think would trouble an agency known for the quality of its modeling. But if it's troubled, IEA hasn't let on.

What's more difficult to figure out is why. Why does the IEA continue to lowball renewables, even in the face of persistent critiques? There are several stories about this floating around, and none of them are entirely satisfying.

(Side note: Virtually every critique described in this post can also be applied to the US Energy Information Administration, which produces forecasts similar to IEA's, focused on the US.)

We'll have a look at the explanations on offer, but first let's establish our premise:

The IEA done slept on wind and solar

That the IEA has historically underestimated wind and solar is beyond dispute. The latest look at the issue comes from Energy Post editor Karel Beckman, who draws on a recent report from the Energy Watch Group (EWG), an independent Berlin-based think tank. The report analyzes the predictive success of previous WEOs.

Here's the history of additions to electric generation capacity by renewables excluding big hydro, along with successive WEO projections:

As you can see, IEA keeps bumping up its projections, but never enough to catch up to reality. It's only now getting close.

It gets even worse when you dig into the details. Here's the bill of particulars:

WEO 2010 projected 180 GW of installed solar PV capacity by 2024; that target was met in January 2015.
Current installed PV capacity exceeds WEO 2010 projections for 2015 by threefold.
Installed wind capacity in 2010 exceeded WEO 2002 and 2004 projections by 260 and 104 percent respectively.
WEO 2002 projections for wind energy in 2030 were exceeded in 2010.
Other, independent analysts (like those at Bloomberg New Energy Finance and Citi) have come closer to accurately forecasting renewables. The only forecasts that match IEA's inaccurate pessimism are those from the likes of BP, Shell, and Exxon Mobil.

Here are IEA's wind and solar projections broken out, from a 2014 post by the folks at eco-consultancy Ecofys:

Back in 2013, energy analyst Adam Whitmore took a look at the IEA's track record on renewables. He found it abysmal, like everyone else. This year, he returned to the WEO to see if it has improved and found that, well, it hasn't.

Here he shows the rate of growth in annual installations of renewables, and what the IEA projects for the future:

(The dashed lines are the standard WEO projections, what happens if nothing changes. The dotted lines are from the "bridge scenario" in the WEO Special Report on Energy and Climate Change, which is supposed to represent some policy ambition.)

As Whitmore says, it's possible that the rate of solar PV installations will suddenly plunge by some 40 percent and then enter a long steady-state period, but there's no reason to think it's particularly plausible.

IEA is particularly, inscrutably bad on solar, as this post from Terje Osmundsen lays out in some detail. For example, here's the IEA's estimate of the investment cost of large-scale PV in 2020, compared with those of other scenarios:

That's a little crazy. Large-scale PV is cheaper than that in some places already.

Enough. The point is made: IEA keeps underestimating renewables. But why?

People have all kinds of stories to explain IEA's pessimism

In reading around and talking to people about this, I've run across a number of theories about IEA, some more plausible than others. Here's a short (likely non-exhaustive) list.

1) IEA is in hock to fossil fuels

Perhaps the most common popular theory is political: The IEA reflects the interests of the most powerful energy sectors, i.e., fossil fuels. Fossil fuel industries are strongly biased toward preserving the status quo; IEA reflects that bias in the assumptions that shape its modeling. Its modeling results, which show the status quo changing very, very slowly, are in turn meant to mislead governments into underinvesting in wind and solar.

But this account of the political economy at work is puzzling. As Beckman says:

even though the IEA may have underestimated the growth potential of solar and wind, it does consistently call on governments to support those technologies vigorously. Similarly, it has been urging policymakers for many years to take action on climate change. In an interview Energy Post had with Fatih Birol last year he said that "radical action is needed to actively transform energy supply and end use." Birol and the IEA have been giving out similar messages for many years.

The IEA has special reports and working groups devoted to clean energy. It advocates for energy transformation at every opportunity. If it were out to undermine wind and solar, why would it do so much to support them?

It could be argued, in fact, that underestimating renewables has given governments more incentive to invest in them, not less.

(Then again, underestimating renewables also serves to exaggerate the level of subsidies needed to support them, as Osmundsen notes. So, yeah, it's complicated.)

I'm not sure how one would prove the case that IEA projections have shaped investment decisions, or show exactly how, but I haven't seen much real evidence offered one way or the other.

2) Institutional conservatism

Whitmore speculates:

There may also be a deeper explanation rooted in institutional conservatism. Taking a conservative view of future prospects in the energy sector can be necessary to avoid being swayed by the latest fad. A conservative view recognises the realities of the long time horizons and vast scale of the world’s energy systems. However it can carry the risk of missing the role of genuinely transformative technologies, as appears to be the case here.

This is important to remember. The global energy system has traditionally been huge and slow-moving. Here's a graph of global electricity market shares from 1980 to 2014, compiled by Jessica Lovering, featured in a post by Jesse Jenkins:

global electricity market shares, 1980-2014

Puts the "explosive growth" of renewables somewhat into perspective, no? Global energy moves slowly. Various hyped trends have come and gone without much changing the big picture. IEA is likely predisposed to small-c conservatism, and rightly so, even if it does raise the likelihood of missing trends that do matter.

3) One simple error

The EWG report has frustratingly little to say about why IEA keeps getting renewables wrong, but it does say this (emphasis added):

A key reason for the high deviations of solar PV and wind capacities in the projections and the historic data is an incorrectly applied growth pattern. The WEO reports assume linear growth, whereas history shows an exponential growth for the new renewable energy (RE) technologies. The current exponential growth is part of long-term logistic growth of new RE technologies.

Disruptive technologies, when introduced, tend to grow slowly at first, then shoot up exponentially, then level out into something like linear growth — this is the S-shaped "logistic growth" curve. Here are several such curves, in a much-celebrated graphic by Nicholas Felton of the New York Times:

As wind and solar mature, reach the top part of the S-curve, and settle into something more like linear growth, IEA's forecasts are likely to get more accurate. (It's already happening a bit with wind.)

Whitmore mentions the exponential growth issue too:

There may have been a reliance on individual jurisdictions’ plans, with more caution than seems with hindsight to have been warranted about the rate at which policy might move. This seems to have led to linear extrapolation of capacities when technologies were in a phase of exponential growth.

This brings us to a fourth possibility.

4) Policy conservatism

The IEA produces several scenarios:

The Current Policies scenario, which assumes current policies remain in place and no new policies are passed.
The New Policies (sometimes Bridge) scenario "broadly serves as the IEA baseline scenario. It takes account of broad policy commitments and plans that have been announced by countries ... even if the measures to implement these commitments have yet to be identified or announced."
The 450 scenario represents what must be done to meet the "internationally adopted goal to limit the rise in long-term average global temperature (with a likelihood of around 50%) to 2 °C."

None of these scenarios is, strictly speaking, a prediction. They are only scenarios, elaborate if-then statements. (This confusion between scenarios and predictions bedevils the relationship between modelers and policymakers.)

The problem, as Ecofys modelers point out, is that the baseline scenario has woefully trailed reality. The scenario that has most closely matched the actual growth of wind and solar is the 450 scenario:

Part of this gap can be traced to oddly high cost assumptions and other features of the WEO model. But part is due to the fact that policy is simply moving faster than the IEA has been willing to contemplate.

Here we might find some sympathy for IEA. It's obviously not realistic to think that current policy will freeze in place. Indeed, there is good reason to think that policy ambition in the next 20 years will far exceed what is being explicitly promised by nations today.

But ... by how much? How will the politics develop? It's not clear that IEA is well-positioned to know, or that anyone is, really.

Policy supports are slowly becoming less essential to wind and solar, but they are essential still, and the evolution of policy in coming years will have an enormous influence on deployment. That introduces a certain irreducible element of uncertainty into wind and solar projections.

5) Modeling is hard

Trying to capture the entire global energy system in a model and project its evolution over several decades is ... well, let's just say it, impossible. To even attempt it, IEA has to use some extremely generalized assumptions that inevitably miss important developments in specific countries or technologies. Adam James, an energy researcher at GTM Research, put it to me this way in an email:

The energy world is very complex, and although a lot of these domino effect changes make sense in retrospect, they are almost never going to be baked into the assumptions made in building these scenarios.

What do I mean by that? Well, looking back, you could have projected that a build-up in polysilicon would lead to a pricing glut; plummeting PV panel prices at the exact moment when Europe had lucrative incentives in place. That then, a PV deployment boom would lead to ratcheting back those incentives, leaving state-backed PV manufacturing in China high and dry. And that to support that industry and meet domestic growth, China would implement their own policy to soak up that supply with demand... but that sure wouldn't have been the Base Case -- and now China is 25% of global PV demand. You could have modeled a nuclear disaster leading to shutdown and then massive power supply shortfall in Japan, and PV rushing to fill that gap -- but again, not likely to be a Base Case. And that's just PV -- now imagine trying to run that kind of scenario-based forecast process across every country for every kind of power generation.

So the reliance on 'easy' metrics like LCOE [levelized cost of energy] and gross/net demand are understandable: [IEA's] Base Case is purely a reflection of their working assumptions, which have to be generalized to hedge against the complexity described above.

In his post, Osmundsen quotes IEA chief economist Fatih Birol, who was questioned about the difference between IEA results and those from industry analysts. Birol said the discrepancy is "because we have other underlying assumptions or because they only look at one technology whereas we look at the power generation sector in totality."

What I take Birol to be saying is that analysts focused only on wind and solar can take the time to do more "bottom-up" work, taking into account actual national policies, market prices, and technology learning curves. The IEA simply can't do that for every country, market, and energy technology in the world — it would need a small army of researchers — so it has to rely on broad, aggregate assumptions based on estimates of GDP and population growth, idealized technology learning curves, and somewhat outdated calculations of cost.

Those broad assumptions inevitably miss things. Research scientist Schalk Cloete took a look at 15 years of IEA forecasts and found that not only did the agency underestimate renewables, it overpredicted oil consumption and underpredicted coal consumption. In fact, in absolute terms, IEA's misses on coal and oil were bigger than its misses on renewables (though the latter is larger in relative terms):

A global energy model like WEO may simply not be built to accurately track emerging, quickly moving energy trends that are highly dependent on policy and regional political dynamics. Such trends are more likely to be spotted and understood by analysts focused on those particular countries and sectors. IEA isn't omniscient.

6) Lousy assumptions

That said, there are assumptions in the WEO that seem difficult to defend on any grounds.

As mentioned, IEA continues to assume that the rate of renewable energy installations will flatten or fall, despite the fact that they've been rising sharply for years:

Why does IEA assume this? No one seems to know.

IEA also assumes that wind and solar cost more than they actually do. I asked technology analyst (and science fiction author) Ramez Naam about this, and he said:

IEA’s [World Energy Model] does attempt to base its forecasts on a dropping cost of solar and wind. But the model’s assumptions are too conservative. In fact, the IEA doesn’t just underestimate the decline of the future cost of solar. It also thinks solar is more expensive right now than it really is. ... Some of what’s happening here is that IEA builds models of what electricity from solar and wind should cost, based on equations that put together the up-front capital cost, the capacity factor of the installations, the availability of good sites, how long those installations should last, and the interest rate the builders pay.

And IEA seems to get it wrong at multiple points. As far as I can tell, their model uses an 8% interest rate (higher than what developers actually pay); assumes that solar installations have a lifetime of 25 years (when evidence is that they decay at less than 1% per year, giving them significantly longer); assumes that good sites are disappearing when, in fact, there’s an abundance of good solar sites, and better wind turbines are opening up new sites for wind; and assumes that capacity factors for solar and wind are static, when, in fact, solar and wind capacity factors are rising over time.

Why does IEA make these gloomy assumptions about renewables when counter-evidence is easily available? No one seems to know.

IEA should update its model assumptions and find new partners

Guess which modeling outfit has done the very best job of predicting the growth of wind and solar power over the past decade?

Greenpeace.

Here's what Sven Teske, the lead author of Greenpeace's Energy [R]evolution reports, told reporter Brian Merchant:

Our projections are much closer to actual renewable energy development than those from IEA because we have monitored global and national renewable energy market development and production capacities carefully since the mid 90s, and discuss possible growth rates with the solar and wind industries. We know what they have in their order books for the next 3 to 5 years and extrapolate it for the next 5 years. This gives us a very good idea about what the renewable energy market will do within the next decade.

This is the bottom-up approach that gives targeted analysts an advantage over generalists like IEA. And that kind of bottom-up work is probably not something IEA is ever going to be able to do, at least in the context of trying to assemble an overview of the entire global energy system.

But what Teske says about Greenpeace's mid- and long-term projections is both interesting and relevant:

Everything beyond projections for the next 10 years is simply a political statement from us, indicating what we want to see happen. This also becomes a work plan for us. If we see a renewable energy market isn’t performing as we want it to, we’ll try to jump in with campaigns—against fossil and nuclear fuels and in favor of renewables.

So Greenpeace doesn't pretend to be able to predict the far future, out past 10 years. It decides what kind of world it wants to see and constructs a scenario to achieve it. Then it sets about making that scenario a reality.

That strikes me as a much healthier way to deploy long-term energy modeling. Currently, policymakers are in a strange ouroboros relationship with IEA projections, deciding what they should do based on scenarios guessing what they might do.

Greenpeace has the better idea: Decide what ought to happen and then use modeling to show that it's possible. Its long-term projections are unapologetically aspirational.

To do something similar, IEA will need to update some its outdated assumptions about wind and solar costs and rates of growth. And though the agency remains focused on a global energy overview, of which wind and solar remain a relatively small fraction, it should work in a concerted way to better understand the internal dynamics of wind and solar markets and not simply transfer the same assumptions that apply to fossil fuel energy.

Osmundsen offers a good idea at the end of his post: IEA should partner with the International Renewable Energy Agency (IRENA), an international think tank created specifically to analyze the development of renewable energy. IRENA's assumptions and projections have proven much more accurate than IEA's. "Governments and stakeholders would be better served if the two organizations put their heads together," says Osmundsen, "and published a joint study on the economics of and potential for renewables in the power sector."

Sounds like a good idea. Rather than pretending to predict what might happen 20 or 30 years from now, lets use energy modeling to give global policymakers something to aspire to.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via