Managing Director & Senior Partner
To achieve net-zero emissions by 2050, electricity systems around the world need to use far more variable renewable energy (VRE) generation, particularly solar and wind power. The transition from fossil fuels to VRE is unlikely to be smooth. As previous articles in this series have shown, VRE creates physical challenges due to its intermittency and variability, with different forms of energy storage emerging as critical solutions. But VRE also creates market challenges for traditional wholesale energy-only markets, with storage unlikely to provide the solution.
Natural gas has a key role to play in setting prices in today’s wholesale energy-only electricity markets. Because of the way prices are set in these markets, gas prices disproportionately determine the price of electricity. As the entry of substantial VRE generation and storage causes coal-fired generators—and subsequently gas-fired generators—to close, it has the potential to undermine this mechanism. Left unchecked, this could lead to high and unstable electricity prices, impairing vital investment signals.
Given the importance of renewables in meeting future global energy needs, are the days of wholesale energy-only electricity markets numbered? Not necessarily. Depending on the dominant technology that emerges to manage the variability of renewable energy and to ensure that adequate supply exists—whether it be zero-emissions gas, energy storage, or flexible demand—market designers and companies can adapt to the new environment, by adjusting the market design or changing their behavior. However, the wholesale energy-only market may play a very different role from the one it does today.
This article is the fourth in a series exploring changes to our electricity and broader energy systems. (See “Other Articles in the Series.”) In this article, we explore how higher penetration of variable renewable energy will impact the way electricity is priced in wholesale energy-only markets and the potential implications for electricity market design.
Starting in the late 1990s, a swath of countries opened their electricity markets to competition to increase efficiency and reduce the cost of electricity. As part of this liberalization, market operators introduced wholesale energy-only markets that permitted different types of power generation plants to be dispatched efficiently and were intended to be the primary signal for new investment. Many different designs for liberalized electricity markets can arrive at similar economic results. As an example, the Australian National Electricity Market (NEM) wholesale energy-only market operates as follows:
Aside from SRMCs, other factors impact generators’ bidding strategies. These include whether there is a limit on the total amount of electricity they can generate (that is, whether they face an energy constraint) and how easily their output can be adjusted. On the basis of these factors, we segment generators into five categories. (See “Electricity Generators Have Different Roles in Price Setting.”)
Typically, gas-fired mid-merit generators (which adjust their output to match demand throughout the day) and peaking generators (which run at times of high demand) set the wholesale spot price of electricity most of the time, either directly or indirectly, so we call them price setters. (See Exhibit 1.) The name is apt not only because they set the price directly when they are the marginal generator (that is, are last in the merit order), but also because they set the price indirectly via the bidding strategies of other generators that use price setters’ bids as a reference price.
We call generators such as hydro generators and energy storage, which have constraints on the amount of energy they can produce, price opportunists because they base their bids on the opportunity cost of deploying that energy at another time. Price opportunists’ supply bids are intended to maximize revenue from the limited amount of electricity they can generate and are typically set just below those of price setters. Consequently, their bids are often impacted by the price setters’ fuel costs.
We call generators with limited flexibility and no energy constraint price followers. These generators typically have a low SRMC. In most instances, price followers bid on the basis of their SRMC. On other occasions, their bids reference the SRMC of the next-highest price-setting technology to earn more revenue when they are the marginal generator—a strategy known in Australia as shadow bidding. Such bidding strategies extend the influence of gas prices on the price of electricity.
The bidding strategies of different types of generators ensure that price setters and the natural gas that powers them have a disproportionate role in determining the price of electricity. The consequence can be seen in Australia: although gas-fired generators contributed only 12% of power generation from 2014 to 2021, they directly or indirectly impacted the electricity price for nearly 75% of total power generation.
Gas-fired generation also plays a dominant price-setting role in the European Union, largely as the marginal generator. There, gas-fired generators represent less than 20% of total generation but set the price more than two-thirds of the time.
As a result of this effect, gas prices materially affect electricity prices, with movements in gas prices leading to significant changes in electricity prices. For example, from 2012 to 2020, the average monthly wholesale spot price in Australia increased by $8/MWh for every $1/GJ rise in the price of gas.
This relationship also means that crises in gas markets, such as the current global natural gas market crisis, spill over into electricity markets. For example, from May 2022 to July 2022, Australian domestic gas prices reached $40/GJ, a four- to five-fold increase over typical prices, and this increase flowed through to electricity prices at a similar multiple of typical prices.
As VRE generation increases, the amount of electricity that gas-fired generators produce declines. It might be tempting to think that greater VRE generation would also reduce the role that gas plays in setting the price of electricity. But its price-setting role persists.
In Australia, the amount of electricity that gas-fired generators produced fell from 15% in 2014 to just 8% in 2021 due to increased electricity supply from VRE sources. Over the same period, the role of gas as the price setter did diminish, although by a comparatively smaller amount—from 70% of the time in 2014 to 50% in 2021. However, this change hides an underlying cycle of events and underplays the persistent role of gas.
At first, as VRE enters the market, the influence of gas as a price setter declines. This is because competition from VRE sources causes coal-fired generators to more frequently bid at their SRMC rather than shadow-bidding on the basis of gas-fired generators’ bids. Subsequently, however, mid-merit coal-fired generators exit the market because they lack the flexibility to operate during long periods when the market price is below their SRMC, and this causes gas to resume its dominant price-setting role. This cycle occurred in Australia in 2017 and is likely to recur in 2023 as additional coal-fired generating capacity closes. The result is that gas remains highly influential in setting the electricity price in Australia, despite commanding a reduced share of generation.
Over the long term, the natural-gas-fired peaking generators that remain in operation hinder the goal of a zero-emissions electricity system. As a result, without zero-emissions fuels or abatement technologies, they would be forced to shut. The withdrawal of these generators, as well as of emissions-intensive price followers, would mean that the only generators left to set the electricity price would be price opportunists, such as storage.
As we mentioned earlier, because price opportunists are energy constrained, they look for the best pricing opportunity at any given time. Consequently, they tend to submit supply bids that take advantage of the high prices that price setters secure at times of peak demand. But if the price setters were to exit the market, price opportunists would have no yardstick for deciding the level of their own bids. Electricity price formation would effectively be decoupled from its key external reference price—the price of gas—causing electricity prices to fluctuate between extremes. This, in turn, would make financial derivatives and hedging contracts extremely expensive, as markets struggle to price in this level of risk. The spot market would no longer be the main driver underpinning investment decisions.
Recent events in Australia show how this situation could play out. In late June 2022, the Australian electricity market became energy constrained. First coal-fired generators (due to outages, flooding at coal mines, and supply chain issues) and then gas-fired generators (due to increased gas demand and lower storage levels) became fuel constrained. As a result, the economics suggests that these generators tended to behave more like price opportunists, generating electricity at the most valuable times. This would imply that electricity prices might be limited not by gas prices but instead by a price-capping mechanism known as the cumulative price threshold. For example, in the week prior to July 17, when gas supply constraints were most challenging, the demand-weighted average electricity price across the mainland NEM states was $500/MWh—above the SRMC of peaking generators.
What role will wholesale energy-only market pricing play in the future? How must electricity markets evolve to produce clear price signals and drive effective investment decisions? We believe that the answers to these questions depend on which of three types of technology emerge as the dominant, physical firming solutions (see Exhibit 3):
External factors such as technology costs, local resources, and the makeup of local industry will determine which technologies are the dominant firming solutions in different markets around the world.
The implications for market players depend on whether the dominant solutions are energy constrained, as in the case of storage, or energy unconstrained, as in the case of gas-fired generators, and whether the solution is on the supply side or the demand side. (See Exhibit 4.)
Generation powered by zero-emission gases such as green hydrogen, biogas, or natural gas with carbon capture, utilization, and storage (CCUS) technology could replace today’s natural-gas-fired generators.
In this scenario, sufficient price setter generating capacity could exist to enable price setters to set electricity prices in the spot market, and wholesale energy-only markets would continue to play a meaningful role in driving investment, as they do today.
Energy users with highly flexible demand could play a crucial role in setting the price of electricity. Until now, end-users have not participated directly in the wholesale energy-only market at scale. Instead, they have used out-of-market demand response mechanisms: reducing their consumption during extreme demand periods.
However, some energy users share important characteristics with today’s price setter generators that make them effective replacements. They have controllable production processes and can vary their output over extended periods. In other words, their electricity demand is not time critical. Most importantly, they can base their production decisions on an external reference price—the revenue they earn by selling their product weighed against the variable cost of the electricity and other inputs needed to produce it.
Some of these users, such as hydrogen producers and crypto miners, can treat production as entirely optional. Others, such as steelmakers using electric arc furnaces (EAFs) and smelters, may have some degree of flexibility in the total volume they produce and can use stockpiles to create flexibility in their production schedules.
This flexibility puts them on a different level from users that can flex their electricity demand for short periods but must meet an external constraint over a longer time period (time-critical demand). Users that fall into the latter category include electric vehicles (EVs), which must be charged to run, and water networks that process and pump water.
To set the price of electricity, highly flexible users such as hydrogen producers and steelmakers would conduct a two-step process:
Significant volumes of price-setting demand could either replace or supplement the role of gas-fired generators in wholesale energy-only markets and drive investment decisions. Rather than rationing flexible supply to meet fixed demand, the wholesale energy-only market would ration flexible demand to meet the variable supply of renewable energy.
Will there be enough energy users possessing these characteristics to make a difference? There certainly could be in markets that produce significant volumes of hydrogen for export or that have large energy-intensive manufacturing industries.
Energy storage technologies are emerging as a promising solution for the physical challenges posed by a high penetration of VRE because they firm VRE supply to meet demand. But they do not solve the market challenges.
Because storage and similar solutions are price opportunists, wholesale energy-only markets would create periods of unacceptably high volatility for both consumers and investors. This would also be the case if flexible demand with price opportunist characteristics (such as EVs with a time critical electricity requirement) were a major firming technology.
Electricity markets would have to undergo a fundamental redesign so that longer-term contract markets, such as capacity markets, or separate markets for variable power (with generation “as available”) and firm power (with generation “on demand”) drove investment decisions. Under this approach, remuneration in the variable power market would be based on contracts for difference, with prices reflecting the long-run marginal cost of existing generators. Emerging market redesign proposals from the governments of Greece and the UK are beginning to countenance such a split.
The wholesale energy-only market would have a limited investment role. Its primary raison d’être would be to signal the efficient dispatch of different types of generators, storage, and flexible demand via a merit order. If this redesign doesn’t occur, companies will need to contract outside of the market—for example, by increasing vertical integration or by entering into long-term contracts for energy and capacity.
As the world moves toward greater VRE generation, liberalized electricity markets face a crisis of purpose. Up to now, thermal generators—particularly natural gas-fired generators—have had a dual function, providing both physical stability and market stability to our electricity systems. But while electricity systems may be able to function physically without natural gas, electricity market pricing will break down if other solutions don’t replace the role that gas plays in setting wholesale spot market prices for electricity. Markets will need to be fundamentally redesigned to adapt to this new environment.
How we best adapt our electricity markets in the long term depends on which types of technology—zero-emissions gas, highly flexible demand, or energy storage—become the dominant firming solutions. As such, market designers can’t make decisions in isolation from technology. Moreover, if they are to make net-zero electricity systems a reality, they must take care that their reforms go beyond solving today’s issues and ensure that their designs are compatible with likely technological developments.
ABOUT BOSTON CONSULTING GROUP
Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we work closely with clients to embrace a transformational approach aimed at benefiting all stakeholders—empowering organizations to grow, build sustainable competitive advantage, and drive positive societal impact.
Our diverse, global teams bring deep industry and functional expertise and a range of perspectives that question the status quo and spark change. BCG delivers solutions through leading-edge management consulting, technology and design, and corporate and digital ventures. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, fueled by the goal of helping our clients thrive and enabling them to make the world a better place.
© Boston Consulting Group 2023. All rights reserved.
For information or permission to reprint, please contact BCG at email@example.com. To find the latest BCG content and register to receive e-alerts on this topic or others, please visit bcg.com. Follow Boston Consulting Group on Facebook and X (formerly Twitter).