This is the first of two articles on how data, tech, and AI are shaking up commodity trading. The second article will focus on the practicalities of transforming power and gas trading.
Commodity trading has entered a new phase. The extraordinary profits generated during the geopolitical shocks of 2022 and 2023 have faded. Geopolitical volatility remains elevated, but is harder to monetize.
During the 2022–2023 crisis, it was enough to have the right asset position, logistics footprint, or market exposure to capture windfall profits. Today, performance depends far more on how quickly a company can process unstructured data, allocate risk, and trade across interconnected markets. These new capabilities also allow incumbent commodity traders to compete against hedge funds and tech-first firms that, spurred by high energy profits of 2022–2023, have successfully expanded into commodity trading with highly automated trading models.
Opportunities such as recent disruptions in the Middle East still appear, but they require quick, decisive action, so slow-moving organizations miss out. Many trading organizations understand this shift in principle. Far fewer embrace the new approach that the market now demands.
The Evolution of Commodity Trading
We originally described this influence of digital technologies on commodity trading in a 2016 article. Yet the trends from that decade not only persist, but have become even stronger. Trading is becoming more systematic, and data- and model-driven strategies are replacing more discretionary approaches. Real-time visibility is taking the place of batch processes. Unified data platforms are replacing fragmented tools, and automation is spreading across the trading life cycle. Operating models are moving from silos to cross-functional teams, with technology and analytics teams often sharing reward for successful strategies.
The role of energy trading and risk management (ETRM) systems is shrinking to trade booking and compliance. Higher-value work—forecasting, pricing, optimization, signal generation, and analytics—is migrating to modular platforms with a shared central data hub. This setup enables real-time data streaming across systems, faster integration of new platforms and applications, and more flexible front-end interactions. (See Exhibit 1.)
Advanced capabilities of a few years ago, such as isolated automation or classical machine learning, have quickly become routine. As markets move toward shorter-term, data-driven optimization, even organizations with a strong physical trading and optimization footprint benefit from scaling automation and analytics.
In 2019 we introduced our “Trader of the Future” framework, which describes four stages of maturity in the digitization of commodity trading, ranging from the simple deployment of electronic platforms to fully autonomous digital intelligence systems. The framework has proven both durable and farsighted, anticipating future developments. But seven years is a long time—look at the advances in AI—so we have tweaked the framework (see Exhibit 2) and the required capabilities to bring it to life. (See the sidebar “The New Capabilities Required.”)
The New Capabilities Required
- Vision defines the organization’s ambition for trading, including the role of technology, AI, data, and culture in driving performance and competitive advantage.
- Governance establishes how digital trading (including AI) is controlled and incentivized, including risk management, compliance, performance metrics, and P&L ownership.
- Team and capabilities cover the structure, skills, and career models required to combine trading, quantitative, and engineering expertise effectively.
- Data determines how this valuable asset is sourced, governed, and made accessible, including its quality, granularity, and integration across the organization.
- Signal creation captures the ability to generate trading insights, including forecasting, optimization, pricing, and the use of advanced analytics, AI, and machine learning.
- Trade execution describes how trades are executed and assets are controlled, ranging from manual execution to fully automated, algorithmic trading.
- Trade life cycle covers end-to-end process automation, from trade capture and confirmation to settlement, reporting, and contract management.
- Risk defines how organizations measure, monitor, and manage the downside, evolving from static metrics to real-time, scenario-based steering.
- Technology describes the underlying architecture, systems, and platforms, including AI and data infrastructure, analytics environment, and cloud capabilities.
Shifting Market Dynamics
In the past seven years, trading performance has increasingly been driven by four structural market dynamics:
- Volatility is becoming more episodic and high-impact. Single events can create outsized P&L swings. But these events are difficult to predict and monetize, shifting risk strategies toward smaller, diversified exposures rather than large directional bets.
- Value pools are shifting toward short-term, data-driven optimization. For example, rising renewables penetration is increasing short-term volatility and compressing trading windows to 15-minute intervals—and even less in the European power market. These developments reduce the role of discretionary trading and increasing the importance of systematic, real-time optimization.
- AI and advanced data capabilities are sufficiently mature to be deployed at scale across trading activities. Advances in data engineering and machine learning enable large-scale signal generation, probabilistic modeling, and tighter integration of analytics into trading decisions. AI agents can handle parts of middle- and back-office processes.
- Technology stacks are widely available to support real-time, integrated trading workflows. Organizations can develop in-house platforms faster than ever, thanks to GenAI, and use strong vendor solutions that enable continuous data ingestion, analytics, and automated execution. Advanced trading setups are accessible to any organization able to implement them effectively.
Uneven Progress
Not all companies are maturing at the same pace in this new data-enabled world. For instance, the trading arms of international and national oil companies, asset-backed owners, and smaller traders still mostly operate with classical or only partially automated setups. Leading European utility traders, select oil companies, and a few global merchants have progressed to level 2, which is characterized by real-time optimization, pricing, and risk management.
The most advanced players—primarily hedge funds, a few leading utility traders, and tech-native entrants—are progressing toward the AI-ready level 3, in which humans oversee trading algorithms. (See the sidebar “The New Competitors.”) Level 4—fully autonomous decision making and execution, run by agentic AI—is still a theoretical state, beyond the requirements of most physical commodity trading organizations due to regulatory constraints, market speed, and the need for human judgment in complex trading activities.
The New Competitors
Different Commodities and Markets Require Different Approaches
All of these dynamics vary by market. In European power and gas, renewables and weather-driven volatility have hugely increased the importance of short-term optimization. Algorithmic trading, real-time forecasting, and fast execution are required to compete. In the US gas market, the growing presence of hedge funds and other financial players and infrastructure bottlenecks have created a more hybrid market. Both model-driven trading and optimization of physical assets still matter. The US power market is becoming more complex and localized. Congestion, nodal pricing, and the rapid growth of day-ahead and real-time (DART), and other short-term markets increase the need for advanced analytics, deep market expertise and scalable technology.
In global seaborne oil and liquefied natural gas (LNG) markets, the edge increasingly comes from reading global flows well and moving quickly on freight, arbitrage, and portfolio opportunities. In onshore and regional liquids markets, value is still shaped more by local supply chains, infrastructure, and physical market insights. In both markets, oil majors and large independent merchants are investing heavily in financial trading, data, and analytics to find new sources of value and protect margins.
What Market Leaders Do Differently
Structural capabilities rather than market conditions are now defining success. To turn advanced capabilities such as real-time data, probabilistic analytics, and automated execution into P&L, firms need to build them into the way teams work.
The old model centered on the individual trader’s market instinct is giving way to a more team-based approach built on shared intelligence. Best-in-class players consistently treat data and technology as core drivers of value creation and align incentives so that quants, analysts, and tech team share directly in the upside from successful trading strategies.
The old model centered on the individual trader’s market instinct is giving way to a more team-based approach built on shared intelligence.
This approach is backed by higher and more targeted technology investment. Leaders often allocate 20% to 25% or more of their cost base to technology and reinvest 10% or more of gross margin into platforms and people capabilities. (See Exhibit 3.)
Three groups of players are using technology to pull ahead in commodity trading, particularly in power and gas:
- Hedge funds run highly integrated platforms that can move swiftly and deploy capital dynamically using probabilistic models. They can simulate the impact of supply chain disruptions in real time. Their in-house meteorology teams often are more accurate than external weather forecast providers. Their willingness to take large positions can produce outsized returns from greater risk. While most of their gains have been captured from proprietary trading in financial markets, some have successfully invested in asset-backed trading to diversify and gain additional information advantage.
- Tech-native players entered commodity trading by building data-centric platforms and strong in-house technology capability from the start. As early movers, they have been able to automate more aggressively, iterate more quickly, and integrate trading and engineering much more tightly. They are especially effective in short-term markets, but are increasingly expanding their long-dated portfolios. While most of these players started in Europe, they have been successfully rolling out their capabilities in less mature geographies such as North America and selected markets in Asia-Pacific.
- Advanced European utility incumbents show that transformation is possible inside legacy organizations. Many have modernized data platforms and automated parts of short-term trading and operations, particularly in power and gas, although older systems and organizational complexity can still slow progress. Lagging players remain constrained by fragmented data, outdated technology, and siloed operating models. They have been unable to leverage modern tech and AI to their full potential.
What Trading Organizations Should Do
The gap between leading and lagging trading organizations is widening. Most organizations need to rethink how trading operates, how technology supports it, and where the business wants to compete.
Start at the end by defining a clear ambition. Leadership teams need to define the role trading should play in the organization, how capital should be allocated, what level of technology maturity they are seeking, and where they want to build an edge. Trading and trading tech leaders should demonstrate commitment to change because transformation will challenge existing ways of working and established profit pools. Value emerges when tech capabilities are scaled and connected across the trading life cycle rather than built in isolation.
Value emerges when tech capabilities are scaled and connected across the trading life cycle.
Clean the tech house. A strong, integrated data and technology foundation should be treated as a core source of value creation. Organizations need architectures that support real-time decision making, shared data layers, and better integration across systems.
Technology investment also needs more discipline. Many firms have spent heavily on scattered upgrades or desktop tools without solving the underlying problems. The real gains come when capabilities are redesigned end to end around a clear business vision.
Challenge how work gets done. The operating model should be redesigned for speed and scalability. Technology and AI will not improve performance if they are laid on top of broken processes and siloed teams. Trading organizations need to reinvent workflows end to end by embedding data and technology capabilities into the business and maintaining a shared backbone of core platforms.
Speed matters here as much as design. Firms that can deploy in waves, learn quickly, and scale what works will have an advantage over those stuck in long, fragmented transformation programs that struggle to move beyond pilots and narrow use cases.
Invest wisely in AI and automation. These tools should be applied selectively and with clear commercial intent. Many firms can still benefit from simpler forms of automation, simplification of the legacy tech stack, and smarter use of third-party tools before moving to more advanced applications. AI has the highest potential in areas such as coding, forecasting, optimization, signal generation, and selected middle- and back-office processes. But leaders need to stay focused on areas with clear business value rather than chasing broad, unfocused, and superficial AI adoption.
Lead the change. Organizational resistance is real. Senior traders may still perform well under the current model, while operations teams may see efficiency gains as a threat. That is why successful transformation requires aligned incentives, a clear case for change across each group, and engaged leadership teams willing to challenge entrenched behaviors. Leaders should treat culture and change management as core parts of the effort rather than side issues.
To become a trader of the future, companies need more than a tech upgrade. They should transform the business, operating model, and the way works gets done.
Acknowledgments
The authors thank Fabian Wagner and Hao Zheng for their contributions to the writing of this article.
