Short-term forecasting in low-carbon energy systems

Jethro Browell, Professor of Statistics and Data Analytics at the University of Glasgow

29 May 2026

Forecasts of energy supply and demand from seconds to years ahead drive the decisions that ultimately determine where the energy we consume comes from, and at what cost. In the long-term, policy is made, infrastructure is built, maintenance is carried out. Energy is bought and sold from years ahead to just a few minutes before delivery. From seconds to days ahead, plans are made and re-made to ensure the reliable physical delivery of energy to end-users. Everything is put in place so that when we flick a switch, the lights turn on.

This blog series has previously highlighted the role of AI in forecasting, and indeed current best practice involves using Machine Learning and AI algorithms to learn patterns to predict energy production/consumption from weather and contextual data, be that power production from a wind farm or electricity consumption for a neighbourhood. However, the transition to net zero is already disrupting the practice of predicting demand and scheduling supply to match it. Weather-dependent renewables with no fuel costs introduce uncertainty on the supply side, and demand is increasingly flexible and less predictable.

Here I use “AI” broadly to include statistics, machine learning, and optimisation, as well as generative AI.

Potential roles for AI

Opportunities for AI in this area fall into two categories: improving forecast performance, i.e. accuracy, and improving use of forecast information leading to better outcomes.

The first category is the easiest to comprehend. Getting better at recognising patterns in ever larger datasets should result in more accurate forecasts and more reliable quantification of uncertainty. However, in recent history, the most significant improvements in energy forecast performance have come from finding new sources of predictability and pairing them with suitable AI/ML algorithms, not from advances in AI/ML per se. I include weather forecasts as a source of predictability here. Improvements in weather forecasting, and the extraction of information from weather forecasts, have both contributed to improvement in energy forecasts. AI weather forecasts may be one of the next new sources of predictability for energy forecasters to exploit. We should continue to search for other new sources of predictability too, and perhaps AI can help with that.

The second category, converting current forecasts into better outcomes, may be the greater opportunity for AI to accelerate decarbonisation but is harder to achieve. Forecast improvement only requires changes to forecasting systems. Changing the way forecasts are used often requires multiple business processes, policies and governance to evolve. Energy market participants and system operators have access to masses of forecast information; this is not the problem. Converting that information into decisions that minimise costs and carbon emissions (or maximise profit) while maintaining reliable supply is the real challenge. We shouldn’t overlook the potential role of AI tooling from chatbots to agents to accelerate deployment and maintenance of state-of-the-art forecasting and decision-support systems. The data and digital skills gap in the energy sector is a major constraint. Increasing the productivity of teams and individuals with the right know-how could have a significant impact. Small tech-focused companies may be well placed to exploit this opportunity, but regulated monopolies and large incumbent utilities are burdened with legacy IT and the necessary security measures that protect critical national infrastructure but hamper innovation.

From forecast improvement to forecast value

The variability and limited predictability of a wind and solar power mean that as these resources expand, short-term forecast uncertainty increases. For system operators and market participants (not just renewable energy producers) who buy/sell their anticipated consumption/production ahead of time, quantifying and managing forecast uncertainty is of increasing importance. Around the world, the energy sector has responded by allowing energy to be traded closer to delivery, in more granular time intervals, and through more diverse market mechanisms. Digitalisation of energy markets has given rise to automatic and algorithmic trading where computers process (forecast) information and execute trades faster than rival human traders, a well-trodden path in financial markets. Control room engineers are also aided by data-driven support systems and varying degrees of automation.

In both cases, managing risk is central and quantifying forecast uncertainty is vital. Uncertainty in energy forecasts is highly structured with implications for users. Imagine, for instance, a weather front crossing the UK that will bring a change in temperature, cloud cover and wind speed: supply, demand and the flow of energy around networks will all be affected. The impact of the front arriving earlier or later than anticipated affects the whole system in a complex but structured manner. Forecasts can describe this structure, but human users struggle to make sense of such vast and complex information. Creating efficient operational plans that include recourse to handle all credible future outcomes has always been the challenge facing energy system operators. In low-carbon energy systems, weather-dependence expands the range and complexity of possible futures, and uncertainty persists closer to real-time. Where automation is not possible, safe or otherwise acceptable, AI has potential to support human operators in myriad ways, including providing advice and insight, even identifying novel risks, and executing decisions that may require many individual actions to be taken.

Finally, we must recognise that the success or failure of these methods in supporting decarbonisation will depend as much if not more on development of the environment in which they operate as it will on advancements in AI itself. Market design, regulation, communication and control systems both constrain and enable AI’s role helping to manage energy systems. For example, demand-side flexibility and storage should alleviate the challenges brought by variable and uncertain production from renewables, but are widely viewed as a forecasting challenge. Mechanisms for matching supply and demand need to be reimagined for the low-carbon future we aspire to create.

Summary

Predictive analytics are already and will continue to improve the efficiency of energy markets and control rooms. AI in all its flavours will play a role in supporting decarbonisation, but realising its benefits will require structural change as part of the wider energy transition as well as adoption by individual actors.