Managing the soaring energy demands of generative AI

Artificial intelligence (AI) is transforming the way we learn, work, and address some of the world's most pressing issues.

It has a role in everything from safeguarding the environment and conserving natural resources to drug discovery and diagnosing health conditions.

Widespread and powerful

Until recently, AI had only a modest impact on global energy consumption. Now, AI applications are becoming more widespread and much more powerful.

Generative AI applications – which can create new data such as stories, conversations, articles, computer code, images, and music in seconds – are especially power-hungry.

For example, the Electric Power Research Institute (EPRI) estimates that a user query on early models of Open AI's ChatGPT application used about 10 times as much electricity as one Google search (this was before Google added generative AI functions to its search results).

Generative AI is increasingly part of many digital services, and while it's not responsible for all data center energy use, its rise is coinciding with an expected doubling of data center power consumption between 2022 and 2026.

By 2030, data centers could consume up to 9.1% of all electricity generation in the U.S., up from 4% in 2024.

Mitigating its impact

In this article, leading researchers and technologists explain what makes generative AI so energy-intensive and highlight the impact it has on power grids and other consumers of electricity.

We also explore the roles academia, industry, policymakers, and end users can play in managing AI's energy demands without stifling innovation.

Many of us have been using "traditional" or "classical" AI for years in applications such as music streaming recommendation engines, voice assistants, and predictive text.

Why does generative AI have a much bigger impact than traditional AI on energy consumption?

Training

"Generative AI is energy-intensive due to the massive computational power required for model training and deployment," says Dr. Soenke Ziesche, co-author of Considerations on the AI Endgame.

"Model training involves optimizing billions of parameters with massive datasets," adds Dr. Lirong Liu, an associate professor in the Centre for Environment and Sustainability at the University of Surrey and a co-investigator at the Energy Demand Research Centre (EDRC).

A constant need to retrain generative AI models is also increasing energy consumption, says Dr. Sukhpal Singh Gill, editor of Applications of AI for Interdisciplinary Research and Editor-in-Chief of the International Journal of Applied Evolutionary Computation (IJAEC):

"Retraining is essential for improving their performance," he says.

Further, Dr. Magdalena Klemun, an assistant professor in the Department of Civil and Systems Engineering at the Johns Hopkins Whiting School of Engineering, highlights that, unlike traditional AI, many generative AI models are designed for a wide range of uses:

"To reach this level of versatility, they are trained on much larger datasets, like a vast and diverse collection of publicly available text for ChatGPT.

"The training, model fine-tuning, and adjusting to a particular application all consume energy," she says.

Usage

"And then, importantly, there is the model's actual use to generate new text or images.

"That also consumes lots of energy, along with keeping enough GPUs [graphics processing units] and memory partially active at any time for the model to respond quickly when prompted – analogous to keeping the car engine running at a stoplight."

"Because these models are used daily by millions, and training occurs during development and fine-tuning, inference [using a model] often consumes more energy over the model's lifetime, especially for widely deployed systems.

"Still, many factors affect this, including model size and algorithm design, how often models are retrained, how users interact with them, and the hardware they run on."

Cooling

Cooling data centers needs vast amounts of electricity too, not to mention millions of liters of water.

"Data centers use substantial energy for cooling and powering servers during deployment," says Dr. Ziesche.

"Data collection itself is less energy-intensive compared to these processes."

"My opinion is that both retraining and cooling are the primary contributors to energy consumption," says Dr. Gill.

Dr. Ziesche highlights that increased greenhouse gas emissions (contributing to climate change) and strain on power grids are major consequences of generative AI's energy use.

Power demand

"In the U.S. in particular, AI has contributed to a shift in how we think about the evolution of power demand," says Dr. Klemun.

"There is more uncertainty now, and technology and politics both play into it.

"U.S. power demand was flat for over a decade, but many forecasts now anticipate an increase, driven partly by AI, as well as new domestic manufacturing and broader trends in electrification, digitalization, EV [electric vehicle] adoption, and Bitcoin mining.

"But keep in mind, forecasts depend on data, and in the AI domain, that data changes rapidly and key gaps remain."

"In regions with high data center concentration, competition for electricity [between data centers and other consumers] will intensify as we electrify our economies to fight climate change and expand low-carbon industrial power use, for example.

"On top of that, we need to factor in potential demand spikes from extreme weather."

Dr. Klemun also highlights how the infrastructure that supports generative AI can affect power grid dynamics:

"AI data centers draw high-density, rapidly varying loads and may transfer to backup power during grid disturbances before re-synchronizing with the grid," she says.

"Together, these behaviors can affect power quality and system stability if not actively managed.

"These problems are all surmountable, but we need to think about them early enough.

"My optimistic take is that the near-term demand from data centers can provide a strategic foundation for a faster economy-wide expansion of low-carbon generation assets and updated grid infrastructure.

"But more pessimistic scenarios, including infrastructure overbuilds and money that could have been better spent, are also important to consider."

Emissions increase

"The use of generative AI can lead to the use of fossil fuels to generate power, increasing carbon emissions," adds Dr. Gill.

Despite the rise in renewable energy generation in recent years, fossil fuels still produce the majority of electricity worldwide. One estimate is that data centers could be powered by 80% renewable energy, with the remaining electricity generated by nuclear power rather than gas or coal.

However, the majority of new nuclear facilities needed to meet data centers' rocketing demand may not be online until the 2030s.

"One important question is where generative AI is really necessary and societally beneficial, particularly during periods and in regions where low-carbon power and transmission capacity are scarce," says Dr. Klemun.

"Besides the global warming potential, the impacts on water should also be noted," adds Dr. Lirong.

What about the use of generative AI in enhancing environmental sustainability?

"The assumed future benefits of AI for sustainability are often presented as if they are guaranteed," says Dr. Jenny Rinkinen, co-author of Conceptualising Demand: A Distinctive Approach to Consumption and Practice and Associate Professor in Social Sciences at LUT University.

"In reality, these outcomes are highly speculative and depend on a wide range of political, economic, and social dynamics.

"We do not yet fully understand the broader sustainability implications of widespread AI adoption, and treating these potential benefits as 'certain' risks overlooking more complex and possibly negative outcomes."

With the rise of generative AI applications and the drive toward artificial general intelligence (AGI), what steps can tech companies, policymakers, and users take to manage or even reduce the demand for electricity – without stifling innovation?

Development and delivery

Citing the IEA's "Energy and AI," report, Dr. Liu highlights three areas where the development and delivery of AI applications could be made more energy efficient:

1. Hardware – e.g., memory, data storage, computer processors, cooling systems
2. Software - e.g., algorithms, code
3. Cross-cutting - e.g., the location of data processing in relation to data storage, co-design of software and hardware, smart energy management

"In terms of hardware innovation, further improvements in the performance-per-watt of current GPUs used for generative AI will occur, but these alone may not be able to address growing energy use," says Dr. Klemun.

"So, more transformative changes will need to be made to chip and system architectures as well."

She says this could include moving:

• Computing closer to end-user devices
• Data processing closer to data storage within computing architectures to reduce the energy loss from data transportation

"In addition, data centers can improve cooling strategies, virtualize servers to boost utilization, and adopt more energy-efficient networking and storage equipment.

"On-site low-carbon power generation can reduce net draws from the public grid and cut emissions."

"Northern European countries are becoming hubs for energy-efficient data centers due to their favorable climate and renewable energy sources," adds Dr. Ziesche.

"The climate is cooler, which reduces cooling costs, and access to renewable energy sources like wind power is available."

"Reducing energy use often involves coordinated improvement in software and hardware," highlights Dr. Klemun.

"For example, to dynamically match workloads to various specialized hardware units and maximize overall efficiency."

Related to software, she highlights the following approaches to boost efficiency:

• "Model optimization, for example, by reducing model size and complexity through the removal of unnecessary components, reduced numerical precision, or development of smaller, efficient models that replicate larger ones – or using tailored rather than full-complexity models for specific applications (such as translation and legal document generation)
• "Training method optimization through early identification and filtering of low-performing model configurations, or preventing overtraining ('early stopping')
• "Limiting the power use of GPUs to strike a balance between training time and energy consumption
• "Better matching computational intensity with hardware performance to avoid using versatile, energy-intensive hardware (such as high-performance GPUs) for models that could be more efficiently run on specialized hardware (such as tensor processing units)

"Much of this is already happening in the R&D space and parts of industry," she says. "But it's not easily accessible to the average consumer interested in greener AI."

However, Dr. Klemun notes that it's difficult to estimate by how much computing efficiency improvements will curb demand:

"We have seen for the last few years that efficiency gains have 'lost' the race, and overall data center power demand has increased despite massive efficiency gains.

"However, that wasn't the case in the first phase of accelerated data center growth, where efficiency gains still offset the increase in computational demand.

"So, a lot of future trajectories are possible from today's perspective, and a key way to address this problem is to improve forecasting methods and make more reliable data [on the energy use of AI models, data centers, and data center components] publicly available."

Interdisciplinary collaborations

So far, we've focused on efficiency gains in individual applications and enterprises.

Dr. Liu highlights how interdisciplinary collaborations involving partners from policy, industry, civil society, and academia can also help the AI industry become more energy-efficient:

"We need to explore an integrated solution that is not only technologically feasible, but also socially acceptable and with appropriate enabling policy supports.

"The application of AI in different end-use sectors such as transport, building, and industry has various but significant potential. We should explore the overall energy use and emissions with the whole system approach."

"AI applications can increase the rate of scientific discovery and the efficiency of our economy, possibly reducing energy use in some [non-AI] areas," adds Dr. Klemun.

"But many applications are new, and we have yet to see how quickly they can be integrated with existing infrastructure and scaled across diverse sectors like buildings and transportation. That will require substantial innovation in deployment processes, in addition to new AI-based products.

"On top of that, however, AI offers consumers an ever-lengthening menu of energy-using services. So we also need to consider the secondary impacts of higher service consumption relative to a non-AI scenario, which in turn adds energy use, known as the 'rebound effect.'

"For example, making services or products cheaper through the use of AI makes them affordable to more consumers, likely raising demand.

"This systems approach can then be boiled down again to create blueprints for cities or companies to make more informed decisions on where to incentivize AI and how to limit its environmental footprint."

Influencing user behavior

Do users of generative AI have a role to play in cutting its energy use?

"Obviously, users may use AI apps more responsibly if they learn about their energy expenses," says Dr. Gill.

But the average user has no idea how much energy they're using when they use generative AI applications.

"The current situation is a bit like running factories without knowing the power ratings of major machines," says Dr. Klemun.

If users are to reduce their use of generative AI or reduce its carbon emissions, she says there needs to be more transparency about how much energy generative AI models and applications use.

"Do all AI tasks need to be addressed right away, or can some tasks be scheduled and delayed to ease power grid stress and reduce emissions? If so, how can we identify these tasks systematically and reward flexible users?

"Users need to be given a choice among options with different environmental consequences. I think that's vital not just for individuals but particularly for decision-makers who are choosing or incentivizing models for entire organizations."

"Educating employees to optimize queries and avoid unnecessary calculations can further cut energy consumption," adds Dr. Gill.

"As AI use grows, it is vital to integrate sustainability-related AI skills into education, including not just technical skills [like model design and prompt design], but also support for decision making," says Dr. Klemun.

"For example, how might students and educators weigh AI benefits like near-term productivity gains against risks like skills erosion."

Challenges for regulators

Whether it's optimizing electricity networks, reducing carbon emissions, forcing AI companies to be more transparent about their energy use, or ensuring the needs of other consumers of energy are met, regulators and policymakers have several challenges to consider.

So what should they focus on first?

Transparency

Dr. Klemun says tech companies should share more information about the performance of their AI models and data centers:

"It's hard to define workable targets without baselines – that's why transparency is key for the development of standard testing conditions, energy ratings for different models, and stronger efficiency requirements for data centers [via mandatory reporting, procurement thresholds, and building codes], moving toward widely adopted minimum performance benchmarks over time.

"We also need to think about water use efficiency and improved water use reporting," she adds.

"Once those baselines are established, we can strategically set targets for reducing energy consumption over time as the technology progresses. We did that successfully for cars, for example, with fuel efficiency standards that got stricter over time.

"Of course, AI models are evolving and spreading much faster, so the challenge is to design standards that can keep up with the pace of innovation – not too limiting, easy to adopt and adapt, and yet protective of the environment and motivating enough for industry to get better and leaner.

Prioritizing energy

Dr. Klemun also poses several questions related to the distribution of electricity:

"What level of residential electricity price is tolerable in regions with high data center concentration, considering that everyone pays for grid upgrades while only a few companies directly benefit from new AI data centers?

"Do we just let markets decide who gets to develop and use AI-based technologies, risking that AI innovation becomes a playground exclusively for a few, energy-wealthy companies?

"How do we define priorities?

"And how much new fossil-fired capacity should we build to power data centers, when there is little uncertainty about the threat of climate change, but tremendous uncertainty about the actual future power demand from data centers?"

Answering questions like these – and taking the appropriate action to rein in generative AI's soaring energy use – won't be easy.