Making a Computer Model of the Most Complex System Ever Built
The eastern part of the North American power grid will likely host an increasing percentage of renewable energy in years to come, and NREL wants to see how that will work.
In the coming decade, the eastern part of the North American power grid, called the Eastern Interconnection, will likely host an increasing percentage of renewable power. Understanding how it could handle that transition is the focus of an intensive, multi-year modeling project at NREL.
The project, called the Eastern Renewable Generation Integration Study (ERGIS), looks ahead to the year 2026 and examines how the Eastern Interconnection might function under four future scenarios. The Eastern Interconnection runs from the east coasts of the United States and Canada all the way to the western borders of the Dakotas, Nebraska, Kansas, and Oklahoma, while skirting around most of Texas. It serves 70% of the U.S. load and 240 million people, and consists of 50,000 transmission lines spanning 459,000 miles. In short, it's huge, and accurately modeling the entire system at five-minute intervals for an entire year has never even been attempted—until now.
"ERGIS is the highest-resolution simulation of the largest power system in the world," said Aaron Bloom, senior project manager at NREL. "It really is the most complex integration study that we've ever done."
ERGIS is also the first high-resolution model of the entire Eastern Interconnection, including Canada.
"Modeling Canada was critically important to this study because of the massive amounts of electricity trade that happen between those two regions," said Bloom. "Indeed, the amount of trade that we can expect to see in the future will be substantial."
To accurately model system operations for the Eastern Interconnection, NREL worked with a Technical Review Committee (TRC) composed of more than 30 industry experts. Under the guidance of the TRC, NREL developed a complex representation of the power plants, or generators, that power the system; the energy-consuming towns and cities, or "loads"; and the complex network of transmission lines that connect the generators to the loads. NREL's resulting ERGIS database closely mirrors existing Eastern Interconnection system operations.
Of Supercomputers and Modeling Tricks
Modeling a huge system like the Eastern Interconnection gets unwieldy quickly. First, a lot of data needs to be fed into the model for it to reach meaningful conclusions; much of this data did not exist before the ERGIS project was launched. For instance, the ERGIS Solar Dataset used 2006 satellite data to estimate actual solar power production at five-minute intervals, and employed a weather model to derive solar power forecasts for four hours ahead and one day ahead—the same type of forecast a grid operator would use to plan ahead. Similar datasets have also been generated for wind power, hydropower generation, and loads in the Eastern Interconnection, all as a function of time. All told, that's about 1,000 gigabytes of data.
Fortunately, NREL has the world's most energy-efficient supercomputer to shoulder the computational burdens. Called Peregrine, the system is capable of 1.3 trillion calculations per second—in computer lingo, 1.3 petaFLOPS. NREL also developed new capabilities to deploy the leading industry tool for modeling power grids—PLEXOS® Integrated Energy Model software—on Peregrine's parallel computing environment.
PLEXOS is able to model how grid operators would make their decisions throughout each day. This includes making "unit commitments"—essentially, plans to turn generators on or off—every hour, one day in advance, while also "dispatching" power plants—telling them how much power to produce—at five-minute intervals. As the modeling software steps forward in time, those decisions affect how the grid operates under changing weather conditions.
"The mathematical challenges associated with a model this complex required the team to build new computational tools to both simulate and analyze the system," said David Palchak, NREL energy systems engineer. "With these capabilities, we can dig deeper into more complex problems than ever before."
Finding a Faster Solution
For ERGIS, NREL modeled all 7,500 generators in the Eastern Interconnection and all the places where transmission lines interconnect, called buses. In the Eastern Interconnection, there are more than 60,000 buses, making the simulation quite complex. In fact, as NREL initially approached the problem, a simulation of the entire Eastern Interconnection at five-minute intervals for one year would have required more than 400 days of computing time. Clearly, a faster solution was needed.
A key issue was that each day's solution depended on the solution for the previous day, so there was no obvious way to solve for multiple time intervals in parallel. But NREL researchers realized they could break the simulation into weekly time periods, with two to three days of overlap between each time period, to dramatically reduce the solve time while still capturing most if not all of the generator starts and stops and other metrics needed to form a coherent view of the grid's performance. This approach allowed NREL to simulate an entire year's operation in only 17 days of computing time—more than an order of magnitude faster than the conventional approach.
Peregrine is capable of running up to 1,442 programs in parallel—in computer terminology, it has 1,442 "nodes." These nodes allowed NREL to break up the huge simulations required for ERGIS into smaller chunks that could all be processed at the same time.
"Peregrine's capabilities allow us to run one week of ERGIS simulations on individual nodes of Peregrine; we can run 52 nodes at a time and therefore run the entire year in parallel," said Aaron Townsend, NREL's technical lead on the project. "We did, at times, simulate multiple scenarios at the same time, so we used up to 500 nodes simultaneously, or approximately 35% of the machine."
To test the model, NREL researchers ran the simulation for 2010 and compared it to the actual performance of the grid during that period. Although the outcomes of the simulation and the actual performance didn't completely line up, the differences were easily explained, and overall, the TRC found the model to be an accurate depiction of Eastern Interconnection operations.
What ERGIS Will Tell Us
For ERGIS, NREL is examining four scenarios for 2026:
- A low-variable-generation scenario, which holds the current amount of wind and solar power constant, removes generators slated for retirement, and adds new generators to meet the load and maintain reliability, as needed.
- A "regional transmission 10% variable generation" scenario, which builds out wind and solar generation to meet 10% of the interconnection's load with a regional transmission expansion.
- A "regional transmission 30% variable generation" scenario, which requires wind and solar generation within each of the ERGIS regions to provide 30% of the region's power. This scenario shares the same transmission expansion as the second scenario.
- An "inter-regional 30% variable generation" scenario, which chooses the best resources in the Eastern Interconnection to meet the 30% target and includes a substantial build-out of new inter-regional transmission facilities.
"What we're looking at is the operational impact of high penetrations of wind and solar on system operations," said Bloom. "How does it change the operations of other generators, like the thermal plants? Do they ramp more, do they start up and shut down more, are they sitting online at idle more often, or are they running full bore all the time? So we want to see what happens to the other power plant assets when you add all these renewables, and then determine if it can be done."
To answer these questions, Bloom and his colleagues are now completing their analyses, and results are expected this summer.
Standing on the Shoulders of Giants
ERGIS will be NREL's most sophisticated modeling of a power grid, but it is far from the first. ERGIS is a follow-on study to the Eastern Wind Integration and Transmission Study, theWestern Wind and Solar Integration Study Phase 1 and Phase 2, and other analyses. These studies show that the variability and uncertainty of wind and solar power at high-penetration levels require new ways of planning and operating electric power systems. However, new questions are being posed about the impacts of future policies, distributed generation, and the siting and timing of new non-renewable generation and transmission. ERGIS will address these follow-up questions and place additional emphasis on the question of how to plan and operate the Eastern Interconnection in the face of generation and transmission uncertainty.
"These studies continue to show the same thing, which is that the integration of large amounts of renewables into our power grid is technically feasible, and we can integrate a lot of variable generation and continue to operate the system in a way that maintains reliability and meets the demands of the system," said David Mooney, director for NREL's Strategic Energy Analysis Center. "That's a really important, fundamental consistency throughout these studies.
"Not only do the reports show that the grids can handle high penetrations of renewables, but perhaps more important is that they identify the pathways to make it happen and the challenges associated with each of those pathways, so we can begin to address them. Some changes will have to be made, and these studies are showing that if we make the right changes, we can integrate large amounts reliably and reasonably cost-effectively."
As everyone awaits the ERGIS results, high penetrations of renewable energy are already a fact in Hawaii. The Hawaii Solar Integration Study is a detailed technical examination of high penetrations of wind and solar energy on the operations of the electric grids of Maui and Oahu. The study found that adding large amounts of new solar power to the electric grids on these islands—enough to achieve 20% renewable energy—will create operational challenges that could affect grid reliability, but also recommended a variety of mitigation strategies that could address those challenges while optimizing the use of renewable energy. The ERGIS report is likely to include similar findings.
Meanwhile, NREL's Renewable Electricity Futures Study found that renewable electricity generation from technologies that are commercially available today, in combination with a more flexible electric system, is more than adequate to supply 80% of total U.S. electricity generation in 2050 while meeting electricity demand on an hourly basis in every region of the country. That finding, and the coming ERGIS report, should give the nation hope as it moves toward a carbon-neutral energy future.
—Written by Kevin Eber