NREL's Supercomputing Drives Innovation

Supercomputing Drives Innovation

Researchers are finding new ways of tackling our nation's energy challenges.

Image of a computer generated 3-dimensional brown and gray wall with a scattering of green and brown blobs floating in the foreground. Enlarge image

Large-eddy simulation (LES) of a neutrally stable atmospheric boundary layer with the wind turbine blade aerodynamics modeled using actuator lines. A vertical cross-section shows streamwise velocity magnitude (red represents high velocities, whereas blue represents low velocities); notice the development of the low velocity wake behind the turbine. The green isosurface is the q-criterion (second invariant of the velocity gradient), showing the vortical structures trailing from the blade.
Credit: Matthew Churchfield and Kenny Gruchalla/NREL

Using NREL's computing power, researchers are identifying optimal components, processes, and systems that are leading to new ways of tackling our nation's energy challenges—avenues that simply are not always viable through trial-and-error or traditional experimentation alone.

NREL's brawny computers help researchers do the heavy lifting across a range of fields, from biofuels to photovoltaics (PV). These computers are also being used to advance energy efficiency in buildings. RedMesa, currently NREL's most powerful high performance computing (HPC) system, has a peak capability of about 180 teraFLOPS, performing 180 trillion "FLOPS," or floating point operations per second (see sidebar). This computer, which is co-located with the larger RedSky HPC system at the U.S. Department of Energy's Sandia National Laboratories, has proved so popular with researchers that NREL scientists have kept the system running at capacity—day and night—for the past year.

For example, during a recent nine-month period, scientists in NREL and its collaborators used 7.5 million computational hours on RedMesa to evaluate more than 30,000 candidates in their pursuit of promising organic photovoltaic (OPV) cells. If these analyses had been done experimentally in a laboratory, they would have taken around 7,000 person-years to create and study that many different materials. So far, this sort of computational analysis has identified more than 300 highly promising materials, 10 of which are in the synthesis pipeline, ready to undergo further evaluation.

As demonstrated by this time savings, NREL is relying on HPC systems to drive technology innovation that is vital to the lab's research and development. The HPC system capabilities now allow scientists and engineers to thoroughly explore a dizzying array of possible designs, configurations, and alternatives in virtual experiments. This potent tool is also being used to explore new horizons in biomass, solar, and wind energy technologies.

High-Performance Computing Accelerates Biofuels Research

Two men stand in front of a projected image on a large screen. Enlarge image

NREL's Steve Hammond, Director of Computational Science Center and Kenny Gruchalla, Senior Scientist, discuss a 3D model of wind plant aerodynamics, showing low velocity wakes and impact on downstream turbines.
Credit: Dennis Schroeder

To ensure that cellulosic ethanol and advanced fuels are produced at costs competitive with fossil fuels, researchers rely on NREL's HPC capability to analyze the complex chemistries and dynamics involved in both thermochemical and biochemical pathways for more efficient biofuels production.

For instance, NREL scientists are using HPC to understand how naturally occurring enzymes from fungi and bacteria break down plants in nature, with the overall aim to then use this knowledge to design enhanced enzymes that will reduce production costs for biofuels. Much about the behavior of these enzymes is unknown, and the plant walls themselves are quite complicated and poorly understood. Over the past two decades, however, research groups have gained crucial understanding about the structures of important enzymes found in nature. Leveraging this knowledge, in recent years, NREL scientists have used supercomputers to simulate 3D models of the primary enzymes and cellulosic materials involved in biofuel production. This work has led to a surge of computational experiments that have identified previously unknown functions in these enzymes, and has suggested further experimental work to tackle this challenging problem.

By employing RedMesa and other supercomputers, scientists can model the complex microfibril structure of cellulose from plants and create visualizations that help explain how enzymes break down feedstock such as corn stover and poplar trees for renewable fuels. For the enzymes that break down plant cellulose, NREL researchers have probed single enzymes from the industrially important fungus, Trichoderma reesei, and enzymes from the bacterium Clostridium thermocellum. These two organisms are very effective biomass degraders in nature, but have significantly different means of breaking cellulose down into basic sugars. Understanding the suite of enzymes from both of these organisms can provide key insights for biologists to produce enhanced enzyme systems for industrial biofuels processes. Overall, this computational approach has helped researchers better understand basic science of both the plant cell wall structure and chemistry, as well as helped them design improved processes involving enzymes to break down the cell wall. See the Continuum article on biomass in this edition.

"The continued rapid advances in supercomputing capability are now enabling scientists to conduct detailed simulations of large wind farms or explore the fundamental properties of entire classes of materials—problems that were intractable just a few years ago."
— Steve Hammond, director of NREL's Computational Sciences Center.

These sorts of inquiries support both the Obama administration's goal of a timely and efficient transformation of the nation's energy system and the U.S. Department of Energy's target of securing American leadership in clean energy technologies. The administration has proposed an 18% cut in daily petroleum consumption in less than a decade, which translates to a reduction of 3.5 million barrels per day, from a 19-million-barrel baseline, by 2020. To replace some of that fossil fuel volume, DOE has set a goal of having 60 million gallons of biofuels in the nation's supply by 2030.

Fine-Tuning Solar Materials with HPC

Biofuels research is not the only program making use of HPC capability. Photovoltaic researchers also employ HPC to help design novel materials that have the exact properties required to dramatically improve solar cell performance. The researchers are also using HPC to identify unique materials that have radically improved thermal energy storage and stability characteristics in order to create low-cost, highly efficient concentrated solar power systems.

Image of a 3-dimensional scatter plot, with gradations of first red, then orange and then yellow multi-sized circles. Enlarge image

A three-dimensional scatter plot showing computed electronic properties of tens of thousands of potential OPV materials that were auto-generated according to combinatorial rules. The points are sized by the overlap of their absorption spectrum with the solar spectrum, colored by "fitness" which is proportional to the predicted OPV device efficiency (yellow is more efficient, red less efficient).
Credit: Ross Larsen and Kenny Gruchalla/NREL

As mentioned previously, OPV cells show great promise as lightweight and flexible devices to convert sunlight to electricity. Currently, the best OPV devices have power conversion efficiencies approaching 10%, and are more than twice as efficient as they were just three years ago. These dramatic gains have been possible because researchers have been able to chemically tune material used in the light-absorbing active layer so that it can better absorb sunlight and generate more power generation for every photon it absorbs. This kind of continued improvement in OPV performance depends on the discovery of novel active layer materials that can convert photons into power more efficiently.

The main constraint is that new materials may require several months to synthesize, which means that researchers have created and tested only a few hundred out of tens of thousands of plausible candidates. To circumvent this bottleneck, NREL computational scientists, along with organic chemists and material scientists, developed a high-throughput computational screening method to test active-layer materials.

Typically, researchers screen these sorts of candidate materials in a four-step process that starts when new materials are generated combinatorially, in which computer scientists analyze large sets of data. Researchers leverage a large library of chemical building blocks and then perform high-level electronic structure calculations on them. The results are processed and automatically imported into a searchable database, creating an OPV material database similar to that of the human genome. The materials with the desirable electronic properties are then shared with chemists to synthesize and characterize.

Supercomputers Provide a Picture for Improved Wind Farms

NREL's New Green Data Center will be a Green Giant

Even though its new HPC system is still under development, NREL is preparing for its expected petaFLOPS level. To understand how powerful this sort of system is, imagine if each of the 7 billion people on Earth had hand calculators and worked together for 24 hours a day, 365 days a year on a calculation. Under that scenario, it would take more than 16 years to do what a petascale supercomputer can do in one day. These HPC systems, the world's most powerful, can perform 1015 operations per second.

As it draws on supercomputer capabilities, NREL remains true to the laboratory's mission of walking the talk in support of sustainability. With the expected completion of the new ESIF facility, the immense computing tool being built to function without using the immense quantities of energy typically associated with cooling and facility infrastructure of a typical HPC system. The new ESIF's ultra-energy-efficient HPC data center is being designed to be the world's most energy efficient data center.

Because of its efficiency, annual operating costs for data center power and cooling will be about half of the operating costs of a typical data center. Additionally, waste heat from the high performance computing equipment will be captured and used as the primary heating source for office and laboratory space, further reducing energy consumption and operational costs for the ESIF.

NREL scientists and engineers develop innovative software tools to simulate the behavior of both individual wind turbines and multi-turbine configurations in complex environments, such as high winds, extreme turbulence, and even hurricanes. This kind of advanced software models the effects of turbulent inflow, including unsteady aerodynamic forces, and the inflow's effect on structural dynamics and drive-train response. Even before offshore wind turbines exist in US waters, these sorts of models can test hydrodynamic loading, helping to ensure that the best possible designs are deployed.

To improve these models, their results must be compared with actual observations. NREL has a long history in collecting data that describes the efficiency and reliability of turbines. In early 2011, an interagency group comprised of NREL, the University of Colorado at Boulder, the National Oceanic and Atmospheric Administration, and DOE's Lawrence Livermore National Laboratory, deployed high-resolution atmospheric instrumentation at NREL's National Wind Technology Center to study the atmospheric flow fields surrounding large wind turbines. The team collected meteorological data in order to validate turbine wake models in a range of atmospheric stability conditions. Understanding and predicting wind plant aerodynamics and the impact of turbulence from upwind turbines on downwind turbines is vital when optimizing wind plant turbine layouts as well as designing turbines with longer lifespans. The interagency study's data will provide valuable insights into the operation and optimization of these large wind turbines.

This type of knowledge could reduce the cost of electricity generated from wind energy, as these insights enhance wind farm efficiency and help extend the life of wind turbines. This research could also help offshore studies, and support the path to DOE's benchmark to cut the levelized cost of wind energy.

NREL's HPC Future is Taking Shape

NREL has already leveraged its current HPC capabilities to help meet critical goals in biofuels, wind energy, and PV for DOE and its Office of Energy Efficiency and Renewable Energy. However, just as the capabilities of computers continue to expand to meet demands, so does NREL's HPC systems capacity, enabling integration across a wide range of applications.

  • Vehicles: NREL's HPC capabilities will be able to model system design and material thermal properties for power electronics, advanced batteries, and renewable fuels combustion properties;
  • Buildings: NREL will be able to determine the optimal cost and energy performance solution sets for energy efficient building retrofits, and for new ultra-efficient and net-zero-energy buildings;
  • Hydrogen: the HPC will simulate the thermal and structural properties of fuel cells for marked improvements in fuel cell performance, durability, and thermal management;
  • SmartGrid: the capacity to model and simulate grids will help the power industry better understand the evolution to a cleaner, reliable, efficient, and secure, grid that includes high penetration renewables.

NREL researchers' ongoing use of the RedMesa HPC system has demonstrated how essential this sort of computer analysis is to their work. When the lab's Energy System Integration Facility (ESIF) is completed in late 2012, NREL plans to acquire a new HPC system with more than double RedMesa's computational capability.

As NREL continues its leadership in advancing renewable energy and energy efficiency technologies—working to help attain administration and DOE energy goals—it continues to find ways to leverage growing HPC capacity to support advances in areas including energy efficient buildings, photovoltaics, wind energy, biofuels, and other renewable energy and energy efficiency technologies. While the petascale era dawns at NREL with the planned petaFLOPS supercomputer, NREL computational scientists and researchers remain committed to staying in the forefront of renewable energy and energy efficiency research while continuing to embrace the laboratory's mission of sustainability.

—Ernie Tucker

Related Links

Advanced Vehicles & Fuels

Buildings Research

Electricity Integration

Hydrogen & Fuel Cells Research

Top of page