SciDAC Review - Advanced Simulation for FAST REACTOR Analysis

NUCLEAR ENERGY

Advanced Simulation for FAST REACTOR Analysis

The past few years have seen renewed interest in the research and development of fast reactors. These reactors offer two major benefits: they enable more efficient use of valuable natural resources, and they reduce the challenges associated with fuel cycle waste management. Key to realizing these benefits is high-performance computing. Using advanced simulation techniques on petascale computers, scientists and engineers are addressing nuclear engineering problems that previously could be addressed only by experiment, or not at all. These advanced techniques promise to improve the efficiency, safety, and cost-effectiveness of future fast reactor facilities.

A Fundamental Shift
The challenge of designing a nuclear reactor is to make it as economical as possible while ensuring its safety. This is not an easy task. The principle of a nuclear reactor is relatively simple—fission creates heat within the nuclear fuel, which conducts to the fuel cladding surface and is subsequently transported by a coolant through heat exchangers and ultimately to a steam conversion plant. But to realize this process efficiently and safely, one must choose among a vast range of competing design parameters. What are the best fuels, structure, and coolant materials and their appropriate ratios? What are the optimum sizes and arrangement of the various components? How does the reactor respond to component failures? And how does one balance these choices given competing goals of performance, lifetime, safety, and capital costs?

Ideally, one would like to base these choices on theory rather than experimental trial and error. Indeed, at the dawn of the nuclear area, the theory of reactor design was an active field of research, driving progress in areas such as transport theory, turbulence, structural mechanics, perturbation theory, and numerical analysis. Advances in these fields were then applied directly to help identify optimal choices for key reactor design parameters. Except for the simplest analyses, model equations typically required numerical solutions of large systems of equations. Thus, in the early 1980s, nuclear engineering was at the forefront of computer applications, pushing the envelope for ever-greater computing resources and related advances in computational science. In one sense, this era of reactor modeling stood as a major early success story in the computational sciences—the computer models reduced the burden of experiment and contributed greatly to the design of the original fleet of reactors.

With the advent of petascale architectures, computing speed has increased a billion-fold since the conception of the current set of reactor analysis tools.

In another sense, however, modeling was severely constrained. Even state-of-the-art machines such as the Cray-1 were unable to explicitly model the key physical phenomena within a reactor. Thus, the early nuclear codes were simplified—using low-dimensional representations, lumped parameter models, and empirical correlations with tunable parameters established largely by experiment. The consequences were twofold: new calculations relied heavily on expensive and often complicated experiments, and the inaccuracy of the predictions resulted in significant design margins that negatively impacted plant economics. More important, exploration of novel design concepts was greatly constrained by fundamental limitations in the predictability of the models.

Figure 1. Idealized layout of the fuel pin lattice for an Advanced Burner Test Reactor fuel assembly. The duct wall providing the structural support to the fuel pin bundle is removed for clarity (right).

With the advent of petascale architectures, computing speed has increased a billion-fold since the conception of the current set of reactor analysis tools. Additionally, comparable advances in related fields of computation, such as scalable solvers, advanced meshing, visualization, and data management, have enabled dramatic improvements in computer applications. The key question now is: Can we revisit our legacy suite of reactor design tools and apply these advances to enable a fundamental shift in the reactor design and licensing process? The timing of this question is critical. As a new nuclear era begins, the adoption of novel strategies and approaches will be a major ingredient for success.

As a new nuclear era begins, the adoption of novel strategies and approaches will be a major ingredient for success.

Benefits of Advanced Simulation
A nuclear plant contains a large number of components, including the fuel assemblies (figure 1), control assemblies, reflectors and shields, reactor vessel, heat exchangers, pumps, steam converters, and containment. Accurate simulation of a reactor requires adequate models for all of these components. However, the same degree of fidelity is not required in the representation of each part. In particular, the reactor core region—containing nuclear fuel and control rods—has the most sensitive and complicated physics and dictates many aspects of the overall plant response. It is thus the starting point for the overall plant design process. Typically, one aims for very high fidelity representations of core physics, coupled to more holistic models for the balance of the plant.

In a nuclear reactor, fission heat should be produced in a controllable manner. The heat should be efficiently removed from the core such that temperatures and temperature gradients are maintained beneath prescribed limits for the chosen core materials (to prevent melting, fatigue, and so on). This is an over-simplification, and there are other considerations to optimize as well, such as meeting the long-term objectives for fuel burn-up and demonstrating safe response across a huge range of transients that can occur during reactor operation. Each of these is a rich area of physics and presents fascinating challenges for simulation, the successful solution of which implies many potential benefits for reactor design.

A benefit of advanced simulation is the ability to evaluate new designs with reduced dependence on experiment.

One important benefit is the reduction of uncertainties in predicted quantities. Uncertainties arise from myriad sources, ranging from fabrication variability and imperfect knowledge of input data to simplifications in models and solution algorithms. These uncertainties typically are combined and augmented by conservative design margins. The result is likely to be an over-designed reactor that carries economic penalties. In fact, an increase of even 1% power output per day translates into millions of dollars a year for a single reactor.

A second benefit of advanced simulation is the ability to evaluate new designs with reduced dependence on experiment. Typically, reactor designers are forced to stay very close to existing designs, unable to make predictions about the potential effects of new materials or new geometric configurations. Even when this is not the case, an entirely new set of experiments is required as a foundation for calibrating the new factors. An important question is whether representations of the key physics at significantly higher fidelity will allow scientists to bypass or minimize this latter step and enable a much faster pace of innovation.

A multi-institution, multidisciplinary team of scientists is currently designing new tools and carrying out high-fidelity simulations aimed at addressing this question. The project, known as SHARP (Simulation-based High Accuracy Advanced Reactor Prototyping), is led by Argonne National Laboratory and supported largely by the U.S. Department of Energy's Global Nuclear Energy Partnership (GNEP). The initial focus is on modeling of the liquid-metal-cooled fast reactor (sidebar "Basic Physics of Fast Reactors"), known as the Advanced Recycle Reactor (ARR; figure 2), although many aspects of the tools and techniques aim to be applicable to a broad class of reactors. Much of this modeling has been done on the Cray XT and IBM Blue Gene leadership-class computers made available through allocations from the U.S. Department of Energy Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program.

Without the power of today's leadership-class computers, much of early reactor theory necessarily involved a number of simplifications and reductions to the transport equation designed to render it manageable on existing resources.

Modeling Heat Generation
In a reactor core, approximately 95% of the heat produced by fission is deposited directly in the nuclear fuel. This heat then conducts outward through the fuel and cladding to the cladding surface, where it is swept away by the coolant. The overall temperature distribution that results from this process is the key quantity of interest for the reactor designer, for it determines both the thermal efficiency and feasibility of the design.

Figure 2. Depiction of the proposed GNEP fuel cycle, showing the use of the ARR in consuming light water reactor waste and playing a major role in reducing the burden of long-term storage.

The heat source can be predicted by solving the transport (Boltzmann) equation. Unfortunately, numerical solutions to the Boltzmann equation are notorious for their computational intensiveness. The situation is further complicated in a reactor core. The nuclear data—the fission, absorption, and scattering probabilities, or cross sections—vary by several orders of magnitude and exhibit strong local fluctuations over the relevant neutron energies. Also, nuclear reactors are enormously complex devices, made up of massive arrays of fuel rods, coolant channels, and control rods, as well as reflectors and shielding that is penetrated by ducting and other geometric irregularities. Without the power of today's leadership-class computers, much of early reactor theory necessarily involved a number of simplifications and reductions to the transport equation designed to render it manageable on existing resources.

The task is daunting, and petascale computing capabilities clearly become essential here.

The fuel assembly for an Advanced Burner Test Reactor (ABTR) illustrates the complexity of the reactor core (figure 1, p13). The assembly comprises four major parts: an array of fuel pins, a duct wall that provides structural support for the pin bundle and isolates the coolant flow path for each assembly, a lower structural support and flow inlet mechanism, and an upper coolant outlet and assembly handling structural component (not shown). Hundreds of such assemblies are usually loaded into the reactor core. The fuel pin spacers define the average mesh size, which for the ARR is a helical wire with diameter of 0.11 cm. Using this as the element scale, one can calculate the total number of elements required in a finite element mesh to be on the order of 10-100 billion. When combined with approximately 100 angular degrees of freedom and approximately 100 energy groups, about 10¹⁵degrees of freedom are needed to solve the transport equation (sidebar "The Fast Reactor Core"). The task is daunting, and petascale computing capabilities clearly become essential here.

Figure 4. Sample UNIC simulation. MeTiS decomposition for 512 processors (left), power distribution inside the fuel portion of an ABTR core (center), and power distribution along the axial midplane (right).

A New Approach
To address this situation, computer scientists and nuclear engineers in the SHARP project have teamed to develop a new state-of-the-art unstructured neutronics solver called UNIC. UNIC is intended to scale seamlessly from desktop to petaflop (and beyond), allowing reactor analysts to choose the level of desired fidelity, from high-turnover early scoping to detailed late-stage design studies. The goal is, through highly accurate simulation, to enable significantly lower uncertainty margins in the analysis of newly proposed reactor designs.

The goal is, through highly accurate simulation, to enable significantly lower uncertainty margins in the analysis of newly proposed reactor designs.

To this end, early work on UNIC has involved design and implementation of new methods on petascale machines and their verification on benchmark problems. In order to handle the full range of important reactor problems as accurately as possible, UNIC combines several strategies. For example, to solve the Boltzmann equation, UNIC reduces the equation to an algebraic form by using either spherical harmonics or a collocation method such as discrete ordinates. Spherical harmonics expansions fit well with the diffusive nature of problems often encountered in reactor physics, especially when homogenization approaches are used; they are also hierarchical, making them ideal for multiresolution and adaptive mesh techniques. Discrete ordinates methods have the advantage that they are comparatively cheap in memory and can more accurately treat the detailed flux distributions for heterogeneous geometries.

To solve the large-scale linear equations, UNIC uses the Portable, Extensible Toolkit for Scientific computing (PETSc) developed at Argonne. In particular, a preconditioned conjugate gradient method is used because it is proven to scale to thousands of processors. The overall performance is determined largely by the choice of preconditioner, which must be guided by the physics of the problem. UNIC uses domain decomposition-based preconditioners (for example, additive Schwarz methods) for scalability. The mesh is partitioned by using the MeTiS package, which attempts to minimize the communication while balancing the computational work load on each processor (figure 4).

To date, UNIC has been run on up to 4,096 processors of Jaguar—the Cray XT4 at Oak Ridge National Laboratory—for a 120-degree periodic sector of the ABTR core.

To date, UNIC has been run on up to 4,096 processors of Jaguar—the Cray XT4 at Oak Ridge National Laboratory—for a 120-degree periodic sector of the ABTR core (figures 5 and 6, p18). The mesh contains 587,458 elements and 33 energy groups for an angular resolution of two. Much higher angular resolutions solutions are currently being prepared, leading to about 100 degrees of freedom for each spatial degree of freedom, combined with more refined meshes (around 10-50 million elements) and a larger number of energy groups (230 for the next set of experiments).

Figure 5. Performance bottlenecks observed for the strong scaling of UNIC on up to 4,096 Cray XT4 processors. The global number of space-angle degrees of freedom for the selected angular resolution (P5 case) is about 12 million per energy group. In practice, higher angular resolutions and more refined spatial meshes are used. Both of these factors will improve scalability. But the case shown in this figure was selected to observe these bottlenecks earlier.

Figure 6. Parallel performance (strong scaling) of UNIC from 512 to 4,096 Cray XT4 processors for 120-degree periodic sector of the ABTR core with 33 energy groups. The mesh contains 587,458 quadratic tetrahedral elements and 793,668 vertices, which corresponds to about 12 million space-angle degrees of freedom per energy group for the selected angular resolution (P5 case).

Scaling results from these simulations indicate approximately 65% parallel efficiency in going from 512 to 4,096 processors. According to the researchers, this is reasonably good performance because these are strong scaling tests for a small problem size per processor (which decreases by a factor of eight in going from 512 to 4,096 processors). Eventually, however, similar performance bottlenecks are expected at a higher processor count for larger problems than observed for this small problem. To deal with these, the researchers are exploring a variety of optimization techniques suitable for petascale computers.

Computational modeling of the coolant flow on leadership-class computers is necessary to understand these limits and to predict and maximize the power output.

Modeling Coolant Flow
In addition to high-performance simulation of the heat source, SHARP team members are studying the coolant flow in liquid-metal-cooled reactors. Fission heat generated in the fuel rods is transferred out of the main vessel by a coolant such as water, helium, or liquid metal. Power output can be increased by increasing the temperature or, at fixed temperature, by increasing the coolant flow rate. Generally, both paths are pursued to increase power output for a given capital cost in order to make the reactors economically viable. The output is ultimately limited by the maximum allowable temperature within the core and by the pumping costs, both of which are governed by the coolant flow. Computational modeling of the coolant flow on leadership-class computers is necessary to understand these limits and to predict and maximize the power output.

To date, the researchers have developed a reactor-specific version of Argonne's state-of-the-art computational fluid dynamics code Nek5000. Applying computational fluid dynamics (CFD) to reactor core design is a major departure from traditional techniques. Thermal-hydraulics analysis has typically been based on subchannel models, which use a few hundred degrees of freedom per channel to represent mass, momentum, and energy balances at axial positions along the channel. However, subchannel models rely on experimentally determined coefficients to account for interchannel mixing. CFD offers the opportunity for detailed analyses that not only can lead to improvements in the sub-channel model coefficients but can also provide a means of validating the applicability of sub-channel models to specific geometries a posteriori.

The researchers hope that the simulations, properly validated, will extend design data beyond the existing experimental data.

The Nek5000 code is ideally suited for petascale science. The spectral element method on which the code is based yields rapid numerical convergence, which implies that simulations of small-scale features transported over long times and distances incur minimal numerical dissipation and dispersion. In effect, accuracy per gridpoint is maximized. The code also features spectral element multigrid and has a parallel coarse-grid solver that has demonstrated scalability to more than 10,000 processors.

Much of the initial work of the SHARP team has focused on large eddy simulation of wire-wrapped rod bundles ranging from a single-pin bundle to a seven-pin bundle. In the seven-pin case (figure 7), there are four principal subchannel interfaces separating interior channels (A-A) interior and edge channels (B-B); corner and edge channels (C-C); and edge and corner channels (D-D). The symmetry-breaking induced by the spiraling wire wrap implies that C-C and D-D are distinct. In general, the number of distinct interfaces will scale as a multiple of the number of pins because the position of the subchannel with respect to the subassembly wall is significant.

Figure 7. Channel cross-sections used to report results.

The seven-pin mesh comprises a total of 132,192 elements. The simulations were supported through a 2007 INCITE award and were carried out on Argonne's Blue Gene/L. The time for a single flow-through calculation of this mesh on the Blue Gene/L is about 240,000 CPU-hours. "While this number appears large," says Dr. Andrew Siegel, a computational scientist at Argonne National Laboratory and participant in the SHARP project, "the wall-clock times on the forthcoming Argonne BG/P will be approximately two hours. Such times should be realizable given Nek5000's established scalability."

Simulations with the Nek5000 code, when compared with experimental data, have proved quite successful in describing cross-section distributions of velocity and pressure (figures 8, 9, and 10). The results show axial velocity peaks in the lower and lower-right edge subchannels. These correlate with fluid being drawn into these channels after the wire passes through them as one moves in the positive Z (axial) direction. This process repeats itself with six-fold symmetry as one progresses in Z. Strong vortices are particularly in the interior and edge subchannels. Time-averaged results indicate that the vortices are persistent. These are not stagnant recirculation zones, however; there is significant pressure variation, which has potential implications for structural response of the subassembly.

The SHARP project is leveraging leadership-class computing to provide insight into design improvements, leading to increased safety and economy of advanced reactors.

Figure 8. Time-averaged steady-state axial velocity component.

The next step, according to the team members, is to conduct petascale simulations at more realistic pin counts. This work will be carried out on the Blue Gene/P at Argonne National Laboratory thanks to an allocation under the INCITE 2008 program. The researchers hope that the simulations, properly validated, will extend design data beyond the existing experimental data.

Multiple Benefits
The SHARP project is leveraging leadership-class computing to provide insight into design improvements, leading to increased safety and economy of advanced reactors. This work is part of a broader initiative to incorporate fundamental physics simulations as a key ingredient in the design and evaluation of next-generation nuclear facilities, thus reducing dependence on costly experiments and facilitating the development of verifiably safe and optimal designs.

Figure 9. Seven-pin large eddy simulation instantaneous distributions of axial velocity (left), pressure (center), and total pressure (right).

Figure 10. Instantaneous velocity distributions for seven-pin large eddy simulations: in a span-wise midplane (left) and close-up of an axial cross section (right).

The work described here only scratches the surface of what is possible. For fast reactors, structural mechanics simulations currently being performed within the SHARP project are expected to shed light on some of the key issues in fast reactor safety, in particular the negative reactivity feedback due to core expansion. More generally, multiscale modeling efforts in nuclear fuel performance and fabrication are under way and hold great promise for reducing the concept-to-qualification timeline for new candidate fuel forms. Similar efforts are being conceived in fuel reprocessing and repository performance and design.

But even at this early stage, fast reactor simulations are being used to guide designers by revealing key physics prior to experiment. Such simulations are seen as complementary to both theory and experiment and, prior to a thorough validation, represent a critical component in the evaluation of novel, potentially superior designs.

Contributors:
Paul Fischer, Dinesh Kaushik, David Nowak, Dr. Andrew Siegel, Won Sik Yang, Gail W. Pieper (senior coordinating writer and editor)—all at Argonne National Laboratory.

Further Reading
Nuclear Energy Research Moves toward Greater Reliance on Computer Simulation
http://www.sciencedaily.com/releases/2007/11/071126121729.htm