DOESciDAC ReviewOffice of Science
LEADERSHIP COMPUTING FOR SCIDAC

Visualization capabilities illuminate high-performance computing data from large-scale climate simulations.

The LCF delivers grand-challenge scale resources for national science endeavors.

More than 40 Tflop theoretical peak performance is provided by high performance computers at the LCF.
The experimental apparatus of computational scientists
The Leadership Computing Facility at Oak Ridge National Laboratory is committed to providing high-performance resources at least 100 times more powerful than existing national capabilities. The National Center for Computational Sciences at Oak Ridge National Laboratory (ORNL) is one of the world's most powerful non-classified computing resources. It is home to the Leadership Computing Facility (LCF), which has been a principal resource for key SciDAC projects. The LCF's suite of resources comprises more than 40 Tflop theoretical peak performance power, and complements diverse types of computational studies by providing scientists with access to tremendous compute capability and dedicated support expertise. The primary production systems - a Cray X1E and a Cray XT3 - are the largest of their kind available, anywhere in the world.
The LCF Cray X1E system is composed of 1,024 multi-streaming vector processors and is the largest vector processor available for non-classified computing in the US. The system has exceptionally high bandwidth and 2 Tbyte of globally addressable memory. The Cray XT3, with 5,296 compute processors, has a peak theoretical performance of 25 Tflop and more than 10 Tbyte of aggregate memory. The LCF also makes available to users a 256-processor SGI Altix with more than 2 Tbyte of shared memory.
The facility also provides advanced connectivity, with fast network connections to national networks such as the Department of Energy's ESnet and UltraScience networks, Internet2, and National LambdaRail, promoting extended collaborations with researchers around the country. Projects using LCF resources may use a large percentage of the resources for single simulations. SciDAC users of LCF systems have carried out some of the largest-ever studies. The vector-based system at the LCF is used for billion-particle simulations for fusion plasma research and the highresolution results needed are achieved in weeks rather than years (see feature "Simulating star power on Earth," p40). Combining SciDAC-developed software with leadership computing capabilities, researchers are providing invaluable insights for the development of the International Linear Collider (see feature "Designing accelerators: precision probes for scientific discovery," p12). The Cray X1E was used for three-dimensional, two-billion-grid-point simulations of the newly discovered stationary accretion shock instability (SASI; see feature "Modeling the first instants of a star's death," p26) of relevance to core collapse supernovae.
Benchmark calculations
Leadership computing is being used to develop, in a realistic time frame, more detailed computer models of combustion and incorporate the effects of turbulence and fuel-air mixing rates, as well as flame extinction and re-ignition. It is also being employed by computational scientists to carry out extraordinarily detailed benchmark calculations of small molecules. Both of these topics are important for health and environmental issues.
In addition to providing advanced computing resources to researchers across the nation, dedicated support staff and PhD-level scientists are committed to working with users to help them achieve maximum results. Once the calculations are complete, sophisticated visualization tools are at hand to assist the researchers in analyzing their work.
The LCF has an aggressive roadmap to sustain leadership for scientific computing. The plan will quadruple the performance of the Cray XT3 system in 2006 to 100 Tflop. By the end of this decade, the LCF's total combined resources will achieve 1,000 Tflop, meeting the growing demand of scientists for faster, more powerful tools to achieve critically needed simulations. These will dramatically impact on the nation's ability to produce a secure energy economy, and increase mankind's understanding of the fundamental physical, chemical, and biological processes surrounding us.

 

"Computational science is becoming synonymous with scientific discovery. Pre-eminence in highperformance computing is increasingly critical to maintaining US scientific and technological leadership. The DOE Leadership Computing Facility at ORNL is enabling breakthroughs that could have significant impact on our daily lives."
THOMAS ZACHARIA
Associate Laboratory Director
ORNL