SciDAC Review - Designing accelerators: PRECISION PROBES for scientific discovery

THE ACCELERATOR SCIENCE AND TECHNOLOGY PROJECT

Designing accelerators:
PRECISION PROBES
for scientific discovery

BY LALI CHATTERJEE

Computational science and simulations are crucial for the design of powerful accelerators and can enable valuable savings in project costs. Using the latest high-performance computing platforms, the SciDAC-funded Advanced Computing for 21st Century Accelerator Science and Technology (AST) project is helping to usher in a new era of accelerator design that will enrich accelerator-based science.

Particle accelerators are extraordinary devices that can produce intense beams for basic and applied science. Developed initially for high-energy physics research, their origins lie in our insatiable quest to understand the true nature of the universe. Subatomic particles, accelerated to billions of electron volts in gigantic accelerators, have been used for decades by particle physicists to probe the structure of matter at its most elemental level. Figure 1 shows the planned ATLAS detector at one of the interaction points of the Large Hadron Collider (LHC). The LHC is currently under construction at CERN (the European Organization for Nuclear Research in Geneva) as an international collaborative project. It is expected to start unraveling outstanding science questions in 2007.

Fig. 1. The ATLAS detector at one of the interaction points of the Large Hadron Collider.

Today, accelerators have expanded their role far beyond high-energy physics and basic research. Accelerator science has led to important discoveries in materials science, nuclear physics, chemistry, and the biosciences, and is used in medicine, industry, and national security. The importance of accelerators at the national level is reflected by the central positioning of accelerator science in the Department of Energy's Twenty Year Outlook report.

The next generation of accelerators will pursue new requirements of beam intensity, precision, and system complexity. These technical challenges, along with the immense size of highenergy accelerators, lead to significant cost challenges. Designing and building the next generation of powerful accelerators therefore requires a daunting optimization of science, technology, design and efficiency.

Fig. 2. These results from a beam dynamics simulation show the formation of a beam halo during the propagation of a high-intensity beam (labeled chronologically from the starting image (a) to the concluding image (f)). The streamlines are used to help visualize the particle dynamics and explore how particles migrate from the inner core out to the tail of the distribution, i.e. out to the beam halo. Understanding and mitigating beam halo formation is an important issue for present and future high-intensity accelerators.

Fig. 3. The Fermi National Accelerator Laboratory (FNAL) has hosted important fundamental discoveries like that of the b and t quarks.

Advanced computing holds the key to meeting this challenge. Innovative designs based on sophisticated computer modeling can reduce the cost of future accelerators, potentially saving capital costs. Improved designs can also increase available beam time for science and maximize the potential for scientific discovery.

The Accelerator Science & Technology (AST) project within the SciDAC program, led by Dr Robert Ryne of Lawrence Berkeley National Laboratory (LBNL) and Dr Kwok Ko of Stanford Linear Accelerator Center (SLAC), focuses on utilizing the latest high-performance computation resources to establish a national cyber capability for accelerator design. This new capability includes a comprehensive suite of terascale simulation codes for beam dynamics and electromagnetic modeling. The codes have been successfully used by researchers nationwide to improve the performance of existing accelerators and to design future accelerators. Parallel simulation codes running on high-end platforms are now seen as critical ingredients of accelerator projects.

Terascale modeling is seen as essential not only to the major accelerator projects, but also for advancing the frontiers of accelerator science and technology. Under the AST project, researchers are developing and using advanced computational capabilities to explore novel accelerator concepts. Based on the incredible intensity of the electromagnetic fields that can be sustained in lasers and plasmas, these hold the potential to address one of the critical needs for future high-energy machines: higher accelerating gradients at lower costs. Projected goals include compact, inexpensive accelerators, and greater gains for research.

Accelerators: real and virtual
Most of the major existing (and planned) accelerator facilities house accelerators with beamlines ranging in length from hundreds of meters in medium-sized facilities to the 27 km ring of the CERN LHC. Typical energies range from giga electron volts to tera electron volts. Figure 3 shows the 6.28 km circular accelerator at Fermi National Accelerator Laboratory (FNAL) in Batavia, Illinois, while figure 4 shows a portion of the beamline inside the accelerator tunnel for comparison of size.

Fig. 4. A section of the tunnel inside the Tevatron, currently the highest-energy accelerator in the world, gives an indication of the scale of these constructions.

Accelerators rely on electromagnetic fields to provide the acceleration, and they typically use magnetic fields to bend and focus the particle beams. Radiofrequency cavities (for acceleration), dipole magnets (for bending), and quadrupole magnets (for focusing) are the most commonly used beamline elements in accelerators. But many more items are present in beamlines, such as high order multipoles (to control the nonlinear behavior of the beams), beam position monitors and other diagnostics, sensors, etc. Many thousands of high-tech devices are therefore spread around a machine many kilometers in length. And the interaction point of a collider is itself extraordinarily complex, as is evident in figure 1. Given the extreme size, cost, and complexity of modernday accelerators, innovative research into improving the design and performance of accelerators becomes as challenging as the basic research and application they enable.

Due to the high cost and extreme complexity of future accelerators, it has become essential to perform exhaustive numerical simulations before embarking on new facilities. These simulations work to optimize the cost-effectiveness of the design, to ensure that it will meet its physics goals, and to verify that the machine will operate safely and reliably. The AST team is developing a virtual accelerator model to perform this cyber modeling and simulation of nextgeneration accelerators. The two main ingredients of a virtual accelerator model are electromagnetic field calculations and beam dynamics calculations.

Electromagnetic field calculations are used to find the fields in geometrically complex accelerating cavities, magnets, and other beamline components. This is done using a variety of techniques, such as finite-difference models and finite-element models. These solve Maxwell's equations using a number of approaches including time-domain solvers, frequency domain solvers, and eigenmode solvers.

Beam dynamics codes are used to predict the motion of charged particles as they travel through an accelerator. Given that a beam may circulate millions of times in a collider, the actual distance travelled may be many billions of kilometers. Computing this behavior is a daunting task. Sophisticated mathematical techniques, such as Lie algebraic methods and differential algebraic techniques, are used to predict the complex, nonlinear dynamics of individual particles in accelerators.

Accelerating particles
The largest and most versatile accelerator is the cosmos itself. Ultra-high-energy cosmic rays with tera electron volts of energy make their way to terrestrial detectors from the farthest corners of the universe. Indeed, before the age of high-energy accelerators for planned research, cosmic-ray events were one of our main observational windows to fundamental physics. Both gravitational and electromagnetic accelerating fields serve as accelerating tools in the natural multi-component accelerators of the universe. Gravitational acceleration spawns star birth and influences large-scale cosmology and astrophysics. Nearer home, electromagnetic acceleration of atmospheric charged particles gives rise to the Aurora Borealis, or Northern Lights, and the Earth's electromagnetic field traps charged particles in the Van Allen Belts around it.	Fig. 5.High-energy particles with energies up to thousands of tera electron volts, believed to have been accelerated in supernova remnants and pulsar nebulae, have been detected in the past. The recent observation of a vast looping structure 20 light-years across, discovered last year, could be due to powerful particle-acceleration near a starforming region in the heart of the Milky Way. Fig. 6.The Ds of a circular cyclotron.
Most early research reactors used electromagnetic fields to accelerate particles in vacuum and this continues to be the basis for linear accelerators. These provide acceleration in a single or series of vacuum cavities exposed to large electrical potential differences. Early in the game, circular machines such as the cyclotron (designed by E. O. Lawrence in 1931) used electromagnetic fields to provide acceleration in short spurts and a magnetic field to confine accelerated particles in expanding circular tracks. A charged particle moving in a magnetic field follows circular trajectories as discussed in the feature "Simulating star power on Earth" on p40.
The radius of the circular path of the particle depends on the velocity of the particle and the strength of the magnetic field. So the particles move in expanded orbits every time they are accelerated. This is demonstrated by figure 6, which shows the cavities that house the accelerating particles. They are conventionally called "D"s because of their shape and they constrain the circulating particles by a magnetic field applied perpendicular to the plane. The particles are accelerated in the region between the Ds by electromagnetic fields and so they start an expanded track when they next enter a D.
As can be expected, such devices are limited by the size of the Ds, which are partially dependent on the extent to which uniformity can be maintained for the magnetic field. The synchrotron - a quick successor to the cyclotron - keeps the particles in a fixed orbit by changing the magnetic field every time the beam experiences acceleration. The acceleration is provided at intervals during the circling process. This synchronization of the magnetic field with the beam energy gives the machine its name.
We have discussed some of the demands of the next-generation accelerators in terms of beam intensity, performance as well as capital and operating costs. In addition, when a charged particle is accelerated, it loses energy by radiating it away. When a particle moves in a circular orbit, this radiation loss is larger and it is also larger for electrons than for heavier particles. These are all factors that need to be considered when designing accelerators or simulating their virtual precursors.

When the particles interact with one another, the widely used particle-in-cell (PIC) method is employed. This method is used not only in beam dynamics simulations, but also in astrophysics, cosmology, and fusion simulations. The basic idea in terms of fusion applications is discussed in the feature "Simulating star power on Earth" on p40. Implementation of the parallel PIC method on today's high-end computers presents many challenges. Depending on the precise algorithm employed, this may involve regular and irregular communication, nearest-neighbor and global communication, and load balancing.

In the case of accelerator beams, the number of charged particles in a bunch ranges from roughly 10⁹ to 10¹⁴ for typical applications. This allows simulations to be run with close to, or within a few orders of magnitude of, the physical number of particles using the present and near-term generation of parallel supercomputers. This may be contrasted with many plasma physics applications (typically 10²⁰ particles) and cosmology (typically 10⁷⁰ in dark matter simulation codes). This distinction is important because it demonstrates the relative closeness of simulation environments to real particle scenarios for accelerator beams.

The International Linear Collider
Accelerator-based high-energy physics research involves large international collaborations, connected by the common goal of probing the nature of our universe. As science advances and its enabling tools increase in precision, we need even sharper probes to push the frontiers of discovery. This corresponds to the need for increasing energies of the particle probes obtained from accelerators. This in turn drives a stronger effort to build effective global collaborations to achieve these objectives.
The International Linear Collider project is intended to explore energy regions previously inaccessible to controlled experiments. Some 12 countries and about 20 group leaders are teaming together to understand the compelling questions that still elude answers - from the structure of the universe and space-time to the nature of dark matter, dark energy and extra dimensions. The US high-energy physics community is involved closely in this effort as is the AST project.	Fig. 7. Model of the Superconducting Low-Loss (LL) Accelerator Cavity design for the ILC. The side view (center) shows the nine cells, while the end views show the front Higher-Order Mode (HOM) coupler (left) and the rear FPC/HOM couplers (right) respectively. Fig. 8. Mesh of simplified test cavity with input coupler only. Fig. 9. Convergence of frequency error calculated with Omega 3P versus required memory (blue is linear elements, red is quadratic elements). Fig. 10. Artist's rendition of the proposed International Linear Collider (ILC).
ILC is designed as a 40 km-long collider that will produce intensely concentrated beams containing billions of high-energy electrons. It will smash these into similar beams composed of positrons, or antiparticles of electrons. As these billions of representatives of our familiar matter particles collide with and annihilate their antiparticles, the extravaganza of available energy is expected to realize new thresholds for particles and resonances and answer some of the outstanding questions about super symmetry, Higgs bosons and much more.
Current projections anticipate the use of superconducting technology for the accelerator cavities, and the planning and designing of this fully international project is being led by an international team headed by Barry Barish of Caltech. As might be expected, computerbased simulations are going to play a dominant role in the planning and designing of this proposed collider. A large part of the machine costs pertain to the 20,000 cavities that will be needed to accelerate the beams to 500 GeV. Simulations for the design of these cavities and their electromagnetic structures, undertaken by AST scientists, are of primary importance as this project moves forward.
Scientists from the German Electron Synchrotron (DESY), the High Energy Accelerator Research Organization (KEK), the Thomas Jefferson National Accelerator Facility (JLab), FNAL, and SLAC are collaborating to investigate an alternate cavity design for the ILC: the lowloss (LL) design. This LL cell shape has less cryogenic loss and higher accelerating gradient than the standard shapes currently in use. While recent experiments demonstrate that a single LL cell can reach 46.5 MV/m as compared to the 35 MV/m achieved with standard cells, this high gradient needs to be reproduced in a nine-cell cavity to mimic ILC. In addition, it is important to find ways to damp beam-generated higher-ordermodes (HOMs) in the cavity that can disrupt beam transport down the beam line.
Omega3P is formulated on tetrahedral mesh with a curved surface and its success is dictated by the combined use of finite element basis functions and high-performance parallel processing. Figure 8 shows the increasing density samples in a test cavity. The accompanying graph demonstrates that quadratic elements provide much higher accuracy for the same available memory and that parallel processing platforms with higher processor capabilities are needed for accurate modeling of the entire cavity of nine cell

In some circumstances the physical phenomena being modeled can be treated using a static or quasi-static approximation. But when particles and fields interact in a way that must be treated self-consistently, fully electromagnetic PIC simulations are required. Electromagnetic PIC codes developed under the AST project are being used to investigate advanced accelerator concepts like the plasma afterburner (see figures 11 and 12).

Fig. 11. A goal is to build a virtual accelerator: a 100+ GeV- on-100+ GeV e^-e⁺ collider based on plasma afterburners.

Fig. 12. Simulating a TeV afterburner stage in three dimensions using QuickPIC. The figure shows color contours of the drive and trailing electron beams and the plasma density. The drive beam is moving from right to left. The trailing beam is bright red.

Computational beam dynamics
Beam dynamics calculations are the foundation of every accelerator design effort. The types of problems encountered in computational beam dynamics are both highly varied and extraordinarily difficult. At one end of the spectrum are the calculations of single-particle, nonlinear beam optics, such as the calculation of "dynamic aperture." This usually involves simulating many millions of turns of a single particle in a collider. At the other end of the spectrum is the simulation of collective effects such as space-charge effects, which may involve the mutual interaction of billions of particles with each other and with the accelerator environment. Most challenging of all, some accelerators require both types of calculations simultaneously. For example, modeling a circular collider like the Tevatron or LHC involves long-term tracking of single particle orbits while at the same time modeling the multiparticle beam-beam effect. Similarly, modeling the International Linear Collider (ILC; see sidebar p17) damping rings involves long-term tracking along with the inclusion of wakefield effects.

Under SciDAC, the AST beam dynamics team has developed a comprehensive suite of codes that has the capability to model, in a single simulation, several of the most important physical phenomena found in accelerators. Codes like the IMPACT suite of codes, MaryLie/IMPACT, Beam- Beam3D, and the Synergia framework are able to model single particle nonlinear dynamics, multiparticle space-charge effects, multi-particle beam-beam effects, and wakefield effects. Thanks to SciDAC, these codes - which are 3D parallel PIC codes - represent a spectacular advance compared with the beam dynamics capabilities of the latter part of the 20th century. At that time, beam dynamics calculations were rarely performed with more than 10,000 simulation particles; this limitation diminished greatly the accuracy, resolution, and ability to model 3D effects. Codes developed under SciDAC using sophisticated multi-physics models eliminate the need for the limited beam dynamics codes of the past. The new parallel computing codes supersede the earlier ones that were based on simplified models and can be run with hundreds of millions of simulation particles.

Fig. 13. MaryLie/IMPACT code development is performed by a large, multidisciplinary team whose members provide specific capabilities needed to produce a coherent, comprehensive, parallel, terascale beam dynamics code.

Figure 13 on p18 describes one such code, MaryLie/IMPACT. This is a hybrid code that combines and extends the functionality of two codes: MaryLie, a high order optics code developed by Alex Dragt of the University of Maryland; and IMPACT, a parallel PIC code developed by Ji Qiang of LBNL. As is evident from the figure, many institutions are involved in the collaborative codedevelopment effort, and many types of researchers (accelerator physicists, applied mathematicians, etc.) contribute. Another example is provided by Synergia, a parallel beam dynamics simulation framework developed under AST. Based on modern programming design, Synergia combines multiple functionality along with a "humane" user interface and standard problem description. According to developer Panagiotis Spentzouris, Synergia is now the main code in use at FNAL for modeling and understanding emittance dilution in the FNAL booster. Figure 14 is a frame from a Synergia simulation showing longitudinal phase space structure formation during the bunching phase of the Booster cycle.

Fig. 14. FNAL booster simulation using Synergia.

Fig. 15. Partitioned mesh of the ILC low-loss accelerating cavity.

Fig. 16. A pair of degenerate dipole modes in the ILC LL cavity exhibiting mode rotation in the time evolution of their electromagnetic field patterns.

According to co-Principal Investigator Dr Robert Ryne, "SciDAC has led to a major shift in the way we do code development. Previously most accelerator codes were developed by individuals or small teams associated with accelerator projects. Under SciDAC we have a comprehensive vision and a recognition that, to succeed, applications scientists need to collaborate with computer scientists, applied mathematicians, visualization experts, and others from the IT community."

Fig. 17. Partitioned mesh of the damped, detuned cell for the next linear collider (predecessor to the ILC).
Fig. 18. Isosurface plots of the electron beam (orange) and electron plasma (green) density as the beam creates a wake in a self-ionized generated plasma. The projections on the walls are color contour plots of the electron plasma density.

The ability to perform fast, accurate, realistic beam dynamics simulations has huge consequences. For example, the computation of dynamic aperture affects the size of the beam pipe; this can have consequences of hundreds of millions of dollars or more in construction costs. Using simulation to predict and mitigate instabilities affects the maximum amount of current that can be transported; this ultimately affects how much science can be done at an accelerator facility. High-resolution simulations are crucial to predicting and minimizing the formation of beam halos and the loss of particles that strike the beam pipe. When too much charge is lost, the beam pipe and surrounding components become radioactive. This can hinder or prevent hands-on maintenance, which reduces the time for accelerator operation (and hence, reduces the science). For example, accelerators like the SNS linac or the proposed FNAL Proton Driver can lose about 1W of power per meter of accelerator. When one considers that the beam power is now approaching 1MW, the loss of 1 W/m represents a very tiny loss indeed. Beam halo is a key issue for future high-intensity accelerators, and high-resolution beam dynamics modeling is a critical tool for designing such ultra-low-loss accelerators.

Electromagnetic modeling
Next-generation accelerators are planned with increasingly challenging specifications in beam energy, precision demands and machine current. As a result, the geometry of the accelerating cavity, for example, has become more complex and the design constraints more stringent. "While numerical modeling is already used extensively in the accelerator community," says Dr Kwok Ko, co-Principal Investigator on the AST project, "more advanced tools are needed to simulate the complicated cavity designs under consideration for future facilities, such as the ILC and the Rare Isotope Accelerator [RIA], to accuracies in speed and problem size previously not possible."

The Electromagnetic System Simulations (ESS) team, working within the AST project, is concerned with the design of the actual hardware of the accelerator, such as the accelerating cavity and the associated beam line sections. Simulating accelerator cavities prior to construction is a practice that has been in vogue for decades. Conventional electromagnetic software for modeling accelerating cavities include MAFIA, Microwave Studio and HFSS. Dr Ko says: "Most present codes are limited to small problem size as they only run on a single computer, while the latest supercomputers consist of thousands of processors with a significantly larger total memory." Evolution of massively parallel computer architecture providing extensive accessible memory invokes the need to develop scalable capability in software tools and computing techniques to harness this increase in compute power and memory. AST codes are filling this need to accelerate advances in accelerator design, including cavities with incredibly complex geometric shape and high accuracy requirements because of tight fabrication tolerances (see figures 15-17).

New plasma acceleration methods

Conventional acceleration techniques, as described in the sidebar "Accelerating particles" on p16, are limited to the energies they can reach by their sizes. These techniques allow accelerating gradients of 20-100 MeV energies per meter traversed by the accelerated particles. In addition, the technology involved in maintaining strong fields across extended regions of space is formidable, and beams may become susceptible to break down conditions. To overcome these potential hurdles, the accelerator community is exploring novel methods to circumvent them for new generations of accelerators.

A promising answer lies in the use of the properties of plasmas to allow high-gradient acceleration when injected with a driver beam (see figures 18-23). As the driver beam cuts through the plasma it generates a wake that can trap plasma particles and carry them forward. This effect is able to accelerate the wake particles to high energies over extremely short length and time scales. The resulting accelerating gradients can be as high as 10-100 GeV per meter, resulting in a factor of 1000 magnification over conventional techniques. Just as nature eludes perfect symmetry, so also there is rarely a "perfect" solution to any problem. In the case of plasma acceleration, the difficulties lie in focusing and stabilizing the driver beams, which are usually lasers or particle beams.

Work in developing novel accelerator technology requires both simulation and experimental studies. AST researchers have designed the "QuickPIC" code - which allows a faster and more efficient use of the PIC method on HPC platforms. Recent experiments have shown significant energy gains by these methods and AST codes were used in the planning of these experiments.

Fig. 19. A 2D slice of a 3D simulation showing the laser envelope (in orange) and the plasma density (in blue). As the laser moves from right to left it blows out the electrons, which rush back to the axis once the laser has passed. There, they feel a strong accelerating force and are self-injected in the laser's wakefield. They are accelerated until they outrun the wake. Fig. 20. Electron beam driven wakefield excitation. A 2D color contour plot of the beam and electron density is obtained from a 3D QuickPIC simulation. The beam is moving from right to left. The black line is the resulting accelerating electric field.

Figures 21-22
Fig. 21. 3D QuickPIC simulations of the E-167 experiment at SLAC. Plots of the energy of the electron beam against one transverse coordinate. Fig. 21.(a) is the image without plasma, showing the energy chirp of the incident beam. Fig. 21.(b) is the image after the beam has propagated through the plasma, showing the acceleration of the tail and the deceleration of the head of the beam. Fig. 22. Comparison of the accelerated electron-beam energy spectra between experimental data and a 3D OSIRIS simulation.

Scientists working on the AST project based at SLAC are developing parallel finite element electromagnetic codes that utilize the massive memory resources of the Department of Energy (DOE)'s Office of Science Supercomputers, e.g. the Cray/X1E (Phoenix) at NLCF and the IBM/SP (Seaborg) at NERSC. The suite of codes includes the eigensolver Omega3P, the S-matrix solver S3P, the time-domain solver T3P, and the particle tracking code Track3P, which can provide significant gains in accuracy, problem size and solution speed when run on the these flagship computers. In particular, Omega3P has proven invaluable in the application to the design of a new low-loss (LL) accelerating cavity for the ILC.

As in the case of the other SciDAC projects, the AST ESS team works in close collaboration with researchers in the Integrated Software Infrastructure Centers (ISICs) and Scientific Application Pilot Program (SAPP). This multi-disciplinary group of scientists is working together to improve a multi-step simulation process that starts with the geometry of the electromagnetic structure in the form of a drawing or base model. The systems are partitioned into meshes and studied on high-performance computers using solver codes. Visualization techniques are important aids for the analysis of the complex data sets resulting from the unstructured grid solutions. The major achievements include meshing, eigensolvers, visualization, refinement, and shape optimization of computational applications for accelerator design.

Fig. 23. Visual rendering of plasma density isosurfaces for colliding electron beams (figure (a)), and colliding electron positron beams (figure (b)). For the electron beam (yellow) the plasma electrons are expelled outward while for the positron beam (red) they are pulled inwards. These threedimensional simulations were done using the fully parallelized code QuickPIC.

Accelerators and society
Particle accelerators and the technologies associated with them have numerous applications for society. Electron linear accelerators used for radiotherapy, acceleratorproduced protons and neutrons used to irradiate tumors, and nuclear diagnostic medicine save countless lives and serve society on a daily basis. Synchrotron radiation facilities and spallation neutron sources impact on the biomedical sciences by allowing protein imaging to understand new biological mechanisms and to guide drug design.	Fig. 24. When he discovered X-rays, Wilhelm Roentgen took an X-ray photograph of his wife's hand bones and wedding ring - an early preview of the power of X-rays to reveal hidden worlds in nature. A century later, LCLS will have the tremendous resolution to illuminate the as-yetunseen world of atoms jostling each other and vibrating from one quantum state to another.
Extending beyond medical and healthcare applications, accelerators and accelerator technology have important industrial applications. They are used in the analysis and modification of physical, chemical, and biological materials with important economic consequences. Examples include particleinduced X-ray emission, protein crystallography, ion implantation, and electron beam welding. Accelerators also play an important role in nanoscience through technologies such as electron beam lithography and in general contribute to economic and industrial competitiveness for the nation.
Stockpile stewardship and national security are also areas in which accelerators have found their placement. Induction linacs are now used for X-ray radiography of hydrodynamic tests. In the future, image sequences may be produced using proton radiography. An important industrial spin-off would be the ability to examine critical structures such as spinning turbines in real time. Accelerator-based terahertz radiation, or "T-rays," have been shown to be effective in imaging through thin layers of material to reveal, for instance, hidden weapons. Portable neutron generators, also based on accelerator technology, are able to see inside luggage.
In regard to energy independence, the Office of Science programs within the DOE have made great strides in developing heavy-ion particle accelerators as drivers for inertial fusion energy. Accelerators have also been proposed to transmute radioactive waste from nuclear reactors, adding to environment clean-up efforts.

New plasma accelerators
The efficiency and power of an accelerator depends on the rate of acceleration, or gradient, of the system. As discussed in the sidebar "New plasma accelerating techniques" above, plasma acceleration techniques could achieve gradients and focusing forces more than 1000 times greater than conventional technology. The challenge is to control and link these high-gradient systems to obtain compact accelerators. "These methods can reduce size from kilometers to meters," says Dr Warren Mori, a University of California, Los Angeles (UCLA) scientist working on the AST project. "Furthermore, we have developed codes that allow us to study the key physics that needs to be understood before a 100+ GeV collider based on plasma techniques can be designed and tested."

While there has been a lot of progress in simulating these new accelerator models, computational efforts have been limited by processor power and lack of codes that perform efficiently on parallel architectures. AST research has enabled the development of innovative algorithms that have been implemented into new codes such as QuickPIC, OSIRIS and VORPAL that can save CPU time and sample larger data sets. Although they are based on the conventional PIC method, these codes are fully parallelized, scale well and incorporate load balancing and particle sorting.

Chasing discovery: accelerators and basic research
Our first evidence for the structure of matter and the atom came from using particle projectiles. Lord Rutherford's pioneering experiment with alpha particles (or helium nuclei) in 1911 demonstrated that most of matter is empty space, and the nuclei that provide most of its mass reside in the cores of its atoms. Decades later, in the late 1960s, a SLAC-MIT group, scattering electrons with energies of 7-17 GeV from a hydrogen target, demonstrated that nucleons are composed of pointlike constituents, or "partons." The leaders of the group, Friedman, Kendall, and Taylor were awarded the Nobel prize in 1990 for this work. The partons of their discovery were later identified as the quarks and gluons of the Standard Model of particle physics.	Fig. 25. Simulation of a Higgs decay to four isolated muons in the CMS detector at the Large Hadron Collider at CERN. The lines denote particles produced from the collision of a pair of ultra-high energy protons. Energy deposits of the particles in the detector are shown in blue.
High-energy experiments using particle accelerators require ever-increasing energies so that they can probe the structure of matter in ever-increasing scales and enable the discovery of new exotic physics. These experiments are of two kinds - fixed-target experiments, in which high-energy projectiles hit a stationary target, or collider experiments where accelerated particles collide head on. In the last 30 years, utilizing such experiments, researchers have discovered the existence of three families of quarks (updown, charm-strange, top-bottom), with similar properties but different masses. The last (and heaviest) quark - the top quark - was discovered near the turn of the century at the Fermilab Tevatron.
In addition to the quark families, high-energy physics experiments utilizing high-energy beams discovered that leptons (electrons and neutrinos) also come in three families. The discovery of the additional flavors of neutrinos was a particularly challenging task since neutrinos only interact weakly (and thus are harder to detect). Leon Lederman, Melvin Schwartz, and Jack Steinberger were jointly awarded the Nobel prize in 1988 for discovering the muon neutrino. The tau neutrino was first directly observed very recently, in 2000, at Fermilab. The picture of the Standard Model of particle physics was almost completed with the discovery of the electroweak gauge bosons W and Z and the precision measurement of their properties, using data from high-energy collider experiments. The only missing ingredient of the Standard Model is the Higgs boson, the particle believed to be responsible for mass generation. Researchers are hoping that the discovery of the Higgs will be the first major discovery of the experiments at the Large Hadron Collider accelerator facility.
Physicists at the Relativistic Heavy Ion Collider (RHIC) at Brookhaven National Laboratory have been smashing beams of highenergy gold nuclei into each other to try to recreate the conditions of the early universe, as part of nuclear physics experiments. This research aims to demonstrate that the nature of the matter produced at the collision points is that of the liquid phase of deconfined quarks and gluons, or "quark-gluon plasma."
As experiments have evolved in energy regimes and complexity, the challenges of data-handling and management, as well as designing and operating accelerators, have become integral parts of the chase for new science. Today, we look to the next generation of accelerators to enable the discovery of new, exciting physics - new kinds of particles, the origin of mass, asymmetries between matter and antimatter, supersymmetric partners of familiar particles and possible extra dimensions of space - to name just a few. Particle accelerators are also drivers of research in material sciences and molecular biology. The Spallation Neutron Source under construction at ORNL will provide intense neutron beams to study matter and its properties at the atomic level. The LCLS will revolutionize our ways of studying chemical reactions at the atomic level and biological phenomena at the molecular level (see feature "Scientific discovery: powerful, unpredictable, aesthetic," p8).
All of these experiments involve challenges of data and its management and analysis, and many of them involve large global collaborations. Accelerator-based research has opened many doors to our understanding of nature and the universe we live in, and is poised to open many more.

In some of these advanced accelerator concepts, the driver beam is either an intense laser pulse or a particle beam. As the drive beam moves through the plasma, it generates a space charge wake arising from radiation pressure and a trailing beam of particles can be accelerated by riding on this. The equations and physics need to be modified and managed differently for the driver and trailing beams and individual trajectories need to be mapped.

Fig. 26. An adaptation of the "Livingston Curve," displaying the center of mass energies of the particle (parton) constituents of existing and planned accelerators (electron-positron and hadron colliders) as a function of the timeline. In the case of the hadron machines, energies have been adjusted to account for quark and gluon constituents. The original NLC (New Linear Collider) point has been updated to the ILC and its expected upgrade.

Fig. 27. LCLS will allow the use of ultra-short X-ray pulses to obtain instantaneous pictures of molecular states. It thus offers the possibility of studying molecular structure in ways that were impossible before.

To give a feel for the numbers, typical simulated systems contain 10⁸ or 10⁹ particles and require 10⁵ time steps. In terms of computational needs, they require 10⁴-10⁵ CPU hours and involve 10-100 Gbyte of data. In most of the PIC-based codes the basic techniques use Maxwell's equations in the field solver to determine the force to be applied to individual particles during the push phase. Modern SciDAC codes, developed through the collaborative efforts of accelerator scientists, applied mathematicians and computer scientists, use special techniques including co-ordinate transformations to optimize use of the supercomputing power currently available. Details of the code can be found in the literature (see Further Reading on p25 for details).

AST's role in accelerator science and discovery
Accelerator-based science is built around large facilities with multiple user groups and large global collaborations. Scientists from the AST project are involved in most of the DOE accelerator facilities. AST codes have played a major role in improving the performance of existing facilities like the Tevatron, PEP-II, and RHIC. Their presence will be felt even more strongly in the design, commissioning, and operation of future facilities.

Before the end of the decade, the Linac Coherent Light Source (LCLS) will begin operation and open the door to new discoveries in ultrafast science. The LHC will start exploring untouched territory at the energy frontier, and the global design effort for the ILC will be completed and ready to begin construction pending project approval. Other accelerator facilities may be built in the next decade, like a Proton Driver for neutrino research, the Rare Isotope Accelerator or an electron-ion collider, or new fourth-generation light sources. For all these facilities the scientific, technological, and economic stakes are high. Advanced computing will play a central role in risk and cost reduction, and in ensuring their successful design, construction, and operation.

Perhaps the greatest challenge facing particle accelerator technology is that current methods are reaching their limits. Just as computer scientists see the end of Moore��s law around 2020, the exponential growth in accelerator technology, described by the so-called ��Livingston curve�� (figure 26), is also reaching its technical limit.

Innovative ideas and advanced accelerator R&D are starting to break through this barrier. Extensive modeling under AST, combined with theory and experiment, has helped to unravel the mysteries and complex phenomena present in laser- and plasma-based accelerators. Other innovative ideas photonic bandgap and dielectric structures, intense neutrino beams, colliding muons, intense ultra-short pulses of radiation �� will help ensure that accelerator-based scientific discovery continues to flourish in the 21st century. And for all of these, advanced computing will continue to be a critical tool used by researchers to advance the frontiers of accelerator science and technology.

Further reading

K. Ko et al. 2005 Impact of SciDAC on accelerator projects across the office of science through electromagnetic modeling J. Phys.: Conf. Ser. 16 195-204.

W. B. Mori et al. 2005 Advanced accelerator simulation research: miniaturizing accelerators from kilometers to meters J. Phys.: Conf. Ser. 16 184-194.

S. P. D. Mangles et al. 2004 Monoenergetic beams of relativistic electrons from intense laser-plasma interactions Nature 431 535.

R. Ryne et al. SciDAC advances and applications in computational beam dynamics 2005 J. Phys.: Conf. Ser. 16 210-214.