SciDAC Review - Modeling the first instants of a STAR'S DEATH

THE TERASCALE SUPERNOVA INITIATIVE

Modeling the first instants of a STAR'S DEATH

BY ROBERT IRION

Scientists funded by the SciDAC Terascale Supernova Initiative (TSI) project are modeling the violent physics of exploding stars. Their work has revealed instability in the shock wave blasts, imparting rotation to the new-born neutron stars in their cores.

Fig. 1. A bright flare of light (lower left) in the outskirts of galaxy NGC 4526, spotted by telescopes in 1994, marked the death of a giant star.

When the biggest stars in the universe have consumed all of their fuel, they don't die quietly. Instead, they explode as supernovae with energy blasts of more than 10⁵³ ergs. These detonations briefly outshine entire galaxies of hundreds of billions of stars (an example being the 1994 supernova shown in figure 1). In our own Milky Way galaxy, astronomers have witnessed several supernovae over the centuries as brilliant "guest stars" that then faded just as mysteriously as they had appeared.

But supernovae are more than just spectacular light shows. They seed space with oxygen, silicon, calcium, iron, and heavier elements - the ingredients of our planet and ourselves. Indeed, astrophysicists believe that many of the atoms in our bodies were propelled into space by nearby supernovae before our solar system formed, about 4.6 billion years ago. So in a way, supernovae represent our most direct connection to the cosmos.

They also produce some of the most exotic physics that nature has to offer. They make neutron stars and black holes, as well as torrents of elusive particles called neutrinos. Their temperatures and densities rise far beyond the kind of conditions physicists will ever be able to create on Earth. For all of these reasons, the effort to understand supernovae is a central quest in astrophysics today.

It is not yet clear exactly what processes make stars explode. Today, nearly four dozen scientists at nine different institutions are tackling this problem with funding from SciDAC in a project called the Terascale Supernova Initiative (TSI). The multidisciplinary team of astrophysicists, nuclear physicists, applied mathematicians, and computer scientists has already developed some of the most sophisticated supercomputer simulations of the first moments of the death of the largest stars, and has unveiled some surprising phenomena deep within the dying stars (see figure 2).

Fig. 2. A 3D visualization reveals lopsided blast waves of gas in the moments after a supernova explodes.

The physics of core collapse
The TSI team, led by astrophysicist Dr Anthony Mezzacappa of Oak Ridge National Laboratory (ORNL), is exploring what happens when the core of a large star - i.e. a star at least eight times as massive as our Sun - runs out of the elements it needs to create energy. The pressures inside these massive stars are intense enough to spark thermonuclear fusion of atomic nuclei beyond hydrogen, the fuel of ordinary stars. Helium, carbon, oxygen, sulfur, and silicon all fuse and release energy in an accelerating sequence of combustion. This sequence builds onion-like shells of these elements. The heavier elements occupy shells closer to the star's hot center, because it takes progressively higher temperatures to ignite fusion as the atomic nuclei grow more massive.

Fig. 3. Simulations explore the physics of turbulent shock waves deep within supernovae.
Fig. 4. Neutrinos streak outward from a collapsed core.

This chain of nuclear fusion releases energy only until iron nuclei form. Thus, fusion stops with iron. As the iron core grows from the fusion of lighter elements in the star into iron, the pressure due to the electrons in the core (which is the dominant source of pressure) can no longer counteract the inward gravitational force of the core. The core implodes, in an event called "core collapse." This ultimately leads to a stellar explosion (see simulation in figure 3) and the synthesis of heavy elements - the hallmarks of a core-collapse supernova. The core collapse also produces neutrinos (at the rate of 10⁵⁷ per second) as shown in the simulation in figure 4 and discussed later.

Within a fraction of a second, this collapse squeezes the inner part of the star's core into a volume just tens of kilometers wide that resists further crushing. The inner core rebounds like a piston, creating a shock wave that rifles back into the still-collapsing outer core. If the shock kept going, it would blow the star apart and spawn the supernova blast. But in all realistic computer models of this process, the shock wave stalls deep inside the star. This is the essence of the core-collapse challenge: what physical processes drive the shock wave violently enough to eject a star's worth of matter into space?

TSI researchers are formulating new ways to examine critical events in the supernova core within the first second after the core bounces. The suite of physics that is required to model these events accurately is so diverse that this challenge may be considered a "renaissance problem" in modern physics.

The cosmic cycles of supernovae

Nature operates in cycles. On Earth, relentless cycles of water, carbon, and minerals shape our dynamic planet. And in space, our solar system and galaxy are part of a cycle of stellar birth and death - a rhythm in which supernovae play a crucial part.

Fig. 5.Cassiopeia A, one of the closest supernova remnants to Earth, exploded into view about 340 years ago. Jets of energy expelled iron, oxygen, and other heavy elements into space.

Stars arise at the centers of cool clouds of gas and dust that take tens of millions of years to contract and heat under gravity. Stellar nurseries create a whole range of objects, from dwarfs to giant stars. Astronomers have determined that most stars are similar in size to our Sun, which is an average star. These stars fuse hydrogen into helium during stable lifetimes lasting billions of years.

When hydrogen runs out, stars like these often burn helium for a short time and become expanded red giants. Some of the outer gas escapes, condensing into dust grains that drift into nebulae for new generations of stars. Usually, a white dwarf, packing a Sun's worth of mass into an Earthsized ball of carbon, is left behind at the core. This dense cinder generally cools off over billions of years until it fades from view. The outcome is different for those rare stars that are at least 8-10 times as massive as our Sun. When the centers of these stars collapse at the end of the nuclear burning phase, the added densities accelerate the actions of the "weak" interaction, resulting in numerous electron captures that produce neutrinos (see sidebar "The importance of particles that barely exist," p33) that transport energy and "lepton number" from the core. They leave behind an ultracompact sphere, largely of neutrons, about 10 km across.

The implosion that creates a neutron star or black hole unleashes a shock wave that propels the rest of the star into space. One or two "core-collapse" supernovae flare in our Milky Way galaxy each century (see figure 5). But in the entire cosmos, a star explodes roughly once every second. This has transformed the universe from a bland fog of hydrogen and helium into the rich mixture of heavier elements we see around us today.

Astrophysicists estimate that the hot-burning cores of massive stars and their deaths in core-collapse supernovae produce most of the cosmic supply of elements with atomic masses between oxygen and iron. Corecollapse supernovae fling these elements into space. And nucleosynthesis in the exploding supernova is believed to create half of the cosmic abundance of elements heavier than iron. Like dust in the wind, these "metals" scatter into space. The debris sets the stage for new stars and solar systems with higher proportions of aluminum, phosphorus, calcium, copper, lead, uranium, and so on. This is the stuff of warm rocky planets and, ultimately, life.

Supernovae also power cycles of energy within a galaxy. The intense shocks produced by the cosmic blasts expand for eons, churning a galaxy's gas and dust. Indeed, the cumulative shocks of many supernovae can eject matter beyond a galaxy's gravitational grasp. In this way, cosmic blasts from supernovae continue to sculpt the universe and the ingredients of its deepest reaches of space.

For example, the dynamics of a collapsing star depend critically on all three fundamental forces of nature: gravity, electroweak, and the strong nuclear forces. Gravity around the compact core is so strong that general relativity is crucial to determining the outcome. The core's density pushes the limits of scientific understanding of the strong nuclear force and the physical states that matter assumes. Magnetic fields in a charged, rapidly rotating medium are difficult to understand and to simulate computationally.

Most critically, the dense object at the heart of the collapse, the new-born "proto-neutron star," is so hot that it radiates huge numbers of neutrinos. The supernova emits more than 10⁵³ ergs of energy in about 10 s - more energy than the rest of the stars in the observable universe combined.

The complex physics of stellar destruction

Astrophysicists know why massive stars explode when they run out of nuclear fuel, but they do not yet understand how the explosions happen. The sudden collapse of a star's core sparks events that push physics far beyond its well-known regimes. With no way to test those extremes in labs on Earth, researchers rely on innovative algorithms that run on supercomputers at very high speeds. The TSI project uses a layered approach to study the complex ingredients of a supernova recipe:

Magnetohydrodynamics. The behavior of fluids in strong magnetic fields is described by magnetohydrodynamics. This is usually applied to stellar research.
Multi-scale physics. Detailed calculations must span more than five orders of magnitude in scale, from the smallest turbulent eddies to the shock wave blasting outward.
Einsteinian gravity. Ultrastrong gravitational fields make it essential to use full general relativity.
Super-nuclear densities. The equation of state describing matter in the newly imploded core - the "proto-neutron star" - requires a deeper grasp of the strong nuclear force inside atomic nuclei.
Microscopic neutrino-nuclear interactions and macroscopic radiation transport. A colossal flux of 10⁵⁷ neutrinos erupts from the core in all directions and at different energies, but details of their impact on the star's infalling matter remains unclear.

This latter challenge is a key frontier for TSI. A solution requires a full description of each neutrino's 3D position and 3D momentum, which is obtained by solving the Boltzman kinetic equations (see figure 7). The computer code must solve this labyrinthine six-dimensional set of algebraic equations (with time as the seventh dimension) to reveal how neutrinos affect the crush of matter cascading inward.

Mathematicians and computer scientists have devised two methods for making this problem tractable. First, "custom preconditioners" applied to the algebraic equations lead to much faster solutions. Second, the solvers run on a massively parallel architecture of thousands of processors. Even so, a typical TSI simulation can require up to 100,000 processor hours to capture a mere 0.03 s of physics at the supernova's core.

Fig. 6. The schematic diagram is indicative of the infalling stellar mass towards the core (M) and its rebound from the tightly packed core (overturn at the gain radius), giving rise to the shock wave. The macroscopic hydrodynamics is influenced by the microscopic neutrino producing reactions in the core and the neutrino absorbing ones from the escaping neutrino flux. The protoneutron star at the core is also shown. Fig. 7. Boltzman kinetic equations.

Fig. 8. TSI researchers devised ways to transmit and store terabytes of data among multiple national sites.

Nearly all of that energy flows out as neutrinos (see figure 4). The explosion relies crucially on how these escaping neutrinos interact with matter rushing inward from the rest of the star. These interactions are dictated by the weak force.

Using new methods, TSI scientists are characterizing neutrino transport and neutrino interactions and grasping how this radiation affects hydrodynamical flows in the collapsing star. One promising line of research involves neutrino oscillations: a phenomenon in which neutrinos of different "flavors" can transform into one another. Theoretical work by physicist Dr George Fuller of the University of California, San Diego, and his colleagues suggests that this mixing occurs over a broad range of energies for both neutrinos and their antimatter counterparts, antineutrinos. Even though the mass differences between neutrino flavors are very small, these rapid-fire transformations could affect the dynamics of the supernova shock wave and the creation of new heavy elements in nucleosynthesis. Dr Fuller's group is now investigating whether this "Background Dominant Solution" for neutrino mixing plays a major role in real supernovae.

New analysis tools
To achieve its scientific result, the TSI team has pushed the boundaries of its field in two distinct ways. First, new computer algorithms have greatly increased the speed and fidelity of analysis in both 2D and 3D simulations of the shock wave and its environment. Second, the team has overcome steep challenges in handling and managing vast quantities of data across networks that span the continent.

To understand neutrino transport at the core of a supernova, the scientists must solve systems of linear algebraic equations underpinning the solution of the neutrino transport equations known as Boltzman kinetic equations (see figure 7). The linear equations themselves are straightforward, but their scale is daunting, involving solution vectors that are terabytes (10¹² bytes) to petabytes (10¹⁵ bytes) in size. To cope with the vast number of calculations, TSI researchers have developed "custom preconditioners" to redefine their linear systems so as to facilitate their solution.

These techniques are used in the context of Newton-Krylov methods for actually solving the systems of equations. Implementations of these preconditioning and solution algorithms for parallel computers were developed through collaborations between TSI scientists at the State University of New York at Stony Brook (SUNY SB) and ORNL with applied mathematicians at the University of Tennessee at Knoxville (UTK) and Santa Clara University in California. These allowed the TSI team to divide the calculations among many thousands of identical computer processors. In this "massively parallel" computing architecture, the neutrino transport equations finally become tractable.

Fig. 9. A community Linux cluster houses data from simulations in the TSI, providing interactive access for team members at different sites. Software divides the supernova data into slabs for faster transport and visualization.

The preconditioners became an enabling technology for Dr Douglas Swesty and Dr Eric Myra of SUNY SB, who say they were able to carry out the most physically detailed 2D core-collapse supernova simulations to date using them. Earlier simulations used a gray approximation, which assumes a shape of the energy distribution spectrum for neutrinos. In contrast, the Swesty and Myra simulations compute the energy spectra of the neutrinos, which change with time and with position as the supernova evolves. That's far more realistic, but it exacts a high penalty in computational cost. For this reason, explains Dr Swesty, the computing resources provided through SciDAC are essential to solving the problem.

Other TSI scientists initially found that the terabyte-sized data streams from powerful supercomputers - including IBM's Seaborg processors at Lawrence Berkeley National Laboratory in California (NERSC) and the Cray X1E at ORNL - were difficult to handle and transfer between workstations. For example, physicist Dr John Blondin at North Carolina State University (NCSU) in Raleigh was unable to transfer the results of his 3D simulations over the Internet or view them on his workstations.

To tackle this challenge of data management, the TSI team worked with computer scientists at the Logistical Computing and Internetworking Laboratory, UTK, and used their Logistical Runtime System to move data on parallel streams, across multiple Internet paths (see figure 8). This tactic increased data-transfer rates by 10-20 times. Moreover, data are now stored on a community Linux cluster at NCSU (see figure 9), which yields interactive access to all team members. The team also worked with Networking researchers at ORNL to move data from the ORNL Cray X1E to NCSU, using the Bearer Channel Control Protocol.

Indeed, the project's challenges have arisen in part from the enormity of the data sets, says Dr Blondin. Pervasive parallelization is the key to solving these challenges. From start to finish, the entire process is divided into multiple components: running the computer codes, writing data onto disks, sending data across networks, and visualizing the results.

The importance of particles that barely exist

Neutrinos are full of surprises. They come in three "flavors" - electron neutrinos, muon neutrinos, and tau neutrinos - with different tiny masses, although scientists still don't know what those masses are. Physicists have confirmed that neutrinos are flavor shifters: they can oscillate from one flavor to another as they move at close to the speed of light. A lone neutrino, dashing through space, could penetrate a light-year of solid lead without noticing other particles. In the parlance of physics, a neutrino's interaction cross-section is vanishingly small. Yet the billion-degree core of a supernova produces so many neutrinos from such a compressed volume - 10⁵⁷ of them - that they "blow" against the star's infalling matter with significant force. Nailing down the details of how neutrinos cascade outward and interact with matter during the earliest stages of a core-collapse supernova is crucial to the supernova science.

Fig. 10. Super-Kamiokande, a neutrino detector in Japan, holds 50,000 tons of ultrapure water surrounded by light tubes. Neutrinos from a nearby supernova that occurred in 1987 sparked flashes of light in smaller water tanks.

The link between supernovae and neutrinos became strikingly real on February 23, 1987, when light from an exploded star, Supernova 1987A, in a small nearby galaxy first swept across Earth (see sidebar "A history of violence in the sky," p35). Although the star was 170,000 light-years away, the flux of neutrinos on our planet was an incredible 100 billion per square centimeter. Of this onslaught, physicists managed to catch only 19 of the fleeting particles when they collided with atomic nuclei in special neutrino detectors, because neutrinos have such low interaction cross sections.

Neutrino detectors have to be very large because of the low interaction rates of neutrinos (see figure 10). Scientists think they are ready to catch a burst of the particles from the next nearby supernova - providing a glimpse of its innermost turbulence. Neutrinos escape from the explosion essentially in an instant, while the explosion itself takes many hours to rip apart the star. Therefore, a pulse of alien neutrinos may provide an "early warning" for astronomers to look skyward for an impending supernova. A computer alert network links the neutrino detectors together for this purpose.

During their wait, physicists are confronting many vexing questions about neutrinos. Most notably, how massive are they? Experiments show that neutrinos possess a smidgen of mass, but their exact value is unclear. Knowing that value will prove critical to specifying the degree to which neutrinos impact on their surroundings at the core of a burgeoning supernova.

TSI researchers believe that oscillations among electron, muon, and tau neutrinos may play an important role in the physics of the explosion. But for the time being, the team's simulations feature mass neutrinos. Endowing them with mass would make the computations too complex, says Dr Mezzacappa, because neutrino transport models are not yet reliable even for zero mass. The models must include full "neutrino distribution functions," which define the energies and 3D motions of the particles at all times. When one includes the spatial co-ordinates within the explosion and the dimension of time itself, solving for neutrino transport becomes a sevendimensional miasma.

It's a lot of fuss over a particle that's as close to nothing as it is possible to get. But in the end, TSI physicists believe that the dynamics of neutrinos will solve the mystery of why supernovae explode at all.

More than the eye can see
That last step - visualization - is vital for both the TSI team and its audience of peers and the public. Visual representation can be very helpful in understanding the details of complex physical analyses. But the outputs from supernova simulations are so gigantic that one cannot portray all aspects of the data at once. Instead, visualization experts are creating ways to highlight specific features of the turbulent flow, such as the flux and energy of neutrinos or the entropy of gas.

Fig. 11. TSI simulations suggest that an unstable spiral shock wave, called a SASI, rotates around the core of a supernova in its first 0.03 s of being.
Fig. 12. Matter falling onto the proto-neutron star gets wound up by this intense flow pattern.

Visualization tools developed by the TSI team have led to new and striking ways to reveal and interpret data. For example, computer scientists at Indiana University at Indianapolis portrayed multiple aspects of 2D hydrodynamics simulations, including neutrino transport, with a technique called "Lagrangian Eulerian Advection."

The resulting images make use of streaks to show neutrino velocities and trajectories, as well as color maps and contours to display entropy and neutrino optical depth. Visually comparing these fields as the shock wave develops in the simulation helps the researchers to verify the accuracy of their work, and provides insights into the processes occurring deep within the supernova.

TSI visualization staff at ORNL have worked with collaborators at Ohio State University to look at data on a whole new scale. Rather than visualizing stellar cores that are tens of kilometers across, the team has developed techniques to visualize microphysics data of the ensemble of atomic nuclei in the stellar core. Custom volume rendering and commercial visualization tools are now available at the physicists' desktop computers.

The researchers may view their data in real time and understand the nuclear structures obtained with new 3D codes. With insightful applications of lighting models and perspective views, the visualizers are opening up new worlds of data interpretation for their colleagues.

A history of violence in the sky

The long-written records of astronomy through the ages are marked by brief apparitions, often called "guest stars." Imagine the shock of ancient observers when the night sky, seemingly constant in its seasonal patterns, suddenly put forth a flare of light that lingered for weeks before fading forever. Some historians believe that such an event - a star exploding in our galaxy - is depicted in the cuneiforms of the Sumerians 10,000 years ago - and may in fact have launched their interest in astronomy and mathematics.

Fig. 13.A blast wave from Supernova 1987A in the Large Magellanic Cloud is overtaking gas ejected long ago by the doomed star.

History's most spectacular recorded supernova occurred in 1054 AD. Chinese astronomers preserved an account of a new star in the constellation Taurus, visible even in daylight. Today, nearly a millennium later, the site is marked by a complex expanding web of debris called the Crab Nebula. At the nebula's heart lies a star that flashes on and off, 30 times each second. This is a rotating neutron star, or pulsar, that emits tight beams of intense radiation.

The next known supernovae in the Milky Way appeared after European scientists started to study the sky in earnest. Danish astronomer Tycho Brahe and German astronomer Johannes Kepler saw and measured the events in 1572 and 1604, respectively. The hot remnants of both explosions are obvious to telescopes and X-ray satellites today. In retrospect, the close timing of the supernovae was a stroke of luck. There hasn't been another one visible in our galaxy since.

But in 1987, skywatchers were treated to the next-best thing: a supernova in one of the Milky Way's companions, a dwarf galaxy called the Large Magellanic Cloud. Neutrino observatories picked up a burst of ghostly particles from "Supernova 1987A," and major telescopes continue to watch its aftermath. By examining previous images of the galaxy, astronomers learned that the doomed star was a blue supergiant about 20 times as massive as our Sun and 100,000 times brighter. And by carefully gauging the explosion's brightness, astronomers verified that the star's death forged huge quantities of heavy elements - primarily a radioactive form of nickel, which decayed to iron and released energy for months.

Today, the blast wave from Supernova 1987A races into space. As it does, it overtakes gaseous material ejected by the unstable star thousands of years ago (see figure 13). The shocks light up this matter in a ring, sparking new outbursts of X-rays and visible light. The celestial fireworks are giving astronomers a direct view of how supernovae impact their surroundings.

The star's destruction may hasten the origins of new stars in the Large Magellanic Cloud. Theorists suspect that supernovae can force clouds of gas and dust to collapse in a process called triggered star formation. This may have sparked our solar system's birth. Some meteorites carry evidence that they contained iron-60, a rare isotope that comes only from supernovae and lasts a few million years before decaying. This metal would have enriched our solar system only if the explosion happened nearby - perhaps much closer than the nearest stars we see today.

TSI scientists also use interactive tools like High-Precision Rendering developed by collaborator Dr Kwan-Liu Ma of the University of California, Davis. These expose the detailed physics happening on all scales throughout the explosion, spanning more than five orders of spatial magnitude. In addition, Scalable Parallel Visualization lets TSI scientists study portions of their data - at full resolution - within the available memory space of the workstations they are using.

TSI researchers can point to several discoveries that visualization has helped to make. One notable find is the Spherical Accretion Shock Instability, or SASI. In essence, simulations suggest that once the rebounding shock wave stalls deep inside the collapsing star, it starts to swirl vigorously (see figure 11). These motions may have profound effects on the shape of the explosion and the fate of the neutron star that the supernova leaves behind.

Surprises inside the supernova core Dr Blondin's team first described the SASI in 2003. According to 2D models developed as part of TSI, the outward-moving shock wave reaches a radius of 100-200 km before it stalls. Matter from the collapsing star streams onto and through this standing shock front, cascading toward the proto-neutron star. But the shock itself behaves like an acoustical cavity, trapping and amplifying sound waves that cannot escape back out past the standing wave. The simulation shows that this process quickly becomes unstable. Within a few hundredths of a second, the shock wave - still stuck above the proto-neutron star - moves violently up and down.

The role of TSI in supernova research
The SciDAC-funded Terascale Supernova Initiative (TSI) grew out of several long-term collaborations among physicists and astrophysicists. Applied mathematicians and computer scientists joined the team to tackle different research areas. Team members share a common philosophy of, and approach to, researching the core-collapse supernova problem. They agree that understanding supernovae is a key priority among the many challenges in astrophysics and cosmology, because exploding stars play such an important role in the cosmic hierarchy.
In particular, supernovae are the dominant source of heavy elements; they give birth to neutron stars and black holes; and they drive the chemical evolution of the galaxy. They are also believed to produce gravitational waves. In one aspect of TSI, researchers predict the waveforms that may be detected by the Laser Interferometer Gravitational-wave Observatory (LIGO) and other facilities. These predictions may help confirm a measurement that would be a threshold in scientific history.
TSI Principal Investigator Dr Mezzacappa notes that a core-collapse supernova could happen in our Milky Way galaxy at any time - perhaps not for 10 years or 30 years, but perhaps tomorrow. If physicists are prepared for the event, they will obtain detailed observations of the neutrinos and gravitational waves from the supernova. Such measurements will yield information from the deepest regions of the exploding star. TSI will provide a three-dimensional multiphysics model of the supernova, allowing physicists to use the event as a laboratory for nuclear and particle physics. This knowledge about the fundamental nuclear and particle physics is critical, because physicists cannot create those extremes of density and composition on Earth.
Moreover, core-collapse supernovae involve just about everything in physics: turbulent fluid flow and instabilities, magnetic fields and rotation, radiation transport in the form of neutrinos and photons, and Einsteinian gravity. In terms of the computing requirements, visualization requirements, and geographic spread of the TSI team, it's a daunting problem for the computational science infrastructure. Similar challenges are faced by other teams working in important areas like combustion modeling, climate modeling, lattice quantum chromodynamics, and fusion science.
Verification of computational results against experiments or observations as well as basic theory is a natural requirement of research programs. The TSI team understands the importance of comparing its computational predictions with real supernovae. A supernova in our galaxy would provide detailed data from neutrinos, gravitational waves, and photons across the electromagnetic spectrum. If the TSI models reproduce those observations across the board, that would be compelling evidence of a valid model.
Astronomers already have dozens of observations of more distant supernovae, containing information on the production of elements and structures in the flows from small scales to large scales. "Ultimately, the true validation is against Mother Nature. That's the bottom line," says Dr Mezzacappa. The team constantly tests its codes, performs parallel simulations within TSI for internal validation, and compares its results with other leading groups. The researchers also run convergence testing, where the virtual grid gets finer and finer with each simulation. If the outcome changes, the model hasn't yet produced a final answer. But if the outcomes are the same, the team has converged on a physical solution, not just a numerical one.
In Dr Mezzacappa's view, SciDAC has allowed physicists to address this problem in all of its complexity for the first time. Until the computers and funding were made available and the teams were assembled, he says, investigators could not have studied this multiphysics problem in earnest.

The scientists knew they would need to demonstrate that the SASI persisted in a 3D simulation. To their surprise, the instability grew. Instead of merely oscillating, the SASI transformed into a spiral shape that whirled around the protoneutron star. This instability amplified as more material plunged inward and added angular momentum (see figure 12 p34), like a child continually spinning a top to make it twirl faster.

The TSI team will continue working to confirm whether the SASI plays a role in driving the stalled blast wave out into the star when the remaining physics is added to the models. Moreover, the spiraling is so pronounced that the SASI may make the inevitable explosion lopsided. Astronomers do observe that supernovae are not spherical. Moreover, young neutron stars often jet through space at speeds of hundreds of kilometers per second. An unbalanced core during the explosion could provide that kick.

In one important consequence, the SASI may be the source of rotational motion of neutron stars. Most neutron stars are born spinning dozens of times per second - a rapid rate for an object approximately 10 km wide. Astronomers spot them as pulsars, which emit beams of radiation that sweep past Earth like lighthouse beacons. If it were confirmed that the SASI sparks that rapid spin, it would solve a long-standing astronomical mystery.

Research must continue. The 3D models still lack some of the detailed physics of the 2D simulations. The V2D code suggests that convection driven by neutrinos - an ingredient not yet included in 3D - is intense in the early stages of the shock rebound. Studies of the convection at higher resolutions than ever before show that it pervades all scales and may even penetrate into the proto-neutron star itself. Convection appears to drag neutron-rich material inwards, feeding the new neutron star in the core. Turbulence between the proto-neutron star and the stalled shock wave may have an effect on the SASI.

Dr Blondin suspects that the spiraling motion will remain the dominant macrophysical process deep inside the nascent supernova. But the TSI physicists will have their answer to this question only when the 3D simulations include the full range of physics that Dr Swesty's 2D model now features. Work to include this physics in 3D models is under way by TSI collaborations between NCSU, ORNL, and Florida Atlantic University (FAU). Work to include magnetic fields in TSI's 2D models is being undertaken by scientists at UCSD, FAU and ORNL.

From gravity to the elements
TSI investigations have an impact far beyond the community of researchers funded by SciDAC. Several TSI scientists have forged collaborations with physicists involved with one of the key searches in science today: the hunt for gravitational waves.

Fig. 14. Major observatories worldwide, such as the twin 10 m Keck Telescopes atop Mauna Kea in Hawaii, continue to study supernovae. Detailed images of many supernovae could help test the latest simulations of how stars explode.

Einstein's general theory of relativity predicts that violent events, such as exploding stars or colliding black holes, will ripple the fabric of space-time like rocks tossed into a pond. By the time they reach Earth, these waves will be extraordinarily small. Still, several gravitational wave observatories in the US, Italy, Germany, and Japan hope to detect the disturbances, which subtly shift the distances between objects in a periodic way.

The experiments rely on matching a precalculated set of "templates," or gravitational waveforms. The templates consist of the exact frequencies, amplitudes, and patterns of waves expected from many combinations of astrophysical events. It's a data-processing challenge of the highest order, similar to that faced by the TSI scientists themselves. To help narrow down the search, the group use the results of their simulations to provide updated templates of possible gravitational wave shapes from supernovae.

Every detail makes a difference; a lopsided supernova would spawn a wave signature distinct from a symmetric one, for instance. Astronomers who observe expanding clouds of matter from supernovae (see figure 14), such as the well-known Crab Nebula and Cassiopeia A remnants, also look to TSI for collaboration. These remnants glow with hot clouds of heavy elements. Recent studies using X-ray satellites have shown that the explosions were far from orderly. In Cassiopeia, for instance, clumps of iron are far more distant from the center of the explosion than oxygen and silicon, as though the star's deepest material was ejected the fastest in an inside-out blast. The results of TSI simulations of convection and turbulence may shed some light on these fascinating astrophysical dynamics.

Beyond the terascale
TSI scientists already have their eye on the next stage in computing: the petascale era, a factor of 1000 more detailed. Computers with petascale capabilities should be available within a few years from now if Moore's law continues apace. Project collaborators agree that such resources will be the only way to run 3D simulations of supernovae that include all essential physical processes, from neutrinos to magnetohydrodynamics to general relativity.

Meanwhile, TSI team members look forward to the continued advances that multidisciplinary research promises to bring. The project's computer scientists, mathematicians, nuclear physicists, and astrophysicists have learned new ways to communicate, to manage a far-flung team and make data compatible, and to allocate precious time and resources. "It's been an eye-opening experience and a tremendous challenge," says one collaborator. "But the combined gains, both from working together as a team and from individual contributions, have been extremely substantial." Enabling these new modes of doing science at a larger scale has been one of SciDAC's great successes.

Further reading

The SciDAC program www.osti.gov/scidac.

The TSI project www.phy.ornl.gov/tsi.

J. M. Blondin 2005 Discovering new dynamics of core-collapse supernova shock waves J. Phys.: Conf. Ser. 16 370-379.

B. Messer et al. 2005 An ADI-Like preconditioner for Boltzmann transport SIAM J. Sci. Comput. 26 810-820.

W. R. Hix et al. 2005 Consequences of nuclear electron capture in core collapse supernovae Phys. Rev. Lett. 91 201102.