Cardiac

A High Performance Computing Cluster for the Heart

Joseph Hargitai

Cardiac, a high-tech supercomputer, is the newest addition to NYU's High Performance Computing (HPC) Center. Suggested by Professor Charles S. Peskin (Dept. of Mathematics, CIMS), the name "Cardiac" is both a playful tribute to ENIAC,¹ the famed supercomputer of the early ages of computing, and a reference to the new system's principal research application -- the study of the heart.

Cardiac will be used by Professor Peskin, along with NYU researchers Boyce Griffith and David McQueen, as a vital tool in their effort to construct a functional computer model of the beating human heart. "This model," writes Professor Peskin, "includes representations of the electrical system that coordinates and controls the heartbeat, the fiber architecture of the muscular heart walls and the delicate heart valve leaflets, and the fluid mechanics of blood in the cardiac chambers. The model formulation employs the immersed boundary (IB) method as a unified mathematical and computational framework for this coupled electromechanical and fluid-structure interaction problem."

Applications of such a virtual heart are considerable. Computer-assisted design of prosthetic cardiac valves, computational optimization of cardiac resynchronization therapy, as well as computer simulation of diseases affecting the heart and its valves, with the goal of improving diagnosis and treatment, are among the areas of research that would benefit from work being done on the Cardiac supercomputer. In addition, "long range goals of the project include the development of patient specific virtual heart models for the individualization of patient care, and the computer simulation of cardiac remodeling as it occurs during the embryonic development of the heart, in disease states, and also in response to therapy."²

Cardiac's three-rack system offers a computational density that was impossible until recently. It is built around a Sun Microsystems, Inc. 8000p Blade Center, with 80 Sun Microsystem X8440 Blade Servers interconnected by Voltaire, Inc.'s low-latency Infiniband switching technology. With a total of 1,280 computational cores, each of the Sun Blade servers employs four CPU sockets populated with Barcelona 8356 processors from Advanced Micro Devices (AMD), Inc. These quad-core processors run at 2.3GHz clock speed and offer four floating points per cycle.

The acquisition of Cardiac, which was made possible by a generous grant from St. Jude Medical to Dr. Larry Chinitz, head of Cardiac Electrophysiology, and Dr. Glenn I. Fishman, Director of the Division of Cardiology at the NYU Langone Medical Center, along with a smaller contribution from ITS, represents a three-way partnership among Sun, Voltaire, and NYU. The CIMS researchers and Dr. Fishman have been collaborating for several years on studies involving mathematical and computer modeling of cardiac electrophysiology. As ITS Executive Director for .edu Services, David Ackerman, points out, "Collaborations and cross-disciplinary partnerships provide ongoing opportunities for NYU researchers to utilize the latest and most powerful technologies available for research -- technologies they may not be able to afford on their own."

"We are delighted to have been selected by NYU to contribute to their important research in the area of cardiovascular disease," notes Asaf Somekh, Vice President of Strategic Alliances at Voltaire. "The Voltaire Grid Director 20 gigabit-per-second switch increases the performance of the computational fluid dynamics applications running on NYU's system, enabling researchers to build more detailed, higher-resolution heart models resulting in useful predictions and accelerated time to discovery."

Above, the flow pattern from one of the model heart computations. In the figure, blood on the left side of the heart (oxygen-enriched) is colored red, blood on the right side of the heart (oxygen-depleted) is colored blue. The blood markers have small tails which show their recent positions, as an aid to visualizing the flow.

In the heart model itself, muscle fibers are shown in a light brown, valve structures are shown in white.

The figure is actually a thin cross-section. Because of the positions of the heart chambers relative to each other and the plane of the cross-section, the figure emphasizes the flow pattern on the left side of the heart.

Heart animations computed by the immersed boundary method can be found at math.nyu.edu/faculty/peskin/myo3D/index.html.

While Voltaire's partnership with NYU/ITS is relatively new, Sun's contribution to and collaboration with NYU spans nearly two decades. "Sun appreciates the opportunity to support NYU's genome, dynamic modeling, and other research efforts with our HPC technology," writes Chief Sun Engineer, Dr. Hung-Sheng Tsao. "It is gratifying to know that this system can potentially provide Professor Peskin and other NYU researchers the tools necessary to provide life-saving cardiovascular and related solutions to the medical field."

Standard applications available on Cardiac include MATLAB, Mathematica, Tecplot, R, Octave, Scipy, Numpy, NETCDF, HDF4, and HDF5. ITS has added to Cardiac's toolbox Totalview, a best-of-breed parallel profiler and debugger, as well as the Compiler Edition of Intel® Cluster Toolkit and Portland Group's Compiler Suite, to address the computational density issue typical of massively multi-core systems.

While Professor Peskin's group will be the principal users of Cardiac, the system will also be available to other research groups on a limited basis. If you are a researcher or advanced student engaged in studies with heavy computational requirements, contact the ITS High Performance Computing support staff at hpc@nyu.edu for further information, access, and help.

At the heart of Cardiac is the Voltaire Grid Director(tm) 2004 switch with 96 ports of 20 Gigabits/sec InfiniBand connectivity providing high bandwidth and I/O throughput between the nodes and very low latency to speed performance of the cluster's applications.

The Sun Blade Center 8000p chassis packs ten blades, two management modules, one Ethernet module, and one Infiniband module in each chassis unit. Theoretical performance of the cluster is estimated at around 12 teraflops.

FOOTNOTES

http://en.wikipedia.org/wiki/ENIAC
Personal communication, Sept. 21, 2008.

Author Biographies

Joseph Hargitai is a Faculty Technology Specialist who works within ITS' High Performance Computing group.