---

---

III. HPCC Research centers and Facilities

Computing, Information, and Communications Research Facilities

These facilities provide capabilities to (1) evaluate early prototype systems and provide valuable feedback to developers, (2) integrate visualization and virtual reality systems into existing high performance systems, (3) run full scale applications, such as Grand Challenges, on systems not otherwise available, and (4) develop parallel software using scaled down systems. In addition, these facilities provide access to innovations in network connectivity that allow large scale applications to run over remote connections to systems at HPCC facilities across the country, thus demonstrating future directions for advanced R&D in academia, industry, and the Federal government.

Researchers at these centers rely on many enabling technologies - high speed networks, supercomputers, parallel architectures, massive data stores, virtual reality display devices - in order to succeed. Many groups in addition to the researchers contribute to this success, among them facility staff, hardware and software vendors, and industrial affiliates. HPCC funding is thus leveraged heavily through equipment and personnel from vendors, discipline-specific agency funds, as well as state and local funds, and industrial affiliate contributions. Industrial affiliation offers a low risk environment for exploring and ultimately exploiting HPCC technology.

Applications software developers from the centers access their resources over the Internet. The wide range of hardware and applications software that is available also makes these centers ideal sites for benchmarking systems and applications and for providing feedback to hardware and software vendors.

All facilities provide extensive K-12 and undergraduate educational opportunities as well as training for researchers, graduate students, and faculty; they also provide for publication of articles in professional journals, annual reports, and newsletters.

The systems listed below are funded by the named agency and receive additional funding from other HPCC agencies. For example, funding for systems at NSF centers also comes from DARPA, NASA, and NIH.

NSF Supercomputer centers

NSF funds four Supercomputer centers, augments the computing facilities at NCAR (the National center for Atmospheric Research), and funds Metacenter activities. The term Metacenter refers to the joint cooperative activities of these centers and others in naturally overlapping research and technology areas. A Metacenter facilitates collaboration, communication, technical progress, and interoperability among participating institutions. Metacenter Regional Alliances (MRAs), which were set up to augment national support activities, are also intended to complement, expand, and strengthen existing Metacenter activities at the regional, state, or local level.

The primary computing resources at CTC are a 512 processor IBM SP-2, an SGI Power Onyx Array, consisting of two 8 processor Power Onyx systems, and a 16 processor SGI Power Challenge. One CTC focus area is a globally scalable computing environment, including mass storage, I/O capability, networking, archival storage, data processing, and graphics power.

Resources include:

	Convex C-3880 with 8 processors and 4 GB memory
	Convex Exemplar with 64 processors and 8 GB memory
	Silicon Graphics Power Challenge with 16 processors and 4 GB memory
	Thinking Machines CM-5 with 512 compute nodes and 16 GB memory

The three-tiered NCSA network consists of (1) Ethernet or FDDI to the desktop; (2) FDDI backbone between buildings, high end systems, and the Internet; and (3) HiPPI between high performance computing systems, mass storage, and high end peripherals.

NCSA is also involved in ATM research in (1) a local area network, (2) a trans-continental 155 Mb/s (SONET OC-3) national network, and (3) the BLANCA gigabit testbed at 622 Mb/s (SONET OC-12).

Resources include:

	The first single-vendor heterogeneous system consisting of a Cray Research T3D (with 512 processors, each with 64 MB of memory) coupled to a C90 (with 16 processors and 4 GB of memory)
	14-processor DEC Alpha workstation cluster
	HiPPI and FDDI network connecting these resources to each other and to storage devices

Resources include:

	Cray Research C90 with 8 vector processors
	Intel Paragon with 400 processors
	Thinking Machines CM-2 with 8,192 processors
	Eight workstation DEC Alpha cluster

NSF HPCC funds provided partial support for the acquisition of a 64 processor Cray Research T3D and an 8 processor IBM SP-1 for use in the global climate modeling Grand Challenge.

NSF Science and Technology centers

Each of these four centers addresses a particular research area. Common to all four is cross-disciplinary focus, knowledge transfer, links to the private sector, and education and outreach. The centers are:

CRPC's aim is to make parallel computing systems as easy to use as conventional computing systems - efforts include HPF, PVM, MPI, and NHSE, HPC++, algorithms for physical simulation, algorithms using parallel optimization, and ScaLAPACK.

This center is building and displaying models that are visually and measurably indistinguishable from real world entities.

This center is applying discrete mathematics and theoretical computer science to solving fundamental problems in science and engineering.

This center studies the human mind through the interaction of disciplines such as psychology, philosophy, linguistics, logic, and computer science. Work in human cognition, perception, natural language processing, and parallel computing has applications in robotic and manufacturing systems, human machine interfaces, and language teaching and translational tools.

NASA Testbeds

NASA maintains testbeds throughout the country to offer diversity in configuration and capability. The testbeds are:

Resources include:

	IBM SP-2 with 160 processors and 20 GB memory
	Intel Paragon with 208 compute nodes, 16 service nodes, and 7 GB memory
	Thinking Machines CM-5 with 128 compute nodes - each consisting of a SPARC processor and 4 vector processors and 4 GB memory, which is also used by the Naval Research Laboratory

Resources include:

Convex SPP-1 with 8 processors

MasPar MP-2 with 16,384 processors and 1 GB memory, SIMD (Single Instruction Multiple Data); MP-1 with 8,192 processors and 512 MB memory; 2 MP-1's with 4,096 processors and 256 MB memory. These four systems have all been connected by a 4-by-4 HiPPI switch. MasPar applications are being modified to take advantage of the HiPPI network in order to distribute the workload across the four MasPar systems. This cluster demonstrates combining SIMD and MIMD (Multiple Instruction Multiple Data) programming styles to enable MasPar to move beyond its current 16,384 processor ceiling.

Resources include:

	Cray Research T3D with 256 processors and 16 GB memory
	Intel Delta with 528 processors at Caltech
	Intel Paragon with 56 compute nodes, and a total of 1.8 GB of memory

Resources include an Intel Paragon with 72 compute nodes.

Resources include an IBM SP-1 with 16 processors.

These include:

	IBM RS6000 workstation cluster with 32 nodes and 3 GB memory, that is part of a 128 node IBM SP-1 consortium at Argonne National Laboratory
	Kendall Square Research KSR-1 with 56 nodes and 1.8 GB memory, at the University of Washington

DOE Laboratories

The Supercomputer Access Program at NERSC provides production computing for investigators supported by the Office of Energy Research in the following areas: material sciences, chemistry, geosciences, biosciences, engineering, health and environmental research, high energy and nuclear physics, fusion energy, and applied mathematics and computational science.

NERSC resources include:

	Cray Computer Cray-2 with 8 processors and 128 megawords (millions of 64-bit words (Mw) memory)
	Cray Computer Cray-2 with 4 processors and 128 Mw memory
	Cray Research C90 with 16 processors and 256 Mw memory
	The National Education Supercomputer, a four processor Cray Research X-MP EL provided by Cray Research and available to high schools over the Internet

These DOE HPCC Research centers provide full scale high performance computing systems for work on Grand Challenge applications and use in scalability studies. These applications require large prototype systems - they cannot be scaled down without removing essential aspects of their physics.

LANL operates a Thinking Machines CM-5 with 1,024 compute nodes and 32 GB memory.

ORNL resources include:

	Intel Paragon XP/S 150 with 1,024 MP nodes (3,072 processors) and a total of 70 GB of memory
	Intel Paragon XP/S 35 with 512 GP nodes (1024 processors) and a total of 16 GB of memory
	Intel Paragon XP/S 14 with 96 MP nodes and a total of 8 GB of memory
	Kendall Square KSR 1 with 64 processors and a total of 2 GB of memory, used to study shared memory algorithms
	nCube2 with 8 processors and a total of 256 MB of memory, used by high school students in the Adventures in Supercomputing program

NIH Systems

The Division of Computer Research and Technology (DCRT) has a 56 processor IBM SP-2 and a 128 processor Intel iPSC/860. Both systems are used by NIH staff in biomedical applications.

The National Cancer Institute's (NCI) Frederick Biomedical Supercomputing center has an 8 processor Cray Y-MP and a MasPar MP-2 with 4,096 processors along with a comprehensive collection of biomedical software available to all scientists who use the facility.

The National center for Research Resources (NCRR) supports various systems for biomedical research applications at its six High Performance Computing Resources centers:

	Resource for Concurrent Biological Computing, Beckman Institute, University of Illinois
	Supercomputing for Biomedical Research, Pittsburgh Supercomputing center
	Theoretical Simulation of Biological Systems, Columbia University
	Parallel Computing Resource for Structural Biology, University of North Carolina, Chapel Hill
	Biomedical Computation Resource, University of California, San Diego
	Parallel Processing Resource for Biomedical Scientists, Cornell Theory center, Cornell University

and two Scientific Visualization Resource centers:

	Interactive Graphics for Molecular Studies, University of North Carolina, Chapel Hill
	Special Research Resource for Biomolecular Graphics, University of California, San Francisco.

NOAA Laboratories

The Forecast Systems Laboratory in Boulder, CO, has a 221 processor Intel Paragon, with 6.5 GB memory. This system is used to parallelize regional and mesoscale forecast models.

The Geophysical Fluid Dynamics Laboratory (GFDL) in Princeton, NJ, has acquired a Cray T-90 (PVP) and a T3E (SPP) and the National centers for Environmental Prediction in Camp Springs, MD, will acquire a scalable computing system in FY 1997. These systems are used for the global climate modeling and weather forecasting Grand Challenges.

EPA Systems

EPA's National Environmental Supercomputing center in Bay City, MI, has a Cray C-94 with 3 processors and 64 Mw memory, and a Cray T3D with 128 processing elements and 8 GB of memory. These systems are dedicated to environmental research, problem solving and related educational programs.