Motivation

The Paradigms for Parallel Computation Project is performing a three phase evaluation of emerging ideas to boost productivity for programmers of high performance computing systems, and in particular to determine which of these ideas could have near-term impact on DoD relevant applications and systems.

The goal of the first phase was to survey the broad set of ideas from reality, determine realistically the readiness of these ideas, and produce a smaller set of programming models upon which to perform a more complete evaluation. The programming models which have been selected, and the languages that represent those models are:

• Partitioned Global Address Space (Unified Parallel C)
• One-sided Communication (shmem)
• Distributed Shared Memory (Cluster OpenMP)
• Fine-grained lightweight tasks (CHARM++)
• Service-Oriented (Publish-Subscribe Query, 100K JBI)
• Concurrent Objects (SCOOP – Phase 3 only)
• Higher Level Language (Matlab +StarP)

Background

For the last decade, high performance computing has been dominated by distributed memory parallel architectures. The most common model for programming these systems has been message passing, specifically as implemented in MPI. The OpenMP standard has been defined for threading in shared memory architectures.

MPI codes have been used to set each new record in performance, including breaking the teraflop and hundred-teraflop barriers. The state of affairs at this time is that codes written in MPI will continue to scale, but the range of applications capable of achieving this scale will remain small, and the effort required to maintain scaling will continue to increase. The real barrier to scalability seems to be the productivity of parallel programmers as the system complexity they are faced with increases. While MPI can exceed the performance of all other techniques, an heroic programming effort is required to achieve this performance. An effort of many years is typical for truly scalable complex scientific applications.

Two technology trends are currently exacerbating this problem. First, the emergence of multi-core microprocessors has led to the development of MPI-OpenMP hybrid codes, using OpenMP for the shared memory within the 8-16 (soon to be more) processors of a node, and using MPI for the distributed memory between nodes. Second, the sheer scale of modern HPC systems, with thousands of nodes and processor cores becoming more common means the need to incorporate fault-tolerance into applications is rapidly growing. In much the same way as high level languages replaced assembly language by trading off some performance for greatly increased programmer productivity, there is now a growing need for a similar move to a different level of abstraction in HPC programming.

A number of different paradigms have been proposed to create this higher level of abstraction. Perhaps the most widely investigated are the PGAS (Partitioned Global Address Space) languages, which include Co-Array FORTRAN (CAF), Unified Parallel C (UPC), and Titanium (a java-based variant). These languages, while still relatively young, are developed to the point where experimental applications can be built, and the CAF extensions are being included in the next FORTRAN standard. Another set of new languages is being developed as part of the DARPA High Productivity Computing Systems (HPCS) effort, which includes X10 (under development by IBM), Chapel (Cray), and Fortress (SUN). The HPCS languages, at this point, can only be evaluated on paper. All of these languages strive to hide the complexity of explicit message passing, though the basic concepts that must be dealt with remain the same: managing where data resides, communicating non-local data, and synchronization. In addition to these relatively new ideas, there are several older ideas that have co-existed with MPI pushing for additional prominence, such as the Global Arrays (GA) toolkit supported by the DOE and the one-sided communication library shmem supported by Cray and SGI. A number of other smaller academic research efforts are also proposing parallel programming models. Among them are CHARM++ and SCOOP. Several commercial technologies are also in development, such as the PSQ (Publish-Subscribe-Query) model, and extensions to Matlab such as Star-P.