Running on Bassi

Communication protocols

Most parallel jobs that run on Bassi do so through an explicit message passing interfaces. The choice of API and protocols must be specified at run time, by setting environment variables for interactive jobs and through a network LoadLeveler keyword for batch jobs (see below and elsewhere on these pages.)

Message Passing APIs

There are two explicit message passing interfaces, or APIs, available. Calls to these libraries are made in source code and a run-time library containing these functions is loaded when the job executes.

MPI

Most codes use MPI, the popular Message Passing Interface. The default programming and interactive run-time environment is configured to support MPI. See the IBM Documentation for details of the IBM implementation of MPI.

Most MPI batch job scripts need to include this line:

#@network.MPI = sn_all,not_shared,us

LAPI

The low-level IBM LAPI interface is also available. See the IBM Documentation for details.

Most LAPI batch job scripts need to include this line:

#@network.LAPI = sn_all,not_shared,us

Interactive jobs that use LAPI must set an environment variable thusly:

bassi% export MP_MSG_API=LAPI   (sh-like shells)
bassi% setenv MP_MSG_API LAPI   (csh-like shells)

Communications Protocols

There are two separate implementations of the communications library that actually ships data packets across the network:

Internet Protocol (IP)

This implementation uses the standard IP protocol that commonly links computers over ethernet networks. IP offers less performance that User Space (see below), but some specialized application may require IP.

Batch job scripts that require IP need to include one of these lines:

#@network.MPI = sn_all,not_shared,ip
#@network.LAPI = sn_all,not_shared,ip

Interactive jobs that use IP must set an environment variable thusly:

bassi% export MP_EUILIB=ip (sh-like shells)
bassi% setenv MP_EUILIB ip (csh-like shells)

User Space (US) Communication Subsystem

This communication subsystem is designed to take advantage of Bassi's high performance switch (HPS).

You will use the "US" protocol in almost all cases. The library is dynamically linked to your code at runtime. You do not have to modify your source code, but you must specify a protocol when you run your program.

For MPI batch jobs using US, the follow line must appear in the batch script:

#@network.MPI = sn_all,not_shared,us