Basic Concepts

Franklin, Bassi, and Jacquard are mixed distributed memory and shared memory machines. Each node consists of multiple CPUs (8 on Bassi and 2 on Franklin and Jacquard) that have direct access to a single node's pool of memory. Each node has its own memory and does not have direct access to the memory on other nodes.

Fortran, C, and C++ provide no mechanism for directly accessing or sharing memory that resides on another node. If a program needs data that is stored on another node, the programmer must incorporate subroutines calls from a library of message passing routines to get that data. The most common portable message passing library is called the Message Passing Interface or MPI. MPI is a standard that allows you to send data from one node to another. MPI also allows you to communicate among MPI tasks, whether they are executing on the same node or not.

While a single node does have multiple CPUs, most codes still use MPI calls to transfer data between all processors, even those on the same node. In many cases compilers and operating systems will recognize when MPI is communicating on-node and will substitute faster shared-memory constructs where appropriate.

In the following, well use the term task and process rather interchangeably to refer to the independent, parallel execution of code. You can have a number of MPI tasks less than or equal to the number of CPUs on the node. (If you are writing a hybrid code, e.g. using OpenMP and MPI, each MPI task can have a number of associated threads.)