Parallel Numerics an der TU München

Barrier, lock and semafore

Allows to receive arbitrary messages, but makes the reception order of the messages also arbitrary.

Standard MPI operations are not synchronized: There is no handshake nor guarantee that they will ever arrive

Blocking: Faster access to data due to data locality

Cyclic: Better load balancing

Shared memory: The processors share a common address space. They communicate by reading and writing to it.

Distributed memory: Each processor has a private address space. They communicate by messaging.

Hybrid: The nodes have a shared address space, but they communicate amongst them by messaging.

It's a send that guarantees synchronous semantics (As long as the matching receive is blocking). This means that it won't return until the matching receive has been posted.

It performs a send operation if matching receive has been posted, otherwise errors. It returns as soon as the buffer is available.

Can be decomposed in parallel with virtually no need to share data

Distribution of the work with the goal of keeping the processors busy at all times

When two processors wait indefinitely for the results of each other

Non-deterministic result of a program depending on the chronology of tasks in a shared memory machine

All tasks of a communicator perform work until reaching the barrier. Then, they wait until the last task of the communicator reaches the barrier and then move on