Comm World handles communication between all the nodes (single computers, or
processing units, if you are using multiple cores per machine). So Each node
can talk to any other node, but ony via Comm World.
When we send a message, we send it to Comm World.
When we receive it, we receive not from the source node, but from Comm
World, in its buffer that it reserved for the origin node.
Reduction - 1
Reduction works much like it does in OpenMP, but with a few significant
That is to say, the result is the same, but the means by which the result is
acheived is substantially different, and would be even if we used MPI only on
a single machine in entirely paralell mode.
Reduction - 2
We do need to talk about the reduce function parameters a little
int MPI_Reduce(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype,
MPI_Op op, int root, MPI_Comm comm)