Mapping Chare Array Elements to Physical Processors

For various reasons (performance, etc.) the programmer may wish to control where the initial chare array elements in a chare array reside (that is, on which physical processor they are located). There are multiple methods the programmer can use to control the initial placement of chare array elements in a chare array on the physical processors.

Inserting Individual Chare Array Elements

The most direct method of controlling which processors the chare array elements are located on initially is to explicitly create the chare array element one-by-one, specifying with processor it should be located on during creation. This can be done by creating the chare array with a call to myArrayProxy = CProxy_myArrayClass::CkNew() with no arguments (or the number of elements set to zero if a CkArrayOptions object is used). Once the chare array has been created, individual chare array elements can be inserted by using the myArrayProxy[i].insert(pe) function call where i is the index of the chare array element to create and pe is the processor number on which the chare array element should be created on. Once all chare array elements have been added to the array, the myArrayProxy::doneInserting() call must be called to indicate to the Charm++ Runtime System that the initial elements of the chare array have been added. Additional chare array elements can be added at a later time followed by an additional call to myArrayProxy::doneInserting().

Using the CkArrayMap Class

Chare arrays also have an associated CkArrayMap class which maps indexes in the arrays to specific processors. By default, a round-robin (RRMap) map object is used. When the chare array is being created, the programmer can specify which map object the array should use via a call to CkArrayOptions::setMap(). For example, instead of round-robin, the programmer might use the BlockMap map class to have the objects mapped in a blocked scheme.

The programmer may create their own map object to be used for a chare array. Simply create a child class of the CkArrayMap class. The important member functions are outlined below. Then, create an instance of the class and associate the map object with the chare array (via CkArrayOptions) during array creation time.

CkArrayMap::populateInitial(): This function is responsible for creating all of the initial array elements located on the physical processor. That is, it is called on every processor and makes a call to the chare array manager's insertInitial() function for each of the chare array elements local to that processor (triggering the creation of the actual chare array elements on the physical processor). This is a virtual function; the child class can provide its own implementation of this function. However, one is provided by CkArrayMap itself which simply uses the CkArrayMap::procNum() function to decide where the chare array elements initially reside.

CkArrayMap::procNum(): This function simply maps an array index (input) to the number of the physical processor (output). For example, the round-robin map class's (RRMap) version of this function simply returns 'index % CkNumPes()'. This function is a pure virtual member function. The child class must provide an implementation of this function.

CkArrayMap::homePe(): This function is similar to the procNum() function, however, the value returned indicates which physical processor is the home processor for the chare array element. This is a virtual function, the child class can provide its own implementation of this function. However, one is provided by CkArrayMap itself which simply returns the same value that procNum() returns.