Shadow Arrays (AKA: Bound Arrays)

It may be useful to have corresponding elements of two or more chare arrays to be located on the same processor. For example, if the corresponding elements of chare arrays X and Y frequently communicate with one another, it would be advantageous to have those elements, X[i] and Y[i], on the same processor to reduce communication costs. Shadow arrays in Charm++ are a way of accomplishing just this type of behavior. When two or more arrays are bound, the Charm++ Runtime System ensures that the objects are always located on the same physical processor.


      // Assumed variable declarations

      int numElements = ...;

      

      // Creation of first array (standard process)

      CProxy_myArrayClass1 myArray1 = CProxy_myArrayClass1::ckNew(...parameters... , numElements);

      

      // Creation of the second array (bound to the first)

      CkArrayOptions ckOptions(numElements);

      opts.bindTo(myArray1);

      CProxy_myArrayClass2 myArray2 = CProxy_myArrayClass2::ckNew(ckOptions);

Figure 1: Code to create two chare arrays, myArray1 and myArray2, which are bound together.

Shadow (or bound) arrays create a relationship in terms of element mapping and load balancing between two or more arrays. The idea is fairly straight forward. First, a chare array is created in the standard manner. Then, a second array is created as a shadow of the first array (or bound to the first array). This is done through the CkArrayOptions parameter to the ckNew call which creates the second array. Additional arrays can also be bound to this set of bound arrays in a similar manner.

There is no restriction to the type or number of chare elements in the arrays. The bound arrays can be instances of the same chare array class or be different chare array classes. Also, the arrays do not have to have the same number of elements. For indexes where the other array doesn't have a corresponding element, the element that exists is free to move between processors arbitrarily. The Charm++ Runtime System simply ensures that when there are corresponding elements (that is, elements at corresponding indexes), in a set of bound arrays, the elements will be located on the same physical processor.

Since the corresponding objects are always located on the same processor, the objects can take advantage of this in various ways. For example, they can obtain local pointers to one another via a call to ckLocal() on the proxy for the element. However, each time the objects migrate (i.e. the objects are unpacked via the Pack-UnPack Routines), the local pointer needs to be refreshed so it is a valid pointer on the new physical processor.

For more information on callbacks, please see Section 3.8.6: Advanced Array Creation: Bound Arrays of the The Charm++ Programming Language Manual