This section describes two new distribution formats. The syntax is extended as follows:
H810 extended-dist-format is BLOCK [ ( int-expr ) ]
or CYCLIC [ ( int-expr ) ]
or GEN_BLOCK ( int-array )
or INDIRECT ( int-array )
The ``generalized'' block distribution, GEN_BLOCK, allows contiguous segments of an array, of possibly unequal sizes, to be mapped onto processors. The sizes of the segments are specified by values of a user-defined integer mapping array, one value per target processor of the mapping. That is, the ith element of the mapping array specifies the size of the block to be stored on the ith processor of the target processor arrangement. Thus, the values of the mapping arrays are restricted to be non-negative numbers and their sum must be greater than or equal to the extent of the corresponding dimension the array being distributed.
The mapping array has to be a restricted expression when used in the DISTRIBUTE directive, but can be an array variable in a REDISTRIBUTE directive. In the latter case, changing the value of the map array after the directive has been executed will not change the mapping of the distributed array.
Let l and u be the lower and upper bounds of the dimension of the distributee, MAP be the mapping array and let BS(i):BE(i) be the resultant elements mapped to the ith processor in the corresponding dimension of the target processor arrangements. Then,
|BS(i)||=||min(BS(i) + MAP(i) - 1, u),|
|BS(i)||=||BE(i - 1) + 1.|
PARAMETER (S = /2,25,20,0,8,65/) !HPF$ PROCESSORS P(6) REAL A(100), B(200), new(6) !HPF$ DISTRIBUTE A( GEN_BLOCK( S) ) ONTO P !HPF$ DYNAMIC B ... new = ... !HPF$ REDISTRIBUTE ( B( GEN_BLOCK(new) )
Given the above specification, array elements A(1:2) are mapped on P(1), A(3:27) are mapped on P(2), A(28:47) are mapped on P(3), no elements are mapped on P(4), A(48:55) are mapped on P(5), and A(56:100) are mapped on P(6). The array B is distributed based on the array new whose values are computed at runtime.
For dynamic arrays, an independent copy of the mapping array will have to be maintained internally so that a change in the values of the mapping array does not affect the access of the distributed array. (End of advice to implementors.)
There are many scientific applications in which the structure of the underlying domain is such that it does not map directly onto Fortran data structures. For example, in many CFD applications an unstructured mesh (consisting of triangles in 2D or tetrahedra in 3D) is used to represent the underlying domain. The nodes of such a mesh are generally represented by a one-dimensional array while another is used to represent their interconnections. Mapping such arrays using the structured distribution mechanisms, BLOCK and CYCLIC, results in mappings in which unrelated elements are mapped onto the same processor. This in turn leads to massive amounts of unnecessary communication. What is required is a mechanism to map a related set of arbitrary array elements onto the same processor. The INDIRECT distribution provides such a mechanism.
The INDIRECT distribution allows a many-to-one mapping of elements of a dimension of a data array to a dimension of the target processor arrangement. An integer array is used to specify the target processor of each individual element of the array dimension being distributed. That is, the ith element of the mapping array provides the processor number onto which the ith array element is to be mapped. Since the mapping array maps array elements onto processor elements, the extent of the mapping array must match the extent of the dimension of the array it is distributing. Also, the values of the mapping array must lie between the lower and upper bound of the target dimension of the processor arrangement.
The mapping array has to be a restricted expression when used in the DISTRIBUTE directive, but can be an array variable in a REDISTRIBUTE directive. In the latter case, changing the value of the mapping array after the directive has been executed will not change the mapping of the distributed array.
Here, the array A is distributed statically using the constant array map1. Thus:!HPF$ PROCESSORS P(4) REAL A(100), B(50) INTEGER map1(100), map2(50) PARAMETER (map1 = /1,3,4,3,3,2,1,4, ..../) !HPF$ DYNAMIC B !HPF$ DISTRIBUTE A( INDIRECT(map1) ) ONTO P !HPF$ DISTRIBUTE map2(BLOCK) ONTO P map2 = ... !HPF$ DISTRIBUTE B( INDIRECT(map2) ) ONTO P ....
A(1) is mapped onto P(1)
A(2) is mapped onto P(3)
A(3) is mapped onto P(4)
A(4) is mapped onto P(3)
A(5) is mapped onto P(3)
A(6) is mapped onto P(2)
A(7) is mapped onto P(1)
A(5) is mapped onto P(4)
The array B is declared dynamic and is redistributed using the mapping array map2.