hpf-builder: a visual environment to transform fortran 90 ... - CiteSeerX

HPF-BUILDER: A VISUAL ENVIRONMENT TO TRANSFORM FORTRAN 90 CODES TO HPF JEAN-LUC DEKEYSER AND CHRISTIAN LEFEBVRE [email protected], [email protected] 

Abstract. FORTRAN 90 is the actual standard in term of data parallel language for scienti c computing. To develop a data parallel algorithm on a distributed memory machine, programmers generally use the High Performance FORTRAN extension of FORTRAN 90, in particular the data mapping directives. HPF-Builder graphical environment goal is to free the HPF programmers of all the syntactic constraints due to the data mapping. All the data distribution and alignment are insured in an interactive and visual way. HPF TEMPLATES and PROCESSORS become the visual support for alignments of arrays and distributions on grids of processors. HPF-Builder automatically generates the corresponding HPF directives and inserts them in the FORTRAN 90 source code. It results in a HPF code.

Key words. Visual programming, data mapping, HPF, distribution, alignment

1. Introduction. With the emergence of parallel and massively parallel machines, where the memory is physically distributed on a large number of processors, new parallel programming techniques have appeared. In order to preserve the knowledge base, a programming model similar to the vector model was proposed: the data parallel model. Here vectors or matrices are distributed across all the processors. Parallel operations are processed simultaneously by the processors. Here again, regular structure handling, involving only regular communication diagrams between processors, has bene ted from the growing power of these new machines. The expression of parallelism at the data level has the advantage of maintaining  Laboratoire d'Informatique Fondamentale de Lille, Universit e des Sciences et Technologies de Lille, Batiment M3, Cite Scienti que, 59655 Villeneuve d'Ascq Cedex , France 1

2

J-L. DEKEYSER AND C. LEFEBVRE

a single control ow. A data parallel algorithm consists of a sequence of elementary instructions applied to scalar or parallel data; an instruction is initiated only if the previous instruction has ended. In the case of data parallel operands, this is the case when all elementary data have been processed. This strong constraint of sequentiallity does not have to be implemented by parallel compilers, provided that the global semantics of the program is guaranteed. Data parallel extensions of FORTRAN 90 promote arrays to rst class citizens of the language and propose restructuring operations for these parallel entities (gather, scatter). Compilation for distributed memory machines relies on the notion of data distribution by the use of mapping directives. These directives specify sets of elementary data that should be allocated on the same processor. HPF (High Performance FORTRAN) [8, 7] is an example of this approach and seems to be becoming the most popular language for data parallel scienti c programming. A distributed data parallel algorithm designer usually starts from a FORTRAN 90 code and inserts HPF directives respecting the HPF syntactic rules. The FORTRAN 90 parts express the data parallel algorithm itself and the HPF directives ensure the mapping of the data without semantic contribution. The eects of these directives are essential in balancing the parallel processing and communications. The programmer has to insert by hand all these mapping directives. Therefore the scienti c programmer must learn a third generation dialect of FORTRAN to take advantage of parallel machines. Like FORTRAN 90, HPF supports regular data structures (multi-dimensional arrays). Furthermore, HPF provides a geometrical support to express the distribution of data among the processors (grid of processors). The expression of parallelism at the data level allows the programmer to have a visual perception of the distribution

3

HPF-Builder environment

of data in space (at least for 1, 2 and 3-dimensional arrays and grids). Often programmers use papers and colour pencils to draw and improve their mapping before translating the drawing to HPF directives. HPF-Builder is a tool to help the programmer at this level. It proposes to replace the paper and pencils by a screen and mouse. Then it automatically generates the HPF directives from the drawing. HPF-Builder graphical environment frees the programmer from all the syntactic constraints due to the data mapping. HPF-builder respects a hierarchical HPF programming model. For each level, HPF-Builder provides a graphical interactive editor. In a WYSIWYG way each editor is able to generate the appropriate directives according to the data manipulation of the programmer. We use a step by step transformation of FORTRAN 90 examples to show the main features of HPF-builder.

2. The hierarchical HPF programming model. A good use of HPF directives respects a three level hierarchical approach (cf. gure 2.1). Arrays

ALIGN REALIGN

Group of aligned object TEMPLATE

DISTRIBUTE REDISTRIBUTE

Abstract processors (user declared cartesian mesh) PROCESSORS

(Implementation dependent directive)

Physical processors

Fig. 2.1. Hierarchical HPF programming model

4

J-L. DEKEYSER AND C. LEFEBVRE

The rst level consists of the processors. It describes a multi-dimensional grid of virtual processors. The second level is the template, with all the templates distributed onto a processor grid. The third level is composed of the arrays, aligned with the templates. This construction ensures a progressive re nement of the data mapping on the physical processors. In this way the programmer is able to group in the same template all the arrays that interact. It avoids a number of levels due to array with array alignments. This three level hierarchy forms the skeleton of HPF-builder.

3. Visual programming in HPF. To replace papers and pencils, a graphical editor has to provide several features : - the graphical editing of mapping directives, - a global view of the hierarchical construction, - the automatic generation of HPF directives, - a WYSIWYG tool to visualize and to modify existing HPF directives, - the interpretation of directives to help the programmer (array partitioning on virtual processors, evaluation of the redistribution and realignment cost in term of communications...). There are few tools to help the HPF programmers. Most of them are limited to visualization, they do not help with directive editing.

3.1. GDDT (Graphical Data Distribution Tool). GDDT [6] supports visualization: from an existing HPF source, this tool allows the visualization of the array distributions and the extraction of some statistics on the processor load and communications.

3.2. Annai/DDV (CSSE/NEC). Annai[4] is a Distributed Data Visualizer (DDV). This environment supports HPF as well as MPI codes. Four features have

HPF-Builder environment

5

been developed:

 User Interface.  Parallelisation Support Tool (PST). PST is an extension of HPF to support irregular mapping.

 Parallel Debugging Tool (PDT). PDT [3] is a parallel debugger for PST, HPF, FORTRAN / C + MPI. This tool visualizes the data parallel distributions from the debugger. Annai allows the user to visualize array or section of array distributions (1D or 2D). Each processor is associated with a colour. Distributions are visualized using these colors.

 Performance Monitor & Analyser (PMA). PMA visualizes the performances. (Statistics, load balancing...)

3.3. DAQV. DAQV [9] (Distributed Array Query and Visualization) is a PTOOLS product. This tool provides a set of routines to visualize arrays and distributions. This approach requires the programmer to insert calls in the source code before the compilation.

3.4. HELP-DRAW. HELP-DRAW [1] is a graphical interface that uses HELP programming model [5]. Parallel data objects are positioned within the hyper-space. During the data parallel processing, the semantics of virtual processors are respected (\a la C*"). HELP-DRAW proposes a visual programming environment according to this model. A visual mapping of the data associated with a visual description of the data parallel algorithms allows the corresponding HPF code to be generated.

6

J-L. DEKEYSER AND C. LEFEBVRE

4. HPF-Builder. HPF-Builder is built according to the HPF programming model. For each level of the data mapping hierarchical representation a graphical editor is de ned to visualize and modify in a WYSIWYG way the corresponding HPF directives (cf. gure 4.1). This procures a step by step transformation from a FORTRAN 90 code towards an HPF version. The data parallel algorithm expressed in FORTRAN 90 is never modi ed. The HPF transformation is concerned exclusively with the data mapping. program example integer i, j, k

128

real Vector(100) real Matrix(100,100) real 3dmat(100,50,100)

128

...

From Fortran 90

Data−Parallel Scientific Computing

Automatic HPF code generation

Graphical Interactive Environment

Templates & Processors

Alignments

1000 = 104*8*1 + 24*7*1

Distributions

Immediate HPF Directive Generation

HPF Skeleton

1000 = 125*8 + 3*0

Portability (C, C++, Tcl/Tk)

To HPF Code program example !HPF$ PROCESSORS maspar(128,128) !HPF$ TEMPLATE templ1(100,100) !HPF$ DISTRIBUTE templ1(CYCLIC(4),*) ONTO maspar integer i, j, k real Vector(100) !HPF$ ALIGN Vector(i) WITH templ1(*,i)

vector

templ1

matrix

maspar

real Matrix(100,100) !HPF$ ALIGN Matrix(i,j) WITH templ1(100−j,50)

3dmat processors

real 3dmat(100,50,100) !HPF$ ALIGN 3dmat(i,j,k) WITH templ1(2*j,k) ...

templ2

cube template

Fig. 4.1. HPF-Builder overview

For each level, we present the corresponding editor with its main characteristics. A visual demonstration is available on the web (http://www.lifl.fr/west/hpf-builder).

HPF-Builder environment

7

4.1. Source editing and parsing. The rst phase of HPF-Builder concerns the parsing of the source le. Adaptor Parsing is used [2]. It supports FORTRAN 90 and almost all the data mapping directives of HPF. From both FORTRAN 90 and HPF code, HPF-Builder is now able to build the hierarchical skeleton of the program (cf. gure 4.2);

F90 code (Matrix vector product) HPF code (Matrix matrix product) Fig. 4.2. Source editing and parsing

An icon is associated with each array of the program and with each identi er of the HPF directives (processors and

template

names). Arrows between these

icons represent alignments and distributions. Two generic icons allow the user to generate new processors and template directives. A full screen editor is opened with the source le. The content of this editor is updated with any interactive graphical manipulation.

8

J-L. DEKEYSER AND C. LEFEBVRE

4.2.

. From the FORTRAN 90 code (Matrix vector product), we rst

Processors

insert a processor grid. We select the corresponding generic icon and the directive is automatically inserted in the source code where we indicated. The

processors

directive is created with default parameters but it is always possible to modify them (shape, size of dimensions and name) (cf. gure 4.3).

Fig. 4.3.

4.3.

processors

. In the same way, we create a template directive in the source

Template

code. This template editor shows a graphical representation of the template. It will be used for the alignment of arrays (cf. gure 4.4). A new icon for the template (and for the processors) is (was) then added to the skeleton view.

Fig. 4.4.

template

9

HPF-Builder environment

4.4. Distribute. When the user selects one template icon and one processors icon, HPF-Builder asks him/her to specify the characteristics of a new distribute directive. The distribute editor is automatically called. The right part of this editor visualizes the processor grid with one colour per processor. The left part visualizes the template. The colour of each element of the template corresponds to the colour of the allocated processor. Interactive modi cations of distribution will change the colouring of this part as well as the directive in the source code. For each dimension of the template, a detailed description of the distribution is processed. The expression 100 = 33  3 + 1  1 = 1  3  3 + 1  (2  3 + 1  1) + 14  2  3 in the gure 4.5b speci es that the rst dimension of

Templ

is distributed in such a way that it

is divided in 33 blocks of 3 columns, and 1 block containing the last column. The rst column of processors contains 3 of these blocks, the second one contains 2 blocks and the block of 1 column, and each of the 14 last columns contains 2 blocks. The expression 100 = 14  7 + 1  2 + 1  0 speci es that the second dimension of the template

is distributed along Procs as 7 lines onto the rst 14 lines of PEs, 2 lines

onto the 15th line, and nothing onto the last one.

(a) block-block

(b) cyclic-block Fig. 4.5. Distributions

All the distributions of HPF are supported. A new arrow in the skeleton represents

10

J-L. DEKEYSER AND C. LEFEBVRE

a distribute directive between the template and the processor grid.

4.5. Alignment. The last level of the programming model concerns the alignment of arrays with templates. In the same way when the user selects an array icon and a template icon, HPF-Builder automatically calls the alignment editor. All the arrays associated with this

template

are associated with one colour. This colour

is used to interactively visualize the array inside the template. From the origin of the array, arrows represent the rst, then the second and the third dimension. Each dimension is aligned with one dimension of the template.

(a) replicated

(b) collapsed Fig. 4.6. Alignments

The replicated alignment is shown by a curve along the corresponding dimension of the template (cf. gure 4.6a). A collapsed alignment is shown by a double arrow for the corresponding dimension of the array (cf. gure 4.6b).

4.6. Zoom eect. Dierent facilities have been added to HPF-Builder. Zooming is used on the template and processors to identify which elements are allocated on which template element or which virtual processor. With particular mixing of distributions and alignments, the eective mapping becomes complicated, e.g. alignment with step mixed with a cyclic distribution of the same step produces a collapse of this dimension on one processor. Zooming highlights this kind of diculty. For each virtual processor, a map of the slices of arrays allocated on this processor is produced

HPF-Builder environment

11

(cf. gure 4.7). Visualization of value indices is used to estimate communications generated by the compiler.

Fig. 4.7. Zoom eect applied to the processor Proc(3,4)

4.7. conclusion. Data parallel programming is still a dicult art. Scienti c programmers have expended a lot of eorts in learning vector (array) programming. Now they have to learn a third generation dialect of FORTRAN to map their data onto distributed memory machines. To succeed in this task, they need some tools to help them to manage their data distributions. HPF-Builder is a rst step in Computer Assisted High Performance Programming. The automatic insertion of HPF directives in a FORTRAN 90 code frees the programmer from the new syntactic constraints. With this step by step interactive re nement of the data mapping, HPF-Builder can be used as an educational tool to learn HPF. Redistribute and realign directives are not supported in the rst version. A knowledge of the value of variables as well as control ow implies to run the program to obtain the visualization. The next HPFBuilder will support this feature by adding a debugger interface to evaluate all the run time dependent elements. HPF-Builder will be available via FTP or the Web in December 1996. All are welcome to use it and report all comments on improving the functionality of this tool.

REFERENCES [1] A. Benalia, J.-L. Dekeyser, and P. Marquet, HelpDraw graphical environment: A step beyond data parallel programming languages, in Fifth Int'l Conf. on Human-Computer In-

teraction, Orlando, FL, Aug. 1993, Elsevier Science Publishers, pp. 591{596. [2] T. Brandes and F. Zimmermann, Adaptor - a transformation tool for hpf programs, in Programming Environments for Massively Parallel Distributed Systems, Birkhauser, April 1994, K.M.Decker and R.M.Rehmann, editors, pp. 91{96. [3] J. F. C. Clemencon and R. Ruhl, Visualization, execution control and replay of massively parallel programs within annai's debugging tool, in Proc. High-Performance Computing Sympo-

sium (HPCS'95, Montreal, Quebec, Canada), Centre de recherche informatique de Montreal, juillet 1995, pp. 393{404. [4] K. M. Decker and B. J. Wylie, Software tools for scalable multi-level application engineering, in Workshop on Environments and Tools For Parallel Scienti c Computing, Aug. 1996. [5] J.-L. Dekeyser, D. Lazure, and P. Marquet, A geometrical data-parallel language, ACM Sigplan Notices, 29 (1994), pp. 31{40. [6] R. K. S. Grabner and J. Volkert, Graphical support for data distribution in spmd parallelization environments, in Proc. IEEE 2nd International Conference on ALgorithms and Parallel

Processing, Singapore, 1996. [7] High Performance Fortran Forum, High Performance Fortran language speci cation, Scienti c Programming, 2 (1993), pp. 1{170. [8] R. Schreiber, Introduction to High Performance Fortran, in Spring School on Data Parallelism, Les Menuires, France, Mar. 1996, Lectures Notes in Computer Science. [9] S. T.Hackstadt and A. D.Malony, Distributed array query and visualization for high performance fortran, in Proc of Euro-Par'96. Lyon. France. August 1996, Aout 1996.

12