EDPEPPSy: An Environment for the Design and Performance Evaluation of Portable Parallel Software
email:
T. Delaitre, G.R. Justo, F. Spies, S. Winter University of Westminster Centre for Parallel Computing 12-14 Clipstone street London W1M 8JS
fdelaitt,justog,spiesf,
[email protected]
Abstract
This paper describes the architecture of a development environment for computer-aided parallel software engineering. The environment comprises tools for program design, simulation, and visualisation. The toolset supports an interactive, performance-oriented software development life-cycle, and the user interface makes extensive use of graphics and animation.
1 Introduction The recent rapid growth of parallel cluster computing has signalled the urgent need for tools to support a parallel program design life-cycle. Concurrent programs are certainly no less complex than serial ones, but whereas serial programmers can rely on a simple and stable programming model (the von Neumann machine), parallel programmers must cope with two additional issues: platform con guration; and mapping (of the concurrent program onto the parallel platform). In high-performance computing, many assumptions made by serial software engineers no longer hold. For example, in serial life-cycle models, performance engineering is a relatively low-priority activity. Yet in high-performance computing, one of the major areas in which parallel processing is currently being applied, performance is clearly the dominant issue. In response to this problem, many researchers have developed tools and methodologies for parallel programming [1]. Typical rst-generation tools, developed largely by hardware vendors, and based on proprietary hardware, are suitable for raw programming, but are not sophisticated enough for large-scale project use. The most recent approaches aim to provide a complete tool-based environment for computer aided parallel software engineering. For example, the SEPP project [2] has developed an overall architecture based on ve principal tool types: Static Design Tools; Dynamic Support Tools; Behaviour Analysis Tools; Simulation Tools; and Visualisation Tools (Figure 1). One of the earliest toolsets for parallel programming was the Transim/Gecko toolset, developed under the Parsifal project at the University of Westminster [3, 4]. This toolset combines simulation of occam-like programs on transputer-based multiprocessors with an animation-based representation of the resulting trace. The tools have been placed in many research establishments, where they have been used with good e ect in the design process. y
This project is funded by an EPSRC PSTPA programme, Grant Number : GR/K40468
173
The approach has been the model for several simulation-based tool environment projects. The HAMLET Application Development System [5], is an environment for developing real-time applications for parallel architectures based on the transputer (primarily) and the PowerPC. The philosophy is essentially that of Transim/Gecko, but with increased sophistication. The environment combines graphical design tools, DES, for both software and hardware con gurations, a simulation tool, HASTE, for predicting execution patterns, and a visualisation tool , TATOO, for examining execution traces. The system assumes contextual support from target tools such as debuggers, pro lers and monitors, which run on the target system. The PSEE (Parallel System Evaluation Environment) project aims to develop a range of interrelated tools, for programming, simulation and visualisation [6]. The project is now closely aligned to the SEPP project. The CAPSE (Computer Aided Parallel Software Engineering) environment [7] assists the development of performance-oriented SPMD (Single Program Multiple Data) programs by integrating tools for performance prediction, and analytical or simulation based performance analysis in the detailed speci cation and coding phase. The ALPSTONE project [8] is comprised of a methodology and tools to guide a programmer of parallel software from the point of performance evaluation (especially prediction and benchmarking). The software development process starts with a formal BACS (Basel Algorithm Classi cation Scheme) description [9] of an algorithm skeleton. From this description, it is possible to generate a time model of the algorithm which allows performance estimation and prediction. The PAMELA project [10] aims to develop of a modelling methodology that yields fast, parameterised performance models for parallel programs running on shared-memory or distributed-memory (vector) machines. The methodology is based on the simulation language PAMELA (PerformAnce ModElling LAnguage). The model description written in PAMELA is compiled into an analytical performance model which allows \fast and crude" predictions or simulation for \slow but accurate" predictions. The goal of the ALPES project [11] is to develop an environment for the performance evaluation of parallel programs on parallel machines. A parallel program is modeled as a valued task graph from which a parallel synthetic program is derived. This synthetic program can be modi ed in order to emulate di erent parameters of parallel machines like, for example, the computation/communication ratio. The synthetic program is executed. Traces are generated and treated. The presentation of performance information is done through a visualization tool. The importance of visualisation as a design aid can hardly be overestimated. Currently, the ability to design complex computational systems is limited only by the imaginations of programmers and designers. Visualisations with high information content will increase the imaginative powers of the users, enabling them to develop increasingly sophisticated, and e ective programs. Some generic tools are already widely used: ParaGraph for instance [12, 13]. However, generic views are not appropriate in every situation, and speci c tools, such as Transim/Gecko, Xmtool [14], and PACVIS [15], naturally de ne unique viewpoints on the underlying design problem. A key feature of a visualisation facility is the ability to relate clearly cause and e ect in a program, relating trends seen in actual (or simulated) behaviour to source code. It is also common, when representing simulated and actual traces, to present the two sets of information within a single framework, in order to facilitate a visual comparison of predicted and actual behaviour of the system. The aim of the EDPEPPS (Environment for Design and Performance Evaluation of Portable Parallel Software) project is to create a tool-based environment which will enable program designers to rapidly synthesize and evaluate methods for optimising the performance of concurrent software executing on a parallel platform. EDPEPPS is based on a rapid prototyping philosophy, in which the designer synthesizes a model of the intended software, which may then be simulated, and the performance subsequently analysed using visualisation. 174
SEPP
Static Design Tools Graphical Design
Hardware Configuration
Tool Architecture Dynamic Support Tools
Mapping Load Balancing
Simulation Tools
Visualisation Tools
Monitoring
Behaviour Analysis Tools Testing / Debugging
Performance Engineering
Kernel
Figure 1: SEPP Tool Architecture. In the next section, the main overall features and philosophy of the EDPEPPS environment are described. The following sections deal with the individual tools, and their respective interfaces in some detail. The simulation section includes some preliminary experimental results in the validation of the simulation model.
2 Overview of EDPEPPS The EDPEPPS toolset is based on three principal tools: a graphical design tool (PVMEdit); a simulation facility; and a visualisation tool. Figure 2 illustrates the simple cyclic relationship between the three tools. Interfaces between the tools are de ned: SimPVM, which represents the graphical program in a purely textual form; and two simulation output les, a trace le and a statistics le, which encapsulate the simulated program behaviour. EDPEPPS is based on an MPMD (Multiple Process Multiple Data) process model, and views the message-passing layer of a distributed operating system as the fundamental (parallel) execution platform. In the current phase of development, the message-passing and concurrency model is PVM [16], which is widely used and supported, and assumes a C+PVM parallel programming style. EDPEPPS is similar in concept to HAMLET, but the di erences are signi cant: EDPEPPS assumes a virtual hardware platform based on a message-passing interface (PVM), whereas HAMLET is based on physical hardware. The programming models di er: HAMLET assumes parallel C, whereas the choice of PVM in EDPEPPS allows a more generic parallel programming style. A corollary is that HAMLET is oriented towards real-time embedded applications, whereas EDPEPPS has more general-purpose applications scope. Load modelling is required in EDPEPPS, on the basis that the application being designed must co-exist with others in its eventual environment. EDPEPPS also has much in common with SEPP. However, the graphical design tool in EDPEPPS is based on PVM, whereas GRAPNEL, the counterpart tool in SEPP, de nes a programming model which may be viewed as a subset of the PVM paradigm. PVMEdit also encourages a mixed graphical/textual style, 175
and does not include graphical icons for low-level programming features. EDPEPPS differs from CAPSE in its support for the development of MPMD (Multiple Program Multiple Data) parallel programs. There is also a fundamental di erence in the way the prototypes or skeletons are described. In EDPEPPS, a graphical language is used and the prototypes describe the behaviour of the MIMD program in terms of tasks, and the way they interact in a PVM-like manner. In CAPSE, however, the prototypes are more abstract and are based on 5 types of speci cation (task structure speci cation, task requirements speci cation, task behaviour speci cation, packet requirement speci cation and packet behaviour speci cation). The ALPSTONE project di ers from EDPEPPS in two ways. Firstly, the language used to describe the program skeletons is a non-graphical abstract language which means that the generation of the implementation passes through many steps using more than one language, for example, ALWAN [9]. Secondly, the approach to performance evaluation adopted by the project is based upon analytical modelling and not simulation. The PAMELA project is similar to EDPEPPS in providing performance prediction. PAMELA is more general than the EDPEPPS graphical language but its environment is much simpler and consists basically of a run-time system library. The philosophy of EDPEPPS emphasizes the role of modelling and simulation as a solution to the parallel program design problem. Simulation is a well-known approach within traditional engineering disciplines (eg. VLSI design) and is invariably used to optimise the performance of the engineered product. For serial software engineers, however, the dominant design issue has been correctness, rather than performance, and the role of simulation as a software engineering tool has been relatively unknown. Based on models of a concurrent program, and the parallel execution platform, a simulation will yield approximate, but nevertheless highly useful information to the designer, facilitating the delivery of acceptable performance in the resulting system. As the design proceeds, the accuracy of the simulation model can be expected to improve, as measurements are gathered from the run-time system, giving more reliable information to designers and managers alike. One problem introduced by simulation-oriented design methods is that of maintaining two models: the executable code, and the corresponding \simulatable" code, ie. the code passed to the simulator. Experience shows that real programmers prefer to maintain their executable code { and neglect all other models (formal designs, documentation, etc). It is therefore of paramount importance to ensure that the programmer only needs to maintain a single source model (which, albeit graphical, should be as manipulable as executable code), and be able to produce both an executable version and a version for simulation from this single source. The approach taken in EDPEPPS, which is based on the PVM programming model, is to provide a graphical design editor, which combines graphical and textual manipulation techniques in a way which allows the programmer to develop his program in a natural and easy manner. The source program itself is based on C+PVM, with additional constructs for controlling the simulation of the program (naturally, these instructions are ignored in the executable code). Simulation of the PVM platform in EDPEPPS is built using SES/WorkbenchTM [17]. The Workbench has sophisticated built-in graphical input and visualisation facilities, which allow the rapid development of discrete-event simulations. The EDPEPPS simulation has four layers: the application layer; the message-passing layer; the operating system layer; and the hardware layer. By comparison, Transim was based on occam and the transputer { a twolevel model. Simulations themselves can consume large amounts of computing power, and a related project at the University of Westminster is directed at the problem of optimising the performance of parallel discrete-event simulations [18]. Visualisation in EDPEPPS is derived from concepts pioneered in Gecko, in which graphical program representations are animated { thus \bringing the program to life". This not 176
Graphical Design & Animation Tool
SimPVM Translator
SimPVM File
task pvm-send
Traces File
Performance Visualisation Tool
Statistics File Simulation Engine
Figure 2: The EDPEPPS Architecture. only maintains the necessary strong relationship between program and trace, but allows the designer to remain in the same visual environment for both the creative and analytic phases of the design process. The visualisation environment also makes extensive use of generic tools (ParaGraph for instance), in order to track a range of issues which originate within the platform, strictly below the 'visible' programming level.
3 Graphical Design of Parallel Programs There are compelling reasons to represent (program) designs graphically. One of them is to expose the software structure as the description of the constituent software components and their patterns of interconnection, which provides a clear and concise level at which to specify and design systems. In terms of message passing parallel programs, their structures are naturally graphical as parallel activities can be represented by the nodes of a graph and message
ow can be denoted by the arcs. This explains the popularity of graphical representations for parallel systems (for a survey refer to [19]). Furthermore, a clear description of a parallel program is important during the mapping and load balancing which are essential activities in the development of a parallel program. Most of the graphical representations for parallel programs can be classi ed into two main groups: one concentrates on an abstract view of the design and speci cation, the other one concentrates on the details of the algorithm or behaviour of the task components. Examples of the rst group are presented in [20, 21]. They represent the components of a design such as the modules (usually tasks or processes) and their relationships (port and message ow) but they do not describe the behaviour (or algorithm) of each task. At the other extreme, the representations in the second group [22, 23] enable us to describe the details of the behaviour of each task in a similar way to a owchart. The advantage of the representations in the rst group is to show the software structure but little can be done in terms of design analysis and evaluation as the behaviour of the tasks is hidden. In the representations of the second group, although the behaviour of each task is described, there is no clear relationship between the behaviour of each tasks and the structure of the whole program. Some attempts have been made to resolve the problem by including more information about the behaviour of the task at the design structure. For example, in VPE [24], one can 177
specify certain details about the tasks or compute nodes but this description is not graphical. The graphical representation we are developing for EDPEPPS tries to balance the aspects of design structure and the behaviour description of the components into a single graphical representation. The principle is that the design of a parallel program consists of nodes (tasks) and arcs (message ow) but the graph must be enriched with special allegories (symbols) which correspond to important aspects of the behaviour of the tasks. Since EDPEPPS aims at PVM programs, these aspects refers to the operations of PVM. In this way, for most of the PVM operations we de ne special symbols. A PVM program design consists basically of a collection of processes (tasks) which interact by message passing. A task is the basic component of the design. The interaction between tasks is carried out by calling PVM operations. A task is denoted by a box with its name, and each operation has its own symbol. In order to simplify the graphical representation, however, we assume at this stage that the actions are divided into two groups: inputs and outputs. An input means that the task will receive some data if that interaction happens. An output means that the task will send some data. A triangle pointing to inside the box means an input. A triangle pointing to outside the box denotes an output. To identify to which operation the action corresponds the user must select one PVM operation when the action symbol is inserted. Figure 1(a) illustrates certain symbols of our graphical representation. Some special PVM operations may correspond to more than one input or output at the same time. A pvm reduce, for example, performs global operations over all the tasks in a group. All group members call pvm reduce with their local data, and the result of the reduction operation appears on the task speci ed as root. For the root task, a pvm reduce corresponds to a \special input" as it receives several messages. This kind of input is de ned as a family of inputs and is represented by two input symbols as illustrated in Figure 3(a). Similarly, pvm mcast, pvm bcast and pvm scatter generate families of outputs which are also illustrated in Figure 3(a). A task may contain sub-tasks forming a hierarchical structure. This type of task, called composed task, is represented with double boxes as shown in Figure 3(a). A task becomes a composed task when it calls the PVM pvm spawn operation. It is worth pointing out that the notion of composed task in PVM does not involve the idea of encapsulation (abstraction) as in Conic [20], for example, where the sub-tasks cannot interact directly with other tasks outside the composed task, except via the ports of the composed task. In PVM, the spawned task is a complete independent task. However, it is still important to document in the design the fact that one task is responsible for creating (pvm spawn) or killing (sending a termination signal to { pvm kill) other tasks. A directed dotted-line from Task 0 to Task 2 means that Task 0 creates (pvm spawn) Task 1 (Figure 3(a)). A directed dotted-line with a cross in the middle from Task 0 to Task 1 means that Task 0 may kill (pvm kill) Task 1. Tasks which have compatible inputs and outputs can be connected. Consistency checks are carried out when two tasks are connected. These checks follow the semantics of the PVM operations. An interaction is denoted by a solid line as illustrated in Figure 3(a). A connection is not like a channel in Occam, for example, as connections are not explicitly declared in PVM. An important part of the design of a parallel program is agglomeration where task are grouped to reduce parallelism or to simply the communication structure [25]. The concept of group in PVM corresponds to a collection of processes that is seen as unit. The main advantage of de ning a group is that certain operations can be applied to all members of the
In later versions, each operation will have its unique graphical symbol.
178
output
family of outputs
spawn operation
Task2
jaguar
family of inputs
input Master simple task
Task3
Task0
Slave
virtual machine
(b) kill operation Task1 group
composed task
(a)
#include "pvm3.h" #define TaskName "Master" #define Slave "Slave" main() { int mytid; mytid=pvm_mytid(); pvm_spawn(Slave, ... );
#include "pvm3.h" #define TaskName "Slave" main() { int mytid; mytid=pvm_mytid(); /* task definition */ pvm_recv(-1,-1); /* input */ pvm_exit(); }
/* task definition */ /* spawn */
pvm_initsend(PvmDataRaw); pvm_pkint( ... ); pvm_send( ... ); /* output */ pvm_exit(); }
(c) Figure 3: (a) EDPEPPS graphical objects. (b) Master-Slave graphical program. (c) Master-Slave PVM Program skeleton. group instead of being applied individually, for example pvm mcast, which sends a message to all members of a group. However, the group primitives are de ned in terms of the more basic primitives. Again, it is important to point out that the notion of group in PVM does not assume the notion of encapsulation, that is, a group is not considered as an abstract unit. The tasks in a group are completely independent and can be addressed directly by any other task that is not part of the group. A group is represented as a blob (Figure 3(a)) that delimits the member tasks of the group. One task can be member of more than one group, therefore group objects can overlap. Mapping tasks to processors is also an important part of the design of a parallel program [25]. Note that, we are not interested here in the hardware description but only in describing the allocation of tasks to virtual nodes. In this way, a virtual node is similar to a group and is also represented by a blob except that it is shaded to di er from a group and cannot be overlapped as one task cannot be allocated to more than one virtual machine at the same time. Virtual nodes are illustrated in Figure 3(a). As described above, every graphical object in our representation corresponds to some PVM segment of code. The reverse is not necessarily true as not every PVM text has an equivalent graphical representation. The segment of code associated with a certain graphical object is supposed to denote its semantics. In order to maintain consistency, checks are carried out to 179
guarantee that any change in a graphical object should a ect its corresponding segment of code and vice-versa. A program skeleton can, therefore, be automatically derived from the graphical representation. Figure 3(b) shows a simple Master-Slave program where Figure 3(c) presents part of the program skeleton. This skeleton can be used for simulation or for derivation of an implementation. Unfortunately, in this paper we are not able to give all the details of the EDPEPPS graphical representation but the description presented in this section outlines the expressiveness of the representation. The rst version of the EDPEPPS graphical, called PVMEdit, has been completed and is under test.
4 SimPVM - A Simulation Oriented Language PVMEdit allows PVM applications to be developed using a combination of graphical objects and text. From this description, executable PVM programs may be generated, but to simulate program execution, annotations must be inserted into the graphical/textual source to control the simulation. This allows both an executable and a \simulatable" version of the program to be generated from the same graphical/textual source. All simulation models in EDPEPPS are based on queueing networks. The \simulatable" code generated by PVMEdit is predominantly a description of the software rather than the execution platform. To simulate the application, a model of the intended platform must be available. Thus, a simulation model is fundamentally partitioned into two submodels: a dynamic model automatically generated from the PVMEdit description, which is comprised of the application software description and some aspects of the platform (eg. number and type of hardware nodes); and a static model which represents the underlying parallel platform, to a suitable level of detail. By building the static descriptions into the simulation system itself, the service is transparent to the application designer, yet provides a virtual, simulation platform for the generated application. The SimPVM language basically contains the following elements:
the list of processes to be initially executed (exec), and the host (identi cation) number where they execute; the description of processes (process) and functions (function). In this version, processes cannot be parameterised; C instructions for variables declaration, loops (for and while), conditional instructions (if else), and assignments; PVM functions for process management (e.g. pvm mytid and pvm spawn), bu er management (e.g. pvm getsbuf) and for point-to-point communication (e.g. pvm send); simulation constructs such as computation delay function and statistical variables.
SimPVM lies above the libpvm level of the simulation model (more details will be presented in Section 5.1) but some special functions are also provided that direct interaction with the kernel model. A SimPVM program is translated into an SES/Workbench simulation model where lines of the program are interpreted as simulation objectsy, which can be a Reference for a process, a Call to a Reference for the PVM functions, or a Speci c Node for C instructions. Figure 4 shows a detailed SimPVM version of the Master-Slave program presented in Figure 3(c). y
The SES/Workbench simulation model language is de ned by graphical nodes
180
exec Master(0,1);
process Master() { int mytid, tc, i, v[100]; mytid=pvm_mytid(); pvm_spawn("Slave", NULL, 0, NULL, 1, &tc);
process Slave() { int mytid, i; mytid=pvm_mytid(); for (i=1; i
