Integrated Tools for Performance-Oriented Distributed Software Development N. Mazzocca
M. Rak
U. Villano
DII, Seconda Università di Napoli via Roma 29 81031 Aversa (CE), Italy +39-081-5010333
DII, Seconda Università di Napoli via Roma 29 81031 Aversa (CE), Italy +39-081-5010227
DING, Università del Sannio c.so Garibaldi 107 82100 Benevento, Italy +39-0824-305844
[email protected] E. Mancini
[email protected]
[email protected]
RCOST, Università del Sannio viale Traiano 82100 Benevento, Italy +39-0824-305540
[email protected] DRAFT PAPER
ABSTRACT This paper presents an integrated set of tools for performance-oriented development of software targeted to distributed heterogeneous systems. Using these tools, software development is carried out through a friendly and handy graphical interface able to exploit the MetaPL notation system for software description and the HeSSE simulation environment for its performance evaluation. MetaPL is an XML-based language that allows the description and the analysis at different level of detail of programs written in any distributed programming language and run-time support. HeSSE is a simulation environment that makes it possible to predict parallel and distributed software performance even in the first steps of software development, when no actual code has yet been written. The paper shows how MetaPL and HeSSE, along with the relative tools, have been integrated in order to provide a flexible and general-purpose graphical development environment and presents an example of its use.
Keywords Performance Analysis, Program Description Language, Distributed Software Systems, Simulation, XML.
1.
INTRODUCTION
The presence of distributed software systems is pervasive in current computing applications. In commercial and business environments, the majority of time-critical applications has moved from mainframe platforms to distributed client-server systems [10]. In the academic and research field, the advances in high-speed networks and improved microprocessor performance have made clusters or networks of workstations an appealing vehicle for costeffective parallel computing [3]. As a matter of fact, nowadays software development for a very wide range of application fields has to take into account the main
characteristics of the above mentioned environments, namely heterogeneity, easy extensibility through the addition of further processing nodes, as well as the presence of not-completely reliable networks, possibly shared with other users or applications. We will refer to these environments as Distributed Heterogeneous Systems (DHS). DHS software design and development has proven to be a challenging matter, due to the inadequacy of the “traditional” approach, commonly followed for sequential or parallel homogeneous systems. Mainframes, supercomputers, workstations or PCs are built on (relatively) stable and well-known hardware architectures and system software, and users can employ customary tools and expertise for software development. The newest DHS systems continuously grow and change, and are subject to subtle effects due to network heterogeneity and sharing. On the software side, there is currently a high number of imperative parallel/concurrent programming languages and environments, which rely on run-time libraries, middleware layers or O.S. calls on the top of communication protocol software. This complexity is not even mitigated by the use of unifying programming approaches. Very often the development of distributed software is still carried out ignoring systematically software engineering principles and methods. Most of the times, the obtained programs are badly structured, difficult to understand, not easy to maintain or re-use. Not surprisingly, they may even fail to meet the desired performance, which is, after all, one of the driving forces towards the adoption of DHS environments. Our research group developed a simulation environment, HeSSE [1,2,6,7] that enables the developer to model DHS systems and to predict their performance. The simulation models built with HeSSE can be defined at any development stage, both when the system is available or
when it is completely missing (at a small accuracy cost), but can provide performance information only for a given execution (i.e., for a given set of input data, under certain working conditions). On the other hand, the MetaPL language [8,9,10] solves complementary problems, making it possible to model applications and, by means of a filter system and user queries, to choose different sets of execution conditions. MetaPL is an XML-based language that can describe programs written in any kind of distributed programming language and run-time support. It can be used at any stage of software development, being possible both to use it to define prototypes of the software to be written, or, conversely, to obtain high-level descriptions of existing applications. Even if it has already been proven that the joint use of these tools can be an effective way to predict complex system performance [11], it is fairly obvious that a high level integration of the tools would lead a more friendly and easy-to-be-used performance prediction and analysis environment. However, making this integration is no trivial task. The problem is not to adapt the tools and to integrate them for the study of a single system and a single application running on it, but to define an integration model that could be sufficiently general to be used for any kind of system and of software performance analysis. The objective of this paper is the study the problem of the tool integration and to show an existing solution, which has shown to be effective and general for any modelling and performance analysis task. In the next section we will give a brief overview of the two set of tools to be integrated, referring the interested reader to previous papers for more details on both HeSSE and MetaPL [6-11]. Then we will discuss the problems related to their integration and thoroughly illustrate the devised solution and the operating mode of the resulting system. The paper closes with a simple case study that will show the practical use of the system. Finally, the conclusions are drawn.
2.
MODELING TECHNIQUES
DHS design is a complex task, hard to manage due to many different conditions that System Designer has to face. In this kind of systems both System topology and application behavior easily change. Our methodology is based on highlevel application description with MetaPL, and system modeling and simulation with HeSSE. The next section will briefly introduce both the modeling techniques and the problems that arise for their integration.
2.1
HESSE
HeSSE (Heterogeneous System Simulator Environment) is a simulation tool that can reproduce the performance
behavior of a given DHS for a given application, under different computing and network load conditions [1,2,3]. As far as DHS simulation is concerned, the distinctive feature of HeSSE is the adoption of a compositional modeling approach. DHS are modeled as a set of interconnected components; each component reproduces the performance behavior of a section of the complete system at a given level of detail. System modeling is thus performed primarily at the logical architecture level. Physical level performances (processor architecture, resulting effects of particular architectural choices in the electronic solutions) are in general modeled with simple analytical models or with integral and not punctual behavioral simulation, e.g., the use of a processor to execute instruction sequences is modeled as the total time spent in the processors, without considering the behavior on a per-instruction basis. HeSSE is capable of obtaining simulation models for Distributed Heterogeneous Systems (DHS) that can be even very complex, thanks to the use of components composition. A component is an object that can reproduce the temporal and functional behavior of a part of the whole system. Applications are in general described in HeSSE through traces. A trace is a file in which all the relevant actions of the program are reproduced. For example, a typical HeSSE trace file for parallel PVM applications is a timed sequence of CPU bursts and of requests to the run-time environment. Each trace represents a single specific execution of a parallel program.
2.2
METAPL
MetaPL is an XML-based language for the description of parallel programs. The main characteristic of MetaPL is its extensibility, obtained exploiting XML distinctive characteristics. The goal of MetaPL is to supply a unified approach to parallel programming, always characterized by a wide number of programming paradigms and models. A ‘core’ that defines a small set of elements for basic process activities (i.e. loops, switches, variable declarations, spawn and wait for process managmet, …) and a set of extensions compose the description language. The core is a really limited, as it aims to give only a basis for program description. The extensions define all the kind of really relevant instructions. Extensions for both message-passing and shared-memory interactions have been developed, along with extensions for procedure and library description, an object oriented extension is in development stage. It is easy to query or parse a MetaPL description, since it is, after all, an XML document. However, in order to do this, dedicated tools should be developed. The MetaPL usage is based on the View and Filter concept.
A View is a less general and more specific description of the program. Examples of views are UML diagrams, HTML documentation, or other MetaPL descriptions. A Filter is an XSLT document that generates a View from a given MetaPL document. The Filter system is extensible, in exactly the same way as the MetaPL language is.
2.3
Using the MetaPL filter system it is possible to generate HeSSE input traces from a MetaPL description. MetaPL descriptions can describe both the program in general (High Level Description or HLD) or a specific execution (Low Level Description or LLD). Trace file generation from an LLD description requires a relatively simple translation (the process is described in [10]. The generation of an LLD description from a given HLD description is instead a hard task. The HLD can be more easily managed from the user point of view, whereas LLD can be straightforwardly used to obtain the simulator input files. In [10] we showed how to obtain an LLD from an HLD by means of an automatically generated script file that asks all necessary execution details to the user.
2.4
etaPL
METHODOLOGY OVERVIEW
The HeSSE simulation models make it possible to describe the more “stable” aspects of the complete system (such as number of processors, operating systems, scheduling policies). We call this System Level Modeling. On the other hand, MetaPL allows to describe the system evolution in time, according to user data and application execution. We call this Behavior Level Modeling. The integrated modeling methodology we propose is based on the integration of the two modeling levels. This is implemented through the exchange of trace files that are produced by the MetaPL filter system and used as input by the HeSSE simulator.
Single Execution Traces
User
INTEGRATION
A very important View is a HeSSE trace file: trace files are very specific description of a program. The trace describes the program behavior in a well-defined execution condition. Traces can be easily obtained (and usually are) through code instrumentation and programs execution on host systems [1,2].
M
System Specification
eSSE
Complete System Behavior Quantitative Analysis
Figure 1.
MetaPL/HeSSE Methodology Schema
The design and analysis cycle is based on a feedback approach: once both System and Behavior level models are obtained, it is possible to produce performance predictions, analyze them and, if needed, perform corrections such as change of application parameters, modifications to the software or to the target computer system, different workload balancing.
3.
THE INTEGRATED TOOLS
Even if the underlying modeling techniques have proven to be fairly effective [11], every new problem needs tuning, adaptation and sometimes modifications of the existing prototypal tools. Experiences in real-world case studies clearly pointed out the need to develop a graphical, user-friendly interface that could be sufficiently general to be used in a wide spectrum of optimization and development problems. The design of such a tool has to tackle three main problems: 1.
independence between Behavior and System Level modeling phases, that is, the need to be able to adopt independent processes for modeling the application and the target system architecture;
2.
interactions between Behavior and System Level modeling phases, that is, management of the influence of the system configuration on trace generation.
3.
flexible management of output graphs (performance indexes can change in different case studies).
The inconsistency between the issues at points one and two is only apparent: software and system modeling are independent tasks, usually carried out by different experts, but they are strongly related. For example, the node number of the modeled system influences the number of application processes, and this influences the behavior of message passing collective primitives.
It should be noted that, given a single case study, all the above problems are completely trivial: performance indexes and system configurations are known, and only the need is an interface able to speed up the modeling steps. On the other hand, the problems are hard to be solved when a generalization process is carried out: the target tool should be able to face any kind of problems, with a minimal effort of the end-user.
Modeling Tool Graphical Interface
M
The output definition is the most difficult problem to solve. After all, it is not possible to foresee the information that the final user would like to gather about a generic target application. Our solution is to automatically query the user about his/her needs and to produce the wanted results interactively. In other words, instead of letting the user choose from a predefined set of performance indexes, as most performance-oriented tool usually do, an active object customized on the basis of the system and behavior level models is automatically generated: the model manager (MM). This prompts the user for the type of desired output indexes and views, and generates them.
GUI
eSSE
Model Manager Tool
M
etaPL
User(s)
In the tool integration that is the object of this paper, independences between the two modeling phases is obtained through the development of two independent tools, described in detail in next sections. The first one (EdMetaPL) is an extended XML editor, able to manage MetaPL descriptions and its filtering system. The second one (HeSSEgraphs) is essentially a graphical editor of HeSSE configuration files. The MetaPL editor has also a project management role. To solve the integration problem, instead, a list of the available configurations is reported in the MetaPL High Level Description. Thanks to the Project management role of the EdMetaPL tool, when lower level descriptions or the HeSSE trace files are generated, it is possible to refer to the right configurations. The MetaPL element adopted to describe the configuration is named Configuration, and its attribute reports information such as the number of processes that the configuration supports, assuring compatibility between generated traces and HeSSE configurations.
GUI
etaPL
Single Execution Traces
eSSE
Output Graphs
Figure 2.
Modified Analysis Cycle
Taking into account the use of the model manager, the integrated methodology can be represented as shown in Figure 2. The user defines the model by means of the HeSSE and MetaPL graphical tools, and a suitable Model Manager is generated. The user interacts with the model, gathering the performance information and, if needed, updating the model or the real hardware/software architecture. The next section will go on to describe the two component tools and the model manager in further detail.
3.1
EDMETAPL
EdMetaPL is an XML editor with additional features intended to assist the user in MetaPL project management. At the definition of a new project, all the filters needed by the user are chosen. The tool is also able to manage the MetaPL extension mechanism, both as far as language and filter extensions are concerned, on a project basis. One of the most interesting features of EdMetaPL is the methodology management system. Methodologies make it possible to define filter chains, in such a way that transformations from HLD to LLD, or from LLD to other views can be automatically handled. Even if at the moment only static filter chains are managed, a forecoming new version of the tools will expand this feature, thus allowing more flexible interactions with the user. In Figure 3 it is possible to see a simple example of use of EdMetaPL. Each MetaPL document can be presented as a tree or as an XML text file. The first view is useful for editing and validating the file, whereas the second one, more flexible, is only for XML experts.
Figure 4.
3.3
HeSSEGraphs screenshot
THE MODEL MANAGER
Once the DHS is completely modeled through the techniques presented above, a customized model manager is generated. A generic MM is composed of: •
a MetaPL HLD, with all the needed DTDs and a collection of suitably-extended filters;
•
a collection of HeSSE configurations (each configuration resides in a folder);
•
a collection of predefined scripts.
Every time that a model manager is started, a new graph is generated. In actual fact, just a data file is produced, to be later on translated by a gnuplot script into a graph.
Figure 3.
EdMetaPL Screenshot
As shown in Figure 5, the model manager tool works in a cyclic way. The main control script is generated through a filter on the basis of the MetaPL HLD. Repeat
The tool is able to manage multiple MetaPL descriptions in a single project, so that multilevel description can be defined and studied together within the graphical environment. In each project it is possible to choose extensions and filters.
3.2
M
HLD etaPL
M
LLD
Trace File generation eSSE
etaPL
Control script generation model_manager.sh
HESSEGRAPHS
HeSSEGraphs is a graphical interface to the HeSSE simulator. It makes it possible to create in an interactive and user-friendly way configurations of (possibly heterogeneous) distributed systems to be simulated. HeSSEGraphs is completely integrated with the simulator, in that it produces HeSSE configuration files describing the hardware/software environment to be simulated, and it allows to launch simulations and to visualize their results from within the graphical interface. Figure 4 shows a screenshot of HeSSEGraph, note the library section on the left side and the system model on the main window.
Store Result
User Query
Figure 5.
Model Manager execution
The MM script, automatically generated by the MetaPL description, contains a list of all relevant parameters. It asks the user which ones should be considered as independent and which as dependent variables, or simply as parameters. It also asks for the configurations to be used, the sample step for the chosen independent variable. Using these data, it generates a secondary script, which is configuration-dependent, and iteratively starts and fills it with the values for each specified parameter. The called script generates the HeSSE trace, starts the simulation and stores the desired output.
3.4
A SIMPLE EXAMPLE
To show how the integrated tools works, we will show here a simple example. The goal of the description is not to give a complete performance analysis of a DHS, but just to show how the integrated tools can be used. Due to space constraints, the example here is just sketched. It will be expanded in the final version of the paper. In the example below, the target hardware system is the Cygnus Cluster, composed of four bi-processor Pentium III systems, plus a front-end. Its model description was built
and tuned by benchmarking application to be executed algorithm [12], an iterative systems of linear equations. modeled in MetaPL.
the real system. The target on the luster is the SOR algorithm adopted to solve The target application was
A little excerpt from the MetaPL description is presented below. It shows some of the variables of the application. In the example, N is the problem dimension, NITER the number of iterations, NPROC the number of processes.
The System section of the description (for details, see the MetaPL papers) contains a list of configuration descriptions, such as: … …
The MM script filter generator finds variable declarations and configuration names in the description (please note that variables are found in loops along with other description elements, it is out of the scope of this paper to point out all the possible cases). The resulting script queries the user about the system configuration to use and some of the variable values; it classifies the variables as parameter, dependent or independent variables and queries the values of the parameters (that will not change in the graph generation). Once this information is available, the script fills the MetaPL description with brand new values, generating a new, more detailed description. In the newly produced MetaPL code, some of the variables are bounded to the value thus assigned, and there is only one configuration description. …
… …
Then the MM script, by means of the MetaPL filter system, generates the new control script, and starts working in an iterative way, launching at each step the new script with suitable parameters (values of the variables in each execution). The control script generates the trace files, putting them in the right folder (4node_1proc in the example), starts HeSSE, and gathers the results putting them in the target gnuplot data file.
4.
CONCLUSIONS
This paper describes a performance-oriented environment that enables the user to gather performance graphs about a DHS system in many different working conditions. As the underlying methodology is based on two modeling techniques, the environment has been set up by integrating two distinct set of tools for code development and description and for system performance prediction. The integration has been obtained by the automatic production of a suitably-tailored active object, the Model Manager, that queries the user for further information on the performance data to be gathered and the working conditions, and generates the required output data.
References [1] Aversa, R., Mazzeo, A., Mazzocca, N., Villano, U. Developing Applications for Heterogeneous Computing Environments using Simulation: a Case Study. Parallel Computing 24 (1998), 741-761. [2] Aversa, R., Mazzeo, A., Mazzocca, N., Villano, U. Heterogeneous System Performance Prediction and Analysis using PS. IEEE Concurrency 6 (1998) 20-29. [3] Buyya, R. High Performance Cluster Computing. Architectures and Systems, Vol. I, Prentice Hall, USA, 1999. [4] Hoeben, F. Using UML models for performance calculation. in Proc. 2nd International Workshop on Software and Performance (WOSP00), ACM Press, New York, NY, USA, (2000) 77-82. [5] Hrischuk, C. E., Woodside, C. M., Rolia, J. A., Iversen, R. Trace-Based Load Characterization for Generating Performance Software Models. IEEE Trans. on Softw. Eng. 25 (1999) 122-135. [6] Mazzocca, N., Rak, M., Villano U. The Transition from a PVM Program Simulator to a Heterogeneous System Simulator: the HeSSE Project. in J. Dongarra et al. (eds.). Recent Advances in Parallel Virtual Machine and Message Passing Interface, Lecture Notes in Computer Science, Vol. 1908. Springer-Verlag, Berlin, DE, (2000) 266-273. [7] Mazzocca, N., Rak, M., Villano, U. Predictive Performance Analysis of Distributed Heterogeneous Systems with HeSSE. in Proc. 2001 Conference Italian Society for Computer Simulation (ISCSI01). [8] Mazzocca, N., Rak, M., Villano, U. MetaPL: a Notation System for Parallel Program Description and Performance Analysis. in Malyshkin., V. (ed.). Parallel Computing
Technologies, Lecture Notes in Computer Science, 2127. Springer-Verlag, Berlin (2001) 80-93. [9] Mazzocca, N., Rak, M., Villano, U. Parallel Program Development using the MetaPL Notation System. in Proc. ParCo2001 Conference, 4-7 September 2001, Naples, Italy, Imperial College Press, pp. 481-489. [10] Mazzocca, N., Rak, M., Villano, U.: The MetaPL approach to the performance analysis of distributed software systems in Proc. 3nd International Workshop on Software and Performance (WOSP02), IEEE Press , 24-27 July 2002, Roma, Italy. [11] Fahringer, Mazzocca, N., Rak, M., Villano, U., Pllana S., Madsen G.: Performance Modeling of Scientific Applications: Scalability Analysis of LAPW0 , in Proc. PDP03 Conference, 3-7 February 2003, IEEE Press, Genova, Italy. [12] Di Martino B., Iannello G. (1994), Parallelization of Nonsimultaneous Iterative Methods for Systems of Linear Equations, in Buchberger, B, Volkert, J. (eds.), Parallel Processing, Lecture Notes in Computer Science, Vol. 854, Springer-Verlag, Berlin 1994, (pp. 253-262) [13] Menascé, D. A., Gomaa, H. A Method for Design and Performance Modeling of Client/Server Systems. IEEE Trans. on Softw. Eng. 26 (2000) 1066-1085. [14] Menascé, D. A., Gomaa, H. Design and performance modeling of component interconnection patterns for distributed software architectures. in Proc. 2nd International Workshop on Software and Performance (WOSP00). ACM Press, New York, NY, USA, (2000) 117-126. [15] Smith, C. U., Williams, L. G. Performance Evaluation of a Distributed Software Architecture. Proc. CMG, CMG, Anaheim, USA (1998). [16] Smith, C. U., Williams, L. G. Performance Engineering Evaluation of Object-Oriented Systems with SPE·ED. in Marie, R. et al. (eds.): Computer Performance Evaluation Modelling Techniques and tools, Lecture Notes in Computer Science, Vol. 1245. Springer-Verlag, Berlin, DE (1997). [17] Special joint issue of IEEE Computer 28 (Nov. 1995) and IEEE Parallel and Distributed Technology (Winter 1995) on Parallel Performance Evaluation Tools. [18] Woodside, C. M. A Three-View Model for
Performance Engineering of Concurrent Software. IEEE Trans. on Software Engineering 21 (1995) 754767.
