Dec 5, 1996 - sors/workstations required for a certain parallel design and parallel ... In the simulation path each C/PVM source code obtained from the .... of boxes which represent tasks (which may include computation) ... 4. pvm send and pvm recv: If send/recv has dynamic connection then need to make sure that.
An Environment for the Design and Performance Evaluation of Portable Parallel Software Report Number : EDPEPPS/20
Speci cations of EDPEPPS Toolset Prototype (DRAFT) T. Delaitre, P. Vekariya, R. Bigeard, G. Justo, S. Winter, M. J. Zemerly Centre for Parallel Computing, University of Westminster, London December 5, 1996
1 Introduction The EDPEPPS project is developing an integrated graphical toolset for performance-oriented design of portable parallel software. The toolset comprises three main tools which are the Graphical Design Tool (PVMGraph), the Simulation utility (SES/Workbench), and the Visualisation Tool (PVMVis). Other tools which are required for the integration process are also discussed here. This report summarises the speci cations and features of the EDPEPPS Toolset prototype. Speci cations for the three main tools available within the toolset are given. These speci cations are intended to give a guide to the user about the capabilities and restrictions of the current versions of the tools.
2 Description of the Integrated EDPEPPS Toolset Traditional parallel programming methods start with porting a sequential code into a parallel machine (or coding the parallel program from scratch), debugging and running the code on the target machine, then nally analysing the performance. This cycle is usually repeated if the performance is not satisfactory. This process is time consuming and requires the use of the target parallel platform. The advantages of the EDPEPPS toolset which o ers rapid prototyping approach to parallel software development over this method are: The performance analysis is done without the need to access the parallel platform. The ease with which the parallel design is changed using the graphical user interface. The user performs the complete cycle of designing, simulation and performance analysis in the same environment without having to leave the toolset. The animation features within the visualisation facility o ers a debugging insight into the execution of the parallel program. The user needs limited knowledge of the parallel programming model language (in this case PVM) syntax to produce the parallel PVM/C code. The PVM platform model is partitioned into three separate layers (the message passing, the operating system and the hardware) and o ers two key features in simulation modelling: modularity and extensibility. 1
The ability to give answers to user questions as for example, the optimal number of processors/workstations required for a certain parallel design and parallel platform.
The EDPEPPS toolset is shown in Figure 1. LINE NUMBERED AND INSTRUMENTED C
Modified TAPE/Pvm
_t.c
SimPVM Translator
Preprocessor
.grf
SES/Workbench PVMGL
TAPE/Pvm DATABASE
Simulation Engine
PVMVis
PVMGraph
MAPS FILE NAMES INTO FILE NUMBERS
Simulation path
.c
Visualisation/animation
TAPE/Pvm Trace File
Real execution path
EDPEPPS Tool EDPEPPS File
USER
EDPEPPS process
Monitoring of the Real Execution
.c _t.c
TAPE/Pvm Preprocessor
Compilation
Figure 1: The EDPEPPS Environment. The gure shows the components of the EDPEPPS toolset. The process starts with the graphical design tool (PVMGraph) by building a graph representing a parallel program design based on the PVM programming model. The graph is composed of computational tasks and communications. The tool provides graphical representation for PVM calls which the user can select to build the required design. Limited knowledge of the syntax of PVM calls is required to complete this task. The generated design is stored in a PVMGL le [2] which is required later in the animation/visualisation stages. The user can then generate (by the click of a button) C/PVM code (.c les) for either simulation or real execution. In the simulation path each C/PVM source code obtained from the PVMGraph is processed using a modi ed version of the Tape/PVM preprocessor [3]. The modi cation is to remove the automatic insertion of the tape start call which is used mainly for clock synchronisation, an action not needed for the simulation. The instrumented C source les have new parameters added to each PVM function call. Some of these parameters are the line number and the le number which are used to relate the trace le back to the design. The instrumented C source les are translated using the SimPVM Translator [8, 9] into a queueing network representation suitable for the Workbench graph (.grf le). The SES/WorkbenchTM [5, 6] utility (sestran), which is di erent than the SimPVM translator used before, is then used to translate the graph le into the Workbench object oriented simulation language called SES/sim [7]. The simulation is based on discrete-event modelling. The technology is well-established, and sophisticated modelling tools are available commercially. SES/Workbench, for example, has wide functionality meeting the requirements of computer system modelling, and includes a time-saving graphical design interface and animationbased visualisation capabilities. Techniques for parallelising discrete-event simulation programs are also well-established, thus providing a route for optimising the run-time performance of the simulator. SES/Workbench has been used both to develop, and simulate platform models. Thus the simulation engine in SES/Workbench is an intrinsic part of the toolset. 2
The platform model and the Workbench simulation engine declarations are then linked by the simulator compiler (sescom) to produce a C source source. This in turn is compiled using the native C compiler with the PVM platform and Workbench run-time libraries to produce an executable model. All these actions are hidden from the user and are executed from the PVMGraph window by a click on the simulation button and hence not shown in the EDPEPPS Environment in Figure 1 but separately in Figure 2. The simulation executable is run using two input les containing parameters concerning the target virtual environment (e.g. number of hosts, host names, architecture, and the UDP communication characteristics, see Figure 3). A shellscript (\build") is used to build/compile the simulation model (with arg `b'), execute it (with arg `e'), or clean (with arg `c') the intermediate generated les automatically. .grf sestran .sim extern declarations of PVM platform model
.inc C preprocessor .ppp sescom .c Native C compiler .o
PVM platform library
Workbench run-time libraries
Linker .exe Executable model
Figure 2: The SES/Workbench Simulation Stages. The simulation model is hidden from the user and will not be discussed here (for detailed description and validation of the model see [1]). The output of the simulation are the execution time, a Tape/PVM event trace le and a statistics le about the virtual machine. These les are then used in the visualisation tool (PVMVis) in conjunction with the PVMGL le to animate the design and visualise the performance of the system. The design can be modi ed and the same cycle is repeated until a satisfactory performance is achieved. In the real execution path the Tape/PVM preprocessor is used to instrument the C source les and these are then compiled and run to produce the Tape/PVM trace le required for the visualisation/animation process. This step can be used for validation purposes and when the performance analysis yields satisfactory results to obtain the nal portable parallel codes and executables. Currently, the user has to provide a Make le suitable for compilation of the C/PVM code (see Appendix A). but future version of PVMGraph will generate suitable Make les for the simulation path and the real execution path automatically. The visualisation tool (PVMVis) o ers the user graphical views (animation) representing the execution of the designed parallel application as well as the visualisation of the performance behaviour of the parallel application from the point of view of the hardware, the design and the network. The animation is an event-based process and is used to locate unwanted behaviour such as deadlocks or bottlenecks. The PVMGraph and PVMVis are incorporated within the same Graphical User Interface where the user can switch between these two possible modes. The performance behaviour visualisation presents graphical plots, bar charts, space-time charts, and histograms for performance measures concerning the platform at three levels (the message 3
// parameters // anubis [0] // cheops [1] // horus [2]
osiris [3] piglet [4] winnie [5]
pvmd.nhosts = 6; pvmd.hostname = {"anubis","cheops","horus","osiris","piglet","winnie"}; pvmd.arch = {"SUN4SOL2","SUN4SOL2","BSD386","BSD386","SUN4","SUN4"}; ipc.ip[0].udp_protocol = {3.08156e-08,0.00067041, -2.17787e-07,0.00141523, -2.49824e-07,0.00193493}; ipc.ip[1].udp_protocol = {4.23286e-08,0.000146233, 2.93521e-08,0.00018409, 4.6293e-08,0.000127103}; ipc.ip[2].udp_protocol = {3.8841e-07,6.1387e-05, 3.34467e-08,0.000537045, -8.74287e-09,0.000691135}; ipc.ip[3].udp_protocol = {2.51621e-07,0.000359086, 7.18212e-08,0.000701099, 7.41616e-08,0.000735463}; ipc.ip[4].udp_protocol = {3.04803e-07,0.00053409, 3.65743e-08,0.00106462, 1.26447e-08,0.00136478}; ipc.ip[5].udp_protocol = {1.79087e-07,0.000432096, 1.50117e-07,0.000537088, 1.50453e-07,0.000597851};
Figure 3: CPC machines characteristics le. passing layer, the operating system layer and the hardware layer). However, not all performance visualisation features are implemented in the prototype version of the toolset. The following sections will summarise the speci cations, features and future plans for the EDPEPPS Toolset.
3 PVMGraph Prototype Speci cations This section summarises the main functions of the PVMGraph tool [2]. Also features and limitations of the prototype tool will be discussed. Suggestions for future planned features are also given. PVMGraph is a graphical programming environment to support the design and implementation of parallel applications. PVMGraph o ers a simple but yet expressive graphical representation and manipulation for the components of a parallel applications. The main function of the PVMGraph is to allow the user to develop PVM applications using a combination of graphical objects and text. Graphical objects are composed of boxes which represent tasks (which may include computation) and arrows which represent communications. The communication actions are divided into two groups: input and output. The PVM actions (calls) are numbered to represent the link between the graph and text in the parallel program. Also di erent types and shapes of arrows are used to represent di erent types of PVM communication calls (or even the same call with di erent parameters such as \-1" for send, receive or multicast). For details of the di erent representation of pvm calls the reader is referred to the original PVMGraph speci cations documents [2, 10]. Parallel programs (PVM/C) (or code for individual tasks) as well as executables can be automatically generated by the click of a button after the completion of the design but the user must insert some PVM parameters (e.g. message size) into the graphical/textual source to control the simulation. Additionally, the user may enter PVM/C code directly into the objects. The code generated by PVMGraph is predominantly a description of the software rather than the target 4
platform. An example of a PVMGraph window for the PPF (Parallel Processing Farm) model used to validate the toolset is shown in Figure 4.
Figure 4: PVMGraph main window. The current features of the prototype PVMGraph are listed below: 1. The most common PVM calls are implemented in the prototype version (see Table 1 later). However, the group functions are not yet implemented. 2. pvm spawn: The parameters for the number of tasks to spawn must be 1. Not checked by PVMGraph. The task name parameter must be manually entered. Must not put pvm spawn into a loop. 3. pvm kill: Fully implemented for design but not during visualisation. If the design uses pvm kill, the design will work (not tested) but it will not be visualised. 4. pvm send and pvm recv: If send/recv has dynamic connection then need to make sure that the sending/receiving task is correct and msgtags match otherwise visualiser may fail. Parameters not checked. 5. pvm gather and pvm bcast: Implemented for design but not for visualisation. Should not be used if the design needs to be visualised. 6. all other send and recv PVM functions are not part of PVMGraph prototype (e.g. pvm nrecv). 5
7. PVMGraph design can be saved and retrieved. 8. C Source code (.c les) can be generated in same directory as the location of the saved design. 9. the user can move graphical objects during visualisation and edit. However, the edit feature is supposed to only allow viewing. Currently, the view can change the code (do not change during visualisation). The code when viewed does not show during visualisation the current function being executed. 10. compiling/translating/linking PVMGraph to simulator: Able to compile the generated C source code (.c les) for a simulator to take place or real execution from within PVMGraph. 11. the graphical objects and textual les are stored separately to enable the users to re-use parts of other applications.
4 PVMVis Prototype Speci cations The main objective of this tool is to o er the user graphical views and animation representing the execution and performance of the designed parallel application. The animation view in PVMVis (see Figure 5) is similar to the design view in PVMGraph except that the pallet is not shown and two extra components for performance analysis are added. These components are barchart view and platform view. The barchart view shows historical states for the simulation and the platform view shows some statistics for selected performance measures at three levels: the message passing layer, the operating system layer and the hardware layer.
Figure 5: PVMGraph main window. Figure 5 Shows a snapshot of the animation window and the platform view of the PVMVis tool for the PPF application. The reader is referred to the original speci cations document [4] for 6
further details. Currently, the barchart view is not implemented but measures such as network load which is obtained from the simulator will be added soon. For the platform view three measures in the network layer, displayed in barcharts in Figure 5, are currently selected (some additions are planned for future versions): packets distribution, collision, and volume.
5 Speci cations of the Prototype Simulator 1. The prototype simulator is recompiled for SunOS5.4 (Solaris 2.4). 2. The platform model library must be recompiled before starting to build an executable model. For the prototype toolset, Workbench's scripts and make les only are required to build the platform model. An EDPEPPS shellscript is provided to build an executable model. In the future this shellscript will be replaced by a Make le provided by PVMGraph. 3. The current platform model (see Figure 6) has been recompiled to handle a virtual machine of 6 hosts. This parameter can be increased but the platform model must be recompiled. To avoid this the platform model can be compiled for a reasonable number of hosts (similar to real environment) and all machines or a subset can be used for simulation. SES/design 3.1 Index What How
Catalog:
pvmsim catalog
pvm apps 0.3
catalog
Thierry’s Main Catalog
test integration
MakeRules
pvmd
libgpvm
ipc
libpvm
host
Ethernet libmodel.a
globals.int
Figure 6: Platform model catalog page 4. There are restrictions on the use of SES/Workbench (currently CPC holds two licenses for each of Workbench's three tools. 5. A CPU model is available in the platform model, but the translator does not make use of it. The translator uses a delay node. 6. The PVM functions listed in Table 1 are implemented in the platform model and are functionally correct. There are no restrictions for the usage of any implemented PVM function. 7. The packing and unpacking functions do not take any simulation time. Time parameters will be addressed in future versions. Only a subset of encoding types are supported. 8. Some extra functionalities have been added to overcome the simulation of programs which require huge amount of data and to optimise the execution time of the simulation. A new set of encoding types have been de ned and are similar to the prede ned PVM encoding types. This allows PVM bu ers to be simulated without allocating memory. 7
proc control msg snd/rcv pvm mytid pvm exit pvm spawn
pvm recv pvm nrecv pvm send pvm bu nfo
msg buf
pvm initsend pvm mkbuf pvm freebuf pvm getsbuf pvm getrbuf pvm setsbuf pvm setrbuf
pack/unpack data pvm pkstr pvm pkbyte pvm pkint pvm pklong pvm pkdouble pvm pk oat pvm upkstr pvm upkbyte pvm upkint pvm upklong pvm upkdouble pvm upk oat
info
pvm parent
Table 1: PVM functions calls implemented in the simulator. 9. Possibility of using intrinsic Workbench simulation (SES/sim) features [6] through the SimPVM translator (probability distributions, random number generator, probabilistic modelling facilities and storage class). These features are available when a parallel program genrated by PVMGraph is translated into a Workbench graph le. Since the SES/sim language is based on C, C++ and some simulation constructs, the parallel program source code may be suitable only for simulation. 10. External loads could be de ned in the future. 11. No time parameters have been addressed for TCP communications within the same host. 12. Establishing TCP links between PVM processes is not allowed in the current version. 13. The allocation and deallocation of bu er do not take any simulation time. 14. The time required to fork and execute a UNIX process takes an arbitrary amount of time. 15. The socket services and the UNIX select system call are functionally correct but does not consume any simulation time. 16. The number of PVM tasks and PVM daemons is limited. This is due to the implementation of the simulated select system call which currently handles 32 le descriptors (1 socket = 1 le descriptor). For instance, when the simulation starts, each daemon requires two sockets, and when a task is created, two sockets are needed. For example, the PPF program (with 5 workers and a virtual machine of 6 hosts) would require 22 le descriptors. 17. The group library and the group server are not implemented. 18. If any run-time errors occur during the execution of the simulation, then the simulation engine should be stopped automatically. Run-time errors in the platform model are not always handled.
5.1 Event Trace File
5.1.1 Instrumented PVM Functions
A subset of PVM functions (pvm mytid, pvm exit, pvm spawn, pvm send, pvm recv, pvm bu nfo, pvm parent) has been instrumented to generate a Tape/PVM trace le (V0.9pl8). 8
5.1.2 Tape/PVM trace le format
Each Tape/PVM event contains 2 parts. The rst one is the header whose format is constant, independent of the event's type. The second is a variable part, depending on the event's type. Each part contains a number of named elds. The header part contains 6 elds. The variable part contains from 0 up to 7 elds. The event header contains the following elds: type task file line
: : : : : date_s : date_us :
event's type pvm task id of the event's generator id of the pvm source file which generated the event line number in the file containing the pvm library call which generated the event events global date in seconds... and in microseconds
The event type, as opposed to the event name, is an integer code named 'event < function >' where < function > is the name of the PVM function call where 'pvm ' has been stripped of. Example: the type of the event generated by 'pvm spawn' is 'event spawn'. 1 0 4 7 6 5 1
TapeMytidEvent TapeExitEvent TapeSpawnEvent TapeRecvEvent TapeSendEvent TapeBufinfoEvent TapeParentEvent
ret ret ret ret ret ret
flag ntask TaskList arrived delta_s delta_us msgtag tid msgtag bytes tid delta_s delta_us msgtag bytes bufid bytes msgtag tid
5.2 Platform Statistics
A statistics le is generated during the simulation. Platform statistics are updated when necessary and low level events may occur between PVM events. Statistics are written into an le. This ASCII le contains lines ordered by time. Each statistics line is composed of two parts. The rst one is a constant header. The second one is a variable part. The header contains 3 elds, the event time, the type of the statistics (e.g. Ethernet, CPU, PVM, etc), and an index value (e.g. host number).
5.2.1 Ethernet
The variable part contains the Ethernet load, the mean number of collisions per packet, and the Ethernet frame size in bytes. This allows the Ethernet load, mean number of collisions per packet, packet size distribution, and packet volume distribution to be plotted. The last three graphs are represented graphically as histograms in PVMGraph.
5.2.2 CPU
The variable part contains the CPU utilisation (mean,current,min,max,variance) and population (mean,current,min,max,variance).
6 SimPVM Translator Version 1 52 :
6.1 Major changes since the last version
Since the last version (see [8, 9]), the features of the SimPVM Translator have considerably changed. The translator can now handle standard C-Files with some limitations explained in this section. The command line is now: 9
translate -o results.grf file1.c file2.c file3.c.......filen.c
The rst le ( le1.c in our example) is the rst process to be executed when the simulation starts.
6.2 Features of the C-Grammar implemented in the SimPVM Translator version 1 52 :
Only signi cant improvements to the previous versions of the translator are mentioned here:
1. As the translator can handle regular C-Files, there is no need anymore to rename the \main" functions to \process". 2. Include les are ignored and not written to the .grf le. Common include les (stdio.h, stdlib.h, math.h...) are already de ned in SES Workbench. 3. #de ne macros are correctly written to the resulting .grf le. However, SES in its preprocessing phase, considers the macros as \global", i.e. common to every program of the design. There are several solutions to avoid this problem (not yet implemented in this version of the translator). One of these solutions would be to preprocess the macros during the translation phase or before. 4. Pointers (single, double or even triple) are supported. 5. C Functions are supported.
6.3 Limitations of Translator
1. Structures (struct or typedef) are not recognized by the translator. 2. Macros are supported but it is not advised to use them as they become global to every programs in the design (see previous section). 3. Preprocessing directives other than #de ne and #include (#ifdef, #ifndef, #endif and so on) are not supported. 4. The use \switch" and \case" is not supported. 5. The .grf les generated cause SES Workbench to crash when clicking on an icon with the draw tool.
References [1] T. Delaitre, G. Justo, F Spies, S. Winter, A Graphical Toolset for Simulation Modelling of Parallel Systems, Proc. 1st Austrian-Hungarian Workshop on Distributed and Parallel Systems (DAPSYS 96), Miskolc, Hungary, October 2-4 1996, pp 25-32. [2] G.R. Justo, PVMGraph: A Graphical Editor for the Design of PVM Programs, Technical report, EDPEPPS/5, University of Westminster, UK, February 1996. [3] E. Maillet, TAPE/PVM an ecient performance monitor for PVM applications - User guide. LMC-IMAG, ftp: image.image.fr in pub/APACHE/TAPE, March 1995. [4] F. Spies, Speci cation of the PVMVis visualisation tool, EDPEPPS EPSRC Project (GR.K40468) deliverable 2.3.1 (EDPEPPS/10), Centre for Parallel Computing, University of Westminster, London, July 1996. 10
[5] Scienti c and Engineering Software Inc, SES/Workbench User's Manual, Release 2.1, Scienti c Engineering Software Inc. 1992. [6] Scienti c and Engineering Software Inc. SES/workbench Reference Manual, Release 2.1, Scienti c Engineering Software Inc., 1992. [7] K. Sheehan and M. Esslinger, The SES/sim Modeling Language Proc. The Society for Computer Simulation, San Diego CA, July 1989, 25-32. [8] F. Spies, Speci cation of the prototype language SimPVM 1.0b EDPEPPS EPSRC Project (GR.K40468) deliverable 2.1.1, Centre for Parallel Computing, University of Westminster, London, November 1995. [9] F. Spies, Speci cation of the prototype language SimPVM 1.1 EDPEPPS EPSRC Project (GR.K40468) report 12, Centre for Parallel Computing, University of Westminster, London, July 1996. [10] P. Vekariya, PVMGraph: Design and Implementation. EDPEPPS EPSRC Project (GR.K40468) Technical report EDPEPPS/11, Centre for Parallel Computing, University of Westminster, London, October 1996.
11
A Listing of the Make le for the PPF Example The make le needed to compile the executable version of the PPF example is presented here. The PPF example is made of three source codes: - ppf master.c First stage of the PPF - ppf worker.c Second Stage of the PPF - ppf T3.c Third Stage of the PPF This Make le builds the executable les for the Tape/PVM instrumented version: ppf master t - ppf worker t - ppf T3 t It builds also the executable les for the non instrumented version: - ppf master - ppf worker - ppf T3 Here is the Make le: ##################################################### # Simple Makefile to instrument 'ppf_master.c' from the # PVM3 distribution (C version) # --------------------------------------------------# TAPE/PVM # E.Maillet, LMC-IMAG, University of Grenoble, France ##################################################### CC
=
gcc
# include and library directories PVMLIBDIR = $(PVM_ROOT)/lib/$(PVM_ARCH)/libpvm3.a\ $(PVM_ROOT)/lib/$(PVM_ARCH)/libgpvm3.a -lnsl -lsocket PVMINCDIR = -I$(PVM_ROOT)/include -I/packages/EDPEPPS/include PVMBINDIR = /home/horus/bigeard/pvm3/bin/$(PVM_ARCH) TAPELIBDIR = $(TAPE_ROOT)/$(PVM_ARCH)/libtape.a /usr/ucblib/libucb.a TAPEINCDIR = -I$(TAPE_ROOT)/include UCB
=
/usr/ucblib/libucb.a
# libraries PVMLIB = -lpvm3 -lgpvm3 -lsocket -lnsl TAPELIB = -ltape all: ppf_master ppf_master_t ppf_worker ppf_worker_t ppf_T3 ppf_T3_t clean: rm -f ppf_master ppf_master_t *.o ppf_master_t.c ppf_worker\ ppf_worker_t ppf_worker_t.c ppf_T3 ppf_T3_t ppf_T3_t.c
12
install: ppf_master ppf_master_t ppf_worker ppf_worker_t ppf_T3 ppf_T3_t cp ppf_master ppf_master_t ppf_worker ppf_worker_t ppf_T3\ ppf_T3_t $(PVMBINDIR)
#################################################### # build non instrumented version (as usual) #################################################### ppf_master: ppf_master.c $(CC) $(PVMINCDIR)
-o ppf_master ppf_master.c $(PVMLIBDIR) $(UCB)
ppf_worker: ppf_worker.c $(CC) $(PVMINCDIR) -o ppf_worker ppf_worker.c $(PVMLIBDIR) $(UCB) ppf_T3: ppf_T3.c $(CC) $(PVMINCDIR) -o ppf_T3 ppf_T3.c $(PVMLIBDIR) $(UCB) #################################################### # build instrumented version #################################################### ppf_master_t: ppf_master_t.c $(CC) -R/usr/ucblib $(PVMINCDIR) $(TAPEINCDIR) -o ppf_master_t\ ppf_master_t.c $(TAPELIBDIR) $(PVMLIBDIR)
ppf_worker_t: ppf_worker_t.c $(CC) -R/usr/ucblib $(PVMINCDIR) $(TAPEINCDIR) -o ppf_worker_t\ ppf_worker_t.c $(TAPELIBDIR) $(PVMLIBDIR) ppf_T3_t: ppf_T3_t.c $(CC) -R/usr/ucblib $(PVMINCDIR) $(TAPEINCDIR) -o ppf_T3_t\ ppf_T3_t.c $(TAPELIBDIR) $(PVMLIBDIR)
# build instrumented source from normal source ppf_master_t.c: ppf_master.c tapepp -fDB ppf_master.c ppf_worker_t.c: ppf_worker.c tapepp -fDB ppf_worker.c ppf_T3_t.c: ppf_T3.c tapepp -fDB ppf_T3.c
13
