OBJECTSIM - THE CONCEPTUAL MODEL OF A GENERIC PARALLEL & DISTRIBUTED COMPUTER ARCHITECTURE SIMULATOR.
Arnold Pears, †Edwige Pissaloux, Samar Singh, Martyn Hodgson, and Tharam S. Dillon School of Computer Science and Computer Engineering, La Trobe University Bundoora, 3083, Melbourne, Victoria, Australia. †IUT de Villetaneuse, Université Paris XIII, Ave. Jean-Baptiste Clément, 93430 Villetaneuse, Paris, France e-mail:
[email protected] KEY WORDS
paper outlines the conceptual design of OBJECTSIM, and demonstrates how the design addresses the key issues identified. The superiority of the OBJECTSIM tool is demonstrated using an example. The paper concludes that the OBJECTSIM approach has produced an efficient and powerful generic tool for simulating parallel and distributed systems. A final paragraph provides a brief discussion of future directions.
object-oriented simulator, generic distributed kernel, parallel and distributed systems. ABSTRACT
This paper describes the design and implementation of OBJECTSIM. OBJECTSIM overcomes the narrow focus of previous architecture simulators by specifying a small number of generic simulation objects (and an associated hierarchy) that are capable of implementing a wide range of target systems. Target systems range from multi-stage interconnection networks (MINs) and ATM and BISDN networks, through to bus and network based multiprocessors.
Simulation is one of the most important basic tools available to designers and implementers of large scale computers. Development of CPU and system architectures can no longer be realistically accomplished using prototyping approaches. There are two reasons for this. First, the cost of prototyping is very high in terms of both hardware fabrication expenses and time. Second, modern systems are becoming increasingly complex. Thus performance and system behaviour are much harder to predict and evaluate than was previously the case.
OBJECTSIM also provides trace, execution and stochastic driver mechanisms. To simulate large systems ( > 256 nodes ) OBJECTSIM uses a distributed object oriented simulation kernel. The combination of this kernel with the object hierarchy method makes OBJECTSIM a unique contribution to the field. OBJECTSIM runs on a network of SUN stations.
Architecture simulation provides a cost effective means to: analyse the performance of a system (in order to better understand its operation); predict system computational power (in terms of the processing capability available through its instruction set); and measure expected temporal performance. Simulation may also be used to evaluate system reliability and fault tolerance and provide support for real-time decisions and interactive training. Programmers also use simulation to evaluate novel programming models for future systems. Simulations are currently used to evaluate processor designs, computer networks, and novel multiprocessor architectures.
INTRODUCTION This paper describes the design and implementation of OBJECTSIM. OBJECTSIM represents a new flexible and generic approach to performance modelling for parallel and distributed systems. The design is based on a powerful combination of a generic object library (with its associated hierarchies) and a distributed simulation kernel. The introduction motivates the development of OBJECTSIM. Design choices are presented in the context of recent research and the problems with related tools are highlighted. The main body of the
Once simulated and precisely evaluated, a design (whether it be for a chip, computer , or network) can be realised with the most up-to-date technology. This allows designers to avoid situations where a final 1
system has been realised incrementally over many years of prototyping, resulting in a product which is obsolete before it reaches the marketplace. Moreover, new developments made in hardware, software or formal system design can easily be integrated into the system at successive stages of the design simulation.
of low level abstract entities like data flows and data transformations. Our work differs from that of Coomber by focussing on developing a high level library of generic classes which can be used to rapidly construct simulations of distributed systems. However, the approach sugessted by Coomber may well prove useful for specifying object class structures. This would allow OBJECTSIM simulation objects to be generated directly from abstract specifications.
DESIGN AND CONTEXT Considerable research into the design of Discrete Event Simulations (DES) exists (Page and Nance ????; Pallacia 1985; Reiher and Jefferson 1990; Veenstra and Fowler 1994). Recent research on the principles and structure of object oriented systems and simulation environments has also provided valuable insights (Coomber 1993; Dillon and Chang ????; Mirra 1986).
Recent architecture simulators include PROTEUS (Brewer 1992), TANGO (Davis, Goldschmidt and Hennessy 1991), DiST (Pears and Francis 1993]) and MINT (Veenstra and Fowler 1994). In this category DiST is unique in providing support for both sharedmemory and distributed-memory programming models, and a range of hardware configurations including memory allocation and mapping schemes and a range of interconnection network topologies.
Page and Nance (Page and Nance ????) make a number of design recommendations that are fundamental to the OBJECTSIM approach. They emphasise the importance of system correctness and testability, software quality, and the ability to provide a good match between the structure of the system being modelled and the forms of expression provided by the simulation environment.
These systems combine the simulation engine and system model in a single monolithic software system. Consequently these simulation suites lack flexibility and generality, especially as they were designed to evaluate hardware and programming model alternatives for specific multiprocessor designs. The ability of these tools to simulate large systems ( >256 nodes) is also limited by the sequential nature of their simulation kernels.
Tanir and Sevinc (Tanir and Sevinc 1994), establish a reference model which characterises the constructs of a simulation environment into five layers based on functional behaviour. They emphasise the choice of a general-purpose programming language and indicate that an object oriented languages with high level data structures is to be preferred. They caution that choice of a strongly typed language may impose restrictions that do not improve the modelling capabilities of the system. In this context C++ was the obvious choice for implementing the OBJECTSIM prototype. The object oriented nature of C++ also assists in realising the correctness and software quality requirements identified by Page and Nance.
In comparison OBJECTSIM provides a much more general framework for a general class of distributed and parallel system simulations. In that respect, and in its distributed nature OBJECTSIM is very similar to the Time Warp Operating System (TWOS) (Reiher 1990). Both are based on Jefferson’s (Jefferson 1985) work on virtual time and use optimistic simulation strategies. The most important differences are that OBJECTSIM is designed to execute on networks of workstations (rather than the BBN Butterfly), and to provide a fully object oriented simulation environment. TWOS objects are written in C. OBJECTSIM uses the flexibility provided by C++ to build a more completely object oriented system. This promises greater modularity and simulation software quality. OBJECTSIM also directly supports object reuse and refinement through the use of the inheritance and polymorphism facilities of C++.
Tanir and Sevinc define eleven requirements for the basic level of a simulation environment. The object hierarchy used by OBJECTSIM to model distributed and parallel systems meets these requirements. In particular the approach enhances abstraction by providing a natural mapping between system components and objects. Object behaviour can be defined using state transition diagrams or petri nets. These models completely define the behaviour of an object in terms of state change responses to incoming messages facilitating correctness checking.
Some recent literature notes that stochastically driven simulations of multiprocessor systems and interconnection networks can encounter accuracy problems since the probability distributions used to drive such simulations are often poorly related to the characteristics of actual program traces (Kim and Chien 1993; Davis, Goldschmidt and Hennessy 1990).
Coomber (Coomber 1993) provides a powerful technique that can be used to generate simulations from abstract specifications based on transformation schemas. Transformation schemas are defined in terms
2
This highlights the importance of program or execution driven simulations in certain contexts. OBJECTSIM addresses the need to permit a blend of stochastic, execution, and trace-driven simulation components (where appropriate) in the construction of system models.
Figure 1 gives an overall conceptual view of an OBJECTSIM simulation. Several important features shown in Figure 1 require further discussion. OBJECTSIM is designed to provide a combination of high flexibility and rapid prototyping. Flexibility is derived from the object library and object oriented approach to building simulations, while efficiency and parallelism are provided by the kernel. The flexibility of the object oriented nature of the simulation components is illustrated in the situation shown in Figure 1, where a simulation employs both execution and stochastic techniques to generate address traces at different places in the simulation model.
CONCEPTUAL MODEL
OBJECTSIM is founded on the concept of a hierarchy of generic simulation objects. These objects are abstractions of the building blocks common to all distributed and parallel systems. Through appropriate refinements this hierarchy provides the ability to model a wide range of systems. An important aspect of OBJECTSIM is the separation of the simulation engine from the system model. The simuation engine (OBJECTSIM kernel) manages the nodes which comprise a simulation and deals with all the host implementation issues.This approach facilitates the development of a parallel simulation kernel, substantially increasing the performance of individual simulations.
Rapid prototyping of simulations is assisted by providing a graphical simulation design tool. This tool will provide the simulation architect with a pallet of object classes and a visual workspace. Object instances can be selected from the pallet and placed in the workspace, where connctions can be drawn between then using the mouse. Clicking on an object instance will reveal its internal structure; a collection of methods and attributes. If necessary a new methods or attributes can be added, or existing methods refined to tailor the object to the precise needs of the simulation.
The simulation kernel is responsible for managing and distributing the nodes which make up the simulation, maintaining simulation correctness, and delivering the messages generated by the nodes. In a parallel simulation there is a kernel on each host machine.
To implement execution driven simulation nodes OBJECTSIM will use annotated source codes. A robust compiler originally implemented to generate annotated workload programs for DiST (Pears and Francis 1993; Pears 1994) can be adapted with minimal effort. The DiST compiler transforms programs written in a procedural parallel language (based on Modula-II) into small annotated code timeslices each represented by a C function. Execution of these functions generates appropriate simulation events, advancing the simulation in a realistic manner.
Simulation Design Tool Configuration File
Simulation Kernel Workload Generator
Scheduler
Router
Node
Interactive debugging and visualisation tools which interface with OBJECTSIM using an external access interface will be able to connect to a simulation during the course of an experiment. Existing tools used with DiST will provide a basis for the development of tools which present cpu workload and memory access statistics in a graphical format.
Mapper
External Interface
Node Workload File
Debugger
Statistics
Compiler
Having provided an overview of OBJECTSIM this paper now proceeds to discuss the two most important components of the system (the simulation kernel and object heirarchy) in some detail.
Figure 1: A Conceptual Overview of OBJECTSIM
An OBJECTSIM system model consists of a set of objects (nodes) and a mapping that defines how they are interconnected. Each node comprises an arbitrary number of queue objects which represent its logical input and output communication channels, and an execution method that is triggered by message arrival.
SIMULATION KERNEL The simulation kernel is responsible for managing and distributing the nodes which make up the
3
simulation, maintaining simulation correctness, and delivering the messages generated by the nodes. In a parallel simulation there is a kernel on each host machine.
node
source
Each kernel contains a router and a scheduler object which manage the simulation nodes. The scheduler stores the nodes resident in the kernel and provides the kernel with interfaces to acquire the node which matches the youngest outstanding event. The router stores the interconnection scheme which defines how the nodes of a simulation are connected together. In a parallel simulations a mapper is also implemented in each kernel. The mapper allows a kernel to discover the location of remote nodes; allowing events to be forwarded to the appropriate kernel.
trace
stochastic
sink
execution forward
bus
MIN
memory
switch
Figure 2: Object Hierarchy Modelling a memory module poses more of a problem. It receives and consumes data requests so it has some characteristics of a sink, but it responds by providing data ( a source). A solution can be obtained by viewing the operation of a memory module as a message transformation, in which case it becomes natural to class it as a forward. Using this insight we can see that a memory can be modelled as a specialised forward node that transforms data as it passes from input to output.
This kernel design allows a simulation architect to build objects without consideration for the global structure into which they might finally be incorporated. For example, a MIN simulation might be constructed by interconnecting nodes which represent switches in an appropriate topology. Message inputs to a switch trigger the local routing function which determines which output channel the message should be directed to using information in the message packet. The important thing to note is that the router need have no knowledge about the global structure of the network. This is important, as it guarantees that object definitions can be truly generic.
Each of the three node sub-classes has been further refined in Figure 2 to provide the basis for the implementation of a class library of standard components. This illustrates the flexibility of the top level classification scheme used. Note that specialisation of the source object class into three subclasses provides the range of driver methods required for all types of simulations. Since these different methods are implemented by specialising the source class they can be used in combination within a single simulation, and since they share a common external interface can replace one another if required. The properties and methods of the three major sub-classes are illustrated using the graphical terminology of Dillon and Tan (Dillon and Tan 1993) in Figure 3.
AN OBJECT HIERARCHY OBJECTSIM provides an hierarchical basis within which a library of general purpose simulation objects classes can be defined. Object instances can be interconnected and specialised to construct an accurate model of any target system. The object hierarchy is founded on the observation that all distributed and parallel systems can be viewed as composed of nodes which are interconnected in some way by communication links. At this level a node may be viewed as a state machine. State transitions are triggered by arriving messages and the resulting activity may produce one or more output messages. The function of each node in a simulation is defined by the structure of its state machine.
sou r ce
Nodes can be classified into three classes according to their primary function in a system, source, forward and sink. For example when modelling a bus based multiprocessor it is natural to classify the CPUs as address generators (sources), while the bus is obviously a transfer medium (forward). Figure 2 shows the principal classes defined by OBJECTSIM.
for w ar d
sin k
sta tisti cs
sta tisti cs
sta tisti cs
in p u t ch s
in p u t ch s
in p u t ch s
ou tp u t ch s
ou tp u t ch s
ou tp u t ch s
clock
d at a s tru ct
d eliver m essa g e
d eliver m essa g e
d eliver m essa g e
sen d m essa g e
sen d m essa g e
sen d m essa g e
g et sta tisti cs
g et sta tisti cs
g et sta tisti cs
con su m e m essa g e
ex ecu te
ro u te m essa g e
Figure 3: Specification of Principal Object Classes
4
From the formal specification in Figure 3 it is obvious that the majority of the attributes and methods are inherited from the node specification. Additional attributes have been provided in the source and forward classes to model time and provide a lookup data structure for routing. The execute, route, and consume methods replace a generic method in the parent which specifies a general interface.
simulation. Dotted lines represent control flows, solid lines are data flows. Object classes use the convention established in Figure 2.
AN EXAMPLE
The interconnection topology is supplied as a startup file to the simulation kernel. This ensures that the routing object can be based on a generic lookup table based algorithm. The schedular in this simulation steps all the active nodes (those with messages in their input buffers) in each simulation timestep. This might not be an appropriate strategy for other simulations. For example a multi-processor architecture simulator might wish to step nodes one at a time in event timestamp order. This can be quickly and easily achieved by replacing the scheduling policy method in the schedular object.
The objects used in this simulation are largely derived from standard classes. Minimal tailoring is necessary to implement specific routing choices and buffer sizes in the switches.
It is interesting to explore the power of this heirarchy by investigating how a MIN simulation might be designed. First the components of the system are characterised into the three primitive classes. Input packets for the network will be generated by stochastic sources, and consumed by sinks. The sinks will also collect statistics based on packet transmission information. External tools collect these statistics and compute measures of interest, such as average delay and throughput. Each switch in the network will be represented by a forward node. Let us consider the design and implementation of each of thesesimulation components in more detail.
CONCLUDING REMARKS This paper presents the conceptual design and application of OBJECTSIM, a generic object oriented tool for simulating the behaviour and performance of parallel and distributed systems and networks.The design has the following key features;
Input source nodes comprise a single input and two logical output channels. One output channel is connected to the input channel, the other to the input of a switch in the network. Arrival of an event on the input channel activates the execution method in the node. If the output channel is free a packet is generated and sent to the network. The source also generates an execute event for itself (time stamped in the future) and sends that on the other channel. Addresses and future timestamps are generated using standard probability distributions.
1. A class library of suitably defined objects, and hierarchies of these, which can be assembled into a suitable system model using simple composition operations. The system takes maximal advantage of polymorphism in ensuring that the simple construction scheme develops sufficient representational power to permit a large range of systems to be accurately modelled. 2. A high level visual specification interface designed to manage the selection, editing, and graphical composition of objects. This will facilitate rapid implementation of simulations. 3. A choice of deterministic and stochastic driving mechanisms. Trace and execution driven techniques are used to provide deterministic input, while a range of standard probability distributions allow network phenomena such as hot-spots to be tackled in the conventional manner.
Figure 4: Components of a MIN Simulation
MIN switches are modelled as forward nodes and consist of two input and two output channels and a data packet routing method. They are connected together into a standard MIN topology. The routing method is activated by the existance of messages in one or more input buffers. It evaluates the state of the input and output buffers on each channel and decides which messages can be transfered from input to output buffers. Figure 4 shows the structure of the final
The flexibility of the proposed object hierarchy has been presented at length in the context of modelling a simple Multi-stage Interconnection Network. The integration of the object class hierarchy with a
5
distributed simulation kernel based on Virtual Time provides the basis for a high performance simulation tool with a wide range of applicability. In addition, as work proceeds a library of simulation objects will develop facilitating object reuse and decreasing simulation implementation overhead. A prototype of the kernel has been implemented on a network of SUN work-stations using C++.
Knowledge and Data Engineering 6, no. 3 (June): 412~419. Misra, J. 1986. “Distributed Discrete-Event Simulation.” Computing Surveys 18, no. 1 (Mar.) 39~65. Page, E. H.; and R.E. Nance. 19??. “Parallel Discrete Event Simulation: A Modeling Methodological Perspective.” In ???
Other areas which remain to be investigated are provision of further generic object classes, automatic object generation from abstract specifications, and the implementation of the graphical simulation design interface.
Pears, A.; and R.S. Francis. 1993. “Execution driven Multicomputer Simulation : the DiST Experience.” In Proceedings of the International Conference on Modelling and Simulation (Melb., Aust.), VUT Melb. Aust., 29~38.
REFERENCES
Pears, A. 1994. “Odin: A Study in Simgle Address Space Multiprocessing.” PhD Thesis, La Trobe University.
Brewer, E.A.; C.N. Dellarocas; A. Colbrook; and W.E. Weihl. 1992. “PROTEUS : A High Performance Parallel Architecture Simulator.” Technical Report TR-516, MIT.
Pollacia, L.F. 1985. “A Survey of Discrete Event Simulation and State-of-the-Art Discrete Event Languages.” In Simulation Digest (Mar.): .8~25.
Coomber, C.J. 1993. “An Object-Oriented Procedure for Simulating Real-Time Systems.” In Technology of Object-Oriented Languages and Systems (TOOLS) Pacific ’93. TOOLS USA, Goleta, CA., 269~278
Reiher, P. L.; and D. Jefferson. 1990. “Virtual Time Based Dynamic Load Management in the Time Warp Operating System.” Transactions of the Society for Computer Simulation 7, no. 2 (June).
Davis, H.; S.R. Goldschmidt; and J. Hennessy. 1990. “Tango: A Multiprocessor Simulation and Tracing System.” Technical Report CSL-TR-90-439. Computer Systems Laboratory, Stanford University.
Reiher, P. L. 1990. “Parallel Simulation Using the Time Warp Operating System.” In Proceedings of the 1990 Winter Simulation Conference, IEEE Picataway N.J.
Davis, H.; S.R. Goldschmidt; and J. Hennessy. 1991. “Multiprocessor Simulation and Tracing Using Tango” In Proceedings of the International Conference on Parallel Processing. Aug 1991
Tanir, O.; and S. Sevinc.1994. “Defining Requirements for a Standard Simulation Environment.”, IEEE Computer 27, no. 2 (Feb): 28~34
Dillon, T.S.; and P.L. Tan. 1993. Object-oriented Conceptual Modeling. Prentice-Hall Australia.
Veenstra, J., E.; and R.J. Fowler. 1994. “MINT : A Front End for Efficient Simulation of Shared-Memory Multiprocessors.” In Proceedings of the 2nd Intrnational Workshop on Modelling, Analysis and Simulation of Computer and Telecommunication Systems MASCOTS (Durham, NC, Jan.). 201~207
Dillon, T.S.; and E. Chang E. 19??. “Object-Oriented Technologies and Engineering.” In ??? Dupuy, A.; J. Schwartz; and Y. Yemini. 1990. "NEST: A Network Simulation Testbed." Communications of the ACM 33, no. 10 (Oct.). Jefferson, D. 1985. “Virtual Time.” ACM Transactions on Programming Languages and System 7, no. 3. Kim, J.H.; and A.A. Chien. 1993. "The Impact of Packetisation on Wormhole-Routed Networks." In Proceedings of PARLE'93, Lecture Notes in Computer Science 694, Springer Verlag, 242~253. Lu, X-M.; and T.S. Dillon. 1994. “An Algebraic Theory of Object-Oriented Systems.” IEEE Trans. on
6
