PROTEUS: A High-Performance Parallel-Architecture Simulator 1

PROTEUS:

A

High-Performance A.

Eric Adrian Parallel

1

Parallel-Architecture

Brewer

(lhrysanthos

Software

N.

William

Colbrook Group*,

NIIT

Laboratory

Simulator

Dellarocas

E. Weihl for

Computer

Science

Introduction PROTEUS

machines. ecutes

is an execution-driven

Like

most

mance. lished

simulations

of simulated PROTEUS vides nally,

accuracy

extends

from

code

the

PROTEUS

too

exploits

its

the nCUBE, the

system,

and for

augmentation

to

with

network memory part

and

exp!licit

also prevents

ahead

in simulated simulation. is available

structure

time, [1].

structure

aspects provides

it allows

users

architecture.

Second,

mod-

protocols,

run-

It

is simple

architecture

while

to ex-

the modular

structure

of a given

by Science

Fellowship.

I Hop-by-hop

I

I

I

I

1,500.000

Caching Cache

Numbers

700,000

1,000,000

400,000

500,000

120,000

are in simulated

cycles per second.

Table 1: The relative system performance of the six combinations of network and cache modules for the 8-queens application running on an 8x8 mesh. The simulations were run on a DECstation 5000. switch between very accurate versions and very fast versions. The required accuracy of a module depends on the end goals of the simulation. For example, users studying scheduling require very accurate costs in the

for

six combinations

module,

promotes which

multiple allows

and Engineering

Email:

brewer@lcs.

Research

im-

users

to

CCRby an Eric

Fellowship, and Adkian

Council

Postdoc-

mit. edu

of cache and

There is more than a ten-fold between

network

difference

the least and most accurate.

modules.

in performance Most simulations

achieve well over one million simulated cycles per second, since high accuracy is not usually required during development. Later, users can switch to more accurate modules

3

without

modifying

their

code.

Performance has an overhead

PROTEUS

for simulating

Brewer is supported by an Office of Naval Research Chris Dellarocas by a Starr Foundation Fellowship, Colbrook

Cost

Coherent

keeping

*Supported by the National Science Foundation, grant 8716884; by DARPA, Contract NOO014-89-J-1988; and equipment grant from Digit al Equipment Corporation.

toral

and

The

such as fulllempty

operations.

of the

simu-

machine,

cache

attributes

two

to tador

We have

multiprocessor. (or bus),

of

unchanged.

plementations

I Analytical

operating system module but may not need detailed net work simulation. The ability to switch among versions of a module provides a simple way to trade accuracy for performance. Table 1 shows the relative system performance

to simplify

of specific

a message-passing

and memory

atomic

periment

Fi-

accuracy

profiling,

modular

a shared-memory

and

display

performance

of the

First,

to a particular

the rest

far

modular

abilities.

PROTEUS

bits

and

“Augmentation

customization

lated

time

Uniform No

pro-

Modularity

The ant

ules control

it

debugging.

to tradeoff

on PROTEUS

and

simulator.

Alewife,

and

maximum

the accuracy

and

both

tool:

collection

detection,

report

replacement import

I

customization.

use of code

costs.

getting

increases

Versatility

the

the

stack-overflow

A technical

very

range

users

Model

requirements.

cent rol of user threads

pub-

a wide

for data

allows

I

produces

several

allows

N’etvvorlc

perfor-

structure

monitoring

to achieve

ex-

has reproduced

parallel-systems

support

high

PROTEUS

is a complete

PROTEUS

which

very

and simplifies

PROTEUS

2

and

nonintrusive

provide

achieve

I

for MIMD

[2], it directly

architectures

performance their

RPPT speed

Its modular

integrated for

to

exceptional

results.

simulator

[3] and

instructions

Despite

accurate

and

Tango

sequential

of several hundred struction.

Overall,

of about

a factor

code, compared

for simulators

with

that interpret

of two a factor each in-

PROTEUS is 10 to 100 times

faster

than such simulators. With PROTEUS, only about 4% of the simulation time is spent in user code. The network and cache simulation require most of the time, which explains the effectiveness of the high-performance modules. Unlike Tango lightweight-threads

[3], PROTEUS package that

uses a prevents

custom context

switches from dominating the simulation overhead. Context switching is reduced to about 4% of the overhead, which improves system performance by an order of magnitude.1 1A new version

247.

Performance

Evaluation

Review,

The threads of Tango

package

uses lightweight

Vol. 20, No. 1, June 1992

performs threads.

partial

context switches if the switch occurs at a procedure call boundary. Invariants hold at procedure boundaries that limit ically, 4

the amount

of state that must be saved. Typ-

98% of all context

switches

very little time. Typically, it takes only a few minutes to define a new data type and specify its interpretation using the graph language.

The

are partial.

Validation

Evidence for the accuracy of PROTEUS comes from several sources. PROTEUS reproduced published results [5] comparing sorting algorithms on an nCUBE/7. We used the original code, modified to use our runtime-

or other

system

copy.

interface,

costs to match

and adjusted those

of the

the simulated nCUBE/7.

hardware The

timing

results

were within 370 for every data point. PRoTEUS also reproduced published results [4] that compared locking algorithms on shared-memory machines (Sequent Symmetry, BBN Butterfly). In general, any effect that we expected to see has actually appeared. Equally important is that all unexpected results have (so far) proven to be real effects rather 5

than

inaccuracies

Monitoring

introduced

and

by PROTEUS.

Debugging

A vital asset of PROTEUS is its support for monitoring and debugging. Real multiprocessors suffer from the addition of monitoring code may the probe effect cause the monitored effect to disappear. This prevents programmers from collecting additional data for debugging. PROTEUS provides nonintrwsive monitoring and debugging: users can add arbitrary debugging code without affecting the behavior or timing of the simulation. PROTEUS’ deterministic nature and its support for nonintrusive debugging allow users to repr~ duce bugs despite the presence of additional debugging code. PROTEUS can reproduce Although deterministic, multiple executions of a nondeterministic application, an ability unique to PROTEUS among multiprocessor simulators. PROTEUS chooses pseudo-randomly between two events with

the same timestamp;

such events

are viewed as a race condition. Since the decision is pseudo-random it is deterministic, which ensures repeatability. At the same time, changing the seed affects the outcome of the race conditions and thus leads to a different 6

Data

execution

of the nondeterministic

Collection

and

application.

simple but powerful graph language that specifies how to interpret the trace file. In addition to several predefine trace-file data types, users can define to generate high-quality

their own, which application-specific

248.

Performance

allows users graphs in

Evaluation

generator

architectural

the X Window

7

produces

line

graphs,

bar

parameter.

The generator

system and produces

uses

POSTSCRIPT

hard-

Conclusion

provides a unique combination of flexibility, performance, and accuracy. Its modular structure PROTEUS

simplifies customization and independent replacement of individual parts of the simulator; this promotes modules for specific architectures and multiple implementations to provide a variety of performance and accuracy combinations. The accurate versions of modules allow PROTEUS to reproduce published results. The validation experiments

provide

a significantly

increased

level of confi-

dence in PROTEUS’ results. PROTEUS ensures repeatability even in the presence of additional monitoring or debugging code, which allows users to track down bugs incrementally. In addition, nonintrusive monitoring supports both performance tuning and accurate data collection. The data collection and display tools give PROTEUS an unusual level of effectiveness – users can collect and display exactly the data they need. The support for user-defined data types and user-specified graphs provides full access to the insight lation. The primary

available

through

simu-

use of PROTEUS so far has been the de-

sign and implement ation of a p ort able parallel language and runtime system. It also has supported research on concurrent algorithms, synchronization, operating systems, and fault tolerance. This work shows that effect ive for real applications: one study involved 3000 simulations of a 10,000-line application. [1]

E. A. Brewer, W. E. Weihl.

C. N.

Dellarocas,

A.

it is over

Colbrook,

and

“PROTEUS: A High-Performance Parallel-Architecture Simulator.” MIT Technical Report MIT/LcS/TR-516, Sept. 1991.

Display

PROTEUS also provides an integrated subsystem for data collection and display. PROTEUS’ graphics capabilities make it simple to evaluate algorithms and architectures: users can create graphs quickly that answer their questions and provide new insight. The key is a

graph

graphs, and tables, and can combine multiple graphs onto the same axes. It also can merge data from multiple simulations; this simplifies comparison of an algorithm across a range of architectures, machine sizes,

[2] R. G.

Covington et Testbed.” Proceedings ference, May 1988.

[3] H. Davis,

“The

S. R. Goldsdunidt, and Tracing Using

Simulation

’91, Aug. 1991. [4] J. M. Mellor-C rummey without

al.

Rice

Processing

and

Con-

J. Hennessy. “Multiprocessor Tango.” P~oceedings of ZCPP M. L.

Proceedings

Contention.’!

Parallel

of the 1988A CM SIGMETRICS

Scott.

“Synchronization

of A SPLOS-IV,

April

1991.

[5] M. J. Quinn. Sorting

29(2):239-250,

Review,

Vol.

“Analysis

Algorithms:

20,

and

Benchmarking

Hyperquicksort

1989.

No.

1, June

1992

of Two

and QuickMerge.”

Parallel

BIT,