ON THE MULTILEVEL SOLUTION ALGORITHM FOR ... - NTRS - NASA

/

NASA

Contractor

ICASE

Report

Report

/

201671

No. 97-17

ANNIVERSARY

ON THE MULTILEVEL SOLUTION ALGORITHM FOR MARKOV CHAINS

Graham

NASA March

Horton

Contract 1997

No. NAS1-19480

Institute for Computer Applications NASA Langley Research Center Hampton, Operated

National Space

VA

and Engineering

23681-0001

by Universities

Aeronautics

Space

and

Administration

Langley Hampton,

in Science

Research Virginia

Center 23681-0001

Research

Association

/

ON THE ALGORITHM

MULTILEVEL SOLUTION FOR MARKOV CHAINS

Gl'ah

all|

(?omputcr

Scicn('e

[!niversily

l)el>artnlent

of Erlangen-Niirnl)erg Martensstr. 3

91058 graham@

nortoll"

Erlang('n.

inf ormat

(',erlnany

ik. uni-erlangen,

de

Abstract \\'e slate

discuss

lh('

solution

recently

of Markov

cil)le whi('h

iltlroduced

(:ha i,s.

is well esl,al)lish('(1 correction,

which

uses

an ol)eralor-(lel)endent

wl,ich

],as been

the

algorithm

is seen

lead

1.o lhe

('oars(,-lev(,l

"This Iration

gley the

research

was

NASA (:ompuler

Research

(:enler,

author

was

supl)orled (lontract

for

l)elmrl.menl

of the

When

operator.

I)rin-

l)rinciple.

a muhi-l('v('l significa.tly

lnel]lO(] faster

('a.sl as a muh.igri(l-likc Approximation SI)e('ial

steady-

a mull il)licalix'(.

aggr('galioll

yields

to be a (',alerkin-Full

('a.ncellal.ion

t'eatur(.s

to give results

in use.

for lhe

on an aggregation

S('h(,ln(,

properlies

of lhe ('Oml)ul.ationally

wil.ll

of lhis

inl.(,nsiv(,

|hall

melho(l, a

l)rolon -

l.(,rms of th('

equations.

m,d('r

[nstitule

and

coarsening,

prolongation

algorithm

is based

al)l)lication

exp('rimentally currently

solution-del)endenl galion

t{ecursive

shown

l.yi)i(:a.l methods

method

in th(' lit('ralur('

('oars('-h'v('l

the

multilevel

The

Hami)toll.

University

the

Na!ional

and

"2:1681-00()1.

ProDssor

of I)enver,

Aeronautics

while

in Science VA

Assistant

by

NASI-Iq,180

Applications

a Visiting

of lhe

iu part No.

author

Engineerillg The

at. the

Denver,

tlw

(:().

work

Mathematics

aml was

Spa(','

(I(:ASE), was

also and

Adminis-

in resid(m('e

al th, _

NASA

Lan-

out.

whil,'

carried (:omputer

Science

1

Introduction

Markov

chains

abilities state

describe

of transitions

between

of tile chain

al_proxima.tely widely the

discrete-state

in stochastic

performance

modeling. and

they

rates

described can

as

be converted is usually

The

goal

formance,

availability, a. certain

also

lhe

repairs.

tic Petri

nets

and

delined

today,

often

in this

as the in the tile

((;SPNs)

are

is used

pal)er

from

It is COmlnon

[1] and such

throughpul

is required about the

t.o solve the Markov abstract model. number

o[" time lools

a system

is al)h' and

to derive

of the

systems the

Markov

chain

are

in

stochasnlemoryless

chains,

useflll

to I)et'possibly

generalized

to Markov

in order

of states

are

on perutilizalion,

failures

[6]. When

equivalent

chain

information

for compul.er

of which networks

are

by a

of equations

oi" component

of componenl,

queueing

is

l)roblems

described

system

is to derive

proportion

models

chain

("I_M(I

problems

a linear

where

Markov

larg_,.

syst.ems

modeling

(1)TM(Is),

the case,

into

extremely

tllosl, inll)ortant

is satisfied,

the

steady-state

presence

abstract

chains case,

represents

as job

condition

l Tnfortunately,

chain

computer

function

use

current

property

chains

systems.

Markov latter

transformation

typh'ally

Various

widespread

our examples

time

In the

Markov

sparse

measuretl

form

this

Markov

of computer

In the

matrix.

of modeling

and

discrete

via. a simple

l Tltin]ately,

which

and

i)robal)ilities.

by a stochastic

DTM(:.

modeling

of lhe

Since

systems,

the prob-

continuous time Markov chains (('TM(Is), in which between states are interl)reted as exponentially dis-

or delays,

a.l'¢' treated

physical

in which solely

property.

\Ve will draw

reliability

processes

a.re a function

memoryless

by many

t.o distinguish between transition coefficients t,rilmt.ed

states

the so-called

satisified

stochastic

and

il

information (and

thus

the

dimension of tile linear system) grows extremely quickly as lhe complexity of the model is increased. There is one unknown for each state that the model

may

Thus,

the

COml)uter fl'om their

be in - a nulnber Markov

in every of the

that

is subject

haw?

on a, wide

of a computer, few months, system

are

range

in which whereas measured

of scales. the rate the rates

in kHz

to a combinatorial

to be solved

models may have tens or hundreds size, one further drawback of typical

of coefficients model

chains

that

_nd

for relatively

coa.rse

of thousands of states. :\parl. Markov chains is the presence

(:onsider,

of component a.ssociated MHz.

even

explosion.

for exa.ml)le, failure with the

a reliability

may

be only once

normal

1)ehaviour

The resulting ing an iterat.ive modelling

community

relaxation found

(SOR)

in [12, S].

require

many

l.em is large can lead

All of these

methods

to reach

principle

of iteralive solution

or the

states

Algebraic

are

mapped

mulligrid

strat, egy for lhe

successive methods iha! are

overmay

they

particularly

nlagnitude

us-

computer |)e may

if the

sys-

presenl..

This

times. (ML) The

solulion method

disaggregalion, chains

algorithm is based

for

on the

a well-established

[7, l_i. 14]. This

nu-

principle

uli-

correction thai is multil)licalive, rather tlmn addicoarse-level x,a.lues are used as _ factor t)v which

st.val.eg.v are

and

and

dry, whack

solution,

in [5].

for Markov

ill the

used

the

a multilevel

aggregation

coarsening

((',S),

varying

introduced

technique

a.pI)rOxinmliolls

coupled

an accurate

of slrongly

was

a coarse-h,vel level i.e. newly ol)lained

fine-lewq

have

numerically

in use

of currenllv

long computation

which

I)¢' solved

(lauss-Seidel

Surveys

we will consider

chains,

merical

methods

are the Power.

io unacceplably

Markov

IIIllSl,

iterative

algorithms.

il.era.tions

paper,

of equat, ions

'l'ypical

or if coetticienl.s

In this

lizes live,

large systems

scheme.

resca.led.

|;'urthermorc.

lhe

aggregatiolh

ill lhat

localh"

strongly

t.o I,_, an attractive

solution

is operalor-dependelfl,

it,self.

together.

(AM(',)

svsloms

[13] is considered

of equal.ions

thai

are

llliSlrllcl,

tlrod

a, lld

which

may

have strongly varying colem

itself

In (.h(' following equations.

The

lh(' multilevel soclion

This

level equations

vighl-hand

side,

from

5 exl)erinlenlal

for the

(FAS)

l.yl)e.

nlethod

ol)era.t.or,

which

combined

from

the coarse itera.l.ion

coarse When

is seen

level Ol)erviewed

as a

1o si.enl fronl

is solul.ion-del_elldenl llDe left or from

has lwo inier('sting d('g('n('rat('s

chains. t,o an algebraic

and the

righl

et['('('i s: the righl-hand

int.o a simple

restri('lion

of t.h('

level ot)('ral of is solution-del)endenl

Io i(.eralion,

('v('n

lhough

the

Markov

is linear. sect.ion,

mul(.il(,vel

melhod

and

method

Scheme

Markov

is equivalcnl

of tim mullihwel

of the prolongation operalor

method

Galerkin

Al)prability

of the

IJv , i --I-u2, and

wilily

._7 is lhen

being

i)robal,ilities

The

A z illay

a llOW Markov

]illlil)illg

corresponding

ilS io g_oileratc

w'e will tile

1.

the

t,i'allsitioii

chain

,_; C 5;1 sigiiifies operation

the

,st'the nnlltilevel method lies in the observationthat, if il is possibleto obtain improved valuesfor the relatire l)robabilil,ies of finelevel states in a comlnol_aggregal.e,then a.nimproved coarse-levelsystem can be set,up and solved,the solution of which t'epresenl.slhe prol)al)ilities of the aggregates.The a.rgunlentcan. of course.1_¢,apl)lied recursively. We choose

to form

stal.es,

small

a.s the (_a.uss-Seidel

relal.ive

l)robabilit.ies

method

in this

different.. are

aggregates

This

smoothed

iteration

of such

conl; j

which

case

consequence,

= R,,I:P

COml_ut¢ lh_ (l,l)

1 i.s in lh_ .1-th

usual

to A t d:fin:d

A l-I Proof:

to the

fl,,ltP 1, 0 .....

a.s th( 0)

From ObservationsI and 3, we draw the following: Observation 4 mulligrid

Th¢

ML

method

i._ tquival:nl

of t h(- above

relationships

have

Haviv [4] discusses various iteratiw' and (ll), and Krieger [8] points out two-level

multigrid

If we consider lar problenl equalion,

with

we may

Observation cHhttion

right

lhc fi,llowing

hand

(9) of lht

a*td lhird

coarse

side (the

r(,sl ricl.ed

is ('()llSlalll

situation

_lll(l

is rew'rse(l:

solul.ions

of the

ML

ask

I)t'operlies

a.1)proximatio]_

ofll_e and

well lhe

defoe!

o[" the coarse

nmlrix

sysl('nl

solution.

II)

(/"lh_ r_ htlir_

ma:/nit udes

the

m('lho(t,

and

nlalrix.

('Oml)ulal.iollal

The

will depend

successive

on the

is sub(lualily

equations

il is worlhwhile

1t,,' converged

malrix

are 1,o (3)

u t- 1 can t,_, lo l lw COl_Verg;ed

b: 1o lh_ _.racl solution

oJ"all ._olulio_ 12

problems,

presenl

is collsl.allt,

lo itel'ali()l_,

solution

,t_ appro;rimation

I_ lh,'

,Since lh(' coats(,

(1) al)l)roxinlal.cs

Definition

,h.[i_

to c_llt-

I)y lhe chang-

of lllc riglli hand side I)ro(lllcl I)('r iieralion.

ML lnel.]_od

iteration

is driven

). ]:or linear

function

changing

w_rv from

coarse

had,,

.,i& oJ" (12).._/i_hli_z:l

sx'sl(,nl

I)re('olnl)ul.('(l.

])e

how ('los(' Ill(" ('On_l)Uied

._moolh,

(12)

,

o/,ralor

Hghl ha,d

the coarse

an(I, lher(,fore, solul ion u 1-_ 1

level equation:

prolong,lio,

line-level

lllaV

I)ya

convergen(c

how

]'oisson

t-, ) = iU.l

l.h(' forcing

are driv(u[

solul.ion-dCl)endenl

out (9) schemes

sucll as a discretized coarse

saving of l.h(' ]all_'r s_']_(,1_l_,in the e\'aluat.ion sl.antial: one fine and on(' coarse ma.trix-vecl.or The

pointing lAD

(tnd ma!t br p_'rcomput_d:

mull igri(l lnethods

hand

authors;

hm to a non singu-

R.4:fi t + IL, t:I'H_):

le:rm._ in lht

tl.] "l uq_ich i._ co,._lant

matrix

side,

algoril

l:A,S-(_alerkin

I¢AL p(

ing righl

I)v previous

nol(" Ill(' tollowing.

of lhr ._co,d

In traditional

noted

multigrid

= H ff -

5 Proptrlq

th_ v_('tor

been

aggregation schemes, the relalionshil)between

the equivalent

a 1loll-trivial

we obtain

for which

lh(,

(;alerkin

algorithms.

al)l)lying

/L, ltp(.:-l)

coarse

I"AS

ilt ralion.

Some

and

to a c_rlain

_,alu_., u,ilhi_

u _ 1o b_

_o('h aggr_gal_

ort

i,

trror

by ,o

more

than

0(l)eared

(lire('l

(;+q,

their

iJnl)ortant

In order

over

explicitly:

task.

mosl

niodelling froni

11, ea('[l

alibi

(hi('

[+'igllre

these

a liiuliipro('essor

rel)air

pl'O('('SSOl'S

method

via

(lev('loped

inipracli(al th('

Petri-n(qs.

nolw'ork

111(' lwo

Balbo

an

used,

queueing

is

('Oli1111Oll

Xlarsall.

MI_

lv

reccnl

]lave

this

ar('

a sl,oCllastic

:{ shows

l']×ample

are seldonl

a niulliprocessor

Petri-liels

Figure

make

sl.ochasli('

well-known

has

slochaslic

1, l)r

and

indir('('l,

Pelri-nel

Multiprocessor

chains

i)ara(lignls

ef[i('ien('v define(I

Markov

in general

modelling

.

Results

niodelling,

al)sl.ra('l

L

3: Simple

Experimental

n

_CM 2

._

Figure

]_

lhe

svSlelll

(;,%

and

of

lhe

of

t]oatin_

units. power

of The

ot" ltt('

l'¢,SOlll'('('s. .MI,

paranielers i)oinl

nielliods are

al)laken

Ol)erations

Figure

needed

to solve

number

,1: (',SPN

the

problem

of processors

chains

varied

from

,v.,..

ill the

('Oml)aring (_S is quile problelns increases. The

dramatic: considered.

SOl{

sl.ochastic

GS and small

of l)roblem

model.

nmul)er

nets

I)('('ause tile Ol)limum

S 1 with

service

ra.l.e of 10 from

schemes,

the

saving

a factor

which

leaving

represenl.

I/O

of st.a.l.es of the

Ma.rkov

we see

that.

the used

gap

world

with service

effort

and

l)robl('nl

the situation

for processing. system.

for lhis

for

problem,

to ])e one.

re('eivos.jobs

There

size

fools

syslem consisting of three in a computer performan('e

leaw" the

1,5

as the

in softwa.r(,

I)a.ra.mel,er is found

rates

t ll('s(" arc of ML over

of 10.9 for the la.rgesl

as a solver

not. improve queueing 1)e used

All l)rol)h'lns

a.lthough

widens

rate .q0 r('l)resent, s a (7!P l: which

SI ma_" then

devices

as the

in computational

that.

is usua.lly

overrelaxa.l.ion

measured

J0.

of 27 for the smallesl,

[2, 10], does

the outside

l.hal, jobs

< l,-

ML

l:igure (i shows a slnall open S('l'vor (|ll('U('S. as might l.ypically Server

The

size

t.o :lNN:] (10 processors).

I1. is a.lso clear

method,

Pet.ri

size.

Svslem

a.s a function

oa

the

of very

of Multipro('essor

.ql (2 processors)

.,,

I)robh'lns

Model

a,t

singlemodel. a l]l('an

is a 70% probal)ilily

Servers

7 a lk(1,5, respectiwqy.

$2

and Tilere

S:] might is a 10_S,

3000 2500 2000 operations

15O0 1000 500 (} 2

0

,1

6

_

# of processors

Figure

5: (:Omlmt_llion_tl

1.o solve

chance have

lhe

A.inlollc

tha.1 a job leaving a finite

Cal)acily

c and

which

of a

t)I)E.

l,he Markov

is somewhat Each

st.ale

(*_1, /_2, _:_), wh¢'r,' I)e int.erl)rel('(l

dimensional

"grid"

reject

incolning

of

the

and

MI, (lower

jobs

processing.

when

has a regular

chain

may

the

number index

The

full.

112-

I}:_

It I,

I/2.

I}:_

I/[,

II 2,

117,

I/l,

112,

1/'1,

112,

\.\.'_' assume

I)e characterized of .iol,s

i_ queue

l]lat

allowing

us

sl.ruc-

discrelizal.ion by

the

vector

i. which

nla.y

in lit(' i-i h ,tilu(,]Lsi(nl of the thr('('-

lrallsit

ions conl ained

(ll

1 4-

-'_

(1}

I

11:¢,

--+

('l

ll:,_

--+

('_1 + 1.._..:;-

16

queues

lhree-dimensional

in the chain

following: (l/l.

curve)

The

distributed,

to 1]lai of a finil.c-elemcn!

as a +(Jri,aill

solulion

titetl,od

('lass of

l)roi)l('liis

Th(' iiicl ho(t is niolivat('(t

aggr(,gation-(lisag;gr('galion

for as a

i echniques

10000

t--1000

_

j-

100

o

/

/

10

/

8 1

0.1

0.01 5

Figure curve:

8: Numerical (',auss-Seidel;

to include

multiple

It is shown with

ol)eral.or.

one and

that

may

the

soltttion,

algorithm

coarse Tile

it leaves

the relative

useful

including,

solution

for example,

coarse

The

algorithm

is shown

obtaining

performance

tude

work

i)rolongalion lnanner,

been ol)served,

35

Nlultitasking

and

o[' tlle

values

40

Model.

(Tpper

met.hod l)rolon -

schelne

probabilities fraction produce

t.o l)erl_)rm

ensures

t)ounded

within

of PI)Es

may

to an FAN multigrid a solution-del)endellt

that. are

mass

corrections

with

aggregates similar

problems, under-

coarse

zero and

unchanged. s on the

tra,dil, ional

over-shoots.

to endent significantly

itlRorililln

faster

is seeii

t>l)craior. Sl>