Automated generation of reliability models - NASA Langley Formal ...

Automated G e n e r a t i o n of R e l i a b i l i C y Models S a l l y C . Johnson; NASA; Hampton Ricky W. B u t l e r ; NASA; Hampton Key Words: R e l i a b i l i t y model g e n e r a t i o n , Markov models, R e l i a b i l i t y a n a l y s i s .

Abstract Markov models can be used t o a n a l y z e t h e r e l i a b i l i t y of v i r t u a l l y any f a u l t - t o l e r a n t s y s t e m . However, t h e p r o c e s s of d e l i n e a t i n g a l l of t h e s t a t e s and t r a n s i t i o n s i n t h e model of a complex system can be d e v a s t a t i n g l y t e d i o u s and e r r o r - p r o n e . The A b s t r a c t Semi-Markov Speci f i c a t i o n I n t e r f a c e t o t h e SURE Tool (ASSIST) program a l l o w s t h e u s e r t o d e s c r i b e t h e Markov model i n a highl e v e l language. I n s t e a d of l i s t i n g t h e i n d i v i d u a l s t a t e s of t h e model, t h e u s e r s p e c i f i e s t h e r u l e s g o v e r n i n g t h e behavior of t h e system, and t h e s e a r e used t o a u t o m a t i c a l l y g e n e r a t e t h e model. A small number of s t a t e m e n t s i n t h e a b s t r a c t language can d e s c r i b e a v e r y l a r g e , complex model. Because no a s s u m p t i o n s a r e made a b o u t t h e system b e i n g modeled, ASSIST can be used t o g e n e r a t e models d e s c r i b i n g t h e b e h a v i o r of any t y p e of system. The ASSIST program and i t s i n p u t language a r e d e s c r i b e d and i l l u s t r a t e d by examples. I n t r oduct i o n N e w advances i n c o m p u t a t i o n , s u c h a s t h e Semi-Markov Range E v a l u a t o r (SURE) program, e n a b l e t h e a c c u r a t e s o l u t i o n of e x t r e m e l y l a r g e and complex Markov models ( R e f s . 1 and 2 ) . ( I n t h i s p a p e r , t h e term Markov w i l l be used t o r e f e r t o both Markov and t h e more g e n e r a l semi-Markov models.) However, t h e g e n e r a t i o n by hand of t h e l a r g e models needed t o c a p t u r e t h e complex f a i l u r e and r e c o n f i g u r a t i o n behavior of most r e a l i s t i c f a u l t t o l e r a n t a r c h i t e c t u r e s has been a n i n t r a c t a b l e problem. Much r e s e a r c h h a s been done on t e c h n i q u e s f o r model pruning and s t a t e a g g r e g a t i o n t o s i m p l i f y t h e models, a t t h e expense o f a c c u r a c y ( R e f s . 3 and 4 ) . Many of t h e e a r l y f a u l t - t o l e r a n t a r c h i t e c t u r e s , s u c h a s t h e S o f t w a r e Implemented F a u l t T o l e r a n c e (SIFT) system, a r e r e l a t i v e l y s i m p l e t o model. Even t h e e a r l y complex systems were u s u a l l y made up of subsystems t h a t can be modeled i n d e p e n d e n t l y . However, a s f l i g h t c r i t i c a l s y s t e m s become more complex and more h i g h l y i n t e g r a t e d , t h e Markov models t o d e s c r i b e them w i l l become enormously complex. The c o m p l e x i t y of t h e model stems from t h e i n t e r a c t i o n s between f a i l u r e and r e c o v e r y p r o c e s s e s of t h e v a r i o u s s u b s y s t e m s , which can no l o n g e r be modeled i n d e p e n d e n t l y . Often even t h e most Complex c h a r a c t e r i s t i c s of a system can be d e s c r i b e d by r e l a t i v e l y s i m p l e r u l e s . The models o n l y become complex b e c a u s e t h e s e few r u l e s combine many times t o form models w i t h l a r g e numbers of s t a t e s and t r a n s i t i o n s between them. The r u l e s d e s c r i b i n g t h e behavior of e a c h subsystem can be developed and v e r i f i e d s e p a r a t e l y , t h e n t h e submodels a r e e a s i l y combined t o a c c u r a t e l y model t h e behavior of t h e e n t i r e i n t e g r a t e d system. An a b s t r a c t , h i g h - l e v e l language f o r d e s c r i b i n g system behavior r u l e s and a methodology f o r a u t o m a t i c a l l y g e n e r a t i n g Markov models from t h e l a n g u a g e were developed by R i c k y B u t l e r ( r e f . 5 ) . The ASSIST computer program (Ref. 6 ) a l l o w s t h e u s e r t o s p e c i f y t h e behavior r u l e s of t h e model i n t h i s a b s t r a c t l a n g u a g e , t h e n t h e Markov model is g e n e r a t e d a u t o m a t i c a l l y from t h e r u l e s . The ASSIST program was w r i t t e n i n P a s c a l and e x e c u t e s on a VAX 11/750. The Markov model is o u t p u t i n t h e format r e q u i r e d f o r i n p u t t o t h e SURE program. For Markov a n a l y s i s programs

1988

r e q u i r l n g a d i f f e r e n t form o f i n p u t f o r t h e Markov model, a s i m p l e program c o u l d be w r i t t e n t o modlfy t h e model d e s c r i p t i o n f i l e . The a b s t r a c t model d e f i n i t i o n and t h e a u t o m a t i c model g e n e r a t i o n s t r a t e g y a r e d e s c r i b e d . A n a l y s i s of a n example f a u l t - t o l e r a n t a r c h i t e c t u r e , a t r i a d of p r o c e s s o r s w i t h c o l d s p a r e p r o c e s s o r s , shows how t h e behavior of a system c a n be c a p t u r e d by a few g e n e r a l r u l e s . The s y n t a x of t h e ASSIST i n p u t language i s t h e n d e s c r i b e d and d e m o n s t r a t e d by c r e a t i n g a model t o d e s c r i b e t h e f a u l t behavior of t h e example a r c h i t e c t u r e . The f l e x i b i l i t y of t h e a b s t r a c t language i s demons5rated by expanding t h e example t o model m u l t i p l e t r i a d s Of p r o c e s s o r s s h a r i n g a pool of c o l d s p a r e p r o c e s s o r s . Model D e f i n i t i o n A Markov model c o n s i s t s of a number of s y s t e m s t a t e s and t r a n s i t i o n s between them. Each s t a t e i s d e f i n e d by a s t a t e v e c t o r , where e a c h element of t h e v e c t o r t a k e s on an i n t e g e r v a l u e w i t h i n a d e f i n e d r a n g e . An element can r e p r e s e n t any meaningful c h a r a c t e r i s t i c , s u c h a s t h e number of good components of one t y p e i n t h e system, O r t h e number of f a u l t y components of a n o t h e r t y p e i n u s e . Each element is a s s i g n e d a n a p p r o p r i a t e v a r i a b l e name f o r e a s e of r e f e r e n c e . The s t a t e s p a c e v a r i a b l e s f o r t h e model and t h e i r v a l i d r a n g e s a r e d e f i n e d i n t h e "spacess s t a t e m e n t . The u s e r s p e c i f i e s t h e i n i t i a l system s t a t e i n t h e " s t a r t " s t a t e m e n t . T h i s e s t a b l i s h e s t h e i n i t i a l v a l u e s of t h e s t a t e s p a c e v a r i a b l e s f o r t h e g e n e r a t i o n of t h e model. The t r a n s i t i o n s r e p r e s e n t t h e e l a p s e d time between system s t a t e s which a r e s t o c h a s t i c p r o c e s s e s d e f i n e d by I n t h e r e s t r i c t e d c l a s s of probability distributions. Markov models, a l l t r a n s i t i o n s a r e e x p o n e n t i a l l y d i s t r i b u t e d and a r e c o m p l e t e l y d e f i n e d by a s i m p l e r a t e p a r a m e t e r . I n t h e more g e n e r a l semi-Markov model, any d i s t r i b u t i o n c a n be used t o d e s c r i b e t h e e l a p s e d time. T r a n s i t i o n s between s t a t e s i n t h e model a r e s p e c i f i e d u s i n g t r a n s i t i o n s t a t e m e n t s . These s t a t e m e n t s have t h r e e main p a r t s : a c o n d i t i o n e x p r e s s i o n , a d e s t i n a t i o n e x p r e s s i o n , and a r a t e e x p r e s s i o n . The f i r s t e x p r e s s i o n is a boolean e x p r e s s i o n t o d e s c r i b e t h e s t a t e s p a c e v a r i a b l e v a l u e s of s t a t e s f o r which t h e t r a n s i t i o n i s v a l i d . The second e x p r e s s i o n d e f i n e s t h e d e s t i n a t i o n s t a t e f o r t h e t r a n s i t i o n i n terms of s t a t e s p a c e v a r i a b l e v a l u e s . The t h i r d e x p r e s s i o n d e f i n e s t h e d i s t r i b u t i o n of e l a p s e d t i m e f o r t h e t r a n s i t i o n . Absorbing s t a t e s of t h e model ( i . e . , s t a t e s w i t h no t r a n s i t i o n s l e a v i n g them) r e p r e s e n t system f a i l u r e . T y p i c a l l y , t h e r e l i a b i l i t y e n g i n e e r must c a l c u l a t e t h e p r o b a b i l i t y of e n t e r i n g a n a b s o r b i n g s t a t e w i t h i n t h e s p e c i f i e d m i s s i o n time. The a b s o r b i n g s t a t e o r Ifdeath" c o n d i t i o n s of t h e model must be d e f i n e d i n terms of s t a t e s p a c e v a l u e s . These f s d e a t h l Tc o n d i t i o n s c o u l d be system f a i l u r e o r t h e o n s e t of d e g a d e d performance o p e r a t i o n o r o t h e r s i t u a t i o n s r e s u l t i n g from f a i l u r e s . The ASSIST program r e a d s a n i n p u t f i l e c o n t a i n i n g t h e model d e f i n i t i o n r u l e s and c r e a t e s two o u t p u t f i l e s : the model f i l e and a n o p t i o n a l l i s t i n g f i l e . The model f i l e d e s c r i b e s a l l of t h e t r a n s i t i o n s between s t a t e s i n t h e Markov model and t h e i r c o r r e s p o n d i n g r a t e s i n t h e The format r e q u i r e d f o r i n p u t t o t h e SURE program. s t a t e s a r e s p e c i f i e d by i n t e g e r s . To make t h e model

U.S. Government work not protected by U.S. Copyright. Annual RELIABILITY AND MAINTAINABILITY

PROCEEDINGS

Authorized licensed use limited to: IEEE Xplore. Downloaded on April 15, 2009 at 09:18 from IEEE Xplore. Restrictions apply.

Symposium

17

easier t o u n d e r s t a n d , t h i s f i l e i s a n n o t a t e d w i t h t h e s t a t e s p a c e v a r i a b l e s of e a c h s t a t e i n comments, which SURE w i l l i g n o r e . For example: 1 ( * 6,0,0

*),

2 ( * 5,1,0

* ) = 3*LAMBDA;

d e n o t e s a t r a n s i t i o n between s t a t e 1 and s t a t e 2 a t e x p o n e n t i a l r a t e 31, where s t a t e 1 i s d e f i n e d i n ASSIST a s ( 6 . 0 . 0 ) and s t a t e 2 i s (5,1,0). I n a d d i t i o n t o a l i s t i n g Of t h e ASSIST i n p u t f i l e , t h e o p t i o n a l l i s t i n g f i l e c o n t a i n s a l i s t of t h e d e s t i n a t i o n s t a t e of each a r c l e a v i n g e a c h non-death s t a t e i n t h e model. D e s t i n a t i o n s t a t e s t h a t a r e d e a t h s t a t e s a r e a n n o t a t e d w i t h a n a s t e r i s k . The l i s t i n g f i l e a l s o c o n t a i n s a l i s t o f t h e mappings between t h e SURE s t a t e numbers and t h e s t a t e s p a c e v a r i a b l e s of t h a t s t a t e i n ASSIST. The o p t i o n a l l i s t i n g f i l e i s u s e f u l f o r v e r i f y i n g t h a t t h e model g e n e r a t e d a c c u r a t e l y d e s c r i b e s t h e i n t e n d e d system b e h a v i o r . C o n s t a n t s and v a r i a b l e s based o n s t a t e s p a c e v a r i a b l e s can be d e f i n e d i n t h e i n p u t l a n g u a g e and used i n s u b s e q u e n t s t a t e m e n t s . S t a t e m e n t s i n t h e ASSIST i n p u t f i l e t h a t , a r e put i n s i d e q u o t e s a r e c o p i e d i n t o t h e model o u t p u t f i l e and a r e n o t o t h e r w i s e p r o c e s s e d by ASSIST. These s t a t e m e n t s a r e p u t i n t h e f r o n t o f t h e SURE i n p u t f i l e w i t h t h e c o n s t a n t d e f i n i t i o n s t a t e m e n t s . Comments may a l s o be i n c l u d e d i n t h e i n p u t f i l e . ASSIST Model G e n e r a t i o n The ASSIST program b u i l d s t h e model from t h e i n i t i a l "start" s t a t e by r e c u r s i v e l y a p p l y i n g t h e t r a n s i t i o n r u l e s . B e f o r e a p p l i c a t i o n of a r u l e , ASSIST checks a l l of t h e "death" c o n d i t i o n s t o see i f t h e c u r r e n t s t a t e i s a "deathTt s t a t e . S i n c e a "death" s t a t e d e n o t e s system f a i l u r e , no t r a n s i t i o n s can l e a v e a " d e a t h " s t a t e . A l l of t h e t r a n s i t i o n rules a r e t h e n e v a l u a t e d , and t r a n s i t i o n s t o new s t a t e s a r e g e n e r a t e d where a p p r o p r i a t e . When a l l p o s s i b l e branches t e r m i n a t e i n a "death" state. model b u i l d i n g i s complete. The o u t p u t f i l e c o n t a i n s a d e f i n i t i o n of e a c h t r a n s i t i o n a n d i t s rate. A l i s t i n g f i l e is a l s o g e n e r a t e d t o a s s i s t t h e u s e r i n d e t e r m i n i n g u h e t h e r t h e model g e n e r a t e d d e s c r i b e s t h e intended s y s t m behavlor. The s p e c i f i c a l g o r i t h m used t o g e n e r a t e t h e model i s a s follows. I n i t i a l l y . t h e READY SET c o n t a i n s o n l y t h e "start" state of t h e model. Each s t a t e i n t h e READY SET is p r o c e s s e d in t h e Pollowlng manner. If t h e s t a t e meets any of t h e " d e a t h m c o n d i t i o n s . t h e n t h a t s t a t e i s a 8tdeath1ts t a t e , and no t r a n s i t i o n s can l e a v e i t , so t h e s t a t e is removed f r c m t h e READY SET. If t h e s t a t e is n o t a "death*' s t a t e . t h e n e a c h t r a n s i t i o n r u l e is a p p l i e d t o t h e s t a t e in t h e f o l l o w i n g manner t o g e n e r a t e a l l p o s s i b l e t r a n s i t i o n s l e a v i n g t h e state. If t h e

Figure 1 .

18

1988 PROCEEDINGS

c o n d i t i o n e x p r e s s i o n of t h e t r a n s i t i o n r u l e e v a l u a t e s t o t r u e f o r t h e current s t a t e , then t h e destination e x p r e s s i o n i n t h e r u l e i s used t o d e t e r m i n e t h e d e s t i n a t i o n s t a t e . If t h i s s t a t e i s w i t h i n t h e bounds of t h e s t a t e s p a c e p a r a m e t e r s , t h e n t h i s i s a v a l i d t r a n s i t i o n . If t h e d e s t i n a t i o n s t a t e h a s n o t a l r e a d y been d e f i n e d i n t h e model, t h e n a unique i n t e g e r i s a s s i g n e d t o t h e new s t a t e , and i t i s added t o t h e READY SET. I f t h e d e s t i n a t i o n s t a t e was a l r e a d y d e f i n e d i n t h e model, t h e n i t was p l a c e d i n t h e READY SET f o r p r o c e s s i n g when i t was f i r s t d e f i n e d . T h e r a t e of t h e t r a n s i t i o n i s d e t e r m i n e d from t h e r a t e e x p r e s s i o n , and t h e t r a n s i t i o n d e s c r i p t i o n i s p r i n t e d t o t h e SURE model f i l e . A f t e r a l l of t h e t r a n s i t i o n r u l e s have been a p p l i e d t o t h e s t a t e , i t i s removed from t h e READY SET. Example A r c h i t e c t u r e The example a r c h i t e c t u r e c o n s i s t s of a t r i a d of p r o c e s s o r s each e x e c u t i n g t h e same program p l u s a pool of two c o l d s p a r e p r o c e s s o r s . The t h r e e p r o c e s s o r s r e c e i v e i d e n t i c a l i n p u t s so a l l n o n - f a u l t y p r o c e s s o r s produce t h e same o u t p u t , a n d t h e t h r e e o u t p u t s a r e voted. Any i n c o r r e c t o u t p u t s a r e masked by t h e v o t i n g a s l o n g a s a m a j o r i t y of t h e a c t i v e p r o c e s s o r s a r e nonf a u l t y . A f a u l t y p r o c e s s o r i s d e t e c t e d by t h e v o t e r and is r e p l a c e d w i t h a c o l d s p a r e p r o c e s s o r i f o n e i s a v a i l a b l e . For s i m p l i c i t y , t h i s p r o c e s s i s assumed t o be e x p o n e n t i a l f o r t h i s example. There i s no f a u l t d e t e c t i o n f o r s p a r e p r o c e s s o r s u n t i l t h e y become a c t i v e . The Markov model t o d e s c r i b e t h i s system is shown i n figure 1. The s t a t e s i n t h e example model a r e d e s c r i b e d by t h e v e c t o r (NP, NFP, NS, NFS), where NP NFP NS NFS

= = =

=

Number Number Number Number

of a c t i v e p r o c e s s o r s , of f a i l e d a c t i v e processors, of c o l d s p a r e p r o c e s s o r s , a n d of f a i l e d c o l d s p a r e p r o c e s s o r s .

The f a u l t and r e c o v e r y b e h a v i o r of t h e example s y s t e m i s d e s c r i b e d by t h e f o l l o w i n g r u l e s : 1 . The f a i l u r e r a t e of e a c h a c t i v e p r o c e s s o r i s 1. 2 . The f a i l u r e r a t e of c o l d s p a r e p r o c e s s o r s i s Y . 3. A f a i l e d a c t i v e p r o c e s s o r is r e p l a c e d by a s p a r e processor a t r a t e 6. 4. System f a i l u r e o c c u r s u n l e s s a m a j o r i t y o f t h e a c t i v e processors a r e non-faulty. Rules 1 t h r o u g h 3 above d e s c r i b e t h e t r a n s i t i o n s between s t a t e s i n t h e model. The f o u r t h r u l e d e s c r i b e s t h e t'deathll s t a t e s of t h e model. The system s t a r t s w i t h t h r e e n o n - f a u l t y , a c t i v e p r o c e s s o r s and two n o n - f a u l t y cold s p a r e processors, thus t h e s t a r t s t a t e is ( 3 , 0, 2 , 0).

Markov model f o r a t r i a d w i t h two c o l d s p a r e s .

Annual

RELIABILITY

AND

MAINTAINABILITY

Authorized licensed use limited to: IEEE Xplore. Downloaded on April 15, 2009 at 09:18 from IEEE Xplore. Restrictions apply.

Symposium

e x p r e s s i o n may c o n t a i n FOR l o o p v a r i a b l e s and s t a t e space v a r i a b l e s . There a r e t h r e e ways of e x p r e s s i n g t h e d i s t r i b u t i o n of e l a p s e d time between s t a t e s i n t h e SURE program t h a t a r e s u p p o r t e d i n ASSIST. Slow e x p o n e n t i a l t r a n ? i t i o n s a r e s p e c i f i e d by t h e t r a n s i t i o n r a t e . F a s t t r a n s i t i o n s may be s p e c i f i e d b y two d i f f e r e n t methods: W h i t e ' s method o r t h e f a s t e x p o n e n t i a l method. The s y n t a x f o r W h i t e ' s method c o n s i s t s of t h r p e r e a l e x p r e s s i o n s :

C r e a t i n g a n Example I n p u t F i l e The b a s i c s t a t e m e n t s used i n ASSIST a r e : SPACE s t a t e m e n t START s t a t e m e n t DEATHIF s t a t e m e n t TR A NTO s t a t em en t s FOR s t a t e m e n t s PRUNEIF s t a t e m e n t s