developing software tools for pre- implementation ...

DEVELOPING SOFTWARE TOOLS FOR PREIMPLEMENTATION TESTING OF DESIGN SUPPORT TOOLS E. Z. Opiyo, I. Horváth and J. S. M. Vergeest 'HSDUWPHQWRI'HVLJQ(QJLQHHULQJ'HOIW8QLYHUVLW\RI7HFKQRORJ\ -DIIDODDQ1/%;'HOIW7KH1HWKHUODQGV HRSL\R#LRWXGHOIWQO ABSTRACT Clear understanding of requirements and performing evaluation prior to implementation can lead to realization of high-quality design support software tools. The problem is that most tests are done on prototypes. Not all abstract implementations e.g. the underlying theories and methods are formally tested. This can influence quality of design support tools. As an attempt to deal with this problem, we have developed a pre-implementation testing methodology that has been named abstract prototyping. Case applications have demonstrated that this methodology can effectively help developers to (i) quickly elicit and formulate requirements, and (ii) systematically test abstract software implementations (i.e. as software product evolves), by systematically involving various stakeholders as subjects. In this paper, we concisely introduce the abstract prototyping concept and present the framework of software for supporting developers in pre-implementation testing of design support tools and give an example on how to use them. 1 INTRODUCTION 7KH QHHGV LQ PDUNHWV FDOO IRU DGRSWLRQ RI PRUH HIIHFWLYH PHWKRGV DQG VRIWZDUH VXSSRUW WRROV WKDW FDQ KHOS LPSURYH TXDOLW\ ORZHU GHYHORSPHQW FRVWV DQG UHGXFH WLPHWRPDUNHWIRUSURGXFWV9DULRXVDXWKRUVHJ>@ DGYRFDWHWKDWPDQ\VRIWZDUH GHYHORSPHQWHIIRUWVGRQRWVXFFHHGLQGHOLYHULQJJRRGTXDOLW\SURGXFWVRQWLPHDQG ZLWKLQEXGJHWV3RVVLEOHUHDVRQVIRUWKLVSUREOHPLQFOXGHL OHQJWK\UHTXLUHPHQWV HOLFLWDWLRQ DQG IRUPDOL]DWLRQ WUDGLWLRQV LL  IX]]\ XQGHUVWDQGLQJ RI UHTXLUHPHQWV DQG LLL  OHQJWK\ DQG H[SHQVLYH WHVWLQJ SUDFWLFHV 0RVW RI WKH FXUUHQW WHVWLQJ WHFKQLTXHVLQYROYHVXVDJHRISURWRW\SHVZKLFKSURYLGHIHHGEDFNIRUWKHGHYHORSHUV WRR ODWH 0HDQZKLOH QRW DOO LQFLGHQWDO LPSOHPHQWDWLRQV DUH IRUPDOO\ WHVWHG 7KLV FDQ LQIOXHQFH TXDOLW\ RI GHVLJQ VXSSRUW WRROV )XUWKHUPRUH VRPH RI WKH FXUUHQW WHVWLQJ WHFKQLTXHV HJ XVDELOLW\ WHVWV WHQG WR LJQRUH RWKHU VWDNHKROGHUV DQG HPSKDVL]HRQWKHDSSUDLVDORIIXOILOOPHQWRIWKHHQGXVHUVUHTXLUHPHQWV'HVSLWHWKH IDFW WKDW WKH GHYHORSHUV DUH VRPHWLPHV LQIRUPDOO\ LQYROYHG LQ WKH UHYLHZV WKH H[LVWLQJWHFKQLTXHVQHYHUWKHOHVVGRQRWSURYLGHDV\VWHPDWLFZD\IRUEULQJLQJWKHP DORQJZLWKWKHHQGXVHUVDQGRWKHUVWDNHKROGHUVLQWRWKHWHVWVIORRU :HKDYHGHYHORSHGDFRPSXWHUEDVHGSUHLPSOHPHQWDWLRQWHVWLQJWHFKQLTXH WKDWKDVEHHQQDPHGDEVWUDFWSURWRW\SLQJ,WDVVLVWVWKHGHYHORSHUVLQGHILQLQJDQG PDLQWDLQLQJFOHDUXQGHUVWDQGLQJRIUHTXLUHPHQWVDQGVXSSRUWVWKHPLQSHUIRUPLQJ HYDOXDWLRQ SULRU WR LPSOHPHQWDWLRQ ,Q WKLV SDSHU ZH GHVFULEH WKH DEVWUDFW

SURWRW\SLQJ FRQFHSW DQG VRIWZDUH WRROV EXLOW WR VXSSRUW WKH GHYHORSHUV LQ SUH LPSOHPHQWDWLRQ WHVWLQJ :H GHVFULEH WKH VWUXFWXUH RI WKH DEVWUDFW SURWRW\SLQJ VRIWZDUHDQGXWLOLWLHVLWRIIHUV:HDOVRJLYHDQH[DPSOHRQKRZWRXVHWKLVFRQFHSW 2 A CONCISE STATE OF THE ART REVIEW 7KHUH DUH PDQ\ WHFKQLTXHV IRU DVVXUDQFH RI VRIWZDUH TXDOLW\ 6RPH RI WKHP DUH DXWRPDWHG LH FRPSXWHU EDVHG UHIHU HJ WR >@ >@  DQG RWKHUV DUH QRW 7KHVH WHFKQLTXHV GLIIHU LQ SDUWLFXODU PHWKRGV DFWLYLWLHV DQG SKDVHV RI GHYHORSPHQW LQ ZKLFK WKH\ FDQ EH XVHG 6RPH RI WKHP VHUYH QRW RQO\ DV VKHOIZDUH  EXW DOVR SURYLGH VRSKLVWLFDWHG IXQFWLRQDOLW\ IRU LQVWDQFH IRU FRGH JHQHUDWLRQ DQG UHYHUVH HQJLQHHULQJ 2WKHU W\SLFDO IXQFWLRQDOLW\ SURYLGHG E\ WKHVH WRROV LQFOXGH IRU H[DPSOHIRUEXLOGLQJDGHWDLOHGVRIWZDUHGHVLJQPRGHOIURPZKLFKLWFDQEHWUDFHG KRZDQGZKHUHDUHTXLUHPHQWLVIXOILOOHG7KHSUREOHPLVWKDWQRWDOOUHTXLUHPHQWV UHODWHGLUHFWO\WRIXQFWLRQV,WLVWKHUHIRUHGLIILFXOWWRXVHVXFKWRROVWRWUDFHZKHWKHU RUQRWWKHUHTXLUHPHQWVWKDWDUHQRW GLUHFWO\FRQQHFWHGWRWKHIXQFWLRQVKDYH EHHQ IXOILOOHG$QRWKHUSUREOHPLVWKDWDXWRPDWHGWHVWLQJPHWKRGVWHQGWRH[FOXGHXVHUV IURP WHVW IORRUV $OVR DXWRPDWLRQ VRPHKRZ GLVWDQFHV WKH H[SHUWLVH RI WKH GHYHORSHUV IURP WHVW UHJLPHV 1RQDXWRPDWLF WHFKQLTXHV IRU WHVWLQJ QHZ VRIWZDUH FRQFHSWVLQFOXGHIRUH[DPSOHXVDELOLW\HQJLQHHULQJWHFKQLTXHVVXFKDVFRQWH[WXDO LQTXLU\ DQG SDUWLFLSDWRU\ GHVLJQ 7KHVH PHWKRGV DGGUHVV WKH XVHU V SDUWLFLSDWLRQ DVSHFWV LQ WKH GHYHORSPHQW E\ PDNLQJ WKHP SDUW RI WKH GHYHORSPHQW WHDP 7KH\ W\SLFDOO\IRFXVRQ JDLQLQJIHHGEDFNIURPWKHHQGXVHUVWKURXJKHQYLVLRQPHQWDQG H[SORUDWLRQ RI WKH XVDELOLW\ SUREOHPV WKH\ IUHTXHQWO\ H[SHULHQFH LQ WKHLU ZRUNLQJ HQYLURQPHQWV HJ >@ >@ >@  7KH XVDELOLW\ HQJLQHHULQJ WHFKQLTXHV WHQG WR LQYROYH WKH HQGXVHUV H[FOXVLYHO\ ZKR W\SLFDOO\ GRHVQ¶W NQRZ DERXW WKH LQIUDVWUXFWXUH RI WKH VRIWZDUH  DQG WKHUHIRUH IRFXV HQWLUHO\ RQ WKH H[SORUDWLRQ RI IXOILOOPHQW RI WKHLU QHHGV 7KLV RYHUZKHOPLQJO\ OHDGV WR WHVWLQJ RI WKH HQGXVHUV UHTXLUHPHQWV ,Q JHQHUDO PRVW WHVWV UHTXLUH VRPH VRUW RI LPSOHPHQWDWLRQV 7KLV PDNHVWKHPOHQJWK\DQGH[SHQVLYHSURFHVVHV 5LJRURXV DSSURDFKHV DUH UHTXLUHG WR HQVXUH TXDOLW\ RI GHVLJQ VXSSRUW VRIWZDUH WRROV ZKLFK DUH FXVWRPDULO\ EXLOW EDVHG RQ WKHRULHV DQG PHWKRGV 7HVWV QHHG WR EH GRQH DW YDULRXV OHYHOV RI DEVWUDFWLRQ RI D VRIWZDUH WRRO DQG DOO VWDNHKROGHUV PXVW V\VWHPDWLFDOO\ EH LQYROYHG :KDW LV QHHGHG LV VWUXFWXUHG GHYHORSPHQWDQGWHVWLQJIUDPHZRUN VWDUWLQJ IURP WKH KLJKHVW OHYHO RI DEVWUDFWLRQ RIWKHXQGHUO\LQJWKHRULHVIROORZHGE\PHWKRGVDOJRULWKPVSLORWLPSOHPHQWDWLRQV DQG ILQDOO\ IXOO SURWRW\SHV ,Q WKH IROORZLQJ VHFWLRQ ZH JLYH DQ RYHUYLHZ RI D FRQFHSW QDPHG DEVWUDFW SURWRW\SLQJ ZKLFK SURYLGHV D IUDPHZRUN IRU SUH LPSOHPHQWDWLRQWHVWLQJRIDEVWUDFWGHVLJQVXSSRUWVRIWZDUHLPSOHPHQWDWLRQV 3 AN OVERVIEW OF THE ABSTRACT PROTOTYPING CONCEPT $EVWUDFWSURWRW\SLQJFDQEHGHVFULEHGDVDUHTXLUHPHQWVEDVHGFRPSXWHUVXSSRUWHG PHWKRGRORJ\ IRU HYDOXDWLRQ RI DEVWUDFW VRIWZDUH LPSOHPHQWDWLRQV ,W SUHVHQWV D IUDPHZRUNE\ZKLFKDOOVWDNHKROGHUVDUHV\VWHPDWLFDOO\LQYROYHGLQWKHHYDOXDWLRQ RI YDULRXV DEVWUDFW LPSOHPHQWDWLRQV DV WKH QHZ VRIWZDUH SURGXFW HYROYHV 7KH

VWDNHKROGHUV  LQYROYHPHQW VFKHPH LV GHVFULEHG LQ >@ 7KH SXUSRVH RI DEVWUDFW SURWRW\SLQJLVWRYHULI\DQGYDOLGDWHQHZVRIWZDUHFRQFHSWV:HUHFRJQL]HWKHRULHV PHWKRGVDOJRULWKPVDQGSLORWLPSOHPHQWDWLRQVDVWHVWDEOHSURWRW\SHV)LJVKRZV WKHUHDVRQLQJPRGHOIRUWKHDEVWUDFWSURWRW\SLQJWHFKQLTXH ,GHD 'HYHORSPHQW7HDP

1HZ

$%675$&7 ,03/(0(17$7,216

5HTXLUHPHQWV 7KHRULHV/HYHO 5HTXLUHPHQWV

7HVWLQJ

7KHRU\

0HWKRGV/HYHO 5HTXLUHPHQWV

7HVWLQJ

0HWKRG

$OJRULWKPV/HYHO 5HTXLUHPHQWV

5HTXLUHPHQWV

7HVWLQJ

$OJRULWKP

'DWDEDVH

H U D Z W I R V  W U R S S X V  Q J L V H '

V V H F R U S W Q H P S R O H Y H G

3LORWV/HYHO 5HTXLUHPHQWV

7HVWLQJ

3LORW ,PSOHPHQWDWLRQ

$SSURYHG 'HVLJQ

)LJ 7KHUHDVRQLQJPRGHOIRUDEVWUDFWSURWRW\SLQJ

7KHUHYLHZLQWKHIUDPHZRUNRIWKHDEVWUDFWSURWRW\SLQJWHFKQLTXHSURFHHGV UHFXUVLYHO\ DPRQJ WKH OHYHOV RI DEVWUDFWLRQ >@ 7KH VXLWDELOLW\ RI WKH HOHPHQWV RI GHVLJQVXSSRUWVRIWZDUHFDQEHWHVWHGDWWKHWKHRULHVDQGPHWKRGVOHYHOVZKLOHWKH ZKROHV\VWHPFDQEHWHVWHGDWWKHDOJRULWKPVOHYHODQGDIWHUSLORWLPSOHPHQWDWLRQ 7HVWLQJLQDEVWUDFWSURWRW\SLQJLVUHTXLUHPHQWVEDVHG7KHUHTXLUHPHQWVDUHEDQGHG DFFRUGLQJWRWKHOHYHOVRIDEVWUDFWLRQ7KHUHTXLUHPHQWVFOXVWHULQJVFKHPHIRUWKH DEVWUDFWSURWRW\SLQJWHFKQLTXHLVGHVFULEHGLQ>@6WDQGDUGVVXFKDV,62DQG ,((( SURYLGH WKH EDVLF TXDOLW\ FKDUDFWHULVWLFV WKDW PXVW EH FRQVLGHUHG GXULQJ IRUPXODWLRQRIUHTXLUHPHQWV2WKHUFKDUDFWHULVWLFVFDQEHLQFRUSRUDWHGDVZHOO)RU HYHU\FKDUDFWHULVWLFUHTXLUHPHQWVWDWHPHQWVDUHIRUPXODWHGDQGXOWLPDWHO\XVHGDV WHVWLQJFULWHULD7KHDGYDQWDJHRIXVLQJVWDQGDUGVVXFKDVWKH,62VWDQGDUGWR GHVFULEHWKHGHVLJQVXSSRUWVRIWZDUHTXDOLW\LVWKDWUHTXLUHPHQWVFDQEHIRUPXODWHG LQDZD\WKDWOHDYHV QRURRPIRU DPELJXLW\ 7KH SRVVLELOLW\ RI PLVXQGHUVWDQGLQJV ZKHQXVLQJVWDQGDUGQRUPVLVJHQHUDOO\PLQLPDO$WHDFKOHYHORIDEVWUDFWLRQWKHUH FDQ EH PDQ\ DOWHUQDWLYH VROXWLRQV DW GLVSRVDO IURP ZKLFK EHWWHU VROXWLRQV DUH VHOHFWHG DQDO\]HG YLVXDOL]HG DQG PRGLILHG XQWLO UHTXLUHPHQWV DUH IXOILOOHG WR D VDWLVIDFWRU\ OHYHO 7KH EDODQFHG FRPSUHKHQVLRQ VFKHPH >@ SURYLGHV D IUDPHZRUN IRUL VHOHFWLRQRIEHWWHUVROXWLRQVDQGLL GHWHUPLQDWLRQRIWKHOHYHOVRIIXOILOOPHQW RIUHTXLUHPHQWVEDVHGRQWKHRSLQLRQVRIVXEMHFWV ,Q WKH IROORZLQJ VHFWLRQ ZH SUHVHQW D VRIWZDUH V\VWHP IRU VXSSRUWLQJ GHYHORSHUV LQ SHUIRUPLQJ DEVWUDFW SURWRW\SLQJ DFWLYLWLHV 7KH EHQHILWV RI

FRPSXWHUL]DWLRQRIWKHDEVWUDFWSURWRW\SLQJDFWLYLWLHVDUHVRPHZKDWSUHWW\REYLRXV 7KHDEVWUDFWSURWRW\SLQJVRIWZDUHHQDEOHWHVWVWREHUXQIDVWHUDQGFRQVLVWHQWO\ 4 THE ABSTRACT PROTOTYPING SOFTWARE 7KH DEVWUDFW SURWRW\SLQJ VRIWZDUH KDV IRXU PRGXOHV QDPHO\ D  UHTXLUHPHQWV GHVFULSWLRQ ZKLFK SURYLGHV VRIWZDUH WRROV IRU LQSXW UHWULHYDO EURZVLQJ DQG PRGLILFDWLRQ RI UHTXLUHPHQWV  DQG E  DEVWUDFW SURWRW\SHV GHVFULSWLRQ ZKLFK RIIHUV VHYHUDO SRVVLELOLWLHV IRU UHSUHVHQWDWLRQ RI WKH LPSOHPHQWDWLRQV HJ YLVXDO WH[WXDO YLGHR DQG DXGLR GHVFULSWLRQ PHDQV  2WKHUV DUH F  LQIRUPDWLRQ JDWKHULQJ ZKLFK SURYLGHV WRROV ZLWK ZKLFK WKH GHYHORSHUV FDQ SUHSDUH WKH LQIRUPDWLRQ JDWKHULQJ IRUPV DQG LQVWUXFWLRQV LQ D VWDQGDUGL]HG ZD\  DQG G  LQIRUPDWLRQ DQDO\VLV ZKLFK SURYLGHV XWLOLWLHV IRU L  DQDO\VLV DQG SUHVHQWDWLRQ RI YLHZV RI VXEMHFWV LL  SULRULWL]DWLRQ RI UHTXLUHPHQWV LLL  SURSRVLQJ LPSURYHPHQW VWUDWHJ\ DQGLY IRUPDWFKLQJDSUREOHPWRH[LVWLQJUHTXLUHPHQWVDQGDEVWUDFWSURWRW\SHV  )LJVKRZVVRPHRIDOJRULWKPVIRUWKHVHXWLOLWLHV Prioritize-Requirements (V, n) Determine-Improvement-Strategy (V,F) Repeat { Recall ViQ Find 9L7 = )

{ Recall Vi

9L Ê Q=

Q



0$; 

Find

9L7 = Ê9L Q =

6 R UW

}

} )

0,1 

{ Recall scores, i.e. MI values End Prioritize-Requirements Compute FI, i.e. )

D

L

Problem-Solution-Matching (P,g,r)

)

0$ ;

)

0,1 

=

0 [ 0 PD[ L

6 R UW

For Problem PI Specify a set of goals/needs J

{

}

= ^JJ  ¡ J  J ` L

L

When Specify a set of obvious/key requirements U

= ^U U  ¡ U 



L

{Set threshold value

UL`

Set threshold value Search the database & match {g} and {r} to abstract prototypes and requirements

ar for the relevance index

af for the fulfillment index

9L7 ˜ a U DQG )U ˜ a I



Return matching abstract prototypes ap and requirements R}

Return requirements list in the orders of relevance and fulfillment salience}

End Problem-Solution-Matching

End Improvement-Strategy

E

F

D 7KHUHTXLUHPHQWVSULRULWL]DWLRQDOJRULWKP9  UHOHYDQFHYDOXHVDQGQLVWKHQXPEHURIVXEMHFWV E 7KHSUREOHPUHTXLUHPHQWVVROXWLRQSURSRVDOPDWFKLQJDOJRULWKP5LVWKHVHWRIWKHDFWXDOUHTXLUHPHQWV F 7KHLPSURYHPHQWVWUDWHJ\DOJRULWKP) UHTXLUHPHQWVIXOILOOPHQWYDOXHVDQGQLVWKHQXPEHURIVXEMHFWV L

)LJ $OJRULWKPV 7KHUHTXLUHPHQWVDQGDEVWUDFWSURWRW\SHVDFFXPXODWHVLHWKH\DUHDYDLODEOH LQ DGYDQFH  DQG WKH\ VKRXOG EH PDGH DFFHVVLEOH WR DOO VWDNHKROGHUV ZLWKRXW UHVWULFWLRQ RU DQ\ GLIILFXOW\ 7KH HYDOXDWLRQ UHVXOWV VKRXOG DOVR EH VWRUHG IRU

SRVVLEOHIXWXUHUHXVH7KHDEVWUDFWSURWRW\SLQJVRIWZDUHSURYLGHV :$5(+286(6

LQ ZKLFK DOO WKHVH LPSRUWDQW SLHFHV RI LQIRUPDWLRQ DUH VWRUHG 7KH IDFLOLWLHV IRU DFFHVVLQJDQGEURZVLQJWKH:$5(+286(6DQGIRUUHWULHYLQJNQRZOHGJHDQGGDWD IURP RWKHU PRGXOHV RI WKH V\VWHP DUH SURYLGHG )DFLOLWLHV IRU LQSXWWLQJ XSGDWLQJ DQGPRGLI\LQJNQRZOHGJHDQGGDWDZLWKLQWKH:$5(+286(6DUHDOVRSURYLGHG 7KH VWUDWHJ\ ZH XVHG WR GHPRQVWUDWH WKH DEVWUDFW SURWRW\SLQJ FRQFHSW ZDV WR

DYRLG UHLQYHQWLQJ WKH ZKHHO  E\ XVLQJ WKH DYDLODEOH HOHPHQWV RI WKH ZLGHO\ XVHG FRPPHUFLDOVRIWZDUHSURGXFWVDVPXFKDVSRVVLEOH7KH:$5(+286(6KDYHEHHQ LPSOHPHQWHGLQ$FFHVVDUHODWLRQDOGDWDEDVHPDQDJHPHQWV\VWHP5'%06 2WKHU H[LVWLQJFRPPHUFLDOVRIWZDUH WRROV QDPHO\ WKH L  0LFURVRIW (TXDWLRQ (GLWRU LL 0LFURVRIW'UDZLQJV(GLWRUDQGLLL 0LFURVRIW:RUG3URFHVVRUSURYLGHWKHDEVWUDFW SURWRW\SLQJV\VWHPZLWKPHDQVIRUPRGHOLQJDQGSUHVHQWDWLRQRIDEVWUDFWVRIWZDUH LPSOHPHQWDWLRQV6RPHRIWKH([FHOXWLOLWLHVKDYHEHHQHQKDQFHGDQGFXVWRPL]HGWR SURYLGH PHDQV IRU DQDO\VLV DQG SUHVHQWDWLRQ RI WKH UHVXOWV 7KH UHDO YDOXH KHUH LV WKDW WKH FDSDELOLWLHV DQG IHDWXUHV LQ WKH H[LVWLQJ VRIWZDUH V\VWHPV DUH DYDLODEOH WR VXSSRUW WKH DEVWUDFW SURWRW\SLQJ LQ D VHDPOHVV XVHUIULHQGO\ HQYLURQPHQW )LJ  VKRZVWKHGDWDIORZGLDJUDPRIWKHDEVWUDFWSURWRW\SLQJV\VWHP 7KH,QWHUQHW 'HGLFDWHG 3URJUDPV

$FFHVV5'%06

7KH$EVWUDFW3URWRW\SLQJ 6\VWHP*8, ([FHO

:RUG3URFHVVRU

)LJ 7KHGDWDIORZGLDJUDPRIWKHSURWRW\SHDEVWUDFW SURWRW\SLQJV\VWHP



5 HOW TO USE THE ABSTRACT PROTOTYPING SYSTEM )LJVKRZVWKH*8,RIWKHDEVWUDFWSURWRW\SLQJVRIWZDUH,WFDQEHXVHGDVIROORZV +DYLQJNQRZQWKH IXQFWLRQDOLW\ DQGWKH OHYHORIDEVWUDFWLRQ WKH:$5(+286(6 XWLOLWLHVHQDEOHVXVHUVWRILQGWKHUHTXLUHPHQWVLQWKHGDWDEDVH7KHQH[WVWHSLVWR ILQHWXQHWKHUHTXLUHPHQWVOLVWLHGLVFDUGLUUHOHYDQWUHTXLUHPHQWVDQGDGGPLVVLQJ UHTXLUHPHQWV  DQG VDYH WKH UHVXOWLQJ UHTXLUHPHQWV PRGHO LQ WKH :$5(+286(0'% GDWDEDVH 0HDQZKLOH WKH $%675$&7 352727