Application of evolutionary algorithms for software maintainability ...

Download PDF
9 downloads 850 Views 260KB Size Report
Comment
development life cycle (SDLC). Software Maintainability. Prediction (SMP) is desirable because firstly the resource planning can be optimized in advance and ...
Application of Evolutionary Algorithms for Software Maintainability Prediction using Object-Oriented Metrics Ruchika Malhotra

Anuradha Chug

DSE, DTU, Delhi-110042, India and DCIS, Indiana University-Purdue University,

University School of Information and Communication Technology, GGS IP University, Dwarka, New Delhi-110077, India

[email protected], [email protected] ABSTRACT

[email protected] PDLQWDLQDELOLW\ >           DQG @ $OWKRXJKVRPDQ\VWDWLVWLFDORUPDFKLQHOHDUQLQJWHFKQLTXHVKDYH EHHQGHYHORSHGIRU603LQODVWGHFDGHVXFKDV)LRUDYDQWL 1HVL >@ XVHG 0XOWL /LQHDU 5HJUHVVLRQ $QDO\VLV $JJDUZDO HW DO >@ DQG7KZLQ 4XDK>@XVHGQHXUDOQHWZRUNV.RWHQ *UD\>@ XVHG %D\HVLDQ QHWZRUN 0LVUD >@ XVHG /LQHDU 5HJUHVVLRQ DQG =KRX HW DO >@ XVHG 0XOWLYDULDWH $GDSWLYH 5HJUHVVLRQ 6SOLQHV 0$56  PRGHO KRZHYHU WKH\ KDYH EHHQ UHSRUWHG WR EH ORZ LQ SUHGLFWLRQDFFXUDFLHVDVSHUWKHFULWHULDODLGGRZQE\&RQWHHWDO >@ DQG .LWFKHQKDP HW DO >@ 7KHUHIRUH LW LV QHFHVVDU\ WR H[SORUHWKHQHZWHFKQLTXHVZKLFKDUHQRWRQO\HDV\EXWDOVRFRVW HIIHFWLYH :H DUH LQWHUHVWHG LQ 603 XVLQJ UHODWLYHO\ QHZ ELRORJLFDO LQVSLUHG ($ ZKLFK DUH VXFFHVVIXOO\ XVHG IRU WKH SUHGLFWLRQV LQ PDQ\ ILHOGV RWKHU WKDQ 603 2YHUDOO VWXG\ GHDOV ZLWKWKHIROORZLQJWKUHH5HVHDUFK4XHVWLRQV 54V  RQ1: (PSLULFDOO\ YDOLGDWH WKH UHODWLRQVKLS EHWZHHQ GHVLJQ PHWULFVDQGVXEVHTXHQWPDLQWDLQDELOLW\LIH[LVWV" RQ2:&DQ(YROXWLRQDU\$OJRULWKPV ($ EHDSSOLHGIRU603" RQ3:,I($FDQEHXVHGIRU603GRWKH\SHUIRUPEHWWHURUZRUVH WKDQWUDGLWLRQDOVWDWLVWLFDODQGPDFKLQHOHDUQLQJPHWKRGV" $VHWRIDOJRULWKPVLQFOXGLQJVWDWLVWLFDOPDFKLQHOHDUQLQJ DQG  ($ EDVHG WHFKQLTXHV ZHUH VHOHFWHG IRU PDNLQJ WKH SUHGLFWLRQ PRGHO XVLQJ WKH .((/ WRRO >@ 3UHOLPLQDU\ UHVXOWV VKRZWKDWLWZDVDEOHWRDFKLHYHSUHFLVLRQDFFXUDF\ZKLFKLV TXLWH FRPSHWLWLYH 5HVW RI WKH SDSHU LV RUJDQL]HG DV IROORZV 6HFWLRQ  GHVFULEHV HYROXWLRQDU\ DOJRULWKPV 6HFWLRQ  SUHVHQWV UHVHDUFKPHWKRGRORJ\6HFWLRQGLVFXVVHVWKHUHVXOWVDQGILQDOO\ 6HFWLRQFRQFOXGHVWKHSDSHUDORQJZLWKIXWXUHGLUHFWLRQV

7KH FRVW LQFXUUHG GXULQJ PDLQWHQDQFH SKDVH RI DQ\ VRIWZDUH FRQVLVWV RI QHDUO\  RI WKH WRWDO SURMHFW FRVW ,Q RUGHU WR FRQWUROLWQHHGVWREHPHDVXUHGLQWKHHDUOLHUSKDVHVRIVRIWZDUH GHYHORSPHQW OLIH F\FOH 6'/&  6RIWZDUH 0DLQWDLQDELOLW\ 3UHGLFWLRQ 603  LV GHVLUDEOH EHFDXVH ILUVWO\ WKH UHVRXUFH SODQQLQJ FDQ EH RSWLPL]HG LQ DGYDQFH DQG VHFRQGO\ LW KHOSV LQ SURGXFLQJ FRVW HIIHFWLYH VRIWZDUH V\VWHPV 6LJQLILFDQFH RI WKH (YROXWLRQDU\ $OJRULWKPV ($  KDV VXEVWDQWLDOO\ LQFUHDVHG LQ UHFHQW WLPHV GXH WR WKHLU FDSDELOLW\ RI PD[LPL]LQJ WKH TXDOLW\ IXQFWLRQ ,QVSLUHG E\ WKH HYROXWLRQDU\ DOJRULWKPV ZH KDYH FRQGXFWHGDQHPSLULFDOVWXG\IRUH[SORULQJWKHDSSOLFDWLRQRI WKH ($ IRU 603 $OWKRXJK VHYHUDO WUDGLWLRQDO PHWKRGV VXFK DV VWDWLVWLFDO DQG PDFKLQH OHDUQLQJ ZHUH DSSOLHG LQ SDVW ZH H[SHULPHQWHG WR DSSO\ ($ IRU WKH ILUVW WLPH IRU 603 7ZR RSHQ VRXUFHVRIWZDUHSURMHFWV$SDFKH3RLDQG$SDFKH5DYH ZULWWHQ LQ -DYD ODQJXDJHV ZHUH XVHG WR FDUU\ RXW WKLV HPSLULFDO LQYHVWLJDWLRQ DQG WKH UHVXOWV ZHUH DQDO\]HG XVLQJ SUHYDOHQW SUHGLFWLRQDFFXUDF\PHDVXUHV:HREVHUYHGWKDWWKHRSWLPL]DWLRQ YDOXHVZHUHDFKLHYHGPRUHDFFXUDWHO\DQGSUHFLVHO\ZLWK($WKDQ WKHWUDGLWLRQDOPHWKRGVWKXVFDQEHVXFFHVVIXOO\DSSOLHGIRU603

Categories and Subject Descriptors ' 62)7:$5( (1*,1((5,1* 0DLQWHQDQFHDQG(QKDQFHPHQW

'

'LVWULEXWLRQ

General Terms ([SHULPHQWDWLRQ

2. EVOLUTIONARY ALGORITHMS

Keywords:

(PSLULFDO 9DOLGDWLRQ (YROXWLRQDU\ $OJRULWKPV 2EMHFW2ULHQWHG 0HWULFV 3UHGLFWLRQ 0RGHOLQJ DQG $QDO\VLV 6RIWZDUH0DLQWDLQDELOLW\3UHGLFWLRQ

7KHHYROXWLRQDU\DOJRULWKPVDUHWKHVHWRIDOJRULWKPVLQVSLUHGE\ WKH PHWDSKRU RI QDWXUDO ELRORJLFDO HYROXWLRQ (DFK WLPH DW HDFK JHQHUDWLRQ YDULRXV RSHUDWRUV VXFK DV VHOHFWLRQ UHFRPELQDWLRQ PXWDWLRQ PLJUDWLRQ ORFDOLW\ DQG QHLJKERUKRRG DUH DSSOLHG RQ SRWHQWLDOVROXWLRQVWRSURGXFHEHWWHUDQGEHWWHU UHVXOWV> DQG@$VVKRZQLQ)LJXUHZKHQWKLVSURFHVVLVUHSHDWHG DJDLQDQGDJDLQLWOHDGVWRWKHHYROXWLRQRISRSXODWLRQVFRQVLVWLQJ RI SRWHQWLDO VROXWLRQV ZKLFK DUH RSWLPL]HG ($ WHFKQLTXHV DUH XVHG HIIHFWLYHO\ LQ VRIWZDUH HQJLQHHULQJ OLNH SUHGLFWLRQ RI GHYHORSPHQW HIIRUW >@SUHGLFWLRQ RI PDLQWHQDQFH HIIRUW >@ DQG SUHGLFWLRQ RI SUHYHQWLYH PDLQWHQDQFH >@ KRZHYHU WKHLU XVH LQ 603 LV IRXQG WR EH H[WUHPHO\ OLPLWHG ,Q RUGHU WR FRPSDUH WKH SHUIRUPDQFHV RI ($ ZLWK WUDGLWLRQDO PHWKRGV ZH LGHQWLILHG WKH IROORZLQJVHWRIDOJRULWKPVGLYLGHGLQWRWKUHHPDMRUFDWHJRULHV DVXQGHU D Statistical Regression /LQHDU/065 //065  DQG 3UR4XDGUDWLF065 34065  E Traditional Machine Learning Algorithms'HFLVLRQ7UHH &$57 DQG 05XOHV  1HXUDO 1HWZRUNV 'HFU5%)15 DQG(QVHPEOH 6XSSRUW9HFWRU0DFKLQH (36,/216955 DQG186955  F Hybrid Algorithms (YROXWLRQDU\ )X]]\ 5XOH /HDUQLQJ *)6*3*5  DQG 7+5,)7  (YROXWLRQDU\ )X]]\ 6\PEROLF

1. INTRODUCTION 6RIWZDUH PDLQWDLQDELOLW\ LV GHILQHG DV WKH HDVH ZLWK ZKLFK WKH VRIWZDUHFRXOGEHPRGLILHGGXULQJRSHUDWLRQDOSKDVH,WFDQQRWEH PHDVXUHG XQWLO WKH VRIWZDUH V\VWHP LV SXW WR XVH IRU D FHUWDLQ SHULRG RI WLPH ,Q WKLV UHJDUG LW EHFRPHV LPSRUWDQW WR GHYHORS PRGHOV ZKLFK FDQ DVVHVV PDLQWDLQDELOLW\ GXULQJ HDUO\ SKDVHV RI SURMHFW GHYHORSPHQW ZLWK WKH KHOS RI VRPH PHDVXUDEOH VRIWZDUH FKDUDFWHULVWLFV ,Q OLWHUDWXUH LW LV HPSLULFDOO\ SURYHG WKDW WKHUH H[LVWLQGHHGDVWURQJOLQNEHWZHHQGHVLJQPHWULFVDQGVXEVHTXHQW FKDUDFWHULVWLFV ,Q OLWHUDWXUH LW LV HPSLULFDOO\ SURYHG WKDW WKHUH 3HUPLVVLRQWRPDNHGLJLWDORUKDUGFRSLHVRIDOORUSDUWRIWKLVZRUNIRU H[LVWLQGHHGDVWURQJOLQNEHWZHHQGHVLJQPHWULFVDQGVXEVHTXHQW SHUVRQDORUFODVVURRPXVHLVJUDQWHGZLWKRXWIHHSURYLGHGWKDWFRSLHV  DUHQRWPDGHRUGLVWULEXWHGIRUSURILWRUFRPPHUFLDODGYDQWDJHDQGWKDW FRSLHVEHDUWKLVQRWLFHDQGWKHIXOOFLWDWLRQRQWKHILUVWSDJH7RFRS\  RWKHUZLVHWRUHSXEOLVKWRSRVWRQVHUYHUVRUWRUHGLVWULEXWHWROLVWV UHTXLUHVSULRUVSHFLILFSHUPLVVLRQDQGRUDIHH %,&7'HFHPEHU%RVWRQ8QLWHG6WDWHV &RS\ULJKWk,&67 '2,LFVWELFW



x :0& :HLJKWHG0HWKRGVSHU&ODVV 7KHVXPRI0F&DEH¶V &\FORPDWLFFRPSOH[LWLHVRIDOOORFDOPHWKRGVLQDFODVV x ',7 'HSWKRI,QKHULWDQFH7UHH 7KHGHSWKRIDFODVVLQWKH LQKHULWDQFHWUHHZKHUHWKHURRWFODVVLV]HUR x 12& 1XPEHURI&KLOGUHQ ,WFRXQWVQXPEHURILPPHGLDWH VXEFODVVHVRIDFODVVLQDKLHUDUFK\ x &%2 &RXSOLQJ EHWZHHQ 2EMHFWV  ,W UHSUHVHQWV WKH QXPEHU RIFODVVHVWRZKLFKWKHJLYHQFODVVLVFRXSOHG x 5)& 5HVSRQVH )RU D &ODVV  7KH QXPEHU RI ORFDO PHWKRGV DQGQRQORFDOPHWKRGVFDOOHGE\FXUUHQWPHWKRGV x /&20 /DFN RI &RKHVLRQ RI 0HWKRGV  7KH QXPEHU RI GLVMRLQW VHWV RI ORFDO PHWKRGV LV FDOFXODWHG E\ FRXQWLQJ WKH LQVWDQFHYDULDEOHVFRPPRQWRWKHWZRRUPRUHPHPEHUV x /2& /LQHVRIFRGH 7KHQXPEHURIOLQHVRIFRGHH[FOXGLQJ FRPPHQWV x &KDQJH 7KH WRWDO QXPEHU RI OLQHV $GGHG 'HOHWHG DQG 0RGLILHGLQQHZYHUVLRQZUWWKHSUHYLRXVYHUVLRQ

5HJUHVVLRQ **65 DQG *665  (YROXWLRQDU\ 1HXUDO 1HWZRUN *$115DQG11(35  7KH GHWDLOV RI HDFK DOJRULWKP DUH JLYHQ E\ )HUQDQGH] HW DO >@ KRZHYHUWKHVHDUHEULHIO\H[SODLQHGDVXQGHU x Linear-LMS-R (LLMSR):,WLVDQDGDSWLYHDOJRULWKPXVHV DQLWHUDWLYHSURFHGXUHWRPLQLPL]HWKHPHDQVTXDUHHUURU x Pro Quadratic -MS-R (PQMSR): ,Q WKLV DOJRULWKP WHUPV DUHSODFHGLQUHVSHFWLYHJURXSVIRUFODVVLILFDWLRQV x CART &ODVVL¿FDWLRQ DQG 5HJUHVVLRQ 7UHHV DUH PDFKLQH OHDUQLQJPHWKRGVIRUFRQVWUXFWLQJSUHGLFWLRQPRGHOVREWDLQHG E\UHFXUVLYHO\SDUWLWLRQLQJWKHGDWDVSDFH x M5-Rules 6DPH DV &$57 EXW KHUH WKH WUHH FDQ KDYH PXOWLYDULDWHOLQHDUPRGHOWRWDFNOHWKHKLJKGLPHQVLRQDOLW\ x Decr-RBFN-R *HQHUDOL]DWLRQ LQ WHUPV RI LQWHUSRODWLRQ EHWZHHQNQRZQSRLQWVLVFUHDWHGIRUVXFKQHWZRUNV x Ensemble,QVWHDGRIUDQGRPVSDFHFRQVWUXFWLYHQRQOLQHDU SURMHFWLRQVDUHFUHDWHGXVLQJQHXUDOQHWZRUNV x EPSILON-SVR-R )LUVW RUGHU DSSUR[LPDWLRQ RI REMHFWLYH IXQFWLRQ LV XVHG WR DFKLHYH IDVWHU FRQYHUJHQFH GXULQJ WKH ZRUNLQJVHWVHOHFWLRQXVLQJ6XSSRUW9HFWRU0DFKLQH x NU-SVR-R,QVWHDGRIILUVWRUGHULQIRUPDWLRQLWXVHVVHFRQG RUGHULQIRUPDWLRQWRDFKLHYHIDVWHUFRQYHUJHQFH x GFS-GPG-R,WFRPELQHVJHQHWLFSURJUDPPLQJDQGJHQHWLF DOJRULWKPVWRVROYHV\PEROLFUHJUHVVLRQSUREOHPV x THRIFT*LYHQE\3KLOLS7KULIWWKHGLVFUHWHQDWXUHRIIX]]\ VWUDWHJLHVLVXVHGGXULQJWKHGLVFRYHU\SURFHVV x GFS-GAP-Sym-R (GGSR) 'HVLJQHG HVSHFLDOO\ IRU HOHFWULFDOHQJLQHHULQJIX]]\ DULWKPHWLFEDVHG *$SURFHGXUH LVDSSOLHGIRUWKHVHDUFKRIDQDQDO\WLFH[SUHVVLRQ x GFS-SAP-Sym-R (GSSR): ,W LV D 6\PEROLF )X]]\9DOXHG 'DWD/HDUQLQJEDVHGRQ*HQHWLF3URJUDPPLQJ2SHUDWRUVDQG 6LPXODWHG$QQHDOLQJ x GANN-R $V LW VWDQGV IRU *HQHWLF $OJRULWKP ZLWK 1HXUDO 1HWZRUN ERWK WHFKQLTXHV DUH FRPELQHG WR JHQHUDWH PRUH UHJXODUFRQQHFWLYHSDWWHUQV x NNEP-R $V LW VWDQGV IRU 1HXUDO 1HWZRUN (YROXWLRQDU\ 3URJUDPPLQJ IRU &ODVVLILFDWLRQ ZKLFK OHDGV WR RYHUDOO SHUIRUPDQFHJDLQIRUUHDOZRUOGKLJKRUGHUIXQFWLRQV 

3.2 Empirical Data Collection 7ZR RSHQ VRXUFH VRIWZDUH $SDFKH 3RL  DQG $SDFKH 5DYH ZHUHDQDO\]HGZKLFKDUHEULHIO\GHVFULEHGKHUH  3.2.1 Appache Poi:ULWWHQLQMDYDODQJXDJH32,6WDQGVIRU 3RRU 2EIXVFDWLRQ ,PSOHPHQWDWLRQ $V WKH QDPH VXJJHVWV ZKHQHYHU WKH FRPPXQLFDWLRQ LV FRQIXVLQJ RU KDUGHU WR LQWHUSUHW WKLV IUHH DQG RSHQ VRXUFH MDYD OLEUDU\ LV XVHG IRU FRQYHUWLQJ GLIIHUHQWIRUPDWV  3.2.2 Appache Rave ,W LV WKH ZHE EDVHG GDWD LQWHJUDWLRQ DSSOLFDWLRQ VRIWZDUH ,W¶V D OLJKW ZHLJKW MDYD SODWIRUP WR FRPELQHV WKH GDWD SUHVHQWDWLRQV RU IXQFWLRQDOLW\ IURP WZR RU PRUHVRXUFHVWRFUHDWHQHZVHUYLFHVIRUH[DPSOH)OLFNHU 

3.3 Prediction Accuracy Measures (PAM) :HDQDO\]HGUHVXOWVXVLQJDFFXUDF\PHDVXUHVJLYHQE\&RQWH>@ DQG.LWFKHQKDP>@DVIROORZV 3.3.1 Magnitude of Relative Error (MRE) ,W LV D QRUPDOL]HG PHDVXUH RI WKH GLIIHUHQFH EHWZHHQ SUHGLFWHG YDOXHV REWDLQHGWKURXJKPRGHODQGWKHUHVSHFWLYHDFWXDOYDOXH

Initial Population

‫ ܧܴܯ‬ൌ  Evaluation of Individual Solutions?

Is improvements possible ?

NO

3.3.2 Mean of Magnitude of Relative Error MMRE 0HDQRI05(LVFDOFXODWHGDVIROORZV

Result as Best Individuals

YES

‫ ܧܴܯܯ‬ൌ

Recombination, Cross over, Migration

Selection on the basis of Survival of Fittest

ȁ‫ ݁ݑ݈ܸ݈ܽܽݑݐܿܣ‬െ ܲ‫݁ݑ݈ܸܽ݀݁ݐܿ݅݀݁ݎ‬ȁ  ‫݁ݑ݈ܸ݈ܽܽݑݐܿܣ‬

σே ௜ୀଵ ‫ܧܴܯ‬௜ ‹•–Š‡—„‡”‘ˆ‘„•‡”˜ƒ–‹‘• ܰ

3.3.3 Prediction Accuracy at 25% and 30% (Pred 0.25 and Pred 0.30): IW PHDVXUHV WKH  SRUWLRQ RI WKH

And Mutation

Evaluation of

SUHGLFWHG YDOXHV ZKR KDYH 05( OHVV WKDQ RU HTXDO WR VSHFLILHG YDOXHLQSHUFHQWDJH

off-spring

ܲ‫݀݁ݎ‬ሺ‫ݍ‬ሻ ൌ  :KHUHTLVWKHVSHFLILHGYDOXHDQG.LVWKHQXPEHURI

Fitness

௄

ே

 Figure 1: Functioning of Evolutionary Algorithms

REVHUYDWLRQV

3. RESEARCH METHODOLOGY 3.1 Independent and Dependent Variables



9DULRXV FKDUDFWHULVWLFV RI RSHQ VRXUFH VRIWZDUH ZHUH PHDVXUHG XVLQJ &KLGDPEHU DQG .HPHUHU > DQG @ PHWULF VXLWH JLYHQ DV IROORZV



$SDFKH3RLDYDLODEOHRQOLQHDWKWWSSRLDSDFKHRUJ



$SDFKH5DYHLVDYDLODEOHDWKWWSVUDYHDSDFKHRUJ



Table 2: Results of Models applied on Apache Poi dataset and Apache RAVE dataset Category of the algorithm

Name of the Algorithm

Max MRE

POI Dataset MMRE Pred(0.25)

6WDWLVWLFDO 5HJUHVVLRQ 'HFLVLRQ7UHH

//065 34065 &$57 05XOHV 'HFU5%)1 (QVHPEOH (36,/216955 186955 *)6*3*5 7+5,)7 **65 *665 *$115 11(35

             

             

1HXUDO 1HWZRUNV 6XSSRUW9HFWRU 0DFKLQH (YROXWLRQDU\ )X]]\ ()  ()6\PEROLF 5HJUHVVLRQ (YROXWLRQDU\ 1HXUDO

             



4. RESULTS AND DISCUSSIONS 7KH UHVXOWV RI YDULRXV DFFXUDF\ PHDVXUHV DFKLHYHG E\ DSSO\LQJ 603PRGHOVDUHFRPSLOHGLQ7DEOHVIRU3RLDQG5DYHGDWDVHWV )LUVW FROXPQ UHSUHVHQW WKH FDWHJRU\ RI WKH DOJRULWKP VHFRQG FROXPQ UHSUHVHQWV WKH QDPH RI WKH DOJRULWKP WKLUG DQG VHYHQWK FROXPQUHSUHVHQWVWKHPD[LPXPYDOXHRI05(IRXUWKDQGHLJKWK FROXPQ UHSUHVHQWV 005( DQG ILIWK VL[WK QLQWK DQG WHQWK FROXPQ UHSUHVHQWV SUHGLFWLRQ DFFXUDF\ DW  DQG VL[WK UHSUHVHQWV SUHGLFWLRQ DFFXUDF\ DW  IRU 32, DQG 5$9( UHVSHFWLYHO\ RQ1: Is relationship exists between design metrics and maintainability? :H GLYLGHG WKH GDWD LQWR  UDWLR EHWZHHQ WUDLQLQJDQGWHVWLQJUHVSHFWLYHO\7KHYDOXHRI005(UHSUHVHQWV WKH JRRGQHVV RI ILW RI WKHSURSRVHG PRGHOV :KHQ ZH FDOFXODWHG WKH DYHUDJH 005( YDOXHV RI DOO  DOJRULWKPV VHOHFWHG LQ WKH FXUUHQW HPSLULFDO VWXG\ LWV YDOXH LV  DQG IRU3RL DQG 5DYHUHVSHFWLYHO\,WLVTXLWHHYLGHQWWKDWWKHYDOXHVRI005(DUH VLJQLILFDQWO\EHWWHU IRUERWKWKHGDWDVHWVDQGTXLWHFRPSHWLWLYHDV SHUWKHVWDQGDUGVRIDFFXUDF\PHDVXUHPHQWV>DQG@ZHFODLP WKDW WKHUH H[LVWV LQGHHG D VWURQJ UHODWLRQVKLS EHWZHHQ GHVLJQ PHWULFVDQGPDLQWDLQDELOLW\

Pred(.3)

MaxMRE

RAVE Dataset MMRE Pred(.25)

Pred(.3)

                                                                      Table 3: Results of Paired T Test with Poi dataset Pair of Algorithm with THRIFT

tvalue

pvalue

&$57 058/(6 //065 34065 (QVHPEOH 'HFU5%)1 (SVLORQ 18B695 **65 *665 *$11 11(3

           

           

Pair of Algorithm with GFSGPG-R &$57 058/(6 //065 34065 (QVHPEOH 'HFU5%)1 (SVLORQ 18B695 **65 *665 *$11 11(3

tvalue            

pvalue            

Table 4 : Results of Paired T Test with Rave dataset

RQ2: Can EA be applied for SMP? 2Q ERWK WKH GDWDVHWV 7+5,)7 LV IRXQG WR EH PRVW DFFXUDWH LI 005( LV WDNHQ DV WKH DFFXUDF\ LQGLFDWRU DQG *)6*3*5 DOJRULWKP RI ZDV IRXQG WR EH PRUH DFFXUDWH LI 0D[05( LV FRQVLGHUHG DV DFFXUDF\ LQGLFDWRU %RWK DOJRULWKPV XQGHU WKH FDWHJRU\ RI ()$ ZHUH IRXQG WREH WKH PRVW DFFXUDWH HYHQ ZKHQ SUHGLFWLRQDFFXUDF\DWDQGLVFKHFNHG7KXVIURPWKH UHVXOWV LW LV VDIH WR FODLP WKDW ()$ FDQ EH XVHG IRU PRUH SUHFLVH PDLQWDLQDELOLW\SUHGLFWLRQV

Pair of Algorithm with THRIFT

tvalue

pvalue

&$57 058/(6 //065 34065 (QVHPEOH 'HFU5%)1 (SVLORQ 18B695

           

           

**65 *665 *$11 11(3

RQ3: Do EA perform significantly better than traditional methods?

Pair of Algorithm with GFSGPG-R &$57 058/(6 //065 34065 (QVHPEOH 'HFU5%)1 (SVLORQ 18B695

**65 *665 *$11 11(3

tvalue            

pvalue            

SDLUHGZLWKUHVWDOJRULWKPVDQGLQVHFRQGVHW*)6*3*5RI ()$FDWHJRU\ZDVSDLUHGZLWKUHPDLQLQJDOJRULWKPV3DLUHG7 7HVW ZDV FRQGXFWHG IRU 5DYH DV ZHOO DV 3RL GDWDVHWV DW  VLJQLILFDQFH OHYHO ZLWK  GHJUHH RI IUHHGRP DQG UHVXOWV DUH VXPPDUL]HG LQ 7DEOH  DQG  UHVSHFWLYHO\  7KH SHUIRUPDQFH RI WKHSDLUHG7WHVWLVPHDVXUHGXVLQJSYDOXHDQGWYDOXH,ISYDOXH LV OHVV WKDQ  ZH FODLP WKDW WKH UHVXOWV RI WKH JLYHQ SDLU DUH VLJQLILFDQWO\GLIIHUHQW:KHWKHUWKLVVLJQLILFDQFHLVEHWWHURUZRUVH GHSHQGVXSRQWKHWYDOXH,IWYDOXHLVQHJDWLYHWKDWPHDQVWKHILUVW DOJRULWKPLVEHWWHURWKHUZLVHVHFRQGDOJRULWKPLVEHWWHU,Q7DEOH  DQG  VW FROXPQ UHSUHVHQW WKH SDLU ZLWK 7+5,)7 DQG WK

,Q RUGHU WR ILQG WKH DQVZHU RI UHVHDUFK TXHVWLRQ 54 ZH SHUIRUPHG SDLUHG WWHVW ZKLFK LV XVHG WR LGHQWLI\ WKH VLJQLILFDQW H[LVWHQFH RI VWDWLVWLFDO GLIIHUHQFHV EHWZHHQ YDULRXV WUHDWPHQWV DSSOLHGWRWKHGDWD:KLOHILQGLQJWKHDQVZHURISUHYLRXVTXHVWLRQ 54ERWKWKHDOJRULWKPVXQGHUWKHFDWHJRU\RI()$ZHUHIRXQG WR EH DV RXWSHUIRUPHU :LWK WKH KHOS RI SDLUHG 7WHVW ZH LQYHVWLJDWHG IXUWKHU WKH SHUIRUPDQFH 7+5,)7 DQG *)6*3*5 ERWKEHORQJVWRWKHFDWHJRU\RI()$,QWRWDOSDLUVZHUHPDGH IRUHDFKGDWDVHWVLQWKHILUVWVHW7+5,)7RI()$FDWHJRU\ZDV



FROXPQ ZLWK *)6*3*5 QG DQG WK FROXPQ UHSUHVHQWV WKH FRUUHVSRQGLQJ WYDOXHV DQG UG DQG WK FROXPQ UHSUHVHQWV FRUUHVSRQGLQJ SYDOXH LQ ERWK WKH WDEOHV IRU 32, DQG 5$9( GDWDVHW UHVSHFWLYHO\ ,W LV TXLWH HYLGHQW WKDW 7+5,)7 LV VLJQLILFDQWO\GLIIHUHQWZLWKRXWRIDOJRULWKPVIRU5DYHGDWDVHW DQG  RXW RI  DOJRULWKPV IRU SRL GDWDVHWV *)6B*3*B5 LV VLJQLILFDQWO\GLIIHUHQWZLWKRXWRIDOJRULWKPVIRU5DYHGDWDVHW DQGRXWRIDOJRULWKPVZLWKSRLGDWDVHWV7KXVZHFODLPWKDW WKH HYROXWLRQDU\ IX]]\ DOJRULWKPV DUH VLJQLILFDQWO\ GLIIHUHQW WKDQ RWKHUDOJRULWKPV1H[WMREZDVWRGHWHUPLQHLIWKH\DUHEHWWHURU ZRUVW $V DOO WKH YDOXH RI WYDOXH LV QHJDWLYH ZKHQ 7+5,)7 DQG *)6*3*5 LV SDLUHG ZLWK RWKHU DOJRULWKPV ZH DOVR FODLP WKDW WKH HYROXWLRQDU\ IX]]\ DOJRULWKPV DUH VLJQLILFDQWO\ EHWWHU WKDQ WKHLUFRXQWHUSDUW

PDLQWDLQDELOLW\ RI REMHFW RULHQWHG GHVLJQ IEEE Transaction on software Engineering   >@ &KLGDPEHU 6 DQG .HPHUHU & $ 0HWULFV 6XLWH IRU 2EMHFW 2ULHQWHG 'HVLJQ IEEE Transactions on Software Engineering   >@ &RQWH6+ 'XQVPRUH 6KHQ9 6RIWZDUH (QJLQHHULQJ 0HWULFVDQG0RGHOVMenlo Park, Benjamin Cummings >@ )HUQDQGH]$/XHQJR-'HUUDF-$OFDOD)GH]-DQG+HUUHUD ) ,PSOHPHQWDWLRQ DQG ,QWHJUDWLRQ RI $OJRULWKPV LQWR WKH .((/ 'DWD0LQLQJ 6RIWZDUH 7RRO ,QWHOOLJHQW 'DWD (QJLQHHULQJDQG$XWRPDWHG/HDUQLQJ±,'($//HFWXUH 1RWHVLQ&RPSXWHU6FLHQFH9ROXPH >@ )LRUDYDQWL)DQG1HVL3(VWLPDWLRQDQGSUHGLFWLRQPHWULFV IRU DGDSWLYH PDLQWHQDQFH HIIRUW RI REMHFW RULHQWHG V\VWHPV IEEE Transactions on Software Engineering     

5. CONCLUSION & FUTURE DIRECTION ,Q WKLV HPSLULFDO VWXG\ ZH HYDOXDWHG WKH SHUIRUPDQFH RI HYROXWLRQDU\ DOJRULWKPV IRU VRIWZDUH PDLQWDLQDELOLW\ SUHGLFWLRQV :H FRPSDUHG WKH SUHGLFWLRQ SHUIRUPDQFH RI HYROXWLRQDU\ IX]]\ HYROXWLRQDU\ QHXUDO DQG HYROXWLRQDU\ QHXUDO V\PEROLF UHJUHVVLRQ PHWKRGVZLWKWUDGLWLRQDOVWDWLVWLFDODQGPDFKLQHOHDUQLQJPRGHOV 'DWDVHWV ZHUH FROOHFWHG XVLQJ WZR RSHQ VRXUFH VRIWZDUH V\VWHPV $SDFKH3RLDQG$SDFKH5DYH7KHUHVXOWVLQGLFDWH($DUHPXFK EHWWHU WKDQ WUDGLWLRQDO DOJRULWKPV DV WKH\ FRXOG DFKLHYH DFFXUDF\ ZLWKLQ WKH UDQJH RI  WR  :H FRQGXFWHG SDLUHG 7WHVW WR FRPSDUH WKHLU SHUIRUPDQFH ZLWK WUDGLWLRQDO PHWKRGV :H IRXQG WKDW WKH ($ SHUIRUPV VLJQLILFDQWO\ EHWWHU WKDQ WKH WUDGLWLRQDO DOJRULWKPV $Q LPSRUWDQW FRQWULEXWLRQ RI WKLV ZRUN LV WKDW VLQFH ZH KDYH FRPSDUHG UHVXOWV XVLQJ WKH 22 PHWULFV VXLWH XVLQJ WZR RSHQVRXUFHVRIWZDUHV\VWHPVZHFDQJHQHUDOL]HRXUUHVXOWV7KLV VWXG\ FRQILUPV WKDW FRQVWUXFWLRQ RI ($ IRU 603 LV IHDVLEOH DGDSWDEOH DQG XVHIXO LQ SUHGLFWLQJ VRIWZDUH PDLQWDLQDELOLW\ 2QH RI WKH ELJJHVW GUDZEDFNV RI ($ LV WKDW WKH VXJJHVWHG VROXWLRQ SURGXFHG LQ HDFK LWHUDWLRQ LV EHWWHU RQO\ LQ FRPSDULVRQ ZLWK WKH SUHYLRXVRQHDQGWKHFDQGLGDWHVROXWLRQLVQRWJXDUDQWHHGRSWLPDO $VWKHILQDOFKRLFHRIZKHQWRVWRSDOZD\VOLHVZLWKWKHXVHULWLV YHU\LPSRUWDQWWRDWWDLQWKHPD[LPDRIFRVWDQGHIIRUWVWUDGHRII ,QIXWXUHZHDUHSODQQLQJWRGHWHUPLQHWKHSDUDPHWHUZKLFKPDNHV WKHHYROXWLRQDU\DOJRULWKPVQRWRQO\HIIHFWLYHEXWHTXDOO\HIILFLHQW DVZHOO

>@ .LWFKHQKDP %$ 3LFNDUG /0 0DF'RQHOO 6* DQG 6KHSSHUG0-:KDWDFFXUDF\VWDWLVWLFVUHDOO\ PHDVXUH IEE Proceedings-Software  ± >@ .RWHQ&9*UD\$5$QDSSOLFDWLRQRI%D\HVLDQQHWZRUN IRU SUHGLFWLQJ REMHFWRULHQWHG VRIWZDUH PDLQWDLQDELOLW\ Information and Software Technology   >@ /L : DQG +HQU\ 6 2EMHFW2ULHQWHG 0HWULFV WKDW 3UHGLFW 0DLQWDLQDELOLW\ Journal of Systems and Software     >@ 0DOKRWUD 5 DQG &KXJ $ 6RIWZDUH 0DLQWDLQDELOLW\ 3UHGLFWLRQ XVLQJ 0DFKLQH /HDUQLQJ $OJRULWKPV 6RIWZDUH (QJLQHHULQJ $Q ,QWHUQDWLRQDO -RXUQDO 6(,-  6HSW     >@ 0DOKRWUD5DQG&KXJ$$SSOLFDWLRQRI*URXS0HWKRGRI 'DWD+DQGOLQJPRGHOIRUVRIWZDUHPDLQWDLQDELOLW\SUHGLFWLRQ XVLQJ REMHFW RULHQWHG V\VWHPV ,QW - 6\VWHPV $VVXUDQFH (QJLQHHULQJ DQG 0DQDJHPHQW  ,QWHUQDWLRQDO -RXUQDO 6SULQJHU9RO,VVXHSDJH   >@ 0DOKRWUD 5 DQG &KXJ $ $ 0HWULF 6XLWH IRU 3UHGLFWLQJ 6RIWZDUH 0DLQWDLQDELOLW\ LQ 'DWD ,QWHQVLYH $SSOLFDWLRQV 7UDQVDFWLRQV RQ (QJLQHHULQJ 7HFKQRORJLHV 6SULQJHU SDJH 

6. REFERENCES >@ $JJDUZDO .. 6LQJK @ 0LVUD 6 0RGHOLQJ GHVLJQFRGLQJ IDFWRUV WKDW GULYH PDLQWDLQDELOLW\ RI VRIWZDUH V\VWHPV Software Quality Journal   >@ 6XQ 3 DQG :DQJ $  $SSOLFDWLRQ RI $QW &RORQ\ 2SWLPL]DWLRQ LQ 3UHYHQWLYH 6RIWZDUH 0DLQWHQDQFH 3ROLF\ IEEE international Conference on Information Science and Technology0DUFK@ $OED ( 3DUDOOHOLVP DQG HYROXWLRQDU\ DOJRULWKPV ,((( 7UDQVRQ(YROXWLRQDU\&RPSXWDWLRQ   >@ %DORJK*=ROWDQ$%DV]HGHV $3UHGLFWLRQRI6RIWZDUH 'HYHORSPHQW (IIRUW (QKDQFHG E\ D *HQHWLF $OJRULWKP Symposium on search based software engineering, SSBSE 6HSWHPEHU7UHQWR,WO\

>@ 7KZLQ0DQG4XDK7$SSOLFDWLRQRIQHXUDOQHWZRUNVIRU VRIWZDUH TXDOLW\ SUHGLFWLRQ XVLQJ REMHFW RULHQWHG PHWULFV Journal of Systems and Software   >@ 9LYDQFR 1 3L]]L )LQGLQJ (IIHFWLYH 6RIWZDUH 0HWULFV WR &ODVVLI\0DLQWDLQDELOLW\8VLQJD3DUDOOHO*HQHWLF$OJRULWKP *HQHWLF DQG (YROXWLRQDU\ &RPSXWLQJ *(&&2  /HFWXUH 1RWHV LQ &RPSXWHU 6FLHQFH      

>@ %DQGL 5 3UHGLFWLQJ PDLQWHQDQFH SHUIRUPDQFH XVLQJ REMHFW RULHQWHG GHVLJQ FRPSOH[LW\ PHWULFV IEEE Transactions on Software Engineering   >@ %DTDLV % $OVKD\HE 0 %DLJ =$ +\EULG ,QWHOOLJHQW 0RGHO IRU 6RIWZDUH 0DLQWHQDQFH 3UHGLFWLRQ in Proceedings of the World Congress on Engineering UG WK  -XO\  /RQGRQ8.

>@ =KRX @ %ULDQG /& %XQVH & DQG 'DO\ -: $ FRQWUROOHG H[SHULPHQW IRU HYDOXDWLQJ TXDOLW\ JXLGHOLQHV RQ WKH