ScienceDirect Designing WorNshops for the

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 28 (2014) 643 – 652

&RQIHUHQFHRQ6\VWHPV(QJLQHHULQJ5HVHDUFK&6(5  (GV$]DG00DGQL8QLYHUVLW\RI6RXWKHUQ&DOLIRUQLD%DUU\%RHKP8QLYHUVLW\RI6RXWKHUQ&DOLIRUQLD 0LFKDHO6LHYHUV-HW3URSXOVLRQ/DERUDWRU\0DULOHH:KHDWRQ7KH$HURVSDFH&RUSRUDWLRQ 5HGRQGR%HDFK&$0DUFK

'HVLJQLQJ:RUNVKRSVIRUWKH,QWURGXFWLRQRI/HDQ(QDEOHUVWR (QJLQHHULQJ3URJUDPV .LOLDQ*HUVLQJD -RVHI2HKPHQE(ULF5HEHQWLVFKF a

Technische Universität München, Arcisstr. 30, 80333 Munich, Germany b Danmarks Tekniske Universitet, Copenhagen, Denmark c Massachusetts Institute of Technology, Cambridge, MA, USA

$EVWUDFW 7KHUHLVDODUJHDQGJURZLQJERG\RINQRZOHGJHUHJDUGLQJVRFDOOHG/HDQEHVWSUDFWLFHVLQFOXGLQJPRVWUHFHQWO\LQWKHDUHDRI SURJUDPPDQDJHPHQWDQGV\VWHPVHQJLQHHULQJ+RZHYHUWKHUHLVOLWWOHHODERUDWLRQRIKRZWKHVHGRFXPHQWHGEHVWSUDFWLFHVDUHWR EHLQWURGXFHGWRDSURIHVVLRQDOZRUNIRUFH2QHZD\RILQWURGXFLQJQHZSUDFWLFHVWRDZRUNIRUFHLVWKURXJKWKHXVHRIWUDLQLQJ ZRUNVKRSV 6XFK ZRUNVKRSV RIWHQ HPHUJH IURP WUDLQLQJ SURJUDPV RU FRQVXOWLQJ DUUDQJHPHQWV EXW WKHUH LV QR ZHOOGHILQHG PHWKRGRUIUDPHZRUNWRV\VWHPDWLFDOO\GHVLJQZRUNVKRSVIRUWKHLPSOHPHQWDWLRQRIDQHZERG\RINQRZOHGJH 7KLVVWXG\IRFXVHVRQWKHGHYHORSPHQWRIDIUDPHZRUNWKDWIDFLOLWDWHVWKHV\VWHPDWLFGHVLJQRIZRUNVKRSVIRFXVHGVSHFLILFDOO\RQ WKH LQWURGXFWLRQ RI /HDQ SULQFLSOHV DQG SUDFWLFHV WR SURJUDP PDQDJHPHQW DQG WKH SURIHVVLRQDO ZRUNIRUFH LQ D SURJUDP HQYLURQPHQW7KHIUDPHZRUNLVEDVHGRQDWKRURXJKUHYLHZRIOLWHUDWXUHRQWUDLQLQJZRUNVKRSGHOLYHU\DQG/HDQSULQFLSOHVDV ZHOODVHPSLULFDOHYLGHQFHREWDLQHGIURPGDWDFROOHFWLRQDQGLQWHUDFWLRQVZLWKWUDLQLQJSURIHVVLRQDOVIURPLQGXVWU\ 7KHIUDPHZRUNSURYLGHVDV\VWHPDWLFGHVLJQSURFHVVIRUZRUNVKRSVLQFOXGLQJD WKHDVVHVVPHQWRIWKHQHHGIRUDZRUNVKRSE  WKHGHILQLWLRQRIZRUNVKRSJRDOVDQGREMHFWLYHVF WKHGHILQLWLRQRIZRUNVKRSWRSLFVG WKHGHYHORSPHQWRIDQDJHQGDDQGWKH WKHRUHWLFDOFRQWHQW RIWKH ZRUNVKRSV DQGH WKH VHOHFWLRQ RI WKH ULJKWWHDFKLQJ WHFKQLTXHV DV ZHOO DVWKH ULJKW VLPXODWLRQV DQG DFWLYHOHDUQLQJGHYLFHV,QHDFKRIWKHVHVWHSVWKHIUDPHZRUNSURYLGHVFRQWHQWIRUWKHSURVSHFWLYHHGXFDWRUV 7KH IUDPHZRUN LV HPERGLHG LQ DQ ([FHOEDVHG WRRO WKDW DOORZV WKH XVHU WR TXLFNO\ DVVHPEOH D VWUXFWXUH IRU WKH ZRUNVKRS LQFOXGLQJWKHWRSLFDQDJHQGDDQGGHILQHGJRDOVDQGDOVRWKHWKHRUHWLFDOFRQWHQWDERXWWKH/HDQSULQFLSOHV$QRWKHUHOHPHQWRI WKHIUDPHZRUNLVWKHGHVLJQDWLRQRIWKHRUJDQL]DWLRQDOKLHUDUFK\OHYHOVWREHDGGUHVVHGZLWKHGXFDWLRQDOWHFKQLTXHVDGDSWHGWR 



&RUUHVSRQGLQJDXWKRU7HO E-mail address:NLOLDQJHUVLQJ#JPDLOFRP

1877-0509 © 2014 The Authors. Published by Elsevier B.V. Open access under CC BY-NC-ND license. Selection and peer-review under responsibility of the University of Southern California. doi:10.1016/j.procs.2014.03.078

644

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

HDFKOHYHO7KHIUDPHZRUNZDVYDOLGDWHGWKURXJKLQWHUDFWLRQVZLWKWUDLQLQJSURIHVVLRQDOVLQDODUJHDXWRPRELOHPDQXIDFWXUHUDQG XVLQJVXEMHFWPDWWHUH[SHUWVIURPDYDULHW\RILQGXVWULDOVHFWRUV

‹7KH$XWKRUV3XEOLVKHGE\(OVHYLHU%9 © 2014 The Authors. Published by Elsevier B.V. Open access under CC BY-NC-ND license. 6HOHFWLRQDQGSHHUUHYLHZXQGHUUHVSRQVLELOLW\RIWKH8QLYHUVLW\RI6RXWKHUQ&DOLIRUQLD Selection and peer-review under responsibility of the University of Southern California. Keyword:/HDQ(QDEOHUV/HDQ3URJUDP0DQDJHPHQW:RUNVKRSV/(,:)UDPHZRUN

,QWURGXFWLRQ /DUJHVFDOH HQJLQHHULQJ SURJUDPV WRGD\ VWLOO DUH RQH RI WKH PRVW GLIILFXOW ULVN\ DQG ± ZKHQ ZHOO GRQH ± UHZDUGLQJ XQGHUWDNLQJV D JRYHUQPHQW RU FRPSDQ\ FDQ DWWHPSW 1XPHURXV EHVW SUDFWLFHV H[LVW LQ WKH GRPDLQV RI SURMHFWPDQDJHPHQWV\VWHPVHQJLQHHULQJDQG/HDQWKLQNLQJWKDWGHOLYHUFRQFUHWHDGYLFHWRSURJUDPPDQDJHUVWR LQFUHDVHWKHLUSURJUDP¶VSHUIRUPDQFH0RVWUHFHQWO\DEHVWSUDFWLFHFROOHFWLRQFDOOHG³/HDQ(QDEOHUVIRU0DQDJLQJ (QJLQHHULQJ 3URJUDPV´ ZDVGHYHORSHG DW WKH 0DVVDFKXVHWWV ,QVWLWXWH RI7HFKQRORJ\ZKLFK VSHFLILFDOO\ DGGUHVVHV FKDOOHQJHV UHODWHG WR WKH HIIHFWLYHQHVV WKH HIILFLHQF\ DQG WKH KXPDQ DVSHFWV LQ ODUJH VFDOH HQJLQHHULQJ SURJUDPV 7KH FRUH RI WKLV JXLGHERRN FRQWDLQV  /HDQ EHVW SUDFWLFHV DOVR UHIHUUHG WR DV WKH /HDQ (QDEOHUV 7KH /HDQ (QDEOHUV DUH FOXVWHUHG LQ VL[ GLIIHUHQW VRFDOOHG /HDQ 3ULQFLSOHV 7KHLU WLWOHV DUH YDOXH YDOXH VWUHDP IORZ SXOO SHUIHFWLRQDQGUHVSHFWIRUSHRSOH7KH/HDQ(QDEOHUVFRQWDLQ6XEHQDEOHUVZKLFKSURYLGHIXUWKHUGHWDLOV 6LQFHWKHVH/HDQ(QDEOHUVDUHEDVHGRQSUDFWLFDOEHVWSUDFWLFHVIURPWKHLQGXVWU\DQGDFDGHPLDPDQ\RIWKHPDUH DOUHDG\ EHLQJ XVHG LQ HQJLQHHULQJ SURJUDPV 7KXV DSSO\LQJ WKH /HDQ (QDEOHUV WR D QHZ RU D UXQQLQJ SURJUDP UHTXLUHV WKH WZR IROORZLQJ VWHSV )LUVW WKH /HDQ (QDEOHUV ZLWK WKH KLJKHVW LPSURYHPHQW SRWHQWLDO QHHG WR EH LGHQWLILHG6HFRQGWKHDSSRLQWHG/HDQ(QDEOHUVQHHGWREHLPSOHPHQWHGLQWKHSURJUDP7KHLPSOHPHQWDWLRQFDQ HLWKHUEHDFKLHYHGWKURXJKDVWUDWHJLFWUDQVIRUPDWLRQRUDFRQWLQXRXVLPSURYHPHQWDSSURDFK 2QFHWKH/HDQ(QDEOHUVZLWKWKHKLJKHVWSRWHQWLDOIRULPSURYHPHQWDUHLGHQWLILHGWKHFKDQJHSURFHVVVWDUWV7KH FKDQJHFRPPXQLFDWLRQLVDFUXFLDOHOHPHQWLQHPSOR\HHV¶UHFHSWLYLW\RIWKLVFKDQJHZKLFKDJDLQLVDQHPHUJLQJDQG VLJQLILFDQWLVVXHIRUWKRVHLQYROYHGLQFUHDWLQJVXFFHVVIXOFKDQJHLPSOHPHQWDWLRQVWUDWHJLHV:RUNVKRSVDUHDPRQJ WKHPRVWSRSXODUWUDLQLQJGHYLFHVLQKLJKHUHGXFDWLRQDQGKDYHDJUHDWHIIHFWLQFKDQJHFRPPXQLFDWLRQ 7KLVSDSHULQWURGXFHVDIUDPHZRUNWKDWVXSSRUWVWKHRSHUDWLRQDOL]DWLRQRI/HDQ(QDEOHUVE\IDFLOLWDWLQJWKHGHVLJQ RI HQJDJLQJ HGXFDWLRQDORULHQWHG LPSOHPHQWDWLRQ ZRUNVKRSV 7KURXJK WKHVH ZRUNVKRSV /HDQ 7KLQNLQJ LV LPSOHPHQWHGLQHQJLQHHULQJSURJUDPV 7KH/HDQ(QDEOHUV 7KH/HDQ(QDEOHUVDUHDVHWRI/HDQEHVWSUDFWLFHVZKLFKKDYHEHHQGHYHORSHGVSHFLILFDOO\IRUWKHLPSURYHPHQW RIODUJHVFDOHHQJLQHHULQJSURJUDPV7KHUHDUH/HDQ(QDEOHUVLQWRWDOZKLFKDUHVRUWHGDFFRUGLQJWRWKHVL[/HDQ 3ULQFLSDOVYDOXHYDOXHVWUHDPIORZSXOOSHUIHFWLRQUHVSHFWIRUSHRSOH 7KH/HDQ(QDEOHUVDUHEURNHQGRZQLQWR  6XEHQDEOHUV 7KH ILUVW /HDQ (QDEOHU RI WKH ILUVW 3ULQFLSOH IRU H[DPSOH VWDWHV ³%XLOG D SURJUDP EDVHG RQ UHVSHFW IRU SHRSOH´ 7KH ILUVW 6XEHQDEOHU RI WKLV (QDEOHU WKHQ VWDWHV ³8QGHUVWDQG WKDW SURJUDPV IDLO RU VXFFHHG SULPDULO\ EDVHG RQ SHRSOH QRW SURFHVVHV 7UHDW SHRSOH DV WKH PRVW YDOXHG DVVHWV QRW DV FRPPRGLWLHV´ 7KH 6XEHQDEOHUVJLYHPRUHGHWDLOVRQZKDWQHHGVWREHXQGHUWDNHQLQRUGHUWRIXOO\LPSOHPHQWWKH/HDQ(QDEOHU7KH *XLGH WR /HDQ (QDEOHUV IRU 0DQDJLQJ (QJLQHHULQJ 3URJUDPV IXUWKHUPRUH JLYHV D QXPEHU RI H[DPSOHV IRU HDFK /HDQ(QDEOHUVKRZVZKLFKSURJUDPSHUIRUPDQFHGRPDLQWKH/HDQ(QDEOHULVUHODWHGWRLQGLFDWHVZKLFKFKDOOHQJH WKH /HDQ (QDEOHU DGGUHVVHV DQG DOVR OLQNV HDFK /HDQ (QDEOHU WR WKH DSSURSULDWH SURFHVV QXPEHU RI WKH V\VWHPV HQJLQHHULQJSURFHVV ,QRUGHUWRGHWHUPLQHWKHLPSDFWRIWKH/HDQ(QDEOHUV³EHVWLQFODVV´SURJUDPVZHUHDQDO\]HGIRUWKHXVHRI/HDQ (QDEOHUV7KHUHVXOWRIWKLVDQDO\VLVZDVWKDWWKHWKUHHPRVWKLJKO\VXFFHVVIXOSURJUDPVRQZKLFKWKHDPRXQWRI GHWDLOHG LQIRUPDWLRQ ZDV HQRXJK IRU WKH DQDO\VLV XVHG  WR  RI DOO WKH /HDQ (QDEOHUV $IWHU WKH URXJK DQDO\VLVDVHFRQGPRUHGHWDLOHGDQDO\VLVZDVFRQGXFWHGEDVHGRQDVXUYH\LQZKLFKWKHSHUIRUPDQFHRIVXFFHVVIXO DQG XQVXFFHVVIXO SURJUDPV DV ZHOO DV WKH GHJUHH WR ZKLFK WKH\ XVHG WKH /HDQ (QDEOHUV ZDV DVNHG 7KH VWURQJ UHVXOWV RI WKH VXUYH\ VKRZHG WKDW VXFFHVVIXO SURJUDPV KDYH PRUH /HDQ (QDEOHUV LPSOHPHQWHG WKDQ XQVXFFHVVIXO

645

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

RQHV )LJ  VXPPDUL]HV WKH UHVXOWV RI WKH VXUYH\ DQG FOHDUO\ HPSKDVL]HV WKH GLIIHUHQFH LQ SHUIRUPDQFH EHWZHHQ VXFFHVVIXODQGXQVXFFHVVIXOSURJUDPV

 )LJ6XFFHVVIXOSURJUDPVKRZVLJQLILFDQWO\KLJKHUSHUIRUPDQFHWKDQXQVXFFHVVIXORQHV

3RWHQWLDO:RUNVKRSVIRUWKH,PSOHPHQWDWLRQRI/HDQ(QDEOHUV 3.1. Identification of Potential Workshops ,Q RUGHU WR XQGHUVWDQG ZKLFK /HDQ (QDEOHUV DUH DOUHDG\ EHLQJ DGGUHVVHG E\ H[LVWLQJ ZRUNVKRSV DQ H[SHUW ZRUNVKRSZDVFRQGXFWHGLQZKLFKWKHSDUWLFLSDQWVZHUHDVNHGWRPDSZRUNVKRSVIURPWKHILHOGRI/HDQDJDLQVWWKH /HDQ (QDEOHUV 7KH IROORZLQJ OLVW VKRZV WKH ZRUNVKRSV WKDW ZHUH VXJJHVWHG E\ WKH SDUWLFLSDQWV RI WKH H[SHUW ZRUNVKRS DQG JLYHV D VKRUW GHVFULSWLRQ WR HDFK RQH 6LQFH WKH PDLQ FRQFOXVLRQ LQ WKLV FKDSWHU LV GUDZQ IURP WKH PDSSLQJLWVHOIWKHUHLVQRYDOXHLQLQWURGXFLQJWKHZRUNVKRSVLQJUHDWHUGHWDLO7KHUHIRUWKHEULHILQWURGXFWLRQLV VHHQDVVXIILFLHQWIRUVKRZLQJWKHGLYHUVLW\RIWKHZRUNVKRSVWKDWZHUHFRQVLGHUHGLQWKLVPDSSLQJ :6 3DLQ 3RLQWV 3ULRULWL]DWLRQ DQG LGHQWLILFDWLRQ RI SDLQ SRLQWV WKURXJK EUDLQVWRUPLQJ DQG MRLQW GLVFXVVLRQV OHDGLQJWRWKHLQLWLDWLRQRIDQLPSURYHPHQWSURMHFW :6 &5(67±&DVKULVNHYDOXDWLRQVFXGRIWHFKQLFDOSHUIRUPDQFH5LVNEDVHGVWXG\OLQNLQJWKHVHSDUDPHWHUV LQDVLQFHUHYLHZRIWKHHVWLPDWHWRFRPSOHWH :6 9DOXH 6WUHDP 0DSSLQJ :RUNVKRS 8QGHUVWDQGLQJ FXUUHQW VWDWH RI D SURFHVV DQG EUDLQVWRUP SURFHVV LPSURYHPHQWVWRSURYLGHLQFUHDVHGYDOXHHIILFLHQFLHV :6 /HDQ*UHHQ%HOW:RUNVKRS'HYHORSPHQWRIVNLOOVUHTXLUHGWRDGGUHVVLQHIILFLHQFLHVLQEXVLQHVVSURFHVVHV DQGWRLPSURYHRUJDQL]DWLRQDOSHUIRUPDQFH :6 &KDQJH5HTXHVW:RUNVKRS5HDOLVWLFFDVHVWXG\WRUDLVHFRQVFLRXVQHVVDERXWGLIILFXOWLHVGXULQJDSURFHVV ([FKDQJHRISRLQWVRIYLHZRQKRZWRLPSURYH :6 7KH/HDQ-RXUQH\:RUNVKRSRQOHDQZLWKLWVSULQFLSOHVDQGWRROVZLWKDVXUYH\EDVHGYHULILFDWLRQRIWKH JDLQHGNQRZOHGJH

646

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

:6 &RQYHQWLRQDO'HVLJQ7HDP:RUNVKRS3UHVHQWDWLRQDQGGLVFXVVLRQRIVHYHUDOGHVLJQVGHFLGLQJRQZKLFK GHVLJQVIXUWKHUWRSURFHHG :6 )ORZ $QDO\VLV :RUNVKRS 0RGHOLQJ DQG XQGHUVWDQGLQJ FXUUHQW VWDWH RI SURFHVV IORZ GLVFXVVLQJ LPSURYHPHQWSRWHQWLDODQGDSSO\LQJWKHPWRPRGHO :6 7HFKQLFDO 3ODQQLQJ :RUNVKRS (YDOXDWLQJ WHFKQLFDO UHTXLUHPHQWV DQG EXLOGLQJ D PDS RI LQWHUUHODWLRQV ZLWKWKHJRDORIPHHWLQJDOOUHTXLUHPHQWV :6 /HDQ  'HVLJQ %XLOG :RUNVKRS (YDOXDWLRQ RI ORZ  PHGLXP  KLJKULVN VROXWLRQV FRQYHUJLQJ  FRQFHSWVLQWRIRXULGHDVDQGUHYLHZLQJWKHPZLWKSURJUDPOHDGHUV :6 3URMHFW&RPPXQLFDWLRQ:RUNVKRS,PSURYLQJFRPPXQLFDWLRQEHWZHHQGLIIHUHQWORFDWLRQVE\GLVFXVVLQJ DQGHYDOXDWLRQGLIIHUHQWVROXWLRQV $ PDSSLQJ RI WKH H[LVWLQJ ZRUNVKRSV DJDLQVW WKH /HDQ (QDEOHUV ZKLFK ZDV GRQH E\ WKH SDUWLFLSDQWV RI WKH ZRUNVKRSUHVXOWHGLQ)LJ7KHILJXUHVKRZVRQHFROXPQIRUHDFK/HDQ3ULQFLSOHOLVWLQJWKH/HDQ(QDEOHUVWKDW IDOO LQ HDFK 3ULQFLSOH 7KH QXPEHU LQ WKH FRORUHG ILHOG LQGLFDWH WKH DPRXQW RI WLPHV WKH /HDQ (QDEOHU KDV EHHQ LGHQWLILHG DV EHLQJ DGGUHVVHG LQ D ZRUNVKRS 7KH WKUHHFRORU FRGH LQGLFDWHV LI WKHUH LV PRUH WKDQ RQH ZRUNVKRS JUHHQ RQO\RQH\HOORZ RUQRZRUNVKRSVDWDOOUHG WKDWDGGUHVVWKHSDUWLFXODU/HDQ(QDEOHU

 )LJ0DSSLQJRI/HDQ(QDEOHUVWRH[LVWLQJZRUNVKRSV )LJVKRZVWKDWRIDOO/HDQ(QDEOHUVDUHEHLQJDGGUHVVHGWKURXJKDWOHDVWRQHRIWKHHOHYHQZRUNVKRSV )XUWKHUPRUHDOPRVWRIWKH/HDQ(QDEOHUVDUHEHLQJDGGUHVVHGE\PRUHWKDQRQHRIWKHZRUNVKRSV7KHPDLQ FRQFOXVLRQ WKDW ZDV GUDZQ IURP WKHVH UHVXOWV ZDV WKDW DOPRVW DOO /HDQ (QDEOHU DUH EHLQJ DGGUHVVHG LQ H[LVWLQJ FRUSRUDWHDQGDFDGHPLFZRUNVKRSV

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

3.2. Concluding the need for a Standardized Framework +DYLQJ D SRRO RI ZRUNVKRSV WKDW DGGUHVV WKH /HDQ (QDEOHUV ZRXOG EH D ILUVW VROXWLRQ IRU WKH ZRUNVKRSEDVHG LPSOHPHQWDWLRQRI/HDQ(QDEOHUV7KHUHDUHWKUHHPDMRUSUREOHPVZLWKWKLVDSSURDFKWKRXJK x 7KHZRUNVKRSVDUHQRWDOZD\VDFFHVVLEOHVLQFHPRVWRIWKHPKDYHEHHQGHYHORSHGLQVLGHDFRPSDQ\RU RUJDQL]DWLRQZKLFKZRXOGQ¶WVKDUHWKHFRQWHQWIRUFRPSHWLWLYHDGYDQWDJHUHDVRQV x 6LQFHWKHZRUNVKRSVFRPHIURPGLIIHUHQWVRXUFHVWKH\DOOXVHGLIIHUHQWIUDPHZRUNVDQGZRUNVKRSWHFKQLTXHV )RUWKHIDFLOLWDWRULWZRXOGEHKDUGWRDOLJQWKHPLQDZD\WKDWWKH\DUHDOOFRQGXFWHGLQDVLPLODUVW\OH x +DYLQJDVHWRISUHGHILQHGZRUNVKRSVZKLFKDOOLPSOHPHQWVSHFLILF/HDQ(QDEOHUVFDXVHVWKHSUREOHPWKDWLID VHWRI/HDQ(QDEOHUVVKDOOEHLPSOHPHQWHGWRJHWKHUPXOWLSOHZRUNVKRSVZRXOGQHHGWREHFRQGXFWHGLQVWHDGRI MXVWRQHWKDWFRYHUVWKHPDOO ,Q H[SHUW LQWHUYLHZV WKHVH FRQFOXVLRQV ZHUH FRQILUPHG  +DYLQJ D FRPSOHWHO\ VWDQGDUGL]HG VHW RI ZRUNVKRSV ZRXOGQRWEHWKHULJKWWKLQJWRGHYHORS³VLQFHWKHSUHVHQWHUXVXDOO\ZDQWVWREHDEOHWRDGDSWWKHPDWHULDOWRKLVRU WKHFRPSDQLHVVW\OHRIWHDFKLQJDQGWUDLQLQJ´ %DVHG RQ WKHVH IDFWV WKH FRQFOXVLRQ LV GUDZQ WKDW D QHZ IUDPHZRUN LV QHFHVVDU\ ZKLFK IDFLOLWDWHV WKH GHYHORSPHQWRILPSOHPHQWDWLRQZRUNVKRSVIRU/HDQ(QDEOHUV7KLVFRQFOXVLRQZDVYDOLGDWHGWKURXJKDIRFXVJURXS DJURXSRIVXEMHFWPDWWHUH[SHUWVDVZHOODVWKURXJKDQH[SHUWLQWHUYLHZ 7KH/HDQ(QDEOHU,PSOHPHQWDWLRQ:RUNVKRS)UDPHZRUN/(,:)UDPHZRUN  4.1. The Requirements of the LEIW-Framework 7KHZRUNVKRSGHYHORSPHQWSURFHVVGHYHORSHGE\0F*LOO8QLYHUVLW\VHUYHGDVDVWUXFWXUHIRUWKHGHYHORSPHQW RIWKHUHTXLUHPHQWV,QDILUVWVWHSWKHSURFHVVZDVEURNHQGRZQLQWRLWVWKUHHPDLQHOHPHQWV WKHSUHGHILQLWLRQ SKDVH   WKH GHYHORSPHQW SKDVH DQG   WKH JHQHUDO FRQGLWLRQV SKDVH 1H[W WKH SURFHVV VWHSV ZHUH FOXVWHUHG LQ WKUHHGLIIHUHQWFDWHJRULHV VWHSVWKDWDIUDPHZRUNFRXOGFRYHU VWHSVWKDWDUHQHHGHGDVLQSXWDQG VWHSVWKDW QHHGWREHUXQWKURXJKE\WKHXVHURIWKHIUDPHZRUNRUWKHFRUSRUDWLRQ7KLVEUHDNGRZQDQDO\VLVLVGHPRQVWUDWHGLQ )LJ

)LJ&OXVWHULQJRIWKHVWDQGDUGZRUNVKRSGHYHORSPHQWSURFHVV

647

648

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

2Q WKH LQSXW VLGH RI WKH /(,:)UDPHZRUN WKHUH DUH WKH /HDQ (QDEOHUV DQ DVVHVVPHQW RI ZKLFK DUHDV WKH ZRUNVKRSVKRXOGIRFXVRQDOVRUHIHUUHGWRDVWKHLPSDFWDUHDVWKHKLHUDUFK\OHYHOWKDWVKDOOEHDGGUHVVHGDQGWKH DYDLODEOH WLPH 7KH /(,:)UDPHZRUN WKHQ XVHV WKLV LQIRUPDWLRQ LQ WR EXLOG D EOXHSULQW RI WKH ZRUNVKRS D UHFRPPHQGDWLRQRQWKHGHPRQVWUDWLRQWRROVDQGZRUNVKRSPHWKRGVDVZHOODVWKHWRSLFDOFRQWHQWIRUWKHZRUNVKRS

)LJ5HTXLUHPHQWVRIWKH/(,:)UDPHZRUN

4.2. The Elements of the LEIW-Framework $FFRUGLQJWRWKHUHTXLUHPHQWVWKHIRXUPDLQHOHPHQWVRIWKH/(,:)UDPHZRUNZHUHGHILQHGDVWKHIROORZLQJ x x x x

$VVHVVPHQWRIWKHKLJKSRWHQWLDO/HDQ(QDEOHUV 'HILQLWLRQRIWKHZRUNVKRSJRDOVREMHFWLYHVDQGZRUNVKRSWRSLFV 3URYLGLQJDQDJHQGDVWUXFWXUHWRSLFDOFRQWHQWDQGDQHYDOXDWLRQVXUYH\ 0DWFKLQJZRUNVKRSVPHWKRGVWRWKHSDUWLFLSDQWVDQGJLYLQJDUHFRPPHQGDWLRQRQZKLFKGHPRQVWUDWLRQWRROV VKRXOGEHXVHGIRUWKHVHOHFWHG/HDQ(QDEOHUV

)LJ7KH/(,:)UDPHZRUN

 ,QWKHIROORZLQJVHFWLRQVHDFKRQHRIWKHIRXUHOHPHQWVZLOOEHGHVFULEHGLQPRUHGHWDLO 4.3. The Assessment Process of the LEIW-Framework $VVKRZQLQ)LJWKHDVVHVVPHQWSURFHVVRIWKH/(,:)UDPHZRUNFRQVLVWVRIWKHIROORZLQJWKUHHVWHSV 

)LJ7KUHHVWHSDVVHVVPHQWSURFHVVRIWKH/(,:)UDPHZRUN

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

7KHSULRULWL]DWLRQRIWKH/HDQ(QDEOHUVLVGRQHRQDULVNEDVHGDSSURDFK7KHDSSURDFKKDVEHHQGHYHORSHGE\ )5,7= ZKLFKVHUYHGDVWKHPDLQVRXUFHIRUWKLVFKDSWHU7KHDVVHVVPHQWEDVLFDOO\FRQVLVWVRIWZRSDUWV  $QHVWLPDWLRQRIWKHFULWLFDOLW\RIWKHWRSWHQFKDOOHQJHVLQHQJLQHHULQJSURJUDPVDQG DPDWXULW\DVVHVVPHQWRI WKH/HDQ(QDEOHUV7KHFULWLFDOLW\RIWKHULVNVLVDVVHVVHGE\ILUVWSURYLGLQJWKHOLVWRIFKDOOHQJHVDVNLQJWRHVWLPDWH WKHLU OLNHOLKRRG WR RFFXU ,Q D VHFRQG VWHS WKHLU LPSDFW LV DVVHVVHG EDVHG RQ D WKUHHOHYHO VFDOH 7KHVH YDOXHV PXOWLSOLHG UHVXOW LQ WKH FULWLFDOLW\ RI HDFK ULVN 7KH PDWXULW\ OHYHO RI WKH /HDQ (QDEOHUV DUH TXHULHG E\ JLYLQJ WKH SRVVLELOLW\RIFKRRVLQJIURPILYHOHYHOVRIPDWXULW\IRUHDFK/HDQ(QDEOHUEDVHGRQWKH&00,6FDOH %DVHGRQDPDSSLQJRIWKH/HDQ(QDEOHUVDJDLQVWWKHWRSWHQFKDOOHQJHVWKHWRROWKHQFKRRVHVWKH/HDQ(QDEOHUV WKDWDGGUHVVWKHPRVWFKDOOHQJHVDQGWKDWKDYHWKHORZHVWGHJUHHRILPSOHPHQWDWLRQ7KLVUHVXOWVLQDOLVWRIWHQ/HDQ (QDEOHUVWKDWKDYHWKHKLJKHVWSRWHQWLDORIUHGXFLQJULVNRQFHWKH\DUHLPSOHPHQWHGWRDKLJKHUGHJUHH,QWKHIXUWKHU FRQWH[WWKHVH/HDQ(QDEOHUVDUHUHIHUUHGWRDVKLJKSRWHQWLDO/HDQ(QDEOHUV ,QRUGHUWRUHGXFHWKHFRPSOH[LW\RIWKH/HDQ(QDEOHUVDQGJLYHWKHXVHURIWKHIUDPHZRUNDEHWWHUXQGHUVWDQGLQJ ZKLFKEURDGHUDUHDWKHLPSURYHPHQWDQGFKDQJHHIIRUWVVKRXOGEHIRFXVHGRQWKHKLJKSRWHQWLDO/HDQ(QDEOHUVDUH PDSSHGDJDLQVWLPSDFWDUHDV7KHLPSDFWDUHDVDUHDQHZO\LQWURGXFHGVHWRIWKUHHDUHDVZKLFKDUHPXWXDOH[FOXVLYH DQGFROOHFWLYHO\H[KDXVWLYH(DFK/HDQ(QDEOHUFDQEHPDWFKHGXQDPELJXRXVO\WRRQHRIWKHWKUHHDUHDV 7KH LPSDFW DUHDV ZHUH GHULYHG IURP WKH 7R\RWD 3URGXFWLRQ 6\VWHP 736  DQG DGMXVWHG WR WKH ILHOG RI /HDQ SURJUDP PDQDJHPHQW DQG ILQDOO\ YDOLGDWHG 7KH 736 FDQ EH VHHQ DV DQ LQWHJUDWHG DQG LQWHUGHSHQGHQW V\VWHP LQYROYLQJ PDQ\ HOHPHQWV WKDW FDQ EH VSOLW LQ D WULDQJOH DV VKRZQ LQ )LJ  FRQVLVWLQJ RI SKLORVRSK\ WRROV DQG WHFKQLTXHVSUDFWLFDO DQGPDQDJHPHQWOHDGHUVKLS ZKLFKLVEXLOWDURXQGWKHSHRSOHRIDFRUSRUDWLRQ

)LJ7KH,PSDFW$UHDVRI/HDQ(QDEOHUV

,W LV WKHQ DVVHVVHG ZKLFK LPSDFW DUHD QHHGV WR EH DGGUHVVHG RQ ZKLFK RI WKH WKUHH KLHUDUFK\ OHYHOV XSSHU OHDGHUVKLSPLGGOHPDQDJHPHQWSUDFWLWLRQHUVOHYHO7KLVDVVHVVPHQWLVGRQHWKURXJKDPDWUL[VWUXFWXUHLQZKLFKWKH XVHUFDQVHOHFWWKHFULWLFDOILHOGVLQDGURSGRZQPHQX 4.4. Defining Topics and Objectives )RUHDFKFRPELQDWLRQRIKLHUDUFK\OHYHODQGLPSDFWDUHDRQHZRUNVKRSWRSLFDQGJRDOLVGHILQHG)LJVKRZV WKHPDWUL[ZLWKWKHQLQHGLIIHUHQWWRSLFVWKDWWKH/(,:)UDPHZRUNSURYLGHV 7KH REMHFWLYH RI HYHU\ ZRUNVKRS LV WKDW KDYLQJ FRPSOHWHG WKH ZRUNVKRS WKH SDUWLFLSDQWV ZLOO KDYH D GHHS XQGHUVWDQGLQJ DERXW WKH LPSOHPHQWHG /HDQ (QDEOHUV DQG ZLOO EH DEOH WR FRQGXFW D ZRUNVKRS RQ WKH FRQWHQW 7KH PLVVLRQ LV WKDW WKH SDUWLFLSDQWV RI WKH ZRUNVKRS ZLOO OHDUQ WKH WKHRU\ DERXW WKH /HDQ (QDEOHUV DQG DOVR GHHSO\ XQGHUVWDQGWKH/HDQ(QDEOHUVWKURXJKHQJDJLQJGHPRQVWUDWLRQWRROV (DFK WRSLF FLUFOH LQ )LJ  RYHUODSV RYHU WZR ILHOGV 7KLV DOOXGHV WR WKH UHFRPPHQGDWLRQ RI XVLQJ D PXOWLSOLHU PRGHODOVRUHIHUUHGWRDVWKH³WUDLQWKHWUDLQHU´DSSURDFKIRUWKHLPSOHPHQWDWLRQRIWKH/HDQ(QDEOHUV,QWKDWFDVH WKHIDFLOLWDWRUVZRXOGRQO\WUDLQWKHXSSHUOHDGHUVKLSLQDZD\WKDWWKH\FDQFDUU\RQWKHNQRZOHGJHWRWKHPLGGOH PDQDJHPHQWDQGVRRQ

649

650

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

)LJ0XOWLSOLHUVWUXFWXUHZLWKWRSLFVRIZRUNVKRSV

4.5. Providing the Workshop Content 7KH /(,:)UDPHZRUN ZLOO DXWRPDWLFDOO\ SURYLGH WKH XVHU ZLWK RQH VOLGH GHFN RQ WKH WRSLFDO FRQWHQW IRU HDFK /HDQ (QDEOHU (DFK VOLGH GHFN KDV WKH VDPH VWUXFWXUH ZKLFK LVEDVHG RQ WKH VWUXFWXUH RI WKH :LNLSHGLD IRU /HDQ SURJUDP PDQDJHPHQW 7KH IROORZLQJ OLVW VKRZV WKH PDLQ WRSLFV RI HDFK HOHPHQW RI WKLV VWUXFWXUH DQG EULHIO\ H[SODLQVZKDWHDFKHOHPHQWLVDERXW  (QDEOHU2YHUYLHZ(DFK/HDQ(QDEOHULVLQWURGXFHGFRQWDLQLQJWKHPRVWLPSRUWDQWIDFWV  6XEHQDEOHUV,QWKLVVHFWLRQHDFK6XEHQDEOHURIWKH/HDQ(QDEOHULVLQWURGXFHGVHSDUDWHO\  ,PSOHPHQWDWLRQVXJJHVWLRQ+HUHDVKRUWOLVWRILPSRUWDQWSRLQWVDERXWWKHLPSOHPHQWDWLRQRIWKH/HDQ(QDEOHU LVJLYHQ7KHVHVXJJHVWLRQVDUHEDVHGRQWKHH[SHULHQFHRIWKHDXWKRUV  &RQFUHWHH[DPSOH(DFKH[DPSOHJLYHVDVKRUWGHVFULSWLRQRIDFHUWDLQSURJUDPLQZKLFKWKH/HDQ(QDEOHUZDV LPSOHPHQWHG VXFFHVVIXOO\ RU LQ ZKLFK WKH /HDQ (QDEOHU FDPH LQWR SOD\ 7KH H[DPSOHV DUH EDVHG RQ WKH H[SHULHQFHRIWKHDXWKRUV  0HWULFV7KHOLVWRIPHWULFVLQWURGXFHVZKLFKPHWULFVVKRXOGEHIRUPHDVXULQJWKHVXFFHVVRIWKH/HDQ(QDEOHU 7KHVHVXJJHVWLRQVDUHEDVHGRQUHVHDUFKFRQGXFWHGDWWKH&RQVRUWLXPIRU(QJLQHHULQJ3URJUDPV  $GGLWLRQDO UHDGLQJ)LQDOO\DOLVWRIERRNVIRUDGGLWLRQDOUHDGLQJVZLOOEHSURYLGHGWRWKHSDUWLFLSDQWVRIWKH ZRUNVKRSV ,QDGGLWLRQWKH/(,:)UDPHZRUNSURYLGHVDQDJHQGDDVZHOODVDQHYDOXDWLRQVXUYH\WKDWFDQEHGLUHFWO\DSSOLHGLQ DZRUNVKRSRUWUDLQLQJ 4.6. Matching Educational Methods, Active Learning Tools and Workshops 7KH HGXFDWLRQDO PHWKRGVIRU DZRUNVKRSRU WUDLQLQJ PXVW IXOILOO WKHIROORZLQJUHTXLUHPHQWV 7KH\PXVW ILW WKH QHHGV DQG WKH SHUIRUPDQFH OHYHO RI HDFK SDUWLFLSDQW DQG LW PXVW DGGUHVV WKH WDUJHW REMHFWLYHV 7KH SURFHVV RI PDWFKLQJWKHHGXFDWLRQDOPHWKRGVWRWKHZRUNVKRSWKDWLVEHLQJEXLOWE\WKHXVHURIWKH/(,:)UDPHZRUNLVEXLOW DURXQGWKHVHWZRPDLQFULWHULD 7KH IROORZLQJ OLVW VKRZV DQGUDJRJLFDO IDFWRUV WKDW KDYH EHHQ LGHQWLILHG DV LPSRUWDQW IRU WKH GHVFULSWLRQ RI WKH GLIIHUHQFHLQQHHGVDQGSHUIRUPDQFHOHYHOVEHWZHHQ\RXQJDQGDGXOWOHDUQHUV x 6HOIHIILFDF\ x 3UHYLRXVH[SHULHQFHV x 0RWLYDWLRQ

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

x *RDORULHQWDWLRQ x $WWLWXGH x &RJQLWLYHDELOLW\ 7KHILUVWIRXUIDFWRUVKDYHEHHQLGHQWLILHGDVWKHPRVWUHOHYDQWIRUGHVFULELQJWKHGLIIHUHQFHEHWZHHQROGHUPRUH H[SHULHQFHG OHDUQHUV DQG \RXQJHU LQH[SHULHQFHG OHDUQHUV ,Q RUGHU WR ILQG WKH ULJKW HGXFDWLRQDO PHWKRGV IRU HDFK KLHUDUFK\ OHYHO WKH DVVXPSWLRQ KDV EHHQ PDGH WKDW WKHUH DUH SDUDOOHOV EHWZHHQ WKH GLIIHUHQFH RI SHGDJRJ\ WR DQGUDJRJ\DQGWKHSUDFWLWLRQHUVOHYHOWRWKHXSSHUOHDGHUVKLSOHYHO7KXVDVPDWXUHDGXOWOHDUQHUVXSSHUOHDGHUVDUH JHQHUDOO\PRUHVHOIGLUHFWHGPRUHH[SHULHQFHGPRUHLQWULQVLFDOO\PRWLYDWHGDQGPRUHSHUIRUPDQFHFHQWHUHGWKDQ SUDFWLWLRQHUV )LJ  VKRZV KRZ WKH FRPSDULVRQ EHWZHHQ SHGDJRJ\ WR DQGUDJRJ\ ZDV XVHG WR H[SODLQ WKH DQGUDJRJLFDOGLIIHUHQFHVEHWZHHQSUDFWLWLRQHUVPLGGOHPDQDJHPHQWDQGXSSHUOHDGHUVKLS

)LJ$QDORJ\RISHGDJRJ\YVDQGUDJRJ\WRSUDFWLWLRQHUVYVOHDGHUVKLS

%DVHG RQ WKLV DVVXPSWLRQ WKH /(,:)UDPHZRUN HYDOXDWHV QXPHURXV HGXFDWLRQDO PHWKRGV LQ D XWLOLW\ YDOXH DQDO\VLV$VDUHVXOWWKHXVHUJHWVDVXJJHVWLRQRIDQXPEHURIPHWKRGVWKDWFDQGLUHFWO\EHDSSOLHGLQWKHZRUNVKRS )RUFXVWRPL]DWLRQUHDVRQVWKHXVHUFDQFKDQJHWKHGHJUHHVRILPSRUWDQFHRIHDFKRIWKHIDFWRUVLQWKHXWLOLW\YDOXH DQDO\VLV )LQGLQJWKHULJKWGHPRQVWUDWLRQWRROVIRUWKHZRUNVKRSLVWKHQH[WJRDORIWKH/(,:)UDPHZRUN7KHVHOHFWLRQ SURFHVV LV EDVHG RQ D VHFRQG XWLOLW\ YDOXH DQDO\VLV 7KH GHJUHH WR ZKLFK D GHPRQVWUDWLRQ WRRO IXOILOOV WKH WDUJHW FULWHULDRILPSOHPHQWLQJHDFKRIWKHVHOHFWHG/HDQ(QDEOHUVZDVHYDOXDWHGDFFRUGLQJWRWKHSURFHVVVKRZQLQ)LJ 

)LJ3URFHVVIRUWKHHYDOXDWLRQRIWKHGHJUHHWRZKLFKDGHPRQVWUDWLRQWRRODGGUHVVHVWKH/HDQ(QDEOHUV

7KURXJK WKLV SURFHVV DQ\ GHPRQVWUDWLRQ WRRO FDQ EH PDSSHG WR WKH /HDQ (QDEOHUV 7KH /(,:)UDPHZRUN LV GHVLJQHGVRWKDWIXUWKHUGHPRQVWUDWLRQWRROVFDQHDVLO\EHLPSOHPHQWHGLQWRWKHIUDPHZRUN7KHWRWDOXWLOLW\YDOXHRI HDFKGHPRQVWUDWLRQWRROLVFDOFXODWHGE\DGGLQJXSWKHIXOILOOPHQWIDFWRUV7KHGHPRQVWUDWLRQWRROZLWKWKHKLJKHVW XWLOLW\YDOXHLVWKHQUHWXUQHGWRWKHXVHURIWKH/(,:)UDPHZRUNDVDUHFRPPHQGDWLRQ

651

652

Kilian Gersing et al. / Procedia Computer Science 28 (2014) 643 – 652

4.7. Validating the LEIW-Framework 7KHIHHGEDFNIURPWKHIRFXVJURXSDVZHOODVWKHLQWHUYLHZSDUWQHUZDVJHQHUDOO\YHU\SRVLWLYH7KHIROORZLQJ SDUDJUDSKVVXPXSWKHIHHGEDFNDERXWWKH/(,:)UDPHZRUN x 7KH/(,:)UDPHZRUNLVHDV\WRXQGHUVWDQGZLWKDOLWWOHVXSSRUW,WLVYHU\SRZHUIXODQGKHOSIXORQFHD FRPSDQ\KDVGHFLGHGWRPRYHLQWKHGLUHFWLRQRIWKH/HDQ(QDEOHUV7KHWRROZRXOGLQVWDQWO\WHOO\RXZKDWWR GRZKLFKLVYHU\KHOSIXO x $QLQWHUHVWLQJXVHFDVHIRUWKHIUDPHZRUNZRXOGEHWRXVHLWDVDJXLGHOLQHLQWKHVWUXFWXUHGGHYHORSPHQWSURFHVV RIDZRUNVKRS7KHIDFLOLWDWRUZRXOGZDONWKURXJKWKHIURQWHQGDQGWKHEDFNHQGPDWHULDORIWKHIUDPHZRUN ZKLOHEXLOGLQJDQHZZRUNVKRS:KLOHGRLQJVRWKHPDWHULDOZRXOGKHOSKLPVWUXFWXUHKLVWKRXJKWVDQGJHW LQWHUHVWLQJLGHDVIRUDOOWKHGLIIHUHQWHOHPHQWVRIWKHIUDPHZRUN x )XUWKHUPRUHWKHIUDPHZRUNGHOLYHUVPDWHULDOVDQGFRQWHQWIRUWKHZRUNVKRSZKLFKFDQEHYHU\KHOSIXODQG VDYHVDORWRIWLPH3XWWLQJWHPSODWHVWRJHWKHUDQGVXJJHVWLQJLPSOHPHQWDWLRQDQGGHPRQVWUDWLRQWRROVLVYHU\ KHOSIXOIRUWKHIDFLOLWDWRU,QPDQ\FDVHVZLOOLQJQHVVWRFKDQJHLVWKHUHEXWXVXDOO\LQHYHU\GD\ZRUNWKHUHLVD ODFNRIWLPHWRWXUQLWLQWRDFWXDOFRQWHQW7KH/(,:)UDPHZRUNZRXOGFOHDUO\VROYHWKLVSUREOHP 5HIHUHQFHV  2HKPHQ-HWDO/HDQ(QDEOHUIRU0DQDJLQJ(QJLQHHULQJ3URJUDPVVWHG&DPEULGJH-RLQW0,730, ,1&26(&RPPXQLW\RI3UDFWLVHRQ/HDQLQ3URJUDP0DQDJHPHQW  30,$*XLGHWRWKH3URMHFW0DQDJHPHQW%RG\RI.QRZOHGJH30%2.*XLGH1HZWRQ6TXDUH3URMHFW 0DQDJHPHQW,QVWLWXWH,QF  +DVNLQV&)RUVEHUJ..UXHJHU06\VWHPVHQJLQHHULQJKDQGERRN,PSURYLQJWKHSURFHVVIRU6( SUDFWLWLRQHUV6DQ'LHJR,1&26(  :RPDFN-3-RQHV'7/HDQ7KLQNLQJ%DQLVK:DVWHDQG&UHDWH:HDOWKLQ\RXU2UJDQL]DWLRQQGHG1HZ