Evaluation of quiz using a statistical calculation in learning ...

(YDOXDWLRQRITXL]XVLQJDVWDWLVWLFDOFDOFXODWLRQLQ /HDUQLQJ0DQDJHPHQW6\VWHP 

0DWHD0DUNLü 8QLYHUVLW\RI0RVWDU )DFXOW\RI6FLHQFHDQG(GXFDWLRQ8QLYHUVLW\RI0RVWDU 0RVWDU%RVQLDDQG+HU]HJRYLQD PDWHDPDUNLF#JPDLOFRP   $EVWUDFW²/HDUQLQJ0DQDJHPHQW6\VWHPVSURYLGHXVTXL]]HV 0RRGOH FDSWXUHV DOO LQIRUPDWLRQ DERXW VWXGHQW DFWLYLWLHV IRUDVVHVVPHQWRIVWXGHQWV¶NQRZOHGJHDQGDOVRDVWDWLVWLFUHSRUW DQG LQ WKH IRUP RI WKH DQDO\WLFDO UHSRUWV SURYLGHV WKH WHDFKHU RIWKRVHTXL]]HV2XUDLPZDVWRXVHVWDWLVWLFUHSRUWVDVDWRROIRU ZLWKLQVLJKWLQWRVWXGHQW¶VEHKDYLRURQWKHV\VWHP,QDGGLWLRQ TXDOLW\DVVHVVPHQWRIWKHILYHTXL]]HVGXULQJWZRDFDGHPLF\HDUV WR UHYLHZLQJ VWXGHQW¶V DFWLYLWLHV VXFK DV WKH QXPEHU RI  DQG  7KH H[SHULPHQW FRYHUHG ILYH 0RRGOH DFFHVVHVDFFHVVWLPHWKHQXPEHURIYLHZHGPRGXOHV0RRGOH TXL]]HV  TXHVWLRQV DQG  VWXGHQWV :H H[DPLQHG WKH RIIHUV D ZHDOWK RI TXL] VWDWLVWLF UHSRUWV PRVW RI ZKLFK QHHG UHOLDELOLW\RIWKHTXL]]HVDVWKH\DUHDQLPSRUWDQWSDUWRIOHDUQLQJ IXUWKHU LQWHUSUHWDWLRQ IRU EHWWHU XQGHUVWDQGLQJ 4XL] VWDWLVWLF DQGWHDFKLQJSURFHVV,WHPDQDO\VLVRIWKHTXL]LVREVHUYHGDVWKH UHSRUWWKDW0RRGOHRIIHUVJLYHVDVWDWLVWLFDODQDO\VLVRITXL]DQG RQH VWHS LQ OHDUQLQJ DQDO\WLFV F\FOH :H DOVR GLVFXVV WKH TXHVWLRQVZLWKLQLW>@ FRQQHFWLRQ EHWZHHQ TXHVWLRQ W\SH DQG VWXGHQWV¶ UHVXOWV RQ TXL]]HV

.H\ZRUGV² /HDUQLQJ DQDO\WLFV 6WDWLVWLFDO DQDO\VLV 4XL] VWDWLVWLFUHSRUW0RRGOH(DVVHVVPHQW

, ,1752'8&7,21 ,QIRUPDWLRQ DQG FRPPXQLFDWLRQ WHFKQRORJ\ ,&7  KDV EHFRPH DQ LQWHJUDO SDUW RI HYHU\GD\ OLIH VR HGXFDWLRQ LV QR H[FHSWLRQ 7KH XVH RI ,&7 LQ HGXFDWLRQ KDV OHG WR WKH HPHUJHQFHRIHOHDUQLQJ7KHWHFKQRORJLFDOEDVLVRIHOHDUQLQJ LV ,QWHUQHW DQG RWKHU UHODWHG FRPPXQLFDWLRQ WHFKQRORJLHV 8VHU¶VDFWLYLWLHVLQHGXFDWLRQDODQGRWKHURQOLQHHQYLURQPHQWV OHDYHWUDFHVWKDWPDQLIHVWLQWKHIRUPRIDODUJHQXPEHURIORJV $ VLJQLILFDQW QXPEHU RI HGXFDWLRQDO ORJ GDWD LV DYDLODEOH WR VFLHQWLVWVWKURXJKSXEOLFDUFKLYHV7KHPDLQTXHVWLRQLVKRZWR XVH WKHVH UHFRUGV WR LPSURYH WKH SURFHVV RI OHDUQLQJ DQG WHDFKLQJ7KHDQVZHUWRWKLVTXHVWLRQLVOHDUQLQJDQDO\WLFV $FFRUGLQJWRWKHGHILQLWLRQJLYHQDW WKH)LUVW,QWHUQDWLRQDO &RQIHUHQFH RQ /HDUQLQJ DQG .QRZOHGJH $QDO\WLFV /$.   OHDUQLQJ DQDO\WLFV LQFOXGHV PHDVXUHPHQW FROOHFWLRQ DQDO\VLVDQGUHSRUWLQJRIGDWDDERXWOHDUQHUVDQGWKHLUFRQWH[WV $ JURXS RI DXWKRUV >@ FRQVLGHUV OHDUQLQJ DQDO\WLFV DV DQ LWHUDWLYHF\FOHZKLFKLQFOXGHWKUHHPDMRUVWHSVGDWDFROOHFWLRQ DQGSUHSURFHVVLQJDQDO\WLFVDQGDFWLRQDQGSRVWSURFHVVLQJ (GXFDWLRQ LQFOXGHV HYDOXDWLRQ RI D VWXGHQW¶V NQRZOHGJH ZKLFKLVFRQGXFWHGWKURXJKTXL]]HVVRWKHRQOLQHHGXFDWLRQDO HQYLURQPHQWVDUHQRH[FHSWLRQ.QRZOHGJHWHVWLQJRQWKH/06 /HDUQLQJ 0DQDJHPHQW 6\VWHP  VXFK DV 0RRGOH 0RGXODU 2EMHFW 2ULHQWHG/HDUQLQJ (QYLURQPHQW  KDV PDQ\ DGYDQWDJHV RYHUWKHWUDGLWLRQDOWHVWV³RQSDSHU´2QHRIWKHDGYDQWDJHVLV DXWRPDWLFDOO\ JUDGLQJ RI WHVWV ZKLFK PDNHV ZRUN HDVLHU IRU WHDFKHUV 6WXGHQWV FDQ DXWRPDWLFDOO\ JHW IHHGEDFN DERXW WKH JUDGHDQGVXFFHVVRQWKHTXL]

,WHP DQDO\VLV RI WKH TXL] LV SDUW RI WKH WKLUG VWHS LQ WKH OHDUQLQJ DQDO\WLFV F\FOH SRVWSURFHVVLQJ 7KH WLPH LW WDNHV WR ³PDQXDOO\´ FRPSXWH VWDWLVWLFDO LQGLFDWRUV FDQ EH XVHG WR DQDO\]H TXL] SDUDPHWHUV LQ RUGHU WR LPSURYH WKH SURFHVV RI HYDOXDWLQJ VWXGHQWV¶ NQRZOHGJH  ,W LV RQH WKH PRVW LPSRUWDQW WRROV WR LQFUHDVH WKH HIIHFWLYHQHVV RI WKH WHVW DQG LW¶V UHFRPPHQGHG WKDW WKH WHDFKHU SHUIRUPV DQDO\VLV RI VWDWLVWLF LQGLFDWRUV DIWHU WKH WHVW ZKLFK DOORZV FRUUHFWLRQV DQG DIIHFWV WKHILQDOUHVXOW %HVLGHWKHILUVWVHFWLRQWKLVSDSHUKDVWKUHHPRUHVHFWLRQV ,Q WKH VHFRQG VHFWLRQ ZH GHVFULEHG WKH PHWKRGRORJ\ DQG H[SHULPHQWIORZ7KLUGVHFWLRQ±'DWDDQDO\VLVDQGGLVFXVVLRQ LV GLYLGHG LQWR WKUHH SDUWV TXL] VWDWLVWLF LQWHUSUHWDWLRQ TXL] VWUXFWXUH DQDO\VLV DQG FRUUHODWLRQ EHWZHHQ TXHVWLRQ W\SH DQG IDFLOLW\DQGGLVFULPLQDWLRQLQGH[ZKHUHZHSUHVHQWUHVXOWVDQG GLVFXVV WKHP ,Q WKH ODVW VHFWLRQ ZH SUHVHQWHG WKH PDLQ FRQFOXVLRQVRIWKHFRQGXFWHGH[SHULPHQW 5HVHDUFKTXHVWLRQV 7KH PDLQJRDORIWKLVUHVHDUFKLVWRDFKLHYHWKHIROORZLQJ REMHFWLYHV  $QDO\]H0RRGOHTXL]VWDWLVWLFUHSRUWVRIILYHTXL]]HV ZKLFKKDYHEHHQKHOGRYHUWZRDFDGHPLF\HDUV  'LVFXVV WKH LPSDFW RI TXHVWLRQ W\SHV RQ WKH RYHUDOO VWXGHQWV¶VXFFHVVRQWKHTXL] ,, 0(7+2'2/2*@:HGLGQ¶WXVHWUXHIDOVHW\SH RITXHVWLRQVEHFDXVHRIWKH ELJFKDQFHRIUDQGRPO\JXHVVLQJ WKHULJKWDQVZHUZLWKRXWXQGHUVWDQGLQJWKHTXHVWLRQ7KHOLVWRI TXL]]HVDQGTXHVWLRQW\SHVSHUHDFKTXL]LVJLYHQLQ7DEOH 7$%/(,,

180%(5$1'7@ 2QH RI WKH GLUHFW XVHV WKDW FDQ EH PDGH RI LQGLFHV RI GLVFULPLQDWLRQLVWKHVHOHFWLRQRIWKHEHVWLWHPVIRULQFOXVLRQLW LQDQLPSURYHGYHUVLRQRIWKHWHVW>@

 )LJ 7RWDOQXPEHURITXHVWLRQVRQILYHTXL]]HVDQGSHUFHQWDJHRIHDFK FDWHJRU\

  )DFLOLW\LQGH[ )DFLOLW\LQGH[LVDFWXDOO\WKHPHDQVFRUHRIWKHVWXGHQWVRQ DQLWHP:HVRUWHGRXWTXHVWLRQVWRZDUGVFDWHJRULHVRIIDFLOLW\ LQGH[)LJ 

   'LVFULPLQDWLRQLQGH[ :H VRUWHG RXW TXHVWLRQV SHU FDWHJRULHV RI GLVFULPLQDWLRQ LQGH[DQGSUHVHQWHGWKHP)LJ 

 )LJ

1XPEHURIWKHTXHVWLRQVSHUHDFKFDWHJRU\RIIDFLOLW\LQGH[



 )LJ 'LVFULPLQDWLRQLQGH[RIWKHILYHTXL]]HVDQGSHUFHQWDJHRIWKHHDFK FDWHJRU\

 9HU\ JRRG GLVFULPLQDWLRQ EHWZHHQ VWXGHQWV LQFOXGHV TXHVWLRQVZLWKGLVFULPLQDWLRQLQGH[DERYHZKDWZHKDYH RQTXHVWLRQVRIWKHWRWDOQXPEHURITXHVWLRQV ZKLFK LPSOLHV WKDW  RI WKH WRWDO QXPEHU RI TXHVWLRQV KDV GLVFULPLQDWLRQ LQGH[ EHORZ  :H KDYH  RI TXHVWLRQV ZLWK QHJDWLYH GLVFULPLQDWLRQ LQGH[ ZKDW PHDQV WKDW WKH\ DUH SUREDEO\LQYDOLG & &RUUHODWLRQEHWZHHQW\SHRITXHVWLRQDQGIDFLOLW\DQG GLVFULPLQDWLRQLQGH[ :H DOVR ZDQWHG WR H[DPLQH FRUUHODWLRQ EHWZHHQ IDFLOLW\ LQGH[DQGW\SHRITXHVWLRQDQGGLVFULPLQDWLRQLQGH[DQGW\SH RITXHVWLRQ

FDWHJRU\ ³)DLUO\ HDV\´ VR ZH FDQ FRQFOXGH WKDW WKH PDWFKLQJ TXHVWLRQVZHUHHDVLHUIRUVWXGHQWV 8VLQJ D VLPSOH DUHD FKDUW WR SUHVHQW WKH SHUFHQWDJH RI PXOWLSOH FKRLFH DQG PDWFKLQJ TXHVWLRQV SHU HDFK FDWHJRU\ RI GLVFULPLQDWLRQ LQGH[ ZH FDPH WR WKH UHVXOWV WKDW PXOWLSOH FKRLFH TXHVWLRQV KDYH KLJKHU YDOXHV RI GLVFULPLQDWLRQ LQGH[  RI PXOWLSOH FKRLFH TXHVWLRQV KDYH ³YHU\ JRRG´ DQG ³DGHTXDWH GLVFULPLQDWLRQ´ 2Q WKH RSSRVLWH VLGH  RI PDWFKLQJTXHVWLRQKDYH³YHU\ZHDNGLVFULPLQDWLRQ´RUWKH\DUH ³SUREDEO\LQYDOLG´)LJ 

$Q LQGHSHQGHQW VDPSOHV 7WHVW ZDV UXQ WR GHWHUPLQH LI WKHUHZHUHGLIIHUHQFHVLQVWXGHQWVVFRUHRQPXOWLSOHFKRLFHDQG PDWFKLQJTXHVWLRQV 7$%/(,9



5(68/762)77(67



9DULDEOH  0DWFKLQJ TXHVWLRQV 

9DULDEOH  0XOWLSOH TXHVWLRQV 

0HDQ





9DULDQFH





2EVHUYDWLRQV 1 





+\SRWKHVL]HG 0HDQ 'LIIHUHQFH





GI





W6WDW





37 W RQH WDLO





W&ULWLFDO RQHWDLO





 )LJ 3HUFHQWDJH RI PDWFKLQJ DQG PXOWLSOH FKRLFH TXHVWLRQV SHU FDWHJRULHVRIGLVFULPLQDWLRQLQGH[

  :H GHFLGHG WR FRQGXFW DQ DGGLWLRQDO VXUYH\ WR UH UHVHDUFKZKHWKHUVWXGHQWV¶RSLQLRQZLOOEHLQDFFRUGDQFHZLWK WKH UHVXOWV RI WWHVW :H XVHG /LNHUW¶V SRLQW VFDOH RQ 7 VWXGHQWV WR H[DPLQH WKHLU RSLQLRQ DERXW VWDWHPHQW ³0DWFKLQJ W\SHTXHVWLRQVDUHHDVLHUWKDQPXOWLSOHFKRLFHW\SHTXHVWLRQV´ $VZHFDQVHHRIWKHVWXGHQWVDJUHHVRPHZKDWDJUHHRU VWURQJO\ DJUHH ZLWK WKH JLYHQ VWDWHPHQW )LJ   7KLV VXUYH\ KDV FRQILUPHG WKDW PDWFKLQJ W\SH TXHVWLRQV ZHUH HDVLHU IRU PRUHWKDQKDOIVWXGHQWV  3% 5%

17%

37 W WZR WDLO





W&ULWLFDO WZRWDLO





3%

25%



36%

:HVHWWKHK\SRWKHVLV 11%

+—PXOWLSOHTXHVWLRQV —PDWFKLQJTXHVWLRQV +—PXOWLSOHTXHVWLRQV—PDWFKLQJTXHVWLRQV



 

76 WSYDOXH  VRZHUHMHFWHG+DQG ZHDFFHSWHG+ :HFDQFRQFOXGHWKDWWKHUHLVDVLJQLILFDQW VWDWLVWLFDO GLIIHUHQFH EHWZHHQ PXOWLSOH FKRLFH DQG PDWFKLQJ TXHVWLRQVXVHGRQWHVWDWWKHDOSKDOHYHO$VZHFDQVHH WKHUH LV D KLJKHU DYHUDJH PHDQ  RI IDFLOLW\ LQGH[HV RQ PDWFKLQJ TXHVWLRQV     7KH PHDQ JUDGH RI DOO PDWFKLQJ TXHVWLRQV EHORQJV WR WKH FDWHJRU\ ³(DV\´ DQG WKH DYHUDJH JUDGH RI PXOWLSOH FKRLFH TXHVWLRQV EHORQJV WR WKH

Strongly disagree

Disagree

Disagree somewhat

Undecided

Agree somewhat

Agree

Strongly agree

)LJ



5HVXOWVRISRLQW/LNHUW¶VVFDOH



,9 &21&/86,21 ,Q WKLV SDSHU ZH KDYH SUHVHQWHG UHVXOWV RI WKH DQDO\VLV RI VWDWLVWLFDO SDUDPHWHUV RI WKH ILYH TXL]]HV :H XVHG 0RRGOH VWDWLVWLFUHSRUWVIRULQWHUSUHWDWLRQRIWKHUHVXOWV ,QWHUPVRIWKHILUVWUHVHDUFKTXHVWLRQZHDQDO\]HG0RRGOH UHSRUWV LQ WZR SDUWV )LUVW SDUW LQFOXGHG DQDO\VLV RI WKH EDVLF VWDWLVWLFDOSDUDPHWHUVRIWKHZKROHTXL]6HFRQGSDUWRIDQDO\VLV LQFOXGHG LQWHUSUHWDWLRQ RI WKH TXL] VWUXFWXUH DQDO\VLV ZLWK HPSKDVLVRQIDFLOLW\LQGH[DQGGLVFULPLQDWLRQLQGH[ $V ZH XVHG PXOWLSOH FKRLFH DQG PDWFKLQJ TXHVWLRQV IRU PDNLQJ TXL]]HV LQ WHUP RI WKH VHFRQG UHVHDUFK TXHVWLRQ ZH ZDQWHG WR H[DPLQH FRUUHODWLRQ EHWZHHQ TXHVWLRQ W\SH DQG IDFLOLW\DQGGLVFULPLQDWLRQLQGH[:HFRQFOXGHGWKDWVWXGHQWV¶ VFRUHV RQ PDWFKLQJ TXHVWLRQV ZHUH KLJKHU ZKDW PDNHV WKHP HDVLHU IRU VWXGHQWV $GGLWLRQDO VXUYH\ KDV FRQILUPHG WKLV VWDWHPHQWDVZHGLUHFWO\DVNHGVWXGHQWVZKDWWKH\WKLQNZKLFK TXHVWLRQW\SHLVHDVLHUIRUWKHP$OVRPDWFKLQJTXHVWLRQVKDYH ORZHUYDOXHVRIGLVFULPLQDWLRQLQGH[0XOWLSOHFKRLFHTXHVWLRQV LQGLFDWHGWKDWWKH\DUHEHWWHUDWVRUWLQJRXWDEOHVWXGHQWVIURP WKRVHZKRDUHOHVVDEOH 2EWDLQHGUHVXOWVFDQEHXVHGIRULPSURYLQJTXL]]HVDQGH DVVHVVPHQWJHQHUDOO\$VWKHOHDUQLQJDQDO\WLFVLVFRQVLGHUHGDV LWHUDWLYH F\FOH DQG TXL] HYDOXDWLRQ LV WKH WKLUG VWHS RI WKDW F\FOHREWDLQHGUHVXOWVFDQEHXVHGIRUWKHQH[WF\FOH  



$&.12:/('*(0(176

7KH SDSHU LV SDUW RI WKH ZRUN VXSSRUWHG E\ WKH 2IILFH RI 1DYDO5HVHDUFKJUDQW1R1 5()(5(1&(6 >@ 0 $ &KDWWL $ / '\FNKRII 8 6FKURHGHU DQG + 7KV ³$ UHIHUHQFHPRGHOIRUOHDUQLQJDQDO\WLFV´,QW-7HFKQRO(QKDQF/HDUQYRO QRS >@ ³4XL] VWDWLVWLFV UHSRUW  0RRGOH'RFV´ >2QOLQH@ $YDLODEOH KWWSVGRFVPRRGOHRUJHQ4XL]BVWDWLVWLFVBUHSRUW >$FFHVVHG 0D\ @ >@ 0 * 6LPNLQ DQG : / .XHFKOHU ³0XOWLSOH&KRLFH 7HVWV DQG 6WXGHQW8QGHUVWDQGLQJ:KDW ,VWKH &RQQHFWLRQ"6XEMHFW$UHDV&RQVWUXFWHG 5HVSRQVH7HVWV6WXGHQW$VVHVVPHQWDQG 0XOWLSOH&KRLFH 7HVWV´'HFLV 6FL -,QQRY(GXFYROQR >@ 3 %XWFKHU ³4XL] UHSRUW VWDWLVWLFV´  >2QOLQH@ $YDLODEOH KWWSVGRFVPRRGOHRUJGHY4XL]BUHSRUWBVWDWLVWLFV>$FFHVVHG-XQ@ >@ 0 %ODQFR DQG 0 *LQRYDUW ³0RRGOH TXL]]HV IRU DVVHVVLQJ VWDWLVWLFDO WRSLFV LQ HQJLQHHULQJ VWXGLHV´ LQ 3URFHHGLQJV RI WKH -RLQW ,QWHUQDWLRQDO,*,3 >@ 0 %ODQFR $EHOODQ DQG 0 *LQRYDUW *LVEHUW ³2Q +RZ 0RRGOH 4XL]]HV &DQ &RQWULEXWH WR WKH )RUPDWLYH H$VVHVVPHQW RI )LUVW@ 5 / (EHO DQG ' $ )ULVELH (VVHQWLDOV RI HGXFDWLRQDO PHDVXUHPHQW)LIWK(QJOHZRRG&OLIIV1HZ-HUVH\3UHQWLFH+DOO,QWHUQDWLRQDO ,1&