An Efficient Mutual "nformation Based Feature Delection ... - GitHub

1 downloads 210 Views 437KB Size Report
Besides feature selection, the other main class of di- .... idea that features that exhibit a high degree of mutual info
          !  "#%$& (' )  *+ -,  .     / 01' 2436587 243:9 %@ ACBEDGFIHKJLD MONKPRQTS?UISWVYX.P[Z]\^VEQ`_`P[abUI_cVEQ4_`NVdUISKSKe[PRfGUI_`PRg8ZhgIi&_`NVkj lK_`l?UIeCP[Zming8X`j%UI_`PRg8Z oqpsrut jvVY_`X`PRfhP[Zb_cg1inVGUI_`lKXcVhQ`lKwQcVY_xQcVYeRVEfY_`PRg8ZyP[ZzfYe{U8QcQ`P}|?fGUI_`PRg8ZySKXcg8wKeRVYjvQG~ psr N?U8QkQcg8jvVvQ`P[aLZKP}|?fGUIZ]_(U8m\LUIZb_€UIa^VEQkQclfcN1U8Q/P[Z]\8UIX`P{UIZfEV%lKZKVYX UIZbhP[Zm‚ ƒ VEfY_`P[\^VT_`X€UIZQuiqg8X`j%UI_`PRg8Z„gIi:\8UIX`P{UIwKeRVEQ!~…†VY\^VYX€UIe p„r w?U8Q€VE%inVGUI_`lKXcV4QcVYeRVEfY_`PRg8Z Qcf€NVYjvVEQN?UG\^VvwWVEVYZ‡Q`lKaLa^VEQ`_cVE-P[Z-_`NVve[P[_cVYX€UI_`lKXcVLˆ‰wKlK_/_`NVY-Q`lmŠ‹VYXVYP[_`NVYX iqXcg8jŒUIZ1P[Z?UIwKP[e[P[_‡_cgsU8fEfEg8lKZb_ iqg8X(fEg8jSKeRVŽ-iqVGUI_`lKXcVP[Zb_cVYX€U8fY_`PRg8Z‘g8Xki’Xcg8j fEg8jSKlK_€UI_`PRg8Z?UIeCP[Zb_`X€U8fY_€UIwKP[e[P[_“”P[Z”Qcg8jvVfEg8ZmP[_`PRg8ZQG~ • ZVY–—jvVY_`Ng]˜PRQKV|ZVE‘–NVYXcVYw]˜fEg8jSKlK_€UI_`PRg8Z™gIi psr \LUIe[lVEQ(iqg8X aLXcg8lKSQšgIi/inVGUI_`lKXcVEQfGUIZ™wV Q`P[aLZKP}|?fGUIZb_`e[™Q`SVEVEKVE›lKSœw]‘j%UIZKP[SKlKe{UI_`P[ZKa fEg8jSWgLQ`P[_cVinVGUI_`lKXcV \LUIe[lVEQGˆbj%UI]P[ZKa P[_SWgLQcQ`P[wKeRV_cgfEg8jSKlK_cV.aLXcg8lKSb–PRQ€V psr \LUIe[lVEQdgIižQ`P[ŸGVEQ(_`N?UI_ – VYXcVP[Z]_`X€U8fY_€UIwKeRVš–P[_`N˜VGUIX`e[PRVYXdjvVY_`Ng†KQG~ MONVZVY– jvVY_`Ng† PRQ.fEg8jS?UIXcVEs_cgvSKXcVY\†PRg8lQ`e[xSKlKwKe[PRQ`NVE„jvg]KVYeRQ4UIZ„UISKSKe[PRVE„_cgU Z]lKjkwWVYX.gIi+fYe{U8QcQ`P}|?fGUI_`PRg8Z SKXcg8wKeRVYjvQž_cgdP[e[e[lQ`_`X€UI_cV/P[_cQžSVYXuing8X`j%UIZfEVL~ ¡x¢b£‹¤ ¥ F ¦CBL§©¨ VGUI_`lKXcV QcVYeRVEfY_`PRg8Z6ˆminVGUI_`lKXcV/X€UIZK]P[ZKaˆjklK_`l?UIe:P[Zming8X`j%UI_`PRg8Z6~ ª

« ¬:­”®+¯&° ±‡²³.®µ´!°­ &¶ ·¸†¹º¼»Y·½L·!¾’·E¿!¹“ÀÂÁbÜĒÅƆÇÄÉȄÊ:ËmÌÎÍYƆÇKÍ(͏ÏIÐEÑWÇWÄÉÒÓWÏxĒǑȄƆÇKÔ*Ð ÕqƆÅÎŏÄÉÖWÐ8Æ]͏ĒËmǘÍYƆŏ×ÅGØdÄÉÇ Ê‹Æ]͏͏ÏIÌÎÇÙÌÎÏIÐIˆÚmÇWÄÉ͏ĒËmÇCۛÜ`Ç*?AI!J K

(ô ՒÕȚÓÍYÓWƆՋĒÇÝÞˆ̏ȄÆ]ÍYÄÉËmÇá‹Æ]ÅÏ ä ÝÞÏ8Æ]͏ÓW̏ÏÅÎÏIÕ’Ï Ð ÍÄ’ËmǔŏÐEÑÏIÈ ÏIÅTÐIƆՒռÝÞËmÌOÍYÑWÏÐ ËmȄÊÓÍEÆbò ÍYĒˆÇ-ˆÝȚÓÍYÓWƆÕCÄÉÇÝÞËm̏ȄÆ]͏ĒËmǔâ]ƆՒÓWÏ Å4ÓWÅÎĒÇWÚxÊWÌÎËmá‹Æ†áĒՒÄ{Í`Ô ä ÏIÇWÅÎÄÉÍ`ԄÝÞÓWÇWР͏ĒËmÇÅIßãÑWÄÉÐEÑ Ä’Ç›ÌÏ8Æ]Õ{òuã.ËmÌÎÕ ä ÊWÌÎËmáWÕ’Ï È„ÅvыÆ8â†Ï ͏Ë1á:Ï ä ÏIÌÎÄÉâmÏ ä ÝÞÌÎËmÈ ä Æ]ÍYÆÛsè¼Ä’ÇWÐ ÏsÆ]ÊWÊW̏Ë8é¼Ä’È„Æ]͏ĒËmÇ Ë†Ý ÍÑWÏIÅÎÏ ä ÏIÇWÅÎÄÉÍ`ÔyÝÞÓWÇÐ ÍYÄÉËmÇWŔÓWÅÎĒÇWÚÐIˆÇ͏ĒÇÓWËmÓŔÝÞÓÇWР͏ĒËmÇWÅxÄÉÅxÇWËmÇKÍYÌÎÄÉâ?ÄqÆ]Õ ÄÉÇKâmËmÕ{âò ĒÇWڔƆÇ-ÄÉÇKÍYÏIچÌYƆÕ:ãÑWĒÐEÑÐ8Æ]ÇWÇWˆÍá:ÏŏËmÕ{âmÏ ä ÏGé¼ÊWՒÄÉÐIÄÉ͏ÕÉÔ !ßÍYÑÏÐEÑWËmÅÏ Ç˜Æ]ÊWÊW̏ËmƆÐEÑ-ĒÅ4ÍË T

ª

CU

2

Ö‹ÌÅ€Í ä ĒÅÎÐIÌÎÏ ÍYÄÉŏÏkÍYÑWÏ(ÝÞÏIÆ]ÍYÓÌÏ â]ƆÕÉÓWÏIÅTÓWŏÄÉÇWښƆÇKԄˆÝƐâ]ƆÌÎÄ’Ï Í`Ô Ë†ÝµÏGé¼Ä’ÅÎ͏ĒÇWÚ ä ĒÅÎÐIÌÎÏ ÍYÄÉÅYÆbò ÍYĒˆÇsÍYÏ ÐEÑWÇWĒÒÓWÏ Å/ÝÞËmÌTÐ ËmÇKâmÏIÌÎŏÄÉËmLjÝÐIËmÇKÍYÄÉÇÓWËmÓWÅTÝÞÏIÆ]ÍYÓ̏Ï(â]ƆÕÉÓWÏIÅ ÛÜ`Ç͏ÑWÏ(ÝÞËmÕÉՒËLãĒÇÚWß ä ĒÅÎÐIÓWÅÎŏĒˆÇ1ÄÉÅTÍYÑWÏ ÌÏ!ÝÞËm̏ÏvÐIˆÇ֋ÇWÏ ä ÄÉÇ͏ÑWÏ ä ËmȄƆÄÉÇhË†Ý ä ĒÅÎÐȈÏ!ÍYÏvÌYÆ]Ç ä ËmÈ#â†Æ]̏ÄqÆ]áWÕ’Ï ÅIÛ å/ÑWÏxÈÓÍYӋÆ]ÕOĒÇÝÞˆ̏ȄÆ]ÍYÄÉËmÇ I(U, V ) á:Ï Í`ã.ÏIÏ ÇœÍ`ã.Ë ä ĒŏР̏Ï!ÍYϔÌYÆ†Ç ä ËmÈ â]ƆÌÎÄqƆáՒÏIÅ Æ†Ç ä V ĒŠä Ï!֋ÇWÏ ä Æ†Å U 3Á

I(U, V ) =

XX u

p(u, v) log

v

p(u, v) , p(u)p(v)

ãÑWÏIÌÎÏ p(u), p(v) Æ†ÌÏ ä ÏIÇWÅÎÄÉÍ`ÔhÝÞÓÇWР͏ĒËmÇWÅ4Æ†Ç ä p(u, v) ĒÅ/Æ ÎˆĒÇKÍ ä Ï ÇWŏÄ{Í`ÔhÝÞÓWÇWÐ!ÍYÄÉËmÇàÛ ô Å„È ÏIÇKÍYÄÉËmÇWÏ ä Æ†á:ËLâmφßƛ׆Ï!Ô Æ ä â†Æ]ÇÍYƆÚmχˆÝȚӼÍYӋƆÕÄÉÇÝÞËm̏ȄÆ]͏ĒËmÇzÄÉńÄ{ÍYŔĒǼò â†Æ]̏ÄqÆ]ÇWÐIϑÓWÇ ä ÏIÌȄƆÇKÔÍY̏ƆÇWŀÝÞËm̏ȄÆ]ÍYÄÉËmÇWÅ Û å/ÑWϑÝÞËmÕÉՒËLãĒÇWڜÍYÑWÏ ËmÌÏ È ÅÎÍYÆ]ÍYÏ Å‡ÍÑWĒŠÊW̏ËmÊ:ÏÌÍ`ÔĒÇ-È Ëm̏ÏÏGéƆРÍkÍYÏ ÌÈ ÅIæ ô(ÅÎŏÓWÈ Ï/ÍYыÆ]Í g ÄÉÅÆ†Ç”Ä’Ç ÎÏIР͏ÄÉâ†Ï OÝÞÓWÇWÐ!ÍYÄÉËmÇàÛ å/ÑÏIǔÝÞËmÌƆÇKÔ ÌYÆ†Ç ä ËmȌâ]ƆÌÎÄqƆáWÕÉÏIÅ U Æ†Ç ä V ߋÄÉÍÑËmÕ ä Å4͏ыÆ]Í I(g(U ), V ) = I(U, V ) Û å˜ˆÓÍYÕÉĒÇWχÆÊW̏Ë?ˆÝcߞÇWˆÍYÏÍYÑWÆ]̈́͏ÑWχȚÓÍYÓWƆÕ/ĒÇÝÞˆ̏ȄÆ]ÍYÄÉËmǙâ]ƆՒÓχÄÉÅ Æ CÏ Ð ÍÏ ä ËmÇWÕÉԔáKÔ ÍYÑWÏÝÞÌÏ ÒÓWÏIÇWÐ!ÔhˆÝà˼РÐIÓWÌÎ̏ÏIÇÐIÏdˆÝàÏ8ƆÐEÑsâ]ƆÕÉÓWÏkÄ’Ç U Æ]Ç ä ÇWˆÍ.áKԔÍYÑWÏkâ†Æ]ՒÓWÏ Å ÍYÑWÏ È„ÅÎÏIÕÉâ†ÏIŠ۞è¼Ä’ÇWÐ Ï g ÄÉÅƆÇxÄÉÇ ÎÏIÐ!ÍYĒˆÇàßmÄÉÍ Ð8ƆÇÇWË†Í ÐEыƆÇÚmÏ4ÍYÑWÏ ÅÏË?ÐIÐ ÓW̏ÌÎÏIÇWÐ ÏkÐ ËmÓWÇKÍYÅ Û ]ÿ üÿ bÿ Rÿ “ÿ †ÿ&ÿ Lÿ 4ÿ bÿ “ÿ Þ4ÿ ÿ ÿ mÿ Âÿ ^ÿ ^ÿ üÿ !ÿ bÿ ÿ ^bÿ ÿ Þmÿ ÿ Âÿ ^ÿ bÿ ¢Â



ÃÅÄÆ_ÇÉÈÆÊÌË

ÎÍ

|4



Ï!ÐÑ'  Ò '$+/ !¢ ¯,& *--  +©Ó҄ ÔVÕÖ  +„ f !%‹×/' X'" ©&  $+ + ‹Ò  !  (0 Ø!ÙÚfÛ Õ " _%/ ¯š !  +o ÔÜÔV ¢| %­Ò-|+$3Ý ! !ΩÕf  „' +ÒÅZ+$Ý ! !Î Õf  .Å' ¢ %Ü0 0e' ©   –fÛ  š/+Å (Z g 0

∀a, b : (g(a) = g(b) ⇒ a = b)

ª 8

ž:ž

Suggest Documents