Four Measures of Data Complexity for Bootstrapping, Splitting and ...

  
   
  
 

 

   

       ! "## $ %&$'( )! *&+, "+--( -. ).-'+/ 0,/1 -. 2(/3 45- 45-3 671$$3 %%89 :0;3 0$ *5$-' ?@5-!+!&= ABCDEFGD HIJKLMJN OJPQQLRNS QTQKNUQ POVLNWN XNQK SNQIJKQ YVNZ KVN LZ[LWL[IPJ OJPQQLRNS \IKMIKQ PSN [LWNSQN PZ[ \] QLULJPS POOISPOT^ _] KVNSN LQ P QIXQKPZKLPJ [L`NSa NZON LZ KVN LZ[LWL[IPJ POOISPOLNQb KVNZ KVN NZQNUXJN ULcVK XN Y\SQN KVPZ KVN QLZcJN XNQK \S NWNZ Y\SQN KVPZ PZ PWNSPcN UNUXNS \] KVN KNPU^ dPSL\IQ UNKVa \[Q VPWN XNNZ MS\M\QN[ ]\S OSNPKLZc QIXQNKQ \] KVN cLWNZ JPXNJN[ [PKP QNKb QIXQNeINZKJT IQN[ K\ KSPLZ KVN LZ[LWL[IPJ OJPQQLRNSQ^ fVSNN QIOV UNKV\[Q PSNg KPhLZc X\\KQKSPM QPUMJNQb QMJLKKLZc KVN [PKP QNK LZK\ [LQi\LZK QIXQNKQb PZ[ SPZ[\U ]NPKISN QPUMJLZc^ j\Ja J\YLZc P SNONZK MIXJLOPKL\Z XT k\ lmnb VNSN YN \`NS P QUPJJ NoMNSLUNZKPJ QKI[T \Z O\UMJNoLKT \] KVN QIXa QNKQ \XKPLZN[ XT KVN KVSNN UNKV\[Q^ j\IS UNPQISNQ \] O\UMJNoLKT VPWN XNNZ IQN[ p KVN KVSNN ]S\U lmng KVN ULZLUPJ QMPZZLZc KSNN qHrfsb KVN P[VNSNZON QIXQNKQ UNPQISN qtuksb KVN UPoLUPJ ]NPKISN Nva OLNZOT qHjwsx PZ[ P OJIQKNS JPXNJ O\ZQLQKNZOT UNPa QISN qyzys MS\M\QN[ VNSN^ {N IQN[ KVN |y_ }YLZN~ [PKPQNK q OJPQQNQb € ]NPKISNQs YLKV  OPQNQ \] OJPQQ QMJLKg €W‚Wb €Wq‚ƒsb ‚Wq€ƒsb PZ[ Wq€ƒ‚sb PZ[ KVN  MPLSYLQN QMJLKQ^ rISMSLQLZcJTb KVN WPJINQ \] KVN ]\IS UNPQISNQ YNSN QIXQKPZKLPJJT [L`NSNZK ]S\U NPOV \KVNSb PJKV\IcV KVN }YLZN~ [PKP QNK LQ MNSONLWN[ PQ KVN KVSNN UNKV\[Q ]\S QNJNOKLZc [PKP QIXa „…†‡ ^KVNˆ] ]NPKISN QNKQb QPUMJLZc VP[ KVN VLcVNQK WPSLPXLJLKT \] KVN [PKP O\UMJNoLKT^ ‰ Š‹ DEŒŽGDŒ ‹ HIJKLMJN OJPQQLRNS QTQKNUQ PSN [NNUN[ K\ XN U\SN POOISPKN KVPZ KVN XNQK LZ[LWL[IPJ OJPQQLRNS LZ KVN NZQNUXJN^ k\YNWNSb KVLQ LQ Z\K ZNONQQPSLJT KSIN ]\S PZT OJPQQLRNS NZQNUXJN^ j\S NoPUMJNb KVN UPi\SLKT W\KN POS\QQ [NMNZ[NZK OJPQQLRNSQ \] KVN QPUN POOIa SPOT UPT XN UIOV Y\SQN \S UIOV XNKKNS KVPZ KVN LZ[LWL[IPJ POOISPOT ln^ uLWNSQLKT LZ KVN KNPU LQ Z\K

KVN \ZJT ]POK\S KVPK P`NOKQ KVN QTQKNU MNS]\SUPZON^ _] KVN LZ[LWL[IPJ OJPQQLRNSQ VPWN QIXQKPZKLPJJT [L`NSa NZK POOISPOLNQb LK LQ IZJLhNJT KVPK KVN O\UXLZPKL\Z YLJJ LUMS\WN \Z KVN XNQK LZ[LWL[IPJ^ _K QNNUQ KVPK LK LQ LUM\SKPZK K\ VPWN QLULJPSJT POOISPKN OJPQQLRNSQ qNWNZ YNPh \ZNQ l‘ns XIK YLKV VLcV [LWNSQLKT^ |Z]\SKIZPKNJTb LZ[NMNZ[NZK KSPLZLZc \] OJPQQLRNSQ [\ Z\K cIPSPZKNN LZ[NMNZ[NZON qLZ QKPKLQKLOPJ QNZQNs \] KVNLS \IKMIKQ ln^ ˆZN PMMS\POV K\ NZVPZOLZc [LWNSQLKT \] KVN LZ[LWL[IPJ OJPQQLRNSQ LQ K\ KSPLZ KVNU \Z [L`NSNZK QIXQNKQ \] KVN PWPLJPXJN JPXNJN[ [PKP QNK^ zNK ’ “ ”•–— ˜ ˜ ˜ —•™ š XN P JPXNJN[ [PKP QNKb •› œ žb Ÿ “ €— ˜ ˜ ˜ —  YLKV   NJNUNZKQ^ fVSNN UNKV\[Q ]\S \XKPLZLZc [L`NSNZK [PKP QIXQNKQ VPWN XNNZ IQN[ LZ SNONZK QKI[LNQ €^ ¡¢¢£¤£¥¦§ ¤¦¨§©ª«¬^ {N QPUMJN ]S\U ’ YLKV SNMNKLKL\Z PQQIULZc IZL]\SU MS\XPXLJLKT POS\QQ KVN NJNUNZKQ \] ’ ^ fTMLOPJJT KVN OPS[La ZPJLKT \] KVN X\\KQKSPM QPUMJN LQ OV\QNZ K\ XN   ^ ­\\KQKSPM QPUMJLZc IZ[NSJLNQ KVN X\\QKLZc UNKV\[ ]\S [NQLcZLZc OJPQQLRNS NZQNUXJNQ l€n^ ‚^ ®¦£¦ ¤§©ª££ª«¬^ fVN LZ[LWL[IPJ OJPQQLRNSQ OPZ XN XILJK \Z [LQi\LZK QIXQNKQ \] ’ b PQ LZ OS\QQWPJa L[PKL\Z NQKLUPKL\Z \] OJPQQLROPKL\Z POOISPOT^ ^ ¯¦«°¢¨ ±²¦£³¥² ¤¦¨§©ª«¬^ {N OPZ XPQN KVN LZ[LWL[IPJ OJPQQLRNSQ \Z [L`NSNZK QIXQNKQ \] ]NPa KISNQb L^N^b \Z [L`NSNZK QIXQMPONQ \] KVN ]NPKISN QMPON ž^ k\ ln QV\YQ KVPK SPZ[\U QPUMJLZc qYLKV\IK SNMNKLKL\Zs K\ cNK P QNK \] ´ µ ¶ ]NPa KISNQ ]S\U KVN LZKNcNSQ ]S\U € K\ ¶b LQ P WLPXJN JLZN ]\S XILJ[LZc UIJKLMJN OJPQQLRNS QTQKNUQ^ _Z KVLQ QKI[T YN PSN LZKNSNQKN[ LZ UNPQISLZc KVN \] KVN [PKP QIXQNKQ \XKPLZN[ KVS\IcV ·¸¹º»„¼½¾‡ KVN KVSNN UNKV\[Q^ {N PMMJT ]\IS UNPQISNQ \] O\UMJNoLKTb KVN ULZLUPJ QMPZZLZc KSNN qHrfsb

KVN P[VNSNZON QIXQNKV\\[ qtuks XPQN[ \Z KVN a ZNLcVX\SV\\[ UNPQISN PQ MS\M\QN[ LZ lmnb KVN UPoa LUIU ]NPKISN NvOLNZOT qHjwsb PJJ KVSNN ]S\U lmnb PZ[ P UNPQISN YVLOV YN OPJJ KVN » †¾„ … „» ¸ UNPQISNQ PSN Noa †½†¾„ ·‡ LZ rNOKL\Z^ fVN‚^ _ZO\UMJNoLKT MJPLZN[ rNOKL\Z  YN [NKPLJ \IS Noa MNSLUNZKPJ SNQIJKQ YLKV KVN }YLZN~ [PKP QNK ]S\U KVN |y_ HPOVLZN zNPSZLZc NM\QLK\ST [PKPXPQN^ rNOa KL\Z m O\ZKPLZQ \IS PZPJTQLQ PZ[ O\ZOJIQL\ZQ^


  
















FCŽE C Œ GŒ 

   

D



QKI[T YN VPWN PcPLZ KPhNZ KVN wIOJL[NPZ [LQKPZON PQ \IS UNKSLO^ tZ P[VNSNZON UPMMLZcb ´ ]S\U KVN M\YNS QNK q s K\ q s LQ QIOV KVPKg ´q s “ ´q s “ qs ´q s “ ´q s ˜ fVN JPScNQK M\QQLXJN P[VNSNZON QIXQNKQ OPZ XN cS\YZ ]\S NPOV M\LZK XT QIOONQQLWNJT NoMPZ[LZc KVN P[a VNSNZON QIXQNK PK NPOV QKPcN YVLJQK NZQISLZc KVPK PJJ ZNYJT LZOJI[N[ M\LZKQ O\UN ]S\U KVN QPUN OJPQQ^ j\S NoPUMJNb ´ q” šs “ ” š— ´ – q” šs “ ´q” šs— ´ q” šs “ ´q ´q” šss ˜ ˜ ˜ b cLWNQ IQ MS\a cSNQQLWNJT VLcVNS \S[NS P[VNSNZON QIXQNKQ^ j\S NPOV M\LZKb \ZJT KVN VLcVNQK \S[NS QIXQNK LQ SNKPLZN[ QIOV KVPK PJJ NJNUNZKQ PSN ]S\U KVN QPUN OJPQQ^ fVLQ MS\a ON[ISN [NRZNQ P MPSKLKL\Z \] KVN [PKP QNK YVNSN NPOV OJIQKNS O\ZKPLZQ [PKP M\LZKQ YLKV KVN QPUN OJPQQ JPa XNJ^ fVN ZIUXNS \] QIOV OJIQKNSQ LQ PZ LZ[LOPKL\Z \] KVN O\UMJNoLKT \] KVN MS\XJNU^ _] KVN OJPQQNQ PSN O\UMPOK PZ[ ]PS ]S\U NPOV \KVNSb KVNZ NPOV OJPQQ YLJJ L[NPJJT ]\SU P QLZcJN QNMPSPKN P[VNSNZON QIXQNK^ {VNZ KVN OJPQQNQ PSN \WNSJPMMLZcb UIJKLMJN OJIQKNSQ PSN JLhNJT K\ PMMNPS^ tcPLZb KVLQ UNKV\[ OPZ XN IQN[ Z\K \ZJT \Z KY\ OJPQQ OPQNQ XIK PJQ\ \Z UIJKLMJN OJPQQ OPQNQ qQNN jLcISN € PsbOs PZ[ jLcISN ‚ PsbOss^ F

G B

           

!""

fVLQ UNKV\[ ]\S UNPQISLZc KVN OJPQQ X\IZ[PST JNZcKVb LQ IQN[ XT k\ lmnb VPWLZc \SLcLZPJJT XNNZ MS\M\QN[ LZ P MPMNS XT jSLN[UPZ PZ[ P]QhT l‚n^ LWNZ P UNKSLOb P ULZLUPJ QMPZZLZc KSNN OPZ XN O\ZQKSIOKN[ YVLOV O\ZZNOKQ PJJ KVN QPUMJN M\LZKQ SNcPS[JNQQ \] KVNLS OJPQQ JPXNJQ^ {N IQN KVN wIa OJL[NPZ [LQKPZON PQ KVN UNKSLO^ yJNPSJT Q\UN N[cNQ \] KVN Hrf YLJJ O\ZZNOK M\LZKQ ]S\U [L`NSNZK OJPQQNQ PZ[ KVN O\IZK \] QIOV N[cNQ cLWNQ IQ P UNPQISN \] KVN JNZcKV \] KVN X\IZ[PST XNKYNNZ KVN OJPQQNQ^ rLZON KVNSN PSN   € N[cNQ ]\S   QPUMJN M\LZKQb KVN O\IZK OPZ XN Z\SUPJLQN[ PQ P MNSONZKPcN \]   lm n^ fVN N[cNQ O\ZZNOKLZc KY\ [L`NSNZK OJPQQNQ PSN QV\YZ YLKV X\J[ JLZNQ LZ KVN NoPUMJNQ LZ jLcISNQ € PZ[ ‚^ k\ lm n \ZJT O\ZQL[NSQ KVN KY\ OJPQQ OPQN XIK KVN UNKV\[ OPZ XN NoKNZ[N[ K\ UIJKLMJN OJPQQNQ qQNN jLcISN € PsbXs PZ[ jLcISN ‚ PsbXss^ ² ¨²£ ¢° €^ fPhN P JPXNJJN[ [PKP QNK \] QL N   PZ[ O\Za QKSIOK KVN ULZLUPJ QMPZZLZc KSNN [LQSNcPS[LZc KVN OJPQQ JPXNJQ^ ‚^ y\IZK KVN ZIUXNS \] N[cNQ LZ KVN ULZLUPJ QMPZZLZc KSNN O\ZZNOKLZc [L`NSNZK OJPQQNQb   ^ ^ fVN O\UMJNoLKT UNPQISN LQ cLWNZ XT q€s “   

#

$

%&

' (%

*

*

+ , - . / 0 1 2 3 4 567

8 9 : " !"  ;"  < = >" ? >

fVLQ UNKV\[ MS\M\QN[ XT k\ lm n O\ZQL[NSQ KVN OJIQa KNSLZc MS\MNSKLNQ \] KVN [PKP^ _K LQ XPQN[ \Z P SNa NoLWNb QTUUNKSLO XLZPST SNJPKL\Z XNKYNNZ KY\ M\LZKQ — LZ P QNK ^ LQ [NRZN[ XT ´q — s µ b YVNSN ´q — s LQ P cLWNZ UNKSLO PZ[ LQ P cLWNZ Z\Za NS\ O\ZQKPZK^ {N [NRZN q s “ ” œ š K\ XN KVN aZNLcVX\ISV\\[ \] ^ j\S \IS @

A

4

7

B

4 7

A

4A 7

C

4 7

)

B E7 A 4

K

K

4

D 4

J F

L

M N OP F

4

F T

D 4 4

7

4

F

4

QL R B

4

F

4

F

4

4

a)

VUVUU VUUVU VVU VUV

^]^]] ^]]^] ^^] ^]^

vuvuu vuuvu vvuuu vu vu uv

hghgg hhggg hg hg

wxw wxw xwxw xwwx x x

ff ff f f

b)

dcdcc cddcc ddc dcd

`_`__ _``__ ``_ `_`

tstss sttss tssst tts tst tst

jijii jiiij jji jij

ZZ ZZ ZZ ZZ

ee ee oo ppoo ee poop ee oo opop opop

XWXWW WXXWW XXW XWX

babaa abbaa bba bab

rqrqq rqqqr rrqqq rrq rrq rqr

lklkk llkkk llkk llkk

YY \[YY\[[ \\[

mm mm mm mm mm mm m nmn nmn nn nn

c)

&

)

 

H F I F

F S

F

 

G B

YY \[YY\[[ \\[

jLcISN €g t ‚ OJPQQ NoPUMJN^ Ps [LQKSLXIa KL\Z \] KVN OJPQQNQb Xs P ULZLUPJ QMPZZLZc KSNN “ b Os KVN P[VNSNZON QIXQNKQ “ ^ + , - . / 0 1 2 3 4 567

L { | . / 0 1 2 3 4 567

} z

y z

² ® ¨²£ ¢° €^ fPhN P JPXNJJN[ [PKP QNK \] QL N   PZ[ ]\S NPOV M\LZK cS\Y KVN JPScNQK M\QQLXJN P[VNSNZON QIXa QNK QIOV KVPK PJJ NJNUNZKQ \] KVN QIXQNK PSN ]S\U KVN QPUN OJPQQ^ ‚^ y\IZK KVN ZIUXNS \] ½ „ „ ¾ P[VNSNZON QIXa QNKQ   ^ ^ fVN O\UMJNoLKT LQ KVNZ cLWNZ XTg “    q‚s

%&

~



&

)

€  

‚

L { | . / 0 1 2 34 567

‚

a) b)









































 

 

(

( )

( )

 

 

(

( )

( )

 

 

(

( )

( )

 

 

(

( )

( )

&

 

 







 

 

 



 



 

 





 

 





& '

&  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

*

*

* +

* +

* +

* +

* +

* +

+

+



 

 



 

 



 

 



 

 

& '

& '

& '

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

&

 

 

& '

! & '

 

!

 

!

 

!

 

 

!

!

 

!

 

!

 

!

 

!

 

 

!

 

 

&

& '

& '

 

 

&

& '

& '

 

 

!

 

!

 

!

 

!

 

!

 

!

 

!

 

#"

+B

 

 

#"

 

 

 

 

 

 

 

#"

 

#"

 

#"

 

#"

 

 

M\LZKQ YLKVLZ KVLQ LZKNSWPJ [NRZNQ KVN O\UMJNoLKT \Z KVPK ]NPKISN PoLQ^ j\S KVN NoPUMJN LZ jLcISN € O^sb KVN \WNSJPM LZKNSWPJ O\ZKPLZQ  \] KVN  M\LZKQb Q\ KVN O\UMJNoLKT \Z KVLQ ]NPKISN LQ “ qKVN NvOLNZOT LQ € s^ ² ¨²£ ¢° €^ fPhN P JPXNJJN[ [PKP QNK q‚ OJPQQNQs \] QL N   ^ ‚^ j\S NPOV ]NPKISN q s MS\iNOK KVN [PKP \Z LKQ PoLQ PZ[ L[NZKL]T KVN \WNSJPM SNcL\Z^ y\IZK KVN ZIUXNS \] M\LZKQ LZ KVPK SNcL\Zb   PZ[ OPJOIa JPKN KVN Hjw O\UMJNoLKT PQ “    ˜ qs ^ fPhN PQ KVN RZPJ O\UMJNoLKT WPJIN

 

$ +B

c)

 

 

 

 

 

 

 

 





 

 





 

 





 

 





 

 

$ %

$ %

 

 

$ %

$ %

 

 

$ %

$ %

 

 

$ %

$ %

 

 

 

 

 

 

 

 

 

 

%&

jLcISN ‚g t  OJPQQ NoPUMJN^ Ps [LQKSLXIa KL\Z \] KVN OJPQQNQb Xs t ULZLUPJ QMPZZLZc KSNN “ b Os KVN P[VNSNZON QIXQNKQ “ ^ + , - . / 0 1 2 3 4 567

,

z

L { | . / 0 1 2 3 4 567

-

-

_Z k\ Q MPMNS lm n KVN OV\LON \] YPQ “ ˜‘‘ YVNSN YPQ KVN ULZLUPJ [LQKPZON XNKYNNZ KY\ M\LZKQ \] [L`NSNZK OJPQQNQ^ _Z P MSNJLULZPST NoMNSa LUNZK YN QKI[LN[ KVN N`NOK \] \Z KVN O\UMJNoLKT WPJIN^ {N IQN[ “ ¶ q ¶s ]\S WPSa L\IQ WPJINQ \] b YVNSN KVN ¶ PZ[ YNSN KVN ULZLUIU PZ[ UPoLUIU [LQKPZONQ LZ KVN [PKP QNK SNcPS[JNQQ \] OJPQQ JPXNJQ^ fPXJN € QV\YQ KVN SNQIJK YLKV P KY\aOJPQQ QMJLK \] KVN [PKP QNK IQN[ LZ \IS ]ISKVNS NoMNSLUNZKQ^ .

/

0

0

05

1

2

3

/

/

/

/

€^ ^ ‘m ^‘‚ ^‘ ^‘‚ €^

L { | . / 0 1 2 3 4 567

^€ ^‚ ^ ^m ^‘

/

/

/

/

/

/

/

4

4

L { | . / 0 1 2 34 567

L { | . / 0 1 2 3 4 567

5

5

L { | . / 0 1 2 3 4 567

6



7

 < 

8

5

?

?

+B

+B



˜ qms

?

;

fVLQ UNPQISN NQKLUPKNQ V\Y YNJJ KVN OJPQQNQ UPKOV KVN M\QQLXJN OJIQKNSQ LZ [PKP^ jLSQK OJIQKNSQ PSN \Xa KPLZN[ \Z KVN YV\JN [PKP QNK SNcPS[JNQQ \] KVN OJPQQ JPXNJQ PZ[ KVNZ KVN JPXNJQ PSN IQN[ K\ O\IZK KVN ZIUXNS ]S\U NPOV OJPQQ YLKVLZ NPOV OJIQKNS^ } ISN~ OJIQKNSQ YLJJ cLWN J\Y O\UMJNoLKT WPJINQ YVNSNPQ }O\ZKPULZPKN[~ OJIQKNSQ YLJJ cLWN VLcV O\UMJNoLKT WPJINQ^ ² ¨²£ ¢° €^ fPhN P JPXNJJN[ [PKP QNK \] QL N   PZ[ OJIQKNS LK LZK\ OJIQKNSQ [LQSNcPS[LZc KVN OJPQQ JPXNJQ^ ‚^ j\S NPOV OJIQKNS q s OPJOIJPKN KVN ·» †¾„ »… „» b PQ KVN ]SPOKL\Z \] KVN UPoLUPJ ·¸ †½†¾„ \]·‡M\LZKQ ZIUXNS \] KVN QPUN OJPQQ JPXNJ Z KVN OJIQKNS ]S\U KVN K\KPJ ZIUXNS \] M\LZKQ LZ KVPK OJIQKNS^ ^ fPhN PQ KVN RZPJ O\UMJNoLKT WPJIN .

%&

{N OPZ QNN ]S\U KVN KPXJN KVPK KVN SNJPKL\ZQVLM XNKYNNZ PZ[ LQ Z\K U\Z\K\ZLO^ _Z[NN[b L] LQ K\\ QUPJJb KVNZ NPOV M\LZK YLJJ XN P OJIQKNS \Z LKQ \YZb PZ[ “ €^ ˆZ KVN \KVNS VPZ[b L] LQ K\\ JPScNb KVNZ KVN a ZNLcVX\ISV\\[ \] YLJJ O\ZKPLZ M\LZKqQs ]S\U P [L]a ]NSNZK OJPQQ^ tcPLZb YLJJ XN UPShN[ PQ P OJIQKNS \Z LKQ \YZb JNP[LZc K\ “ €^ rLZON KVNSN QNNUQ K\ XN Z\ OJNPS SNPQ\Z ]\S OV\\QLZc P MPSKLOIJPS b YN MLOhN[ “ ˜€^ 

)

D

4

/

2

&

>

0 F4

fPXJN €g y\UMJNoLKT WPJINQ ]\S WPSTLZc WPJINQ \] 2

=

y z

. / 0 1 2 3 4 567