California )nstitute of Tec0nology 3asadena5 CA 7889:
A new bottom>u? visual saliency model5 Ara?0>Based Cisual Saliency EABCSF5 is ?ro?osedG )t consists of two ste?sH Irst forming activation ma?s on certain feature c0annels5 and t0en normaliJing t0em in a way w0ic0 0ig0lig0ts cons?icuity and admits combination wit0 ot0er ma?sG T0e model is sim?le5 and biologically ?lau> sible insofar as it is naturally ?aralleliJedG T0is model ?owerfully ?redicts 0uman Ixations on LM7 variations of 8NO natural images5 ac0ieving 7OP of t0e RRC area of a 0uman>based control5 w0ereas t0e classical algorit0ms of )tti S Toc0 EU9V5 UWV5 UMVF ac0ieve only OMPG
Xost vertebrates5 including 0umans5 can move t0eir eyesG T0ey use t0is ability to sam?le in detail t0e most relevant features of a scene5 w0ile s?ending only limited ?rocessing resources elsew0ereG T0e ability to ?redict5 given an image Eor videoF5 w0ere a 0uman mig0t Ixate in a Ixed>time free> viewing scenario 0as long been of interest in t0e vision communityG Besides t0e ?urely scientiIc goal of understanding t0is remarYable be0avior of 0umans5 and animals in general5 to consistently Ixate on Zim?ortantZ information5 t0ere is tremendous engineering a??lication5 eGgG in com?res> sion and recognition U8WVG T0e standard a??roac0es EeGgG5 U9V5 U7VF are based on biologically mo> tivated feature selection5 followed by center>surround o?erations w0ic0 0ig0lig0t local gradients5 and Inally a combination ste? leading to a Zmaster ma?ZG Recently5 Bruce U:V and ot0ers UMV 0ave 0y?ot0esiJed t0at fundamental [uantities suc0 as Zself>informationZ and Zsur?riseZ are at t0e 0eart of saliency\attentionG ]owever5 ultimately5 Bruce com?utes a function w0ic0 is additive in feature ma?s5 wit0 t0e main contribution materialiJing as a met0od of o?erating on a feature ma? in suc0 a way to get an activation5 or saliency5 ma?G )tti and Baldi deIne Zsur?riseZ in general5 but ultimately com?ute a saliency ma? in t0e classical U9V sense for eac0 of a number of feature c0annels5 t0en o?erate on t0ese ma?s using anot0er function aimed at 0ig0lig0ting local variationG By organiJing t0e to?ology of t0ese varied a??roac0es5 we can com?are t0em more rigorouslyH iGeG5 not ^ust end> to>end5 but also ?iecewise5 removing some uncertainty about t0e origin of observed ?erformance differencesG T0us5 t0e leading models of visual saliency may be organiJed into t0e t0ese t0ree stagesH Es8F H extract feature vectors at locations over t0e image ?lane Es9F H form an Zactivation ma?Z Eor ma?sF using t0e feature vectors EsWF H normaliJe t0e activation ma? Eor ma?s5 followed by a combina> tion of t0e ma?s into a single ma?F )n t0is lig0t5 U:V is a contribution to ste? Es9F5 w0ereas UMV is a contribution to ste? EsWFG )n t0e classic algorit0ms5 ste? Es8F is done using biologically ins?ired Ilters5 ste? Es9F is accom?lis0ed by subtracting feature ma?s at different scales E0encefort05 Zc>sZ for ZcenterZ > ZsurroundZF5 and ste? EsWF is accom?lis0ed in one of t0ree waysH 8G a normaliJation sc0eme based on local maxima
U9V E Zmax>aveZF5 9G an iterative sc0eme based on convolution wit0 a difference>of>gaussians Ilter EZ_oAZF5 and WG a nonlinear interactions EZNaZF a??roac0 w0ic0 divides local feature values by weig0ted averages of surrounding values in a way t0at is modelled to It ?syc0o?0ysics data U88VG be taYe a different a??roac05 ex?loiting t0e com?utational ?ower5 to?ogra?0ical structure5 and ?ar> allel nature of gra?0 algorit0ms to ac0ieve natural and efIcient saliency com?utationsG be deIne XarYov c0ains over various image ma?s5 and treat t0e e[uilibrium distribution over ma? locations as activation and saliency valuesG T0is idea is not com?letely newH BrocYmann and Aeisel UOV suggest t0at scan?at0s mig0t be ?redicted by ?ro?erly deIned aevy cig0ts over saliency Ields5 and more recently Boccignone and derraro ULV do t0e sameG )m?ortantly5 t0ey assume t0at a saliency ma? is 5 and offer an alternative to t0e winner>taYes>all a??roac0 of ma??ing t0is ob^ect to a set of Ixation locationsG )n an un?ublis0ed ?re>?rint5 aGdG Costa UeV notes similar ideas5 0owever offers only sYetc0y details on 0ow to a??ly t0is to real images5 and in fact includes no ex?eriments involving IxationsG ]ere5 we taYe a uniIed a??roac0 to ste?s Es9F and EsWF of saliency com?uta> tion5 by using dissimilarity and saliency to deIne edge weig0ts on gra?0s w0ic0 are inter?reted as XarYov c0ainsG fnliYe ?revious aut0ors5 we do not attem?t to connect features only to t0ose w0ic0 are some0ow similarG be also directly com?are our met0od to ot0ers5 using ?ower to ?redict 0uman Ixations as a ?erformance metricG T0e contributions of t0is ?a?er are as followsH E8F A com?lete bottom>u? saliency model based on gra?0 com?utations5 ABCS5 including a frame> worY for ZactivationZ and ZnormaliJation\combinationZG E9F A com?arison of ABCS against existing benc0marYs on a data set of grayscale images of natural environments EviJG5 foliageF wit0 t0e eye>movement Ixation data of seven 0uman sub^ects5 from a recent study by gin0huser etG alG U8VG
Aiven an image !5 we wis0 to ultimately 0ig0lig0t a 0andful of isigniIcantj locations w0ere t0e image is iinformativej according to some criterion5 eGgG 0uman IxationG As ?reviously ex?lained5 t0is ?rocess is conditioned on Irst com?uting feature ma?s Es8F5 eGgG by linear Iltering followed by some elementary nonlinearity U8:VG ZActivationZ Es9F5 ZnormaliJation and combinationZ EsWF ste?s follow as described belowG Su??ose we are given a feature ma?8 " : "n# G Rur goal is to com?ute an activation ma? A : "n# 5 suc0 t0at5 intuitively5 locations $i& '% "n# w0ere !5 or as a ?roxy5 " $i& '%& is some0ow unusual in its neig0bor0ood will corres?ond to 0ig0 values of AG
Rf course ZunusualZ does not constrain us sufIciently5 and so one can c0oose several o?erat> ing deInitionsG Z)m?robableZ would lead one to t0e formulation of Bruce U:V5 w0ere a 0is> togram of " $i& '% values is com?uted in some region around $i& '%5 subse[uently normaliJed and treated as a ?robability distribution5 so t0at A$i& '% & log$($i& '%% is clearly deIned wit0 ($i& '% & Pr" $i& '%nei*hborhood0 Anot0er a??roac0 com?ares local ZcenterZ distributions to broader ZsurroundZ distributions and calls t0e TullbacY>aeibler tension between t0e two Zsur?riseZ UMVG
8 in t0e context of a mat0ematical formulation5 let , G Also5 t0e ma?s 5 and later 5 are ?resented as s[uare E F only for ex?ository sim?licityG Not0ing in t0is ?a?er will de?end critically on t0e s[uare assumtion5 and5 in ?ractice5 rectangular ma?s are used insteadG
be ?ro?ose a more organic Esee belowF a??roac0G aet us deIne t0e dissimilarity of " $i& '% and " $(& 1% as " $i& '% d$$i& '%$(& 1%% log 0 " $(& 1%
T0is is a natural deInition of dissimilarityH sim?ly t0e distance between one and t0e ratio of two [uantities5 measured on a logarit0mic scaleG dor some of our ex?eriments5 we use " $i& '% " $(& 1% instead5 and we 0ave found t0at bot0 worY wellG Consider now t0e fully>connected di> rected gra?0 2 5 obtained by connecting every node of t0e lattice " 5 labelled wit0 two indices $i& '% "n# 5 wit0 all ot0er n , nodesG T0e directed edge from node $i& '% to node $(& 1% will be assigned a weig0t 3 $$i& '%& $(& 1%% d$$i& '%$(& 1%% 4 $i (& ' 1%5 w0ere a 0 b 4 $a& b% exp 0 16 6 is a free ?arameter of our algorit0m9 G T0us5 t0e weig0t of t0e edge from node $i& '% to node $(& 1% is ?ro?ortional to t0eir dissimilarity and to t0eir closeness in t0e domain of " G Note t0at t0e edge in t0e o??osite direction 0as exactly t0e same weig0tG be may now deIne a XarYov c0ain on 2 by normaliJing t0e weig0ts of t0e outbound edges of eac0 node to ,5 and drawing an e[uivalence between nodes S states5 and edges weig0ts S transition ?robabilities G T0e e[uilibrium distribution of t0is c0ain5 rececting t0e fraction of time a random walYer would s?end at eac0 node\state if 0e were to walY forever5 would naturally accumulate mass at nodes t0at 0ave 0ig0 dissimilarity wit0 t0eir surrounding nodes5 since transitions into suc0 subgra?0s is liYely5 and unliYely if nodes 0ave similar " valuesG T0e result is an activation measure w0ic0 is derived from ?airwise contrastG be call t0is a??roac0 ZorganicZ because5 biologically5 individual knodesl EneuronsF exist in a con> nected5 retinoto?ically organiJed5 networY Et0e visual cortexF5 and communicate wit0 eac0 ot0er Esyna?tic IringF in a way w0ic0 gives rise to emergent be0avior5 including fast decisions about w0ic0 areas of a scene re[uire additional ?rocessingG Similarly5 our a??roac0 ex?oses connected Evia 4 F regions of dissimilarity Evia 3F5 in a way w0ic0 can in ?rinci?le be com?uted in a com?letely ?arallel fas0ionG Com?utations can be carried out inde?endently at eac0 nodeH in a sync0ronous environment5 at eac0 time ste?5 eac0 node sim?ly sums incoming mass5 t0en ?asses along mea> sured ?artitions of t0is mass to its neig0bors according to outbound edge weig0tsG T0e same sim?le ?rocess 0a??ening at all nodes simultaneously gives rise to an e[uilibrium distribution of massG T0e e[uilibrium distribution of t0is c0ain exists and is uni[ue because t0e c0ain is ergodic5 a ?ro?erty w0ic0 emerges from t0e fact t0at our underlying gra?0 2 is by construction strongly connectedG )n ?ractice5 t0e e[uilibrium distribution is com?uted using re?eated multi?lica> tion of t0e XarYov matrix wit0 an initially uniform vectorG T0e ?rocess yields t0e ?rinci?al eigen> vector of t0e matrixG T0e com?utational com?lexity is t0us 7$n 8% w0ere 8 n is some small number of iterations re[uired to meet e[uilibriumW G T0e aim of t0e ZnormaliJationZ ste? of t0e algorit0m is muc0 less clear t0an t0at of t0e activation ste?G )t is5 0owever5 critical and a ric0 area of studyG garlier5 t0ree se?arate a??roac0es were men> tioned as existing benc0marYs5 and also t0e recent worY of )tti on sur?rise UMV comes into t0e saliency com?utation at t0is stage of t0e ?rocess Ealt0oug0 it can also be a??lied to s9 as mentioned aboveFG be s0all state t0e goal of t0is ste? asH G )f mass is not con> centrated on individual activation ma?s ?rior to additive combination5 t0en t0e resulting master ma? may be too nearly uniform and 0ence uninformativeG Alt0oug0 t0is may seem trivial5 it is on some level t0e very soul of any saliency algorit0mH concentrating activation into a few Yey locationsG 9 )n our ex?eriments5 t0is ?arameter was set to a??roximately one tent0 to one Ift0 of t0e ma? widt0G Results were not very sensitive to ?erturbations around t0ese valuesG W Rur im?lementation5 not o?timiJed for s?eed5 converges on a single ma? of siJe in fractions of a second on a 9GM A]J 3entiumG
Armed wit0 t0e mass>concentration deInition5 we ?ro?ose anot0er XarYovian algorit0m as followsH T0is time5 we begin wit0 an activation ma?M A : "n# 5 w0ic0 we wis0 to ZnormaliJeZG be construct a gra?0 2 wit0 n nodes labelled wit0 indices from "n# G dor eac0 node $i& '% and every node $(& 1% Eincluding $i& '%F to w0ic0 it is connected5 we introduce an edge from $i& '% to $(& 1% wit0 weig0tH 3 $$i& '%& $(& 1%% A$(& 1% 4 $i (& ' 1%0 Again5 normaliJing t0e weig0ts of t0e outbound edges of eac0 node to unity and treating t0e resulting gra?0 as a XarYov c0ain gives us t0e o??ortunity to com?ute t0e e[uilibrium distribution over t0e nodes: G Xass will cow ?referentially to t0ose nodes wit0 0ig0 activationG )t is a mass concentration algorit0m by construction5 and also one w0ic0 is ?aralleliJable5 as before5 0aving t0e same natural advantagesG gx?erimentally5 it seems to be0ave very favorably com?ared to t0e standard a??roac0es suc0 as Z_oAZ and ZNaZG
be ?erform saliency com?utations on real images of t0e natural world5 and com?are t0e ?ower of t0e resulting ma?s to ?redict 0uman IxationsG T0e ex?erimental ?aradigm we ?ursue is t0e fol> lowingH for eac0 of a set of images5 we com?ute a set of feature ma?s using standard tec0ni[uesG T0en5 we ?roccess eac0 of t0ese feature ma?s using some activation algorit0m5 and t0en some nor> maliJation algorit0m5 and t0en sim?ly sum over t0e feature c0annelsG T0e resulting master saliency ma? is scored Eusing an RRC area metric described belowF relative to Ixation data collected for t0e corres?onding image5 and labelled according to t0e activation and normaliJation algorit0ms used to obtain itG be t0en ?ool over a cor?us of images5 and t0e resulting set of scored and labelled master saliency ma?s is analyJed in various ways ?resented belowG Some notes followH ]ereafter5 Zgra?0 EiFZ and Zgra?0 EiiFZ refer to t0e activation algorit0m described in section 9G8G9G T0e difference is t0at in gra?0 EiF5 t0e ?arameter 6 & 1025 w0ereas in gra?0 EiiF5 6 & 2G Zgra?0 EiiiFZ and Zgra?0 EivFZ refer to t0e an iterated re?itition of t0e normaliJation algorit0m described in section 9G9G T0e difference is t0e termination rule associated wit0 t0e iterative ?rocessH for gra?0 EiiiF5 a com?licated termination rule is used w0ic0 looYs for a local maximum in t0e number of matrix multi?lications re[uired to ac0ieve a stable e[uilibrium distributione 5 and for gra?0 EivF5 t0e termination rule is sim?ly Zsto? after M iterationsZG T0e normaliJation algorit0m referred to as Z)Z corres?onds to Z)dentityZ5 wit0 t0e most naive normaliJation ruleH it does not0ing5 leaving activations unc0anged ?rior to subse[uent combinationG T0e algorit0m Zmax>aveZ and Z_oAZ were run using t0e ?ublicly available Zsaliency toolboxZL G T0e ?arameters of t0is were c0ecYed against t0e literature U9V and UWV5 and were found to be almost identical5 wit0 a few slig0t alterations t0at actually im?roved ?erformance relative to t0e ?ublis0ed ?arametersG T0e ?arameters of ZNaZ were set according to t0e better of t0e two sets of ?arameters ?rovided in U88VG be wis0 to give a reward [uantity to a saliency ma?5 given some target lo> cations5 eGgG5 in t0e case of natural images5 a set of locations at w0ic0 0uman observers IxatedG dor any one t0res0old saliency value5 one can treat t0e saliency ma? as a classiIer5 wit0 all ?oints above t0res0old indicated as ZtargetZ and all ?oints below t0res0old as ZbacYgroundZG dor any ?articular value of t0e t0res0old5 t0ere is some fraction of t0e actual target ?oints w0ic0 are labelled as suc0 Etrue ?ositive rateF5 and some fraction of ?oints w0ic0 were not target but labelled as suc0 anyway Efalse ?ositive rateFG Carying over all suc0 t0res0olds yields an RRC curve U8MV and t0e area beneat0 it is generally regarded as an indication of t0e classifying ?ower of t0e detectorG T0is is t0e ?er> formance metric we use to measure 0ow well a saliency ma? ?redicts Ixation locations on a given imageG M To be clear5 if is t0e result of t0e eigenvector com?utation described in 9G85 iGeG5 if t0e gra?0>based activation ste? is concatenated wit0 t0e gra?0>based normaliJation ste?5 we will call t0e resulting algorit0m ABCSG ]owever5 may be com?uted using ot0er tec0ni[uesG : be note t0at t0is normaliJation ste? of ABS can be iterated times to im?rove ?erformanceG )n ?ractice5 we use G 3erformance does not vary signiIcantly in t0is regime wit0 res?ect to G e wit0 t0e intuition being t0at com?etition among com?eting saliency regions can settle5 at w0ic0 ?oint it is wise to terminate L 0tt?H\\wwwGsaliencytoolboxGnet
)n a study by gin0huser et alG U8V5 0uman and ?rimate Ixation data was collected on 8NO images5 eac0 modiIedO in nine waysG digure 9 s0ows an exam?le image from t0is collection5 toget0er wit0 ZxZs marYing t0e Ixation ?oints of t0ree 0uman sub^ects on t0is ?articular ?ictureG )n t0e ?resent study5 LM7 uni[ue modiIcations of t0e 8NO original images5 and 9M8M7 0uman Ixations from U8V were usedG Rnly ?ictures for w0ic0 Ixation data from t0ree 0uman sub^ects were available were usedG gac0 image was cro??ed to 344 544 ?ixels and was ?resented to sub^ects so t0at it tooY u? 63 22 of t0eir visual IeldG )n order to facilitate a fair com?arison of algorit0ms5 t0e Irst ste? of t0e saliency algorit0m5 feature extraction Es8F5 was t0e same for every ex?erimentG Two s?atial scales & were used5 and for eac0 of t0ese5 four orientation ma?s corres?onding to orientations 9 & 4 & 52 & 74 & ,82 were com?uted using Aabor Ilters5 one contrast ma? was com?uted using luminance variance in a local neig0bor0ood of siJe 94945 and t0e last ma? was sim?ly a luminance ma? Et0e grayscale valuesFG gac0 of t0ese 89 ma?s was Inally downsam?led to a 1286 raw feature ma?G Zc>sZ Ecenter>surroundF activation ma?s were com?uted by subtracting5 from eac0 raw feature ma?5 a feature ma? on t0e same c0annel originally com?uted at a scale M binary orders of magnitude smaller in overall resolution and t0en resiJed smoot0ly to siJe 12 86G )n U9V5 t0is overall sc0eme would be labelled c & 1& 85 for and 5 and ; & 55 corres?onding to a scale c0ange of 5 ordersG T0e ot0er activation ?rocedures are described in section 9G8G9 and 9G8G8G T0e normaliJation ?rocedures are all earlier described and namedG digure 9 s0ows an actual image wit0 t0e resulting saliency ma?s from two different Eactivation5 normaliJationF sc0emesG EaF Sam?le 3icture bit0 dixation
EbF Ara?0>Based Saliency Xa?
EcF Traditional Saliency Xa?
RRC area m NGLM
RRC area m NG:L
digure 9H EaF An image from t0e data>set wit0 Ixations indicated using xjsG EbF T0e saliency ma? formed w0en using Eactivation5normaliJationFm Egra?0 EiF5gra?0 EiiiFFG EcF Saliency ma? for Eactivation5normaliJationFmEc>s5_oAF dinally5 we s0ow t0e ?erformance of t0is algorit0m on t0e cor?us of imagesG dor eac0 image5 a mean inter>sub^ect RRC area was com?uted as followsH for eac0 of t0e t0ree sub^ects w0o viewed an image5 t0e Ixation ?oints of t0e remaining two sub^ects were convolved wit0 a circular5 decaying Yernel wit0 decay constant matc0ed to t0e decaying cone density in t0e retinaG T0is was treated as a saliency ma? derived directly from 0uman Ixations5 and wit0 t0e target ?oints being set to t0e O
XodiIcations were made to c0ange t0e luminance contrast eit0er u? or down in selected circular regionsG Bot0 modiIed and unmodiIed stimuli were used in t0ese ex?erimentsG 3lease refer to U8V5 U89VG
Ixations of t0e Irst sub^ect5 an RRC area was com?uted for a single sub^ectG T0e mean over t0e t0ree is termed Zinter>sub^ect RRC valueZ in t0e following IguresG dor eac0 range of t0is [uantity5 a mean ?erformance metric was com?uted for various activation and normaliJation sc0emesG dor any ?articular sc0eme5 an RRC area was com?uted using t0e resulting saliency ma? toget0er wit0 t0e Ixations from all W 0uman sub^ects as target ?oints to detectG T0e results are s0own belowG EaF Activation Com?arison
EbF NormaliJation Com?arison Comparison of Normalization Algorithms
Comparison of Activation Algorithms 0.7
0.7
0.65 mean ROC value for algorithm
mean ROC value for algorithm
0.65
c-s graph (i) graph (ii) self-info
0.6
0.55
0.5
0.45 0.55
0.6
0.65 0.7 inter-subject RO C value
0.75
0.8
graph (iii) graph (iv) ave-max NL DoG
0.6
0.55
0.5
0.45 0.55
0.6
0.65 0.7 inter-subject RO C value
0.75
0.8
digure WH EaF A mean RRC metric is com?uted for eac0 range of inter>sub^ect RRC valuesG gac0 curve re?resents a different activation sc0eme5 w0ile averaging over individual image numbers and normaliJation sc0emesG EbF A mean RRC metric is similarly com?uted5 instead 0olding t0e normaliJation constant w0ile varying t0e activation sc0emeG )n bot0 digures W and M5 T0e boundary lines above and below s0ow a roug0 u??er7 and strict lower bounds on ?erformance Ebased on a 0uman control and c0ance ?erformanceFG digure WEaF and dig> ure WEbF clearly demonstrate t0e tremendous ?redictive ?ower of t0e gra?0>based algorit0ms over standard a??roac0esG digure M demonstrates t0e es?ecially effective ?erformance of combining t0e best gra?0>based activation and normaliJation sc0emes5 contrasted against t0e standard )tti S Toc0 a??roac0es5 and also t0e Zself>informationZ a??roac0 w0ic0 includes no mention of a normaliJation ste? E0ence5 set 0ere to Z)ZFG Co m p a ri s on o f A l g ori thm s E n d -to-E n d
mean ROC value for algorithm
0 .7
0 .6 5
g ra p h s el f-in fo a ve -m a x NL Do G
0 .6
0 .5 5
0 .5
0 .4 5 0 .5 5
0 .6
0 .6 5 0 .7 i n ter-s u bj e c t R OC va l u e
0 .7 5
0 .8
digure MH be com?are t0e ?redictive ?ower of Ive saliency algorit0msG T0e best ?erformer is t0e met0od w0ic0 combines a gra?0 based activation algorit0m wit0 a gra?0 based normaliJation algorit0mG T0e combination of a few ?ossible ?airs of activation sc0emes toget0er wit0 normaliJation sc0emes is summariJed in Table 85 wit0 notes indicating w0ere certain combinations corres?ond to estab> lis0ed benc0marYsG 3erformance is s0own as a fraction of t0e inter>sub^ect RRC areaG Rverall5 we Ind an median RRC area of NG:: for t0e )tti S Toc0 saliency algorit0ms U9V on t0ese imagesG )n U8V 7
To form a true u??er bound5 one would need t0e Ixation data of many more t0an t0ree 0umans on eac0 imageG
t0e mean is re?orted as NG:L5 w0ic0 is remarYably close and ?lausible if you assume slig0tly more so?0isticated feature ma?s Efor instance5 at more scalesFG Table 8H 3erformance of end>to>end algorit0ms
nBruce S Tsotsos U:V n)tti S Toc0 UWV n)tti5 Toc05 S Niebur U9V naee5 )tti5 Toc05 S Braun U8NV
Alt0oug0 a novel5 sim?le a??roac0 to an old ?roblem is always welcome5 we must also seeY to answer t0e scientiIc [uestion of 0ow it is ?ossible t0at5 given access to t0e same feature information5 ABCS ?redicts 0uman Ixations more reliably t0an t0e standard algorit0msG be Ind ex?erimentally t0at t0ere are at least two reasons for t0is observed differenceG T0e Irst observation is t0at5 because nodes are on average closer to a few center nodes t0an to any ?articular ?oint along t0e image ?eri?0ery5 it is an emergent ?ro?erty t0at ABCS ?romotes 0ig0er saliency values in t0e center of t0e image ?laneG be 0y?ot0esiJe t0at t0is Zcenter biasZ is favorable wit0 res?ect to ?redicting Ixations due to 0uman ex?erience bot0 wit0 ?0otogra?0s5 w0ic0 are ty?ically taYen wit0 a central sub^ect5 and wit0 everyday life in w0ic0 0ead motion often results in gaJing straig0t a0eadG Notably5 t0e images of foliage used in t0e ?resent study G Rne can [uantify t0e ABCS>induced center bias by activating5 t0en normaliJing5 a uniform image using our algorit0msG ]owever5 if we introduce t0is center bias to t0e out?ut of t0e standard algorit0msj master ma?s Evia ?oint> wise multi?licationF5 we Ind t0at t0e standard algorit0ms ?redict Ixations better5 but still worse t0an ABCSG )n some cases EeGgG5 Z_oAZF5 introducing t0is center bias only ex?lains 9NP of t0e ?erformance ga? to ABCS o in t0e best case EviJG5 Zmax>aveZF5 it ex?lains 7NP of t0e differenceG be con^ecture t0at t0e ot0er reason for t0e ?erformance difference stems from t0e robustness of our algorit0m wit0 res?ect to differences in t0e siJes of salient regionsG gx?erimentally5 we Ind t0at t0e Zc>sZ algorit0m 0as trouble activating salient regions distant from ob^ect borders5 even if one varies over many c0oices of scale differences and combinations t0ereofG Since most of t0e standard algorit0ms 0ave Zc>sZ as a Irst ste?5 t0ey are weaYened ab initioG Similarly5 t0e Zself>infoZ algorit0m suffers t0e same weaYness5 even if one varies over t0e neig0bor0ood siJe ?arameterG Rn t0e ot0er 0and5 ABCS robustly 0ig0lig0ts salient regions5 even far away from ob^ect bordersG be note 0ere t0at w0at lacYs from ABCS described as above is any notion of a multiresolution re?resentation of ma? dataG T0erefore5 because multiresolution re?resentations are so basic5 one may extend bot0 t0e gra?0>based activation and normaliJation ste?s to a multiresolution version as followsH be begin wit05 instead of a single ma? A : "n# 5 a collection of ma?s A 5 wit0 eac0 A : "n # re?resenting t0e same underlying information but at different resolutionsG 3roceeding as we did before5 we instantiate a node for every ?oint 5 introducing edges again between every ?air of nodes5 wit0 weig0ts com?uted same as before wit0 one caveatH t0e 8N
?erformance 0ere is measured by t0e ratio of ERRC area using t0e given algorit0m for Ixation detectionF to ERRC area using a saliency ma? formed from t0e Ixations of sub^ects on a single ?ictureF
distance ?enalty function 4 $a& b% acce?ts two arguments eac0 of w0ic0 is a distance between two nodes along a ?articular dimensionG )n order to com?ute 4 in t0is case5 one must deIne a distance over ?oints taYen from different underlying domainsG T0e aut0ors suggest a deInition w0erebyH E8F eac0 ?oint in eac0 ma? is assigned a set of locations5 E9F t0is set corres?onds to t0e s?atial su??ort of t0is ?oint in t0e 0ig0est resolution ma?5 and EWF t0e distance between two sets of locations is given as t0e mean of t0e set of ?airwise distancesG T0e e[uilibrium distribution can be com?uted as beforeG be Ind t0at t0is extension Esay5 ABCS Xultiresolution5 or ABCSXF im?roves ?erformance wit0 little added com?utationG T0erefore5 we 0ave ?resented a met0od of com?uting bottom>u? saliency ma?s w0ic0 s0ows a re> marYable consistency wit0 t0e attentional de?loyment of 0uman sub^ectsG T0e met0od uses a novel a??lication of ideas from gra?0 t0eory to concentrate mass on activation ma?s5 and to form acti> vation ma?s from raw featuresG be com?ared our met0od wit0 establis0ed models and found t0at ours ?erformed favorably5 for bot0 of t0e Yey ste?s in our organiJation of saliency com?utationsG Rur model is extensible to multiresolutions for better ?erformance5 and it is biologically ?lausible to t0e extent t0at a ?arallel im?lementation of t0e ?ower>law algorit0m for XarYov c0ains is trivially accom?lis0ed in 0ardwareG T0e aut0ors ex?ress sincere gratitude to bolfgang gin0huser for 0is offering of natural images5 and t0e Ixation data associated wit0 t0em from a study wit0 seven 0uman sub^ectsG be also acYnowledge NSd5 N)]5 _AR3A5 and RNR for t0eir generous su??ort of our researc0G U8V bG gin0huser5 bG Truse5 TG3G ]offmann5 S 3G Tpnig Z_ifferences of XonYey and ]uman Rvert Attention under Natural ConditionsZ5 9NNeG U9V aG )tti5 CG Toc05 S gG Niebur ZA model of saliency based visual attention for ra?id scene analysisZ5 UWV aG )tti S CG Toc0 ZA saliency>based searc0 mec0anism for overt and covert s0ifts of visual attentionZ5 5 9NNN UMV aG )tti5 S 3G Baldi ZBayesian Sur?rise Attracts ]uman AttentionZ5 U:V NG Bruce S JG Tsotsos ZSaliency Based on )nformation XaximiJationZ5 UeV aGdG Costa ZCisual Saliency and Attention as Random balYs on Com?lex NetworYsZ5 9NNe ULV AG Boccignone5 S XG derraro ZXodelling gaJe s0ift as a constrained random walYZ5 9NNM UOV _G BrocYmann5 TG Aeisel ZAre 0uman scan?at0s aevy cig0tsrZ5 U7V _G 3arY0urst5 TG aaw5 S gG Niebur ZXodeling t0e role of salience in t0e allocation of overt visual attentionZ5 5 9NN9 U8NV _GTG aee5 aG )tti5 CG Toc05 S JG Braun ZAttention activates winner>taYe>all com?etition among visual featuresZ5 5 8777 U88V aG )tti5 JG Braun5 _GTG aee5 S CG Toc0 ZAttention Xodulation of ]uman 3attern _iscrimination 3sy> c0o?0ysics Re?roduced by a suantitative XodelZ5 U89V bG gin0huser S 3G Tpnig5 Z_oes luminance>contrast contribute to saliency ma? for overt visual attentionrZ5 9NNW U8WV fG Rutis0auser5 _G balt0er5 CG Toc05 S 3G 3erona Z)s bottom>u? attention useful for ob^ect recognitionrZ5 U8MV BGbG Tatler5 RGJG Baddeley5 S )G_G Ailc0rist ZCisual correlates of Ixation selectionH gffects of scale and timeGZ 9NN: U8:V JG XaliY S 3G 3erona Z3reattentive texture discrimination wit0 early vision mec0anismsZ 877N
