Rats' invariant object recognition relies on tracking ...

Rats' invariant object recognition relies on tracking salient features across object views

I-90

1,*

1,2

2,*

Alireza Alemi-Neissi , Federica Rosselli , Davide Zoccolan

COSYNE 2011

1

Cognitive Neuroscience and

2

Neurobiolgy Sectors, International School for Advanced Studies (SISSA), Trieste, Italy

* A.A.N. and F.R. equally contributed to this work

Range of size variation

35˚

In fact, although some recent study [1] has shown that rats can recognize objects despite considerable variation in their appearance (e.g., size and viewpoint changes), it is difficult to exclude the possibility that rats’ invariant recognition relies on detection of some low-level image cues that are somewhat preserved across most appearances of the tested objects. In this study, to better understand what perceptual strategy underlies rat object vision, we have exploited an image masking technique [2] that has allowed identifying what “salient” image features rats rely upon when required to recognize objects in spite of image variation.

30˚

25˚

The masks that lead to correct identification of Object 2

Sum of the masks that lead to correct identification of Object 2

The masks applied to Object 2

Sum of all the masks

Trials in which Object 2 was masked

The saliency map resulting from dividing the two planes above

20˚

Range of azimuth rotation

Saliency map superimposed on Object 2 -60˚

0˚

-30˚

30˚

60˚

Correct Trials Incorrect Trials

After learning to discriminate the default objects views, rats were trained to tolerate the range of variation in object size and in-depth azimuth rotation shown, respectively, in the top and bottom row of the figure. Rats were also trained to tolerate variation in the horizontal position of the objects across a ± 18º range (not shown).

Methods 3. The Bubbles masking technique

Methods 1. Visual objects and behavioral task A

Significantly salient image patches (p < .05)

By properly sorting and processing (see figure) the masks that lead, respectively, to the correct or incorrect identification of a given object, we obtained saliency maps in which bright/dark pixels indicated regions that were salient/anti-salient, i.e., likely/unlikely to lead to correct identification of that object [2]. A Z-score test (p < .05) was run to assess what portions of the saliency maps were significantly salient and anti-salient (shown, respectively, as red and cyan patches in the figure).

Saliency maps (A) and juxtaposed salient and anti-salient regions (B) for three rats trained with object pair 2.

Default views of object pair 2

tio po s.

s. de f

au l

tv

tv au l

de f

si

m az i

ze si s. tv

B

au l

size=20˚ size=30˚ size=30˚ size=30˚ size=30˚ azimuth=0˚ azimuth=-40˚ azimuth=20˚ azimuth=0˚ azimuth=0˚ position=0˚ position=0˚ position=0˚ position=-18˚ position=18˚

de f

A

Default size=35˚ azimuth=0˚ position=0˚

Rat A3

To identify what visual features rats used to correctly recognize the visual objects, we presented masked versions of the objects. Masks consisted of opaque patches punctured by a number of randomly located transparent windows (or bubbles) that revealed only partially the object [2].

n

ut h

Results 1. Salient and anti-salient visual features for three rats of group 1 Default views of object pair 1

Probably because of the higher similarity between objects in this pair (as compared to object pair 1) the pattern of salient and anti-salient features looks more complex and sparser. As for object pair 1, there are salient image features that appear to be tracked across position changes (see yellow arrows). In some other cases, it seems that rats had the tendency to rely on image patches that were quite preserved across some transformations (see, for instance, those indicated by the magenta arrows), but, for those transformations in which such “invariant” patches were not preserved, rats still successfully tracked the corresponding salient features (see features indicated by green arrows).

Summary

Methods 4. Trial examples Object 1: Lick left feeding tube

A

Time

Object 2: Lick right Stimulus presentation feeding tube

A. Default views (0º in-depth rotation) of the visual objects that rats were trained to discriminate (each object’s default size was 35° visual angle). A group of six rats was trained with object pair 1, while another group of six animals was trained with object pair 2. B. The behavioral task. Rats learned to interact with a set of three sensors to trigger stimulus presentation (central sensor) and report the identity of the presented test object (either left-side or right-side sensor).

1

Tri al

2

Tri al

3

Tri al

4

Tri al

5

Tri al

6 Rat A5

Tri al

Lick central sensor to trigger stimulus presentation

By exploiting an image masking technique, we have identified what salient image features rats rely upon, when required to discriminate two visual objects in spite of substantial variation in their appearance along a variety of dimensions: size, azimuth and position changes.

Rat A2

B

B

Tri al

1

Tri al

2

Tri al

3

Tri al

4

Tri al

5

Tri al

6

In the final phase of the experiment, presentation of the fully visible target objects, shown across the trained size/rotation/position ranges (see Methods 2), were interleaved with masked versions of the objects (see Methods 3). Masks were first applied to the default object views (A), and then to a subset of the transformed object appearances (see, for instance, masks applied to -40º azimuthrotated objects in B). Each object appearance that was tested with the bubble masks was presented to the rats for 4-8 weeks before switching to the next one. Depending on the fluency of each individual rat in the task, a different number of such masked appearances could be tested.

default vs. B position

Significantly anti-salient image patches (p < .05)

Masked Stimulus

Stimulus

Mask

Significance of the saliency map according to a Z-score test

A

Default size=30˚ size=30˚ size=35˚ position=0˚ position=-18˚ position=18˚

Rat A8

Investigating the neuronal basis of invariant object recognition is a formidable challenge that would greatly benefit from the array of powerful experimental approaches (e.g., genetics and molecular) that are available in rodent studies. However, it remains controversial whether rodents possess the visual processing machinery that can support invariant recognition of visual objects.

Methods 5. Building saliency maps for the visual objects

Rat A9

Introduction

Results 2. Salient and anti-salient visual features for three rats of group 2

Rat A11

Methods 2. Transformation ranges of the visual objects

Salient regions Anti-Salient regions

A. Saliency maps with significantly salient (in red) and anti-salient (in cyan) visual features obtained for three rats trained with object pair 1. Overall, the following pattern emerges: 1. rat recognition appears to rely on a combination of multiple salient and anti-salient visual features; 2. for any given transformation, the pattern of salient and anti-salient features of one object is in “opposition of phase” with the one of the other object; 3. critically, at least some object features (e.g., the top/left lobe of Object 2) did not preserve their position/size/orientation across the transformations the objects underwent, but, nevertheless, appear to be tracked by rats (see yellow arrows), i.e., used as salient features and relied upon to invariantly recognize the target objects. B. Salient and anti-salient regions obtained for different object appearances are shown juxtaposed (the corresponding object views are also displayed on the background as a reference). This helps appreciating the fact that, in many instances, the image features used by the rats to recognize the objects across the tested transformations did not overlap. This excludes the possibility that rats’ invariant recognition trivially relies just on some transformation-invariant visual cues.

This approach revealed that the salient image features for recognition of a given object “tracked” the object’s transformations, i.e., changed in position, size, and orientation, as the object translated, shrunk and rotated in the animals’ visual field. These results indicate that rats’ recognition of visual objects does not trivially rely on some transformation-preserved low-level cues, but, rather, depends on neuronal representations of object features that are truly and largely tolerant to a wide range of image variations.

References [1] Zoccolan D, Oertelt N, DiCarlo JJ, Cox DD, A rodent model for the study of invariant visual object recognition, Proc Natl Acad Sci USA, 2009 [2] Gosselin F, Schyns, PG (2001). Bubbles: A technique to reveal the use of information in recognition tasks. Vision Research, 2001

Acknowledgments Supported by an Accademia dei Lincei- Compagnia di San Paolo grant, and by a Programma di Neuroscienze grant from the Compagnia di San Paolo.