Interpreting LOC Cell Responses

Interpreting LOC Cell Responses David S. Bolme and Bruce A. Draper Computer Science Department Colorado State University Fort Collins, CO 80523 U.S.A. [email protected]

Abstract. Kourtzi and Kanwisher identify regions in the lateral occipital cortex (LOC) with cells that respond to object type, regardless of whether the data is presented as a gray-scale image or a line drawing. They conclude from this data that these regions process or represent structural shape information. This paper suggests a slightly less restrictive explanation: they have identified regions in the LOC that are computationally down stream from complex cells in area V1.

1 Introduction The lateral occipital complex (LOC) is an early part of the human ventral visual pathway. In [2], Kourtzi and Kanwisher present functional magnetic resonance imaging (fMRI) data for previously uninterpreted regions in the LOC. In particular, they identify regions that are not part of the early vision system and respond to both gray scale images and line drawings. Most importantly, they observe adaptation, in the identified regions, across the format type indicating that the identified regions respond to the type of object presented, and not to the presentation format (image or line drawing). Kourtzi and Kanwisher interpret their data as identifying regions in the LOC that process and/or represent structural (shape) information [2]. This paper proposes a simpler explanation for their data: they have identified regions in the LOC that are computationally downstream from complex cell responses in V1. Our hypothesis is supported by a computer simulation that implements standard models of simple and complex cells. It shows that the complex cell responses to images and line drawings of the type used in [2] are indistinguishable. These two interpretations are not incompatible. It may be that the LOC computes shape features based on complex cell responses. However, our explanation is less restrictive, since there are other properties that can be extracted from complex cell responses, including texture features and size features. At the same time, there are other regions of the brain that could provide input for format invariant shape descriptors. More experiments are therefore needed to distinguish between these hypotheses.

2 LOC response data Prior to [2], it was known that regions in the LOC respond better to intact images of objects than to scrambled images, and respond equally to familiar and unfamiliar objects.

2

David S. Bolme and Bruce A. Draper

Kourtzi and Kanwisher hypothesized that some of these LOC regions might process shape information. To test this hypothesis, they generated images and line drawings of synthetic objects, similar to the ones shown in Figure 1, in both scrambled and intact forms. They focused their attention on voxels in the LOC that are activated more strongly by intact than scrambled objects, but respond equally to images and line drawings. They then searched for cellular adaptation to probe what these regions might compute. In particular, they show that activation in these regions is suppressed on the second consecutive presentation of the same object, but not when two different objects are presented in succession. This suggests that different cells within the LOC are responding to each object type, and these cells are subject to adaptation. They then showed that the adaptation happens across format type (image or line drawing), suggesting that the same cells respond to an image of an object and its line drawing. Their hypothesis is that these cells must be processing structural or shape information. Kourtzi and Kanwisher performed additional experiments that examined the region localized in this part of the study. Two more similar experiments were presented in [2]. The second experiment repeated the first, using familiar household objects rather than synthetic shapes. The other used partially occluded images and line drawings. In both cases, the same cells responded to object shape accross stimulus formats. Additional exploration of this region was presented in [3]. This study attempts to determine discern if the region in the LOC responds to percieved shape or simple contours.

3 Complex Cells in V1 Area V1 of the visual cortex receives input directly from the lateral geniculate nucleus (LGN), and is the starting point for higher cortical regions of the ventral visual pathway. We know from single cell recording studies dating back to Hubel and Wiesel [1] that there are at least three types of cells in V1: simple cells, complex cells, and hypercomplex cells. Of these, complex cells make up the majority (approximately 75%) of the cells in V1 [6]. ˜ msec post stimulus preIt is now generally accepted that the early responses (40 sentation) of simple cells in V1 can be modeled as half-rectified Gabor filters, where the Gabor filters are parameterized by location, orientation, scale and phase (sometimes called parity) [7]. Depending on the phase, a Gabor filter can act like either a bar or edge detector at a particular location, orientation and scale. It is also generally accepted that complex cells combine the responses of simple cells that differ in phase, but not orientation (except for 180 degrees) or scale. The complex cell response is the total energy at a given frequency, orientation, and location [7]. As a result, complex cell responses combine simple edge and bar responses. Complex cells therefore respond similarly to gray scale images and line drawings of an object. More specifically, complex cells respond strongly to edges in gray-scale images, which are formed by the object’s silhouette and by internal contours. Since they also respond to the lines (bars) in line drawings, they should respond similarly to both formats.

Interpreting LOC Cell Responses

3

It should be noted that Zipser et al [8] observed contextual adaptation in V1 cells. While the initial responses of V1 cells can be modelled in terms of Gabor functions, later responses (80 msec or more post stimulus) are modified by contextual factors outside of the cell’s classically defined receptive field. This contextual adaptation enhances or suppresses a cell’s response to a bar or edge. These observations are confirmed by Lee, et al [5], who also observed a second wave of adaptation approximately 200 msec post stimulus.

4 A Computational Simulation This paper depends on the hypothesis that complex cells in V1 respond indistinguishably to images and line drawings of an object. To verify this, we simulate the initial responses of complex cell to gray scale images and line drawings of objects. The simulation replicates the setup of Kourtzi and Kanwisher’s experiments in terms of formats, and uses correlation to measure the similarity of complex cell responses. 4.1 Test Imagery The test imagery is based on 256 randomly generated, three dimensional shapes. These stone-like shapes are composed of smooth faces and sharp contours. The objects were rendered using orthographic projection from a random viewing angle. Images and line drawings were rendered for each object. The gray scale images were generated using OpenGL, with realistic lighting techniques and smooth shading. Line drawings were created by rendering lines for the outline and internal contours of the three dimensional objects. The images were rendered at 300 X 300 pixels and rescaled to 150 X 150 pixels for use in the simulation. See Figure 1 for sample images.

Fig. 1. Gray scale images and line drawings for eight sample objects.

4.2 Cell Models The cell models are based on Gabor filters [7, 4]. Simple cell responses (S) are computed for every pixel in the image using four different orientations and one frequency. The frequency was chosen to be in a range that responds well to sharp edges and thin lines. The Gabor kernels were generated using the following equation:

4


x 2

2σ2 S x y θ ϕ e cos 2π xλ x x cos θ y sinθ y x sin θ y cosθ γ2 y 2

ϕ

π where λ 4, γ 0 5, σ 0 56λ, θ 0 π4 π2 3π 4 , and ϕ 0 2 . (These parameters are consistent with biological models of simple cells, as discussed in [4].) The complex cell response (C at any point and orientation is based on the simple cells for that same location and orientation, and combines the symmetric (bar-like) and anti-symmetric (edge) filter responses. The model can be computed as [7]:

C x y θ S x y θ 0

2

S x y θ

π 2 2

Because Kourtzi and Kanwisher’s first experiment [2] did not exploit texture boundaries, stereo disparity or color, the initial and adapted V1 cell responses to their images should be approximately the same. Hence the data gives no basis for distinguishing whether the LOC regions are responding to the initial or adapted responses. See Figure 2 for examples of complex cell responses. 4.3 Results Example complex cell responses are shown in Figure 2. Qualitatively, the outline of the gray scale images is the dominant feature. The internal contours are present, however they exhibit a much lower response. For line drawings, the complex cells respond equally well to both the outline and internal contours. Dispite this difference the complex cell responses still show a remarkable qualitative similarity. Standard linear correlation was used to measure the similarity of the complex cell responses. In particular, we correlated the complex cell responses to images and line drawings of the same object. As a baseline, we correlated the responses to images and line drawings of different objects. Histograms were produced to illustrate the distributions of these correlation scores (see Figure 3), and show that the complex cell responses to images and line drawings of the same object are virtually the same. It is also neccesary to examine the relationship between the correlation values for different objects of the same format to the correlation values for different formats. These results shown in Figure 4 illustrate that complex cell responses are virtually unchanged by format type.

5 Conclusions Kourtzi and Kanwisher have identified regions in the LOC that respond to specific objects regardless of the format of the data. They conclude that these regions respond to the structure (shape) of the object. The simulations conducted in this paper show that complex cell responses are qualitatively and quantitatively similar for line drawings and gray scale images. Therefore, we believe that the simplest explanation is that Kourtzi and Kanwisher have identified regions of the LOC that rely indirectly on complex cell


5

Fig. 2. Complex cell responses for one object. The top two images are the raw data. The middle images show the complex cell responses for each orientation. The bottom images show the sum of all four complex cell outputs.

6


0.3

Same Object/Different Format Different Object/Different Format

Normalized Count

0.25

0.2

0.15

0.1

0.05

0 -0.2

0

0.2

0.4

0.6

0.8

1

Correlation

Fig. 3. Histograms of the (high) correlations between complex cell responses to images and line drawing of the same object, and the (low) correlations between responses to images and line drawings of different objects.

0.3

Same Object/Different Format Different Object/Different Format Different Object/Line Format Different Object/Gray Format

Normalized Count

0.25

0.2

0.15

0.1

0.05

0 -0.2

0

0.2

0.4

0.6

0.8

1

Correlation

Fig. 4. Histograms comparing correlation scores between different objects presented as gray scale/gray scale, gray scale/line drawing, and line drawing/line drawing. This shows that complex cell responses do not depend on image format. The correlation histogram for the same objects/different format is also included in purple as a reference point.


7

responses in V1. We believe that this explanation fits the latter experiments in their paper as well. Unfortunately, we are unable to distinguish from the data whether these LOC regions rely on the initial or adapted V1 cell responses. Kourtzi and Kanwisher note that of the regions in the LOC that respond to either type of stimulus, a majority of them responded to both types of stimuli. Since 75% of the cells in V1 are complex cells, it is highly plausible that many of the regions in the LOC would respond to the outputs of complex cells.

Acknowledgments I would like to thank Zoe Kourtzi for providing initial data and for her words of encouragement. David Bolme was supported under grant DUE-0094514 from the National Science Foundation Computer Science, Engineering, and Mathematics Scholarship Program.

References 1. D. Hubel and T. Wiesel. “Receptive Fields of Single Neurons in the Cat’s Striate Cortex,” Journal of Physiology (London), 148:574-591. 2. Z. Kourtzi and N. Kanwisher. “Cortical Regions Involved in Perceiving Object Shape,” Neuroscience 20(9):3310-3318, 2000. 3. Z. Kourtzi and N. Kanwisher. “Representation of Perceived Object Shape by the Human Lateral Ocipital Complex,” Science 293:1506-1509, 2001. 4. P. Kruizinga, N. Petkov and S.E. Grigorescu. “Comparison of texture features based on Gabor filters,” International Conference on Image Analysis and Processing, Venice, p.142147, 1999. 5. T.S. Lee, D. Mumford, R. Romero and V.A.F. Lamme. “The Role of the Primary Visual Cortex in Higher Level Vision,” Vision Research, 38:2429-2454, 1998. 6. S. Palmer. Vision Science: Photons to Phenomenology, MIT Press, Cambridge, MA, 1999. 7. D. Pollen, J. Gaska and L. Jacobson. “Physiological Constraints on Models of Visual Cortical Function,” Models of Brain Functions, M. Rodney and J. Cotterill (eds.), Cambridge University Press, New York, 1989. 8. K. Zipser, V.A.F. Lamme, P.H. Schiller. “Contextual Modulation in Primary Visual Cortex,” Neuroscience, 16(22):7376-7389, 1996.