Enhancement of Object Representations in Primate Perirhinal Cortex ...

J Neurophysiol 97: 1298 –1310, 2007. First published November 15, 2006; doi:10.1152/jn.00167.2006.

Enhancement of Object Representations in Primate Perirhinal Cortex During a Visual Working-Memory Task Sidney R. Lehky1 and Keiji Tanaka1,2 1

Cognitive Brain Mapping Laboratory, RIKEN Brain Science Institute, Saitama; and 2Graduate School of Science and Engineering, Saitama University, Saitama, Japan Submitted 16 February 2006; accepted in final form 6 November 2006

The inferior temporal cortex (IT) plays a central role in visual object processing. Neurons in dorsal and ventral regions of anterior IT (TEad and TEav), the last unimodal visual areas, are selective for complex object features (Desimone 1984; Rolls and Tovee 1995; Tamura and Tanaka 2001; Tanaka 1996), thus providing the capacity to perceptually identify and discriminate stimuli. Perirhinal cortex (PRh) is believed to have an involvement in the mnemonic aspect of recognition for complex objects in addition to perhaps contributing further to perceptual processing. It is a polymodal structure medially adjacent to TEav on the ventral surface of IT. TE provides major anatomical inputs to PRh and in return receives strong feedback projections from PRh (Lavenex et al. 2002; Saleem and Tanaka 1996; Suzuki and Amaral 1994; Van Hoesen 1982). PRh in particular appears to play a multifaceted role in object processing. In the context of long-term visual memory, recognition of complex objects seems to preferentially involve the PRh over neighboring areas. Lesions to PRh produce greater behavioral deficits for object recognition than lesions to nearby limbic structures, during delayed match to sample (DMS) or similar tasks (Alvarado and Bachevalier 2005; Baxter and Murray 2001; Meunier et al. 1993; Murray and Mishkin 1998; Zola-Morgan et al. 1989). Lesions to PRh also produce greater deficits to recognition memory than lesions to the adjacent visual cortex, area TE (Buckley et al. 1997; Buffalo et al. 1999). Moreover, disrupting cholinergic input to PRh but

not TE or dentate gyrus impairs recognition memory (Tang et al. 1997; Turchi et al. 2005). As the delay tasks in these studies used trial-unique stimuli and delays stretching into several minutes, they were most likely primarily probing long-term memory. Long-term memory has also been examined at the single-cell level in temporal cortex. Miyashita and colleagues studied associative memory between visual stimuli (Miyashita 1993; Naya et al. 2001; Takeda et al. 2005), finding a critical role for PRh in the formation of associations. Stimulus-reward associative memory was studied by Liu and Richmond (2000) and Mogami and Tanaka (2006), who both found a larger involvement of PRh over TE. There is also an extensive literature on familiarity and recency effects on single-cell activity in TE and PRh (reviewed by Brown and Aggleton 2001), which to a large extent also involve long-term memory. In addition to long-term memory, working memory within temporal cortex has been the subject of monkey studies (Baylis and Rolls 1987; Davachi and Goldman-Rakic 2001; Eskandar et al. 1992; Fuster 1990; Fuster and Jervey 1981; Holscher and Rolls 2002; Miller and Desimone 1994; Miller et al. 1991; Miyashita and Chang 1988; Nakamura and Kubota 1995; Sobotka et al. 2005). Working memory is believed to be closely related to attention (Awh and Jonides 2001; Awh et al. 2006; Barnes et al. 2001; de Fockert et al. 2001; Desimone 1996). In this study, we shall be examining single-cell activity in both TE and PRh associated with the performance of a working-memory task. Working-memory effects within temporal cortex are likely to be closely tied to the perceptual properties of various structures within that region. Although the perceptual role of area TE is clear, that of PRh is less so. However, there is mounting evidence that in addition to mnemonic functions, PRh may play a role in perceptual processing. This evidence indicates that PRh is necessary when object identification requires the conjunction of a more complex set of visual features than TE neurons are capable of handling (Buckley and Gaffan 1997; Buckley et al. 2001; Bussey et al. 2002, 2003, 2005; Eacott et al. 1994; Lee et al. 2005; Murray and Bussey 1999; Tyler et al. 2004; see also Gaffan 2002). This perspective treats PRh, in terms of its visual perceptual properties, as the next link beyond TE in a chain of cortical areas stretching back to V1 with increasingly complex receptive fields. However, such a perceptual role for PRh remains controversial, and others have suggested that the function of PRh can be under-

Present address and address for reprint requests and other correspondence: S. R. Lehky, Computational Neuroscience Lab, The Salk Institute, 10010 N. Torrey Pines Road, La Jolla, CA 92037 (E-mail: [email protected]).

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

INTRODUCTION

1298

0022-3077/07 $8.00 Copyright © 2007 The American Physiological Society

www.jn.org

Downloaded from jn.physiology.org on February 20, 2007

Lehky SR, Tanaka K. Enhancement of object representations in primate perirhinal cortex during a visual working-memory task. J Neurophysiol 97: 1298 –1310, 2007. First published November 15, 2006; doi:10.1152/jn.00167.2006. We compared single-cell activities in perirhinal cortex (PRh) as well as adjacent visual cortex (area TE) across two tasks. One task required the monkey to identify any stimulus repetition within a sequence of object stimuli. In the other task, the same stimuli were presented, but the monkey didn’t have to remember them. PRh responses during the object-memory task were elevated relative to those during the second task. In TE, on the other hand, there were no significant task-related differences in responses. We did not observe task-related differences related to repetition effects in either brain area. The onset of the enhanced signal in PRh during the object-memory task occurred with a latency of 80 ms after the onset of the stimulus response, suggesting that it was the result of top-down feedback.

ENHANCED PERIRHINAL ACTIVITY DURING A WORKING-MEMORY TASK

J Neurophysiol • VOL

(the sample), we used a serial-recognition task that required keeping multiple stimulus images in working memory simultaneously. If prefrontal cortex was providing a control signal to representations in temporal cortex, then it might be more visible if prefrontal cortex was keeping track of multiple items rather than just one. Although we used a working-memory task, there is evidence for extensive overlap between mechanisms of working memory and attention, as will be detailed in the DISCUSSION, and we shall consider the data within the context of both those factors. METHODS

Two female Japanese macaque monkeys (Macaca fuscata) were used, weighing 7 and 9 kg. All experimental procedures conformed to National Institutes of Health guidelines on the care and use of laboratory animals. Recordings were conducted on the right side of the brain in both cases, ranging over anterior-posterior positions A17–A19 in one monkey and A14 –A17 in the other (Fig. 1). Chambers were angled at 10° so that electrodes slanted laterally as they advanced.

Recordings There were three target structures, TEad, TEav, and PRh, each adjacent to the next as one moves down ventrally along the inferotemporal cortex. TEad is in the middle temporal gyrus, extending ventrally into the lateral lip of the anterior medial temporal sulcus (amts). TEav is located around the medial bank and medial lip of amts. PRh, or area 35/36, is located on the lateral bank and lateral lip of the rhinal sulcus (rs) (Saleem and Tanaka 1996; Suzuki and Amaral 1994). Because these structures are located so close to each other in the mediolateral direction, electrode positions had to be determined with precision. To do this, stereotactic coordinates of the target structures were first obtained from MRI scans. Then, after each recording session, an X-ray was taken with the electrode still in place within the brain, using a portable veterinary X-ray machine. From the X-ray images, the lateral distance between the electrode guide tube tip and the brain midline was measured. This in turn allowed determination of the mediolateral position of the tip of the slanting electrode. Slight rotations of the head relative to the fronto-parallel plane of the X-ray machine were mathematically corrected using information from small

A

C

Monkey 1

Monkey 2

1 cm

B

D

A18

amts rs

sts

sts

hc

A16

hc

1 cm

amts rs

FIG. 1. Recording sites, indicated in gray. Gray areas on the lateral views of the brains indicate anterior-posterior ranges of recording. rs, the rhinal sulcus; amts, the anterior middle temporal sulcus; sts, the superior temporal sulcus.

97 • FEBRUARY 2007 •

www.jn.org


stood purely in mnemonic terms (Hampton 2005; Hampton and Murray 2002; Levy et al. 2005; Squire et al. 2004). The primary structure associated with working memory is lateral prefrontal cortex. This association is based on lesion studies extending as far back as Jacobsen (1936) and Mishkin (1957) as well as observations of delay activity in prefrontal neurons during delayed-match-to-sample or delayed-response tasks (Fuster 1973; Fuster and Alexander 1971; Kubota and Niki 1971; Niki 1974). The portrayal of prefrontal cortex as a working-memory buffer received an influential formulation by Goldman-Rakic (1987). Although there is widespread agreement that prefrontal delay activity plays an important role in working memory, there is disagreement about what that role is. On one hand, there is the view that prefrontal delay activity actually represents the contents of working memory, including specific stimulus features (reviewed by Courtney 2004). By this view, representations in visual working memory do not engage the same posterior neural structures that are involved in perceptual representations of stimuli. Alternatively, there is the view that the contents of visual working memory are indeed represented in posterior sensory cortices. By this view, prefrontal delay activity serves as a control signal engaged in the attentional activation, manipulation, or monitoring of the posteriorly held contents of working memory (Curtis and D’Esposito 2003; Lebedev et al. 2004; Petrides 1994, 2000; Postle 2006; Postle et al. 1999). In the particular case of TE and PRh, the possibility of feedback interactions with prefrontal cortex of the sort outlined in the preceding text is supported by the existence of extensive anatomical connections running both ways (Lavenex et al. 2002; Rempel-Clower and Barbas 2000; Suzuki and Amaral 1994; Ungerleider et al. 1989; Webster et al. 1994). If prefrontal cortex was indeed projecting a control or activating signal to posterior representations during a working-memory task, as the later hypothesis in the preceding text indicates, then PRh and TE, given their physiological properties and anatomical connections, would be likely places to look for such a signal in the context of remembering complex objects. Delay activity is considered the primary neural substrate of working memory in prefrontal cortex. The fact that prefrontal delay activity survives disruption by intervening visual stimuli between sample and match in a DMS task (Miller et al. 1996) reinforces that belief. However, delay activity does not survive intervening stimuli in PRh, making it unlikely that it is the substrate for working memory in that structure (Miller et al. 1993). Subsequently it was suggested that a repetition enhancement effect for the match stimulus in a DMS task represented the neural substrate of working memory in PRh cortex (Miller and Desimone 1994). In this study, we have taken a different approach to identifying activity in temporal cortex associated with working memory. Rather than comparing neural responses during different epochs within a single working-memory task, we made a paired comparison across two tasks. In one task, the monkey was required to keep a set of serially presented object stimuli in working memory. In the second task, the monkey again viewed the same stimuli but without the memory requirement while being kept busy encoding into memory a different set of unrelated stimuli. Instead of using a delayed-match-to-sample task, which requires keeping just one item in working memory

1299

1300

S. R. LEHKY AND K. TANAKA

(1 mm) titanium markers implanted in the skull at known stereotactic coordinates, which were visible on the X-ray images. At the end of the experiments, the animals were killed, and recording sites were verified histologically through examination of electrode tracks. In a few cases, tracks fell too far medially in the rhinal sulcus, and the corresponding cells were excluded from the data. In an attempt to reduce selection bias, all isolated cells with detectable activity, whether spontaneous or stimulated, were accepted for complete testing without informal prescreening of cells for responsiveness.

Behavioral task and stimuli

FIG. 2. Example stimulus images. These are a subset of the 110 images each cell was initially tested on. Based on responses during this preliminary screening, 10 images were selected for further testing (a different set for each cell). Those 10 images were not the 10 most effective stimuli but included a full range of effectiveness from the best to the worst.



www.jn.org


Stimuli were a set of 110 color photographs of real-world objects, both natural and man-made (examples shown in Fig. 2). They were presented against a neutral gray background that matched the general background of the video display, set at 40 cd/m2, so that the stimulus boundary box was not visible. The boundary box subtended a visual angle of 6° (full width). The video display had a resolution of 1,024 ⫻ 768 pixels and a frame rate of 75 Hz and was viewed from a distance of 48 cm. Eye position was monitored with an optical eye tracker. The monkeys were repeatedly exposed to the same set of 110 object stimuli during several months of training, so all the stimuli were highly familiar to them by the time recordings were started. The general testing procedure for each cell was as follows. First, a preliminary test using the entire 110 object stimulus set was performed. Based on that, 10 stimuli were chosen for the main phase of the testing, in which two tasks, an object-memory task and a dotmemory distractor task, were interleaved. A serial-recognition task was chosen rather than a serial delayed-match-to-sample (sDMS) task in an attempt to increase the response difference from the distractor task (see following text). A serial-recognition task requires keeping multiple stimuli in memory simultaneously and not just a single sample stimulus as in sDMS. Thus it provides a heavier memory load than a sDMS task. The object-memory task and the dot-memory distractor task, as will be described in the following text, were both essentially the same working-memory task operating on different stimulus sets. The strat-

egy was to select visual patterns for each task that stimulated different populations of neurons, taking advantage of the narrow pattern selectivity of cells in inferotemporal cortex. As the monkey switched back and forth between remembering one set of stimuli or the other, memory-related activity would be expected to switch back and forth between the two neural populations. Because we would be recording from just one of those two populations (those neurons stimulated by the images for the “object-memory” task), the memory-related signal would appear to switch on when the monkey had to remember the stimuli for that task and off when the memory related signal was attached to a different population of neurons forming representations for the other class of stimuli. A “memory-related signal” in this context could include an attentional aspect to it, as directing attention to a stimulus may be a condition for the stimulus to enter working memory. During the preliminary testing phase, the monkey was performing the object-memory task. Each of the 110 objects was presented a minimum of two times (mean: 2.5). After that, stimuli were sorted in order of the firing rate elicited (effectiveness), and the top four were selected, as well as six additional stimuli evenly spaced over the remaining range of effectiveness. In the object-memory task, which was a variant of a serial recognition task, the monkey viewed a series of object stimuli (Fig. 3A). The task was to report any repetition in the series by releasing a touch sensitive bar within 800 ms of repetition onset. The trial terminated after the first repetition occurred. The stimulus series length randomly varied from two to five images. Each object stimulus was presented for 600 ms, with a 700-ms inter-stimulus interval between them. A square fixation spot (0.3o) was presented at the start of each trial and during interstimulus intervals within the series with no temporal gaps between the fixation spot and adjacent object stimuli. In the second task, the dot-memory distractor task, the monkey viewed the same serially presented object stimuli but was not required to remember them. To discourage the monkey from covertly remembering the object stimuli, the dot-memory task was performed as a distractor during each trial (Fig. 3B). During this task, a round dot (0.5° diam), the color of which was randomly selected from six possibilities, was displayed, starting before the first object stimulus and then afterward during each interstimulus interval between object stimuli. The monkey’s task was to respond to any repetition in the color of the dot within 800 ms of stimulus onset. The object stimuli had no significance for correct performance of the task now. As was the case for the object-memory task, two to five object stimuli were presented during each trial but now with the colored dot distractor displayed in the period preceding each object stimulus. The location of the colored dot was randomly placed anywhere within 2° of fixation during each presentation within a trial, in an attempt to discourage the monkey from maintaining a narrow focus of spatial attention. For the distractor task, object stimulus presentations remained 600 ms, but the interstimulus intervals were increased to 900 ms. The color-dot distractor stimulus was displayed during the central 500 ms of the interstimulus intervals, so that there was a 200-ms gap before and after the distractor dot separating it from the object stimuli. A square fixation spot was displayed simultaneously with the distractor dot in exactly the same manner that the fixation spot had been displayed during the periods between object stimuli in the objectmemory task (Fig. 3). During the distractor task, 50% of the trials presented each object stimulus just once within the trial, whereas the other 50% included a repeat presentation of one of the objects. For this task, however, a repeat presentation of an object stimulus had no significance to the monkey. The two tasks were presented in interleaved blocks of trials, two blocks for each task for a total of four blocks, while the activity of each neuron was recorded. Within each block, each object stimulus was presented a minimum of 10 times (mean: 12). Trials were also


A

B

700-ms SOA between the dot stimuli and object stimuli in our dot-memory task. Therefore we don’t think that limitations in the temporal dynamics of deploying attention were a significant factor in our experiment.

Object-memory task

Time

release bar

Data analysis

Dot-memory distractor task

Fixation spot on Object-stimulus on Distractor-stimulus on

1301

release bar

included which contained only the colored dots of the distractor task, with the object stimulus periods left blank, to test sensitivity of cells to the color stimuli. The colored dots in the distractor task were presented with the fixation spot on, using the same gray background as that used for object stimuli. For both tasks, the task-relevant stimulus repetition occurred with equal likelihood at the four possible time positions in the series (positions 2–5). We calculated chance performance conservatively by assuming that the monkeys had learned the basic structure of the task, so that they knew this probability of stimulus repetition at each position. In that case, if they ignore the stimuli and decide at the start of each trial to respond randomly at one of those four positions, they would have 25% chance of correct performance. Although the distractor task included an additional color dot in the periods between object stimuli (Fig. 3), we believe it unlikely that this would result in a response difference between the two tasks due to temporal crowding or masking effects. A fixation spot, similar in size and shape to distractor color dot, preceded the object stimuli under both tasks, with an ISI of 0 ms. Presenting an additional color dot would likely have added little if any masking over that already produced by the fixation spot, particularly since the ISI between the color dot and object stimuli was 200 ms, and neural masking effects terminate with ISIs of 100 –150 ms (Kondo and Komatsu 2000; Kovacs et al. 1995; Macknik and Livingstone 1998; Rolls and Tovee 1994; Rolls et al. 1999). Furthermore, in our data there was no perturbation to the early part of the object-stimulus response (Fig. 7C), whereas suppression of the early response was a significant feature in the physiological masking studies of Macknik and Livingstone (1998) as well as Kondo and Komatsu (2000). For these reasons, we believe the effects reported here reflect cognitive processing (attention, memory) rather than sensory responses to the stimuli, although the possibility of a sensory contribution cannot be completely excluded. Besides masking, another possible effect during target detection within serial presentation of stimuli is attentional blink (Raymond et al. 1992). However, our tasks proceeded at a very slow pace relative to the rapid serial visual presentation (RSVP) procedure used during attentional blink experiments, with a 1,200-ms SOA (stimulus onset asynchrony) between object stimuli in our object-memory task and J Neurophysiol • VOL

S E ⫽ HG ⫺ H共r兲

(1)

where HG is the entropy of a Gaussian, and H(r) is the entropy of a cell’s response distribution taken from the data (Lehky et al. 2005). Equating high selectivity with low entropy captures the characteristic of a highly selective cell of having little response to most stimuli and a large response to a few. The entropy of the Gaussian reference distribution is given by Rieke et al. (1997) 1 H G ⫽ log2共2␲e␴2兲 ⫽ 2.074 bits 2

(2)

with variance normalized to one. Entropy of the probability distribution of the data are H共r兲 ⫽ ⫺ 兰p共r兲log2共p共r兲兲dr

(3)

where r is the response and p(r) is the response probability density function (pdf). For practical calculations Eq. 3 is discretized to

冘 M

H共r兲 ⫽ ⫺

p共rj兲log2共p共rj兲兲⌬r

(4)

j⫽1

where responses from n stimulus images have been placed in M bins. Expressing the selectivity index SE (Eq. 1) in terms of Eqs. 2 and 4, we get


www.jn.org


FIG. 3. A: object-memory task: monkey responded to any repetition within a series (2–5) of object stimuli. The duration of object stimuli was 600 ms and the interstimulus interval was 700 ms. B: dot-memory distractor task: monkey viewed a series of object stimuli, but was not required to remember them. To discourage the monkey from covertly retaining object stimuli in memory, a distractor task was included. The distractor involved displaying a small dot with a randomly selected color during the inter-stimulus intervals between image presentations. Each dot presentation was at a random location to discourage the monkey from maintaining a narrow focus of attention. The monkey responded to any repetition in dot color. Duration of object stimuli was 600 ms, duration of distractor dot stimuli was 500 ms, and the interval between object stimuli was 900 ms.

Response was defined as average firing rate over the stimulus duration shifted by latency. Latency was determined as time to half-height of the first peak in the grand average PSTH, incorporating data from all cells within a given cortical area to all stimuli. Onset time for the stimulus presentation was corrected for the time it took the video electron beam to reach the center of the screen, ⬃8 ms. Estimates of response standard errors were made using Bayesian methods (Lehky 2004), and the SEs were used to determine those stimuli producing significant responses. Spontaneous activity was determined from a 150-ms window preceding each object stimulus presentation. Cells that showed ⬎2 spike/s change in responses to the colored dot distractor stimuli in trials of the distractor task with no object stimulus presentation were excluded from the analysis regardless of the significance of that response. This was done to reduce the possibility that object-stimuli responses would be contaminated by afterdischarges from responses to the dot stimuli. In some cases, the activity in the excluded cells may have reflected above-average variability in the background firing rather than responses to the dot stimuli per se. For grand average comparisons of neural activities between the two tasks, the last stimulus presentation within each trial, which overlapped the monkeys’ behavioral responses, were omitted from the analysis. For comparisons between repeat presentations of a stimulus within single trials, all stimulus presentations were considered in the analysis, but with inclusion of only the first 300 ms of the PSTH in each case, so that there was no overlap with behavioral responses. Visual selectivity can be measured in terms of the shape of the probability density function (pdf) of a neuron’s responses over the stimulus set (Lehky et al. 2005; Simoncelli and Olshausen 2001). Here we define selectivity SE as the change in entropy of a neural response pdf relative to the entropy of a Gaussian distribution, which has maximum entropy for a fixed variance

1302


冘 M

S E ⫽ 2.074 ⫹

p共rj兲log2共p共rj兲兲⌬r

(5)

j⫺1

The pdf determined from the data must be normalized to a variance of 1.0 before performing this calculation. Bin size ⌬r was set by dividing the data range in M bins equal to the square root of the number of images in the stimulus set. To examine repetition effects, we calculated, for each trial, the ratio of the response during the second presentation of a stimulus image relative to the first presentation. The array of response ratios was then log transformed to normalize the distribution, and a t-test was conducted on the data for each cell, examining if log(ratio) was significantly different from zero. For plots of the probability distribution of response ratios, the average ratio for each cell was calculated as the geometric mean over all trials, and then response ratios were incorporated into the probability plots on a cell-by-cell basis.

RESULTS

probability density

0.10 TEad TEav PRh

0.08 0.06 0.04 0.02 0

0

10

20

30

spikes/sec FIG.

4. Probability density functions (pdf’s) of neural responses to object stimuli. Responses of each cell to 110 stimuli were used to calculate that cell’s pdf, and then the median pdf was determined for the different cortical areas. These curves show that PRh neurons have much lower probabilities of producing large responses than TE cells do. Peaks of the distributions approximately coincide with spontaneous activity. The number of units underlying these curves are TEad: 65, TEav: 181, and PRh: 115. J Neurophysiol • VOL


www.jn.org


Data were analyzed from 65 cells in TEad, 181 cells in TEav, and 115 cells in PRh, collected while the macaque monkey performed two tasks, an object-memory task (Fig. 3A) and a dot-memory distractor task (Fig. 3B). Histology showed that the TEad cells came from the middle temporal gyrus near the lateral lip of the anterior medial temporal sulcus (amts), TEav cells from the medial bank and medial lip of amts, and PRh cells from the lateral bank and lateral lip of the rhinal sulcus (rs; Fig. 1). Each cell underwent preliminary testing with a large set of object images (110 images) to select effective stimuli for the main phase of the experiment. ANOVA analysis of this data indicated visual selectivity in 100% of TEad cells, 97% of TEav cells, and 98% of PRh cells (P ⬍ 0.01). The probability density function of the response magnitudes for each cell to all 110 stimuli was determined using Gaussian kernel smoothing methods. Figure 4 shows the median pdf in each cortical area. They were determined by finding the median probability density value at each firing rate, over all cells. The PRh cells differ from those in TE primarily by having

more symmetrical pdf”s with less of a tail of large responses extending to the right. Visual selectivity to complex stimuli (natural images, objects) can be quantified by measures of the overall shape of the response probability density function (Lehky et al. 2005). Here we define selectivity SE as the change in entropy of a response pdf relative to a Gaussian pdf. The Gaussian is referenced as the density function for a minimally selective neuron (see METHODS). Applying this selectivity measure to the pdf of each cell and taking median values, we find SE ⫽ 0.33 for TEad, SE ⫽ 0.32 for TEav, SE ⫽ 0.22 for PRh, on a scale where SE ⫽ 0 for the low-selectivity Gaussian distribution. After preliminary testing of each cell with 110 object stimuli, 10 objects were selected for use in the main phase of the experiment comparing responses during the object-memory task (Fig. 3A) and dot-memory distractor task (Fig. 3B). Overall correct performance for the object-memory task was 84% and for the dot-memory distractor task it was 74%. This was calculated on the basis of trials that were not aborted because of broken fixation. If one includes aborted trials, correct performance for the object-memory task was 74% and for the dot-memory task 68%. Both monkeys showed poorer performance on the distractor task. Performance on distractor trials that included object stimulus repetition (78% correct) was close to performance during those trials that did not include repetition (85% correct). This indicates that the monkeys were indeed selectively responding to the color dot stimuli during the distractor task and not indiscriminately to any stimulus repetition that appeared on the screen. Behavioral statistics are presented in more detail in Fig. 5. Performance declined as the length of the stimulus series increased (Fig. 5A). On the other hand, performance as a function of the separation (number of items) between the first presentation (cue) and second presentation (match) of the target stimulus formed a U-shaped curve (Fig. 5B). Although the distractor curves are shifted down slightly, the shapes of the curves for both tasks are similar, and there is no indication here that the monkeys were using different strategies for performing these two serial recognition tasks. Statistics for trials aborted because the monkeys broke fixation are also similar for the two tasks with the abort rate slightly lower for the dot-memory task (Fig. 5C). Yakovlev et al. (2004) provide a detailed consideration of behavioral statistics for monkeys performing essentially the same serial recognition task we used. The U-shaped curves in Fig. 5B are examples of wellknown serial position effects that occur when a list of sequentially presented items is held in short-term memory. The upturn of the curve at the left is an indication of the recency effect, as it occurs when the cue immediately precedes the match stimulus. The upturn at the right is an example of the primacy effect, as it occurs when the cue stimulus is the first item presented in the stimulus series. Primacy and recency memory effects occur in both human subjects (Atkinson and Shiffrin 1968, 1971; Glanzer and Cunitz 1966; Murdock 1962) and monkeys (Castro and Larsen 1992; Sands and Wright 1980; Wright et al. 1985). The poorer correct identification of the match stimulus on the dot-memory distractor task may be related to lower saliency of the stimuli for that task. The stimuli in the dotmemory task were small (0.5°), simple geometric shapes that differed along only one dimension (color). On the other hand,


A percent correct

100 80 60 40 20 0

2

3

4

5

stimulus series length

B 80 60

percent broke fixation

n=65

spikes/sec

2

16

3

match position difference 50

8 0

40

B

30 20 10 0

TEad

24

20 0 1

C

A

40

0

200

400

n=181

spikes/sec

percent correct

100

600 TEav

24

16

2

3

4

5

stimulus series length FIG. 5. Behavioral statistics for the 2 tasks. A: percentage correct as a function of the length of the stimulus time series, which varied randomly between 2 and 5 in different trials. Percentages were calculated excluding aborted trials. Chance performance is set at 25% on the assumption that for each trial the monkey picks at random one of the four possible time positions of the target stimulus. B: percentage correct as a function of the difference in position along the stimulus time series between the 1st presentation of the target stimulus and the 2nd presentation (match). C: percentage of trials that were aborted because the monkey broke fixation as a function of the length of the stimulus time series.


0

C

0

200

400

n=115

600 PRh

24

spikes/sec

the object-memory patterns were large (up to 6°), with each comprising a complex multidimensional mixture of shape, texture, and color. The roughly equal propensity for the monkeys to abort trials by breaking fixation under the dot- and object-memory tasks suggests that the overall attentional or motivational state of the monkeys did not differ during the two tasks. Those factors are therefore not likely to account for differences in correct performance or the task-related differences in neural activities we observed.

8

16 8 0

0

200

400

msec

600

FIG. 6. Population averages of response spike trains (PSTHs) of cells in TEad, TEav, and PRh to the colored dot displayed during the distractor task, pooled over all cells responses. The plots have been smoothed using a Gaussian kernel with ␴ ⫽ 5 ms.


www.jn.org


We checked sensitivities of cells to the colored dot distractors presented during the distractor task. The concern was that responses to distractors might overlap responses to object stimuli. Trials were included which presented the colored dots in the normal task setting (Fig. 3B) but with the “object stimuli” periods held blank. Figure 6C shows the average PSTH in PRh to the colored dots, indicating low sensitivity of this population to the distractor stimuli. TEad and TEav produced similar results (Fig. 6, A and B). Despite population insensitivity to distractors, cells were examined individually and any cell whose average response showed a deviation from baseline activity of ⬎2 spike/s (either positive or negative) was excluded from further analysis regardless if that activity was significant or not. That left 86% (56/65) of the cells in TEad and 76% (137/181) of TEav cells remained, whereas 94% (108/115) of PRh cells remained. From the pool of remaining cells, we included in the analysis only those stimuli that produced activity elevation in the object-memory task that was both significant (P ⬍ 0.05) and large (⬎2 spike/s above spontaneous). For some cells none of the 10 object stimuli met those criteria, so that after this second

chance distractor

obj-memory

1303

1304


A

B

C

FIG. 7. Population averages of response spike trains (PSTHs) in TEad, Teav, and PRh to object stimuli, pooled over all cell responses that met our significance criteria.


Probability distributions of the peak height ratios for the three areas are shown in Fig. 8. The lack of bimodality in these curves supports the idea that the dual peaks seen in the pooled PSTHs are not due to combining responses from two singlepeaked populations of neurons. However, the rightward hump in the PRh curve does leave open the possibility of a subpopulation in that area the responses of which are dominated by the first peak. If one associates the first peak with feedforward sensory processes and the second peak with feedback cognitive processes, then it is possible that the relative heights of the two peaks could depend on whether a particular task were more sensory intensive or more cognitive intensive. The double peak response in PRh may explain the extremely long 66-ms latency difference between TE and PRh reported by Liu and Richmond (2000). Such a latency difference would seem surprising given the dense direct projections between those adjacent areas and might reflect a latency calculation procedure that tended to miss the first peak. These data show essentially no task differences in the PSTHs for TEad or TEav. However, in PRh there is an elevation in activity during the object-memory task compared with the dot-memory distractor task. This activity increase starts after a latency of ⬃80 ms after the start of stimulus response, and seems to coincide with the onset of the second peak. Histograms of average response magnitude over the stimulus duration (Fig. 9A) again indicate no task effect in the two TE areas (mirroring the PSTHs in Fig. 7). These included data only from cells that met our criteria for significance, described above. In contrast to TE, there is a notable elevation in PRh activity during the object-memory task. Figure 9B shows the ratios of responses during the two tasks, calculated from paired comparisons of responses to the same stimulus in the same cell. Response ratios were calculated on a cell-by-cell basis, and then the geometric mean over all cells was calculated. The ratios in TEad and TEav were 1.01 and 1.05, respectively, not significantly different from 1.0, as indicated by their Z-test scores. In PRh, however, the ratio was 1.33 with a multiplicative SE factor of 1.05, significantly elevated above 1.0 (P ⬍ 0.01). A Wilcoxon signed-rank test for paired comparisons again confirmed no significant task differences in TE responses, but a significant difference in PRh (P ⬍ 0.01). There were no significant serial position effects of the response ratios, performing ANOVA as a function of the position of the object image along the stimulus series. The preceding ratios were calculated based on stimuli the responses of which passed our significance criteria during the object-memory task. If instead the selection were based on responses that met the significance criteria during both the object-memory task and the distractor dot-memory task, then the ratios in TEad and TEav were both 1.01, whereas the ratio in PRh was 1.21. Also shown in Fig. 9A is spontaneous activity. Spontaneous activity increased in all cases in the dot-memory distractor task relative to the object-memory task. The increase was largest in PRh (1 spike/s) and also significant in that area (P ⬍ 0.05) under a paired comparison while being insignificant in TEav and TEad. Because spontaneous activity was smaller during the object-memory task, a change in spontaneous activity cannot explain the increase in activity during stimulus presentations for the object-memory task. This small increase in


www.jn.org


round of selection, remaining cells amounted to 80% (52/65) in TEad, 72% (131/181) in TEav, and 31% (36/115) in PRh, a lower percentage in PRh because of low activity levels in that area. After these selection procedures, the total number of object stimuli out of the original 110 that survived in use, over the ensemble of recorded cells, was 87 for TEav, 108 for TEav, and 43 for PRh. Average PSTHs to object stimuli are shown in Fig. 7. Latencies from the pooled PSTHs are 69 ms in TEad, 72 ms in TEav, and 83 ms in PRh. PRh responses are markedly smaller than those in TE. PSTHs in all cortical areas show two peaks of activity. The first peak occurs with a latency of ⬃90 ms in both TE areas and 100 ms in PRh. The second peak occurs at 170 ms in TEad, 150 ms in TEav, and 180 ms in PRh, relative to the time of the first peak. In both TE areas, the second peak is smaller than the first, while the opposite is true in PRh. The shapes of the PRh PSTHs are sensitive to the significance level of the stimulus responses included in constructing them. As the significance level increases (smaller p), the size of the first peak diminishes relative to the second peak.


0.4

object task (spikes/sec)

probability

A

TEad TEav PRh

0.5

0.3 0.2 0.1 0

1/16

1 4 1/4 ratio 1st/2nd peak

16

25

25

20

20

15

15

10

10

5

5

distractor

spont.

8 6 4

0

TEad

0

2

TEav

0

B obj-memory/distractor

spikes/sec

obj-memory

PRh

1.4

1.0

0.6

0.2

TEad TEav PRh

FIG. 9. A: average response magnitudes in different cortical areas under the 2 tasks. Spontaneous levels of activity are shown in gray. Data were pooled over stimulus duration shifted by latency. B: ratios of responses under the object-memory and distractor tasks. Error bars indicate SE. The number of cells in each brain area is the same as indicated in Fig. 7.


60 40 20


B

0

80


20

40

60

80

40

60

80

TEav

60 40 20 0

C

TEad

0 25

20 PRh

20 15 10 5 0

0 10 15 20 25 5 distractor task (spikes/sec)

FIG. 10. Responses of individual cells plotted as average firing rate during the object-memory task vs. average firing rate during the distractor task. In both areas of TE, responses to the two tasks were almost equal, whereas in perirhinal cortex there was greater activity during the memory task. Green points indicate cells whose response ratios for the 2 tasks were different from 1.0 at the P ⫽ 0.01 level of significance.

effect in either TEad or TEav. In PRh, on the other hand, the curve for the object-memory task was shifted upward parallel to that of the distractor task. The parallel nature of the shift indicates that stimulus selectivity has not been changed by the task. Under the object-memory task, repetition of a stimulus within a trial indicated a memory target. Under the distractor task, object stimulus repetition had no significance. We examined the ratio of responses to the second stimulus presentation compared with the first within a trial and determined the significance of the ratio for each cell by means of a t-test, using only cells whose responses were large enough to meet the


www.jn.org


spontaneous activity during the dot-memory task may be a cognitive effect, perhaps reflecting increased resources (mnemonic/attentional) directed at cells during the periods between object stimuli presentations. Alternatively, it could be a sensory effect reflecting responses to the distractor dot stimuli, although this seems improbable as there is no evidence for such sensory responses in Fig. 6. We examined scatterplots that compared firing rates of individual cells during the two tasks (Fig. 10). This shows again that responses in TEav during the two tasks were almost the same, as was the case in TEad, but responses in PRh were elevated during the object-memory task. The linear regression slope in TEav was 0.98 ⫾ 0.01, in TEad, 0.97 ⫾ 0.01, and in PRh, 1.25 ⫾ 0.04. Correlation coefficients between responses for the two tasks were 0.99 in both TEav and TEad, and 0.94 in PRh. Cells whose response ratios were significantly different from 1.0 at the P ⫽ 0,01 level under a Z-test are shown in green in the figure. The proportion of such significant cells was 6% (2/52) in TEav, 4% (5/131) in TEav, and 44% (16/36) in PRh. The 10 object stimuli used during the task comparison were selected for each cell, from a preliminary set of 110, to span a range of stimulus effectiveness from high to low. To determine if stimulus selectivities are affected by the task, we examined response magnitudes over the ordered set of the 10 stimuli (Fig. 11). These curves were constructed by ranking the stimuli for each cell by response magnitude in the object-memory task and then averaging over all cells. As before, there was no task

80

0

FIG. 8. Probability distributions of the relative heights of the 2 peaks seen in the PSTHs (Fig. 7). The ratio of the peaks was determined for each cell that met our significance criteria, and the pdf was then generated by kernel smoothing estimation. The number of units in each brain area is the same as indicated in Fig. 7.

A

1305


25

20

20

15

15

10

10

5

5 2

4

6

8 10

response rank

distractor

8

PRh

7 6 5 4 2

4

6

8

10

response rank

2

4

6

8

10

response rank

FIG. 11. Average response as a function of response rank for the 10 stimuli presented to each cell. Each cell had a different set of 10, selected from 110 objects based on preliminary testing. The parallel shift in PRh indicates that memory processing does not change the stimulus selectivity of cells. The number of units in each brain area is the same as indicated in Fig. 7.

DISCUSSION

Dealing first with the perceptual aspect of perirhinal responses, our data indicate lower stimulus selectivity in PRh compared with TE. Selectivity determinations were based on calculating entropies of response probability density functions (Fig. 4). The pdf of PRh lacks a rightward tail of large responses, making the distribution closer to normal than that of TE. Under an entropy measure of selectivity (as well as a kurtosis measure), that is indicative of lower selectivity. In contrast, Naya et al. (2003) report that they found essentially no difference in stimulus selectivity between PRh and TE. Whether PRh is less selective than TE, as we observed, or equally selective as observed by Naya et al. (2003), the J Neurophysiol • VOL

A TEad

obj-memory distractor

2.0

1.0

0

B

0.25 0.5

1.0

2.0 TEav

2.0

4.0


1.0

0

C

0.25 0.5 PRh

0.6

1.0

2.0

4.0


0.4 0.2 0

0.25 0.5 1.0 2.0 4.0 ratio 2nd/1st presentation

FIG. 12. Repetition suppression and repetition enhancement. These are pdf’s of the ratios of response magnitudes during 1st and 2nd presentations of the same stimulus within the same trial. The pdf’s show that stimulus repetition effects are not significantly different for the 2 tasks in any of the cortical areas. The number of units in each brain area is the same as indicated in Fig. 7.


www.jn.org


criteria we set. For TEav, repetition enhancement was seen in 1% (1/131) of the units during the object-memory task and 3% (4/131) during the distractor task. TEav repetition suppression was seen in 19% (25/131) for the object-memory task and 11% (15/131) for the distractor task. For TEad, enhancement was seen in 2% (1/52) for the object-memory task and 8% (4/52) for the distractor task. Suppression occurred in 11% (6/52) for both tasks. Finally, for PRh, enhancement occurred for 0% (0/36) during the object-memory task and 6% (2/36) during the distractor task, whereas suppression was seen in 6% (2/36) for the object-memory task and 3% (1/29) during the distractor task. These data are broadly consistent with previous reports (Baylis and Rolls 1987; Fahy et al. 1993; Miller and Desimone 1994; Miller et al. 1993) in which repetition effects were observed in only small fractions of the recorded cells. The pdf’s over all response ratios for units that met the response criteria described in the preceding text are plotted in Fig. 12. In no case is the distribution average significantly different from 1.0. The most notable difference between TE and PRh in these plots is the greater variance of the PRh distribution. That may be due to the lower activity in PRh, leading to greater uncertainty in estimating firing rates within a fixed time window. The pdf’s for the two tasks are not significantly different in any brain area, under a KolmogorovSmirnov test, nor are the median response ratios significantly different for the two tasks under a Wilcoxon signed rank paired comparison. Furthermore, there were no significant serial position effects on the magnitude of the ratio, performing ANOVA as a function of the number of items separating the first and second presentation of the stimulus image. Thus we see no influence of the memory task on repetition effects.

significant point is that PRh is not more selective than TE. Thus these data seemingly fail to support the perceptualmnemonic model of PRh function (Buckley et al. 2001; Bussey et al. 2005; Murray and Bussey 1999), which holds that PRh plays a perceptual role in vision by forming more complex conjunctions of features than TE in addition to its mnemonic role. If PRh was forming more complex conjunctions of features, one would expect it to show greater visual selectivity than TE. Nevertheless, we shall offer two suggestions how observation of lower selectivity in PRh could still be compatible with the perceptual-mnemonic model. The first possibility is that the measured selectivity in PRh has been underestimated because of the limited size of the stimulus set (110 objects). To see how this might happen, imagine how the response statistics of a very highly selective cell might appear if the (rare) effective stimuli for that cell weren’t included in the stimulus set. Lacking any large responses, the cell’s pdf would likely be quasi-normally distrib-

probability density

spikes/sec

25

TEav

probability density

obj-memory

TEad

probability density

1306



The designs of the tasks used here do not allow us to associate the elevated PRh activity with specific stages within the working-memory process. Unlike the delayed match to sample task, in which encoding, maintenance and retrieval are separated in time, in our serial recognition task all three processes occur concurrently, as the sequence of stimulus items are presented within each trial. In compensation for this loss of temporal separability among working-memory subprocesses, our task offered a more complex memory load that increased the contrast between the memory and nonmemory (distractor) conditions. Working-memory effects have been detected in a number of earlier visually responsive areas (V1, V4, MT), using simple synthetic stimuli such as spots, bars, and stripes (Bisley et al. 2004; Haenny et al. 1988; Maunsell et al. 1991; Motter 1994b; Pasternak and Greenlee 2005; Supe`r et al. 2001). Given those observations, the question arises why the memory-related effects we saw were specific to PRh and did not include the earlier area TE. A possible explanation arises when one considers psychophysical evidence that for complex stimuli, visual working-memory stores integrated objects rather than individual features (Lee and Chun 2001; Luck and Vogel 1997). It was suggested in the preceding text that PRh is the site where feature clusters are integrated into more holistic object representations, whereas TE still maintains representations at the feature level. If that were so, then working-memory effects using more naturalistic complex objects might then be preferentially associated with PRh. Response PSTHs in all cortical areas had two peaks. The first peak most likely represents feedforward flow of responses up from the retina. The second peak appears around 150 –180 ms after the first peak and may represent recurrent input. Elevated PRh activity during the object-memory task is associated with the second peak (Fig. 7C), starting ⬃80 ms after the onset of the stimulus response. The latency between the onset of the stimulus response and the onset of the putative memoryrelated activity suggests the possibility that such activity is dependent on top-down feedback. Given the close association between working memory and lateral prefrontal cortex that was outlined earlier, a likely source of this feedback would be prefrontal cortex, although these data cannot exclude other sources. We therefore view these data as consistent with models of working memory postulating that object representations within posterior cortices (forming the contents of working memory) are controlled or activated by feedback signals from prefrontal cortex (Curtis and D’Esposito 2003; Petrides 1994, 2000; Postle 2006; Postle et al. 1999). This prefrontal cortex signal may serve to tag particular posterior visual representations as behaviorally significant for the task at hand. Thus under the interpretation here, as the monkey switches back and forth between the object-memory task and the dotmemory distractor task (both basically the same task, but with different stimuli), the memory-related feedback signal from prefrontal cortex switches back and forth between the neural populations forming the representations of the two sets of stimuli (Fig. 13). The cells we recorded from were selected to be responsive to just one stimulus class (the complex stimuli of the object-memory task) and not the other (the simple patterns of the dot-memory-distractor task). Therefore we observed the memory related signal turning on and off depending on which neural representation was the target of prefrontal control.


www.jn.org


uted around some low firing rate rather similar to what we see in the PRh data, giving the false appearance of low selectivity. Including just one effective stimulus dramatically changes the selectivity calculations, but it is unlikely that we would by chance include something approximating that image within a small stimulus set. Because of this potential problem, the measurement here of lower selectivity in PRh relative to TE is open to question. It may take thousands of object stimuli to properly characterize neurons whose receptive fields have this level of complexity. Alternatively, the lower selectivity observed here in PRh relative to TE may not be a sampling artifact, as suggested in the preceding text, but rather may reflect the possibility that PRh is involved in visual categorization to a greater degree than TE. Object categorization requires cells to generalize across stimuli and therefore be less selective, contrary to the high selectivity required for object identification. Cells in inferotemporal cortex are known to be involved in visual categorization (Freedman et al. 2003; Hung et al. 2005; Sigala and Logothetis 2002; Vogels 1999), although that aspect of their properties has not been compared between PRh and TE. Therefore under this view, the advance from TE to PRh, rather than continuing the pattern of increasing selectivity as one ascends in visual pathways, may mark the transition to a qualitatively distinct and more abstract functional mode in which object-based rather than feature-based representations predominate. Studies indicating that PRh is involved in processing more complex visual stimuli than TE (Buckley et al. 2001; Bussey et al. 2002, 2003, 2005; Eacott et al. 1994; Murray and Bussey 1999; Tyler et al. 2004), together with the observation here that PRh neurons are visually less selective than those of TE, are consistent with the idea that PRh is preferentially involved in object-based categorization rather than feature-based processing. Moving from perception to memory, there was a significant increase in activity in perirhinal cells during the object-memory task compared with the distractor task (Figs. 7, 9, and 10). Similar task-related changes in responses were not seen in the adjacent visual areas that project directly to PRh, areas TEad and TEav. As our task involved the use of a small set (10) of highly familiar stimuli, in which every member of the set appeared repeatedly within a short period of time (several times per minute), solving the task would invoke predominantly working-memory mechanisms rather than long-term mechanisms. That being the case, we would interpret the increased PRh activity during the object-memory task as reflecting some aspect of the neural activity required when visual objects are being processed by working memory. Although we observed a task-specific increase in neural responses for objects being held in working memory, we did not observe any task-specific changes in repetition effects, either repetition enhancement or repetition suppression. Thus there is no basis in these data for associating such effects with processing during our working-memory task. A dissociation between performance during a working-memory DMS task and repetition suppression has previously been shown by Miller and Desimone (1993). During systemic application of a cholinergic blocker, performance in the task was disrupted, but there was no change in repetition suppression in PRh. As yet, this test has not been applied to repetition enhancement or other putative working memory correlates.

1307

1308

S. R. LEHKY AND K. TANAKA Object-memory task

response and the onset of the memory-related activation, consistent with a top-down feedback source of the activation. The nature of this feedback activation during working memory may overlap extensively with feedback activation associated with attentional selection. These results support models of visual working memory in which prefrontal cortex exerts a controlling influence on visual representations held within posterior cortices.

Dot-memory distractor task

Prefrontal Cortex

ACKNOWLEDGMENTS

Perirhinal Cortex

We thank K. Saleem for assistance with histology, J. Maunsell for providing data-acquisition software as well as M. Mishkin, A. Sereno, and L. Chen for comments on the manuscript.

Neural representation of object stimulus

GRANTS

Neural representation of dot stimulus

As there is a close connection between working memory and attention (Awh and Jonides 2001; Awh et al. 2006; Barnes et al. 2001; de Fockert et al. 2001; Desimone 1998), the mechanisms underlying the memory-related activation we observed in PRh could extensively overlap with mechanisms involved in attentional selection. The changes in PRh activity cannot be ascribed to shifts in spatial attention (for example, see Moran and Desimone 1985), as the stimuli used in the object-memory and distractor tasks were in overlapping locations. On the other hand, it is more difficult to exclude the possibility of a contribution from object attention, if indeed working memory for objects and selective attention for objects are independent mechanisms. It is well established that attention can select features or objects nonspatially as demonstrated psychophysically (Awh et al. 2001; Duncan 1984; Lee and Chun 2001; Vecera and Farah 1994), neurophysiologically (Chelazzi et al. 1993; Hayden and Gallant 2005; McAdams and Maunsell 2000; Motter 1994a; Sereno and Amador 2006; Treue and Trujillo 1999) and by brain imaging (Corbetta et al. 1991; Liu et al. 2003; O’Craven et al. 1999; Serences et al. 2004). Also, given that relevant and nonrelevant stimuli in our task occurred at predictable times, another possible contributor to our results is temporal attention, the ability to direct attention to particular points in time. Although not as extensively studied as object attention, temporal attention has been demonstrated with human brain imaging (Coull and Nobre 1998; Griffin et al. 2002; Nobre 2001) as well as monkey single-cell recordings (Ghose and Maunsell 2002). Again suggestive of an overlap between working memory and attentional mechanisms, an analogy may be drawn between what appears to be feedback activation of PRh during our working-memory task and feedback activation of various posterior cortices during attention (reviewed by Pessoa et al. 2003). To summarize, in a paired comparison between two tasks we have observed activity in PRh associated with the requirement for keeping complex objects in visual working memory. The activity did not extend into the adjacent visual cortex, area TE. There was a long latency between the onset of the stimulus J Neurophysiol • VOL

This research was partly supported by a Grant-in-Aid for Scientific Research on Priority Areas 17022047 from Japanese Ministry of Education, Sports, Science and Technology. REFERENCES

Alvarado MC, Bachevalier J. Comparison of the effects of damage to the perirhinal and parahippocampal cortex on transverse patterning and location memory in rhesus macaques. J Neurosci 25: 1599 –1609, 2005. Atkinson RC, Shiffrin RM. The control of short-term memory. Sci Am 225: 82–90, 1971. Atkinson RC, Shiffrin RM. Human memory: A proposed system and its control processes. In: The Psychology of Learning and Motivation: Advances in Research and Theory, edited by Spence KW and Spence JT. San Diego, CA: Academic, 1968, vol. 2, p. 89 –195. Awh E, Dhaliwal H, Christensen S, Matsukara M. Evidence for two components of object-based selection. Psychol Sci 12: 329 –334, 2001. Awh E, Jonides J. Overlapping mechanisms of attention and spatial working memory. Trends Cogn Sci 5: 119 –126, 2001. Awh E, Vogel EK, and Oh S-H. Interactions between attention and working memory. Neuroscience 139: 201–208, 2006. Barnes LL, Nelson JK, Reuter-Lorenz PA. Object-based attention and object working memory: overlapping processes revealed by selective interference effects in humans. Prog Brain Res 134: 471– 481, 2001. Baxter MG, Murray EA. Opposite relationship of hippocampal and rhinal cortex damage to delayed nonmatching-to-sample deficits in monkeys. Hippocampus 11: 61–71, 2001. Baylis GC, Rolls ET. Responses of neurons in the inferior temporal cortex in short term and serial recognition memory tasks. Exp Brain Res 65: 614 – 622, 1987. Bisley JW, Zaksas D, Droll JA, Pasternak T. Activity of neurons in cortical area MT during a memory for motion task. J Neurophysiol 91: 286 –300, 2004. Brown MW, Aggleton JP. Recognition memory: what are the roles of the perirhinal cortex and hippocampus? Nat Rev Neurosci 2: 51– 61, 2001. Buckley MJ, Booth MCA, Rolls ET, Gaffan D. Selective perceptual impairments after perirhinal cortex ablation. J Neurosci 21: 9824 –9836, 2001. Buckley MJ, Gaffan D. Impairment of visual object-discrimination learning after perirhinal cortex ablation. Behav Neurosci 111: 467– 475, 1997. Buckley MJ, Gaffan D, Murray EA. Functional double dissociation between two inferior temporal cortical areas: perirhinal cortex versus middle temporal gyrus. J Neurophysiol 77: 587–598, 1997. Buffalo EA, Ramus S, Clark R, Teng E, Squire LR, Zola SM. Dissociation between the effects of damage to perirhinal cortex and area TE. Learn Mem 6: 572–599, 1999. Bussey TJ, Saksida LM, Murray EA. Perirhinal cortex resolves feature ambiguity in complex visual discriminations. Eur J Neurosci 15: 365–374, 2002. Bussey TJ, Saksida LM, Murray EA. Impairments in visual discrimination after perirhinal cortex lesions: testing “declarative” vs. “perceptual-mnemonic” views of perirhinal cortex function. Eur J Neurosci 17: 649 – 660, 2003. Bussey TJ, Saksida LM, Murray EA. The perceptual-mnemonic/feature conjunction model of perirhinal cortex function. Q J Exp Psychol 58B: 269 –282, 2005. Castro CA, Larsen T. Primacy and recency effects in nonhuman primates. J Exp Psychol Anim Behav Process 18: 335–340, 1992.


www.jn.org


FIG. 13. Model of prefrontal-perirhinal interactions during visual working memory. During the object-memory task, feedback from prefrontal cortex activated population-coded neural representations of the object stimuli in perirhinal cortex. During the dot-memory task, the prefrontal feedback signal switched to neural representations of the dot stimuli. As we recorded only from neurons that were part of the object representations, the prefrontal signal appeared to turn on and off as it was switched back and forth between the 2 neuronal populations.



Hayden BY, Gallant JL. Time course of attention reveals different mechanisms for spatial and feature-based attention in area V4. Neuron 47: 637– 643, 2005. Holscher C, Rolls ET. Perirhinal cortex neuronal activity is actively related to working memory in the macaque. Neural Plas 9: 41–51, 2002. Hung CP, Kreiman G, Poggio T, DiCarlo JJ. Fast readout of object identity from macaque inferior temporal cortex. Science 310: 863– 866, 2005. Jacobsen CF. The functions of the frontal association areas in monkeys. Comp Psychol Monogr 13: 1– 60, 1936. Kondo H, Komatsu H. Suppression on neuronal responses by a metacontrast masking stimulus in monkey V4. Neurosci Res 36: 27–33, 2000. Kovacs G, Vogels R, Orban GA. Cortical correlate of pattern backward masking. Proc Natl Acad Sci USA 92: 5587–5591, 1995. Kubota K, Niki H. Prefrontal cortical unit activity and delayed alternation performance in monkeys. J Neurophysiol 34: 337–347, 1971. Lavenex P, Suzuki WA, Amaral DG. Perirhinal and parahippocampal cortices of the macaque monkey: projections to the neocortex. J Comp Neurol 447: 394 – 420, 2002. Lebedev MA, Messinger A, Kralik JD, Wise SP. Representation of attended versus remembered locations in prefrontal cortex. PLOS Biol 2: e365, 2004. Lee ACH, Bussey TJ, Murray EA, Saksida LM, Epstein RA, Kapur N, Hodges JR, Graham KS. Perceptual deficits in amnesia: challenging the medial temporal lobe “mnemonic” view. Neuropsychologia 43: 1–11, 2005. Lee D, Chun MM. What are the units of visual short-term memory, objects or spatial locations? Percept Psychophys 63: 253–257, 2001. Lehky SR. Bayesian estimation of stimulus responses in Poisson spike trains. Neural Comput 16: 1325–1343, 2004. Lehky SR, Sejnowski TJ, Desimone R. Selectivity and sparseness in the responses of striate complex cells. Vision Res 45: 57–73, 2005. Levy DA, Shrager Y, Squire LR. Intact visual discrimination of complex and feature-ambiguous stimuli in the absence of perirhinal cortex. Learn Mem 12: 61– 66, 2005. Liu T, Slotnick SD, Serences JT, Yantis S. Cortical mechanisms of featurebased attentional control. Cereb Cortex 13: 1334 –1443, 2003. Liu Z, Richmond BJ. Response differences in monkey TE and perirhinal cortex: stimulus association related to reward schedules. J Neurophysiol 83: 1677–1692, 2000. Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature 390: 279 –281, 1997. Macknik SL, Livingstone MS. Neuronal correlates of visibility and invisibility in the primate visual system. Nat Neurosci 1: 144 –149, 1998. Maunsell JH, Sclar G, Nealey TA, DePriest DD. Extraretinal representations in area V4 in the macaque monkey. Vis Neurosci 7: 561–573, 1991. McAdams CJ, Maunsell JH. Attention to both space and feature modulates neuronal responses in macaque area V4. J Neurophysiol 83: 1751–1755, 2000. Meunier M, Bachevalier J, Mishkin M, Murray EA. Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys. J Neurosci 13: 5418 –5432, 1993. Miller EK, Desimone R. Scopolamine affects short-term memory but not inferior temporal neurons. Neuroreport 4: 81– 84, 1993. Miller EK, Desimone R. Parallel neuronal mechanisms for short-term memory. Science 263: 520 –522, 1994. Miller EK, Erickson CA, Desimone R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J Neurosci 16: 5154 –5167, 1996. Miller EK, Li L, Desimone R. A neural mechanism for working and recognition memory in inferior temporal cortex. Science 254: 1377–1379, 1991. Miller EK, Li L, Desimone R. Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J Neurosci 13: 1460 – 1478, 1993. Mishkin M. Effects of small frontal lesions on delayed alternation in monkeys. J Neurophysiol 20: 615– 622, 1957. Miyashita Y. Inferior temporal cortex: where visual perception meets memory. Annu Rev Neurosci 16: 245–263, 1993. Miyashita Y, Chang HS. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331: 68 –70, 1988. Mogami T, Tanaka K. Reward association affects neuronal responses to visual stimuli in macaque te and perirhinal cortices. J Neurosci 26: 6761– 6770, 2006. Moran J, Desimone R. Selective attention gates visual processing in the extrastriate cortex. Science 229: 782–784, 1985.


www.jn.org


Chelazzi L, Miller EK, Duncan J, Desimone R. A neural basis for visual search in inferior temporal cortex. Nature 363: 345–347, 1993. Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE. Selective and divided attention during visual discrimination of shape, color, and speed: functional anatomy by positron emission tomography. J Neurosci 11: 2383–2402, 1991. Coull JT, Nobre AC. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J Neurosci 18: 7426 –7435, 1998. Courtney SM. Attention and cognitive control as emergent properties of information representation in working memory. Cogn Affect Behav Neurosci 4: 501–516, 2004. Curtis CE, D’Esposito M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn Sci 7: 415– 423, 2003. Davachi L, Goldman-Rakic P. Primate rhinal cortex participates in both visual recognition and working memory tasks: functional mapping with 2-DG. J Neurophysiol 85: 2590 –2601, 2001. de Fockert JW, Rees G, Frith CD, Lavie N. The role of working memory in visual selective attention. Science 291: 1803–1806, 2001. Desimone R. Stimulus-selective properties of inferior temporal neurons in the macaque. J Neurosci 4: 2051–2062, 1984. Desimone R. Neural mechanisms for visual memory and their role in attention. Proc Natl Acad Sci USA 93: 13494 –13499, 1996. Desimone R. Visual attention mediated by biased competition in extrastriate visual cortex. Philos Trans R Soc Lond B Biol Sci 353: 1245–1255, 1998. Duncan J. Selective attention and the organization of visual information. J Exp Psychol Gen 113: 501–517, 1984. Eacott MJ, Gaffan D, Murray EA. Preserved recognition memory for small sets, and impaired stimulus identification for large sets, following rhinal cortex ablations in monkeys. Eur J Neurosci 6: 1466 –1478, 1994. Eskandar EN, Richmond BJ, Optican LM. Role of inferior temporal neurons in visual memory. I. Temporal encoding of information about visual images, recalled images, and behavioral context. J Neurophysiol 68: 1277– 1295, 1992. Fahy FL, Riches IP, Brown MW. Neuronal activity related to visual recognition memory: long-term memory and the encoding of recency and familiarity information in the primate anterior and medial inferior temporal and rhinal cortex. Exp Brain Res 96: 457– 472, 1993. Freedman DJ, Riesenhuber M, Poggio T, Miller EK. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J Neurosci 23: 5325–5246, 2003. Fuster JM. Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. J Neurophysiol 36: 61–78, 1973. Fuster JM. Inferotemporal units in selective visual attention and short-term memory. J Neurophysiol 64: 681– 697, 1990. Fuster JM, Alexander GE. Neuron activity related to short term memory. Science 173: 652– 654, 1971. Fuster JM, Jervey JP. Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli. Science 212: 952–955, 1981. Gaffan D. Against memory systems. Philos Trans R Soc Lond B Biol Sci 357: 1111–1121, 2002. Ghose GM, Maunsell JH. Attentional modulation in visual cortex depends on task timing. Nature 419: 616 – 620, 2002. Glanzer M, Cunitz AR. Two storage mechanisms in free recall. J Verbal Learn Verbal Behav 5: 351–360, 1966. Goldman-Rakic P. Circuitry of the prefrontal cortex and the regulation of behavior by representational memory. In: Handbook of Physiology.The Nervous System. Higher Functions of the Brain., edited by Plum F. Bethesda: Am. Physiol. Soc., 1987, sect. 1, vol. V, part 1, p. 373– 417. Griffin IC, Miniussi C, Nobre AC. Multiple mechanisms of selective attention: differential modulation of stimulus processing by attention to space or time. Neuropsychologia 40: 2325–2340, 2002. Haenny PE, Maunsell JH, Schiller PH. State dependent activity in monkey visual cortex. II. Retinal and extraretinal factors in V4. Exp Brain Res 69: 245–259, 1988. Hampton RR. Monkey perirhinal cortex is critical for visual memory, but not for visual perception: reexamination of the behavioural evidence from monkeys. Q J Exp Psychol 58B: 283–299, 2005. Hampton RR, Murray EA. Learning of discriminations is impaired, but generalization to altered views is intact, in monkeys (Macaca mulatta) with perirhinal cortex removal. Behav Neurosci 116: 363–377, 2002.

1309

1310



Sands SF, Wright AA. Primate memory: retention of serial list items by a rhesus monkey. Science 209: 938 –940, 1980. Serences JT, Schwarzbach J, Courtney SM, Golay X, Yantis S. Control of object-based attention in human cortex. Cereb Cortex 14: 1346 –1357, 2004. Sereno AB, Amador SC. Attention and memory related responses of neurons in the lateral intraparietal area during spatial and shape delayed match-tosample tasks. J Neurophysiol 95: 1078 –1098, 2006. Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415: 318 –320, 2002. Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annu Rev Neurosci 24: 1193–1216, 2001. Sobotka S, Diltz MD, Ringo JL. Can delay-period activity explain working memory? J Neurophysiol 93: 128 –136, 2005. Squire LR, Stark CEL, Clark RE. The medial temporal lobe. Annu Rev Neurosci 27: 279 –306, 2004. Supe`r H, Spekreijse H, Lamme VAF. A neural correlate of working memory in the monkey primary visual cortex. Science 293: 120 –124, 2001. Suzuki WA, Amaral DG. Perirhinal and parahippocampal cortices of the macaque monkey: cortical afferents. J Comp Neurol 350: 497–533, 1994. Takeda M, Naya Y, Fujimichi R, Takeuchi D, Miyashita Y. Active maintenance of associative mnemonic signal in monkey inferior temporal cortex. Neuron 48: 839 – 848, 2005. Tamura H, Tanaka K. Visual response properties of cells in the ventral and dorsal parts of the macaque inferotemporal cortex. Cereb Cortex 11: 384 –399, 2001. Tanaka K. Inferotemporal cortex and object vision. Annu Rev Neurosci 19: 109 –139, 1996. Tang Y, Mishkin M, Aigner TG. Effects of muscarinic blockade in perirhinal cortex during visual recognition. Proc Natl Acad Sci USA 94: 12667–12669, 1997. Treue S, Trujillo JC. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399: 575–579, 1999. Turchi J, Saunders RC, Mishkin M. Effects of cholinergic deafferentation of the rhinal cortex on visual recognition memory in monkeys. Proc Natl Acad Sci USA 102: 2158 –2161, 2005. Tyler LK, Stamatakis EA, Bright P, Acres K, Abdallah S, Rodd JM, Moss HE. Processing objects at different levels of specificity. J Cogn Neurosci 16: 351–362, 2004. Ungerleider LG, Gaffan D, Pelak VS. Projections from inferior temporal cortex to prefrontal cortex via the uncinate fascicle in rhesus monkeys. Exp Brain Res 76: 473– 484, 1989. Van Hoesen GW. The parahippocampal gyrus: new observations regarding its cortical connections in the monkey. Trends Neurosci 5: 345–350, 1982. Vecera SP, Farah MJ. Does visual attention select objects or locations? J Exp Psychol Gen 123: 146 –160, 1994. Vogels R. Categorization of complex visual images by rhesus monkeys. II. Single-cell study. Eur J Neurosci 11: 1239 –1255, 1999. Webster MJ, Bachevalier J, Ungerleider LG. Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys. Cereb Cortex 4: 470 – 483, 1994. Wright AA, Santiago HC, Sands SF, Kendrick DF, Cook RG. Memory processing of serial lists by pigeons, monkeys, and people. Science 229: 287–289, 1985. Yakovlev V, Bernacchia A, Orlov T, Hochstein S, Amit D. Multi-item working memory–a behavioral study. Cereb Cortex 15: 602– 615, 2004. Zola-Morgan S, Squire LR, Amaral DG, Suzuki WA. Lesions of perirhinal and parahippocampal cortex that spare the amygdala and hippocampal formation produce severe memory impairment. J Neurosci 9: 4335– 4370, 1989.


www.jn.org


Motter BC. Neural correlates of attentive selection for color or luminance in extrastriate area V4. J Neurosci 14: 2178 –2189, 1994a. Motter BC. Neural correlates of feature selective memory and pop-out in extrastriate area V4. J Neurosci 14: 2190 –2199, 1994b. Murdock BB. The serial position effect in free recall. J Exp Psychol 64: 482– 488, 1962. Murray EA, Bussey TJ. Perceptual–mnemonic functions of the perirhinal cortex. Trends Cogn Sci 3: 142–151, 1999. Murray EA, Mishkin M. Object recognition and location memory in monkeys with excitotoxic lesions of the amygdala and hippocampus. J Neurosci 18: 6568 – 6582, 1998. Nakamura K, Kubota K. Mnemonic firing of neurons in the monkey temporal pole during a visual recognition memory task. J Neurophysiol 74: 162–178, 1995. Naya Y, Yoshida M, Miyashita Y. Backward spreading of memory-retrieval signal in the primate temporal cortex. Science 291: 661– 664, 2001. Naya Y, Yoshida M, Miyashita Y. Forward processing of long-term associative memory in monkey inferotemporal cortex. J Neurosci 23: 2861– 2871, 2003. Niki H. Prefrontal unit activity during delayed alternation in the monkey. I. Relation to direction of response. Brain Res 68: 185–196, 1974. Nobre AC. Orienting attention to instants in time. Neuropsychologia 39: 1317–1328, 2001. O’Craven KM, Downing PE, Kanwisher N. fMRI evidence for objects as the units of attentional selection. 401: 584 –587, 1999. Pasternak T, Greenlee MW. Working memory in primate sensory systems. Nat Rev Neurosci 6: 97–107, 2005. Pessoa L, Kastner S, Ungerleider LG. Neuroimaging studies of attention: from modulation of sensory processing to top-down control. J Neurosci 23: 3990 –3998, 2003. Petrides M. Frontal lobes and working memory: evidence from investigations of the effects of cortical excisions in nonhuman primates. In: Handbook of Neuropsychology, edited by Boller F and Grafman J. New York: Elsevier, 1994, p. 59 – 82. Petrides M. Dissociable roles of mid-dorsolateral prefrontal and anterior inferotemporal cortex in visual working memory. J Neurosci 20: 7496 – 7503, 2000. Postle BR. Working memory as an emergent property of the mind and brain. Neuroscience 139: 23–38, 2006. Postle BR, Berger JS, D’Esposito M. Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. Proc Natl Acad Sci USA 96: 12959 –12964, 1999. Raymond JE, Shapiro KI, Arnell KM. Temporary suppression of visual processing in an RSVP task: an attentional blink? J Exp Psychol Hum Percept Perform 18: 849 – 860, 1992. Rempel-Clower NL, Barbas H. The laminar pattern of connections between prefrontal and anterior temporal cortices in the Rhesus monkey is related to cortical structure and function. Cereb Cortex 10: 851– 865, 2000. Rieke F, Warlan D, de Ruyter van Steveninck R, Bialek W. Spikes: Exploring the Neural Code. Cambridge, MA: MIT Press, 1997. Rolls ET, Tovee MJ. Processing speed in the cerebral cortex and the neurophysiology of visual masking. Proc R Soc Lond B Biol Sci 257: 9 –15, 1994. Rolls ET, Tovee MJ. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. J Neurophysiol 73: 713–726, 1995. Rolls ET, Tovee MJ, Panzeri S. The neurophysiology of backward visual masking: information analysis. J Cogn Neurosci 11: 300 –311, 1999. Saleem KS, Tanaka K. Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal cortices in the macaque monkey. J Neurosci 16: 4757– 4775, 1996.