Dissecting Neural Responses to Temporal Prediction

Cerebral Cortex Advance Access published November 11, 2014 Cerebral Cortex, 2014, 1–12 doi: 10.1093/cercor/bhu269 Original Article

ORIGINAL ARTICLE

Dissecting Neural Responses to Temporal Prediction, Attention, and Memory: Effects of Reward Learning and Interoception on Time Perception 1

National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD 20892, USA, and 2National Institute on Drug Abuse, Bethesda, MD 20892, USA

Address correspondence to Dardo Tomasi, PhD, 10 Center Dr, Rm B2L304, Bethesda, MD 20892-1013, USA. Email: [email protected]

Abstract Temporal prediction (TP) is needed to anticipate future events and is essential for survival. Our sense of time is modulated by emotional and interoceptive (corporal) states that are hypothesized to rely on a dopamine (DA)-modulated “internal clock” in the basal ganglia. However, the neurobiological substrates for TP in the human brain have not been identified. We tested the hypothesis that TP involves DA striato-cortical pathways, and that accurate responses are reinforcing in themselves and activate the nucleus accumbens (NAc). Functional magnetic resonance imaging revealed the involvement of the NAc and anterior insula in the temporal precision of the responses, and of the ventral tegmental area in error processing. Moreover, NAc showed higher activation for successful than for unsuccessful trials, indicating that accurate TP per se is rewarding. Inasmuch as activation of the NAc is associated with drug-induced addictive behaviors, its activation by accurate TP could help explain why video games that rely on TP can trigger compulsive behaviors. Key words: dopamine, fMRI, reward, timekeeping

Introduction Adaptive organisms must predict future events in time scales that are useful for survival. Temporal prediction (TP), the ability to estimate the short-term evolution of future events, is continuously shaped by reward/aversion learning processes as a function of sensory information (Schultz et al. 1997; Nobre et al. 2007) as well as emotional and interoceptive states (Pollatos et al. 2014). Our sense of time and the temporal relationships between previous events help us to anticipate future events and to make faster responses (Jazayeri and Shadlen 2010). Multiple processes seem to determine our subjective perception of time in the range of hundreds of milliseconds to several seconds, including interoceptive signals in the insular cortex and an “internal clock” or internal timekeeper, presumably located in the basal ganglia and modulated by DA (Buhusi 2003; Coull et al. 2011; Merchant et al. 2013). Specifically, it is hypothesized that interval timing is encoded by coincident activity of cortical neurons that

are detected by striatal neurons and conveyed through the thalamus into the cortex (Matell and Meck 2004). Since dopamine (DA) regulates striato-thalamo-cortical circuits, this could explain how DA signaling could modulate time perception (Meck 1986, 2006; Drew et al. 2007; Wiener et al. 2014). However, the neural bases of TP, including its cortical representations, are still poorly understood. Behavioral and neuroimaging studies have shown that the sense of time is slower in elders and in psychiatric disorders of abnormal basal ganglia function (schizophrenia, Parkinson’s disease, attention deficit hyperactivity disorder, and addiction; Allman and Meck 2012; Allman et al. 2014), which is consistent with DA’s role in timekeeping operations (Bäckman et al. 2006; Dreher et al. 2008). Similarly, pharmacological studies in animals also support DA’s role in timekeeping operations in the striatum (Maricq and Church 1983; Meck 1986; Oprisan and Buhusi 2011). These include the effects of drugs of abuse such as stimulants,

Published by Oxford University Press 2014. This work is written by (a) US Government employee(s) and is in the public domain in the US.

1

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Dardo Tomasi1, Gene-Jack Wang1, Yana Studentsova1 and Nora D. Volkow1,2

2

| Cerebral Cortex

Materials and Methods Subjects The 36 healthy participants (age: 27 ± 6 years, mean ± SD; 34 right handed and 2 left handed; 18 females) in this study were recruited from advertisements in local newspapers. Written

informed consent was obtained from all participants prior to the study. Participants were excluded from the study if they had 1) history of major psychiatric or neurological disease; 2) medical conditions that may alter cerebral function (i.e., cardiovascular, endocrinological, oncological, or autoimmune diseases), 3) head trauma with loss of consciousness for >30 min, or 4) use prescribed psychoactive medications. The participants were instructed to discontinue any over the counter medication 2 weeks prior to the study. Food and beverages (except for water) were discontinued at least 4 h prior and cigarettes for at least 2 h prior to the study. Females were scanned in their midfollicular phase. The study was approved by the Committee on Research in Human Subjects at Stony Brook University.

Prediction Learning Paradigm During functional scans, participants engaged in 4 consecutive tasks (Supplementary Fig. 1A) involving SM coupling, TOP, spatial WM, and SA. The tasks had identical duration (4 min), number of trials (20), and intertrial interval (ITI = 12 s; 2 s jittering). Button press responses were recorded to determine reaction time (RT; average time difference between targets and responses) and performance accuracy ( percentage of successful events relative to the total number of events). In addition, the subject’s responses were used for general linear modeling (GLM) of brain activation responses during successful prediction as well as during prediction errors. The next paragraphs describe the trials for each of the 4 tasks in this paradigm. The SM task involves visual perception and measures the RT required for the subjects to respond to the presence of a target, an open circle covering 10% and 12% of the horizontal and vertical visual fields (Supplementary Fig. 1B). A static white fixation cross was shown at the center of the black screen during 96% of the ITI. During each trial, the target was randomly flashed for 200 ms at 1 of the 4 corners of the screen in the peripheral visual field. The subjects’ task was to respond to the appearance of the target by pushing an in-house MRI-compatible response button with their right index finger as quickly as possible upon target presentation. The 600-ms response window that followed the target was used to classify a button press response as successful (RT ≤ 600 ms) or unsuccessful (RT > 600 ms). After an expectation period of 1.3 s, an outcome message, “$” for a hit or “X” for a miss, briefly (500 ms) replaced the fixation cross at the center of the visual field as a feedback, to inform the subjects about their accuracy during the trial; after the outcome only the fixation cross was displayed for 9–11 s, until the onset of the next target. The TOP task is based on the observation that subsequent TPs are generally correlated in a way that incremental learning allows for dynamic adjustment of the prediction strategy leading to more accurate predictions in the near future. The subject’s time predicting ability was optimized using temporal learning, a reinforcement learning model assuming that subsequent predictions are increasingly accurate. Each trial was initiated by a 1-s long cue displaying nonrepeating natural numbers (1–4), randomly, at the 4 quadrants of the central visual field, which revealed the spatial sequence of future events (circles subsequently displayed at the corners of the screen; Fig. 1 and Supplementary Fig. 1B). During the 3-step TD learning module, which started 200 ms after the cue, white circles were subsequently flashed for 200 ms every 1100 ms, one at each of the 4 corners of the visual field, sequentially as indicated by the cue. The subjects were instructed to press a button within a brief 300-ms response window at the onset of the last circle of the cue (i.e., to predict the onset of the target). The subjects were informed that they would not be able

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

which lead to overestimations of time duration (Cheng et al. 2006; Wittmann et al. 2007), or alcohol and cannabinoids, which lead to underestimations of time duration (Tinklenberg et al. 1976). Accumulating evidence from human and nonhuman primates suggests that DA neurons in the midbrain code reward prediction and support reward learning (Schultz 1998). Temporal difference (TD) learning (Sutton and Barto 1981; Sutton 1988), a differential learning method, has been proposed to explain reward-depending learning (Schultz et al. 1997). Specifically, the firing rate of midbrain neurons appears to mimic the TD error function (Schultz et al. 1997), and functional magnetic resonance imaging (fMRI) studies on appetitive (taste) and aversive ( pain) conditioning showed that reward and aversive signals in the ventral striatum and anterior insula seem to mimic the sequential learning signals predicted by TD models (McClure et al. 2003; O’Doherty et al. 2003; Seymour et al. 2004). Successful TP requires accurate timekeeping (supported by TD reward learning) as well as attention/vigilance and working memory (WM); processes that are coupled because reward learning is associated with attention shifts (Maunsell 2004) supported by WM (Knudsen 2007). Thus, it is important to identify what neural system mediates TP from those that mediate attention/ vigilance and WM, processes that are concurrently active during a given TP task. To understand the neurobiology of TP, we asked subjects to predict the onset of circles periodically displayed in the screen while corresponding brain activity was assessed with 4 T fMRI. Along with dissecting TD learning and outcome components from the target onset prediction (TOP) task, control tasks of sensorimotor (SM), spatial attention (SA), and WM contributions to the TOP task were also studied to determine whether the TP activation signature could be distinguished from the control functions occurring concurrently with TP. As we hypothesized, correct performance on the TOP task elicited fMRI activation responses in the ventral striatum [location of the nucleus accumbens or NAc, key reward brain nucleus (Breiter et al. 1997; Breiter and Rosen 1999)] and anterior insula [key cortical hub involved in interoception and timekeeping (Livesey et al. 2007; Wittmann et al. 2010; Bueti and Macaluso 2011)]. Activation in the NAc and anterior insula was significantly greater in the TD learning condition than in the outcome condition of the TOP task, pointing to the importance of these regions for learning associated with reward, as has been shown with reinforcement learning (O’Doherty et al. 2003; Seymour et al. 2004; O’Doherty et al. 2006) and anticipation of uncertain outcomes following prospect theory (Breiter et al. 2001). TP-based activation was significantly greater than any SM, SA, and WM contributions, showing that TP function could be distinguished in a specific fashion from concurrent cognitive processes. Prior reward/aversion studies, including studies of TP have not to our knowledge explicitly shown this segregation from attention and WM functions, which are also modulated by midbrain DA function. This differentiation of TP from attention and WM processes was further accentuated by observation of significant correlation between the speed of the behavioral responses and TD learning activation to errors in the region of the ventral tegmental area (VTA), a source of midbrain DA neurons.

The Neural Basis of Temporal Prediction Tomasi et al.

| 3

highlighting the nature and timing of the cues as well as the TD learning. Dashed arrows indicate the sequential advent of circles in the screen; dashed arrows are not shown in the screen during the task. (C) TD learning module: After the first circle is displayed on one of the corners of the screen, the 3 remaining circles are sequentially displayed at fixed intervals at the corners of the screen, as indicated by the cue in B. The participants predict the timing of each circle [W(t), FWHM Gaussian solid color curves], which might not match perfectly the correct timing of the circles [V(t), FWHM Gaussian black solid curve centered at t1, t2, and t3]. The error function, δ(t), measures the time difference between the predicted events and the correct events (dotted color curves). The MRI signal, assumed to be proportional to the time integral of δ(t), is expected to be higher for unsuccessful (miss) than for successful (hit) prediction trials.

to make such fast responses if they waited to see the target, and that in order to achieve faster responses they must develop a target prediction strategy based on the regular onset of the previous circles in the TD learning module. Thus, target prediction was based on the regular timing of the previous events in the 3.3-s long TD learning module (Fig. 1C), which was grounded in TD learning principles. Basically, we assumed that the observation of an event allows reinforcement-based adjustments of brain circuitry that lead to improved predictions of future events. To better predict the onset of the circles, midbrain DA neurons fire in proportion to the TD error function, δ = V − W, the difference between the ideal prediction, V, and the current prediction, W (Fig. 1C). Note that in this simple model we set the TD discount factor to 1 because the time scale of the blood oxygenation level-dependent (BOLD)-fMRI response (∼20 s) prevents assessment of this decay parameter within the length of the TD learning module (3.3 s). The outcome was presented as in the SM task, and followed by the fixation cross that was displayed after the outcome until the onset of the next cue, 4.5–6.5 s. Because the TOP task also involves spatial WM and SA, the proposed battery also includes additional WM and SA tasks to assess the potential confounds of these cognitive domains on TD learning. The WM task assesses the memory burden of the TOP task, which requires holding in memory the screen locations of subsequent circles until the end of the TD learning epoch. For each trial, a 1-s long cue showing a spatial random (nonrepeating) sequence of the natural numbers 1–4 was displayed for visual memory encoding as in the TOP task (Supplementary Fig. 1B). Subsequently, 200 ms after the cue, circles were flashed at each of the 4 corners of the visual field (WM retrieval epoch), as in the TD learning epoch. However, the appearance of the circles in the screen matched the spatiotemporal sequence suggested by the cue only in 50% of the trials. Subjects were instructed to press the button as fast as possible when the circles appeared in a spatial–temporal sequence that matched the cue; as for the

SM task, responses that occurred within 600 ms from the target were considered successful. Then, the outcome and the fixation cross were presented as in the TOP task until the onset of the next cue (4.5–6.5 s after the outcome). The SA task assesses the vigilance burden of the TOP task, which requires sustained attention to detect the appearance of circles at the corners of the screen during the TD learning epoch. For each trial, a cue displayed a circle, randomly, at 1 of the 4 corners of the central visual field highlighting 1 of the 4 quadrants of the screen (Supplementary Fig. 1B). Subsequently, 200 ms after the cue, circles were flashed randomly and sequentially at each of the 4 corners of the visual field (focused SA epoch), like in the TD learning epoch. The subject was instructed to press a button when the circle appeared at the target quadrant originally highlighted by the cue; as for the SM task, responses that occurred within 600 ms from the target were considered successful. Then, the outcome and the fixation cross were presented as in the TOP task until the onset of the next cue (4.5–6.5 s after the outcome). During the outcome, a money symbol ($) was used to indicate that responses were accurate (within time window). However, subjects were not rewarded with money for their RT or performance accuracy nor punished for their errors. Subject reimbursement for experiment participation was not affected by their performance. Post task questionnaires were not used to assess how rewarding the task was. The visual stimuli were presented to the subjects on MRI-compatible goggles (Resonance Technology, Inc., Northridge, CA, USA) connected to a personal computer. The software used to display the stimuli and record subject’s responses was developed in Visual Basic and Visual C (Visual Studio; Microsoft Corp., Redmond, WA, USA) and was synchronized precisely with MR acquisition using an MRI trigger pulse. Prior to the MRI session, the participants completed a brief training session (5–10 min) on the tasks outside the MRI scanner to ensure that subjects understood and were able to perform the tasks.

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Figure 1. (A) Time course of the TOP task, highlighting the onset and durations of the visual cues (brown), TD learning, response window (green), and the presentation of outcome ( pink) within one interstimulus interval (ISI = 11–13 s). (B) Spatiotemporal distribution of events, targets, responses, and outcomes for the TOP task within the ISI,

4

| Cerebral Cortex

MRI Data Acquisition

Data Processing Image reconstruction was performed using a phase correction method that produced minimal ghost artifacts (Caparelli and Tomasi 2008). The first 4 imaging time points were discarded to avoid nonequilibrium effects in the fMRI signal. The statistical parametric mapping package SPM8 (Wellcome Trust Centre for Neuroimaging, London, UK) was used for subsequent analyses. Ascending linear interpolation was used for the temporal realignment of the slices (i.e., “slice time correction”), which were acquired in a sequential order. Spatial realignment was performed with a fourth degree B-spline function without weighting and without warping; head motion was miss differential TOP activation and its association with RT in anterior insula, ventral striatum, and in a medial midbrain region that encompasses VTA.

Functional Region-of-Interest Analyses Functional region-of-interest (ROI) measures were used to report average statistical values or BOLD-fMRI responses in a volume comparable to the image smoothness (e.g., resolution elements or “resels”) rather than single-voxel peak values as well as to test potential linear and nonlinear (u-shaped) relationships between brain activation and RT. The volume of the resels was estimated using the random field calculation in SPM8 as a nearcubic volume with Cartesian FWHM = 15.6, 13.9, and 16.1 mm. Thus, 9-mm isotropic masks containing 27 voxels (0.73 mL) were defined at the centers of relevant activation/deactivation/ correlation clusters to extract the average % BOLD signal from individual contrast maps. These masks were created and centered at the precise coordinates listed in Table 1 and were fixed across subjects.

Results Behavioral Responses As expected, the average RT across subjects was shorter for TOP (0.26 ± 0.08 s; mean ± SD) than for SM (0.64 ± 0.08 s), WM (0.45 ± 0.15 s), and SA (0.46 ± 0.08 s) (P < 0.00001, paired t-tests; Fig. 2). Thus, TD learning enabled significantly faster responses than

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

A 4-T whole-body Varian (Palo Alto, CA)/Siemens (Erlangen, Germany) MRI scanner with a T2*-weighted single-shot gradient echo-planar imaging (EPI) pulse sequence [time echo (TE)/time repetition (TR) = 20/1600 ms, 4-mm slice thickness, 1-mm gap, 35 coronal slices, 64 × 64 matrix size, 3.125 × 3.125 mm2 in-plane resolution, 90° flip angle, 157 time points, 200.00 kHz bandwidth) with ramp-sampling and whole brain coverage was used to collect functional images with BOLD contrast. Padding was used to minimize motion. Earplugs (−28 dB sound pressure level attenuation; Aearo Ear TaperFit 2; Aearo Co., Indianapolis, IN, USA), headphones (−30 dB sound pressure level attenuation; Commander XG MRI Audio System, Resonance Technology Inc., Northridge, CA, USA), and a “quiet” EPI acquisition approach (Tomasi and Ernst 2003) were used to minimize the interference effect of scanner noise during fMRI (Tomasi et al. 2005). Anatomical images were collected using a T1-weighted three-dimensional modified driven equilibrium Fourier transform pulse sequence (TE/TR = 7/15 ms, 0.94 × 0.94 × 1.00 mm3 spatial resolution, axial orientation, 256 readout and 192 × 96 phase-encoding steps, 16 min scan time) and a modified T2-weighted hyperecho sequence (TE/TR = 0.042/10s, echo train length = 16, 256 × 256 matrix size, 30 coronal slices, 0.86 × 0.86 mm2 in-plane resolution, 5 mm thickness, no gap, 2 min scan time) to rule out gross morphological abnormalities of the brain.

BOLD-fMRI signal change from baseline (black screen with a fixation cross) caused by the SM, TOP, WM, and SA cues were obtained for each subject. An additional GLM was used to estimate BOLD-fMRI responses corresponding to the TD learning and the outcome epochs of the TOP task, independently for hits and errors (GLM2). Specifically, GLM2 included 12 regressors, 4 of which modeled the onsets of the TD learning and outcome epochs for successful (hit) and unsuccessful (miss) prediction trials and their first- and second-order time derivatives, which accounted for the remaining 8 regressors. GML2 was convolved with low- and high-pass filtering and the 6 motion covariates as GLM1.

The Neural Basis of Temporal Prediction Tomasi et al.

| 5

Table 1: Statistical significance of BOLD-fMRI signals to SM, TOP, WM, and SA in regions where TOP activation was stronger than the co-activation to SM, WM, and SA Regions

4 3 2 18 18 17 13 47 38

24 24 47 38 13 6 44 7 6 2 19 37

MNI coordinates (mm)

k

TOP > SM and WM and SA

SM

TOP

WM

SA

x

y

z

voxels

T

T

T

T

T

−39 −54 −63 3 12 6 45 39 57 9 3 12 12 −3 −15 0 0 −42 −51 −51 −24 −48 30 48 45 −39 −45

−25 −25 −22 −82 −91 −91 14 20 14 5 −16 11 −22 −19 8 2 29 17 11 −1 −1 5 −67 2 −34 −79 −58

64 55 25 34 22 10 −5 1 −8 4 4 −8 −17 4 −5 37 31 1 −5 4 52 31 37 31 43 −11 −14

8.4 6.3 3.8 7.9 6.6 6.5 6.3 5.3 4.6 6.3 5.4 5.6 4.5 5.4 4.2 5.8 5.2 4.9 4.8 3.9 −7.3 −6.5 −7.0 −7.0 −6.9 −6.5 −5.5

4.6 2.5 4.0 8.0 2.1 3.6 2.1 −3.1 3.9 NS 3.5 −2.9 NS 3.8 NS 3.4 2.7 NS 1.9 NS −2.9 −2.6 NS NS NS NS NS

11.6 4.9 7.2 13.0 10.5 11.5 10.7 7.4 8.1 8.6 8.6 4.9 6.9 8.3 4.9 8.3 8.0 6.5 7.4 5.5 −9.5 −6.5 −6.9 −5.6 −5.2 −9.0 −5.0

NS NS 2.0 −2.2 NS 2.8 3.2 5.5 NS NS NS NS 3.4 NS NS NS NS 3.6 NS NS 3.7 5.6 3.8 5.5 5.7 1.6 5.3

3.6 NS 3.3 8.2 8.0 7.9 6.3 2.3 4.8 3.0 4.7 NS 2.5 4.7 NS 3.6 3.2 NS 4.9 3.3 −7.3 −2.4 NS NS NS −7.2 −2.2

727

1732

460

1308

630 481

556 1734

922

Note: Within-subject ANOVA.

preparation effects, because the instructions were different for the TOP task than for the SM, WM, and SA tasks. Performance accuracy was better for TOP (77 ± 18%; mean ± SD) than for SM (46 ± 23%) and for WM (95 ± 9%) than for TOP (P < 2 × 10−9; Fig. 2). Accuracy did not show linear increases or decreases and remained stable during the performance of the TOP task (mean accuracy/trial = 77 ± 8%), demonstrating the lack of accumulative learning effects during the TOP task. Accuracy for the WM and SA tasks had ceiling effects (Fig. 2), which suggests the lack of sensitivity to learning effects in these tasks. Indeed, the rates of hits and false alarms did not change over the course of the WM and SA tasks.

Brain Activation

Figure 2. Column scatter graph showing the average behavioral responses across trials for each participant (circles) during the SM (blue), TP (green), WM (red), and SA ( pink) fMRI tasks. RT was significantly different for all tasks (P < 5 × 10−6; paired t-test). Performance accuracy was significantly different for all tasks (P < 5 × 10−6), except for the WM and SA which did not show significant differences in accuracy (P > 0.08).

strategies purely based on perception (SM) as well as those also based on WM and sustained attention (WM and SA). However, the faster responses for the TOP task could partially reflect

Voxel-wise statistical parametric mapping for the TOP task demonstrated positive fMRI responses (activation) in anterior cingulum (dACC), orbital fronto-insular, prefrontal, parietal, temporal and limbic cortices, somatosensory and motor areas, medial visual cortex, cerebellum, thalamus, midbrain, caudate, and NAc; the TOP task also caused negative fMRI responses (deactivation) in lateral occipital and parietal regions, supplementary motor area, paracentral lobule, precentral gyrus, and superior and middle frontal gyri (Supplementary Fig. 2 and Table 1; PFWE < 0.05, corrected for multiple comparisons at the cluster level with the random field theory and a family-wise error). To identify areas uniquely involved in TP, we contrasted TOP activation responses to the co-activation responses of the SM, WM, and SA tasks using voxel-wise one-way within-subject

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Precentral Postcentral Postcentral Cuneus Cuneus Calcarine Anterior insula Anterior insula Temporal pole Caudate Medial thalamus NAc Midbrain Thalamus NAc Anterior cingulum Anterior cingulum Anterior insula Temporal pole Rolandic oper Superior frontal Precentral Middle occipital Precentral Supra marginal Inferior occipital Inferior occipital

BA

6

| Cerebral Cortex

increased activation during the TOP task, compared against the SM (A), WM (B), and SA (C) tasks as well against the co-activation caused by the SM, WM, and SA tasks (D); one-way within-subject ANOVA. (E) Column scatter graph showing the average BOLD responses elicited by each of the fMRI task within 27 voxels cubic ROI centered at the locations of the medial thalamus, right anterior insula (bilateral), and right NAc (see MNI coordinates of the ROIs in Table 1).

ANOVA. We found that TP increasingly engaged DAergic regions (NAc, caudate, midbrain, and medial thalamus), anterior insula, dACC and the temporal pole, as well as motor, visual, and somatosensory cortices and disengaged superior frontal, parietal, and inferior and middle occipital cortices (Fig. 3 and Table 1; PFWE < 0.05). These results were consistent with our hypothesis that activation of the insula and DAergic regions is essential for TP. To quantify the amplitude of the fMRI responses in brain regions involved in TP, we averaged the % BOLD signal changes from baseline within 27-voxel cubic ROIs centered at the cluster locations in Table 1. These ROI analyses confirmed the higher activation/deactivation responses for TOP than for SM, WM, and SA, separately, for all clusters highlighted in Table 1 (P < 0.05). As an example, Figure 3E emphasizes the higher fMRI responses in the right anterior insula and NAc and medial thalamus for TOP than for SM, WM, and SA. A direct comparison of brain activation caused by TD learning and outcome periods (Fig. 1A) demonstrated that TD learning caused stronger brain activation responses than outcome such that TOP activation was driven by TD learning (Fig. 4). Whereas TD learning engaged the same cortical and subcortical regions that showed TOP activation/deactivation, outcome strongly activated dorsal striatum and deactivated ventral caudate, bilateral pars opercularis (BA 44), and cerebellar vermis (Fig. 4). This suggests differential involvement of brain areas in TD learning from those involved in outcome processing.

Correlation Between RT and TOP activation Significant correlation between RT and TOP activation responses in the left and right cerebellum (anterior lobe, PFWE < 0.02) and in the right orbital fronto-insular cortex (OFIC, x, y, z = 54, 11, −5 mm; t-score = 3.9, cluster size 86 voxels, PFWE < 0.02; small volume correction within a 10-mm spherical volume) emerged from voxel-wise linear regression analyses (Fig. 5A). These results are consistent with the role of the anterior insula in timekeeping operations such that the stronger the activation in OFIC the faster (i.e., more accurate) the TPs of the targets. The functional ROI analysis confirmed the negative correlation between RT and TOP activation responses in the cerebellum and OFIC (R < −0.37; P < 0.03; Fig. 5A). In addition, linear regression revealed a negative correlation across all tasks and subjects between RT and activation responses in the OFIC and NAc ROIs (R < −0.24; P > 0.005; Fig. 5B), but less so in the cerebellum (R < −0.20; P = 0.02), which suggests variable timekeeping demands across tasks in these regions.

Successful Versus Unsuccessful Prediction Behavioral and functional responses during successful prediction (hits: 74.1% of the trials) and unsuccessful predictions (errors or misses: 25.9% of the trials) were analyzed for each of the 34 subjects that did not achieve perfect accuracy during TOP task performance but not for the 2 subjects who had RT < 300 ms for

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Figure 3. Statistical significance of brain activation differences superimposed on axial and sagittal views of the human brain (radiological convention) showing the

The Neural Basis of Temporal Prediction Tomasi et al.

| 7

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Figure 4. Statistical significance of brain activation evoked by the TD learning and outcome epochs of the TOP task and their difference (TD learning > outcome) superimposed on axial views of the human brain (radiological convention).

all 20 trials ( perfect performance) in the TOP task. Naturally, RT was significantly longer for misses (0.52 ± 0.11 s; mean ± SD) than for hits (0.17 ± 0.07 s; P = 5 × 10−14, paired t-test; Fig. 6A). Voxel-wise within-subject ANOVA showed that, in NAc and medial OFC, brain activation responses to the outcome were significantly lower for miss trials than for hit trials (PFWE < 0.0005; cluster size = 800 voxels; Fig. 6B), finding that was confirmed by the ROI analysis (Fig. 6C). Given the role of NAc in reward processing, this suggests that failure outcomes in TP are associated with negative DAergic reward signaling, or alternatively,

that successful outcomes are by themselves rewarding. Withinsubject ANOVA also showed that activation responses to TD learning in a medial midbrain region that includes VTA were significantly lower for hits than for misses (PFWE < 0.03; x, y, z = 0, −13, −14 mm; cluster size 90 voxels; peak level t-score = 3.4, small volume corrections within a 10-mm spherical volume; Fig. 6B), which was consistent with our hypothesis on the activation of midbrain DA neurons during errors. Conversely, brain activation responses to TD learning in the cerebellum (vermis) were significantly higher for hits than for misses (PFWE < 0.0005;

8

| Cerebral Cortex

Association between RT and activation responses in the right anterior insula and NAc (linear regression; df: 106) across subjects and tasks. Statistical significance (red– yellow color maps) is superimposed on axial views of the human brain (radiological convention).

cluster size = 476 voxels; Fig. 6B). The analysis of functional ROI measures confirmed the findings for cerebellum and VTA (P < 0.02; Fig. 6D). Linear regression analysis revealed a significant negative correlation between RT and VTA responses to misses, but not to hits during TD learning (PFWE = 0.02; x, y, z = 0, −13, −14 mm; cluster size 38 voxels; peak level t-score = 3.6; small volume corrections within a 10-mm spherical volume), which was confirmed with ROI measures from VTA (R = −0.46, P < 0.006; Fig. 7) and is also consistent with activation of midbrain DA neurons during errors. Furthermore, direct comparison of linear regressions for misses and hits revealed a success × RT interaction effect on VTA responses to TD learning (P = 0.036, slope; P < 0.0005, intercept) such that increased VTA activation was associated with faster responses for unsuccessful (miss) trials (no association between RT and VTA responses was found for hits). Similar correlations for the cerebellum did not reach significance (Fig. 7).

Discussion The timekeeping operations required for precise TP of event onset engaged a reward-motivation network (Fig. 4D) that comprised regions modulated by the DA mesolimbic and mesocortical pathways including anterior insula, NAc, ventral caudate, and

medial thalamus (Wise 2002; Breiter and Gasic 2004; Kelley et al. 2005; Liu et al. 2011; Haight and Flagel 2014). These observations suggest that accurate TOP performance is experienced by subjects as rewarding. Specifically, the novel TOP task caused stronger activation than the SM, WM, and SA tasks in regions involved in salience attribution (OFIC and dorsal anterior cingulum; Seeley et al. 2007) and DAergic nuclei in the basal ganglia (ventral caudate, NAc, and midbrain) and middle thalamus. These findings are consistent with the modulatory role of DA in the feedback control of interval timing (Lustig and Meck 2005; Meck 2005) and with the effects of DAergic drugs on the speed of the internal clock (Lake and Meck 2013).

Behavioral Responses RT, the time difference between target onsets and button press responses, was significantly shorter for the TOP task than for the SM, WM, and SA tasks (Fig. 2). Previous studies have shown that temporal processing skills include time perception in the form of verbal time estimation and motor timing, the ability to temporally adjust a motor response to sensory stimulation (Rao et al. 2001). Subject’s responses were 2.5 times faster for the TOP task compared with the perceptual motor task (SM), which demonstrates that subjects anticipated their responses in a way

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Figure 5. (A) Association across subjects between RT and activation responses in the cerebellum and right anterior insula during the TOP task (linear regression; df: 34). (B)

The Neural Basis of Temporal Prediction Tomasi et al.

| 9

superimposed on coronal and axial views of the human brain in the radiological convention (within-subject ANOVA); (C) column scatter graph showing the average BOLD responses in NAc elicited by TD learning and outcome, independently for hit and miss trials; (D) average BOLD responses in the cerebellum (CB) and VTA elicited by TD learning, independently for hit and miss trials (27-voxel cubic ROI centered at x, y, z = 0, −13, −12 mm).

that the identification and interpretation of visual information necessary for the perception of the target was incomplete at the time of the response. Thus, processing speed was not based on perception and the speed of motor responses during the TOP task. Whereas RT could also depend on attention and WM skills (Livesey et al. 2007), the responses in the present study were significantly faster for the TOP task than for the WM and SA tasks. This suggests that subjects’ prediction abilities were not supported solely by attention and WM processing based on the spatial information embedded in the cues. We propose that, in the present study, TP was based on incremental learning based on the temporal information of events in the TD learning module. However, since there were no behavioral changes over the course of tasks (habituation/sensitization effects or reinforcement learning), further work is needed to test the hypothesis that TP is based on incremental learning. It is noteworthy too that despite responses were significantly faster, accuracy was significantly higher for the TOP task than for the SM task (Fig. 2), which underscores the value of prediction for improving performance.

Anterior Insula The engagement of the anterior insula and the basal ganglia during TP is consistent with previous studies on time perception and supports an important role of these brain regions in timekeeping. Previous studies have shown activation in middle and posterior insula during the reproduction of time durations of auditory and visual stimuli, consistent with the role of the insula in time encoding (Livesey et al. 2007; Wittmann et al. 2010; Bueti and Macaluso 2011). Recent studies on attention to interoceptive processes showed that fear caused subjective time dilation, suggesting that experience of time emerges from emotional and interoceptive signals in the insular cortex (Pollatos et al. 2014). Here, we show a correlation between activation responses in the anterior insula and RT during the TOP task (Fig. 5). This

finding is also consistent with the accumulation function of the anterior insula, which postulates that our sense of time reflects the accumulation of physiological changes in body states (Wittmann et al. 2010; Bueti and Macaluso 2011). The anterior insula is also involved in TD learning (Sutton and Barto 1981; Sutton 1988) during Pavlovian conditioning. The TOP task contains a TD learning module (Fig. 1B), which allows reinforcement of the prediction circuitry to accurately forecast target onsets. Thus, the activation of the anterior insula during the TOP task, but less so during SM, WM, and SA, suggests an important role of reward learning on event onset forecasting during the TD learning module. The correlation between activation responses in the anterior insula and RT across tasks and subjects (Fig. 5B) suggests differential timekeeping reinforcement for SM, SA, WM, and TOP that increase in proportion to the predictability of the events.

Basal Ganglia The basal ganglia have been implicated in WM and some attention processes (Schroll et al. 2012). This study is the first to test the involvement of the basal ganglia in TP while controlling for other concurrently active mental processes (WM, SA, and SM). The stronger activation of the ventral striatum (caudate and NAc) for the TOP task than for the SM, WM, and SA tasks is consistent with the involvement of basal ganglia in timing and reward learning (Rao et al. 2001; Schultz 2007). Converging evidence from psychopharmacological studies as well as studies in animals, healthy volunteers and patients with Parkinson’s disease, schizophrenia, or attention deficit disorder implicate basal ganglia pathways in timekeeping (Maricq and Church 1983; Jahanshahi et al. 2006; Jones et al. 2008; Carroll et al. 2009; Rubia et al. 2009; Allman and Meck 2012; Coull et al. 2012; Noreika et al. 2013). The NAc, an important nucleus for reward and for prediction of reward related activity, is involved in

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Figure 6. (A) Column scatter graph showing the average RT for successful (hits; green circles) and unsuccessful (misses; red circles) trials during the TOP task; (B) statistical significance (t-score) of differential activation responses for successful (hit) and unsuccessful (miss) trials, independently for TD learning and outcome epochs,

10

| Cerebral Cortex

Accurate Prediction and Reward

Figure 7. Linear association across subjects between RT and TD learning activation responses in VTA, and cerebellum independently for hit and miss trials (df: 34).

procedural learning and in reward learned associations (Saddoris and Carelli 2014). Indeed, studies on Pavlovian conditioning and decision-making implicate the ventral striatum in reward learning (O’Doherty et al. 2003; Rolls et al. 2008). Activation in the ventral striatum also showed significant correlation with RT across tasks and subjects (Fig. 5B), supporting differential reinforcement for SM, SA, WM, and TOP.

The last decade has seen a proliferation of video games that can be highly rewarding and can result in compulsive gaming. Video gaming reward requires accurate performance, which in many instances needs prediction of the onsets of the visual stimuli to respond prior to target detection. Our simple task emulates some of these properties and shows that TP activates the mesolimbic pathway, which is consistent with previous studies that associated video gaming performance with striatal volume (Erickson et al. 2010), activation of the ventral striatum (Hoeft et al. 2008; Kühn et al. 2011), and striatal DA release (Koepp et al. 1998). In addition, the higher activation to the outcome during accurate versus inaccurate performance highlights the rewarding nature of accurate performance in and of itself, which further reinforces behaviors in video gaming. Inasmuch as compulsive gaming behavior emerges in some individuals that play these video games, the performance-dependent reward and concomitant activation of the DA pathways could underlie the addiction patterns reported with video gaming.

Summary Here, we show a role of the DA meso-accumbens pathway and its connection to insula in time prediction. Our findings on NAc activation in the absence of explicit reward stimuli also provide evidence that accurate task performance in and of itself is rewarding, which could underlie the motivation triggered by challenging activities.

Prediction Errors

Supplementary Material

Midbrain neurons process rewarding and reward predicting stimuli, and electrophysiological data from alert monkeys (Mirenowicz and Schultz 1994) showed that the firing rate of midbrain neurons is associated with the error function in the TD learning algorithm (Schultz et al. 1997). In the present study and compared with successful predictions, prediction errors caused decreased NAc activation to outcome and increased

Supplementary material can be found at: http://www.cercor. oxfordjournals.org/.

Funding This research was supported by the National Institute of Health’s Intramural Research Program (National Institute on Alcohol

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

midbrain activation to TD learning (Fig. 6C), such that greater VTA activation was associated with less inaccurate predictions (i.e., shorter RT; Fig. 7). This is consistent with the role of the NAc in reward processing and with the increased involvement of the midbrain in learning (D’Ardenne et al. 2008; Glimcher 2011; Iglesias et al. 2013), and suggests that VTA is increasingly activated during error trials to improve the accuracy of the predictions. Here, we propose reward learning as a potential mechanism behind the increased activation of VTA to errors compared with that of successful predictions (Fig. 1C). Previous studies have shown that midbrain neurons fire in proportion to the TD error function (Schultz et al. 1997), δ, which in this study reflects the time difference between sensory information corresponding to the onsets of the circles at t1, t2, and t3 (V; correct prediction) and the time predictions of these onsets (W). We assume that if the TD learning module is sufficiently prolonged, DAergic reinforcement of accurate predictions should continue until time differences between predictions and events are null (successful prediction or hit). For a time-limited TD learning module, such as the one in this work, successful prediction can be achieved (hit) or not (miss) during each trial. Since the fMRI response is proportional to the time integral of the firing rate (Logothetis et al. 2001; Sander et al. 2013), activation responses of DA neurons in VTA should be higher for miss than for hit trials as shown in our data (Fig. 6D).

The Neural Basis of Temporal Prediction

Abuse and Alcoholism; 2RO1AA09481) and was carried out using the infrastructure of Brookhaven National Laboratory under contract DE-AC02-98CH10886.

Notes We thank Millard Jayne, Frank Telang, Pauline Carter, and Barbara Hubbard for subject care and protocol oversight; Karen Apelskog for protocol coordination; Ruiliang Wang for MRI data acquisition; and the reviewers for constructive criticism and meaningful suggestions. We also thank the subjects who volunteered to participate in this study.

References

| 11

Dreher J, Meyer-Lindenberg A, Kohn P, Berman K. 2008. Age-related changes in midbrain dopaminergic regulation of the human reward system. Proc Natl Acad Sci USA. 105:15106–15111. Drew M, Simpson E, Kellendonk C, Herzberg W, Lipatova O, Fairhurst S, Kandel E, Malapani C, Balsam P. 2007. Transient overexpression of striatal D2 receptors impairs operant motivation and interval timing. J Neurosci. 27:7731–7739. Erickson K, Boot W, Basak C, Neider M, Prakash R, Voss M, Graybiel A, Simons D, Fabiani M, Gratton G, et al. 2010. Striatal volume predicts level of video game skill acquisition. Cereb Cortex. 20:2522–2530. Glimcher P. 2011. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci USA. 108(Suppl 3):15647–15654. Haight J, Flagel S. 2014. A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses. Front Behav Neurosci. 8:79. Hoeft F, Watson C, Kesler S, Bettinger K, Reiss A. 2008. Gender differences in the mesocorticolimbic system during computer game-play. J Psychiatr Res. 42:253–258. Iglesias S, Mathys C, Brodersen K, Kasper L, Piccirelli M, den Ouden H, Stephan K. 2013. Hierarchical prediction errors in midbrain and basal forebrain during sensory learning. Neuron. 80:519–530. Jahanshahi M, Jones C, Dirnberger G, Frith C. 2006. The substantia nigra pars compacta and temporal processing. J Neurosci. 26:12266–12273. Jazayeri M, Shadlen M. 2010. Temporal context calibrates interval timing. Nat Neurosci. 13:1020–1026. Jones C, Malone T, Dirnberger G, Edwards M, Jahanshahi M. 2008. Basal ganglia, dopamine and temporal processing: performance on three timing tasks on and off medication in Parkinson’s disease. Brain Cogn. 68:30–41. Kelley A, Baldo B, Pratt W, Will M. 2005. Corticostriatal-hypothalamic circuitry and food motivation: integration of energy, action and reward. Physiol Behav. 86:773–795. Knudsen E. 2007. Fundamental components of attention. Annu Rev Neurosci. 30:57–78. Koepp M, Gunn R, Lawrence A, Cunningham V, Dagher A, Jones T, Brooks D, Bench C, Grasby P. 1998. Evidence for striatal dopamine release during a video game. Nature. 393:266–268. Kühn S, Romanowski A, Schilling C, Lorenz R, Mörsen C, Seiferth N, Banaschewski T, Barbot A, Barker G, Büchel C, et al. 2011. The neural basis of video gaming. Transl Psychiatry. 1:e53. Lake J, Meck W. 2013. Differential effects of amphetamine and haloperidol on temporal reproduction: dopaminergic regulation of attention and clock speed. Neuropsychologia. 51:284–292. Liu X, Hairston J, Schrier M, Fan J. 2011. Common and distinct networks underlying reward valence and processing stages: a meta-analysis of functional neuroimaging studies. Neurosci Biobehav Rev. 35:1219–1236. Livesey A, Wall M, Smith A. 2007. Time perception: manipulation of task difficulty dissociates clock functions from other cognitive demands. Neuropsychologia. 45:321–331. Logothetis N, Pauls J, Augath M, Trinath T, Oeltermann A. 2001. Neurophysiological investigation of the basis of the fMRI signal. Nature. 412:150–157. Lustig C, Meck W. 2005. Chronic treatment with haloperidol induces deficits in working memory and feedback effects of interval timing. Brain Cogn. 58:9–16. Maricq A, Church R. 1983. The differential effects of haloperidol and methamphetamine on time estimation in the rat. Psychopharmacology (Berl). 79:10–15.

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Allman MJ, Meck WH. 2012. Pathophysiological distortions in time perception and timed performance. Brain. 135:656–677. Allman MJ, Teki S, Griffiths TD, Meck WH. 2014. Properties of the internal clock: first- and second-order principles of subjective time. Annu Rev Psychol. 65:743–771. Bäckman L, Nyberg L, Lindenberger U, Li S, Farde L. 2006. The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neurosci Biobehav Rev. 30:791–807. Breiter H, Aharon I, Kahneman D, Dale A, Shizgal P. 2001. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 30:619–639. Breiter H, Gasic G. 2004. A general circuitry processing reward/ aversion information and its implications for neuropsychiatric illness. In: Gazzaniga M, editor. The cognitive neurosciences III. Cambridge: MIT Press, p. 1043–1065. Breiter H, Gollub R, Weisskoff R, Kennedy D, Makris N, Berke J, Goodman J, Kantor H, Gastfriend D, Riorden J, et al. 1997. Acute effects of cocaine on the human brain activity and emotion. Neuron. 19:591–611. Breiter HC, Rosen BR. 1999. Functional magnetic resonance imaging of brain reward circuitry in the human. Ann N Y Acad Sci. 877:523–547. Bueti D, Macaluso E. 2011. Physiological correlates of subjective time: evidence for the temporal accumulator hypothesis. Neuroimage. 57:1251–1263. Buhusi C. 2003. Dopaminergic mechanisms of interval timing and attention. In: Meck W, editor. Functional and neural mechanisms of interval timing. Boca Raton, FL: CRC Press. p. 317–338. Caparelli E, Tomasi D. 2008. K-space spatial low-pass filters can increase signal loss artifacts in echo-planar imaging. Biomed Signal Process Control. 3:107–114. Carroll C, O’Donnell B, Shekhar A, Hetrick W. 2009. Timing dysfunctions in schizophrenia span from millisecond to several-second durations. Brain Cogn. 70:181–190. Cheng R, MacDonald C, Meck W. 2006. Differential effects of cocaine and ketamine on time estimation: implications for neurobiological models of interval timing. Pharmacol Biochem Behav. 85:114–122. Coull J, Cheng R, Meck W. 2011. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology. 36:3–25. Coull J, Hwang H, Leyton M, Dagher A. 2012. Dopamine precursor depletion impairs timing in healthy volunteers by attenuating activity in putamen and supplementary motor area. J Neurosci. 32:16704–16715. D’Ardenne K, McClure S, Nystrom L, Cohen J. 2008. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 319:1264–1267.

Tomasi et al.

12

| Cerebral Cortex

Saddoris M, Carelli R. 2014. Cocaine self-administration abolishes associative neural encoding in the nucleus accumbens necessary for higher-order learning. Biol Psychiatry. 75:156–164. Sander C, Hooker J, Catana C, Normandin M, Alpert N, Knudsen G, Vanduffel W, Rosen B, Mandeville J. 2013. Neurovascular coupling to D2/D3 dopamine receptor occupancy using simultaneous PET/functional MRI. Proc Natl Acad Sci USA. 110:11169–11174. Schroll H, Vitay J, Hamker F. 2012. Working memory and response selection: a computational account of interactions among cortico-basalganglio-thalamic loops. Neural Netw. 26:59–74. Schultz W. 2007. Behavioral dopamine signals. Trends Neurosci. 30:203–210. Schultz W. 1998. Predictive reward signal of dopamine neurons. J Neurophysiol. 80:1–27. Schultz W, Dayan P, Montague P. 1997. A neural substrate of prediction and reward. Science. 275:1593–1599. Seeley W, Menon V, Schatzberg A, Keller J, Glover G, Kenna H, Reiss A, Greicius M. 2007. Dissociable intrinsic connectivity networks for salience processing and executive control. J Neurosci. 27:2349–2356. Seymour B, O’Doherty J, Dayan P, Koltzenburg M, Jones A, Dolan R, Friston K, Frackowiak R. 2004. Temporal difference models describe higher-order learning in humans. Nature. 429:664–667. Sutton R. 1988. Learning to predict by the methods of temporal differences. Mach Learn. 3:9–44. Sutton R, Barto A. 1981. Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev. 88:135–170. Tinklenberg J, Roth W, Kopell B. 1976. Marijuana and ethanol: differential effects on time perception, heart rate, and subjective response. Psychopharmacology (Berl). 49:275–279. Tomasi D, Caparelli EC, Chang L, Ernst T. 2005. fMRI-acoustic noise alters brain activation during working memory tasks. Neuroimage. 27:377–386. Tomasi D, Ernst T. 2003. Echo planar imaging at 4 Tesla with minimum acoustic noise. J Magn Reson Imaging. 18:128–130. Wiener M, Lee Y, Lohoff F, Coslett H. 2014. Individual differences in the morphometry and activation of time perception networks are influenced by dopamine genotype. Neuroimage. 89:10–22. Wise R. 2002. Brain reward circuitry: insights from unsensed incentives. Neuron. 36:229–240. Wittmann M, Leland D, Churan J, Paulus M. 2007. Impaired time perception and motor timing in stimulant-dependent subjects. Drug Alcohol Depend. 2–3:183–192. Wittmann M, Simmons A, Aron J, Paulus M. 2010. Accumulation of neural activity in the posterior insula encodes the passage of time. Neuropsychologia. 48:3110–3120.

Downloaded from http://cercor.oxfordjournals.org/ by guest on November 12, 2014

Matell M, Meck W. 2004. Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes. Brain Res Cogn Brain Res. 21:139–170. Maunsell J. 2004. Neuronal representations of cognitive state: reward or attention? Trends Cogn Sci. 8:261–265. McClure S, Berns G, Montague P. 2003. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 38:339–3456. Meck W. 1986. Affinity for the dopamine D2 receptor predicts neuroleptic potency in decreasing the speed of an internal clock. Pharmacol Biochem Behav. 25:1185–1189. Meck W. 2005. Neuropsychology of timing and time perception. Brain Cogn. 58:1–8. Meck W. 2006. Neuroanatomical localization of an internal clock: a functional link between mesolimbic, nigrostriatal, and mesocortical dopaminergic systems. Brain Res. 1109:93–107. Merchant H, Harrington D, Meck W. 2013. Neural basis of the perception and estimation of time. Annu Rev Neurosci. 36:313–336. Mirenowicz J, Schultz W. 1994. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 72:1024–1027. Nobre A, Correa A, Coull J. 2007. The hazards of time. Curr Opp Neurobiol. 17:465–470. Noreika V, Falter C, Rubia K. 2013. Timing deficits in attentiondeficit/hyperactivity disorder (ADHD): evidence from neurocognitive and neuroimaging studies. Neuropsychologia. 51:235–266. O’Doherty J, Buchanan T, Seymour B, Dolan R. 2006. Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron. 49:157–166. O’Doherty J, Dayan P, Friston K, Critchley H, Dolan R. 2003. Temporal difference models and reward-related learning in the human brain. Neuron. 38:329–337. Oprisan S, Buhusi C. 2011. Modeling pharmacological clock and memory patterns of interval timing in a striatal beat-frequency model with realistic, noisy neurons. Front Integr Neurosci. 5:52. Pollatos O, Laubrock J, Wittmann M. 2014. Interoceptive focus shapes the experience of time. PLoS ONE. 9:e86934. Rao S, Mayer A, Harrington D. 2001. The evolution of brain activation during temporal processing. Nat Neurosci. 4:317–323. Rolls E, McCabe C, Redoute J. 2008. Expected value, reward outcome, and temporal difference error representations in a probabilistic decision task. Cereb Cortex. 18:652–663. Rubia K, Halari R, Christakou A, Taylor E. 2009. Impulsiveness as a timing disturbance: neurocognitive abnormalities in attention-deficit hyperactivity disorder during temporal processes and normalization with methylphenidate. Philos Trans R Soc B. 364:1919–1931.