Mutual Information Expansion for Studying the ... - Semantic Scholar

LETTER

Communicated by Stefano Panzeri

Mutual Information Expansion for Studying the Role of Correlations in Population Codes: How Important Are Autocorrelations? A. Scaglione∗ [email protected] School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, U.S.A.

G. Foffani∗ [email protected] and [email protected] School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, U.S.A., and Neurosignals Group, Hospital Nacional de Parapl´ejicos, SESCAM, 45071 Toledo, Spain

G. Scannella [email protected]

S. Cerutti [email protected] Department of Biomedical Engineering, Politecnico di Milano, 20133 Milano, Italy

K. A. Moxon [email protected] School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, U.S.A.

The role of correlations in the activity of neural populations responding to a set of stimuli can be studied within an information theory framework. Regardless of whether one approaches the problem from an encoding or decoding perspective, the main measures used to study the role of correlations can be derived from a common source: the expansion of the mutual information. Two main formalisms of mutual information expansion have been proposed: the series expansion and the exact breakdown. Here we clarify that these two formalisms have a different representation of autocorrelations, so that even when the total information estimated differs by less than 1%, individual terms can diverge. More precisely, the series expansion explicitly evaluates the

∗ The first two authors equally contributed to this work. G. Foffani is the corresponding author.

Neural Computation 20, 2662–2695 (2008)

C 2008 Massachusetts Institute of Technology

Mutual Information Expansion and Autocorrelations

2663

informational contribution of autocorrelations in the count of spikes, that is, count autocorrelations, whereas the exact breakdown does not. We propose a new formalism of mutual information expansion, the Poisson exact breakdown, which introduces Poisson equivalents in order to explicitly evaluate the informational contribution of count autocorrelations with no approximation involved. Because several widely employed manipulations of spike trains, most notably binning and pooling, alter the structure of count autocorrelations, the new formalism can provide a useful general framework for studying the role of correlations in population codes.

1 Introduction A central problem in computational neuroscience is to establish the role of correlations in the information conveyed about stimuli by the activity of populations of neurons (Averbeck, Latham, & Pouget, 2006; Averbeck & Lee, 2006; Brenner, Strong, Koberle, Bialek, & de Ruyter van Steveninck, 2000; Latham & Nirenberg, 2005; Nirenberg & Latham, 2003; Panzeri & Schultz, 2001; Panzeri, Schultz, Treves, & Rolls, 1999; Panzeri, Treves, Schultz, & Rolls, 1999; Rolls, Franco, Aggelopoulos, & Reece, 2003). This role has been analyzed from two different perspectives: encoding and decoding (Amari & Nakahara, 2006; Averbeck & Lee, 2006; Panzeri, Pola, Petroni, Young, & Petersen, 2002). From the encoding perspective, to establish the role of correlations consists of evaluating the contribution of correlations to the information conveyed by single-trial responses of populations of neurons (Panzeri, Schultz, et al., 1999; Schneidman, Bialek, & Berry, 2003). From the decoding perspective, to establish this role is to determine whether the knowledge of correlations is crucial to decode the neural activity (Amari & Nakahara, 2006; Averbeck & Lee, 2006). Both the main encoding and decoding measures used in the literature can be derived from the mutual information expansion, of which two formalisms have been proposed: the series expansion (Panzeri, Schultz, et al., 1999) and the exact breakdown (Pola, Thiele, Hoffmann, & Panzeri, 2003). Encoding and decoding measures derived from the mutual information expansion offer powerful tools for studying the role of correlations in population codes. However, the impact of autocorrelations on these measures has not been fully investigated. Autocorrelations mean correlations between spikes emitted by the same neuron. Mathematically, they can be of two main types: (1) correlations in the timing of spike arrivals, which we call timing autocorrelations, and (2) correlations in the count of spikes, which we call count autocorrelations. Timing autocorrelations represent correlations between bins within the same neuron and are conceptually identical to the cross-correlations between bins in different neurons. Count autocorrelations are somewhat less

2664

A. Scaglione et al.

intuitive, as they represent correlations within the same bin, defined as a deviation of the count distribution from the Poisson condition of independence (see, e.g., appendix A in Panzeri, Schultz, et al., 1999). Count autocorrelations do not have a cross-correlation counterpart. Here we clarify that the series expansion (Panzeri, Schultz, et al., 1999) and the exact breakdown (Pola et al., 2003) have a different representation of autocorrelations. Namely, although the two formalisms are equivalent for timing autocorrelations (Panzeri & Schultz, 2001; Pola, Petersen, Thiele, Young, & Panzeri, 2005), the series expansion explicitly evaluates the informational contribution of count autocorrelations, whereas the exact breakdown does not. This is an important difference, because several widely employed manipulations of spike trains, most notably binning and pooling, alter the structure of count autocorrelations. We therefore propose a new formalism of mutual information expansion that introduces Poisson equivalents in order to estimate the informational contribution of count autocorrelations with no approximation involved. This letter is organized as follows. We first provide the conceptual background of our work and present the mathematical formalisms of the series expansion and the exact breakdown. We then analytically compare the two formalisms, showing their differences when count autocorrelations are considered. Next, we quantify the impact of these differences on simulated data. Finally, we present and validate the new formalism of mutual information expansion: the Poisson exact breakdown. 2 Conceptual Background In recent years, information theory and especially mutual information (I) have provided several quantitative tools to investigate the role of correlations in population coding (Dayan & Abbott, 2001; Gawne & Richmond, 1993; Latham & Nirenberg, 2005; Montani, Kohn, Smith, & Schultz, 2007; Nirenberg, Carcieri, Jacobs, & Latham, 2001; Nirenberg & Latham, 2003; Panzeri & Schultz, 2001; Panzeri, Schultz, et al., 1999; Rieke, 1997; Treves, Panzeri, Rolls, Booth, & Wakeman, 1999). On one hand, from the encoding perspective, two main measures have been proposed to quantify the impact of correlations: (1) the synergy/redundancy (Isyn ), which quantifies whether a population of neurons is more informative (synergy) or less informative (redundancy) than the sum of its individual neurons, and (2) the shuffled term of the mutual information (Ishuffle ), which quantifies the difference between the information conveyed by a population of neurons and the information conveyed by the same population after destroying correlations by shuffling the responses between individual neurons across trials (Brenner et al., 2000; Gawne & Richmond, 1993; Panzeri, Schultz, et al., 1999; Schneidman et al., 2003). On the other hand, from the decoding perspective, the main measure proposed is the average across neural responses of the Kullback-Leibler distance between the probability of having


2665

received a particular stimulus given the real neural responses and the probability of having received the same stimulus given the shuffled neural responses. This measure (I) provides an upper bound of the information lost by an optimal decoder that assumes absence of correlations compared to an optimal decoder that considers correlations (Nirenberg et al., 2001; Nirenberg & Latham, 2003; Panzeri, Schultz, et al., 1999; Panzeri, Treves, et al., 1999; Pola et al., 2003). These measures allowed investigators to gain important physiological insights into the impact of correlations in real neural data (Montani et al., 2007; Montemurro, Panzeri, et al., 2007; Nirenberg et al., 2001; Panzeri, Petroni, Petersen, & Diamond, 2003; Panzeri, Pola, & Petersen, 2003; Petersen, Panzeri, & Diamond, 2001; Rolls, Aggelopoulos, Franco, & Treves, 2004; Rolls et al., 2003), but also led to substantial disagreements (Latham & Nirenberg, 2005; Schneidman et al., 2003). Interestingly, each of the above measures can be derived from a common source, the expansion of the mutual information (Panzeri, Schultz, et al., 1999; Panzeri, Treves, et al., 1999; Pola et al., 2003), which therefore provides the basic building blocks for investigating the role of correlations in the neural code (see Figure 1). In order to understand the expansion of the mutual information, it is important to distinguish between signal correlations and noise correlations. Signal correlations are correlations in the average neural responses across all the stimuli; for example, neurons with very similar receptive fields or tuning curves have strong signal correlations. Noise correlations are correlations contained in trial-to-trial response variability; for example, neurons that are synchronized by reciprocal projections have strong noise correlations. The expansion of the mutual information accounts for both signal and noise correlations and divides the total information into the sum of four terms: a linear term (Ilin ), a signal-similarity term (Isig-sim ), a stimulus-independent correlation term (Icor-ind ), and a stimulus-dependent correlation term (Icor-dep ). The linear term measures the information conveyed by the neurons as if they were statistically independent from each other. The signal-similarity term quantifies the amount of redundancy in the information introduced by the similarity between neurons in their responses to the stimuli. The stimulus-independent correlation term represents information carried by correlations among the neuronal activity that are not dependent on the particular stimulus delivered. The stimulusdependent correlation term measures the information related to changes in neuronal correlations across stimuli. The sum of the last three terms of the expansion (Isig-sim + Icor-ind + Icor-dep ) is formally equivalent to the encoding measure of synergy/redundancy Isyn and quantifies the overall contribution of both signal and noise correlations to the mutual information. The sum of the last two terms of the expansion (Icor-ind + Icor-dep ) is formally equivalent to the shuffled term Ishuffle and quantifies the overall contribution of noise correlations and their interaction with signal correlations. Finally, the last term of the expansion Icor-dep is formally equivalent to the decoding measure I and depends on only noise correlations. Overall, the

2666

A. Scaglione et al.

Figure 1: Expansion of the mutual information. Information conveyed by neural responses R about the stimulus set S can be expanded as a sum of a linear term (Ilin ), that accounts for information conveyed (≥0) independently by the neurons, and a synergy/redundancy term (Isyn ), that accounts for information gained (≥0) or lost (≤0) if the neurons do not act independently. This last term (Isyn ) can be further expanded as a sum of a signal similarity term (Isig-sim ), which accounts for information lost (≤0) due to the similarity of the neurons’ tuning curves, and a shuffled information term (Ishuffle ), which accounts for information gained (≥0) or lost (≤0) due to the presence of correlations in the trial-to-trial variability of the neurons. Finally, this last term can be further expanded as a sum of a stimulus-independent correlation term (Icor-ind ) that accounts for information gained (≥0) or lost (≤0) due to the presence of stimulus-independent correlations, and a stimulus-dependent correlation term D) that accounts for information gained (≥0) due to the pres(Icor-dep , I or ence of stimulus-dependent correlations. The last term also represents an upper bound on the information lost by an optimal decoder that ignores correlations compared to an optimal decoder that considers correlations.

expansion of the mutual information offers a sophisticated and integrated tool to study the role of correlations in the neural code (see Figure 1). Two main expansions of the mutual information have been derived: the series expansion and the exact breakdown. To develop the series expansion, the expression of the total mutual information, or brute force, was first expanded using a Taylor series and then truncated after the second order (Panzeri, Schultz, et al., 1999; Panzeri, Treves, et al., 1999; Rolls, Treves, Tovee, & Panzeri, 1997). The series expansion is therefore valid only when the truncated Taylor series provides an accurate approximation of the total


2667

information, which is not always true in real neural data. To develop the exact breakdown, the average rates and pair-wise correlations used in the series expansion were replaced with full probability distributions, thereby eliminating all the assumptions that limited the application of the series expansion (Pola et al., 2003). When the assumptions of the second-order approximation are not verified, the series expansion and the exact breakdown obviously diverge. However, if the assumptions of the approximation are indeed verified, whether and when the two formalisms can be considered equivalent was not completely tested. 3 Series Expansion and Exact Breakdown of the Mutual Information We consider a neural population of C cells whose responses, after the presentation of a stimulus s in the time window [0, t], can be represented with the vector r(s, t). In the following, only the case of spike count code strategy will be considered, so that the vector r has C components ri (s, t) (i = 1, . . . , C), each one representing the firing rate of the i-th cell in the time window [0, t]. Thus, the relationship between the vector r(s, t) and the number of spikes ni (s, t) emitted by the cell i in the time window [0, t] can be defined as r(s, t) = [r1 (s, t) r2 (s, t) ri (s, t) =

···

ri (s, t)

···

rC (s, t)]

where (3.1)

ni (s) . t

Real neural populations display a stochastic behavior: they emit different responses after different presentations of the same stimulus. Therefore, we can model the response vectors r(s, t) as being drawn with a certain probability from the set R(s, t). R(s, t) thus represents the set of possible responses obtained when the stimulus s is presented to the neural population. The set of all possible responses R(t) is obtained when all the stimuli of the stimulus set S are presented to the neural population. Once the above quantities have been defined, the information that the set R of neural responses conveys about the set S of stimuli can be quantified using the equation of the mutual information (Cover & Thomas, 1991; Shannon & Weaver, 1949): I (R, S) =

s∈S r∈R

P(s)P(r | s) · log2

P(r | s) , P(r)

(3.2)

where P(r | s) and P(s) represent, respectively, the probability of obtaining the response r by the neural population given the stimulus s and the probability that the stimulus s has been drawn from the stimulus set S, and P(r) = s∈S P(s)P(r | s). The time dependency of each quantity was omitted in equation 3.2 for ease of representation.

2668

A. Scaglione et al.

3.1 The Series Expansion. The series expansion of the mutual information is obtained under the main assumption that the conditional probability P(r | s) of observing the neural response r given the stimulus s scales proportionally with time (Panzeri, Schultz, et al., 1999; Panzeri, Treves, et al., 1999; Rolls et al., 2003). In this way, the probabilities in equation 3.2 depend explicitly on time. The Taylor expansion, stopped at the second order, can then be applied to separate the mutual information, equation 3.2, into the sum of a linear term and three correlation terms. In order to consider the correlation contributions, it is necessary to introduce the signal correlation and noise correlation coefficients. The signal correlation coefficient is represented by νi j =

ni (s)n j (s)s − 1, ni (s)s n j (s)s

(3.3)

where ni (s) represents the number of spikes emitted by cell i when stimulus s is presented (see equation 3.1), the notation (. . .) indicates the average across the trials, and the notation (· · ·)s indicates the average across the stimulus set weighted by the prior probabilities P(s) of the stimuli. The noise correlation coefficient is represented by  ni (s)n j (s)    −1 if i = j   (ni (s)n j (s)) γi j (s) = (3.4)   (ni (s)2 − ni (s))   − 1 if i = j.  ni (s)2 Substituting the response probabilities into the equation of the mutual information, equation 3.2, and keeping terms only up to second order, the mutual information becomes I (R, S) = Ilin + Isig-sim + Icor-ind + Icor-dep , where Ilin =

C

ni (s) log2

i=1

Isig-sim =

ni (s) ni (s )s

(3.5)

s

C C 1 1 ni (s)s n j (s)s νi j + (1 + νi j ) ln ln 2 1 + νi j i=1 j=1

Icor-ind =

C C [ni (s)n j (s)γi j (s)s ] log2 i=1 j=1

Icor-dep =

1 1 + νi j

(3.6)

C C (1 + γi j (s))ni (s )n j (s )s ni (s)n j (s)(1 + γi j (s)) log2 . ni (s )n j (s )[1 + γi j (s )]s s i=1 j=1


2669

3.2 The Exact Breakdown. The exact breakdown of the mutual information is obtained from the series expansion by substituting the average rates and pair-wise correlations with the full response probabilities (Pola et al., 2003). Similar to the series expansion, the mutual information formula is expanded into the sum of a linear term and three correlation terms. However, this time, no assumptions need to be made about the real response probabilities of the neurons. The signal correlation and noise correlation coefficients need to be rewritten. It is first necessary to introduce Pind (r | s), which represents the statistically independent conditional response probability, and Pind (r), which is the independent probability to obtain the same response r considering it as stimulus-independent: Pind (r | s) =

C

P(ri | s)

(3.7)

i=1

Pind (r) = Pind (r | s)s .

(3.8)

The signal correlation coefficient is then expressed as  Pind (r)   P(ri ) = 0  P(r ) − 1 if i i i . ν(r) =   if P(ri ) = 0 0

(3.9)

i

The expression of the noise correlation coefficient becomes   P(r | s) − 1 γ (r | s) = Pind (r | s)  0

if Pind (r | s) = 0

.

(3.10)

if Pind (r | s) = 0

Once the above quantities have been defined, the expression of the exact breakdown becomes I (R, S) = Ilin + Isig-sim + Icor-ind + Icor-dep ,

(3.11)

where P(ri | s) Ilin = P(ri | s) log2 P(ri ) s ri i

1 1 Isig-sim = P(ri ) ν(r) + (1 + ν(r)) ln ln 2 r∈R i 1 + ν(r)

(3.12)

(3.13)

2670

A. Scaglione et al.

Icor-ind =

Pind (r | s)γ (r | s)s log2 r∈R

Icor-dep =

1 1 + ν(r)

(3.14)

Pind (r | s)(1 + γ (r | s))

r∈R

× log2

(1 + γ (r | s))Pind (r | s )s Pind (r | s )[1 + γ (r | s )]s

.

(3.15)

s

4 Comparison Between the Series Expansion and the Exact Breakdown Based on Analytical Considerations The series expansion and the exact breakdown, as presented in the previous sections, have several analogies: (1) they both converge to the mutual information expressed by equation 3.2 (Panzeri, Schultz, et al., 1999; Pola et al., 2003); (2) they expand the mutual information into a sum of four terms that have the same overall meaning and share the same mathematical formalism (Pola et al., 2003); and (3) they quantify the impact of correlations on the information by means of the signal correlation and noise correlation coefficients. Indeed, the series expansion is meaningful only when the assumptions of the second-order approximation are verified; otherwise it does not converge to the mutual information, thus diverging from the exact breakdown. The second-order approximation also implies the inability of the series expansion to consider correlations of order higher than two, which are instead considered by the exact breakdown. This fundamental difference was fully discussed in the original letter introducing the exact breakdown (Pola et al., 2003). A careful consideration of the basic mathematical definitions of signal correlation and noise correlation used in series expansion and in the exact breakdown reveals the presence of more subtle differences between the two formalisms, even when the assumptions of the second-order approximation are verified. Namely, the signal and noise correlation coefficients consider both count autocorrelations and cross-correlations in the series expansion, while they consider only cross-correlations neglecting count autocorrelations in the exact breakdown. In the following sections, unless otherwise noted, we use autocorrelation to mean “count autocorrelation.” 4.1 Differences in the Noise Correlation Coefficient. A difference exists in the noise correlation coefficient between the series expansion and the exact breakdown: the noise autocorrelation in the series expansion is zero only for particular spike count distributions, whereas in the exact breakdown, it is always zero. This can be easily seen in the case of one neuron. In the series expansion, the noise correlation coefficient for one neuron (see


2671

equation 3.4) can be rewritten as γii (s) =

var[n(s)] − n(s) + n(s) n(s)

2

2

− 1,

(4.1)

where var[n(s)] is the variance of the spike count distribution. Equation 4.1 shows that noise autocorrelations are present in the series expansion if the variance of the spike count distribution is different from its mean. Therefore, neurons that have their firing rate distributed according to a Poisson distribution will show no noise autocorrelations. Conversely, in the exact breakdown, the noise correlation coefficient for one neuron is always zero, because Pind (r | s) takes into account only deviation from the independency for cross-correlations between cells. Thus, if only one neuron is considered, Pind (r | s) equals P(r | s), always leading to a null value of the noise correlation coefficient (see equation 3.10). In summary, even when the assumptions of the second-order approximation are verified, the noise correlation coefficient of the series expansion differs from that of the exact breakdown when noise autocorrelations are present, that is, if the spike counts are not distributed accordingly to a Poisson distribution. 4.2 Differences in the Signal Correlation Coefficient. As obtained for the noise correlation coefficient, a similar difference can be found in the signal correlation coefficient between the series expansion and the exact breakdown. Namely, the signal autocorrelation in the series expansion is zero only for particular spike count distributions, whereas in the exact breakdown, it is always zero. Again, this can be easily seen in the case of one neuron. In the series expansion, the signal correlation coefficient for one neuron is null if and only if the average spike count is the same across all stimuli, that is, if n(s1 ) = n(s2 ) for every s1 , s2 = 1, . . . , s, . . . , S (see appendix A for a demonstration). Conversely, in the exact breakdown, the signal correlation coefficient for one neuron is always zero because Pind (r) takes into account only deviations from the independency for crosscorrelations between cells. Thus, if only one neuron is considered, Pind (r) equals i P(ri ), always leading to a null value of the signal correlation coefficient (see equation 3.9). In summary, even when the assumptions of the second-order approximation are verified, the signal correlation coefficient of the series expansion differs from that of the exact breakdown when signal autocorrelations are present, that is, when the average spike counts change across stimuli. Note that signal autocorrelations are not related to Poisson distributions. One might wonder whether these differences between the series expansion and the exact breakdown lead to appreciable differences in the individual terms. To address this issue, in the following we compared the

2672

A. Scaglione et al.

two formalisms of mutual information expansion on simulated data obtained with a stochastic neural network model that allows us to control the presence and strength of both autocorrelations and cross-correlations. 5 Comparison Between the Series Expansion and Exact Breakdown Based on Simulated Data 5.1 The Model. 5.1.1 Overall Architecture. In order to test and compare the individual terms of the exact breakdown and the series expansion, we created a stochastic neural model based on simulated response probabilities. The model consists of an analytically tractable three-neuron network (see Figure 2). Two neurons N1 and N2 are used to transmit information about the discrimination between two stimuli: they carry stimulus-related information by changing their average firing rate as a function of the stimuli. The third neuron Nn is used to provide shared noise; it represents a source of noise cross-correlations. The spike count distribution of each neuron in response to the stimuli was modeled as a stationary Poisson process. The rationale for choosing a Poisson process is that it is, by definition, free of count autocorrelation in the sense that the occurrence of a spike in a Poissonian spike train tells us nothing about the occurrence of the next one. The parameter of the Poisson distribution for each neuron, λ1 , λ2 , or λn , defines the average number of spikes per stimulus emitted in a fixed time window of length t after stimulus onset. The objective of the model was to construct the conditional probability distributions P(r | s) that provide the input for the mutual information expansion. To clarify the computational steps performed to construct P(r | s) in the presence of different types of correlation, we use a matrix notation in the following sections. Every simulation modeled the response of neurons to two stimuli, and therefore the two stimulus-related neurons, Ni , had two response distributions: one for each stimulus, with the corresponding parameter represented as λ11 and λ12 for the first stimulus and λ21 and λ22 for the second stimulus. For each stimulus, the two-dimensional (2D) joint probability distribution As (s = 1, 2), modeling the responses of the stimulus-related neurons, was constructed assuming independence between the two neurons: As (u, v) = P1 (u)P2 (v). Therefore, in As , the activity of one stimulusrelated neuron did not influence the activity of the second one, and there were no noise correlations. 5.1.2 Adding Noise Autocorrelations. In order to introduce noise autocorrelations in the model, we performed a manipulation of the 2D joint probability distributions of the stimulus-related neurons. In particular, the joint probability distributions As were truncated after two spikes emitted by each neuron, N1 and N2 , in response to the stimuli. The resulting 2D


2673

Figure 2: Stochastic neural model. The model consists of two stimulus-related neurons, N1 and N2 , and a third neuron Nn representing a common source of noise cross-correlation. Each of these neurons fires according to a stationary Poisson process with mean and variance λ1 , λ2 , or λn . Two stimuli are considered. λ11 , λ12 represent the average number of spikes emitted by neuron N1 and neuron N2 , respectively, when stimulus S1 is applied. λ21 , λ22 represent the responses of the same neurons when stimulus S2 is applied. Finally, λn represents the average number of spikes emitted by neuron Nn .

joint probability distributions were then normalized, creating a different stimulus-related square matrix, As-auto . It is worth reiterating that by autocorrelations, we are specifically referring to count autocorrelations, that is, correlations within the same bin in a spike count context, and not to timing autocorrelations, that is, correlations between bins within the same neuron in a spike timing context. We chose to truncate the Poisson distributions to add count autocorrelations for two reasons: (1) the series expansion explicitly considers count autocorrelations as deviations from the Poisson distribution, and (2) big deviations from this distribution can easily violate the assumptions of the series expansion and therefore would not allow us to compare the series expansion to the exact breakdown.

2674

A. Scaglione et al.

5.1.3 Adding Noise Cross-Correlations. We introduced noise crosscorrelations as a common noise source exciting both stimulus-related neurons. A 2D probability distribution (diagonal matrix N) was constructed for the third neuron, Nn , so that the elements of the principal diagonal equaled the distribution of the common noise source P(λn ): diag(N) = P(λn ). The joint distribution of the two neurons As (or As-auto ) was then convolved with the noise matrix N, producing a new stimulus-dependent square matrix Ans (or Ans-auto ). This procedure represents an extension to the trial-adding procedure used by Pola et al. (2003) to introduce cross-correlations between neurons. In fact, adding the trials of the noise neuron to the trials of both stimulus-related neurons is equivalent to convolving the 2D joint distributions of the two stimulus-related neurons with the 2D distribution of the noise neuron. 5.1.4 Using Mutual Information Expansions with the Model. The exact breakdown of the mutual information depends on the probability distributions of neural activity, so it could be directly applied to the model (i.e., using Ans or Ans-auto ). Conversely, the series expansion depends on three main variables that are estimated from the single trials of the neural activity: the mean firing rate (¯r ), the signal correlation coefficient (ν), and the noise correlation coefficient (γ ). In order to apply the approximated series expansion to the model, we rewrote the three variables above in a form that depends on only the marginal and joint probability distributions of the stimulus-related neurons (i.e., on Ans or Ans-auto ; refer to appendix B). We therefore eliminated the need for the series expansion to simulate single trials from the defined distributions. The main advantage of applying both the series expansion and the exact breakdown directly on probability distributions is that we avoid the problem of bias correction due to limited sample data (Golomb, Hertz, Panzeri, Treves, & Richmond, 1997; Panzeri, Senatore, Montemurro, & Petersen, 2007; Panzeri & Treves, 1996; Treves & Panzeri, 1995). Finally, to distinguish the effects due to autocorrelations from the effects due to the cross-correlations, we evaluated both the auto- and the cross-correlation parts of each second-order term of the series expansion as done by Rolls et al. (2004, 2003). 5.2 Simulations. 5.2.1 Regions of Convergence. In order to compare the individual terms between the series expansion and the exact breakdown, we defined a region of convergence in which the total information value (Itot ) of the series expansion differed by less than 1% compared to the exact breakdown. The


2675

Figure 3: Iso-information curves and region of convergence for the total information of the series expansion and the exact breakdown for four different correlation scenarios simulated. In each graph, the x-axis and y-axis represent the average number of spikes emitted by the neurons in response to stimulus S1 and S2 , respectively. The solid and dashed lines represent the total value of the mutual information evaluated with the exact breakdown and with the series expansions, respectively, as a function of the neurons’ average number of spikes in the two stimuli. For both of these lines, the information values are colorcoded, changing from black to white as information values increase; the text labels on each line of the exact breakdown give the value in bits. The white area within the plot shows the region of convergence (see methods for the definition). (A, B) Iso-information curves and regions of convergence obtained for the reduced one-neuron model in absence (A) and presence (B) of noise autocorrelations. (C, D) Iso-information curves and regions of convergence obtained for the two-neuron model with positive signal cross-correlations (λ11 = λ12 and λ21 = λ22 ), in the absence (C) and presence (D) of noise autocorrelations. Note that both introducing noise autocorrelations and increasing the number of neurons reduced the region of convergence.

acceptable error (1%) was chosen to be more stringent compared to previous work on real neural data (Petersen et al., 2001). All comparisons between the series expansion and the exact breakdown were performed based on the differences observed on the border of the region of convergence. We considered five cases: (1) a reduced one-neuron model without truncation (impact of signal autocorrelations; see Figure 3A); (2) a reduced

2676

A. Scaglione et al.

one-neuron model with truncation (impact of noise autocorrelations; see Figure 3B); (3) the two-neuron model without truncation (impact of signal cross-correlations; (see Figure 3C); (4) the two-neuron model with truncation (integration between signal cross-correlations and noise autocorrelations; see Figure 3D); and (5) the two-neuron model with common noise (impact of noise cross-correlations). The last case could not be displayed in a 2D plot because of the additional degree of freedom given by the firing rate of the noise neuron; therefore, a representative projection was chosen. In all cases, we tested whether the four terms Ilin , Isig-sim , Icor-ind , and Icor-dep differed between the series expansion and the exact breakdown within the region of convergence and quantified the differences on the border. 5.2.2 Impact of Signal Autocorrelations. To study the impact of signal autocorrelations, we considered a reduced version of the model with only one of the two stimulus-related neurons, and we set the noise neuron to 0 so that Ans = As . Note that in this reduced one-neuron model, As is a vector rather than a square matrix. The average rate λ of the stimulus-related neuron was set to range between 0 and 5 spikes per stimulus in both stimuli. The simulations of the one-neuron model without truncation (see Figure 4A) confirmed that the series expansion and the exact breakdown provide a different representation of signal autocorrelations. In this reduced model, signal autocorrelations are the only type of correlations possibly present, and, as described above, they are not present only when the average firing rate of the neuron is the same for the two stimuli (see appendix A), that is, when the total information is zero. Accordingly, in the series expansion, the noise correlation terms Icor-ind and Icor-dep were zero, and the presence of signal autocorrelations was detected by the signal similarity term Isig-sim . Conversely, in the exact breakdown, all three correlation terms (Isig-sim , Icor-ind and Icor-dep ) were zero. The average difference in the Isig-sim term between the series expansion and the exact breakdown, on the border of the region of convergence, was 14.85 ± 13.28% of the total information. This difference was compensated for by an equivalent but opposite difference between the two expansions in the linear term Ilin (13.87 ± 13.28% of the total information). Therefore, in presence of signal autocorrelations, even when there is no significant difference in the total information, the linear and signal similarity terms of the exact breakdown can substantially differ from the corresponding terms of the series expansion. 5.2.3 Impact of Noise Autocorrelations. To study the impact of noise autocorrelations, we used the same one-neuron model as above, but we truncated the number of spikes per stimulus after two spikes emitted by the neuron (Ans-auto = As-auto ). Indeed, As-auto is also a vector in this reduced model. After the truncation, the average rate of the neuron ranged between 0 and 1.62 spikes per stimulus.


2677

Figure 4: Impact of signal and noise autocorrelations. Total value (left two panels) and single terms (six panels on the right) of the exact breakdown (solid lines) and the series expansion (dashed lines) in the absence (A) and presence (B) of noise autocorrelations when only one stimulus-related neuron is considered in the model (C). In each graph, the x-axis represents the average number of spikes emitted by the neuron for the first stimulus (λ11 ), whereas the y-axis represents the information extracted by the two expansions in bits. For illustration purposes, the average number of spikes emitted in response to stimulus 2 (λ21 ) was set to a fixed value in order provide representative examples of the differences between the two formalisms (λ21 = 0.48 spikes/stimulus in A, 0.24 spikes/stimulus in B). The two formalisms should be compared within the region of convergence (white areas), whereas only the exact breakdown is meaningful outside the region of convergence (gray areas).

The simulations in the one-neuron model with truncation (see Figure 4B) confirmed that the series expansion and the exact breakdown also provide a different representation of noise autocorrelations. In this reduced model, both signal and noise autocorrelations are present. Accordingly, in the series expansion, the presence of signal autocorrelation was again detected by the signal similarity term Isig-sim , and the presence of noise autocorrrelation was detected by the stimulus-independent correlation term Icor-ind . Conversely, in the exact breakdown, all three correlation terms (Isig-sim , Icor-ind , and Icor-dep ) were still equal to zero. The average differences in the Isig-sim and Icor-ind terms between the series expansion and the exact breakdown on the border of the region of convergence were 10.42 ± 4.65% and 28.50 ± 8.50% of the total information, respectively. Note that the differences in the two terms had opposite signs. These differences were compensated for by an equivalent difference between the series expansion and the exact breakdown in the linear term (19.32% ± 4.26% of the total information, close to

2678

A. Scaglione et al.

28.50% − 10.42%). Similar results were obtained when stimulus-dependent autocorrelations were introduced by truncating the responses up to two spikes for one stimulus and up to one spike for the other stimulus (data not shown): in the series expansion, stimulus-dependent autocorrelations were detected by the stimulus-dependent correlation term Icor-dep , whereas in the exact breakdown, Icor-dep remained equal to zero, and the difference was compensated for by the linear term. These results confirm and quantify the important discrepancy between the series expansion and the exact breakdown: they have a different representation of autocorrelations. 5.2.4 Impact of Signal Cross-Correlations. To study the impact of signal cross-correlations, we considered both stimulus-related neurons, but we set the noise neuron to 0 so that Ans = As (square matrix). Four independent variables were then used to describe this two-neuron model: the average number of spikes per stimulus of two neurons responding to each of two stimuli. To obtain two-dimensional representations without loss of generality, we reduced the number of independent variables from four to two by introducing positive signal correlations, imposing λs1 = λs2 (s = 1, 2). The average rate of the neurons was set to range between 0 and 5 spikes per stimulus. The simulations using the two-neuron model without truncation (see Figure 5A) revealed that the series expansion and the exact breakdown provide the same representation of signal cross-correlations, but individual terms still diverge due to the inescapable presence of signal autocorrelations. The average difference in the Isig-sim term between the series expansion and the exact breakdown on the border of the region of convergence was 6.85% ± 4.85% of the total information, but this difference was entirely due to signal autocorrelations. In fact, the Isig-sim term of the exact breakdown matched the cross-correlation part of the corresponding term of the series expansion (see Figure 5A inset). A difference in the Ilin term (5.9% ± 4.85%) compensated, as always, for the difference in the Isig-sim term. Therefore, despite the different representation of autocorrelations, the series expansion and the exact breakdown have the same representation of signal crosscorrelations. 5.2.5 Interaction Between Signal Cross-Correlations and Noise Autocorrelations. To study the possible interaction between signal cross-correlations and noise autocorrelations, we used the same two-neuron model with the noise neuron set to zero, as above, and we introduced noise autocorrelations by truncating the number of spikes per stimulus after two spikes emitted by the neurons (Ans = As-auto , square matrix). In the case of positive signal correlations, after the truncation, the average firing rate of the neurons ranged between 0 and 1.62 spikes per stimulus. The simulations using the two-neuron model with truncation (see Figure 5B) did reveal a tricky


2679

Figure 5: Impact of signal cross-correlations and interaction between signal cross-correlations and noise autocorrelations. Total value and single terms of the exact breakdown (solid lines) and series expansion (dashed lines) in the absence (A) and presence (B) of noise autocorrelations when two stimulus-related neurons are considered (C). See Figure 4 for a description of the axes. Positive signal cross-correlations were imposed (λ11 = λ12 and λ21 = λ22 ). As in Figure 4, for illustration purposes, the average number of spikes emitted in response to stimulus 2 (λ21 ) was set to a fixed value in order to provide representative examples of the differences between the two formalisms (λ21 = 0.5 spikes/stimulus in A, 0.42 spikes/stimulus in B). The insets show the exact breakdown (solid lines) and the cross-correlation component of the approximated series expansion (crossed lines). As in Figure 4, white areas are within the region of convergence, and gray areas are outside the region of convergence.

interaction between signal cross-correlations and noise autocorrelations. In the series expansion, the Icor-ind term detected noise autocorrelations, and the Isig-sim term detected both signal autocorrelations and signal crosscorrelations, consistent with the previous simulations. In the exact breakdown, one might expect the Icor-ind term to be zero and the Isig-sim term to match the cross-correlation part of the corresponding series expansion term. The first expectation was indeed verified, but the second one was not. Namely, the average difference between the Isig-sim term of the exact breakdown and the cross-correlation part of the series expansion Isig-sim term was 8.25% ± 5.57% of the total information (see Figure 5B, inset). Note that the average difference is relatively subtle, but it could be as high as 17.19% of the total information along the border of the region of convergence, and on average it represents 48.23 ± 14.28% of the signal similarity detected by the exact breakdown.

2680

A. Scaglione et al.

This puzzling result that the Isig-sim term of the exact breakdown did not match the cross-correlation part of the corresponding series expansion term can be intuitively explained as follows. The absence of noise autocorrelation information in the Icor-ind term of the exact breakdown was compensated for by the Ilin term, which considers the two neurons independently. Any redundancy in the noise autocorrelation information between the two neurons was therefore transformed into redundancy in the linear information, which was captured by the Isig-sim term of the exact breakdown. The Isig-sim term of the exact breakdown therefore overestimated the redundancy that would be exclusively expected from signal cross-correlations. Information related to autocorrelations is therefore explicitly represented by the correlation terms in the series expansion, whereas it is implicitly represented by the linear term leaking into the signal similarity term in the exact breakdown. 5.2.6 Impact of Noise Cross-Correlations. So far we have shown that the series expansion and the exact breakdown have a fundamentally different representation of both signal and noise autocorrelations. Conversely, they have the same representation of signal cross-correlations. We extended the equivalence between the series expansion and the exact breakdown to all cross-correlations (i.e., both signal and noise cross-correlations) by performing simulations with the full two-neuron model with common noise (Ans ), in the presence of positive signal correlation (see Figure 6). To obtain 2D representations, we fixed the firing rates of the stimulus-related neurons to representative informative values (λ11 = 0.4 spikes per stimulus and λ21 = 0.1 spikes per stimulus, and the other neuron calculated according to the imposed signal correlation). The noise neuron was set to range between 0 and 0.4 spike per stimulus. In separate simulations, noise autocorrelations were also introduced by truncating the number of spikes per stimulus after two spikes emitted by the neurons (Ans-auto ). Similar to what was found when studying the impact of signal crosscorrelations, we observed significant differences in the Ilin and Isig-sim terms. Namely, the difference in the Isig-sim term between the series expansion and the exact breakdown on the border of the region of convergence was 3.7% of the total information. Note that because the region of convergence changes as the firing rate of the noise neuron changes, these differences are not average values but representative values on the minimal region of convergence in the chosen projection. Consistent results were obtained in all projections tested. The differences were entirely due to the different representation of signal autocorrelations. In fact, the Isig-sim term of the exact breakdown matched the cross-correlation part of the corresponding term of the series expansion (see Figure 6 insets). As expected, the difference in Isig-sim was compensated with a difference in the in the Ilin term (3.9%). The remaining two second-order terms, Icor-ind and Icor-dep , were similar between the series expansion and the exact breakdown; that is, their difference on the border of the region of convergence was smaller than the accepted error (1% of


2681

Figure 6: Impact of noise cross-correlations. (A) Total value and single terms of the exact breakdown (solid lines) and series expansion (dashed lines) in the presence of noise cross-correlations when two stimulus-related neurons and the common noise neuron are considered (B). Positive signal cross-correlations were imposed (λ11 = λ12 and λ21 = λ22 ) In each graph, the x-axis represents the average number of spikes λn emitted by the common noise neuron Nn , which is stimulus independent, and the y-axis represents the information values extracted by the two expansions in bits. Similar to the previous figures, for illustration purposes, the average number of spikes emitted by the stimulus-related neurons N1 and N2 was set to fixed values in order to provide a representative example of the comparison between the two expansions as a function of the firing rate of the noise neuron. The insets show the exact breakdown (solid lines) and the crosscorrelation component of the approximated series expansion (crossed lines). White areas are within the region of convergence, and gray areas are outside the region of convergence.

the total information). When we introduced noise autocorrelations, Icor-ind and Icor-dep differed between the series expansion and the exact breakdown respectively by 9.6% and 3.5% of the total information (data not shown). Importantly, Icor-ind and Icor-dep of the exact breakdown matched the crosscorrelation part of the corresponding terms of the series expansion (data not shown). Overall, these results suggest that despite the different representation of autocorrelations, the series expansion and the exact breakdown have the same representation of cross-correlations.

2682

A. Scaglione et al.

6 Poisson Exact Breakdown of the Mutual Information The exact breakdown of the mutual information quantifies the specific contributions of cross-correlations to the total information carried by a neural population about the stimulus set. If one is interested in the impact of autocorrelations, the series expansion of the mutual information becomes the only tool available. However, the series expansion has stringent assumptions that need to be verified and limit its application to real neural data. Here we propose a new formalism of mutual information expansion that using Poisson equivalents explicitly quantifies the impact of noise autocorrelations within the framework of the exact breakdown. 6.1 Partial Poisson Exact Breakdown: The Contribution of Noise Autocorrelations. In order to evaluate the contribution of noise autocorrelations to the mutual information with no approximation involved, we define Poisson equivalents of the linear and signal similarity terms of the exact breakdown. These Poisson equivalents are obtained from the exact breakdown (see equation 3.11) by substituting the real neural probability distributions with Poisson distributions. Poisson equivalents have the same mean of the real probability distributions for each cell c. Given the mean µi of the real ˆ i ) is probability distribution P(ri ), the Poisson equivalent distribution P(r obtained by using

ˆ i = k) = P(r

µik e−µi . k!

(6.1)

We then define the signal correlation coefficient as

νˆ (r) =

 Pˆ ind (r)   −1  ˆ  P(r )

if

i

ˆ i ) = 0 P(r ,

i

i

   0

if

(6.2)

ˆ i) = 0 P(r

i

ˆ i ) are obtained in the same way as in the exact breakwhere Pˆ ind (r) and P(r down but this time using Poisson equivalent distributions rather than real probability distributions. Once the above quantities have been introduced, we define the partial Poisson exact breakdown as I (R | S) = Iˆlin + Iˆsig-sim + Icor-ind + Icor-dep + Iˆshuffle-auto ,

(6.3)


2683

where Icor-ind and Icor-dep have the same expression defined in the exact breakdown (see equations 3.14 and 3.15) and the other terms are defined by Iˆlin =

i

ri

ˆ i | s) log P(r 2

ˆ i | s) P(r ˆ i) P(r

(6.4) S

1 1 ˆ i ) νˆ (r) + (1 + νˆ (r)) ln Iˆsig-sim = P(r ln 2 r∈R i 1 + νˆ (r) Iˆshuffle-auto = Ilin + Isig-sim − ( Iˆlin + Iˆsig-sim ).

(6.5) (6.6)

It can be easily proved that the sum of the above terms leads to the mutual information and that the ν(r) ˆ respects the same normalization constrain required for ν(r) (Pola et al., 2003). The linear term of the Poisson exact breakdown Iˆlin quantifies the information carried by the neurons when all spikes are considered as independent, assuming absence of cross-correlations and noise autocorrelations. The absence of noise autocorrelations is guaranteed by the employment of Poisson equivalent distributions. Note that Poisson equivalent distributions do not change the means and therefore do not modify what the series expansion considers as signal autocorrelations. The linear term of the Poisson exact breakdown is therefore the exact extension of the linear term plus the autocorrelation part of the signal similarity term of the series expansion. The signal similarity term of the Poisson exact breakdown Iˆsig-sim quantifies the redundancy introduced exclusively by the presence of signal crosscorrelations. The employment of Poisson equivalent distributions ensures that noise autocorrelations do not leak into the signal similarity term as they did in the exact breakdown. The signal similarity term of the Poisson exact breakdown is therefore the exact extension of the cross-correlation part of the signal similarity term of the series expansion. Finally the noise autocorrelation term Iˆshuffle-auto quantifies the overall redundancy or synergy introduced by noise autocorrelations. It represents the exact extension of the autocorrelation part of the stimulus-independent correlation term plus the autocorrelation part of the stimulus-dependent correlation term of the series expansion. 6.2 Validation of the Partial Poisson Exact Breakdown on Simulated Data. To validate the partial Poisson exact breakdown, we applied it to simulated data in two different conditions: (1) presence of noise autocorrelations and (2) presence of interaction between noise autocorrelations and signal cross-correlations. The data were simulated using the stochastic neural model previously described, with the same parameters used for the simulations presented in sections 5.2.3 and 5.2.5. To validate the partial

2684

A. Scaglione et al.

Figure 7: Validation of the partial Poisson exact breakdown with the one-neuron model in the presence of noise autocorrelations. In each graph, the x-axis and y-axis represent the average number of spikes emitted by the neuron in response to stimuli 1 and 2, respectively. The white area represents the region in which the considered terms differ by at most 1% of the total information. (A) Region of convergence for the total information. (B) Region of convergence between the linear term of the Poisson exact breakdown and the sum of the linear and the autocorrelation part of the signal similarity term of the series expansion. (C) Region of convergence between the noise autocorrelation term of the partial Poisson exact breakdown and the sum of the autocorrelation parts of the stimulus-independent and stimulus-dependent terms of the series expansion.

Poisson exact breakdown, we compared the individual terms with the corresponding combinations of terms of the series expansion; then we verified that (1) corresponding terms had the same overall behavior and (2) the region of convergence for individual terms was similar to (or larger than) the region of convergence for the total information. The maximum error allowed was always 1% of the total information. 6.2.1 Validation in the Presence of Noise Autocorrelations. The simulation obtained with the one-neuron model with truncation (see Figure 7) validated the individual terms of the Poisson exact breakdown. In this reduced model, both signal autocorrelations and noise autocorrelations are present. As expected, the linear term of the Poisson exact breakdown Iˆlin did not consider the contribution of noise autocorrelations and included only the autocorrelation part of the signal similarity of the series expansion. In fact, the region of convergence for this term with the corresponding combination of terms of the series expansion (see Figure 7B) was much greater than the region of convergence for the total information (see Figure 7A). The contribution of noise autocorrelations was perfectly estimated by the noise


2685

Figure 8: Validation of the partial Poisson exact breakdown with the twoneuron model in the presence of both noise autocorrelations and signal cross-correlations. See Figure 7 for a description of the axes. (A) Region of convergence for the total information. (B) Region of convergence between the linear term of the Poisson exact breakdown and the sum of the linear and the autocorrelation part of the signal similarity term of the series expansion. (C) Region of convergence between the signal similarity term of the Poisson exact breakdown and the cross-correlation part of the signal similarity term of the series expansion. (D) Region of convergence between the noise autocorrelation term of the partial Poisson exact breakdown and the sum of the autocorrelation parts of the stimulus-independent and stimulus-dependent terms of the series expansion.

autocorrelation term Iˆshuffle-auto . In fact, the region of convergence for this term with the corresponding combination of terms of the series expansion (see Figure 7C) was virtually identical to the region of convergence for the total information (see Figure 7A). 6.2.2 Validation in the Presence of Interaction Between Noise Autocorrelations and Signal Cross-Correlations. The simulation obtained with the twoneuron model with truncation (see Figure 8) also validated the individual terms of the Poisson exact breakdown. In this reduced model, both signal and noise autocorrelations are present together with the presence of signal cross-correlations. As expected, again, the linear term of the Poisson exact breakdown Iˆlin did not consider the contribution of noise autocorrelations and included only the autocorrelation part of the signal similarity of the series expansion. In fact, the region of convergence for this term with the corresponding combination of terms of the series expansion (see Figure 8B) was much greater than the region of convergence for the total information (see Figure 8A). More important, the signal similarity term of the Poisson exact breakdown Iˆsig-sim was specific for the redundancy induced by signal

2686

A. Scaglione et al.

cross-correlations without any leaking from noise autocorrelations. In fact, the region of convergence for this term with the cross-correlation part of the signal similarity term of the series expansion (see Figure 8C) was remarkably large and almost entirely included the region of convergence for the total information (see Figure 8A). Finally, the noise autocorrelation term Iˆshuffle-auto provided the expected estimate of the overall contribution of noise autocorrelations. In fact, the region of convergence for this term with the respective combination of terms of the series expansion (see Figure 8D) was remarkably similar to the region of convergence for the total information (see Figure 8A). 6.3 Full Poisson Exact Breakdown: The Contribution of StimulusDependent Noise Autocorrelations. The noise autocorrelation term Iˆshu f f le−a uto of the partial Poisson exact breakdown quantifies the overall contribution of noise autocorrelations to the total information but is not able to differentiate between stimulus-independent and stimulus-dependent noise autocorrelations. To overcome this limitation, we propose a pseudoequivalent Poisson representation Iˆcor-dep of the stimulus-dependent correlation term. The first step is to define the pseudo-equivalent noise correlation coefficient γˆ (r | s) as   P(r | s) − 1 for Pˆ (r | s) = 0 ind γˆ (r | s) = Pˆ ind (r | s) , (6.7)  0 for Pˆ ind (r | s) = 0 where Pˆ ind (r | s) is obtained substituting the real neural probability distributions in equation 3.7 with their Poisson equivalent distributions. Once the above quantity has been introduced, we define the full Poisson exact breakdown: I (R | S) = Iˆlin + Iˆsig-sim + Icor-ind + Iˆcor-ind-auto + Icor-dep + Iˆcor-dep-auto ,

(6.8)

where Iˆlin and Iˆsig-sim have been defined above and the other terms are expressed by Iˆcor-dep = Pˆ ind (r | s)(1 + γˆ (r | s)) r∈R

× log2

(1 + γˆ (r | s)) Pˆ ind (r | s )S Pˆ ind (r | s )[1 + γˆ (r | s )]S

(6.9) S

Iˆcor-dep-auto = Iˆcor-dep − Icor-dep Iˆcor-ind-auto = Iˆshuffle-auto − Iˆcor-dep-auto .

(6.10)


2687

As for the partial Poisson exact expansion, it can be easily proved that equation 6.8 converges to the mutual information and that γˆ (r | s) respects the same normalization constraints required for γ (r | s) (Pola et al., 2003). The stimulus-dependent autocorrelation term Iˆcor-dep-auto conceptually corresponds to the autocorrelation part of the stimulus-dependent correlation term of the series expansion. However, we could not validate this claim on simulated data, because the magnitude of Iˆcor-dep-auto in simulations that respect the assumptions of the series expansion is often of the same order of the allowed error, making the regions of convergence meaningless. Nonetheless, this term has a precise mathematical meaning, because Iˆcor-dep represents the average across neuronal responses of the Kullback-Leibler divergence between the probability of having received a particular stimulus given the real neural responses and the probability of having received the same stimulus given the Poisson equivalent shuffled neural responses. Iˆcor-dep can therefore be interpreted from a decoding perspective, as it estimates the amount of information that is lost by a decoder that ignores both cross-correlations and noise autocorrelations. As a final remark, the Poisson exact breakdown is intuitively related to the general information-theoretic formalism recently introduced by Montemurro, Senatore, and Panzeri (2007). However, our Pˆ ind (r | s) does not verify the assumptions required to construct the simplified models of correlations Psimp (r | s) in Montemurro’s formalism. Their formalism was indeed inspirational for this work but cannot be used to readily derive the Poisson exact breakdown. 6.4 Proof of Concept of the Poisson Exact Breakdown: Pooling and Binning. To provide a proof of concept of the Poisson exact breakdown, we applied it on simulated data representing two widely employed manipulations of spike trains that alter the structure of count autocorrelations: pooling and binning. Pooling consists of summing the activity of different neurons into a single variable (see, e.g., Panzeri, Petroni et al., 2003). Binning consists of dividing the time window containing the neural responses into smaller bins. Mathematically, binning and pooling are closely related: the neural activity in the “unbinned” time window is nothing but the pooled activity of individual bins. Exploiting this relation, we performed a simulation implementing the pooling and used the same simulation to discuss the binning. We used the stochastic neural model previously described to simulate the activity of two neurons in the presence of noise cross-correlations. We employed the full two-neuron model with common noise (Ans ) and positive signal correlation, with the same parameters as in section 5.2.6. To pool the activity of the two neurons into a single variable, we collapsed the joint conditional probabilities into pooled conditional probabilities by summing the antidiagonals of Ans . We studied the behavior of all terms of the full Poisson exact breakdown as a function of the common noise (see Figure 9A)

2688

A. Scaglione et al.

Figure 9: Proof of the concept of the full Poisson exact breakdown of the mutual information. (A) Total information and individual terms of the full Poisson exact breakdown with the two-neuron model with common noise, after pooling the responses of the two neurons. The x-axis represents the average number of spikes emitted by the common noise neuron λn , while the y-axis represents the amount of information in bits extracted by the terms of the full Poisson exact breakdown. (B) Comparison of individual terms of the full Poisson exact breakdown between the two-neuron model and the pooled model in the absence or presence of common noise. Note that the same simulation can be interpreted from a binning perspective, considering the two correlated neurons as two correlated bins of the same neuron.

and quantified the differences between the two-neuron model and the pooled model in absence and presence of common noise (see Figure 9B). In the absence of common noise, the total information was the same in the two-neuron model and in the pooled model. In the presence of common noise, as expected, the total information was smaller in the pooled model than in the two-neuron model, and the Poisson exact breakdown uncovered the following relationships: (1) Iˆlin in the pooled model was identical to Iˆlin + Iˆsig-sim in the two-neuron model; (2) Iˆcor-ind-auto in the pooled model was identical to Icor-ind in the two-neuron model; and (3) Iˆcor-dep-auto in the pooled


2689

model was present but was smaller than Icor-dep in the two-neuron model, the difference corresponding exactly to the difference of total information between the two models. The Poisson exact breakdown is therefore able to quantify in informationtheoretic terms the transformation, imposed by pooling, of noise crosscorrelations between neurons into count autocorrelations. Because pooled distributions are much better sampled than joint distributions, the Poisson exact breakdown combined with pooling can be used to obtain rigorous lower bounds of the role of noise cross-correlations in the information conveyed by large populations of neurons. Similar considerations arise if the pooling simulation is interpreted from a binning perspective, that is, if the two correlated neurons are considered as two correlated bins of the same neuron. In this case, binning imposes a transformation of count autocorrelations into noise cross-correlations between bins. The Poisson exact breakdown is therefore able to exactly distinguish all the components of the additional information that is conveyed by spike timing compared to spike count. 7 Discussion Our results confirm that the series expansion (Panzeri, Schultz, et al., 1999) and the exact breakdown (Pola et al., 2003) provide a substantially equivalent representation of cross-correlations. However, the two formalisms have a different representation of count autocorrelations, so that even when the total information estimated by the two formalisms differs by less than 1%, individual terms can diverge. The correlation terms of the exact breakdown are basically blind to count autocorrelations, which are instead included in the linear term, with a subtle leakage into the signal-similarity term. Conversely, the series expansion explicitly considers count autocorrelations in all second-order terms. We proposed a new formalism of mutual information expansion, the Poisson exact breakdown, which introduces Poisson equivalents in order to explicitly evaluate the informational contribution of count autocorrelations with no approximation involved. Because several widely employed manipulations of spike trains, most notably binning and pooling, alter the structure of count autocorrelations, the new formalism can provide a useful general framework for studying the role of correlations in population codes. 7.1 Considerations about the Bias Problem. The model we used to perform comparisons on simulated data and to validate the new formalism of mutual information expansion is based on simulations of probability distributions rather than simulation of single trials. Both the exact breakdown and the Poisson exact breakdown depend on only the probability distributions of neural activity (Pola et al., 2003), so they could be directly applied to the model. Conversely, the series expansion depends on variables that are

2690

A. Scaglione et al.

estimated from the single trials of the neural activity (Panzeri, Schultz, et al., 1999; Petersen et al., 2001); therefore, these variables were rewritten in a form that depended on only the probability distributions (see appendix B). The effort of applying all formalisms directly on probability distributions offered the advantage of avoiding the need to correct the mutual information expansion for the presence of the positive upward bias (Panzeri et al., 2007; Panzeri & Treves, 1996; Treves & Panzeri, 1995). All our results and conclusions are therefore bias free. Of course, appropriate bias corrections should be adopted when applying these methods to study the neural code with neurophysiological data (Golomb et al., 1997; Panzeri et al., 2007; Panzeri & Treves, 1996; Treves & Panzeri, 1995). Although bias correction is beyond the scope of this letter, it should be noted that the new terms introduced in the Poisson exact breakdown have good bias properties because they require the estimation of only the mean rather than of the entire probability distribution of spike counts. 7.2 Using Poisson Distributions. In this letter, we made wide use of Poisson distributions for both comparing the series expansion and the exact breakdown on simulated data and defining the Poisson exact breakdown. Poisson distributions were chosen not for simplicity, but because of their elegant and exclusive mathematical property of being, by definition, free of count autocorrelations. On the one hand, we took advantage of this property to specifically compare the series expansion and the exact breakdown in the absence of noise count autocorrelations and to introduce noise count autocorrelations as small departures from the Poisson assumptions (by means of truncation). Poisson distributions therefore allowed us to quantify the basic difference between the series expansion and the exact breakdown in their representation of autocorrelations. On the other hand, the property of Poisson distributions being free of count autocorrelation offered the convenient opportunity to estimate the contribution of count autocorrelations simply by substituting full probability distributions with Poisson equivalents in the individual terms of the exact breakdown. The terms introduced in the new formalism are therefore parametric with only one degree of freedom (the mean of the distribution) and have clear mathematical interpretations related to the Poisson assumptions. 7.3 Autocorrelations, Spike Count, and Spike Timing. As presented in section 1, there are two main types of autocorrelations: count autocorrelations and timing autocorrelations. Here we focused on count autocorrelations—correlations within the same bin in a spike count context. Nonetheless, the extension from spike count to spike timing (Panzeri & Schultz, 2001; Pola et al., 2005) is conceptually similar to the extension from one neuron to two neurons: in both cases, the extension consists of adding variables (bins or neurons). Thus, our simulations with two neurons plus noise can be conceptually interpreted as simulations with one neuron


2691

and two correlated bins. Hence, our results that the series expansion and the exact breakdown have the same representation of noise cross-correlations also support that the two formalisms have the same representation of timing autocorrelations. Consequently, the Poisson exact breakdown can be easily extended to evaluate the informational contribution of both count and timing autocorrelations. The ability of the Poisson exact breakdown to consider both count autocorrelations and timing autocorrelations is valuable not only when count autocorrelations are the measure of interest, but also when using manipulations of spike trains that alter the count autocorrelation structure of the data—most notably, binning and pooling. On one hand, binning is widely used in the analysis of spike trains and can produce a complex interaction between count autocorrelations and timing autocorrelations when it is performed with different bin size on the same data. This is typically performed when the contribution of spike timing to the neural code is compared to the contribution of spike count (Foffani & Moxon, 2004; Foffani, Tutunculer, & Moxon, 2004; Foffani, Chapin, & Moxon, 2008; Ghazanfar, Stambaugh, & Nicolelis, 2000; Montemurro, Senatore, et al., 2007; Nicolelis et al., 1998; Panzeri, Petersen, Schultz, Lebedev, & Diamond, 2001; Petersen et al., 2001; Pola et al., 2005). On the other hand, pooling has also been applied on neurophysiological data (Panzeri, Petroni et al., 2003) and is particularly appealing for studying the role of correlations in large populations of neurons because this manipulation transforms count cross-correlations between neurons into count autocorrelations. The Poisson exact breakdown can therefore provide a useful general framework for studying the role of correlations in population codes. Appendix A: Zeros of the Signal Autocorrelation Coefficient of the Series Expansion Given the stimulus set S = {s = 1, . . . , S}, let q s be the number of times the stimulus s is presented to the neural population. Thus, Q = s q s represents the total number of trials collected. The distribution of the stimulus presentations is then given by P(s = k) = qQk . Therefore, we can rewrite the signal autocorrelation coefficient (see equation 3.4) as νii =

2 qs s Q n (s)

qs s Q n (s)

qs s Q n (s)

− 1,

(A.1)

where the averages across the stimulus set are expanded and the P(s) have been replaced with the distribution of stimulus presentations. The zeros of this coefficient can be found by analyzing the zeros of the numerator of

2692

A. Scaglione et al.

equation A.1. It can be rewritten as qs s

Q

2

n(s) −

qs s

Q

n(s)

qs Q

s

n(s) = 0.

(A.2)

Defining the two auxiliary indices s1 , s2 = 1, . . . , s, . . . , S and with some algebra, the above term can be written as s1

2

q s1 q s2 n(s1 ) (1 − δs1 s2 ) −

s2

s1

q s1 q s2 n(s1 )n(s2 )(1 − δs1 s2 ) = 0,

s2

(A.3) where δs1 s2 represents the Kronecker delta that assumes the value 1 when s1 = s2 and 0 elsewhere. After collecting the terms with the same q s1 q s2 coefficient, equation A.3 becomes s1

q s1 q s2 (n(s1 ) − n(s2 ))2 (1 − δs1 s2 ) = 0.

(A.4)

s2

Since the term of the two summations is always positive, equation A.4 is verified if and only if n(s1 ) = n(s2 ) for every s1 , s2 = 1, . . . , s, . . . , S with s1 = s2 . Therefore, the signal autocorrelation coefficient is zero for the considered neuron if and only if this neuron keeps the same average number of spikes for every stimulus. Appendix B: Calculation of the Series Expansion from the Probability Distributions of the Neurons The unbiased values of each term of the series expansion can be obtained rewriting the three basic measures that need to be estimated to calculate the individual terms of this expansion: the average number of spikes emitted in the window T = [0, t] when the stimulus s is presented (n¯ i (s)), the signal correlation coefficient (νi, j ), and the noise correlation coefficient (γi, j (s)). The calculation of the average number of spikes from the probability distributions is straightforward. Let p(ni = n | s) be the firing probability distribution of the neuron Ni (i = 1 . . . C) for the stimulus s (s = 1, . . . , S) with n ∈ N, where N represents the set of all possible responses emitted by the population of cells. Therefore, the average number of spikes emitted by the neuron i in the time window T when the stimulus s is presented is given by ni (s) =

n∈N

p(ni = n | s) · n.

(B.1)


2693

Using the above formula, the signal correlation coefficient (νi, j (s)) for each couple of cells i,j(i,j = 1, . . . , C) is then given by νi, j =

s∈S

ni (s) · n j (s) · p(s) , ni · n j

(B.2)

where n¯ i (s) or n¯ j (s) is obtained with equation B.1 and ni = s∈S ni (s) · p(s). Finally, the gamma correlation coefficient (γi, j (s)) for each couple of cells i,j(i,j = 1, . . . , C) is given by  2  n∈N n · P(ni = n | s) − n∈N n · P(ni = n | s)    2  ni (s) γi, j (s) =  n · n · p(n = n | s) j  i, j  n∈Ni i   ni (s) · n j (s)

i= j , i = j (B.3)

where ni, j = [ni , n j ], ∈ N2 , is a random vector whose components represent the number of spikes emitted by the neurons Ni and Nj respectively in the time window T. References Amari, S., & Nakahara, H. (2006). Correlation and independence in the neural code. Neural Comput., 18(6), 1259–1267. Averbeck, B. B., Latham, P. E., & Pouget, A. (2006). Neural correlations, population coding and computation. Nat. Rev. Neurosci., 7(5), 358–366. Averbeck, B. B., & Lee, D. (2006). Effects of noise correlations on information encoding and decoding. J. Neurophysiol., 95(6), 3633–3644. Brenner, N., Strong, S. P., Koberle, R., Bialek, W., & de Ruyter van Steveninck, R. R. (2000). Synergy in a neural code. Neural Comput., 12(7), 1531–1552. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience: Computational and mathematical modeling of neural systems. Cambridge, MA: MIT Press. Foffani, G., Chapin, J. K., & Moxon, K. A. (2008). Computational role of large receptive fields in the primary somatosensory cortex. J. Neurophysiol., 100(1), 268–280, doi: 10.1152/jn.01015.2007. Foffani, G., & Moxon, K. A. (2004). PSTH-based classification of sensory stimuli using ensembles of single neurons. J. Neurosci. Methods, 135(1–2), 107–120. Foffani, G., Tutunculer, B., & Moxon, K. A. (2004). Role of spike timing in the forelimb somatosensory cortex of the rat. J. Neurosci., 24(33), 7266–7271. Gawne, T. J., & Richmond, B. J. (1993). How independent are the messages carried by adjacent inferior temporal cortical neurons? J. Neurosci., 13(7), 2758–2771.

2694

A. Scaglione et al.

Ghazanfar, A. A., Stambaugh, C. R., & Nicolelis, M. A. (2000). Encoding of tactile stimulus location by somatosensory thalamocortical ensembles. J. Neurosci., 20(10), 3761–3775. Golomb, D., Hertz, J., Panzeri, S., Treves, A., & Richmond, B. (1997). How well can we estimate the information carried in neuronal responses from limited samples? Neural Computation, 9(3), 649–665. Latham, P. E., & Nirenberg, S. (2005). Synergy, redundancy, and independence in population codes, revisited. J. Neurosci., 25(21), 5195–5206. Montani, F., Kohn, A., Smith, M. A., & Schultz, S. R. (2007). The role of correlations in direction and contrast coding in the primary visual cortex. J. Neurosci., 27(9), 2338–2348. Montemurro, M. A., Panzeri, S., Maravall, M., Alenda, A., Bale, M. R., Brambilla, M., et al. (2007). Role of precise spike timing in coding of dynamic vibrissa stimuli in somatosensory thalamus. J. Neurophysiol., 98(4), 1871–1882. Montemurro, M. A., Senatore, R., & Panzeri, S. (2007). Tight data-robust bounds to mutual information combining shuffling and model selection techniques. Neural Comput., 19(11), 2913–2957. Nicolelis, M. A., Ghazanfar, A. A., Stambaugh, C. R., Oliveira, L. M., Laubach, M., Chapin, J. K., et al. (1998). Simultaneous encoding of tactile information by three primate cortical areas. Nat. Neurosci., 1(7), 621–630. Nirenberg, S., Carcieri, S. M., Jacobs, A. L., & Latham, P. E. (2001). Retinal ganglion cells act largely as independent encoders. Nature, 411(6838), 698–701. Nirenberg, S., & Latham, P. E. (2003). Decoding neuronal spike trains: How important are correlations? Proc. Natl. Acad. Sci. USA, 100(12), 7348–7353. Panzeri, S., Petersen, R. S., Schultz, S. R., Lebedev, M., & Diamond, M. E. (2001). The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron, 29(3), 769–777. Panzeri, S., Petroni, F., Petersen, R. S., & Diamond, M. E. (2003). Decoding neuronal population activity in rat somatosensory cortex: Role of columnar organization. Cerebral Cortex, 13(1), 45–52. Panzeri, S., Pola, G., & Petersen, R. S. (2003). Coding of sensory signals by neuronal populations: The role of correlated activity. Neuroscientist, 9(3), 175–180. Panzeri, S., Pola, G., Petroni, F., Young, M. P., & Petersen, R. S. (2002). A critical assessment of different measures of the information carried by correlated neuronal firing. Biosystems, 67(1–3), 177–185. Panzeri, S., & Schultz, S. R. (2001). A unified approach to the study of temporal, correlational, and rate coding. Neural Computation, 13(6), 1311–1349. Panzeri, S., Schultz, S. R., Treves, A., & Rolls, E. T. (1999). Correlations and the encoding of information in the nervous system. Proceedings of the Royal Society of London Series B—Biological Sciences, 266(1423), 1001–1012. Panzeri, S., Senatore, R., Montemurro, M. A., & Petersen, R. S. (2007). Correcting for the sampling bias problem in spike train information measures. J. Neurophysiol., 98(3), 1064–1072 Panzeri, S., & Treves, A. (1996). Analytical estimates of limited sampling biases in different information measures. Network-Computation in Neural Systems, 7(1), 87–107.


2695

Panzeri, S., Treves, A., Schultz, S., & Rolls, E. T. (1999). On decoding the responses of a population of neurons from short time windows. Neural Comput., 11(7), 1553–1577. Petersen, R. S., Panzeri, S., & Diamond, M. E. (2001). Population coding of stimulus location in rat somatosensory cortex. Neuron, 32(3), 503–514. Pola, G., Petersen, R. S., Thiele, A., Young, M. P., & Panzeri, S. (2005). Data-robust tight lower bounds to the information carried by spike times of a neuronal population. Neural Computation, 17(9), 1962–2005. Pola, G., Thiele, A., Hoffmann, K. P., & Panzeri, S. (2003). An exact method to quantify the information transmitted by different mechanisms of correlational coding. Network-Computation in Neural Systems, 14(1), 35–60. Rieke, F. (1997). Spikes: Exploring the neural code. Cambridge, MA: MIT Press. Rolls, E. T., Aggelopoulos, N. C., Franco, L., & Treves, A. (2004). Information encoding in the inferior temporal visual cortex: Contributions of the firing rates and the correlations between the firing of neurons. Biol. Cybern., 90(1), 19–32. Rolls, E. T., Franco, L., Aggelopoulos, N. C., & Reece, S. (2003). An information theoretic approach to the contributions of the firing rates and the correlations between the firing of neurons. J. Neurophysiol., 89(5), 2810–2822. Rolls, E. T., Treves, A., Tovee, M. J., & Panzeri, S. (1997). Information in the neuronal representation of individual stimuli in the primate temporal visual cortex. Journal of Computational Neuroscience, 4(4), 309–333. Schneidman, E., Bialek, W., & Berry, M. J., II. (2003). Synergy, redundancy, and independence in population codes. J. Neurosci., 23(37), 11539–11553. Shannon, C. E., & Weaver, W. (1949). Mathematical theory of communication. Urbana: University of Illinois Press. Treves, A., & Panzeri, S. (1995). The upward bias in measures of information derived from limited data samples. Neural Computation, 7(2), 399–407. Treves, A., Panzeri, S., Rolls, E. T., Booth, M., & Wakeman, E. A. (1999). Firing rate distributions and efficiency of information transmission of inferior temporal cortex neurons to natural visual stimuli. Neural Computation, 11(3), 601–631.

Received August 20, 2007; accepted February 20, 2008.

This article has been cited by: