arXiv:1705.02773v1 [nlin.AO] 8 May 2017
The concurrence of form and function in developing networks: An explanation for synaptic pruning Ana P. Millána∗, J.J. Torresa , S. Johnsonb , and J. Marroa a
Institute Carlos I for Theoretical and Computational Physics, University of Granada, Spain
b
Warwick Mathematics Institute and Centre for Complexity Science, University of Warwick, Coventry CV4 7AL, UK. Corresponding author (
[email protected]).
May 9, 2017 Abstract A fundamental question in neuroscience is how structure and function of neural systems are related. We study this interplay by combining a familiar autoassociative neural network with an evolving mechanism for the birth and death of synapses. A feedback loop then arises leading to two qualitatively different behaviours. In one, the network structure becomes heterogeneous and dissasortative and the system displays good memory performance. In the other, the structure remains homogeneous and incapable of pattern retrieval. These findings are compatible with experimental results on early brain development, and provide an explanation for the existence of synaptic pruning. Other evolving networks – such as those of protein interaction – might share the basic ingredients for this feedback loop, and indeed many of their structural features are as predicted by our model.
1
Introduction
The fundamental question in neuroscience of how structural and functional properties of neural networks are related has recently been considered in terms of large-scale connectomes and functional networks obtained with various imaging techniques [1]. But at the lower level of individual neurons and synapses there is still much to learn. From this point of view, the brain can be regarded as a complex network in which nodes (or vertices) represent neurons, and edges (or links) stand for synapses. It is then possible
1
to use mathematical models which capture the essence of neural and synaptic activity to study how a great many such elements can give rise to collective behaviour with at least some of the characteristics of cognition [2, 3]. Following his strategy, structural properties of the underlying network have been found to affect the behaviour of the system in various ways [4, 5, 6, 7]. For instance, the robustness to noise of memory performance is influenced by the heterogeneity of the degree distribution (the degree of each node being the number of nodes it is connected to) as well as by degree-degree correlations [8, 9]. And indeed, networks that connect neurons by means of synapses have been found to exhibit degree heterogeneity (according to roughly scale-free distributions) and negative degree-degree correlations (dissasortativity), properties which, it has been suggested, may contribute to robustness and efficiency [10]. It is also known that these networks are not static: new synapses grow and others disappear in response to neural activity [11, 12, 13, 14]. Models in which networks are gradually formed, for instance by addition of nodes and edges or by rewiring of the latter, have long been studied in many contexts. Typically, the probabilistic addition or deletion of elements at each time step is a function of the existing structure. For example, in the familiar Barabási-Albert model, a node’s probability of receiving a new edge is proportional to its degree [15]. These rules often give rise to phase transitions (almost invariably of a continuous nature), such that different kinds of network topology can ensue, depending on parameters [16]. One such model was used to interpret data on brain development, and it was found that several topological characteristics of empirical networks were reproduced close to the critical point separating phases of homogeneous and heterogeneous degree distributions [17]. Here we combine this model of brain network development with the Amari-Hopfield neural network [2, 3]. The probabilities of synaptic growth and death are set to depend on neural activity, as has been empirically observed, instead of on node degree. This is in the spirit of other co-evolving network models which have been studied in various fields [18, 19, 20, 21, 22]. We find that, for certain parameter ranges, the phase transition between memory and randomness becomes discontinuous (i.e. resembling a first order thermodynamic transition). Depending on initial conditions, the system can either evolve towards heterogeneous networks with good memory performance, or homogeneous ones incapable of memory. The reason is a feedback loop between structure and function, as we shall go on to show. Previous studies of co-evolving brain networks have focussed on the temporal evolution of mean degree [23], particular microscopic mechanisms [24], the development of certain computational capabilities [25], or the effects of specific growth rules [26]. However, to the best of our knowledge, this is the first time that a feedback loop, and the ensuing discontinuous transition, have been identified. In many species, early brain development seems to be dominated by a remarkable process known as synaptic pruning. The brain starts out with a relatively high density of synapses, which is gradually reduced as the individual matures. In humans, for example, synaptic density at birth is about twice what it will be at puberty, and certain disorders, such as autism and schizophrenia, have been related to this reduction process [27, 28,
2
29, 30]. It has been suggested that such synaptic pruning may represent some kind of optimization, perhaps to minimise energy consumption or related genetic information [24]. But why begin with so many more synapses than required? Our proposal can give a hint in this regard. That is, it seems that the strategy could be to start with a homogeneous, randomly wired network which would then evolve under the effects of a competition between form and function to develop in this way the gradual optimization of the structure. A possible - and coherent with experiments - strategy to provide memory performance during this phase could be as simple as maintaining a highly connected energy consuming network during infancy. Our results thus suggest a simple explanation for the existence of an initial explosion of connections followed by a synaptic pruning process. Finally, according to some indications which we shall describe, the chances are that the mechanism in this paper is general enough to describe the underluing dynamics os other biological systems, such as protein interaction networks.
2
Model definition
Consider an undirected network with N nodes, and edges which can change in discrete time. At time t, the adjacency matrix is {eij (t)} (for i, j = 1, . . . , N), with elements 1 or 0 according to whether there exists or not an edge between the pair of nodes (i, j), P respectively. The degree of node i at time t is ki (t) = N j eij (t), and the mean degree −1 PN of the network is κ(t) = N i ki (t). The neural network model. Each node represents a neuron, and each edge a synapse. We shall use the Amari-Hopfield model, according to which each neuron has two possible states at each time t, given by si (t) = {0, 1} [31]. Each edge (i, j) has a P synaptic weight wij , and the field at neuron i is hi (t) = N j=1 wij eij (t)sj (t). The states of all neurons are updated in parallel at every time step according to the transition probabilities 1 [1 + tanh (β [hi (t) − θi ])] , 2 P −1 is a ‘noise’ where θi = 21 N j=1 wij is a neuron’s threshold for firing, and β = T parameter controlling stochasticity (analogous to the inverse temperature in statistical physics) [32]. The synaptic weights can be used to encode information in the form of a set of P ‘patterns’ {ξiµ ; µ = 1, . . . , P }, with mean a0 = hξ µ i, specific configurations of neural states which can be regarded as memories. We shall use the Hebb learning rule: P {si (t + 1) = 1} =
wij = [κ0 a0 (1 − a0 )]−1
P X
(ξiµ − a0 )(ξjµ − a0 ).
µ=1
The canonical Amari-Hopfield model [31] which is here a reference is defined on a fixed fully connected network (eij = 1, ∀i = j), and it exhibits a continuous phase transition at the critical temperature T = 1.
3
The pruning model. The structure of the network also changes in time. For this we shall adapt the model put forward in Ref. [17]. At each time t, each node has a probability Pig (t) of being assigned a new edge to another node, randomly chosen. Likewise, each node has a probability of losing one of its edges, with probability Pil (t). These probabilities are given by Pig = u (κ) π (Ii ) ,
Pil = d (κ) η (Ii ) ,
(1)
where we have dropped the time dependence of all variables for clarity. The first terms on the right-hand side of each equation represent a global dependence to account for the fact that such processes rely in some way on diffusion of different molecules through large areas of tissue, and we take the mean degree κ at each time as a proxy (note that, since the number of nodes is fixed, the mean degree is proportional to the total number of synapses). The simplest choice for this effect consistent with empirical data on synaptic pruning in humans [17, 23] is n κ u (κ) = , N 2κ∞
n κ d (κ) = 1− , N 2κ∞
(2)
where n is the number of edges to be added or removed at each step, which sets the speed of the process, and κ∞ is the stationary mean degree (i.e. the mean degree of the system κ tends to κ∞ as t → ∞). It should also be noticed, however, that experimental studies [23] have revealed a fast initial overgrowth of synapses associated with the transient existence of different growth factors. This and other particular mechanisms could be accounted for by adding extra factors in Eqs. (2). For example, initial overgrowth has been reproduced by adding the term a exp(−t/τg ) in the probability of growth [17]. The second terms of the right hand of Eqs. (1) were previously considered [17] to be functions of ki . This was meant as a proxy for the empirical observations that synaptic growth and death are determined by neural activity [11, 12, 13, 14]. However, we now make this dependence explicit, by considering a dependence on the local current at each neuron: Ii = |hi − θi |. This couples neural dynamics with the evolving network. Monte Carlo simulations. The initial conditions for the neural states are random. For the network topology we can draw an initial degree sequence from some distribution p(k, t = 0), and then place edges between nodes i and j with a probability proportional to ki (0)kj (0), as in the configuration model. Time evolution is then accomplished in practice via computer simulations as follows. First, the number of links to be created and destroyed are chosen according to two Poisson distributions with means Nu (κ) and Nd (κ), respectively. Then, as many times as needed according to this draw, we choose a node i with probability π (Ii ) to be assigned a new edge, to another node randomly chosen; and similarly we choose a new node j according to η (Ij ) to lose an edge from one of its neighbours, randomly chosen. This procedure uses the BKL algorithm to assure proper evolution towards stationarity [32]. Each node can then be chosen via two paths: either through the process with probability π (Ii ) for a gain (or η (Ii ) for a loss), or when it is randomly connected to (or disconnected from) an already chosen
4
node. Therefore, the effective values of the second factors in (1) are πe = π (Ii ) + 1/N and ηe = η (Ii ) + ki /(κN). For the sake of simplicity, we shall consider πe and ηe to be power laws, as in Ref. [17], which allows one to move smoothly from a sub-linear to a super-linear dependence with a single parameter: πe ∝ Iiα and ηe ∝ Iiγ . This leads to: π (Ii ) =
2 N hIi
!α
!
hI α i I − , 2 α
Iiγ η (Ii ) = γ . hI i N
(3)
This assumption is most important, as it couples current activity and structure. However, the particular functions are an arbitrary choice and other ones could be considered. In our scenario, the parameters α and γ characterize the dependence of the local probabilities on the local currents and account for the different proteins and factors that control synaptic growth. These could be obtained experimentally, although to the best of our knowledge this has not yet been done. The time scale for structure changes is set by the parameter n in (2), whereas the time unit for activity changes is the number hs of Monte Carlo Steps (MCS) between each configuration updating. Our study shows a low dependence on this parameter in the cases of interest, so we only report results here for hs = 10 MCS. The macroscopic state. This may be characterized by the overlap among the neuP µ ral activity and each of the memorized patterns, mµ = [Na0 (1 − a0 )]−1 N i=1 (ξi − a0 ) (si − a0 ) , P and by the mean neural activity, M = N1 N i=1 si . Also of interest is the degree distribution at each time, p (k, t), whose heterogeneity may be measured via g = exp (−σ 2 /κ2 ), where σ 2 is the variance. g equals one for the homogeneous case p (k) = δ (k − κ) and tends to zero for highly heterogeneous networks. E the Pearson correD We also monitored 2 2 2 2 lation coefficient applied to the edges: r = hki k knn (k) − hk i / hki hk 3 i − hk 2 i , where knn (k) is the mean nearest-neighbour-degree function. This characterizes the degree-degree correlations, which have important implications for network connectedness and robustness. That is, whereas most social networks are assortative (r > 0), almost all other networks, whether biological, technological or information-related, seem to be generically dissasortative (r < 0), meaning that high degree nodes tend to have low degree neighbours, and vice-versa. In fact, previous studies showed that heterogeneous networks favour the emergence of dissasortative correlations [33, 34].
3
Main results
Consider first what we shall call the topological limit of the model, in which there is no neuron dynamics and one substitutes Ii → ki , so that ηi = η (ki ) and πi = π (ki ). This immediately leads to the master equation: dP (k, t) /dt = u (κ) πe (k − 1) p (k − 1, t) + d (κ) ηe (k + 1) p (k + 1, t) − [u (κ) πe (k) + d (κ) πe (k)] p (k, t), which is exact in the limit of no degree-degree correlations between nodes. This equation can be numerically integrated for each α and γ to obtain the corresponding degree distribution. Three
5
qualitatively different behaviours, leading in practice to different phases, follow for ki ≫ 1 (see figure 1a): 1. α < γ =⇒ ηe > πe , i.e., high degree nodes are more likely to lose than to gain edges. Excluding low k, a relatively homogeneous degree distribution emerges, with the probability of having high degree nodes very rapidly vanishing from a maximum (circles in figure 1a). 2. α > γ =⇒ ηe < πe , i.e., high degree nodes are more likely to continue to gain than to lose edges. Since the stationary number of edges Nκt is fixed, this leads to kind of a bimodal degree distribution, with high probability for very high degree nodes (squares in fig. 1a). 3. α = γ =⇒ ηe = πe , i.e., very connected nodes are as likely to gain as to lose edges. Excluding low degrees, the stationary distribution then decays as a power law with exponent µ close to 2.5 (diamonds in figure 1a), in accordance with long-range connections observed in the human brain [7] and measures in protein interaction networks [35]. Notice that the condition ηe = πe (for ki ≫ 1) is mandatory to obtain this critical behaviour, despite previous studies suggesting the opposite [17] (see fig. 1b). This description is valid for more general choices of both ηe and πe , as long as both are monotonous functions. Notice also that even for α < γ the stationary network presents some degree of heterogeneity, that is, the degree distribution is not of the form δ(k − κ∞ ), due to the intrinsic noise associated with edge dynamics. One can easily prove that γ = 1 corresponds in the topological limit to links chosen at random for removal by noticing that the probability of choosing any edge for removal 1 is then pij = k1i ηe (ki )+ k1j ηe (kj ) = hkiN . This can be seen as a first order approximation to the pruning dynamics, and it also induces powerful simplifications during computations. Furthermore, the relevant parameter is the relation between α and γ, whereas their absolute values only affect quantitatively, e.g., to the exponent of the critical degree distribution. Therefore, we shall mainly refer here to the case γ = 1. Concerning the full model, in which pruning depends on Ii , computer simulations depict a rich emergent phenomenology depending on the levels of stochasticity and heterogeneity. In particular, we focus on the influence of both the noise level T and the local probability parameter α. Other parameters are set as in a previous study [17]; this is for simplicity and also because it then follows that the present model agrees with experimental data as reported there. New preliminary simulations suggested dependence on the heterogeneity of the initial condition (IC), so that we considered two different types of IC, namely, homogeneous, p (k, t = 0) = δ(k −κ0 ), and heterogeneous, p (k, t) ∝ k −2 networks, with fixed κ0 . Phase diagram. Our coupling dynamics leads to a rich phenomenology, including discontinuous transitions and multistability. Three different phases can be identified by monitoring the order parameters g(α, T ), m(α, T ) and r(α, T ), as illustrated in figure
6
100
1
g
(a)
(b)
-2 0
p(K)
10
0.5
0
0.5
1
1.5
Normalized local eld
10-1
10-4
10
0
10
1
10
2
10
Degree (k)
3
(c) 10-2
10
-3
10-4 10-4
10
-3
10
-2
10-1
Normalized degree
Figure 1: Topological limit. (a) Degree distributions for the three different phases: subcritical (α = 0.8, purple circles), critical (α = 1.0, green diamonds) and supercritical (α = 1.2, blue squares). N = 3200 and γ = 1.0. Notice how p(k) decays slowly in the critical case, approximately as k −µ with µ ≈ 2.5. (b) g(α) for the topological case (purple squares), and for a different set of local probabilities (green circles) that do not fulfil the condition ηe = πe for ki ≫ 1 (in particular we set here ηi ∝ ki and πi ∝ kiα , with α rescaled so that both transitions occur at α = 1). The transition in the latter case is a discontinuous one, which fails to produce scale-free networks. The indicated lines follow from integration of the corresponding master equations, whereas the points are Monte Carlo data. Deviations from simulations are due to the emergence of small degree-degree correlations that are not taken into account by the master equation. Full model: (c) Normalized local field of each neuron as a function of its (normalized) degree, showing the correlation between activity and the topology that emerges in the system. This is almost one-to-one in the critical case for every node degree, whereas homonegeneous networks present a very broad correlation, as bimodal ones do for low degrees (notice the double logscale). Data symbols meaning as in (a).
7
1
g
1
(a)
(b)
0
0
1 0.5
(d)
m
(c)
0
0 0
1 T
2
0
1
α
2
Figure 2: Behaviour along the dotted lines marked in the diagram of figure 3 when N = 1600, κm = 10 and n = 5. Left: g (T ) (top) and m (T ) (bottom) curves for different values of α, as indicated, for both homogeneous and heterogeneous IC. Right: g (α) (top) and m (α) (bottom) curves for different temperatures, as indicated. Solid points are for homogeneous IC; empty are for heterogeneous IC.
8
0
1
2
Figure 3: Phase diagram as obtained from analysis of the control parameters (m, g and r) as defined in the main text. The transition lines, corresponding to αct (T ) and αcm (T ) as labelled, are indicated either with solid or with dashed lines regarding whether they correspond to continuous or discontinuous transitions. The horizontal and vertical dotted lines correspond to the cases illustrated in figure 2.
9
2 (our data for r(α, T ) is not shown here since it does not provide new information). Analysis of these and similar curves for different parameter values leads to the phase diagram in figure 3. That is, there is a homogeneous memory phase for low values of both α and T which is characterized by high overlap m, high heterogeneity g and low (negative, almost zero) correlation r (see the curves in fig. 2a,c for α = 0.5 and low T and in fig. 2b,d for T < 1.2 for low α). The system is then able to reach and maintain memory, while the topological process leads to a homogeneous network configuration. Due to the memory, there is a strong correlation between the physiological state of the network, as measured by the currents Ii , and its topology, via the degrees ki (see figure 1c). A continuous increase of α leads through a topological phase transition to heterogeneous (bimodal) networks which corresponds to the line α = αct (T ) in the diagram of fig. 3, along which one finds scale-free distributed networks. The phase transition is revealed in the g(α) curves of fig. 2b (for T = 0.9, 1.0 and 1.1) as a fast (and continuous) decay to zero, and it also shows in the overlap curves m(α) (figure 2d) where memory initially grows monotonously with α, to slightly decrease in the transition, and then it approaches a constant value. This behaviour reveals a strong coupling between structure and memory, and it also shows how scale-free distributed networks can optimize memory recovery for a given set of control parameters. We call this the heterogeneous memory phase (figure 3), which is characterized by high overlap m, very low (almost zero) g and high negative r, indicating a memory state with heterogeneous (bimodal) dissasortative structure. Interestingly, this phase expands up to high noise levels as a consequence of network heterogeneity, which happens to help memory recovery. Even more, memory recovery in turn favours network heterogenization in a feed-back manner, enhancing the stability of the state. Dynamics is finally governed by noise as T is further increased, and the stationary network is then homogeneous. It results in a homogeneous noisy phase, characterized by low (almost zero) overlap, high g and low (almost zero) r (see fig. 2a,c for α = 0.5 and 1.0 and high T and fig. 2b,d for T > 0.9 and low α). Interestingly, the transition from memory to noise strongly differs depending on the initial low T phase, i.e., on the value of α. For low α values (homogeneous memory phase), a noise increase causes a transition from memory to noise through the line α = αcm (T ) (see fig. 2a and c for α = 0.5 and the diagram of fig. 3), which also leads to more homogeneous networks, since structure evolution is driven by noise. Our results indicate that this transition has a continuous nature. On the other hand, the transition at higher α from heterogeneous memory has a discontinuous nature, and it includes a bi-stability region (see the striped area in figure 3), which is evident when one compares g(α, T ) and m(α, T ) for homogeneous and heterogeneous IC, particularly for T = 1.2 (fig. 2 b and d) and for α = 0.5 (fig. 2 a and c). The existence of multistability illustrates how memory promotes itself in a heterogeneous network, which is a direct consequence of the coupled dynamics in our model. To understand this effect, we recall previous studies on stationary networks [8, 9] which showed that heterogeneous networks present higher memory recovery than homogeneous
10
ones for high noise levels, particulary for T > 1. Besides this effect, in our model the presence of memory leads to some degree of network heterogenization, which depends on α, giving rise to a memory-heterogeneity feed-back loop. This implies differences between homogeneous and heterogeneous IC, which translates for high α and T into a multistability region and the presence of memory for T > 1. To further visualize this effect, notice how the complex transition lines discovered here allow us to trace isolines (either constant α or constant T ) that go through the three phases. Particularly, for T = 1.0 and 1.1 (fig. 2b and c) the system is initially in the homogeneous memory phase, and an increase in α leads to networks that are heterogeneous enough to maintain memory through the transition line αcm (T ). The memory-heterogeneity feed-back loop then appears, favouring higher heterogenization and causing a "fall" of g (α) to memory levels. A final increase in α leads to a topological transition at αct (T ) towards heterogeneous networks. Similarly, the system visits for α = 1 (fig. 2 a and c) the three phases: at very low T , α is high enough to develop heterogeneous memory networks, but a slight noise increase is enough to suppress heterogeneity (the system crosses αm (T ) from above), until noise is too high and memory is also lost (thus leading to the homogeneous noisy phase). Therefore, a main observation here is that the model shows an intriguing relation between memory and topology which induces complex transitions that might be relevant to understand actual systems. In particular, the inclusion of a topological process allows for memory recovery when T > 1, whereas the presence of thermal noise shifts the topological transition (which in the topological limit occurs at α = 1) to α > 1. These findings can also have implications for network design, e.g. to help memory recovery optimization in a noisy environment. Synaptic pruning with initial overgrowth. A question that naturally arises here is the significance of considering a heterogeneous structure as the initial network state, which has been artificially placed so far. However, there are mechanisms not taken into consideration here, such as additional growth factors, that could lead to the same effects - that is, to an enhanced stability of the memory state. In fact, some preliminary simulations show that homogeneous IC could strongly improve memory retrieval after (and also during) pruning by including the term a exp(t/τg ) in the growth probability. This is due to the combination of two effects. First, the inclusion of such a term creates a long transitory with high κ, which allows the network to maintain a memory. Second, since the preferential attachment process is already taking place, network structure begins heterogenizing and hubs appear, which will ensure memory retrieval when κ diminishes. Therefore, it would seem that a rapid initial growth mechanism could be essential in helping the subsequent synaptic pruning to assure a proper brain development. Protein interaction networks. Many complex systems can be described as networks which evolve under the influence of node activity. It is therefore likely that the structure-function feedback loop we have described for neural networks will play a role in other settings too. For instance, this mechanism could be compatible with experimental and theoretical studies concerning protein interaction networks. These gather
11
different types of metabolic interactions, whether physical, due to a physico-chemical contact between two proteins, or through an action on the corresponding gene. Interactions can then be either inhibitory or excitatory and have different strength, as edges in our model. This structure is known to change on a evolutionary time scale, that is, during the evolution of species rather than that of individuals, and these changes combine a random origin (since they are typically due to mutations) with a "force" driven by natural selection, which is likely to be activity dependent (see [35] for a detailed analysis of their characteristics). The similarities between this picture and our model suggest a parallel concerning the resulting network topologies as well. To make a more specific contact, note that measurements on these networks show scale-free distribution of some important node measurements with respect to node degree: (a) degree distributions p(k) ∝ k −µ with µ ≈ 2.5 [35]; (b) clustering coefficient C(k) ∝ k −θ with θ ≈ 1, 2 depending on the type of network, namely, metabolic networks are for θ ≈ 1, while protein interaction networks are for θ ≈ 2 [35, 36]; and (c) mean nearest neighbour degree, K(k) ∝ k −ν with ν ≈ 0.6 [37]. Our model also produces scale-free distribution for these magnitudes, thus reproducing basic qualitative properties of these networks. Even more, we even reproduce the observed exponent µ, indicating a general mechanism in biological systems. Also interestingly, in our model we find that C(k) decays as k −1 for control parameters in the heterogeneous memory phase and close to the topological transition αct , the same region where power-law distributed degree-distributions emerge. Moreover, the function K(k) decays in our model as a power law for almost every value of the parameters. According to [37], this is related with the intrinsic topological dynamics of the model, which create an asymmetry between the couple of nodes that gain (or loose) an edge. We find ν ≈ 1.0 for homogeneous networks and ν ≈ 2.0 for heterogeneous bimodal networks, whereas scale-free distributed networks lie in between those, with ν ≈ 1.5. Therefore, besides the qualitative reproduction of these functions, we are able to approximately obtain µ and θ for metabolic networks. Also, given that the particular values of these parameters depend on the local probabilities in our model, we expect that they can be accurately reproduced by appropriately tuning the parameters α and γ. In conclusion, there are indications that some of the main topological properties of protein interaction networks could be qualitatively reproduced with simple adjustments or extensions of the model. This suggests a general mechanism underlying the dynamics of different biological systems, which is likely to extend as well to other contexts.
4
Final discussion
It is well known from a large body of work that the brain stores information in synaptic conductances, which mediate neural activity; and that this in turn affects the birth and death of synapses. To explore what this feedback might entail, we have coupled an auto associative or attractor neural network model with a model for the evolution of its network topology which describes synaptic pruning. The resulting picture allows
12
for a two-way functional coupling between physiological activity in the network and its topological structure, which is very likely to occur in nature. This opens a way to relate brain cognitive abilities, such as memory related processes, with known biophysical dynamics at the cellular level. Taken separately, each of the two models involved exhibit continuous phase transitions: between a phase of memory and a noise one in the neural network, and between homogeneous and heterogeneous network topologies in the synaptic pruning model. The coupled model continues to display both of these transitions for certain parameter regimes, and a new transition emerges which is discontinuous. One of the hallmarks of such a discontinuous transition is the coexistence of phases. In other words, situations with the same parameters but slightly different initial conditions can lead to markedly different outcomes. In this case, whether the attractor neural network is initially capable of memory retrieval influences the emerging network structure, which feeds back into the memory performance. There is therefore an interesting feedback loop between structure or form and function which determines the capabilities of the eventual system which the process yields. Our picture thus relates well-defined mathematical properties with specific dynamic processes in the brain and, in particular, it addresses how neural activity can impact on early brain development. The models we have coupled in this work are the simplest ones able to reproduce the behaviour of interest - namely, associative memory and network topologies with realistic features. Our choices were made in the spirit of identifying the minimum ingredients necessary for the feed-back loop between structure and function to arise. However, there are no indications to believe that this feedback will disappear when using more complex models. On the other hand, neural systems may not be the only ones to display the properties we found to be sufficient for the existence of this feedback loop between structure and function. Networks of proteins, metabolites or genes also adopt specific configurations (often associated to attractors of some dynamics), and the existence of interactions between nodes might depend on their activity. It is thus noteworthy that we find some evidence that protein networks seem to have topological features which emerge naturally from the coupled models we have considered here. One may expect that the interplay between form and function we have described in this work could play a role in many natural, complex systems. Finally, an unresolved question in neuroscience is why brain development proceeds via a severe synaptic pruning. Fewer synapses require less metabolic energy, so why not just begin with the ideal number? Presumably, much less genetic information is needed to build a randomly wired neural network than to determine a priori the exact locations of synapses, or even to specify other topological properties, such as the degree distribution. As our study shows, a heterogeneous topology vastly increases memory efficiency in a noisy environment. The way to achieve this seems to be to start with an initial overgrowth in the synaptic density, and then follow a pruning process guided by early brain activity. This is enough to provide a heterogeneous network configuration that efficiently supports memory after pruning greatly reduces the number of synapses.
13
This is compatible with the view that the process serves to reduce energy consumption. If this view is correct, then the model suggests that the probabilities of birth and death of synapses as functions of the current arriving at each neuron should be approximately linear, so as to produce optimal (scale-free) network topologies. This prediction could be tested experimentally. We are grateful for financial support from the Spanish MINECO (under grant FIS2013-43201-P; FEDER funds) and from the "Obra Social La Caixa".
References [1] Ed Bullmore and Olaf Sporns. "Complex brain networks: graph theoretical analysis of structural and functional systems." Nature Reviews Neuroscience, 10:186, 2009. [2] S Amari. "Characteristics of random nets of analog neuron-like elements." Artificial neural networks: theoretical concepts. IEEE Computer Society Press 2: 643, 1972. [3] J J Hopfield. "Neural networks and physical systems with emergent collective computational abilities." Proc.Nat.Acad.Sci., 79:2554, 1982. [4] D S Modha and R Singh. "Network architecture of the long-distance pathways in the macaque brain." Proc.Nat.Acad.Sci.USA, 107:13485, 2010. [5] Qawi K Telesford et al. "The brain as a complex system: using network science as a tool for understanding the brain." Brain Connectivity, 1:295,2011. [6] N Voges and L Perrinet. "Complex dynamics in recurrent cortical networks based on spatially realistic connectivities." Front Comput Neurosci, 6:41, 2012. [7] MT Gastner and Géza Ódor. "The topology of large open connectome networks for the human brain." Sci. Reports, 6:27249 ,2016. [8] JJ Torres, MA Munoz, J Marro and PL Garrido. "Influence of topology on the performance of a neural network." Neurocomputing, 58, 229, 2004. [9] S de Franciscis, S Johnson and J J Torres. "Enhancing neural-network performance via assortativity." Phys. Rev. E 83:0.6114, 2011. [10] S Navlakha, A L Barth, and Z Bar-Joseph. "Decreasing-rate pruning optimizes the construction of efficient and robust distributed networks." PLoS Comput Biol, 11:e1004347, 2015. [11] K S Lee, F Schottler, M Oliver and G Lynch "Brief bursts of high-frequency stimulation produce two types of structural change in rat hippocampus." Journal of Neurophysiology, 44:247, 1980.
14
[12] E Frank. "Synapse elimination: For nerves it’s all or nothing." Science, 275:324, 1997. [13] A Y Klintsova and W T Greenough. "Synaptic plasticity in cortical systems." Current Opinion in Neurobiology, 9:203, 1999. [14] M De Roo, P Klauser, P Mendez, L Poglia and D Muller. "Activity-dependent psd formation and stabilization of newly formed spines in hippocampal slice cultures." Cerebral Cortex, 18:151, 2008. [15] AL Barabási and R Albert. "Emergence of scaling in random networks." Science, 286:509, 1999. [16] S Johnson, J J Torres, and J Marro. "Nonlinear preferential rewiring in fixed-size networks as a diffusion process." Phys. Rev. E 79.5:050104, 2009. [17] S Johnson, J Marro, and J J Torres. "Evolving networks and the development of neural systems." Journal of Statistical Mechanics: Theory and Experiment, P03003, 2010. [18] F Vazquez, Víctor M. Eguiluz, and M San Miguel. "Generic absorbing transition in coevolution dynamics." Phys. Rev. Lett., 100:108702, 2008. [19] T Gross and B Blasius. "Adaptive coevolutionary networks: a review." Journal of The Royal Society Interface, 5:259, 2008. [20] P Kaluza, A Kölzsch, M T Gastner and B Blasius. "The complex network of global cargo ship movements." Journal of the Royal Society Interface, 7:1093, 2010. [21] C Meisel, A Storch, S Hallmeyer-Elgner, E Bullmore and T Gross. "Failure of adaptive self-organized criticality during epileptic seizure attacks." PLoS Comput Biol, 8:1, 2012. [22] H Sayama, I Pestov, J Schmidt, BJ Bush, C Wong, J Yamanoi and T Gross. "Modeling complex systems with adaptive networks." Computers & Mathematics with Applications, 65:1645, 2013. [23] P R Huttenlocher and A S Dabholkar. "Regional differences in synaptogenesis in human cerebral cortex." Journal of Comparative Neurology, 387:167, 1997. [24] G Chechik, I Meilijson, and E Ruppin. "Neuronal regulation: A mechanism for synaptic pruning during brain maturation." Neural Computation, 11:2061, 1999. [25] V M Eguiluz, D R Chialvo, G A Cecchi, M Baliki and A-V Apkarian "Scale-free brain functional networks." Phys.Rev.Let., 94:018102, 2005. [26] J Iglesias, J Eriksson, F Grize, M Tomassini and AE Villa "Dynamics of pruning in simulated large-scale spiking neural networks. Biosystems.", 79:11, 2005.
15
[27] M S Keshavan, S Anderson, and J W Pettergrew. "Is schizophrenia due to excessive synaptic pruning in the prefrontal cortex? the feinberg hypothesis revisited." Journal of Psychiatric Research, 28:239, 1994. [28] D H Geschwind and P Levitt. "Autism spectrum disorders: developmental disconnection syndromes." Current Opinion in Neurobiology, 17:103, 2007. [29] G Faludi and K Mirnics. "Synaptic changes in the brain of subjects with schizophrenia." International Journal of Developmental Neuroscience, 29:305, 2011. [30] A Fornito, A Zalesky, and M Breakspear. "The connectomics of brain disorders." Nature Reviews Neuroscience, 16:159, 2015. [31] D J Amit. Modeling brain function: The world of attractor neural networks. Cambridge University Press, 1992. [32] A B Bortz, M H Kalos, and J L Lebowitz. "A new algorithm for monte carlo simulation of ising spin systems." Journal of Computational Physics, 17:10, 1975. [33] S Boccaletti, V Latora, Y Moreno, M Chavez and DU Hwang. "Complex networks: Structure and dynamics." Phys.Rep. 424:175, 2006. [34] S Johnson, J J Torres, J Marro, and M A Muñoz. "Entropic origin of disassortativity in complex networks." Phys. Rev. Lett. 104: 108702, 2010. [35] R Albert. "Scale-free networks in cell biology." J. of cell science 118:21, 2005. [36] S Maslov and K Sneppen. "Specificity and stability in topology of protein networks." Science 296:5569, 2002. [37] J Berg et al. "Structure and evolution of protein interaction networks: a statistical model for link dynamics and gene duplications." BMC Evolutionary Biology 4:1, 2004.
16
