June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
Advances in Complex Systems, Vol. 14, No. 3 (2011) 317–339 c World Scientific Publishing Company ! DOI: 10.1142/S0219525911003050
NETWORK AUTOMATA: COUPLING STRUCTURE AND FUNCTION IN DYNAMIC NETWORKS
DAVID M. D. SMITH Centre for Mathematical Biology, Oxford University, Oxford OX1 3LB, UK Oxford Centre for Integrative Systems Biology, Department of Biochemistry, Oxford University, South Parks Road, Oxford, OX1 3QU, UK
[email protected] JUKKA-PEKKA ONNELA Department of Biomedical Engineering and Computational Science, Helsinki University of Technology, P.O. Box 9203, FIN-02015 HUT, Finland CABDyN Complexity Centre, Oxford University, Oxford, OX1 1HP, UK CHIU FAN LEE Max Planck Institute for the Physics of Complex Systems, Noethnitzer Str. 38, 01187 Dresden, Germany MARK D. FRICKER Department of Plant Sciences, University of Oxford, Oxford, OX1 3RB, UK NEIL F. JOHNSON Physics Department, University of Miami, Coral Gables, Florida FL 33124, USA Received 26 May 2010 Revised 12 February 2011
We introduce Network Automata, a framework which couples the topological evolution of a network to its structure. To demonstrate its implementation we describe a simple model which exhibits behavior similar to the “Game of Life” before recasting some simple, wellknown network models as Network Automata. We then introduce Functional Network Automata which are useful for dealing with networks in which the topology evolves according to some specified microscopic rules and, simultaneously, there is a dynamic process taking place on the network that both depends on its structure but is also capable of modifying it. It is a generic framework for modeling systems in which network structure, dynamics, and function are interrelated. At the practical level, this framework allows for precise, unambiguous implementation of the microscopic rules involved in such 317
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
318
D. M. D. Smith et al. systems. To demonstrate the approach, we develop a family of simple biologically inspired models of fungal growth. Keywords: Complex system; functional, dynamic networks; computer simulation; integrative biology. PACS Number(s): 89.75.Fb, 87.15.A-, 87.85.Xd
1. Introduction The network framework has proved very successful in the study of various complex interacting systems. In the network description, the interacting elements are depicted as nodes and the interactions between the elements are represented by links connecting the corresponding nodes. The science of complex networks has progressed very quickly in the last few years, and some excellent reviews have been written covering both the methodology and key results [1, 6, 21]. The strength of the complex network paradigm lies in its ability to capture some of the essential structural characteristics of interacting systems while reducing the details of both the elements and their interactions. Consequently, the early complex network literature was almost exclusively focused on structural properties of networks. In many out-of-equilibrium growing networks, the evolution at a given time is dependent on the configuration of the network at that time, as exemplified in the preferential-attachment model of Barab´ asi and Albert [4]. In this paper, we develop Network Automata (NA), which can be seen as a natural extension of Cellular Automata (CA) [18, 32]. In contrast to Cellular Automata on a network, in which the states of nodes evolve and whose neighborhoods are defined by the network [7, 19, 20, 29], in the NA the network topology itself changes in time. We describe some different variants of NA in Secs. 2 and 3 and to demonstrate the versatility, we show how some familiar network models can be recast as NA in Sec. 4. Whilst structural properties remain important in constraining the behavior of a system [19], there is also significant interest in understanding dynamical processes taking place on networks [16]. Indeed, the marriage between structure and dynamic processes is so strong that the behavior of dynamic processes can be used to detect structure [14]. While a network’s topology constrains the type of dynamics that may unfold on it, in many scenarios the dynamical process may influence the subsequent evolution of the topology — meaning that the structural properties of the network are coupled to its function. A real-world example might be the evolution of transport links within a city. The dynamics of the human population using this network in turn affect the reinforcement or removal of those transport links and the feedback process is apparent. A similar situation may arise in the context of social networks, where one’s current social opportunities and dynamics are limited by the existing network structure, but they can be widened by extending the network. There have been several specific attempts in the literature to inter-relate a network’s structure, dynamics, and function [7, 11, 15, 22, 29, 33]. While many network
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
319
generating algorithms and Cellular Automata models can be easily recast within the Network Automata framework [24], its usefulness is exemplified when it is applied to systems in which topological evolution is coupled to a dynamic process occurring upon the network. We call these systems Functional Network Automata (FNA) and introduce the concept in detail in Sec. 5. Unlike certain CA models on networks in which the topology is evolved according to a global fitness function such as the ability of the system to perform a specific function (e.g. the density classification problem) [7, 29], the topological update in FNA is performed simultaneously with the functional process resulting in configurations which are emergent phenomena of the system [13]. FNA are conceptually similar to Structurally Dynamic Cellular Automata (SDCA) in which topological and functional evolution are coupled [2, 13]. Whilst topological updates in SDCA are governed by the CA states of the nodes, in the FNA the network evolution can be coupled to any functional process occurring on the network. Its contribution to the network field lies in the fact that it enables a precise specification of the microscopic rules underlying the structural and functional evolution of a given network-based system. This is a desirable feature which circumnavigates issues commonly associated with simulating complex systems whereby microscopic implementation biases can have emergent macroscopic consequences [23] and any comparison of the macroscopic behavior of the system might be rendered redundant. The (evolving) topology inherent in FNA removes the constraint of spacial homogeneity associated with CA models [23], further bridging the gap between CA and Multi-Agent Systems. As a demonstration of such an application, we introduce a family of progressively more realistic, biologically-inspired models of woodland fungal growth in Sec. 6. 2. Network Automata Consider an arbitrary weighted or unweighted, directed or undirected network at some time t which is to be grown to some size Ntot . As nodes might be added to the system at each timestep, the network would conventionally be considered to be growing. An alternative representation is to treat the system as being of size Ntot at all times where at some time t many of the nodes have no links. Information regarding the network’s topology is entirely encompassed within the adjacency matrix A(t) which is of dimension Ntot × Ntot . The matrix holds information about which links exist, their direction and, perhaps, weights. One might consider the evolution of the network as a process that alters the elements within this adjacency matrix, updating the attributes of any of the possible links which could exist in the system. We base a framework of network growth around this concept. If the microscopic ruleset governing the network’s evolution is solely related to quantities which can be derived from the network’s current topology (and hence from A(t)) then the evolution of the network might be expressed in terms of some operation F acting upon the adjacency matrix: A(t + 1) = F (A(t)).
(1)
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
320
D. M. D. Smith et al.
The ruleset employed could relate to any property of the nodes (their degree, betweenness, clustering and so on) or the links (weights or direction) and might be deterministic or stochastic. This concept can be achieved in practice by visiting all possible links within the adjacency matrix A(t) for a network comprising Ntot nodes (whether part of a component or not) once every timestep.a The update as to the nature of the link at the next time step (its existence or its weight or direction) is then prescribed by the ruleset. Because each link can see the requisite information at the start of the time step, the system is synchronously updated. The ruleset can take the form of a lookup table in which the state of a link is evaluated and the color (existence/direction/weight) is prescribed for the next timestep. This update process is analogous to the update of a cell within a conventional Cellular Automaton [18, 32] except that it acts upon the connectivity of an node, thereby generating A(t + 1). This link-orientated update is a generic description of a dynamic network and all the essential features of that network’s evolution are then contained within the exhaustive ruleset. We describe such a system as a Network Automaton.b No restriction has yet been made as to the directionality or weight of links. Unlike Structurally Dynamic Cellular Automata [13], in the simplest instance of NA the nodes do not carry values, the network update is purely governed by current topology.c 3. Restricted Network Automata Having defined Network Automata we shall now explore a simple example that can encapsulate some familiar Cellular Automata (CA) behavior [30]. We will impose some constraints such that this example is a small subclass of NA. First, we shall look at the undirected graph with unweighted links such that the existence of a link at time t can be described as the element Ai,j (t). The ruleset governing the system’s evolution determines the existence of this link at the next time step Ai,j (t + 1). At the start of the update process, we let the information that the ruleset can act upon be simply the current existence of the link and the degrees of the two nodes which it could possibly connect namely ki (t), kj (t). We impose some simple rules to govern the evolution of the system, namely that the existence of a link at time t + 1 is a function of the combined degree of the adjacent nodes ki (t) + kj (t) at the beginning of the timestep and its own existence Ai,j (t). We now restrict the network to an underlying lattice, U such that only those links that exist within the underlying structure can be formed in A. Note that the a The process of running through all links might be streamlined to include only those eligible for updating. This might be particularly useful in scenarios where the subset of links whose states might change is significantly less than total corresponding to all pairs of nodes or in models in which many nodes might remain unconnected indefinitely. b The concept of a Network Automaton has been alluded to before although not formally defined [32]. c This was proposed as a possible extension to SDCA in [13].
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
321
underlying lattice is undirected such that Ui,j = Uj,i . For visual clarity, we shall make this an undirected, degree 4 lattice with cyclic boundary conditions. Clearly this construction fulfills the criterion of being a Network Automaton, but since it is restricted to an underlying static network U, we call it a Restricted Network Automaton (RNA). The evolution of the state of a specific link can be described as some operation # # Ai,j (t) + Ai,j (t). (2) Ai,j (t + 1) = F Ai,j (t), Ui,j , j
i
As an example of the RNA framework, let us construct the rules of the game in the nomenclature of the “life-like” CA models [30]. The survival and birth of a link is determined according to the value of ki (t) + kj (t). We can express a ruleset for the survival process of the link in terms of a number (or a set of numbers) xs such that if ki (t) + kj (t) = xs and the state of the link Ai,j (t) = 1, then Ai,j (t + 1) = 1 and is zero otherwise. Likewise, for link birth, we have a number xb such that if ki (t) + kj (t) = xb and Ai,j (t) = 0, then Ai,j (t + 1) = 1. These rules are conventionally described in terms of a fertility interval and survival interval [30] or birth set/survival set [8], and here we follow the latter convention. The rules in this particular example relate the number of neighboring links to the future existence of a link. For example, according to rule B2/S3 a non-existent link will be “born” if the combined degrees of the two nodes between which it might exist is 2 and a link will survive if the combined degree of the two nodes it connects is 3. This ruleset is given explicitly in Table 1. The top three rows of Table 1 refer to the birth of links and the next two refer to link survival. Clearly, if a link exists then the degree of the nodes at each end must be greater than zero. As such, the fourth and fifth lines of Table 1 cover all eventualities of link survival for this ruleset. The explicit inclusion of the (symmetric) underlying matrix U reflects the restricted nature of the automaton. Naturally, ki (t) and kj (t) can be expressed in terms of the network’s adjacency matrix A(t) as the ith and jth element of A(t) 1 where 1 is a vector with all elements equal to one and of Table 1. The rules of game B2/S3 on an arbitrary underlying network U. A link will be born if it has 2 neighboring links and it must have only 1 neighbor to survive. Time t
Time t + 1
ki (t)
kj (t)
Ai,j (t)
Aj,i (t)
Ai,j (t + 1)
Aj,i (t + 1)
1 0 2 2 1
1 2 0 1 2
0 0 0 1 1
0 0 0 1 1
Ui,j Ui,j Ui,j 1 1
Ui,j Ui,j Ui,j 1 1
0
0
Otherwise
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
322
D. M. D. Smith et al.
Fig. 1. A simple spaceship of period two in the B2/S3 game with underlying lattice of degree 4. Repeated application of the automaton rules will clearly perpetuate its motion.
dimension Ntot . More concisely, in terms of some operation F & ' A(t + 1) = F A(t), U .
(3)
We can now observe the evolution of the automaton on some initial configurations of A(0) using the rule set B2/S3 and underlying lattice of degree 4. We can observe “blinkers” which are motifs which return to their original position after some period. There are also motifs that replicate themselves after a number of timesteps but are spatially translated as shown in Fig. 1. These are known as “spaceships” in the CA nomenclature because they propagate through the space. There are many other interesting configurations and many rulesets to explore, even with the degree 4 underlying lattice. There are a number of “still lifes” (objects that remain unchanged), blinkers of long periods, and “puffers” (debris leaving spaceships) which have been found [24]. It is interesting to note that this particular Network Automata example could be interpreted as a conventional Cellular Automata on a network which is the line graph [3] of the underlying lattice used in this example. Nodes in the line graph represent edges in the original network and are inter-connected if the edges they represent in the original network share a common node. An implementation of the equivalent Cellular Automata on this line graph would comprise of each node being assigned a binary state {0, 1} and the ruleset acting on the neighborhood of a given node in the new network, i.e. birth/survival is related to the number of neighboring nodes in state 1. 4. Stochastic Network Automata We can also use Network Automata to construct more conventional evolving networks. To do this we augment the NA, which consists of purely deterministic rules, by adding one or more stochastic rules and, thus, arrive at Stochastic Network Automata (SNA). To implement SNA, we need two additional definitions. We denote the outcome of a Bernoulli trial with B(p) defined as ( P (B(p) = 0) = 1 − p P (B(p) = 1) = p.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
We also define the Heaviside-like step function φ(x) as ( 1 if x > 0 φ(x) = 0 if x ≤ 0.
323
(4)
Note that SNA can be restricted to a fixed underlying lattice (network) U, resulting in Restricted Stochastic Network Automata (RSNA). We follow this approach in developing the biologically-inspired model in Sec. 6, but emphasize that in this section there is no constraining underlying lattice structure imposed. 4.1. Random attachment model The random attachment algorithm for building a single component network is straightforward. We consider a simple case in which at each time step a new node is connected to the existing network with one undirected link. Growing networks such as this one are known as non-equilibrium networks to distinguish them from equilibrium networks in which the number of nodes is constant. The conventional, master-equation analysis [6] provides the degree distribution of such a system, P (k) = ck = 2−k . We now emulate this process as an SNA. Suppose we wish to grow the network to Ntot nodes, so that the adjacency matrix is of dimension Ntot . At each time step, we consider the update of all possible links in the network but only wish to update the links from nodes within the connected network component to nodes outside of it. Consequently, there are a total of N (t)(Ntot − N (t)) links that may be added, of which we wish that, on average, only one link will be added. Having identified those links which may be added, the required probability associated with one of them being added is P (t) = [N (t)(Ntot − N (t)]−1 . The explicit rules for this particular SNA which replicates random attachment are expressed in Table 2. They state that the link can only be born if the node i is already part of the component and node j is not in the component (or vice-versa) and B(P (t)) = 1. If the link already exists, it stays. We seed the automaton with initial configuration of A1,2 (0) = A2,1 (0) = 1 reflecting a single component of two nodes. All links are only considered once in the update stage and both Ai,j (t+1) and Table 2. The stochastic ruleset to replicate random-attachment network growth. The resulting distribution is shown in Fig. 2. Time t
Time t + 1
Ai,j (t)
Aj,i (t)
φ(ki (t))
φ(kj (t))
Ai,j (t + 1)
Aj,i (t + 1)
0 0 0 0 1
0 0 0 0 1
0 0 1 1 1
0 1 0 1 1
0 B(P (t)) B(P (t)) 0 1
0 Ai,j (t + 1) Ai,j (t + 1) 0 1
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
324
D. M. D. Smith et al. 0
10
BA Network Automaton BA Master Equation RA Network Automaton RA Master Equation
−1
10
−2
P(k)
10
−3
10
−4
10
−5
10
−6
10
0
10
1
10
2
degree k
10
3
10
Fig. 2. The resulting degree distributions P (k) for the random-attachment (RA) model and the preferential-attachment (BA) model. The analytically-obtained degree distributions for these models are P (k) = 2−k and P (k) = 2m(m + 1)/[k(k + 1)(k + 2)] respectively [1, 6], with m = 1 with here. These are plotted with lines. Superimposed, are the corresponding distributions for one realization of the SNA, grown to only 10,000 nodes. Clearly the two approaches are consistent.
Aj,i (t + 1) are simultaneously updated.d The comparison of the degree distribution of the network generated by the SNA to that of the master-equation analysis is shown in Fig. 2. It is clear from the binomial process governing the addition of new links (and nodes) to the existing component of the network that, on average, one new link and one new node are added, although clearly more than one new node could be attached with more than one new link allowing cycles to be formed. 4.2. Barab´ asi–Albert model SNA can also emulate a preferential-attachment model such as that by Barab´ asi and Albert [4]. In the BA model the probability for a new node to attach to an existing node is proportional to the degree of the existing node, i.e., the attachment probability is a linear function of the node degree. This attachment mechanism is achieved in the SNA by simply modifying the probability of link birth, and the ) ruleset for this process is presented in Table 3, where K(t) = i ki (t) denotes the sum of degrees over all nodes in the network. d This
might be achieved in practice by running sequentially through all nodes i ∈ {1, . . . , Ntot } and then potential neighbors j ∈ {1, . . . , i} (the lower triangular portion of the adjacency matrix).
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
325
Table 3. The stochastic ruleset to generate preferential-attachment network growth. A small simulation distribution is shown in Fig 2. Time t
Time t + 1
Ai,j (t)
Aj,i (t)
φ(ki (t))
φ(kj (t))
Ai,j (t + 1)
0
0
0
0
0
0
0
0
1
B
0
0
1
0
0 1
0 1
1 1
1 1
B
“
“
kj (t) K(t)(Ntot −N(t)) ki (t) K(t)(Ntot −N(t))
0 1
Aj,i (t + 1) ” ”
0 Ai,j (t + 1) Ai,j (t + 1) 0 1
The analytical result obtained using the master-equation approach is given by P (k) = 2m(m + 1)/[k(k + 1)(k + 2)] where m is the (fixed) degree of the new node entering the network [1]. Asymptotically this leads to P (k) ∼ k γ with γ = 3. The analytically-obtained distribution is plotted in Fig. 2 together with the corresponding distribution obtained from one realization of an SNA simulation. The match between the two approaches is very good. 4.3. Watts–Strogatz model We have demonstrated how non-equilibrium growing networks can be generated as an SNA. Here we turn to equilibrium networks and discuss the rules necessary to generate small-world networks emulating those of Watts and Strogatz [31]. The WS model starts from considering a one-dimensional lattice comprising N nodes with all nodes having the same degree k (through connections to nearest neighbors, then next nearest neighbors, etc.) and cyclic boundary conditions.e An initial configuration for k = 4 is shown in Fig. 3, as an example. Each link in the network is visited and it is rewired with probability p. The original rewiring mechanism was
Fig. 3.
e Note
The initial network configuration of the Watts-Strogatz model for k = 4.
that a two-dimensional lattice of degree 4 would be inappropriate in that it has zero clustering.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
326
D. M. D. Smith et al.
such that one end of the link remained where it was and the other vertex was chosen at random from the rest of the network. In practice, the addition of shortcuts is the important aspect of this model so we choose a slightly simpler mechanism such that both ends of the rewired link are chosen at random. We implement this modified process as an SNA because it results in a somewhat simpler ruleset than the original model whilst retaining the salient features of the original model. Initially, there are N k/2 links within the system. The expected number which are to be rewired is pN k/2. This process might be considered “link death”. The number of nodes remains constant, and we wish the number of links to remain constant too. The expected number of links to be born is therefore set equal to the expected number of links which are removed. This is comparible to the notion of link rewiring in the original model of Watts and Strogatz although we note that the number of links removed and those added are not necessarily equal. Assuming no loops of length one (self-connected nodes) such that Ai,i = 0, the total number of links in the system that are not alive (and therefore capable of being born) is N (N − 1) N k − . 2 2
(5)
We can then describe the time-independent birth probability of links by P (t) = pb =
kp . (N − k − 1)
(6)
The ruleset for this system, which is is given in terms of the link-removal probability p and the link-birth probability pb , is shown in Table 4. In an actual implementation of the model, one runs through all possible links, whether they exist or not, and updates their state according to the rules. The result of applying this ruleset is shown in Fig. 4, depicting the normalized clustering coefficient and mean shortest path for the networks generated. It is interesting to note that this implementation of the SNA requires only one time step. After sufficient successive applications of the automata, all the original links will have be removed and any remaining links will exist between random pairs of nodes. The result will be a network comparable to the classic random graphs of Erd¨ os and R´enyi [9]. Table 4. The simple, stochastic ruleset for emulation of the Watts and Strogatz small-world network model. The top row represents the removal of a link with probability p and the bottom, the rewiring of a link with probability pb as defined in Eq. (6). The network is undirected, yielding a symmetric adjacency matrix A. Time t = 0
Time t = 1
Ai,j (0)
Aj,i (0)
Ai,j (1)
Aj,i (1)
1 0
1 0
1 − B( p) B( pb )
Ai,j (1) Ai,j (1)
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
327
Clustering Mean Shortest Path
l(p)/l(0) C(p)/C(0)
1
0.8
0.6
0.4
0.2
0 −4 10
−3
10
−2
10
Rewiring Probability p
−1
10
0
10
Fig. 4. The normalized clustering coefficient and average shortest path of the networks generated using the Network Automata mechanism outlined in Table 4 to emulate the Watts–Strogatz smallworld model. Here, k = 10 and N = 400. Each marker represents a simulated network, with the line being the mean values over 100 simulations. The normalizing coefficients C(0) and l(0) are the clustering coefficient and mean shortest path of the network prior to any rewiring.
5. Functional Network Automata We now consider a situation in which the topology of a network evolves while there is simultaneously some process taking place on the network. At any given time the topology of the network constrains the type of dynamics that may unfold on it. However, the dynamical process may influence the subsequent topological evolution of the network, so that its structural properties are coupled to its function and vice-versa. The ruleset governing the topological update process relates not only to network-derived quantities but also functional aspects of the nodes and/or links. Since the functional process requires a network on which to perform, we decouple the evolution of the network into two distinct phases, namely, that affecting its topology and that governing the functional process. Writing the functional information (relating to nodes and/or links) at some time t as some matrix S(t), the formal description of the evolution can be expressed in terms of some operations F and G as A(t + 1) = F (A(t), S(t)),
(7)
S(t + 1) = G(A(t + 1), S(t)).
(8)
This expression states that the network evolves according to some process, which is determined by its own current topology A(t), and also by some attributes of its nodes and links that includes function-based information S(t). The functional
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
328
D. M. D. Smith et al.
process then occurs on this network to generate the new set of information S(t + 1). The global state of the system is encompassed by the matrices A and S.f The system is synchronously updated by running through all possible links between all pairs of nodes and updating the state of each link in accordance to the ruleset employed. 6. Biologically Inspired Model We now construct a series of simple models of woodland fungi [5, 10, 26, 27] to demonstrate the versatility of Functional Network Automata. Although the models, which describes the growth of the fungi and distribution of resources within it, are biologically inspired, their aim is not to incorporate a large number of biological details. Instead, we adopt a minimalist approach to emulate fungal growth and internal nutrient transport from a small set of microscopic rules. These models are, nevertheless, capable of producing structures qualitatively similar to mycelial network development (e.g. Fig. 5). This example is Phanerochaete velutina, a foraging saprotrophic woodland fungus that builds an adaptive network to translocate
Fig. 5. (Color online) Transport of non-metabolized, radioactively-labeled amino acid within Phanerochaete velutina, a foraging, woodland fungi. The fungi was grown in a Petri dish from a 12 mm agar inoculum plug across a scintillation screen. The image is of an area approximately 10 × 10 cm and depicts relative intensities of photon counts derived from movement of a radiolabeled amino acid integrated over a 60 minute interval with dark blue indicating the lowest and dark red indicating the highest relative intensity [26–28]. f Here,
we have defined the topological and functional updates as being sequential to each other. In general, a topologically and functionally coupled system system could also be defined synchronously, A(t + 1) = F (A(t), S(t)) and S(t + 1) = G(A(t), S(t)) although any (dis)advantage from this formulation is unclear. The suitability of either formulation is likely to be situation (model) dependent and related to the relative timescales of the functional and topological evolution.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
329
resource [5, 27, 28]. The transport flux is oscillatory and still not fully understood [10, 25, 27, 28]. We start by specifying biologically-naive and mathematically-simple update rules (Ruleset a) which govern the topological part of the system’s evolution as in Eq. (7). This is paired to a simple functional update process (Process 1) representing Eq. (8). The ruleset and the process fully define the model. We then modify both the topological rules (Ruleset b and c) and functional process (Process 2) according to some basic physical and biological considerations providing six distinct models. The end product may serve as a platform for more elaborate future models of fungal growth, demonstrating the effectiveness of using the framework. Consider a system of agents who might each be interpreted as a cell in a twodimensional lattice. The connectivity between agents is North, South, East and West reflecting a maximum possible connectivity of d = 4 for all agents who are subsequently restricted to local information as long-range communication is assumed unlikely in the biological system. The agent layer is superimposed on a resource layer as in Fig. 6. The rules of the system are very simple. If an agent is above a resource, it absorbs that resource at some rate RE . The objective of each agent is two-fold: to grow into available space in the search for more resource, and to redistribute excess internal resource to support further exploration. We allow each agent to grow only one new neighbor (in a random direction) at a particular time, but only if the agent has resources to do so. To mimic active transport of resources to the growing tip, an agent passes resources to its neighbor cell provided that the neighbor does not pass resources to it. We now endeavor to categorize this simple multi-agent system as a Restricted Stochastic Network Automaton. This
Agent Layer
RE
RE
Resource Layer Fig. 6. (Color online) A biologically-inspired multi-agent model whereby the agent layer is superimposed upon a resource layer. Agents above a resource can accumulate resources at some rate RE .
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
330
D. M. D. Smith et al.
serves not only to clarify any ambiguities that arise in the programming of a multiagent system [23], but also as a potential aid to improving efficiency in that the required iteration and information storage/retrieval aspects are clearly defined by the rulesets imposed. Let us first look at the growth (topological) stage. Each cell or agent represents a node and the boundary between two cells through which resource is passed is represented by a directed link. Consider that the information upon which the topological ruleset will act to update the attributes of a link in the network is simply the amount of resource that each of the two nodes has at each end of the link and their in and out-degrees. We can simply write the functional information as a vector such that Si (t) refers to the resource (functional variable) that agent (node) i has at time t. For clarity, we can express some topological information as vectors too, such that the element ki (t) represents the total degree of a node, out ki (t) its out-degree which, in turn, provides its in-degree in ki (t), each of which is obtainable from the adjacency matrix A(t). We will grow this Network Automaton in an unweighted but directed adjacency matrix A so that if Ai,j = 1 the link exists and is directed from i to j, whereas if Aj,i = 1 the link exists and is directed from j to i. If neither Ai,j = 1 nor Aj,i = 1 then the link does not exist. Here Ai,j = 1 and Aj,i = 1 are mutually exclusive. As each node has limited possible connectivity (here 4), we only consider the subset of links in the system which could possibly exist. The structural update process runs through all of these possible links and each pair of nodes which could be connected is considered once. The link attributes Ai,j and Aj,i are then updated at the same time. We can now write the network update procedure for this topological update as Ruleset a in terms of an exhaustive truth table as in Table 5. Note that if Ai,j (t) = Aj,i (t) = 0, the total degree of both nodes i and j is less than the connectivity d of the lattice in which the system evolves. Again, we denote the outcome of a Bernoulli trial as B(p) such that P (B(p) = 1) = p and conversely P (B(p) = 0) = 1 − p. Again, we employ a step function φ(x) defined as φ(x) = 1 for x > 0 and φ(x) = 0 for x ≤ 0 to distinguish active or “alive” agents. That is, an agent i is active if it has some amount of resource, φ(Si (t)) = 1. The first rule in Ruleset a states that if there does not exist a link between two inactive nodes at the present timestep, no link shall be formed at the next timestep. The second and third rules reflect an agent growing by establishing a link towards (on average) one neighbor in the lattice. Once fully connected (ki (t) = d), an agent cannot generate any further links. The fourth rule reflects two active agents who are adjacent in the lattice but not connected. A link is established between them with its direction determined via a coin toss. The remaining eight rules in Ruleset a simply state that once a link is established, it remains indefinitely: there is no rewiring. We now describe the resource distribution (functional) stage and start by mapping the adjacency matrix A(t + 1) to a normalized transition matrix T(t + 1)
Aj,i (t)
0
0
0
0
1 1 1 1
0 0 0 0
Ai,j (t)
0
0
0
0
0 0 0 0
1 1 1 1
0 0 1 1
0 0 1 1
1
1
0
0
φ(Si (t))
Time t
0 1 0 1
0 1 0 1
1
0
1
0
φ(Sj (t))
B
1 1 1 1
0 0 0 0
2
`1´
1 d−ki (t)
B
“
0
0 ”
Ai,j (t + 1) “
0
0
1 d−kj (t)
”
0 0 0 0
1 1 1 1
1 − Ai,j (t + 1)
B
Aj,i (t + 1)
Time t + 1
Model a
2
`1´
1 d−ki (t)
B
“
”
1 − Aj,i (t + 1)
1 1 1
0 0 0 ` ´ δ(in ki , d)B d1
B
0
0
Ai,j (t + 1) “
0
0
1 d−kj (t)
”
0 0 0 ` ´ δ(in ki , d)B d1
1 − Ai,j (t + 1)
1 1 1
1 − Ai,j (t + 1)
B
Aj,i (t + 1)
Time t + 1
Model b
2
`1´
g d−ki (t)
B
“
”
1 − Aj,i (t + 1)
1 1 1
0 0 0 ` ´ δ(in ki , d)B d1
B
0
0
Ai,j (t + 1)
“
0
0
g d−kj (t)
”
0 0 0 ` ´ δ(in ki , d)B d1
1 − Ai,j (t + 1)
1 1 1
1 − Ai,j (t + 1)
B
Aj,i (t + 1)
Time t + 1
Model c
Table 5. Different rulesets (the columns labeled ‘time t + 1’) for three biologically inspired models: (a) the simplest scenario, (b) incorporating conservation of resources, and (c) implementing a delay factor (see text for details). As Ai,j=1 and Aj,i = 1 are mutually exclusive, there are only 12 possible states of a link at time t.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks 331
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
332
D. M. D. Smith et al.
describing the flow of resource between adjacent cells: ( Ai,j (t + 1)/out ki (t + 1) for out ki (t + 1) > 0 Ti,j (t + 1) = 0 for out ki (t + 1) = 0 ( 0 for out ki (t + 1) > 0 Ti,i (t + 1) = 1 for out ki (t + 1) = 0.
(9)
Through this process, an agent distributes resource equally amongst those neighbors which are not transmitting resource to it. We can then write the update for the resource distribution process as S(t + 1) = T" (t + 1)S(t) + ξ(t),
(10)
where the vector ξ corresponds to the accumulation of resource by agents from the resource (substrate) layer and % denotes the matrix transpose. We impose the constraint that only “alive” (i.e. active) agents can accumulate resource through this process so ξη (t) = RE φ(Sη (t)) Lη ,
(11)
where the vector L denotes the (binary) existence of resource at position of node (agent) η in the resource layer, RE is the rate at which an agent accumulates the resource and φ(x) is the step function defined earlier to identify active agents. This resource accumulation from the substrate could be made time dependent (i.e. finite resources such that L represents quantities of resource at specific locations) and the accumulation rate could also be made site specific reflecting resource “quality” although here we will not consider these effects. Equations (9)–(11) constitute Process 1 and represent the functional update stage of Eq. (8). For the example of Fig. 7, the amount of resources that node i has at time t + 1 is Si (t + 1) = Sj (t)
Aj,i (t + 1) Am,i (t + 1) + Sm (t) k (t + 1) out j out km (t + 1)
+ Sr (t)
Ar,i (t + 1) Aq,i (t + 1) + Sq (t) out kr (t + 1) out kq (t + 1)
= Sj (t)Tj,i (t + 1) + Sr (t)Tr,i (t + 1) + Sq (t)Tq,i (t + 1).
(12)
In this example node i is not accumulating resource from the substrate layer. The resource it receives is from neighbors to whom it is connected and whose links are directed towards node i. The amount it receives from a given neighbor is simply determined by the out-degree of that neighbor. We can now observe the Network Automaton in operation as shown in Fig. 8. We start with a single node η above a single food source so that at time t = 0 the agent has some resource. In the initial configuration the adjacency matrix is all zeros Ai,j (0) = 0 ∀ i, j, and the resource information vector is all zeros except
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
333
m
i
j
r
q
Fig. 7. The influx of resource into node labeled i. The amount this node receives from node j is related to the out-degree of node j. This amount would be expressed as Sj (t)/out kj (t+1). Similarly, node i receives an amount Sq (t)/out kq (t + 1) of resource from node q and Sr (t)/out kr (t + 1) from node r whereas it receives nothing from node m.
t =0
t =2
t =4
t =6
Fig. 8. (Color online) The evolution of the biologically-inspired NA over 6 timesteps. The colors represent the amount of resource Si (t) a node has (the functional aspect) superimposed on the evolving, directed network (the structural aspect). Dark blue denotes the agent with the lowest amount of resource and dark red the highest.
Sη (0) = RE such that the initial agent has amount RE . For this example, the resource accumulation vector ξ(t) is also all zeros except ξη (t) = RE ∀ t. We observe both the network and functional aspect of the system. The nodes (agents) are superimposed on the directed network, and the amount of resources a node has is indicated by its color ranging from blue (low concentration) to dark red (high concentration) [10, 24]. Only nodes that have resources are included and the result is independent of the choice of RE . A longer simulation is shown in Fig. 9.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
334
D. M. D. Smith et al.
a1
b1
c1
a2
b2
c2
Fig. 9. (Color online) Simulated output for the six models pairing each of the topological updates, Rulesets a, b and c, and functional updates of Processes 1 and 2. For all models, the lattice is of dimension 400 × 400 with periodic boundary conditions. Each was seeded with a single initial agent above a single (infinite) resource. The rate of resource accumulation for that agent is RE = 80, 000 and the simulation was run for 2000 time steps for each model. For Process 2 the residual consumption rate of each agent is set at R C = 1. For Ruleset c, the delay factor is g = 0.1. Again, dark blue denotes the agent with lowest amount of resource and dark red the highest.
Note that under this ruleset and functional update stage an agent might accumulate resource indefinitely. By allowing a node which has in-degree equal to the maximum connectivity (in ki = d) to randomly flip a direction of one of its links overcomes this biologically-unfeasible behavior and results in the topological update of (Ruleset b) where the Kronecker delta function is defined as δ(x, y) = 0 for x '= y and δ(x, y) = 1 for x = y. A simulation of this ruleset coupled with the functional update of Process 1 is shown in Fig. 9 and emergent canalized flux channels are clear. Such channels have been observed experimentally in a wide class of real biological fungi [27]. In Rulesets a and b, the physical transport of nutrient is of comparable speed to that of the growth, which is clearly not reasonable for most biological systems. A further development can be made to incorporate two different time scales in the model so that the growth and redistribution of resources take place at different rates, leading to Ruleset c. To establish this feature within the model, we can delay the growth process by introducing a parameter g ∈ (0, 1] to the stochastic growth terms in the topological ruleset, so that an agent grows, on average, one neighbor
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
335
every 1/g time steps. This is depicted explicitly in Table 5. Whilst this additional development slows the rate of growth, the automaton will grow indefinitely until the domain is fully occupied (the system illustrated in plot c1 of Fig. 9 is still growing). We can also incorporate consumption of resources by agents in the model by making further modifications to the functional update stage, resulting in Process 2. If an agent has more than some residual consumption amount, RC , then this rate of consumption is deducted. If the agent has less than this value, all of that agent’s resources are removed and that agent might be considered dead. We introduce an intermediate stage in the functional update process which can be described in a similar manner to Eq. (10): S ∗ (t + 1) = T" (t + 1)S(t) + ξ(t),
(13)
where the resource distribution is described by the transition matrix T(t+1) defined in Eq. (9) and the accumulation of resource from the substrate is described by the vector ξ(t) defined in Eq. (11). We can then write the update for S(t + 1), including consumption of resource as Si (t + 1) = φ(Si∗ (t + 1) − RC )(Si∗ (t + 1) − RC ),
(14)
which makes use of the step function φ(x). Only agents (nodes) active in the network can accumulate resources from the resource layer. The effect of this “cost” of living clearly limits the potential size of the system. Equations (9), (11), (13) and (14) constitute Process 2 and the effect of using this functional update are illustrated in Fig. 9. Whilst the aim of development of these models of fungal growth is to serve as an illustration of an application FNA, we can qualitatively observe the effects of the different rulesets and functional update processes, a pairing of which constitute a fully-defined model. We observe from the upper row of plots in Fig. 9 that the functional update process, Process 2 is robust to the imposed variations in the topological updates (a, b and c) with respect to coverage of the domain. The model c1 will eventually occupy the full space and is still growing at the point illustrated in the plot c1 of Fig. 9. Interstingly, comparison of plots a1 and a2 demonstrates that the incorporation of resource conservation in ruleset b results in stronger canalization of flow than the simplest ruleset a. The lower plots in Fig. 9 depict the use of Process 2 incorporating the consumption of resource. Models a2 and b2 have reached a steady state in which no more growth occurs whereas in model c2, the peripheral islands unconnected to the main population body will eventually die out. 7. Concluding Remarks and Discussion In this paper we have developed the concepts of Network Automata (NA) together with its restricted (RNA) and stochastic (SNA) variants. This generic framework can encompass the behavior of many familiar models. We have demonstrated its
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
336
D. M. D. Smith et al. Functional
SDCA
BIM
BA
GOL
RA WS
Restricted
Stochastic
Fig. 10. The functional, stochastic and restricted behaviors which can be encompassed by (Functional) Network Automata. Also illustrated is the position of the Barab´ asi–Albert (BA) network growth model, the random attachment (RA) network growth model, the Watts–Strogatz smallworld model (WS), the Game of Life Cellular Automata (GOL), the biologically inspired models (BIM) introduced in Sec. 5 and Structurally Dynamic Cellular Automata [13].
ability to mirror Cellular Automata behavior in Sec. 3, and also non-equilibrium growing network models (random attachment, Barab´asi-Albert) and equilibrium (Watts–Strogatz) network models in Sec. 4 which are illustrated schematically in Fig. 10. Whilst these pedagogical cases (which might be implemented directly more efficiently) have comprised undirected, unweighted networks, these features can be easily incorporated within the NA framework. The development of NA naturally leads to the concept of Functional Network Automata which couples the evolution and function of complex networks by using simple microscopic rules at the level of nodes and links. We have demonstrated the practicality of the framework by applying it to a family of biologically inspired models, which produce qualitatively similar canalized flow patterns to those observed in real woodland fungi [10, 25–27]. This suggests that for organisms which have to adapt their morphology to a variable environment, function may play a crucial role in determining structure. The well-defined and simple rulesets not only make replication straightforward but also aid implementation at the programming level. In the study of emergent phenomena involving networks, use of an FNA implementation enables concise and clear establishment of microscopic rules such that the observed differences in resulting behavior are known to be resultant from the rules as opposed to any emergence from implementation biases. Speculatively, we might infer that the growth of “limbs” in models b2 and c2 evident in Fig. 9 might constitute a primitive emergent search strategy in that a larger distance from the initial resource has been reached than would have been if the same number of agents had grown with isotropic radial coverage. One can easily think of more complex
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
337
rulesets to more accurately model a real system, such as adding transport costs or finite resources, both of which can easily be accomplished, or even a time-dependent ruleset. It is then interesting to ask what level of complexity is required to more accurately model real biological systems. We would expect certain limitations of FNA consistent with the advantages of Multi-Agent Systems over conventional CA approaches. Specifically, MAS are well suited to mixed populations of agents [23] and implementation of such models as FNA could prove difficult. However, we also expect there to be many application domains for Network Automata from social to biological systems such as modeling the evolution of genetic regulatory networks where selection pressure at the functional (phenotypic) level influences evolution at the topological (genetic) level [12]. One can envisage applying FNA to Diffusion Limited Aggregation (DLA) systems. That DLA has been recently recast as a CA suggests that the FNA might be suitable for this type of problem [23] or, indeed, any system in which the dynamics of network topology is related to the function performed thereon. There are certain biological systems which can quasi-solve increasingly complex problems in a constant time. An example is Physarum polycephalum, the true slime mold, which approximately identifies the Steiner points when placed upon multiple food sources [17]. Given that it might be possible to model these systems within the NA framework, it might suggest how to design a hardware-based implementation to perform similar calculations in constant time. It is interesting to pose the question as to what kind of problems could be solved by such a system and how complex the microscopic rules would be for a given problem. This would reflect the minimum length of ruleset that would have to be employed by the system in both the network and functional update stages. Acknowledgments D.M.D.S. acknowledges funding by EPSRC and BBSRC and J.-P.O. by a Wolfson College Junior Research Fellowship (Oxford). D.M.D.S acknowledges funding from the European Union (MMCOMNET) for part of this research. We thank Felix Reed-Tsochas for comments and suggestions. References [1] Albert, R. and Barab´ asi, A.-L., Statistical mechanics of complex networks, Rev. Mod. Phys. 74 (2002) 47–97. [2] Alonso-Sanz, R., A structurally dynamic cellular automaton with memory, Chaos Soliton. Fract. 32 (2006) 1285–1295. [3] Balakrishnan, V. K., Schaums Outline of Graph Theory (Mcgraw-Hill Publishing Company, New York, 1997). [4] Barab´ asi, A.-L. and Albert, R., Emergence of scaling in random networks, Science 286 (1999) 509–512. [5] Bebber, D. P., Hynes, J., Darrah, P. R., Boddy, L. and Fricker, M. D., Biological solutions to transport network design, Proc. R. Soc. B 274 (2007) 2307–2315.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
338
D. M. D. Smith et al.
[6] Dorogovtsev, S. N. and Mendes, J. F. F., Evolution of networks, Adv. Phys. 51 (2002) 1079–1187. [7] Darabos, C., Giacobini, M., Tomassini, M., Performance and robustness of cellular automata computation on irregular networks, Adv. Complex Syst. 10 (2007) 85–110. [8] Eppstein, D., Growth and decay in life-like cellular automata, in Adamatzky, A. (ed.), Game of Life Cellular Automata (London: Springer, 2010). [9] Erd¨ os, P. and R´enyi, A., On random graphs, Publ. Math. Debrecen 6 (1959) 290. [10] Fricker, M. D., Lee, J. A., Bebber, D. P., Tlalka, M., Hynes, J., Darrah, P. R., Watkinson, S. C. and Boddy, L., Imaging complex nutrient dynamics in mycelial networks, J. Microscopy 231 (2008) 317–331. [11] Fronczak, P., Fronczak, A. and Ho!lyst, J. A., Self-organized criticality and coevolution of network structure and dynamics, Phys. Rev. E 73 (2006) 046117. [12] Foster, D. V., Kaufmann, S.A. and Socolar, J. E. S., Network growth models and genetic regulatory networks, Phys. Rev. E 73 (2006) 031912. [13] Ilachinski, A. and Harpern, P., Structurally dynamic cellular automata, Complex Systems 1 (1987) 503–527. [14] Lambiotte, R., Delvenne, J.-C. and Barahona, M., Laplacian dynamics and multiscale modular structure in networks, arXiv:0812.1770v3 (2009). [15] Meisel C. and Gross, T., Self-organized criticality in a realistic model of adaptive neural networks, arXiv:0903.2987v1 (2009). [16] Motter, A. E., Mat´ıas, M. A., K¨ urths, J. and Ott, E., Dynamics on Complex Networks and Applications, Physica D 224 (2006) 7–8. [17] Nakagaki, T., Kobayashi, R., Nishiura, Y. and Ueda, T., Obtaining multiple separate food sources: Behavioural intelligence in the physarum plasmodium, Proc. Biol. Sci. 271 (2004) 2305–2310. [18] von Neumann, J., The Theory of Self-Reproducing Automata (University of Illinois Press, Urbana, 1966). [19] Marr, C. and H¨ utt, M.-T., Topology regulates pattern formation capacity of binary cellular automata on graphs, Physics A 354 (2005) 641–662. [20] Marr, C. and H¨ utt, M.-T., Outer-totalistic cellular automata on graphs, Phys. Lett. A 373 (2009) 546–549. [21] Newman, M. E. J., The structure and function of complex networks, SIAM Rev. 45 (2003) 167–256. [22] Rohlf, T. and Bornholdt, S., Self-organized criticality and adaptation in discrete dynamical networks, arXiv:0811.0980v1 (2008). [23] Spicher, A., Fat´es, N. and Simonin, O., Translating discrete multi-agents systems into cellular automata: application to diffusion-limited aggregation, Filipe, J., Fred, A. and Sharp, B. (eds.), CCIS 67 Proceedings of ICAART09–Revised Selected Papers, 270–282 (Berlin: Springer, 2010). [24] Smith, D. M. D., Agents, Games and Networks, D. Phil. thesis, University of Oxford, UK (2007). [25] Tlalka, M., Bebber, D. P., Darrah, P. R., Watkinson, S. C. and Fricker, M. D., Emergence of self-organised oscillatory domains in fungal mycelia, Fungal Genet. Biol. 44 (2007) 1085–1095. [26] Tlalka, M., Bebber, D. P., Darrah, P. R., Watkinson, S. C. and Fricker, M. D., Quantifying dynamic resource allocation illuminates foraging strategy in Phanerochaete velutina, Fungal Genet. Biol. 45 (2008) 1111–1121. [27] Tlalka, M., Hensman, D., Darrah, P. R., Watkinson, S. C. and Fricker, M. D., Noncircadian oscillations in amino acid transport have complementary profiles in assimilatory and foraging hyphae of Phanerochaete velutina, New Phytol. 158 (2003) 325–335.
June 18, 2011 19:22 WSPC/S0219-5259 169-ACS S0219525911003050
NA: Coupling Structure and Function in Real-World Networks
339
[28] Tlalka, M., Watkinson, S. C., Darrah, P. R. and Fricker, M. D., Continuous imaging of amino-acid translocation in intact mycelia of Phanerochaete velutina reveals rapid, pulsatile fluxes, New Phytol. 153 (2002) 173–184. [29] Tomassini, M., Giacobini, M. and Darabos, C., Evolution and dynamics of smallworld cellular automata, Complex Systems 15 (2005) 261–284. [30] de La Torre, A. C. and M´ artin, H. O., A survey of cellular automata like the “game of life”, Physica A 240 (1997) 560–570. [31] Watts, D. J. and Strogatz, S. H., Collective dynamics of ‘small-world’ networks, Nature 393 (1998) 440. [32] Wolfram, S., A New Kind of Science (Wolfram Media, 2002). [33] Zimmerman, M. G., Egu´ıluz, V. M. and San Miguel, M., Coevolution of dynamical states and interactions in dynamic networks, Phys. Rev. E 69 (2004) 065102(R).
