Evolutionary dynamics on stochastic evolving ... - APS link manager

PHYSICAL REVIEW E 84, 046111 (2011)

Evolutionary dynamics on stochastic evolving networks for multiple-strategy games Bin Wu,1,* Da Zhou,2,3,† and Long Wang1,‡ 1

Center for Systems and Control, State Key Laboratory for Turbulence and Complex Systems, College of Engineering, Peking University, Beijing 100871, China 2 School of Mathematical Sciences, Peking University, Beijing 100871, China 3 MOE Key Laboratory of Bioinformatics and Bioinformatics Division, TNLIST, Department of Automation, Tsinghua University, Beijing 100084, China (Received 28 April 2011; revised manuscript received 19 August 2011; published 21 October 2011) Evolutionary game theory on dynamical networks has received much attention. Most of the work has been focused on 2 × 2 games such as prisoner’s dilemma and snowdrift, with general n × n games seldom addressed. In particular, analytical methods are still lacking. Here we generalize the stochastic linking dynamics proposed by Wu, Zhou, Fu, Luo, Wang, and Traulsen [PLoS ONE 5, e11187 (2010)] to n × n games. We analytically obtain that the fast linking dynamics results in the replicator dynamics with a rescaled payoff matrix. In the rescaled matrix, intuitively, each entry is the product of the original entry and the average duration time of the corresponding link. This result is shown to be robust to a wide class of imitation processes. As applications, we show both analytically and numerically that the biodiversity, modeled as the stability of a zero-sum rock-paper-scissors game, cannot be altered by the fast linking dynamics. In addition, we show that the fast linking dynamics can stabilize tit-for-tat as an evolutionary stable strategy in the repeated prisoner’s dilemma game provided the interaction between the identical strategies happens sufficiently often. Our method paves the way for an analytical study of the multiple-strategy coevolutionary dynamics. DOI: 10.1103/PhysRevE.84.046111

PACS number(s): 89.75.Hc

I. INTRODUCTION

Evolutionary game theory is the study of frequencydependent selection in the process of evolution. It provides a powerful mathematical framework for the study of biology [1–3], economics [4], and social sciences [5]. It is widely employed to investigate how cooperative behavior emerges among selfish individuals [2,3], how biodiversity sustains [6], and how language evolves over time [7]. Traditionally, the population is assumed to be well-mixed and infinitely large, i.e., each individual is equally likely to interact with any other ones. In this case, the well-known replicator dynamics, a system of ordinary differential equations, drives each strategy to evolve [8,9]. Recent advances in this theory have focused mainly on the relaxations of previous assumptions in two areas: the finite population size effect and the structured population. A natural question arises: how does the finite size and the structure affect the replicator dynamics? For finite population size, stochastic processes instead of deterministic equations are employed [10–12]. How does the finite-size effect impact the replicator dynamics? Traulsen et al. have shown that for the linear pairwise update rule, the evolution of strategy in finite populations can be captured by a Langevin equation, in which the drift term is identical to the replicator equation in the sense that they both share the same equilibrium and Lyapunov stability, while the diffusion term scales with the inverse of the square root of the population size [12,13]. Thus the Langevin equation acts as if it were driven by the replicator equation provided the population size is sufficiently large. This result bridges the deterministic

*

[email protected] [email protected] ‡ [email protected] †

1539-3755/2011/84(4)/046111(8)

replicator dynamics in an infinite population and the stochastic game dynamics in a finite population. For structured populations, networks are often employed. Nodes and links represent individuals and the connections via which the payoffs are obtained [14–17]. Originally, researchers focused on static networks in which individuals are only allowed to adjust their behaviors. In this case, the evolutionary game on networks results in the replicator dynamics with a rescaled payoff matrix under weak selection [18,19]. In recent years, however, individuals on the networks have been allowed not only to change their strategies but also to alter their partnerships [20–40]. On the one hand, this coevolution of strategy and structure is often studied in the framework of the prisoner’s dilemma or snowdrift, i.e., 2 × 2 games, whereas the general coevolutionary dynamics for n × n games is seldom addressed [37]. On the other hand, analytical methods are still lacking while many intriguing and valuable results have shown that coevolution, can greatly promote cooperation [23,27,28]. One pioneering step toward this is the model proposed by Pacheco et al. [29–31]. The linking dynamics therein is described by a system of ordinary differential equations. In other words, their linking dynamics is deterministic. However, when the uncertainties of social ties are taken into account, the linking dynamics is more suitably considered as a stochastic process due to random fluctuations [38]. In this study, we generalize the stochastic linking dynamics proposed in [38] to arbitrary n × n games. The old question arises again: how is the replicator dynamics affected by the stochastic linking dynamics? Motivated by this, we investigate analytically how the fast linking dynamics affects the strategy evolution in n × n games with general imitation processes. We find that it results in a replicator dynamics with a rescaled matrix. Quite intuitively, each entry of the rescaled matrix is

046111-1

©2011 American Physical Society

BIN WU, DA ZHOU, AND LONG WANG


the product of the original entry and the average duration time of the corresponding link. The structure of the paper is as follows: the model is proposed in Sec. II, the analysis is performed in Sec. III, applications of the main result are given in Sec. IV, and finally we end with the conclusion and discussion in Sec. V. II. MODEL

We consider the coevolution of strategy and structure. The strategy interaction is captured by an n × n matrix game M = (Mij ). Initially, the whole population of size N is situated on vertices of a connected regular network with degree L. We further assume that N L, implying that on average each individual has a very limited number of neighbors compared with the population size. At each time step, strategy evolution occurs with probability w (see Fig. 1). Here w governs the dynamical time scales of these two coupled updating moves. It has been shown that the game dynamics on a static network, i.e., w → 1 in our model, results in a transformation of the original payoff matrix under weak selection [19]. Here, we consider the other limit, the fast linking dynamics referring to w 1 [29–31]. In this case, individuals are much more reluctant to change strategies than to adjust their partnerships. The general imitation processes are adopted [41] to capture the strategy evolution. We are aiming to obtain the coevolutionary dynamics of the strategy evolution under the fast linking dynamics. The rules of strategy updating and partner switching are defined as follows: Strategy evolution: We adopt the general imitation process on networks. Each individual is allowed to play with all of its current neighbors and obtains an accumulated payoff. A player a is selected at random and subsequently a player b is randomly selected among a’s current neighbors. Then a takes b’s strategy with probability g[β(Pb − Pa )], where Pa and Pb are the accumulated payoffs for a and b; β 0 denotes the imitation intensity, measuring how strongly the imitation behavior depends on the payoff difference. Here g(x) is a strictly increasing function with g(0) > 0 [41]. For simplicity, we assume that g(x) is differentiable. In particular, when g(x) = 1/[1 + exp(−x)], this refers to the well-known Fermi update rule [42,43]. Linking dynamics: Denote ij as the type of link whose two extremes are of strategy i and strategy j . The collection of all the possible social ties yields Sn = {ij |1 i j n}.

(1)

There are at most n(n + 1)/2 types of social ties in the population. In order to characterize widely observed uncertainties in various kinds of relationships or the noise effect in the topological evolution [38], we denote kij as the probability with which the ij link breaks. In what follows, we describe the linking dynamics in detail: Step (i): A link is randomly selected and we assume the link is of type ij ∈ Sn . Step (ii): The ij link is broken with probability kij . Otherwise it remains connected. Step (iii): If the link ij is broken, a player will be randomly chosen between the two individuals occupying the

FIG. 1. (Color online) The coevolutionary dynamics of strategies and links. With probability w, the strategy update happens. In this case, select a random individual (e.g., a strategy 1 player) in the population and it imitates the strategy of another randomly selected neighbor (e.g., a strategy 2 player) based on the payoff difference. The richer the strategy 2 player is, the more likely it is to be imitated by the strategy 1 player. Otherwise, the linking dynamics happens with probability 1 − w. A random link is selected, and it breaks with a prescribed probability dependent on the link type. If it indeed breaks, randomly select an extreme of the broken link and then switch to a random individual in the population who is not its current neighbor. As an illustration, the dashed link is selected and broken with probability k12 , where 12 indicates the link type, and the strategy 1 player of the broken link is selected, eventually it connects with another strategy 2 player. In this way, a 12 links becomes another 12 link.

two extremes of the broken link. Subsequently, the player switches to a random player who is not its current neighbor in the population (see Fig. 1). Here, for step (i), if one of the nodes of the selected link has only one neighbor, then we directly go back to step (i) again. It is noteworthy that kij is time-invariant and also an intrinsic quantity of the linking dynamics. The duration time of the ij link obeys the geometric distribution with parameter kij [44,45]. That is, the inverse of kij can be taken as the the average duration time or the interaction rate between i and j . In other words, kij is a measurement of the loyalty of the ij link. Besides, the total number of links H = LN/2 is constant in the coevolution process as in [23,28,40]. This constraint implies a limited resource environment. In fact, individuals obtain their payoffs via links. In other words, links are the media via which individuals get payoffs. Thus links can be referred to as some kind of resource. Thus the invariance in the total number of links can be regarded as a limited resource. III. ANALYSIS AND RESULTS

Initially, we assign each link a number α, ranging from 1 to H , as its name, where H = LN/2 is the total number of links. We specify a link α and denote this link as α 0 , where

046111-2

EVOLUTIONARY DYNAMICS ON STOCHASTIC EVOLVING . . .

the subscript 0 denotes the time step. When linking dynamics happens, one of the following two cases occurs: Case 1. α is chosen and broken; subsequently a new link is established. Case 2. α is not broken and remains unchanged. For Case 1, we denote the newly established link as α 1 . For Case 2, we denote the original α as α 1 . Since either Case 1 or Case 2 happens, α 1 is well-defined. In analogy to α 1 , α s , where s > 1 is the time step in the linking dynamics, is iteratively well-defined. Remarkably, if we denote the link type of α s as T (α s ), then it is found that T (α s ) takes a random walk in the state space Sn . Once the specified link is selected, let us denote the conditional transition matrix as V . Otherwise, the link is not selected and the type of it is not altered, thus the conditional transition matrix is the identity matrix In(n+1)/2 . Therefore, the transition matrix of such a Markov chain is given by G = (1/H )V + [(H − 1)/H ]In(n+1)/2 .

(2)

For the conditional transition matrix V = [V(XY )(ZW ) ][n(n+1)/2]×[n(n+1)/2] , each entry V(XY )(ZW ) indicates the transition probability with which a link of type XY transforms to a link of type ZW provided it is selected. Moreover, its entries can be categorized into three classes: (i) If {X,Y } {Z,W } = ∅, then V(XY )(ZW ) = 0, since it is impossible that the original link and the transformed one share no identical extreme under our linking dynamics. (ii) If {X,Y } {Z,W } = {X}, without loss of generality we assume X = Z. Further, (a) if Y = W and X = Y , then V(XY )(ZW ) = kXY xW /2, since the original and the transformed link differ from each other and share only one extreme, thus the original link has to be broken (with probability kXY ), and the node with strategy X is selected (with probability 1/2); finally it switches to another individual of strategy W (with probability xW , where xW is the fraction of strategy W ); (b) if Y = W and X = Y , after the original link is broken with probability kXY , no matter which node is selected, it is definitely of strategy X; then V(XY )(ZW ) = kXX xW ; (c) if X = Y = Z = W , then V(XX)(XX) = (1 − kXX ) + kXX xX , since this happens either when the link is not broken (with probability 1 − kXX ) or when the link is broken (with probability kXX ) and the selected node that must be X switches to another strategy X individual (with probability xX ). (iii) If {X,Y } {Z,W } = {X,Y }, without loss of generality we assume X = Z, Y = W , and X = Y ; then V(XY )(ZW ) = kXY (xX + xY )/2 + (1 − kXY ). In particular, for 2 × 2 games,

V = 11 12 22

11 ⎛ 1 − k11 x2 ⎝ k12 x1 /2 0

12 k11 x2 1 − (k12 /2) k22 x1

22 ⎞ 0 k12 x2 /2 ⎠, 1 − k22 x1


(see Appendix A), which paves the way to obtain the stationary distribution πij = a(x)(2 − δij )xi xj /kij , ij ∈ Sn ,

(4)

where δij indicates the Kronecker delta, x = (x1 ,x2 , . . . ,xn )T , and a(x) = [ ij ∈Sn (2 − δij )xi xj /kij ]−1 normalizes the distribution. Thus the normalization factor a(x) is dependent on both the uncertainties of the linking dynamics kij ’s and on the composition of the population. The normalized stationary distribution provided in Eq. (4) thus indicates the fraction of ij links in the population [44]. The stationary number of XY links can be given by H πXY , i.e., E(UXY ) = H πXY .

(5)

In particular, if the linking dynamics is sufficiently fast compared to the strategy evolution, i.e., w 1, every link in the population is almost in the stationary state when the strategy evolution occurs. Therefore, the average fitness function of strategy i reads ⎛ ⎞ n fi = ⎝ E(Uij )Mij (1 + δij )⎠ (N xi ). (6) j =1

Here the numerator and the dominator of the right-hand side (r.h.s.) of Eq. (6) represent the total payoff of strategy i and the number of the strategy i individuals. Taking Eqs. (4) and (5) as well as H = LN/2 into Eq. (6) leads to ⎛ ⎞ n M˜ ij xj ⎠ , fi = La(x) ⎝ (7) j =1

where M˜ ij = Mij /kij .

(8)

For a general imitation process, the strategy evolution can be approximated by high-dimensional stochastic differential equations [13]. When the population size is sufficiently large, the stochastic term vanishes. The approximation degenerates to an ordinary differential equation, x˙i = xi

n

xj {g[β(fi − fj )] − g[β(fj − fi )]}.

(9)

j =1

We will show that this dynamics bears the identical internal equilibria as the replicator dynamics with the payoff matrix ˜ i.e., Eq. (8), M, x˙i = xi

n ˜ ˜ i − x T Mx]. [(Mx)

(10)

j =1

(3)

where 11, 12, and 22 indicate the three types of links in the population [38]. Note that this Markov chain is irreducible and aperiodic provided ni=1 xi XY ∈Sn kXY = 0. In this case, there exists a unique stationary distribution π , satisfying π G = π . From Eq. (2), π is equivalently determined by π V = π . Further, we find that this chain V fulfills the detailed balance condition

Further, the same equilibrium also has the same local behavior provided it is hyperbolic. For the internal equilibria of Eq. (9), they fulfill g[β(fi − fj )] − g[β(fj − fi )] = 0. Let F (z) = g(z) − g(−z), then F (z) = 2g (z) > 0, thus F (z) is an increasing function. Therefore, there exists a unique solution for F (z) = 0. Obviously, we have F (0) = 0, thus z = 0 is the solution. In other words, the ˜ i = (Mx) ˜ j . Besides, internal equilibria fulfill fi = fj or (Mx) the normalization condition ni=1 xi = 1 should be fulfilled. Note the internal equilibria of Eq. (10) also fulfill these two

046111-3



conditions [8]. Thus Eqs. (9) and (10) share the identical internal equilibria. For the local behavior of the internal equilibrium in Eq. (9), denoted as x ∗ , we linearize the r.h.s. of Eq. (9) in the vicinity of x ∗ , which leads to the Jacobian matrix J˜ij = g (0)βLa(x ∗ ) 2xi∗ (M˜ ij − M˜ sj )xs∗ , (11)

s

∗

where Jij is the Jacobian matrix of Eq. (10) at x . Note that J˜ij scales Jij with g (0)βLa(x ∗ ) > 0; these two matrices share the same eigenvalues up to this positive factor. Thus, if the fixed point is hyperbolic, i.e., zero is not the eigenvalue of Jij , the local stability of Eq. (9) around x ∗ is the same as that of Eq. (10). For n × n games with general imitation processes, we analytically get that the evolution of strategy can be approximated by the replicator equation with the rescaled matrix in Eq. (8), where each entry is the product of the original one and the lifetime of the corresponding link. IV. EXAMPLES

We have shown that fast linking dynamics results in the replicator dynamics with a rescaled matrix. In this subsection, we employ this result to investigate how the fast linking dynamics influences the biodiversity in the rock-paper-scissors (RPS) game and the evolutionary stability of the conditional cooperation strategy in the repeated prisoner’s dilemma. A. Rock-paper-scissors game

The rock-paper-scissors game is a two-player game with three strategies: “rock”(R), “paper”(P), and “scissors”(S). They follow the cyclic dominance rule: R beats S, S beats P, and P beats R. For simplicity, we assume the winner gets 1 and the loser gets −a with a > 0. When it is a tie, each gets 0 [46]. Thus the payoff matrix can be denoted as R ⎛ 0 ⎝ 1 −a

P −a 0 1

S 1 ⎞ . −a ⎠ 0

⎛

R

P

0 R ⎜ P ⎝ 1/kPR −a/kSR S

Jij

R P S

case (see Appendix B) leads to a replicator dynamics with a transformed payoff matrix,

−a/kPR 0 1/kSP

S

⎞ 1/kRS ⎟. −a/kPS ⎠ 0

(13)

The internal equilibrium of the replicator equation for Eq. (13) is λ[kPR kRS (a 2 kRS kSP + akPR kRS + kPR kSP ), kPR kSP (a 2 kRS kPR + akPR kSP + kRS kSP ), kRS kSP (a 2 kPR kSP + akSP kRS + kPR kRS )],

(14)

where λ > 0 is the normalization factor. To study the stability of this equilibrium, we also have the determinant of Eq. (13), 1 − a3 . kRS kSP kPR When it is positive, i.e., 0 < a < 1, the internal equilibrium is globally stable, and it is globally unstable when a > 1 [8]. Therefore, the fast linking dynamics only changes the position of the equilibrium with the stability of the equilibrium unchanged (see Fig. 2). In particular, for a = 1, Eq. (13) also represents a zero sum game as Eq. (12) for all the nonzero breaking probability kij . In general, increasing the breaking probability between two strategies only lightens the competition between the winner and the loser. Furthermore, the loser loses as much as the winner gains, thus the two forces offset each other. Thus the stability of the internal equilibrium is not altered. Figures 2 and 3 show both analytically and numerically that the internal fixed point moves away from P as kRS increases. Intuitively, increasing kRS decreases the interaction rate between R and S. In this way, R’s meet more rarely with S’s. This benefits S’s, which eventually leads to a decrease in the population size of P’s.

(12)

This game is widely employed for the study of the maintenance of biodiversity [6,47,48]. The internal equilibrium of the replicator dynamics for this game in structureless populations is (1/3,1/3,1/3). The stability of the equilibrium is determined by the sign of the determinant of the payoff matrix in Eq. (12), 1 − a 3 . When it is positive, i.e., 0 < a < 1, the internal equilibrium (1/3,1/3,1/3) is globally stable, implying that biodiversity is maintained, since all three strategies (or species) coexist for a long time; for a > 1, the internal equilibrium is globally unstable [8], leading to the loss of biodiversity. Now we consider our coevolutionary RPS game. Since the population size is sufficiently large, we neglect the finite population effect that two of the three species become extinct sooner or later [49,50]. The fast linking dynamics in this

FIG. 2. (Color online) The internal fixed points and flow fields of the replicator equations for the rock-paper-scissors game with active linking dynamics in large populations. Each of the three plots has an internal equilibrium. The fixed points are all stable, which is in agreement with our theoretical prediction. But the internal fixed point moves away from P as kRS increases. In other words, the more fragile the RS link is, the less likely P is to be prosperous. Intuitively, increasing kRS decreases the interaction rate between R and S. Thus the dynamic flow around the edge RS is slower. This paves the way for the equilibrium to move toward the RS link or far away from P. For all the plots, kPR = kSP = 0.5, a = 0.7. (Images are generated by a modified version of the DYNAMO package [51].)

046111-4



the population is invaded by some defectors, defection is likely to dominate. Thus TFT is not an evolutionary stable strategy. Here evolutionary stable strategy refers to the strategy that can resist the invasion of any strategy provided the mutant is sufficiently rare. This is one of the drawbacks of TFT [56–58]. Now we incorporate the repeated PD with the fast linking dynamics. The fast linking dynamics leads to the replicator dynamics with a rescaled matrix, ALLC ALLD TFT

⎛

ALLC

(b − c)m/k ⎜ ⎝ bm/γ (b − c)m/γ

ALLD −cm/γ 0 −c/γ

TFT

⎞ (b − c)m/γ ⎟. b/γ ⎠ (b − c)m/k

(16)

where k and γ refer to the breaking probabilities between two identical and different strategies. By the last column of Eq. (16), when (b − c)m/k > b/γ and (b − c)m/k > (b − c)m/γ , i.e., FIG. 3. (Color online) Frequencies for rock and scissors at the stationary regime. The frequency of paper can be given by 1 − xR − xS . For different initial conditions and the same kRS , the stationary equilibria obtained by the simulations are almost the same. This indicates the stability of the internal equilibrium. Concerning the stationary frequencies of rock, xR , and scissors, xS , simulation results show that they both increases with the interaction rate between rock and scissors, kRS , as predicted by Eq. (14). The simulation results agree well with the analytical predictions. Here the population size N = 400, selection intensity β = 10, and strategy updating frequency w = 0.01. Besides, kPR = kSP = 0.5 and a = 0.7, which are the same as that in Fig. 2. For each data point, it is averaged over 50 independent runs. In each run, for a specified strategy, we set the mean value over a time window of 103 generations to be the final frequency after a transient time of 106 generations.

k/γ < min{1,m(1 − c/b)},

(17)

TFT becomes an evolutionary stable strategy (see Fig. 4) [59]. By inequality (17), the necessary condition for a stable TFT is k < γ , i.e., the interaction between the identical strategies happens more often than that between different strategies. This ensures that ALLC cannot easily increase in number by random drift any longer. Instead, TFT and ALLC play like a coordination game such that TFT can resist the invasion of ALLC provided the mutant is rare. This is because the

B. Repeated prisoner’s dilemma

The prisoner’s dilemma (PD) is a game with two strategies: cooperation and defection. A cooperator (C) pays a cost c and the opponent receives a benefit b, while a defector (D) pays nothing and the opponent gets nothing [52]. It is noteworthy that this donor-recipient version of PD is only a special case for simplicity. In this dilemma, cooperation cannot be favored unless additional mechanisms are involved [53]. One such mechanism is direct reciprocity [31,54,55]. Imagine the game is repeated m times. Consider the following three strategies: tit-for-tat (TFT), which cooperates in the first round and does whatever the opponent did in the previous round; ALLC, which cooperates all through the m rounds; and ALLD, which defects all through the m rounds [5]. The payoff matrix for the repeated PD then reads ALLC ALLD TFT

⎛

ALLC

(b − c)m ⎜ ⎝ bm (b − c)m

ALLD −cm 0 −c

TFT

⎞ (b − c)m ⎟. b ⎠ (b − c)m

(15)

In a homogenous population of TFT, TFT can resist the invasion of ALLD, provided m > b/(b − c). But this is not the whole story. When the population is composed of ALLC and TFT, the frequency of cooperators can increase by random drift since they both have the same payoff. In this case, when

ALLC ALLD ALLC ALLD ALLC ALLD

FIG. 4. (Color online) The frequency for TFT at the stationary regime. The simulation result shows when the population is large in the frequency of TFT (or small in the frequencies of ALLC and ALLD); TFT finally takes over the whole population for k < 0.5. In other words, it resists the invasion of cooperators and defectors. This implies that TFT becomes an evolutionary stable strategy for k < 0.5. This is consistent with the analytical prediction by the inequality (17) with our simulation parameters b = 5, c = 1, m = 5, and γ = 0.5. Other parameters are the same as those in Fig. 3. For k > 0.5, the simulation results do not perfectly agree with the analytical one. This is due to the finite population effect as well as the inaccuracy of the transition matrix for the linking dynamics [38].

046111-5



interaction between TFT’s happens more often than TFT and ALLC, which leads to a higher effective payoff of TFT over ALLC when ALLC is rare in number.

strategy j is proportional to the interaction time between them. ACKNOWLEDGMENTS

V. CONCLUSION AND DISCUSSION

Coevolutionary dynamics of strategy and structure has received considerable attention over the past decade [20,22,23,27–31,37]. It reveals a universal principle in nature that not only do the states of individuals evolve, but also the relations among them vary. Thus coevolution is an important source of dynamical complexities [60]. There are, however, few analytical methods for the general dynamics of coevolution. This is mainly because complications arise from the coupled dynamics between the strategy update and the linking dynamics. Compared to the 2 × 2 coevolutionary games, the general multiple-strategy cases are significantly more difficult as complications arise from both the dynamical coupling and the high dimension [13]. Inspired by these, we generalize our previous linking dynamics [38] to an arbitrary number of strategies and investigate the coevolution dynamics for an n × n matrix with general imitation processes. We have revealed that, in our coevolutionary model, the effective payoff of strategy i against strategy j is the corresponding original payoff multiplying the average duration time of the ij link. This rescaled matrix arises from nonuniform interaction rates between different strategies, kij , caused by the linking dynamics. Technical difficulties for the generalization occur because the number of states for the Markov chain describing the linking dynamics increases quadratically with the number of the strategy n, making it difficult to analytically obtain the stationary distribution. In our model, the detailed balance condition helps us to overcome this obstacle. Intuitively, the coevolutionary dynamics of strategy with active linking dynamics could lead to more complex behaviors than that without linking dynamics. In contrast, we have shown that the evolution of n × n games by the fast linking dynamics can also be captured by the replicator dynamics only with the matrix rescaled. In other words, our coevolutionary mechanism with fast linking dynamics does not increase the dynamical complexity in the sense of the maximum number of equilibria: both the original and the transformed bear at most n − 1 internal equilibria [61]. More interestingly, this result is robust to a wide range of imitation processes. Furthermore, based on this replicator dynamics with the rescaled matrix, we can anticipate the evolutionary fate of any strategy in the long run. Here, it is worth noting that our approach is of the bottom-up style. That is to say, we first establish the stochastic microscopic linking dynamics, from which we analytically derive the macroscopic dynamics of the coevolutionary system. In agreement with the work of Pacheco et al. [29], timescale separation also plays an important role in our study. In both models, fast linking dynamics paves the way to getting the stationary regime of the topology structure. Our linking dynamics, however, explicitly involves the stochastic effect. More importantly, the resulting payoff matrix transformed by our linking dynamics has a more explicitly intuitive explanation: the effective payoff of strategy i against

We greatly acknowledge the comments from A. Traulsen, F. Fu, X. J. Chen, Y. L. Zhang, and J. L. Zhang. We also thank the referees for a careful reading of the paper and comments. This work was supported by 973 Program (2012CB821203) and NSFC (61020106005 and 10972002). APPENDIX A: DETAILED BALANCE CONDITION FOR THE LINKING DYNAMICS

In this appendix, we discuss the reversibility of the Markov chain in Eq. (2). Kolmogorov’s criterion [45] states that a Markov chain is reversible if and only if its transition probabilities satisfy pi1 ,i2 pi2 ,i3 · · · pin−1 ,in pin ,i1 = pi1 ,in pin ,in−1 · · · pi3 ,i2 pi2 ,i1

(A1)

for all finite sequences of states. Kolmogorov’s criterion indicates that, starting from some state i1 , the probability of any path that ultimately returns to i1 must be equal to the probability that this path is traced in the other direction. It is not difficult to show that V = [V(XY )(ZW ) ][n(n+1)/2]×[n(n+1)/2] satisfies Kolmogorov’s criterion. For example, given a starting state XX, consider a path XX → XY → Y Z → ZX → XX, where X = Y = Z = X. Then V(XX)(XY ) V(XY )(Y Z) V(Y Z)(ZX) V(ZX)(XX) = kXX xY × kXY 12 xZ × kY Z 12 xX × kZX 12 xX = 18 kXX kXY kY Z kZX xX2 xY xZ

(A2)

and V(XX)(XZ) V(XZ)(ZY ) V(ZY )(Y X) V(Y X)(XX) = kXX xZ × kXZ 12 xY × kZY 12 xX × kY X 12 xX = 18 kXX kXZ kZY kY X xX2 xY xZ .

(A3)

Note that kXY = kY X , kXZ = kZX , and kY Z = kZY , which means that Eqs. (A2) and (A3) are equal to each other. For other paths, we can check the criterion in the same way. Therefore, with Kolmogorov’s criterion, it can be shown that our stochastic linking dynamics satisfies the detailed balance condition, that is, πXY V(XY )(ZW ) = πZW V(ZW )(XY ),

(A4)

where πXY is the stationary distribution of V . This paves the way to obtaining the stationary distribution in Eq. (4). APPENDIX B: TRANSITION MATRIX FOR n = 3

For n = 3, such as a rock-paper-scissors game, there are six types of links. The conditional transition matrix V(XY )(ZW )

046111-6



is given by RR SS PP RS RP SP

RR SS 0 ⎛ 1 − kRR (1 − xR ) 0 1 − k SS (1 − xS ) ⎜ ⎜ 0 0 ⎜ ⎜ kRS xR kRS xS ⎜ ⎜ 2 2 ⎜ kRP xR 0 ⎝ 2 0

kSP xS 2

PP 0 0 1 − kPP (1 − xP )

RS kRR xS kSS xR 0 1−

0 kRP xP 2 kSP xP 2

kRS (1+xP ) 2 kRP xS 2 kSP xR 2

RP kRR xP 0 kPP xR 1

kRS xP 2 S) − kRP (1+x 2 kSP xR 2

SP 0 kSS xP kPP xS

1

kRS xP 2 kRP xS 2 R) − kSP (1+x 2

⎞ ⎟ ⎟ ⎟. ⎟ ⎟ ⎟ ⎟ ⎠

The stationary distribution of this chain is πij = a(x)(2 − δij )xi xj /kij . Here a(x), where x = (xR ,xS ,xP ), is given by −1 a (x) = xR2 kRR + xP2 kPP + xS2 kSS + (2xR xP /kRP ) + (2xR xS /kRS ) + (2xS xP /kSP ) .

[1] J. M. Smith, Evolutionary Genetics (Oxford University Press, Oxford, 1998). [2] M. A. Nowak, Evolutionary Dynamics (Harvard University Press, Cambridge, MA, 2006). [3] K. Sigmund, The Calculus of Selfishness (Princeton University Press, Princeton, NJ, 2010). [4] D. Friedman, Econometrica 59, 637 (1991). [5] R. Axelrod, The Evolution of Cooperation (Basic Books, New York, 1984). [6] T. Reichenbach, M. Mobilia, and E. Frey, Nature (London) 448, 1046 (2007). [7] E. Matsen and M. A. Nowak, Proc. Natl. Acad. Sci. (USA) 101, 18053 (2004). [8] J. Hofbauer and K. Sigmund, Evolutionary Games and opulation Dynamics (Cambridge University Press, Cambridge, 1998). [9] J. Weibull, Evolutionary Game Theory (MIT Press, Cambridge, 1995). [10] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Nature (London) 428, 646 (2004). [11] L. A. Imhof and M. A. Nowak, J. Math. Biol. 52, 667 (2006). [12] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev. Lett. 95, 238701 (2005). [13] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev. E 74, 11901 (2006). [14] K. Lindgren and M. G. Nordahl, Physica D 75, 292 (1994). [15] G. Szabó and C. Hauert, Phys. Rev. Lett. 89, 118101 (2002). [16] F. C. Santos and J. M. Pacheco, Phys. Rev. Lett. 95, 098104 (2005). [17] G. Szabó and G. Fáth, Phys. Rep. 446, 97 (2007). [18] H. Ohtsuki, C. Hauert, E. Lieberman, and M. A. Nowak, Nature (London) 441, 502 (2006). [19] H. Ohtsuki and M. A. Nowak, J. Theor. Biol. 243, 86 (2006). [20] B. Skyrms and R. Pemantle, Proc. Natl. Acad. Sci. (USA) 97, 9340 (2000). [21] V. Bala and S. Goyal, J. Econ. Theory 17, 101 (2001). [22] H. Ebel and S. Bornholdt, e-print arXiv:cond-mat/0211666. [23] M. G. Zimmermann and V. M. Egu´ıluz, Phys. Rev. E 72, 056118 (2005).

(B1)

(B2)

[24] J. Poncela, J. Gómez-Gardeñes, A. Traulsen, and Y. Moreno, New J. Phys. 11, 083031 (2009). [25] N. Hanaki, A. Peterhansl, P. S. Dodds, and D. J. Watts, Manag. Sci. 53, 1036 (2007). [26] C. Biely, K. Dragosit, and S. Thurner, Physica D 228, 40 (2007). [27] F. Fu, C. Hauert, M. A. Nowak, and L. Wang, Phys. Rev. E 78, 026117 (2008). [28] F. C. Santos, J. M. Pacheco, and T. Lenaerts, PLoS Comput. Biol. 2, 1284 (2006). [29] J. M. Pacheco, A. Traulsen, and M. A. Nowak, Phys. Rev. Lett. 97, 258103 (2006). [30] J. M. Pacheco, A. Traulsen, and M. A. Nowak, J. Theor. Biol. 243, 437 (2006). [31] J. M. Pacheco, A. Traulsen, H. Ohtsuki, and M. A. Nowak, J. Theor. Biol. 250, 723 (2008). [32] S. V. Segbroeck, F. C. Santos, T. Lenaerts, and J. M. Pacheco, Phys. Rev. Lett. 102, 058105 (2009). [33] S. V. Segbroeck, F. C. Santos, A. Nowé, J. M. Pacheco, and T. Lenaerts, BMC Evol. Biol. 8, 287 (2008). [34] G. Zschaler, A. Traulsen, and T. Gross, New J. Phys. 12, 093015 (2010). [35] A.-L. Do, L. Rudolf, and T. Gross, New J. Phys. 12, 063023 (2010). [36] T. Gross and B. Blasius, J. R. Soc. Interface 5, 259 (2008). [37] M. Perc and A. Szolnoki, Biosystems 99, 109 (2010). [38] B. Wu, D. Zhou, F. Fu, Q. Luo, L. Wang, and A. Traulsen, PLoS ONE 5, e11187 (2010). [39] S. V. Segbroeck, F. C. Santos, T. Lenaerts, and J. M. Pacheco, New J. Phys. 13, 013007 (2010). [40] K. Fehl, D. J. van der Post, and D. Semmann, Eco. Lett. 14, 546 (2011). [41] B. Wu, P. M. Altrock, L. Wang, and A. Traulsen, Phys. Rev. E 82, 046106 (2010). [42] L. E. Blume, Games Econ. Behav. 5, 387 (1993). [43] G. Szabó and C. T˝oke, Phys. Rev. E 58, 69 (1998). [44] S. Karlin and H. M. A. Taylor, A First Course in Stochastic Processes, 2nd ed. (Academic, London, 1975). [45] R. Durrett, Probability: Theory and Examples (Duxbury Press, Belmont, CA, 2005).

046111-7



[46] J. C. Claussen and A. Traulsen, Phys. Rev. Lett. 100, 058104 (2008). [47] B. Kerr, M. A. Riley, M. W. Feldman, and B. J. M. Bohannan, Nature (London) 418, 171 (2002). [48] M. Frean and E. R. Abraham, Proc. R. Soc. London, Ser. B 268, 1323 (2001). [49] R. Durrett and S. A. Levin, Theor. Pop. Biol. 46, 363 (1994). [50] M. Berr, T. Reichenbach, M. Schottenloher, and E. Frey, Phys. Rev. Lett. 102, 048102 (2009). [51] W. H. Sandholm and E. Dokumaci, http://www.ssc.wisc.edu/ whs/dynamo (2007). [52] R. Axelrod and W. D. Hamilton, Science 211, 1390 (1981). [53] M. A. Nowak, Science 314, 1560 (2006). [54] R. L. Trivers, Q. Rev. Biol. 46, 35 (1971).

[55] L. A. Imhof, D. Fudenberg, and M. A. Nowak, Proc. Natl. Acad. Sci. (USA) 102, 10797 (2005). [56] R. Selten and P. Hammerstein, Behav. Brain Sci. 7, 115 (1984). [57] R. Boyd and J. Lorberbaum, Nature (London) 327, 58 (1987). [58] L. A. Imhof, D. Fudenberg, and M. A. Nowak, J. Theor. Biol. 247, 574 (2007). [59] J. Maynard Smith, Evolution and the Theory of Games (Cambridge University Press, Cambridge, 1982). [60] Adaptive Networks: Theory, Models and Applications Understanding Complex Systems, edited by T. Gross and H. Sayama (Springer-Verlag, New York, 2009). [61] C. S. Gokhale and A. Traulsen, Proc. Natl. Acad. Sci. (USA) 107, 5500 (2010).

046111-8