A Chemical Model of the Naming Game

0 downloads 0 Views 914KB Size Report
ysis, Artificial Chemistry, The Naming Game, Idiotypic Networks. 1 Introduction ... Both players write a list of names on a paper, and papers are exchanged before.
A Chemical Model of the Naming Game Joachim De Beule Artificial Intelligence Lab, Vrije Universiteit Brussel Pleinlaan 2, 1050 Brussel [email protected] http://arti.vub.ac.be/~joachim

Abstract. A key feature of many biological distributed systems is that they have the capacity to behave in highly coordinated ways. In the domain of language, such coordination dynamics have been studied within the framework of language games. As yet however, a fundamental understanding that goes beyond the simplest cases is still missing. In this paper, a novel approach is proposed for investigating coordination problems. I illustrate the approach for a simple but well studied case called the naming game. I will therefore bring together a number of ideas from Artificial Chemistry and Chemical Reaction Network Theory, Semiotic Dynamics and Immunology, and conclude by arguing why the proposed approach provides a good starting point for tackling more complex coordination problems as well. Key words: Coordination, Semiotic Dynamics, Agent Response Analysis, Artificial Chemistry, The Naming Game, Idiotypic Networks

1

Introduction

A key feature of many biological distributed systems is that they have the capacity to behave in collective and highly coordinated ways. In the domain of language, such coordination dynamics have already extensively been studied using the concept of language games[10]. In a language game, a population of agents (language users) needs to bootstrap a ‘language’ by engaging in pairwise interactions. For example, in the naming game they have to agree upon a name for an object. In more complex games, agents would benefit from using more complex encodings that go beyond simple naming and involve syntax, similar to natural languages inducing a conventional and multi-levelled mapping between hierarchically structured meanings and forms [6, 9], or to the immune system which is capable of responding appropriately in a virtually infinite number of different situations [7]. Although (especially in recent years) the naming game has become very well understood (see e.g. [2]), as yet only very few results exist that go beyond it and a fundamental understanding of the dynamics involving the emergence and evolution of syntax is still lacking.

2

Joachim De Beule

In this paper, I propose to model agents as artificial cellular organisms interacting with their environment through the absorption and secretion of artificial chemical substances (henceforth called species) and in turn modelled as continuous stirred flow tank reactors with entrapped species [3]. Within a cell/reactor, species interact themselves and new species are formed according to a reaction network corresponding to the learning or entrenchment of linguistic constructions in more traditional agents. This approach lends itself particularly well for performing an agent response analysis [11, 4], which allows do draw conclusions about the behavior of a population of agents on the basis of a single agent only. In the following section, this will be illustrated for the case of the naming game. A reactor agent solving the naming game will be defined and a response analysis will be performed on it. This will lead to the identification of a design principle for synthesizing artificial agents solving more complex coordination problems, as will be discussed in the final section of the paper.

2

A Reactor Agent Solving the Naming Game

Consider the following game. Every round two (random) players enter a room. Both players write a list of names on a paper, and papers are exchanged before they leave the room again. The game ends if all players consistently write down one and the same name. This game is called the naming game. In traditional approaches, players are called agents. They typically keep a memory of names, possibly with a preference measure (a score), and apply one or the other lateral inhibition mechanism after each game: names in the other player’s list are promoted at the expense of other names. It is not primarily clear however how this approach scales to more complex languages. Now consider the chemical reactor tank in Figure 1(b) as a model for a cellular organism playing the naming game (Figure 1(a).) On the left side, the

fi0 fi

fi0

fi fi0

fi and ci according to (2)

fi , ci

(a)

fi

(b)

Fig. 1. A cellular agent (a), modelled as an isothermal homogeneous continuous flow stirred tank reactor (b). The reactor is supplied with a continues feed of form species (names), with molar concentrations fi0 . Inside the reactor/cell new species ci are formed according to the idiotypic reaction network shown in Figure 2. Forms are also continuously extracted from the reactor/cell (with fi the molar concentrations of form species i both in the reactor/cell and in the outflow.) The new species remain entrapped in the cell.

A Chemical Model of the Naming Game

3

reactor is fed with a continues supply of name species corresponding to the other player’s lists of names (fi0 ).1 More formally, fi0 denotes the molar concentration of names of type i in the influx (where i = 1..nf , nf being the number of different names). With nf X fi0 ≡ 1 , (1) i

which is arbitrary, you can think of fi0 as the frequency with which other players use name i. Similarly, fi denotes the frequency with which the agent under investigation uses name i in a game itself. Because of the stirred tank hypothesis, it also corresponds to the molar concentration of name i withing the agent. Inside the agent, each form fi lives in a co-existential balance with an antiidiotypic species ci according to the first two reactions in Figure 2. These thereby form a kind of memory similar to what was already proposed in [7].

k

1 ci fi −→

k

2 fi ci −→

k

3 ci + fi cj + fi −→

Fig. 2. An artificial chemical reaction network. According to the first two reactions, form species fi trigger the existence of an anti-idiotypic species ci and vice versa. Setting the reaction rate k2 = 1 + fi renders species fi autocatalytic.

I will however add a crucial element to this picture, namely that the equilibrium balance ratio ci /fi depends on the amount of available species (i.e. ci and/or fi ). This can be accomplished in several ways. For example, assume that k1 = k3 = 1 and that k2 = 1+fi . The last condition renders species fi autocatalytic. It makes the equilibrium ratio’s ci /fi = k1 /k2 induced by the first two idiotypic reactions in Figure 2 depend on the amount of available fi . This will turn out to be crucial when considering the interplay with the third reaction. To see this, let us turn back to the complete reactor model. Define X X σf ≡ fi ; σc ≡ ci , (2) i

i

and assume a mass-action kinetics. If form species are supplied and extracted at a volumetric flow rate ρf (with dimension volume/time) and with c∗i ≡ fi /(1 + fi ) , 1

(3)

I use f to denote names to emphasize the generality of the approach: their is no restriction on the sort of species in the influx, and in more complex games instead of names there will also be other ‘elementary’ as well as more complex ‘parts of form’.

4

Joachim De Beule

we arrive at the following system of differential equations describing the agent state change over time when confronted with a particular external (population) behavior fi0 : f˙i = ρf (fi0 − fi ) − (1 + fi )(c∗i − ci ) , ci σc − ). c˙i = (1 + fi )(c∗i − ci ) + fi σf ( σf fi

(4)

Note that because of the reactor hypothesis we can be certain that fi > 0 if ρf > 0 and fi0 > 0. The first term in f˙i represents the in and out flux of form species. It drives the agent to behave in the same way as the population does (fi0 ). The second term is opposite to the first term in c˙i . It induces the abundance-dependent equilibrium ratio’s as, by definition: (ci = c∗i ) ⇒ (ci /fi = 1/(1 + fi )) .

(5)

In words: less abundant forms will have higher equilibrium ratios. The final term however tries to make all ratios equal to σc /σf . Furthermore, it does so by converting species with higher ratios to species with lower ratios. In other words: by converting already less abundant forms to already proliferating forms. This interplay makes that this agent solves the naming game as will be shown in the next section.

3

Agent Response Analysis

The idea to investigate coordination problems through an agent response analysis was already introduced in [11] and [4]. In this section it will be shown how this method also applies naturally to the reactor agent defined in the previous section. For this, and with ℜ+ denoting all positive real numbers, define the agent state space Q = (ℜ+ )2∗nf as the set of all possible agent states q = hfi , ci i, and let the behavior space F = [0, 1]nf be the set of all distributions over forms (i.e. behaviors). Then the agent’s behavior function becomes: f : Q → F : hfi , ci i 7→ fi /σf .

(6)

Furthermore, the agent’s transition function becomes a function: δ : Q × F × ℜ+ → Q ,

(7)

where q(t) = δ(q(0), f 0 , t) is the solution to (4) when integrated over a time t and starting from an initial state q(0) at time 0. Finally, an agent’s response behavior is defined as the agent’s behavior when confronted with a constant population behavior for a very long time. If an agent is ergodic, then this limiting behavior will be independent of the initial state q(0) and converge to a steady state under all circumstances. In this case it makes sense to define the agent’s response function: φ : F → F : f 0 7→ limt→∞ f (δ(q(0), f 0 , t)) .

(8)

A Chemical Model of the Naming Game

5

It that maps a (constant) external population behavior f 0 to the agent’s unique limiting behavior induced by it. Now define f ∗ = c∗

σf , σc

(9)

with c∗ = f ∗ /(1 + f ∗ ) .

(10)

Thus, f represents the form concentration in the influx that occurs with the same concentration in the outflux and corresponds to the equilibrium form concentration of the system (4) under the condition that fi = fi0 .2 Solving for σc and substituting the result into equations (4) with f˙i and c˙i set to zero results in an expression for the agent’s response function φ in terms of fi0 and containing the parameters ρf , σf 0 and f ∗ . It is shown in Figure 3 for σf 0 = 1 and ρf = f ∗ = 0.5 together with the identity relation fi = fi0 . ∗

Fig. 3. The curved line represents the response function of the cellular agent defined in section 2 for values σf = σf 0 = 1 and f ∗ = 0.5 (see text.). It determines the agent’s outflow behavior fi = φ(fi0 ) (Y-axis) given the inflow behavior fi0 (X-axis) and relative to the invariant frequency f ∗ . Input frequencies below f ∗ are dampened while those above it are amplified.

As can be seen, inflow frequencies below the invariant frequency f ∗ are dampened while those above it are amplified. This suggests that if this agent were to 2

Note that, because of the reactor hypothesis, in equilibrium it will hold that σf =

nf X

fi0 ≡ σf 0 ,

(11)

i

and since then also σc + σf is constant, f ∗ is entirely determined by the input distribution f 0 .

6

Joachim De Beule

interact with other agents like itself, they would gradually converge to a state in which only one form remains, thus solving the naming game. In particular this holds if the agent were to interact with itself (i.e. when fi0 = fi or, equivalently, ρf = 0). In [4] it is suggested how, under mean field conditions, the dynamics induced by such a closed system indeed correspond to the average population dynamics. This can be understood as follows. Consider an agent randomly interacting with other agents in the population at times k = 1, 2, .... Let q(k) and f 0 (k) represent the agent’s state and the average population behavior at time k respectively. Every interaction the agent is stochastically influenced by the population behavior and vice versa. With f and δ the agent’s behavior and transition functions, this results in the following set of stochastic difference equations: f 0 (k + 1) = (1 − β)f 0 (k) + βf (q(k)) ; q(k + 1) = δ(q(k), f 0 (k)) ,

(12)

with β ∈ [0, 1] a constant parameterizing the degree of influence an agent has on the population. For large populations, β will be relatively small, and f 0 will remain relatively constant over a large number of interactions, meaning that the agent’s behavior will approach it’s response behavior φ(f 0 ). If we now define β = α, a constant, then the f 0 (k) = f 0 (k∆t) = f 0 (k) and let ∆β → 0 with ∆t following ordinary differential equation is obtained: d 0 f = α(φ(f 0 ) − f 0 ) , dt

(13)

relating the evolution of the average population behavior f 0 to that of the fixed points of the response function φ of the agents constituting the population. In our case these correspond to the equilibrium states of the closed reactor system (i.e. equations (4) with ρf = 0. The top of Figure 4 shows the evolution of such a system in case of nf = 100 forms. As predicted, only one form type remains in the end. The bottom graph shows the corresponding evolution of the coherence and the synonymy (scaled by the number of form types). With wi = fi /σf , these are defined respectively as: Coh(t) =

X

2

(wi (t))

and

nf X Syn(t) = exp( −wi (t)log(wi (t)))

i=1

(14)

i=1

and can be related to the communicative success and the number of words in the population. As can be seen, a very sharp transition occurs around t = 850, in accordance with previous findings resulting from multi-agent simulations (see e.g. [1].)

4

Discussion and Conclusion

In this paper, it was shown how a simple coordination problem like the naming game can conveniently be studied by combining concepts from (artificial)

A Chemical Model of the Naming Game

7

Fig. 4. Evolution of a system governed by equations (4) with ρf = 0 and nf = 100 and starting from a random initial state. The top graph shows the relative total abundances (fi + ci )/(σf + σc ) for all form index types i = 1..100. The bottom graph shows the corresponding evolution of the coherence and (scaled) synonymy of the system, roughly corresponding to the communicative success and the number of words in more traditional setups.

chemical reaction network theory and systems biology with earlier notable attempts to systematize the investigation of coordination problems through an agent response analysis. I argue that this approach is also particularly well suited for investigating more complex coordination problems. The response analysis not only suggests a relation between the behavior of single agents and that of a population, it also points towards an intuitive agent design principle: an agent will only be fit for a particular coordination problem if it can solve it when interacting with itself. By modelling agents as continues flow reactor tanks this condition is easily checked by feeding back an agent its own outflow and performing a stability analysis on the induced set of differential equations. In other words, designing an agent boils down to finding a reaction network inducing a dynamics with the desired properties according to the coordination problem under investigation. Moreover, for this we will now have at our disposal a wealth of powerful insights about reaction networks coming from artificial chemistry and chemical reaction network theory (CRNt) [5, 8]. For example, the question whether a reaction network supports multiple positive stable equilibria is particularly well studied in CRNt. Because coordination problems typically require that agents

8

Joachim De Beule

are able to escape from incompatible (sub-optimal) configurations, I predict that this will be of importance for more complex coordination problems. We will also be able to draw upon (and perhaps even help understand) an increasing amount of available data and insights from systems biology and related fields regarding biological reaction networks. As a first example, I have shown how an idiotypic reaction mechanism, first identified within the context of the immune system, induces a sharp transition from a random and incoherent state to a highly coordinated one thus solving the naming game. I predict that other well studied mechanisms like covalent modification and phosphorylase will also turn out to be useful for tackling more complex coordination problems.3

References 1. A. Baronchelli, M. Felici, E. Caglioti, V. Loreto, and L. Steels. Sharp transition towards shared vocabularies in multi-agent systems. J. Stat. Mech., P06014, 2006. 2. Andrea Baronchelli and Vittorio Loreto. In-depth analysis of the naming game dynamics: The homogeneous mixing case. International Journal of Modern Physics C, 19(5):785–812, May 2008. 3. Gheorghe Craciun and Martin Feinberg. Multiple equilibria in complex chemical reaction networks: Extensions to entrapped species models. Systems Biology, 153(4):179–186, 2006. 4. Bart De Vylder. The Evolution of Conventions in Multi-Agent Systems. PhD thesis, VUB Artificial Intelligence Lab, 2007. 5. Martin Feinberg. Lectures on chemical reaction networks. http://www.che.eng.ohio-state.edu/ FEINBERG/LecturesOnReactionNetworks/, 1979. Lectures given at the Mathematics Research Center of the University of Wisconsin-Madison. 6. The Five Graces Group. Language is a complex adaptive system. DOI: SFI-WP 08-12-047, SFI Working Papers, 2008. 7. Niels Jerne. The generative grammar of the immune system. Bioscience Reports, 5(6):439–451, 1985. Nobel Lecture Note. 8. N. Matsumaru, T. Hinze, and P. Dittrich. Organization-oriented chemical programming for distributed artefacts. to appear, 2009. 9. Luc Steels. Language as a complex adaptive system. In Marc Schoenauer, Kalyanmoy Deb, G¨ unter Rudolph, Xin Yao, Evelyne Lutton, Juan Julian Merelo, and Hans-Paul Schwefel, editors, Proceedings of Parallel Problem Solving from Nature VI, Lecture Notes in Computer Science. Springer, Berlin, Germany, 2000. 10. Luc Steels. Language games for autonomous robots. IEEE Intelligent Systems, sept-oct 2001:17–22, 2001. 11. Bart De Vylder. Coordinated communication, a dynamical systems perspective. In Proceedings of the European Conference on Complex Systems (ECCS06), 2006.

Acknowledgements This work was partly funded by the ComplexDis European sixth framework project (FP6-2005-NEST-PATH). The author would like to thank Luc Steels, Peter Dittrich, Karel van Acoleyen and Bart De Vylder for useful comments. 3

In fact, in a subsequent paper, I will show how the introduction of domains akin to protein domains allows to decompose a more complex coordination problem called the guessing game into several instances of the naming game.