Knowledge, Rationality and Action Wiebe van der Hoek Department of Computer Science University of Liverpool Liverpool L69 7ZF, United Kingdom Email:
[email protected] Abstract I argue that the notions of knowledge, rationality and action form a natural and interesting triple when studying agent systems: first of all, the combination and interplay between them is still not fully explored, and secondly, they give rise to rich theories especially in multi-agent systems. The paper provides an informal overview of formal approaches to the three notions.
1. Introduction This paper accompanies my invited talk at AAMAS 2004, and as such it is not a report on new results in my area of research. Neither is it a survey: the number of pages that I have to my disposal cannot do justice to all the relevant current research in the field— let alone to give a concise overview or even reference to its predecessors or original roots. The aim of this paper is to provide informally the reader with a feel of the formal —and indeed, logical— approaches to Knowledge, Rationality and Action, with the aim of convincing him and her that this is an intriguing and mathematically rich area, facilitating the modeling of many domains in which agents interact. The bibliography should be helpful for those readers whose appetite is whetted. A well-accepted principle in planning in good old fashioned AI, especially in robotics (where it is often referred to as “reactive planning”), is the sense-plan-act cycle, of which the name is almost self-explanatory: the robot senses its environment, tries to fire one of its plans, which generates an action to be taken, after which the robot inspects the outside world again, possibly taking into account the effect of its action, which might lead to a modified plan, which generates a next action to be taken. The paradigm of agents extends this model, but at the same time leaves the principle the same: agents constantly collect information in order to keep their knowledge accurate, which is then, based
on some rationality principles, used to make an appropriate decision in the form of an action. The discipline of research in agents extends the senseplan-act cycle since it provides each of the phases with a richer dimension, the most important arising from the fact that, apart from the ‘outside world’, every agent is also assigned its own mental state, providing a stage for informational, motivational and even emotional attitudes. This implies for instance that we can identify introspective agents, who not only sense the outside world, but who also can inspect their own beliefs, desires, and intentions. In a similar fashion, planning becomes more sophisticated since agents can conditionalize on the knowledge that is available, the current desires, and possibly many other features of the agent’s ‘state of mind’. Finally, the actions themselves can be actions in the real world, but they can also refer to the agent’s mental state: the agent can modify his beliefs, adapt his desires, and change his plans — either as a side effect of the ‘physical’ action, but even as a fully fledged mental actions, in which the agent just ‘changes his mind’. My favourite terminology to refer to the enriched sense-plan-act cycle is by the trio Knowledge, Rationality and Action. I mean these terms in a broad sense: each of them summarising and referring to a massive amount of research that has already been done, but as a triple ‘agentize’ them and setting the agenda for future research. ‘Knowledge’ is meant to capture the whole spectrum of processing information in an intelligible way, from gathering information, (reasoning, possibly resource bounded, about) knowledge, belief, uncertainty, and changing situations, as studied in the areas of belief revision and updates. ‘Rationality’ then refers to the decision making process in the agent. Here, notions like utilities, preferences, strategies, and solution concepts in games are classics on the research agenda, but the agent community has broadened this to include notions like satisficing (rather than optimal) behaviour, competitive versus cooperative behaviour, emergent and computational models of rational behaviour.
‘Action’ is of course closely related to decision making, it can be conceived of as its output. It is also responsible for a dynamic component in the whole analysis: actions change the world and the mental states of the agents. There is a rich literature on reasoning about change (or, for that matter, time) but studying it together with informational and rational notions is quite a new and exciting enterprise. This brings me to one of two other points I wish to make before we start our journey through a more detailed analysis of our tripartite: part of the real challenge in formalising them is in their combination. For instance, the interplay between uncertainty and actions, although having already a respectable history (in the seventies, R.C. Moore published a paper called Reasoning about Knowledge and Action ([1])), a stream of publications starting around the year 2000 on Dynamic Epistemic Logic (see Section 4) showed that there are many unresolved but important issues here: actions themselves can be purely epistemic, and the agent may be unsure not only about what the effect of an action is, but also which action precisely is being executed. Game theory demonstrates that the combination of knowledge and rationality is more than just adding the two notions together. The earliest encounters in game theory ([2]) did not refer to knowledge at all: this did not mean that the players (or the agents) did not have any knowledge, to the contrary, that work modelled games in which the players share common knowledge about each other’s rationality, everybody’s payoffs, the game structure and the history in the game. Thus, ironically, adding the notion of knowledge made it possible to reason about games in which players have incomplete information, i.e., players do not necessarily know ’where in the game they are’. This led game theorists and logicians to join forces in the 1990’s (see the TARK ([3]) and LOFT ([4]) conferences), and recent work on knowledge and games still further deepens the connection: for instance, winning conditions can be purely epistemic ([5]), and epistemic logic can be used to characterize memory in games ([6]). (As an aside, one can even give up the assumption that the agents know which game exactly they are involved in, ending up in games of imperfect information, an area that is becoming popular in agents’ research under the name of ‘evolutionary game theory’.) The second point I wish to make here is that studying Knowledge, Rationality and Action is especially intriguing in the context of Multi- Agent Systems. An intelligent agent then not only has to model the world and its own mental state, but also that of others. And this modeling of the other agent’s state might include a model of that agent modeling the modeler! This is exactly why reasoning in game theory and decision making is so complex and often prone to circular reasoning, like: ”obviously, my best decision here is to do d. But if this is so obvious, my competitor C will know
this also, so it might be better to opt for d ′ . But C can see this as well, so d might be better still, in the end”. And that it is hard to maintain all these models of other agents (including their models) in the context of knowledge, can immediately be affirmed by the few readers who have incidentally ever told a lie, which is much harder to manage, on an informational level, than to tell the truth! Studying action also is most interesting in a multi-agent context, since notions like alternation, communication and cooperation need to be analyzed and understood. In this respect the framework of ATL (Alternating- time Temporal Logic, see [7]) proposes a nice shift from a temporal framework CTL (for reasoning what happens over time) to a coalitional one, in which one reasons about what states of affairs groups of agents can guarantee to bring about, using their combined strategies, possibly with the use of an epistemic component ([8]). The rest of the paper is organised as follows. In Section 2 I present a standard approach to epistemic logic, highlighting its intuitive semantics. In Section 3 I then informally relate knowledge with rationality and action. Section 4 adds an action component to the epistemics, and in Section 5 I briefly give some remarks about the relative value of the approach. Before we start off then, I like to thank Rineke Verbrugge to kindly let me use our [9] as a basis for Section 2. Also, Hans van Ditmarsch and Barteld Kooi welcomed letting me found Section 4 on our joint [10]. Finally, Peter McBurney was so nice to read and comment on a first draft of this paper: Of course, all trivialities and errors remain mine.
2. Knowledge: the Static Case Epistemic logic is the logic of knowledge and ignorance: Is it harmful if, at a literature-exam you don’t know the contents of a chapter? No, as long as you know that the examiner does not know that you do not know it! In negotiations, it will harm you to let the other party know your ‘bottom-line’, but it may be helpful to disclose other information about yourself, for example about some of your values. The subject of epistemic logic started to flourish after Kripke’s invention of a possible worlds semantics for modal logic in the early sixties. The first full-length book about epistemic logic, J. Hintikka’s “Knowledge and Belief” [11], applies these semantical ideas to epistemic logic, although his definitions are not quite the same as the standard ones used today. Since [11] epistemic logic has been a subject of research in philosophy [11], computer science [12], artificial intelligence [13] and game theory [14]. The latter three application areas made it more and more apparent that in multiagent systems higher-order information, knowledge about other agents’ knowledge, is crucial.
We are now ready to define the formal language for knowledge of individual agents in a group of m agents. The atomic propositions in the language P typically denote facts about the world, or about a particular game (for instance, an atom can denote that player Alice holds the King of Hearts). Definition 2.1 Let P be a non-empty set of propositional variables, and m ∈ IN be given. The language L is the smallest superset of P such that
p, ¬q t• B s• p, q
We also assume to have the usual definitions for ∨, → and ↔ as logical connectives.
The intuition of formulas in the epistemic language L is best explained by looking at the semantics of Ki ϕ. Given a situation or world s (for the moment, think of it as a truthassignment to atoms), Ki ϕ is true in s if ϕ is true in all situations t that agent i cannot distinguish from s. For instance, let p denote ‘it is sunny in Liverpool’ and q ‘it is sunny in Amsterdam’. Then a priori there are four possible situations, and for agent B who is in Liverpool, the two situations s in which p ∧ q is true, and t in which (p ∧ ¬q) is, are indistinguishable. He is also unable, given his information in Liverpool, to distinguish the two possibilities u and v in which it rains in Liverpool (see Figure 1). We are now ready to define the semantics for our modal language L formally: a Kripke model M will be a tuple M = hS, π, R1 , . . . , Rm i where S is a non-empty set of worlds or states s, π gives, for every state s the truth-value π(s)(p) for every atom p, and each Ri for i ≤ m is a binary equivalence relation between worlds (since the relation ‘is indistinguishable from’ is an equivalence relation). In order to determine whether a formula ϕ ∈ L is true in s (if so, we write (M, s) |= ϕ), we look at the structure of ϕ: iff iff iff iff
A M
•v ¬p, q
A
¬p, ¬q • u′
[π] =⇒ s′ • p, q
A M′
′
•v ¬p, q
A1 A2 A3 A4 A5 R1 R2
axioms and rules for S5m any axiomatization for propositional logic (Ki ϕ ∧ Ki (ϕ → ψ)) → Ki ψ Ki ϕ → ϕ Ki ϕ → Ki Ki ϕ ¬Ki ϕ → Ki ¬Ki ϕ |= ϕ, ⊢ ϕ → ψ ⇒ ⊢ ψ |= ϕ ⇒ ⊢ Ki ϕ Table 1. Epistemic axioms and rules: i ≤ m
2.1. Kripke Semantics
M, s |= p M, s |= (ϕ1 ∧ ϕ2 ) M, s |= ¬ϕ M, s |= Ki ϕ
B
′
Figure 1. A simple two-agent Kripke Model
ϕ, ψ ∈ L ⇒ ¬ϕ, (ϕ ∧ ψ), Ki ϕ ∈ L (i ≤ m).
The intended meaning of Ki ϕ is ‘agent i knows ϕ’. Thus, for the simplest kind of epistemic logic, where the knowledge of individual agents in a group about the world and the other agents is modeled, it is sufficient to enrich the language of classical propositional logic by unary operators Ki . Here, an agent may be a human being, a player, a robot, a machine, or simply a ‘process’.
A
p, ¬q t •
¬p, ¬q •u
π(s)(p) = true M, s |= ϕ1 and M, s |= ϕ2 not M, s |= ϕ ∀s(Ri st ⇒ M, t |= ϕ)
Under such a definition, we say that Ki is the necessity operator for Ri . We also say that i considers t possible in s, if Ri st.
It is worthwhile to spend some words on the simple Kripke model M (comprising the states s, t, u and v) of Figure 1. First of all, since we assume that all epistemic accessibility relations are equivalences, we omit reflexive arrows in all pictures, from now on (hence, although no agent can distinguish state s from itself, we leave this implicit in the drawing). More interestingly, we can use that model to demonstrate a crucial feature of modal logic, called intensionality: note that we have M, s |= (p ↔ q) ∧ (KB p ∧ ¬KB q), saying that, although p and q are equivalent in state s, still there are formulas (i.e., KB ·) that are true for the one, but not for the other. A final point about this model is that it allows us to reason about a multi-agent system in a very natural way. Note that in s for instance, we have KB p ∧ ¬KB q ∧ ¬KB ¬q ∧ KA q, saying that although B knows p, he does not know q, nor ¬q, although A knows q. But we can without further ado immediately express higher order information: in s we have for instance that, although B does not know whether q, he does know that A is in Amsterdam and knows the weather condition there: KB (KA q ∨ KA ¬q). It is an interesting question to determine what the properties of any modal logic are, i.e., properties ϕ that are valid in every Kripke model. Choosing each Ri to be an equivalence relation, gives a system called S5m : its properties are summarized in Table 1. For example, A1 says that knowledge is closed under consequences and R1 expresses that agents know all validities. Table 1 reflects idealized notions of knowledge, that do not necessarily hold for human beings, or even compu-
tational systems. This is known as the problem of logical omniscience. For a discussion on weakening these properties we refer to [12, 13]. A4 and A5 are also known as positive and negative introspection, respectively. Although we can express one agent’s knowledge about that of another, the presented system gets a real multiagent flavour by introducing the following. First of all, let Eϕ ≡ (K1 ϕ ∧ · · · ∧ Km ϕ) denote that everybody knows ϕ. Although we have positive introspection (A4 above) for every individual’s knowledge, this does not immediately carry over to everybody’s knowledge: it is perfectly well possible that everybody in a department knows that a specific colleague is leaving, without everybody knowing that everybody knows it. Therefore, it makes sense to define the notion of common knowledge Cϕ, which intuitively captures the notion Eϕ ∧ EEϕ ∧ EEEϕ ∧ · · ·. Semantically, ϕ is common knowledge in state s, if, no matter where we end up in the model following arbitrary arcs from s on, we only visit worlds in which ϕ holds. In other words, we have that ϕ is not common knowledge in s, if somebody holds it for possible that somebody holds it for possible, that . . . ¬ϕ holds. Example 2.2 The example of the muddy children is certainly not new, but the treatment of its dynamics within the object language is novel and interesting (see Section 4). In this example the principal players are a father and k children, of whom m (with m ≤ k) have mud on their forehead. The father calls all the children together: None of them knows whether it is muddy or not, but they can all accurately perceive the other children and judge whether they are muddy. This all is common knowledge. Now father has a very simple announcement Φ to make: At least one of you is muddy. If you know that you are muddy, please come forward. After this, nothing happens (except in case m = 1). When the father notices this, he literally repeats the announcement Φ. Once again, nothing happens (except in case m = 2). The announcement and subsequent silence are repeated until the father’s m-th announcement: then, all m muddy children step forward! The semantical analysis of this situation is given in model U of Figure 2: worlds are denoted as triples xyz. The world 110 for instance denotes that child a and b are muddy, and c is not. Given the fact that every child sees the others but not itself, we can understand that agent a ‘owns the horizontal lines’ in the figure, since a can never distinguish between two states 0yz and 1yz. Similar arguments apply to agents b and c. The only propositional atoms we use are mi (i = a, b, c) with meaning ‘child i is muddy’. In state s = 110, we then have for instance ¬(Ka ma ∨Ka ¬ma ) (agent a does not know whether it is muddy), and also Ka (mb ∧ ¬Kb mb ) ∧ Ka (¬mc ∧ ¬Kc ¬mc ) (a knows that b is muddy without knowing it,
and also that c is mudless without knowing that). Regarding group notions, we observe the following, in s. Let ℓ denote that at least one child is muddy (ℓ ≡ ma ∨ mb ∨ mc ). 1. Eℓ ∧ ¬Ema ∧ ¬Emb ∧ ¬Emc ; everybody knows that there is at least one muddy child, but nobody is known by everybody to be muddy 2. Kc Eℓ ∧ ¬Kb Eℓ; c knows that everybody knows that there is at least one muddy child, but b does not know that everybody knows at least one child to be muddy. 3. ¬Cℓ; It is not common knowledge that there is at least one muddy child! This follows immediately from the previous item, but also directly from the model: one can find a path from s = 110 via 010 to 000, the latter state being one in which no child is muddy. We even have U, 111 |= ¬Cℓ.
c
a
011
c a
010
110 b
b U
b
b c
000
111
001 a
a c
101
100
Figure 2. The Muddy Children
3. Knowledge, Rationality and Action 3.1. Knowledge and Rationality As mentioned in the introduction, we use ‘rationality’ as it is used in economic theories: every agent has his own preferences, and acts in accordance to, and only to, them. In this section I like to show how a notion like common knowledge of rationality supports a technique used in game theory, i.e. backward induction, and thereby lays a theoretical foundation for the notion of Nash equilibrium (see also [14]). As the scene for this demonstration, I like to use the game in extensive form, depicted in the left hand side of Figure 3. It supposes we have two players, A and B, and A has to decide in the nodes labeled a, e, i and u, whereas B decides in b and c. The leaves of the tree are labeled with payoffs, the one in the left-most leave for instance denoting that A would receive 1, and B would get 6. A natural question now is to ask: “suppose you are agent A. What would your decision be in the top node a? A prominent procedure to determine A’s ‘best’ move is to reason starting from the leaves. It runs as follows. Suppose the
game would end up in node u. Since A is rational, he prefers an outcome of 4 over 1, and hence he would move ‘left’ in u; this is demonstrated using the thick lines in the game on the right hand side of Figure 3. Now, B is rational as well, and he moreover knows that A is rational, so, when reaching node c, player B knows he has in fact a choice between a payoff of 1 (going ‘left’ in c) and 3 (going right, and knowing what A will do in u). We do the same reasoning over nodes e and i, and end up with the choices with a thick arc in the figure: A would go ‘right’ and ‘left’, respectively. Again, since B is rational and knows that A is rational, his payoffs in node b are 4 and 3 respectively, and he will choose ‘left’. Now A is in a position to decide in node a, being aware that he would receive 2 when going left, and 1 when choosing right here. Note that his choice for going left is based on (i) the fact that he is rational; (ii) the fact that he knows that B is rational and (iii) the fact that A knows that B knows that A is rational! a
J
J
J
J
a
J
J
J
J b B B
B e i E E E E E E 1 2 7 6 6 4 3 5
1 4
c B B
B u E
4 3
E E 1 5
b B B B e i E E E E E E 1 2 7 6 6 4 3 5
c B B B u 1 E 4 E E 4 1 3 5
Figure 3. A Game in Extensive Form
3.2. Knowledge and Action Knowledge and action interfere with each other in many ways: for some actions, in order to be able to do them properly, some knowledge is required, and, on the other hand, actions may add to an agent’s knowledge. A major ongoing research problem in AI planning is that of correctly formulating knowledge preconditions for actions and plans: knowledge that is necessary to successfully carry out the plan (like knowing the combination of a safe to open it). Many formalisms have been developed for this problem — see [15] for a survey. Actions that add to the agent’s knowledge may be some kind of scientific experiments in the real world, but they may also be purely ‘epistemic’ (like reading the combination of the safe in a message). Here, we also have strategic reasoning about knowledge, since it is often
not only crucial that some agents obtain certain knowledge, but also that third parties remain ignorant about it! The formal approaches to reasoning about knowledge and action (or, for that matter, change) differ quite a lot in the expressiveness of the object language. I briefly mention a few approaches, without the intention of being complete, here. Belief revision ([16]) studies dynamic and epistemic attitudes on a meta level: beliefs are represented as sets of propositional formulas K, and the change is analyzed via postulates in the meta language, like ϕ ∈ K + ϕ, saying that the agent should always incorporate anything that he accepted to incorporate. The temporal approaches usually mix a temporal language with an epistemic one, allowing to express properties like perfect recall: Ki ϕ → Ki ϕ ( is a ‘next’ operator, referring to the next situation in a path). This approach is particularly stimulated by the interpreted systems paradigm, see [12] for further details. Currently, there is a lot of interest in social formalisms to reason about change, the question not being when something will happen, but rather who can bring it about ([17]). In ATEL, which is ATL with an epistemic component, one can for example express that two agents need to know a shared key to open a safe, while being both unable to open it on their own (Ki keyi ∧K2 key2 → (hh{i, j}ii♦o∧¬hhiii ♦o∧ ¬hhjii♦o) and that they also have a way to ensure that a third party will never come to know one of the keys: hhi, jii ¬(Kh keyi ∨Kh keyj ). The fact that currently a proper semantics for ATEL is under debate ([18]), underlines it expressivity and usefulness, rather than undermining it. There are many agent logics that try to combine several agent attitudes with a dynamic component, the most prominent being represented in the BDI stream ([19]), which fits well with the temporal approaches mentioned above. Few of these logics however, incorporate dynamic logic as a component ([20]), in which one can express how certain states of the world can be brought about. A ‘dynamized’ version of perfect recall would then read Ki [α]ϕ → [α]Ki ϕ, expressing that if i knows that action α will bring about ϕ, he will indeed remember that ϕ is the case having done α. Although the work in this area still seems in essence to be pretty much single agent oriented, in fact Dynamic Epistemic Logic (DEL) can be seen as a specific way to fill this gap. In DEL, a variant of perfect recall would read Ki [α]ϕ → [Ki α]Ki ϕ: in order to ensure that the agent indeed knows ϕ, the action α has not just to happen: the agent must know that it happens!
4. Dynamic Epistemic Logic The famous paper [16] by Alchourr´on et al. put the change of information, or belief revision, as a topic on the philosophical and logical agenda: it was followed by a large stream of publications and much research in belief revision,
fine-tuning the notion of epistemic entrenchment, revising (finite) belief bases, differences between belief revision and belief updates, and the problem of iterated belief change. However, in all these approaches the dynamics is studied on a level above the informational level, making it impossible to reason about change of agents’ knowledge and ignorance within the framework, let alone about the change of other agents’ information. Dynamic Epistemic Logic ( DEL) tries to overcome this. I use DEL for approaches that not only dynamize the epistemics, but also epistemize the dynamics: the actions that (groups of) agents perform are epistemic actions. Different agents may have different information about which action is taking place, including higher-order information. This rather recent tradition treats all of knowledge, higherorder knowledge, and its dynamics on the same footing. Following an original contribution by Plaza in 1989 [21], a stream of publications appeared around the year 2000 and the city of Amsterdam: the reader is referred to [10] for references.
4.1. Belief Revision and DEL In this section I try to explain the basic semantic issues in DEL and at the same time to demonstrate what are the technical problems compared with belief revision. Referring back to Section 2, an agent knows ϕ at state s if ϕ is true in all the situations t that the agent cannot (epistemically) distinguish from s. Hence, the more of such states t there are, the less the agent knows, and, at the other extreme, if the agent manages to rule out all alternatives different from s, he has perfect knowledge. This leads to a first naive approach to DEL, as depicted in Figure 4.
s p, q •
N
t • ¬p, q
[L?p] =⇒
s′ N ′ p, q •
Figure 4. A Simple Learning Action
In the model at the left-hand side of Figure 4 (in situation s, say), the agent does not now p. Now suppose he learns p, which we denote with [L?p]. This should result in a new model, in which the agent knows that the alternative t can now be ruled out as a possible real world, resulting in the Kripke model with only the state s′ . Learning new information (which is compatible with what the agent already knew) is captured in belief revision using an expansion operator, where K+ϕ denotes the belief base K expanded with ϕ. Now, one postulate about this operator reads ψ ∈ K ⇒ ψ ∈ K + ϕ: we should not lose existing information when incorporating new information. The belief set of interest in our example is Ks = {ψ | N, s |= Kψ}. Then, the result of
expanding Ks would be Ks +p = Ks′ = {ψ | N ′ , s′ |= Kψ}. However, for the full epistemic language, the postulate we just gave is not valid anymore: note that ¬Kp is a member of Ks , but should not be in Ks′ : an agent that is aware of his ignorance (see A5 in Table 1) about p, should give up his ignorance when he indeed learns that p! A property that is derivable from the postulates on expansion, is that one can always painlessly accept information that is consistent with the belief base: if K ∪ ϕ is consistent, then so is K + ϕ. But even this weak property is not true for DEL, witnessed by ϕ = (p ∧ ¬Kp): although the claim ‘p is true but not known’ makes perfect sense (as long as it is uttered by an outsider), it is impossible to update one’s knowledge with it! (More precisely, any agent i with the properties of knowledge described in Section 2, can never know that (p ∧ ¬Ki ), let alone update with it.) As a final example of the intricacies of multi-agent updating, let us use the idea of ‘learning is removing alternatives’ and apply it to the model of Figure 1. Suppose, that agent B (who is in Liverpool) finds out about the weather in Amsterdam: if q, he learns q, if not, he learns ¬q. Semantically, one could remove the links labeled ‘B’ and obtain M ′ as in Figure 1, but in the resulting model A learnt that B knows: everywhere in the model we have KA (KB q ∨ KB ¬q), which is absurd, because A was not involved in the learning of p by A in the first place!
4.2. Public Announcements The epistemic program π that transforms N into N ′ in Figure 1 is in fact a public announcement, it corresponds with the situation that agents A and B talk to each other, B telling A: ‘I know what the weather in Amsterdam is’. Letting ψ be (KB q ∨ KB ¬q), the program would result in situations in which KB ψ, KA ψ, KB KA ψ, KA KB ψ, KB KA KB ψ, . . .. Indeed, in N ′ , it is common knowledge that ψ! Public announcements are those epistemic actions about which it is common knowledge that they take place. The semantics of such announcements is relatively simple. Let [φ]ψ mean that after public announcement with φ, the formula ψ is true. Then M, s |= [φ]ψ if M, s |= φ ⇒ M|φ, s |= ψ, where M|φ, s is simply the restriction of the model M to all those states where φ is true. A warning is in place that announcements don’t have to refer to objective facts only: it is perfectly possible to model the following scenario: Agent C does not know whether a Agent A does not know whether a Agent A asks C: ”do you know whether a”? C answers truthfully: ”yes, a!” (Imagine C being a program committee member, of an event that notified all rejected papers last week, A is an au-
thor of a submission, which may have been accepted (a) or not. His question is in fact an announcement ”¬KA a”. Suppose we initially have KC (¬KA a ↔ a) (the pc member knows that authors of accepted papers are exactly those that don’t know their status yet), then indeed we get ¬KC a∧ [¬KA a]KC a!) Example 4.1 Recall our semantics of announcements [φ]ψ: first the model is updated with φ, and in the resulting model, we verify ψ. In state U, 110 of Example 2.2, we can represent the announcement Φ as onemuddy := m1 ∨ m2 ∨ m3 . Model U1 = U|Φ is obtained from U by removing all states in which onemuddy is false, i.e., 000 (see Figure 5). Thus, we have (M, 110) |= [onemuddy]Conemuddy, since Conemuddy holds in U1 , s. Now suppose the real situation in U1 would be 100. Then a knows there is a muddy child, and, since it does not see any, it knows it is muddy itself, and would step forward. However, father repeats ‘Φ’, and hence 100 cannot have been the actual situation. Thus, we can represent Φ when given in U1 as standstill := ¬(K1 m1 ∨K1 ¬m1 )∧¬(K2 m2 ∨K2 ¬m2 )∧¬(K3 m3 ∨K3 ¬m3 ). The model U2 = U1 |standstill is obtained by removing those worlds in which standstill is false (that is, those worlds with only one occurrence of 1 in their representation), giving the model U2 . Summarising, we have: (M, 110) |= [onemuddy]Conemuddy (M, 100) |= [onemuddy]K1 m1 (M, 110) |= [onemuddy]standstill (M|onemuddy), 110 |= [standstill]¬standstill (M, 110 |= [onemuddy][standstill]¬standstill
c
a
011
c a
010
111
110
a
011
c
111
110
b
b
b
101
101
b 001
a c 100 U1
U2
Figure 5. Muddy Children Dynamized
An interesting phenomenon in the previous example is [standstill]¬standstill, representing an unsuccessful update of the form [φ]¬φ; by announcing φ, it becomes false (father saying ‘you still don’t know whether you are muddy’ makes the children a and b realise that they in fact are).
4.3. Lecture or Amsterdam The following, possibly simplest example in the setting of multiagent systems (two agents, one atom) attempts to demonstrate that the notions of higher-order information and epistemic actions are indeed important and may be subtle. In particular, some actions may increase the uncertainty of the agents, so that, semantically speaking, adding worlds becomes the issue, rather than removing them. Anne and Bert are in a bar, sitting at a table. A messenger delivers a letter that is addressed to Anne. The letter contains either an invitation for a night out in Amsterdam, or an obligation to give a lecture instead. Anne and Bert commonly know that these are the alternatives. This situation can be modelled as follows: There is one atom q, describing ‘the letter invites Anne for a night out in Amsterdam’, so that ¬q stands for her lecture obligation. Whatever happens in each of the following action scenarios, is publicly known. Also, assume that in fact q is true. The initial situation is modelled in model L of Figure 6: black dots denote q worlds, at open dots q is false. In the first scenario (tell) Anne reads the letter aloud. This is in fact a public announcement, the resulting model consisting of one world, in which q is true, giving (in L, s) that [tell]Cq. In another scenario, Bert sees that Anne reads the letter. The resulting model is (like) the model in the right hand side of Figure 1 (ignore the atom p). Now Cq does not hold, but still, Bert learns something: L, s |= [read]KB (KA q ∨ KA ¬q). An interesting action is mayread: Ann orders a drink at the bar so that Bert may have read the letter. The resulting model is L1 in Figure 6. Note that A is not sure whether B knows q: in s1 she admits the possibility that q∧KB q but also that q ∧ ¬KB q. Finally, we get model L2 if the action bothmayread is executed in L, s: Bert orders a drink at the bar while Anne goes to the bathroom. Both may have read the letter. Note that both agents, no matter what the value of the atom is, reckon the possibility that the other agent knows it, or is ignorant about it — and that again is common knowledge! For more on this kind of updates, I refer to the references in [10].
5. Conclusion I have advocated formal approaches to frameworks that model knowledge, rationality and action, as a theory for multi-agent systems. Admittedly, ’classical’ approaches to such systems have been under debate for a long time. Brooks ([22]) argues in favour of emergent rather than rationalized behaviour, Gigerenzer ([23]) underlines social and heuristic rather than logical intelligence, and in fact the
s q•
B L
◦ ¬q
A
•
•
◦ L1
s1 •
◦
L2 s2 •
•
•
◦
◦
◦
◦
Figure 6. Reading Letters area of evolutionary game theory ([24]) is successful in areas where classical game theorists’ assumptions are just too simple, which is also observed by many researchers in the area of cognitive systems. However, I think that understanding idealized behaviour, under circumstances that may be necessarily idealistic, is a good starting point for understanding complex systems that behave under much less understood constraints. Moreover, the formalisms and languages that we came across are extremely helpful and powerful in reasoning about and understanding complex multi-agent systems. Finally, I am an optimist and a believer, and I hope the few initiatives ([25]) combining the sub-symbolic with the logical, in the end to be successful.
References [1] R. C. Moore, “Reasoning about knowledge and action,” in Proceedings of the Fifth International Joint Conference on Artificial Intelligence (IJCAI-77), Cambridge, MA, 1977. [2] J. V. Neumann and O. Morgenstern, Theory of Games and Economic Behaviour. Princeton University Press: Princeton, NJ, 1944. [3] “Theoretical aspects of reasoning about knowledge (TARK),” http://www.tark.org. [4] “Logic and the foundations of game and decision theory (LOFT),” http://www.econ.ucdavis.edu/faculty/bonanno/loft.html. [5] H. v. Ditmarsch, “Knowledge games,” Bulletin of Economic Research, vol. 53, no. 4, pp. 249–273, 2001. [6] G. Bonanno, “A characterization of von neumann games in terms of memory,” Knowledge, Rationality and Action, pp. 281–296, 2004, KRA is a special section of Synthese. [7] R. Alur, T. A. Henzinger, and O. Kupferman, “Alternatingtime temporal logic,” Journal of the ACM, vol. 49, no. 5, pp. 672–713, Sept. 2002. [8] W. van der Hoek and M. Wooldridge, “Cooperation, knowledge, and time: Alternating-time temporal epistemic logic and its applications,” Studia Logica, vol. 75, no. 1, pp. 125– 157, 2003.
[9] W. van der Hoek and R. Verbrugge, “Epistemic logic: a survey,” in Game Theory and Applications, L. Petrosjan and V. Mazalov, Eds. New York: Nova Science Publishers, 2002, vol. 8, pp. 53–94, iSBN 1-59033-373-. [10] H. van Ditmarsch, W. van der Hoek, and B. Kooi, “Concurrent dynamic epistemic logic,” in Knowledge Contributors, K. J. V.F. Hendricks and S. Pederson, Eds. Kluwer Academic Pusblishers, 2003, pp. 105–143. [11] J. Hintikka, Knowledge and Belief. Cornell University Press, 1962. [12] R. Fagin, J. Y. Halpern, Y. Moses, and M. Y. Vardi, Reasoning About Knowledge. The MIT Press: Cambridge, MA, 1995. [13] J.-J. C. Meyer and W. van der Hoek, Epistemic Logic for AI and Computer Science. Cambridge University Press, 1995. [14] R. Aumann and A. Brandeburger, “Epistemic conditions for Nash equilibrium,” Econometrica, no. 63, pp. 1161–1180, 1995. [15] M. Wooldridge and N. R. Jennings, “Intelligent agents: Theory and practice,” The Knowledge Engineering Review, vol. 10, no. 2, pp. 115–152, 1995. [16] C. Alchourr´on, P. G¨ardenfors, and D. Makinson, “On the logic of theory change: partial meet contraction and revision functions,” Journal of Symbolic Logic, vol. 50, pp. 510–530, 1985. [17] M. Pauly, “A modal logic for coalitional power in games,” Journal of Logic and Computation, vol. 12, no. 1, pp. 149– 166, 2002. [18] W. Jamroga and W. van der Hoek, “Agents that know how to play,” 2004, accepted for Fundamenta Informaticae. [19] A. S. Rao and M. P. Georgeff, “Modeling rational agents within a BDI-architecture,” in Proceedings of Knowledge Representation and Reasoning (KR&R-91), R. Fikes and E. Sandewall, Eds. Morgan Kaufmann Publishers: San Mateo, CA, Apr. 1991, pp. 473–484. [20] W. van der Hoek, B. van Linder, and J.-J. C. Meyer, “An integrated modal approach to rational agents,” in Foundations of Rational Agency, M. Wooldridge and A. Rao, Eds. Kluwer Academic Publishers: Dordrecht, The Netherlands, 1999, pp. 133–168. [21] J. Plaza, “Logics of public communications,” in Proceedings of the 4th International Symposium on Methodologies for Intelligent Systems, M. Emrich, M. Pfeifer, M. Hadzikadic, and Z. Ras, Eds., 1989, pp. 201–216. [22] R. Brooks, Cambrian Intelligence: The Early History of the New AI. MIT Press, 1999. [23] G. Gigerenzer, Adaptive thinking: Rationality in the real world. Oxford University Press, 2000. [24] H. Gintis, Game Theory Evolving. Princeton University Press, 2000. [25] R. Blutner, “Nonmonotonic inferences and neural networks,” to appear in Knowledge, Rationality and Action.
