Pattern Recognition

Pattern Recognition Prof. Christian Bauckhage

outline additional material for lecture 10

inference with Markov chains

summary

inference with Markov chains

in lecture 10, we discussed maximum likelihood estimation of the (transition) parameters Pij of a Markov chain over a finite set of states a student then asked the very valid question where or how Markov models are used in pattern recognition to provide an answer to this question, we recall a didactic example from our lectures on game AI that is, we show how Markov models allow us to classify sequences of symbols

game AI example

consider a simple game map consisting of 3 rooms assume the rooms form the state space  S = A, B, C for discrete Markov chains

A

B treasure

C

game AI example (cont.)

also assume you observe a game agent (player or NPC) moving from room to room in particular, you observe the following sequence of moves   O = AABBBBCBBA the obvious question is: what is the agent doing? what kind of behavior would correspond to these movements?

A

B treasure

C

game AI example (cont.)

experience tells us that –on this part of the map– opponent players typically show two behaviors: “patrolling the area” or “guarding the treasure in room B” for both behaviors, we happen to have a Markov model, namely 0.3

0.3

0.8

0.25

0.2

A

0.1

B

0.5

0.5

A 0.25

0.5

0.2

0.2

C

B

0.5

0.1

0.25

0.5

C

0.3

0.25

λ1 ⇔ patrolling

λ2 ⇔ guarding

game AI example (cont.)

under these conditions, the question of what the agent is doing can be answered by evaluation the two probabilities

p O λ1 and p O λ2

game AI example (cont.)

for λ1 we have


p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A

· p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025

game AI example (cont.)

for λ1 we have


p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A

· p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025 for λ2 we have
p AABBBBCBBA = 1 · 0.25 · 0.5 · 0.8 · 0.8 · 0.8 · 0.1 · 0.5 · 0.8 · 0.1 ≈ 0.00026

game AI example (cont.)

for λ1 we have


p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A

· p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025 for λ2 we have
p AABBBBCBBA = 1 · 0.25 · 0.5 · 0.8 · 0.8 · 0.8 · 0.1 · 0.5 · 0.8 · 0.1 ≈ 0.00026 ⇒ according to the two models we have available, it seems as if the opponent is guarding the treasure in room B

note


above we tacitly assumed p X1 = A = 1 as we do not have any information as to what is the most likely symbol for a sequence to begin with, we should acknowledge our ignorance and assume


p X1 = A = p X1 = B = p X1 = C
of course we could argue that p X1 = A = 1/3 because there are three rooms but whether or not the first factor in the two products on the 1 previous slide not matter; in either case we
is /3 or 1 does
find p O λ1 < p O λ2

note

if the given sequence of observations is really long, it is numerically much safer to evaluate
log p O λi instead of
p O λi

note

we can also use Markov models to anticipate the future for instance, now that we “know” that the opponent is guarding the treasure, we might ask: if the opponent is currently in room B, how likely will he stay there for the next τ time steps? to answer this question, we compute
p Xt = B, Xt+1 = B, . . . , Xt+τ−1 = B, Xt+τ 6= B

= p Xt+1 = B | Xt = B · · · p Xt+τ 6= B | Xt+τ−1 = B = pτ−1 BB · (1 − pBB )

we now know about

how to use (a set of) Markov models to classify sequences of symbols or histories of (discrete) observations