Annotated Slides - University of Washington

3 downloads 105 Views 583KB Size Report
May 29, 2013 ... Representation. Machine Learning – CSE446. Carlos Guestrin ... Proposition: P statisfies (X ⊥ Y) if and only if. □ P(X,Y) = P(X) P(Y). ©Carlos ...
Bayesian Networks – Representation Machine Learning – CSE446 Carlos Guestrin University of Washington May 29, 2013

©Carlos Guestrin 2005-2013

1

Handwriting recognition

Character recognition, e.g., kernel SVMs rr r r r c ac r z bc

©Carlos Guestrin 2005-2013

2

1

Webpage classification Company home page vs Personal home page vs University home page vs …

©Carlos Guestrin 2005-2013

3

Handwriting recognition 2

©Carlos Guestrin 2005-2013

4

2

Webpage classification 2

©Carlos Guestrin 2005-2013

5

Today – Bayesian networks n  n  n  n 

One of the most exciting advancements in statistical AI in the last decades Generalizes naïve Bayes and logistic regression classifiers Compact representation for exponentially-large probability distributions Exploit conditional independencies

©Carlos Guestrin 2005-2013

6

3

Causal structure n 

Suppose we know the following: ¨  ¨  ¨  ¨ 

n 

The flu causes sinus inflammation Allergies cause sinus inflammation Sinus inflammation causes a runny nose Sinus inflammation causes headaches

How are these connected?

7

©Carlos Guestrin 2005-2013

Possible queries Flu

Inference

n 

Most probable explanation

n 

Active data collection

Allergy

Sinus

Headache

n 

Nose

©Carlos Guestrin 2005-2013

8

4

Car starts BN n 

18 binary attributes

n 

Inference ¨ 

n  n 

P(BatteryAge|Starts=f)

216 terms, why so fast? Not impressed? ¨ 

HailFinder BN – more than 354 = 58149737003040059690390169 terms

©Carlos Guestrin 2005-2013

9

Factored joint distribution Preview Flu

Allergy

Sinus

Headache

Nose

©Carlos Guestrin 2005-2013

10

5

What about probabilities? Conditional probability tables (CPTs)

Flu

Allergy

Sinus

Nose

Headache

©Carlos Guestrin 2005-2013

11

Number of parameters Flu

Allergy

Sinus

Headache

Nose

©Carlos Guestrin 2005-2013

12

6

Key: Independence assumptions Flu

Allergy

Sinus

Headache

Nose

Knowing sinus separates the variables from each other 13

©Carlos Guestrin 2005-2013

(Marginal) Independence n 

Flu and Allergy are (marginally) independent Flu = t Flu = f Allergy = t Allergy = f Flu = t

Flu = f

Allergy = t Allergy = f ©Carlos Guestrin 2005-2013

14

7

Marginally independent random variables n  n 

Sets of variables X, Y X is independent of Y if ¨  P

n 

Ⱶ (X=x⊥Y=y), ∀x∈Val(X), y∈Val(Y)

Shorthand: ¨  Marginal

n 

independence: P Ⱶ (X ⊥ Y)

Proposition: P statisfies (X ⊥ Y) if and only if ¨  P(X,Y)

= P(X) P(Y)

©Carlos Guestrin 2005-2013

15

Conditional independence n 

Flu and Headache are not (marginally) independent

n 

Flu and Headache are independent given Sinus infection

n 

More Generally:

©Carlos Guestrin 2005-2013

16

8

Conditionally independent random variables n  n 

Sets of variables X, Y, Z X is independent of Y given Z if ¨  P

n 

Ⱶ (X=x⊥Y=y|Z=z), ∀x∈Val(X), y∈Val(Y), z∈Val(Z)

Shorthand: ¨  Conditional

independence: P Ⱶ (X ⊥ Y | Z) ¨  For P Ⱶ (X ⊥ Y |∅), write P Ⱶ (X ⊥ Y) n 

Proposition: P statisfies (X ⊥ Y | Z) if and only if ¨  P(X,Y|Z)

= P(X|Z) P(Y|Z) ©Carlos Guestrin 2005-2013

17

The independence assumption Flu

Allergy Sinus

Headache

Nose

Local Markov Assumption: A variable X is independent of its non-descendants given its parents

©Carlos Guestrin 2005-2013

18

9

Local Markov Assumption: A variable X is independent of its non-descendants given its parents

Explaining away Flu

Allergy

Sinus

Headache

Nose

©Carlos Guestrin 2005-2013

19

Naïve Bayes revisited Local Markov Assumption: A variable X is independent of its non-descendants given its parents

©Carlos Guestrin 2005-2013

20

10