Planning as ModelChecking

Planning as Model Checking Piergiorgio Bertoli Alessandro Cimatti Marco Pistore Marco Roveri Paolo Traverso

Automated Reasoning Systems (SRA) Division ITC-IRST - Institute for Scientific and Technological Research 38050 Povo - Trento, Italy

The Planning Problem Domain = States (block positions) + Actions (moves)

Initial State (“Red on Table”, “Blue on Table”, “Green on Blue”)

Goal State (“Blue on Green”, “Green on Red”, “Red on Table”)

Plan (“Move Green on Red”, then “Move Blue on Green”)

Planning Problem: Given a domain (states and actions), an initial and goal state, the planning problem is the problem to find a plan of actions that leads from the initial state to the goal

   

A Hard Problem!

5 locations, 3 piles, 3 trucks, 100 containers




states!

Automatic Planners

Domain

Observations

Initial

Planner Plan

Goal

Controller

System Actions

The Planner automatically generates the plan of actions

The Controller observes the current state and executes actions

Initial

Goal

Controller

Plan

Planner

The “Classical” Planning Problem

Domain

Observations

System Actions





Goal: a set of final desired states

Full Observability: complete knowledge about the state of the system

Deterministic Domain: actions from one state lead to a single state

Restrictive assumption: no uncertainty, complete knowledge



?

Non-deterministic domains

(before moving)

(after moving)

Partial Observability

(top view)

? (side view)

Extended Goals

For instance: “Keep the system infinitely often through a given state”

MANUAL

3 A

4

SDB

PROCESS i

SCHEDULER

SDB

Safety Logic

SDB

A realistic example (I)

OPERATOR

COMMANDS

PROCESS 1

...

5

SDB 6

SDB 7

PERIPHERAL STATUS PERIPHERAL CONTROLS

PROCESS n

...

SDB

B

8

SDB

PHERIPHERAL DEVICES

9

Dir. Gas

Controller

A realistic example (II)

Flussostato Bassa Pressione

Termica Comandi

Compressore Compressore Ventole

Temperatura

Controllo Ventole

Condensatore

Alta pressione

Controllo Valvola Flussostato

E v a p o r

Temperatura termoregolazione

Bassa Pressione

Pressione tasti Controllo led

Condensatore

Interfaccia Utente

Dir. Gas

ON/OFF ALARM/RESET ALTA PRESSIONE FN ON/OFF/RESET REQUEST OPERATION TERMICA

State of the art

State of the art w.r.t. planning in non-deterministic domains, under partial observability, and for extended goals. 1. Classical Planning: no applicable 2. (Limited) extensions to classical planning: no practical

3. Expressive frameworks (e.g., deductive): no automatic planning

Practical automated planning with non-determinism, partial observability and extended goals is still an open problem!

Structure of the talk

Planning as Model Checking in Non-Deterministic Domains:

1. Planning via Symbolic Model Checking (Alessandro Cimatti) 2. Planning under Null Observability (Marco Roveri) 3. Planning under Partial Observability (Piergiorgio Bertoli) 4. Planning for Extended Goals (Marco Pistore) 5. Conclusions and ... Dreams

Structure of the talk

Planning as Model Checking in Non-Deterministic Domains:

1. Planning via Symbolic Model Checking (Alessandro Cimatti) 2. Planning under Null Observability (Marco Roveri) 3. Planning under Partial Observability (Piergiorgio Bertoli) 4. Planning for Extended Goals (Marco Pistore) 5. Conclusions and ... Dreams

Model Checking



model checking algorithms: exhaustive traversal of the model

satisfies certain properties, represented as temporal logic formulae

a system, represented as Finite State Machine, (FSM)

Model Checking is a formal verification technique if



q

temporal formula G(p -> Fq)

p finite-state model

Remark: high industrial applicability!

Model Checker

q

p

counterexample

no!

yes!

A Model Checker is a software tool for system verification:



Planning as Model Checking







Plan as counterexample traces;

Planning as exploration of the FSM

Observations as Machine Outputs

Action effects as transitions

Actions as inputs to the FSM

Domain represented as Finite State Machine

locked & not loaded ->

locked loaded

eventually loaded & locked

locked not loaded finite-state transition system

counterexample

loaded

locked

no!

yes!

Reducing a Planning problem to a Model Checking problem:



temporal formula



Model Checker



Symbolic Model Checking Key issue: extremely large state spaces! Symbolic Model Checking:

Represent sets of states of the FSM symbolically, by means of boolean formulas;

Boolean formulas as Ordered Binary Decision Diagrams (BDDs)

1

0

1 0

Y

0

X

0

1

1

0

0

X

1

1



Y

1

1

Y

0 1

A BDD represents the assignments satisfying (and falsifying) a boolean formula

0

1





Operations over sets of states (e.g. union, intersection) as boolean operations (e.g. conjunction, disjunction) implemented as transformations over BDDs

X

0

0











   

Symbolic State-space Exploration

set_x flip_x

5

3

reset

flip_y set_y

1

4

X

1

Y

0 1

0

0

X

0

1

Y

1

1

The state-space is explored by iterating preimages (i.e. states from which a certain set is reachable)

1

2

Termination when fix point is reached

set_y flip_y

0

0

Implemented as transformations over BDDs

1

Y

1

0

0



 

The NuSMV Symbolic Model Checker NuSMV: A symbolic model checker developed at IRST A joint IRST-CMU development; Over 150 downoloads since july 1999: – basis for several ph.d. theses; – used for teachnig courses in several universities;

Used (by IRST) in technology transfer projects in design of industrial controllers;

Soon to be released for OpenSource development under Mozilla Public License: – free availability for commercial usage;

– copyleft: improvements made available for distribution.

NuSMV is available at http://sra.itc.it/tools/nusmv/

Planning via Symbolic Model Checking The Planning via Symbolic Model Checking paradigm: Inherits the basic technology of symbolic model checking; Extends symbolic model checking: Planning is a harder problem than model checking!

functionalities: conditional planning under full observability;

based on the NuSMV model checker;

MBP: a Model Based Planner



  

planning for temporally extended goals.

planning under partial observability;

planning under null observability (conformant planning);

MBP is available at http://sra.itc.it/tools/mbp/



 

  

Structure of the talk

Planning as Model Checking in Non-Deterministic Domains:

1. Planinng via Symbolic Model Checking (Alessandro Cimatti) 2. Planning under Null Observability (Marco Roveri) 3. Planning under Partial Observability (Piergiorgio Bertoli) 4. Planning for Extended Goals (Marco Pistore) 5. Conclusions and ... Dreams

B

table C

door

out

Actions: G O pos , P ICK U P, L ET G O G O O UT requires key at D OOR selects randomly an object among the possible ones.

box

Domain:

Robot position unknown. Robot hand empty. Either one of objects A or B has a key. objects initially located at DOOR are known to be red. object located at OUT.

PICKUP

Initially:

RED



Goal:



A simple example: The Blind Robot Problem (Michie’74)

A

   

   

       

Box

F

F

F

F

able F

Door

Out

Door

able F

Door Door Door Door

Door

None None None None

None

Table None

Box

Table None

Box

Table

Table

A

B

Box

Box

Box

F

F

Box

PickUp Box

F

Box

F

T

Door

Box

Box

Door Table None

Table None

Box

T

Door

Box

Box

Table

Door Table

B

A

PickUp Box

F

Door

T

Box

Door Table

A

B

Door

Door

T

T

Door

Door

Door Table

Door Table

B

A

Out

GotoTable PickUp GoOut

Door

C

Door Table None

LetGo

Table

Door Table None

Door

Door

Box

Box

Box

T

Door

T

Door

LetGo Door

F

Table None

Door

GotoBox

BDD.

Out

Door

F

GotoDoor

Box

T

Door

GotoDoor

Box

Box

Uncertainty in the initial condition. Non-deterministic action effects. Sets of indistinguishable states.

Conformant Plan for the Blind Robot Problem Key Problems:

Box

PickUp GotoTable LetGo

Box

GotoDoor

Box

Box

Box

Box

Belief States: Box Box Box Box

Box

Box GotoBox

Represent a Belief State as a

ROBOT P OS K EYAT D OOR O BJAP OS O BJ BP OS O BJ CP OS I N H AND

Our Approach:

Conformant Planning: Algorithms Intuitions

Bs1 Bs2 Bs3

γ

δ

α

G

are applicable in

δ β δ

I

β γ α β δ

γ

δ

α

β

Bs−3 Bs−2 Bs−1

δ

Role of Symbolic Model Checking Techniques:

β

γ

stored in a hash table.

symbolic expansion of belief states (extends symbolic expansion primitives).

BDDs,

G

Backward Search

and result in Bs1, Bs2, Bs3.

Basic search step: expansion of a Belief State Forward: , , α β γ



Search style: breadth-first, depth-first, best-first, A*, Termination: Bs , or search space exhausted.

I

Forward Search



visited belief states represented as







 



Conformant Planning: Results Two classes of algorithms:

Algorithms allow for different selection functions thus exploiting different search strategies (e.g. depth-first, breadth-first, best-first, A*, ).

a solution for problems admitting a conformant solution. with a failure otherwise.







Fully-symbolic search: additional BDD variables to encode the plan associated to a belief states. Heuristic-symbolic search: the plan is stored in the hash table of visited belief states. The algorithms are guaranteed to terminate with  

 

 

Experimental results show that the algorithms are effective in practice.

When the selection function is admissible (A* style search) solutions found are optimal (i.e. of minimal length).







    

CPU Search time (sec)

2

!

CPU Search time (sec)

2

!

4

4

BTUC(n)

8 10 # of packages CUBE (n) CENTER

8 10 side length

12

12

14

14

16

BMTC(n,6) Low Uncertainty

BMTC(n,6) High Uncertainty

1000

100

1000 100

10

1000 100

10

10

10

9

1

8

1

5 6 7 # of packages

1

4

0.1

3

0.1

2

0.1

10

0.01

9

0.01

8

0.01

5 6 7 # of packages

1000

0.001

4

URING(r)

100

1000

0.001 3

RING(r)

100

1000 100

10

10

9

10

8

10

6 7 # of rooms

1

5

1

4

1

3

0.1

2

0.1

10

0.1

9

0.01

8

0.01

6 7 # of rooms

0.01

5

0.001 4

0.001 3

0.001 2

0.001 2

Experimental Evaluation: Planning Domains

6

6

MBP fully-symbolic search. MBP heuristic-symbolic search. heuristic, non-symbolic planner (best competitor). GPT

From Planning to Circuit Analysis

The reset sequence problem: find a sequence of inputs that takes a system into a known state from any initial state. e.g. the Pentium processor at power-up. Goal: the set of “certain” (singleton) belief states.

Our approach: forward algorithms with modified termination tests.

Results: our technique compares positively with state-of-the-art (BDD based) specialized algorithms.

Structure of the talk

Planning as Model Checking in Non-Deterministic Domains:

1. Planning via Symbolic Model Checking (Alessandro Cimatti) 2. Planning under Null Observability (Marco Roveri) 3. Planning under Partial Observability (Piergiorgio Bertoli) 4. Planning for Extended Goals (Marco Pistore) 5. Conclusions and ... Dreams

PO Planning: examples

Under PO, in general status cannot be exactly determined: – ...because the initial situation is uncertain... Initial

– ...because actions may have unpredictable results...

Thus, we reason about sets of states: belief states (BS)

PO Planning: key concepts An action transforms a BS into a BS

?WallS No

Yes

?WallS

Observing some feature may split a BS

Yes

No

Yes

?WallS

No

Goal

GoWest

Yes

GoEast

WallN?

GoNorth

GoSouth

No

Search space and conditional plan

GoEast

GoWest

GoWest

GoEast

GoEast; if WallN then GoSouth; GoWest else GoWest #

"

Search algo: a DFS search



Planning algorithms:

Plans: strong acyclic (non-iterative) conditional.

Goals: Propositional formulas on final states.

Results



Experimental results show that the algorithms works in practice.

Implementation of the planning algorithms by BDD based symbolic model model checking techniques.

guaranteed to terminate for problems admitting such a plan. Failure otherwise. A variety of search styles.

  

100

10

MBP GPT SGP

4

2

6

Medical

3 # of illnesses

Empty room

8 10 12 Room size

14

4

16

18

5

100

10

1

0.1

0.01

0.01

0.1

1

10

100

1000

$ 20

MBP GPT SGP

2

MBP GPT

0

3

10

4

BT with metal detection

5 6 7 # of packages

Maze

20 30 Size of maze

8

40

9

10

$ 50

$

1

1

2

MBP GPT SGP

0

CPU Search time (sec)

CPU Search time (sec)

0.1

0.01

0.01

0.1

1

10

100

1000

$

CPU Search time (sec)

CPU Search time (sec)

$

$

Experimental Results

CPU Search time (sec)

CPU Search time (sec)

100

10

1

0.1

0.01 4

4

6 7 # of rooms

Ring

5 6 7 # of packages

5

8

8

9

9

10

10

BT with metal detection, sniffing, xray and ticking

3

3

MBP GPT

MBP GPT SGP

2

10000 1000 100 10 1 0.1 0.01

2

CPU Search time (sec)

% CPU Search time (sec)

%

1000 100 10 1 0.1 0.01

MBP GPT

2

MBP GPT SGP

0

10000 1000 100 10 1 0.1 0.01 2

3

4

4

6

Empty room

# of rooms

6

Ring

8 10 12 Room size

5

7

14

8

16

9

18

10

20

CPU Search time (sec)

%

1000 100 10 1 0.1 0.01

MBP GPT

0

10

Maze

20 30 Size of maze

40

50

Structure of the talk

Planning as Model Checking in Non-Deterministic Domains:

1. Plannning via Symbolic Model Checking (Alessandro Cimatti) 2. Planning under Null Observability (Marco Roveri) 3. Planning under Partial Observability (Piergiorgio Bertoli) 4. Planning for Extended Goals (Marco Pistore) 5. Conclusions and ... Dreams

objects in

&

given rooms”

Extended Goals: some examples

“Deliver

&

Extended Goals: some examples “Producer rooms”

“ “Keep delivering objects to given rooms as they are generated”

Extended Goals: some examples “Kid doors”

“ “Keep TRYING to deliver objects to given rooms as they are generated”

p

)

q

U

p

U U

W

Gp Fp

p

W

p

EXp

p

U

W

W

p

p

p

p

+

Extended Goals are CTL formulas Extended Goal

p

Abbreviations:

p

Intuitions: Xp pUq p

AXp

p

'

'

9

9

'32 +

6 '

9

'

' -+ 7 ,

7

8' 8'

;

6 8*

' 9 -+ 7

' -+ 5 7

'34 + 8'

'

) )

' :

;

+10 . 8'

,

+5

'

+7

8* 8'

7
K

G

> J >

DE

> BC FL

F H

BC

“move to door”

=

“object grasped”

=

“open door”

Domain

Observations

=

“door closed”

=

“enter room”

=

“door open” =