Self-Loop Aggregation Product (SLAP). AM: s0 s1 s2 s3 s4 s5 s6 s7 a¯bc a¯b¯c a¯bc a¯b¯c ab¯c a¯bc. ¯ab¯c. ¯abc. A¬&: q0 q1 a¯b b. Self-Loop Aggregation ...
Self-Loop Aggregation Product — A New Hybrid Approach to On-the-Fly LTL Model Checking Alexandre Duret-Lutz (LRDE/EPITA) Kais Klai (LIPN/Paris 13) Denis Poitrenaud (LIP6/Paris 6) Yann Thierry-Mieg (LIP6/Paris 6)
ATVA 2011 October 2011 http://move.lip6.fr/software/DDD/ltl_bench.html ATVA’11
Self-Loop Aggregation Product
1 / 17
Automata-Theoretic Explicit LTL Model Checking High-level model M
State-space generation
State-space automaton AM Synchronized product L (A¬ϕ ⊗ AM ) = L (A¬ϕ ) ∩ L (AM )
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
Product Automaton A¬ϕ ⊗ AM
Emptiness check ? L (A¬ϕ ⊗AM ) = ∅
M |= ϕ or counterexample
2 / 17
Automata-Theoretic Explicit LTL Model Checking High-level model M
State-space generation
State-space automaton AM Synchronized product L (A¬ϕ ⊗ AM ) = L (A¬ϕ ) ∩ L (AM )
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
Product Automaton A¬ϕ ⊗ AM
Emptiness check ? L (A¬ϕ ⊗AM ) = ∅
M |= ϕ or counterexample
2 / 17
Automata-Theoretic Explicit LTL Model Checking High-level model M
On-the-fly generation of state-space automaton AM
Synchronized product L (A¬ϕ ⊗ AM ) = L (A¬ϕ ) ∩ L (AM )
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
Product Automaton A¬ϕ ⊗ AM
Emptiness check ? L (A¬ϕ ⊗AM ) = ∅
M |= ϕ or counterexample
2 / 17
Automata-Theoretic Explicit LTL Model Checking High-level model M
On-the-fly generation of state-space automaton AM
On-the-fly synchronized product L (A¬ϕ ⊗ AM ) = L (A¬ϕ ) ∩ L (AM )
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
Emptiness check ? L (A¬ϕ ⊗AM ) = ∅
M |= ϕ or counterexample
2 / 17
Explicit Approach
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ATVA’11
ab¯ A¬ϕ :
Self-Loop Aggregation Product
q0
> b
q1
3 / 17
Explicit Approach
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
A¬ϕ ⊗ AM : q0 , s3
¯ abc q0 , s2
¯c ¯ ab¯ c q , s ab¯ c q ,s q0 , s0 abc q0 , s4 ab¯ 1 5 1 4 ¯ abc ¯ ab¯ c q ,s 0 1
¯ abc q1 , s6
¯abc ¯ab¯ c q ,s 1 7
Emptiness check = search for an accepting cycle in the product State explosion problem ATVA’11
Self-Loop Aggregation Product
3 / 17
Symbolic Observation Graph (SOG)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
For stuttering invariant properties (e.g., LTL\ X) Ignore non-observable propositions Aggregate Kripke states with homogeneous labels Represent aggregates using BDDs
K. Klai and D. Poitrenaud. MC-SOG: An LTL model checker based on symbolic observation graphs. In Proc. of PN’08, vol. 5062 of LNCS, pp. 288–306. Springer ATVA’11
Self-Loop Aggregation Product
4 / 17
Symbolic Observation Graph (SOG)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
ab¯ { ss02
SOG: ab¯ ab¯
s1 s3
q0
> b
q1
ab¯ {s5 }
}
{s4 }
{s6 s7 }
ab
¯ab
K. Klai and D. Poitrenaud. MC-SOG: An LTL model checker based on symbolic observation graphs. In Proc. of PN’08, vol. 5062 of LNCS, pp. 288–306. Springer ATVA’11
Self-Loop Aggregation Product
4 / 17
Product Sizes: Kripke vs. SOG A¬ϕ ⊗ AM : q0 , s3
¯ abc q0 , s2
¯c ¯ ab¯ c q , s ab¯ c q ,s q0 , s0 abc q0 , s4 ab¯ 1 5 1 4 ¯ abc ¯ ab¯ c q ,s 0 1 q0 , { ss02
A¬ϕ ⊗ SOG: ab¯
¯ abc
¯abc ¯ab¯ c q ,s 1 7
q1 , s6 s1 s3
}
ab¯
q0 , {s4 }
ab
ab¯ q0 , ab¯
ab
q1 , {s4 }
¯ab
q1 , {s5 } ab¯ q1 , {s6 s7 }
K. Klai and D. Poitrenaud. MC-SOG: An LTL model checker based on symbolic observation graphs. In Proc. of PN’08, vol. 5062 of LNCS, pp. 288–306. Springer ATVA’11
Self-Loop Aggregation Product
5 / 17
BCZ: Multiple-State Tableaux
Similar to SOG with only 1 step par aggregate: supports full LTL no need to search for livelock cycles
Low aggregation power: on our example with low branching, it does not reduce the Kripke structure.
A. Biere, E. M. Clarke, and Y. Zhu. Multiple state and single state tableaux for combining local and global model checking. In Correct System Design, vol. 1710 of LNCS, pp. 163–179. Springer, 1999 ATVA’11
Self-Loop Aggregation Product
6 / 17
Building a Product Directly High-level model M
On-the-fly generation of state-space automaton AM
On-the-fly synchronized product L (A¬ϕ ⊗ AM ) = L (A¬ϕ ) ∩ L (AM )
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
Emptiness check ? L (A¬ϕ ⊗AM ) = ∅
M |= ϕ or counterexample
7 / 17
Building a Product Directly High-level model M
Dynamic and on-the-fly generation of an automaton D such that L (D) = ∅ ⇐⇒ L (A¬ϕ ⊗ AM ) = ∅.
Emptiness check ? L (D) = ∅
LTL property ϕ ATVA’11
LTL translation
Negated property automaton A¬ϕ
Self-Loop Aggregation Product
M |= ϕ or counterexample
7 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
ATVA’11
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
> b
q0
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
q1 , { ss46
s5 s7
}
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
> b
q0
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
q1 , { ss46
s5 s7
}
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
> b
q0
q1
Self-Loop Aggregation Product:
q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
q1 , { ss46
s5 s7
}
>
8 / 17
Self-Loop Aggregation Product (SLAP)
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
> b
q0
q1
Self-Loop Aggregation Product: > q0 ,
n s0
ATVA’11
s1 s2 s3 s4
o
>
q1 , {s5 }
>
Self-Loop Aggregation Product
q1 , { ss46
s5 s7
}
8 / 17
Fully Symbolic Approach (FS)
Encode the Kripke structure and the property automaton using BDDs. Combine the two BDDs. (Symbolic product.) Use fixpoint computations to decide whether the product contains an accepting cycle. Two symbolic emptiness checks compared: EL OWCTY K. Fisler, R. Fraer, G. Kamhi, M. Y. Vardi, and Z. Yang. Is there a best symbolic cycle-detection algorithm? In Proc. of TACAS’01, vol. 2031 of LNCS, pp. 420–434. Springer F. Somenzi, K. Ravi, and R. Bloem. Analysis of symbolic SCC hull algorithms. In Proc. of FMCAD’02, vol. 2517 of LNCS, pp. 88–105. Springer ATVA’11
Self-Loop Aggregation Product
9 / 17
SLAP-FST FST = Fully Symbolic search in Terminal automata states When reaching a state of A¬ϕ which is terminal (and accepting) use a fully symbolic search.
AM :
¯c ab¯ s3
¯ abc s0
ab¯ c s4
¯ abc s5
s2
s1
s7
s6
¯ abc
¯c ab¯
¯abc
¯ab¯ c
ab¯ A¬ϕ :
q0
> b
q1
> SLAP-FST: ATVA’11
q0 ,
n s0
s1 s2 s3 s4
o
>
q1 , {s5 }
Self-Loop Aggregation Product
10 / 17
Summary of methods
logic prop. aut. data str. empt. chk
Explicit BCZ LTL graph
SOG SLAP LTL\ X explicit BDD+graph explicit
PDP LTL
FS
symb./expl. BDD[] BDD symbolic (OWCTY/EL)
PDP = Property Driven Partitioning, R. Sebastiani, S. Tonetta, and M. Y. Vardi. Symbolic systems, explicit properties: on hybrid approches for LTL symbolic model checking. In Proc. of CAV’05, vol. 3576 of LNCS, pp. 350–363. Springer ATVA’11
Self-Loop Aggregation Product
11 / 17
Experimental Framework Implemented algorithms BCZ, SOG, SLAP, SLAP-FST, EL, OWCTY Tools used SDD Hierarchical Set Decision Diagrams (http://ddd.lip6.fr/) Hierarchy (memory gain) Automatic saturation (improves fixpoint computations) Spot Model checking library (http://spot.lip6.fr/) Good LTL-to-TGBA translation Explicit emptiness-check algorithms Data Models Scalable toy models with 106 . . . 1066 states. Formulas Random formulas (filtered), plus a set of (random) weak fairness properties. ATVA’11
Self-Loop Aggregation Product
12 / 17
Cumulative Plot: Violated Formulas 4500 SLAP SLAP-FST SOG BCZ EL OWCTY
4000
3500
3000
2500
2000
1500
1000
500
0 0 ATVA’11
20
40
60
Self-Loop Aggregation Product
80
100
120 13 / 17
Cumulative Plot: Verified Formulas 3500 SLAP SLAP-FST SOG BCZ EL OWCTY
3000
2500
2000
1500
1000
500
0 0 ATVA’11
20
40
60
Self-Loop Aggregation Product
80
100
120 14 / 17
Scatter Plots: SLAP-FST vs. Hybrid Runtime in seconds. Timeout at 120s. 1000
1000
empty non-empty unknown
100
10
10 BCZ
SOG
100
empty non-empty unknown
1
1
0.1
0.1
SLAP-FST 0.01 0.01
0.1
ATVA’11
1
SLAP-FST 10
100
1000
0.01 0.01
0.1
Self-Loop Aggregation Product
1
10
100
1000
15 / 17
Scatter Plots: SLAP-FST vs. Fully Symbolic Runtime in seconds. Timeout at 120s. 1000
1000
empty non-empty unknown
100
10
10 EL
OWCTY
100
empty non-empty unknown
1
1
0.1
0.1
SLAP-FST 0.01 0.01
0.1
ATVA’11
1
SLAP-FST 10
100
1000
0.01 0.01
0.1
Self-Loop Aggregation Product
1
10
100
1000
16 / 17
Conclusion SLAP, a new hybrid approach: Aggregates states at the product level, not in the Kripke structure Can therefore adjust dynamically to the features (self-loops and acceptance conditions) present in the property automaton. SLAP-FST: a variant using a fully-symbolic algorithm for terminal states.
SLAP-FST most competitive algorithm of those we compared, on this benchmark On-the-fly computations have strong impact when counterexamples exist. Hybrid approaches outperform fully symbolic approaches in these cases. http://move.lip6.fr/software/DDD/ltl_bench.html ATVA’11
Self-Loop Aggregation Product
17 / 17
