Minimal Critical Subsystems as Counterexamples for ω-Regular ...

Minimal Critical Subsystems as Counterexamples for ω-Regular DTMC Properties Ralf Wimmer Bernd Becker Albert-Ludwigs-University Freiburg, Germany {wimmer,becker}@informatik.uni-freiburg.de

Nils Jansen

´ Erika Abrah´ am Joost-Pieter Katoen RWTH Aachen, Germany

{jansen,abraham,katoen}@informatik.rwth-aachen.de

Abstract We propose a new approach to compute counterexamples for violated ω-regular properties of discrete-time Markov chains. Whereas most approaches compute a set of system paths as a counterexample, we determine a critical subsystem that already violates the given property. In earlier work methods have been introduced to compute such subsystems for safety properties, based on a search for shortest paths. In this paper we use mixed integer linear programming to determine minimal critical subsystems for arbitrary ω-regular properties.

1. Introduction Systems with uncertainties often act in safety-critical environments. In order to use the advantages of formal verification, formal models are needed. A popular modeling formalism for such systems are discrete-time Markov chains (DTMCs). State-of-the-art model checking algorithms verify probabilistic safety properties like “The probability to reach a safety-critical state is at most 10−3 ” or, more generally, ω-regular properties [1], efficiently by solving linear equation systems [2]. Thereby, if the property is violated, they do not provide any information about the reasons why this is the case. However, this is not only strongly needed for debugging purposes, but it is also exploited for abstraction refinement in CEGAR frameworks [3, 4]. Therefore, in recent years much research effort has been made to develop algorithms for counterexample generation for DTMCs (see, e. g., [5, 6, 7, 8, 9, 10, 11, 12, 13]). With the exception of [12], which handles LTL specifications, all of these approaches are restricted to reachability properties. Furthermore, the algorithms in [6, 7, 8, 9] yield path-based counterexamples for violated safety properties, i. e., counterexamples in the form of a set of finite paths that all lead from the initial state to a safety-critical state and whose joint probability mass exceeds the allowed limit. Unfortunately, the number of paths needed for a counterexample is often very large or even infinite, in particular if the gap between the allowed probability and its actual value is small. The size of the counterexample may be several orders of magnitude larger than the number ∗

This work was partly supported by the German Research Council (DFG) as part of the Transregional Collaborative Research Center “Automatic Verification and Analysis of Complex Systems” (SFB/TR 14 AVACS) and the DFG-Project “CEBug – CounterExample Generation for Stochastic Systems using Bounded Model Checking”

of system states, rendering the counterexample practically unusable for debugging purposes. Different proposals have been made to alleviate this problem: [6] represents the path set as a regular expression, [7] detects loops on paths, and [8] shrinks paths through strongly connected components into single transitions. As an alternative to path-based counterexamples, the usage of small critical subsystems has been proposed in [5, 10]. A critical subsystem is a part of the Markov chain such that the probability to reach a safety-critical state (or, more generally, to satisfy an ω-regular property) inside this part exceeds the bound. This induces a path-based counterexample by considering all paths leading through this subsystem. Contrary to the path-based representation, the size of a critical subsystem is bounded by the size of the model under consideration. Different heuristic methods have been proposed for the computation of small critical subsystems: The authors of [5] apply best first search to identify a critical subsystem, while [10] is based on a hierarchical abstraction of DTMCs in combination with heuristics for the selection of the states to be contained in the subsystem. Both approaches use heuristic methods to select the states of a critical subsystem. However, we are not aware of any algorithm that is suited to compute a minimal critical subsystem, neither in terms of the number of states nor of the number of transitions. In this paper we fill this gap. We provide a formulation as a mixed integer linear program (MILP) which yields state-minimal critical subsystems of DTMCs. We also present some optimizations which significantly speed up the computation times in many cases. Experimental results on some case studies are provided, which show that our approach yields significantly more compact counterexamples than the heuristic methods even if the MILPs cannot be solved to optimality due to time restrictions. We present our algorithms first for probabilistic reachability properties and extend them afterwards to general ω-regular properties. Structure of the Paper. In Sec. 2 we introduce the foundations of DTMCs and critical subsystems. In Sec. 3 we present our approach for the computation of state-minimal subsystems for DTMCs. We discuss experimental results in Sec. 4 and finally draw a conclusion in Sec. 5. 2. Foundations We first introduce discrete-time Markov chains, ω-regular properties, and critical subsystems. Then we take a look at mixed integer linear programs, which we use for the computation of minimal critical subsystems. 2.1. Discrete-Time Markov Chains and ω-Regular Properties Definition 1 Let AP be a set of atomic propositions. A discrete-time Markov chain (DTMC) is a tuple M = (S, sI , P, L) with S being a finite set of states, sI P ∈ S the initial state, and P : S × S → [0, 1] the matrix of transition probabilities such that s0 ∈S P (s, s0 ) ≤ 1 for all s ∈ S.1 L : S → 2AP is a labeling function which assigns to each state s ∈ S the propositions that are valid in s.

1

Please note that we allow sub-stochastic distributions. Usually, the sum of probabilities is required to be exactly 1. This can be obtained by defining M 0 = (S ∪ {s⊥ }, sI , P 0 , L0 ) with s⊥ a fresh sink state, P 0 (s, s0 ) = P (s, s0 ) and L0 (s) = L(s) for all s, s0 ∈ S, L0 (s⊥ ) = ∅, P 0 (s⊥ , s⊥ ) = 1, and finally P (s, s⊥ ) = 1 − P (s, S) and P 0 (s⊥ , s) = 0 for all s ∈ S.

Let in the following M = (S, sI , P, L) be a DTMC the set AP propositions. of atomic  over 0 P (s, s0 ) > 0 , the set of We denote the set of transitions of M by EM = (s, s ) ∈ S × S 0 0 successors ofstate s ∈ S by succM (s) = s ∈ S (s, s ) ∈ EM , and its predecessors by 0 0 predM (s) = s ∈ S (s , s) ∈ EM . A finite (infinite) path in M is a finite (infinite) sequence π = s0 s1 s2 . . . of states such that (si , si+1 ) ∈ EM for all i. We denote the set of finite (infinite) paths starting in s by Pathsfin (s) (Pathsinf (s), resp.). The trace of π, trace(π), is the sequence L(s0 )L(s1 )L(s2 ) . . . A linear-time property over AP is a set L of infinite sequences α0 α1 α2 . . . with αi ⊆ AP for all i. A path π satisfies a linear-time property L if trace(π) ∈ L. In this paper we will deal with a certain subclass of linear-time properties, namely ω-regular properties. In order to define them, we need the notion of Rabin automata2 . Definition 2 A deterministic Rabin automaton (DRA) is a tuple A = (Q, qI , Σ, δ, F ) such that Q is a finite, non-empty set of states, qI ∈ Q an initial state, Σ an input alphabet, δ : Q × Σ → Q a transition function, and F ⊆ 2Q × 2Q an acceptance condition. Let w = σ0 σ1 σ2 . . . ∈ Σω be an infinite word over Σ. The run r of A on w is given by the state sequence q0 q1 q2 . . . ∈ Qω with q0 = qI and qi+1 = δ(qi , σi ) for all i. We denote the set of states which occur infinitely often on a run r by inf(r). Given the acceptance condition F = {(Li , Ki ) | i = 1, . . . , n}, a run r is accepting if, for some i ∈ {1, . . . , n}, inf(r) ∩ Li = ∅ and inf(r) ∩ Ki 6= ∅. The set of words with an accepting run is denoted by L(A). Definition 3 A linear-time property L over AP is ω-regular if there is a deterministic Rabin automaton A = (Q, qI , 2AP , δ, F ) such that L = L(A). For a state s of the DTMC M and an ω-regular property L, we denote the probability to walk along a path which satisfies L when starting in state s ∈ S by PrsM (L) = PrsM ({π ∈ Pathsinf (s) | trace(π) ∈ L}). We consider ω-regular properties where this probability may be at most a given bound λ ∈ [0, 1]. Such properties are denoted by P≤λ (L). We assume that the initial state of M violates this property. The goal is now to identify a smallest possible part of M which already violates P≤λ (L). Definition 4 A subsystem of a DTMC M = (S, sI , P, L) is a DTMC M 0 = (S 0 , s0I , P 0 , L0 ) such that S 0 ⊆ S, s0I ∈ S 0 , L0 (s) = L(s) and P 0 (s, s0 ) > 0 implies P 0 (s, s0 ) = P (s, s0 ) for all s, s0 ∈ S 0 . s0 For a property P≤λ (L), we call M 0 critical if s0I = sI and PrMI 0 (L) > λ. We want to compute a minimal critical subsystem (MCS) of M for P≤λ (L). Minimality can be defined in terms of the number of states or the number of transitions. Though in this paper we focus on state-minimality, our approach can be easily adapted to transition-minimality. For a proposition a ∈ AP , the property to reach an a-state is specified by L = {α0 α1 α2 . . . | ∃i ≥ 0 : a ∈ αi }. We denote this reachability property by the PCTL-Formula P≤λ (♦ a). To check reachability properties, specialized algorithms have been developed, which are much more efficient than the algorithms for arbitrary ω-regular properties. To be more precise, the 2

It is more common to use nondeterministic B¨uchi automata to define ω-regular properties, which have the same expressiveness as deterministic Rabin automata, but can be much more compact. However, we need to construct a product automaton of a DTMC and an appropriate ω-automaton. In case of a deterministic Rabin automaton, the result is again a DTMC, but not in case of a nondeterministic B¨uchi automaton.

probability to eventually reach an a-state from a state s is the unique solution of a linear equation system [2, p. 760] containing for each state s ∈PS the equation ps = 1 if a ∈ L(s), ps = 0 if there is no path from s to any a-state, and ps = s0 ∈S P (s, s0 ) · ps0 in all other cases. Definition 5 Let M = (S, sI , P, L) be a DTMC and a ∈ AP . A state s ∈ S is relevant for a if Sn−1 there is a path π = s0 s1 s2 . . . sn with s0 = sI , a 6∈ i=0 L(si ), a ∈ L(sn ) and s = sj for some j ∈ {0, . . . , n}. A transition (s, s0 ) ∈ EM is relevant if both s and s0 are relevant and a 6∈ L(s). rel rel We write SM for the set of relevant states of M and EM for the set of relevant transitions. States and transitions that are not relevant can be removed from a DTMC without changing rel rel the probability to reach a target state. The following lemma shows that SM and EM can be determined in linear time in the size of the DTMC by two simple graph analyses.

Lemma 1 Assume a DTMC M = (S, sI , P, L) and a proposition a ∈ AP , and let T = {s ∈ − S|a ∈ L(s)}. Let furthermore EM = (s, s0 ) ∈ S × S (s0 , s) ∈ EM be the set of reversed − transitions of M . We consider the directed graphs G = (S, EM ) and G− = (S, EM ). A state s ∈ S is relevant for a iff s is reachable from sI in G and s is reachable from a state t ∈ T in G− . A transition (s, s0 ) ∈ EM is relevant iff s is reachable from sI in G and s0 is reachable from a state t ∈ T in G− , and s 6∈ T . 2.2. Mixed Integer Linear Programming A mixed integer linear program optimizes an objective function under a condition specified by a conjunction of linear inequalities. A subset of the variables in the inequalities are restricted to take only integer values, which makes solving MILPs NP-hard. Definition 6 Let A ∈ Qn×m , B ∈ Qk×m , b ∈ Qm , c ∈ Qn , and d ∈ Qk . A mixed integer linear program (MILP) consists in computing min cT x + dT y such that Ax + By ≤ b and x ∈ Rn , y ∈ Zk . MILPs are typically solved by a combination of a branch-and-bound algorithm with the generation of so-called cutting planes. These algorithms heavily rely on the fact that relaxations of MILPs which result from removing the integrality constraints, can be solved efficiently. MILPs are widely used in operations research, hardware-software codesign and numerous other applications. Efficient open source as well as commercial implementations are available like S CIP or C PLEX. We refer the reader to, e. g., [14] for more information on solving MILPs. 3. Computing Minimal Critical Subsystems for DTMCs We give an MILP formulation3 of the problem to find state-minimal critical subsystems for DTMCs and violated reachability properties in Sec. 3.1. We report on optimizations in Sec. 3.2 and extend our approach to general ω-regular properties in Sec. 3.3. 3.1. Minimal Critical Subsystems for Reachability Properties Let M = (S, sI , P, L) be a DTMC, a ∈ AP a proposition, T = {s ∈ S | a ∈ L(s)} the set of so-called target states, and P≤λ (♦ a) a reachability property which is violated by M . We want to 3

We also worked out Sat-Modulo-Theories formulations, but they showed to be less efficient.

 1 X  − ps I + xs 2 rel

(1a)

ps I > λ

(1b)

rel ∩ T : p s = xs ∀s ∈ SM

(1c)

minimize

s∈SM

such that ∀s ∈

rel SM

\ T : ps ≤ x s

(1d) X

ps ≤

s0 ∈succ

P (s, s0 ) · ps0 .

(1e)

rel M (s)∩SM

Figure 1: MILP formulation for state-minimal critical subsystems of DTMCs

identify a state-minimal critical subsystem of M such that the probability to walk from sI to a state in T in the subsystem is larger than λ. This can be formulated by the MILP shown in Fig. 1. rel We introduce a variable xs ∈ [0, 1] ⊆ Z for each s ∈ SM , indicating if s belongs to the subsystem rel (xs = 1) or not (xs = 0). A real-valued variable ps ∈ [0, 1] ⊆ R for each state s ∈ SM contains the probability of reaching a target state from s inside the MCS. Selected target states have rel probability one, non-selected target states zero (line 1c). For non-target states s ∈ SM \ T, either s is not selected and ps = 0, or s is selected and ps is the sum of the probabilities ps0 of all relevant successor states s0 , weighted by the according transition probabilities P (s, s0 ). For non-selected states s, line 1d ensures that ps = 0. To keep our inequations linear, line 1e defines the above-mentioned sum to be an upper bound on the probabilities, covering both selected and unselected states. Line 1b assures that the subsystem is critical. In order to obtain a state-minimal critical subsystem, we have to minimize the number of xs variables with value 1, or equivalently, the sum over all xs -variables. However, due to the inequations in line 1e, minimizing this sum would give us only lower bounds on the exact probabilities of reaching target states in the resulting MCS. We can achieve to compute the exact probabilities by forcing the solver to additionally maximize psI (line 1a). This way the solver even computes a minimal MCS with maximal probability. The factor 1/2 is needed because if we only subtract the probability of the initial state, the solver may add an additional state if this results in psI = 1. 3.2. Optimizations In the following we describe optimizations by adding redundant constraints to the problem, which may help the solver to detect unsatisfiable branches in the search space earlier. 3.2.1. Successor and Predecessor Constraints We can guide the solver not to select states that are unreachable in the subsystem by adding the following constraints to the MILP formulation in Fig. 1: X rel ∀s ∈ SM \ T : − xs + xs 0 ≥ 0 (2a) rel )\{s} s0 ∈(succM (s)∩SM

rel ∀s ∈ SM \ {sI } : − xs +

X rel )\{s} s0 ∈(predM (s)∩SM

xs 0 ≥ 0 .

(2b)

The first set of constraints (2a), which we call forward cuts, states that each non-target state in the MCS must have a proper successor state in the MCS. Proper means that self-loops are ignored. The second set of constraints (2b), called backward cuts, requires that each non-initial state in the MCS has a proper predecessor in the MCS. Forward and backward cuts do not modify the feasible solutions, but add cutting planes which tighten its LP-relaxation. 3.2.2. SCC Constraints The forward respectively backward cuts do not encode that all states of the MCS are forwards respectively backwards reachable, since they allow loops that are unreachable in the MCS. To strengthen the effect of forward and backward cuts, we make use of strongly connected components. Formally, a strongly-connected component (SCC) of a DTMC M = (S, sI , P, L) is a maximal subset C ⊆ S such that each state s ∈ C is reachable from each state s0 ∈ C visiting only states from C. The input states In(C) = {s ∈ C | ∃s0 ∈ S \ C : P (s0 , s) > 0} of an SCC C have an in-coming transition from outside the SCC. The output states Out(C) = {s ∈ S \ C | ∃s0 ∈ C : P (s0 , s) > 0} of C are those states outside C which can be reached from C via a single transition. A state of an SCC can be reached from the initial state only through one of the SCC’s input states. Therefore we define an SCC input cut for each SCC C assuring that, if none of C’s input states is included in the MCS, then the MCS does not contain any states from C: X X xs ≤ C \ In(C) · xs . (3a) s∈C\In(C)

s∈In(C)

Analogously, starting from a state inside an SCC, all paths to a target state lead through one of the SCC’s output states. Therefore, the SCC output cut states that if no output state of an SCC C is selected then we do not want to select any state from C: X X xs ≤ C · xs . (4a) s∈C

s∈Out(C)

3.2.3. Complete Reachability Encoding The SCC cuts still do not encode reachability exactly, since they allow the selection of unreachable loops in reachable SCCs. For a complete encoding of forward reachability, we introduce a rel variable rs→ ∈ [0, 1] ⊆ R for each state s ∈ SM . The values of these variables define a partial order on the states, which we use to encode that for each selected state s there is a path s0 . . . sn → in the MCS from the initial state s0 = sI to sn = s such that ri→ < ri+1 for all 0 ≤ i < n. For 0 each transition from a state s to s we additionally need an integer variable t→ s,s0 ∈ [0, 1] ⊆ Z representing the choice of the predecessor state in the above paths. Note that proper values always exist, e.g., choosing for each state s the longest loop-free path from sI to s and setting rel rs→ = ns /|SM | with ns the length of the path. Forward reachability is encoded as follows:

rel rel ∀s∈SM ∀s0 ∈ (succM (s)∩SM ) : 2t→ s,s0 ≤ xs + xs0

(5a)

rs→ < rs→0 + (1 − t→ s,s0 ) X rel \ {sI } : (1 − xs ) + ∀s ∈ SM

(5b) t→ s0 ,s ≥ 1 .

(5c)

rel s0 ∈predM (s)∩SM

Lines 5a and 5b encode that each transition from s to s0 with t→ s,s0 = 1 connects selected states → → with rs < rs0 . Under this assumption, the constraints in line 5c imply by induction that for each selected state there is a reachable selected predecessor state. Backward reachability is encoded analogously using variables rs← and t← s,s0 , which we substitute → → for the variables rs and ts,s0 in the above constraints. rel rel ∀s∈SM ∀s0 ∈ (succM (s)∩SM ) : 2t← s,s0 ≤ xs + xs0

(6a)

rs← < rs←0 + (1 − t← s,s0 ) X rel ∀s ∈ SM \ T : (1 − xs ) + s0 ∈succ

(6b) t← s,s0 ≥ 1 .

(6c)

rel M (s)∩SM

These encodings come at the cost of new variables, but they exclude all subsystems with unreachable states. 3.3. General ω-Regular Properties We now show how the techniques described so far for reachability properties can be generalized to arbitrary ω-regular properties L with an upper bound λ on the probability of satisfying L, i. e., properties of the form P≤λ (L). To do so we need the product automaton of the DTMC under consideration with the DRA of L. Definition 7 Let M = (S, sI , P, L) be a DTMC and A = (Q, qI , 2AP , δ, F ) with F = {(Li , Ki )| 1 ≤ i ≤ n} a DRA for the ω-regular property L. The product automaton M ⊗ A is the DTMC M ⊗ A = (S × Q, (sI , δ(qI , L(sI )), P 0 , L0 ) such that P 0 ((s, q), (s0 , q 0 )) equals P (s, s0 ) if q 0 = δ(q, L(s0 )) and 0 otherwise. We use the sets Li , Ki as new atomic propositions for labeling the states, i. e., for H ∈ {Li , Ki | 1 ≤ i ≤ n}, H ∈ L0 ((s, q)) iff q ∈ H. SCCs of M ⊗ A that cannot be left, the so-called bottom SCCs (BSCCs), play a crucial role in the computation of PrsM (L). We call a BSCC T of M ⊗ A accepting iff there is an index i ∈ {1, . . . , n} such that T ∩ (S × Li ) = ∅ and T ∩ (S × Ki ) 6= ∅. Lemma 2 (Theorem 10.56 of [2]) Let M be a DTMC, s a state of M , A a DRA for L, and let T be the union of all accepting BSCCs in M ⊗ A. Then (s,δ(q ,L(s)))

PrsM (L) = PrM ⊗AI

(♦ T ).

This means, computing probabilities of ω-regular properties reduces to a reachability analysis in the product automaton. The probability to finally reach a BSCC of a DTMC and to visit all of its states infinitely often equals 1 [2, Theorem 10.27]. This implies, in order to obtain an MCS, we

 1 minimize − p(sI ,δ(qI ,L(sI ))) + 2

X

xs



such that p(sI ,δ(qI ,L(sI ))) > λ

(7b)

rel ∀(s, q) ∈ T ∩ SM ⊗A : xs = ps

∀i ∈ {1, . . . , k}∀s ∈ Ti ∩ ∀(s, q) ∈

rel SM ⊗A

rel SM ⊗A

(7c)

: xTi = xs

(7d)

\ T : p(s,q) ≤ xs p(s,q) ≤

(7a)

rel s∈SM ⊗A |M

(7e) X

P 0 ((s, q), (s0 , q 0 )) · p(s0 ,q0 ) . (7f)

rel (s0 ,q 0 )∈succM ⊗A (s,q)∩SM ⊗A

Figure 2: Minimal critical subsystems for ω-regular properties

either have to take all states of an accepting BSCC of M ⊗ A or none of them. One can show that the size of the resulting MCS is the same for all DRAs = L. Sk A with L(A) rel Let T1 , . . . , Tk be the accepting BSCCs of M ⊗ A, T = i=1 Ti , and SM ⊗A the relevant states of M ⊗ A (regarding the reachability of T ). For a set R ⊆ S × Q we denote by R|M the projection of R on S, i. e., R|M = {s ∈ S | ∃q ∈ Q : (s, q) ∈ R}. Since we want an MCS of M , but compute the probabilities in M ⊗ A, we introduce one variable xs ∈ [0, 1] ⊆ Z for each state rel rel s ∈ SM ⊗A |M and one variable p(s,q) ∈ [0, 1] ⊆ R for each pair (s, q) ∈ SM ⊗A . Furthermore we use a variable xTi ∈ [0, 1] ⊆ Z for each accepting BSCC Ti which indicates if the BSCC is relevant for the MCS. The MILP formulation for MCSs is shown in Fig. 2. It follows the ideas of MCSs for reachability (see Fig. 1), with the exception that we minimize the projection of the subsystem onto the original DTMC. Constraint (7d) takes care of the fact that we either have to take all states of a BSCC or none by requiring that for all s ∈ Ti |M the variables xs have the same value. All optimizations like SCC cuts or reachability cuts can be applied to ω-regular properties in a straightforward way. 4. Experimental Evaluation We implemented our approach in a tool called S UB S YS in C++ and applied it to two series of test cases. For all benchmarks, we used P RISM [15] models available at http:// prismmodelchecker.org. (1) The crowds protocol [16] provides a mechanism for anonymous web browsing by routing messages through a network of N nodes. If a node wants to send a message, it has a probabilistic choice whether to deliver the message directly to its destination or to forward it to a randomly selected successor node. This procedure preserves anonymous sending of messages, as the original sender of a message cannot be determined. One instance consists of R rounds of message deliveries. We denote the different instances by crowdsN -R. The set T of target states contains all those states where a bad group member could identify the sender of a message. (2) The synchronous leader election protocol [17] models the selection of a distinguished leader node in a ring of N identical network nodes. In each round, every node randomly selects an integer number from {0, . . ., K}. The node with the highest number becomes the leader, if this number is unique. Otherwise a new round starts. We denote the instances for different N and K by leaderN -K.

Table 1: Sizes of the benchmark models and comparison with the heuristic local search method of [10] Model |S| |EM | |T | λ |SMCS | |EMCS | |Sheur | |Eheur | crowds2-3 crowds2-4 crowds2-5 crowds3-3 crowds3-4 crowds3-5 crowds5-4 crowds5-6 crowds5-8 leader3-2 leader3-3 leader3-4 leader4-2 leader4-3 leader4-4 leader4-5 leader4-6

183 356 612 396 901 1772 3515 18817 68740 22 61 135 55 256 782 1889 3902

243 476 822 576 1321 2612 6035 32677 120220 29 87 198 70 336 1037 2513 5197

26 85 196 37 153 425 346 3710 19488 1 1 1 1 1 1 1 1

0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

22 22 22 37 37 37 72 72 72 15 33 70 34 132 395 946 1953

27 27 27 51 51 51 123 123 123 18 45 101 41 171 522 1257 2600

23 23 23 40 40 40 94 145 198 17 40 76 44 170 459 1050 2103

27 27 27 56 56 56 156 253 356 20 54 108 54 220 605 1393 2797

All experiments were performed on a computer with four 2.3 GHz AMD Opteron Quad-Core CPUs and 64 GB memory, running Ubuntu 10.04 Linux in 64-bit mode. We aborted any experiment which did not finish within 7200 s (denoted by “– TL –”) or needed more than 4 GB of memory (“– ML –”). For solving the MILPs we used a number of state-of-the-art solvers, from which we selected, after a series of preliminary experiments, the most efficient ones, namely the publicly available S CIP 2.0.2 solver (http://scip.zib.de) and the commercial C PLEX 12.3 solver (http: //www-01.ibm.com/software/integration/optimization/cplex-optimizer). Table 1 contains statistics on our benchmarks. The columns contain (from left to right) the model name, the number of states, the number of transitions, the number of target states, the probability bound, the number of states in the MCS, and finally the number of transitions in the MCS. The last two columns contain the sizes of heuristically computed critical subsystems using the local search approach of [10]. For all instances the heuristic tool terminated within 6 min. In Table 2 we give the running times of S CIP and C PLEX in seconds. C PLEX supports a parallel mode, in which we started 16 parallel threads. Therefore we give for C PLEX the accumulated times of all threads and, in parentheses, the actual time from start to termination of the tool. The block of columns entitled “w/o redundant constraints” contains the running times of the solvers without any optimizations. The block “optimal conf.” lists the optimal times, i. e., the times achieved by adding the combination of optional constraints that leads to the smallest computation time. These running times can be obtained in general by using a portfolio approach which runs the different combinations of redundant constraints in parallel. The optimal running times were in some cases by orders of magnitude smaller, especially for the large benchmarks were the standard formulation could not be solved within the time limit. To illustrate the effects of the different optimizations, we give more detailed results for crowds5-6 and leader4-5 in Table 3. The left column block lists running times without SCC cuts. The column “–” contains the values without forward and backward cuts, “→” with forward cuts, “←” the with backward cuts and “↔” with both. The values for the four different combinations of

Table 2: Running times S CIP and C PLEX for computing MCSs w/o redundant constraints optimal conf. Model S CIP C PLEX S CIP C PLEX crowds2-3 0.16 1.33 (0.28) 0.12 0.06 (0.11) crowds2-4 0.47 0.30 (0.24) 0.30 0.30 (0.24) crowds2-5 0.90 0.56 (0.45) 0.60 0.56 (0.24) crowds3-3 0.64 0.49 (0.33) 0.35 0.38 (0.30) crowds3-4 4.29 5.53 (2.07) 1.45 0.89 (0.58) crowds3-5 23.49 6.66 (2.77) 5.58 1.51 (0.87) crowds5-4 743.84 14.23 (5.07) 13.28 12.51 (4.89) crowds5-6 – TL – 302.03 (38.39) 1947.46 100.26 (23.52) crowds5-8 – TL – – TL – – TL – 1000.79 (145.84) leader3-2 0.07 0.62 (0.22) 0.01 0.21 (0.13) leader3-3 91.89 0.43 (0.22) 0.06 0.02 (0.06) leader3-4 2346.59 0.70 (0.36) 0.40 0.07 (0.09) leader4-2 0.23 0.45 (0.21) 0.07 0.24 (0.17) leader4-3 1390.79 22.33 (3.38) 0.21 0.49 (0.37) leader4-4 – TL – – TL – 1.49 1.88 (1.21) leader4-5 – TL – – TL – 1.15 4.06 (2.80) leader4-6 – TL – – ML – – TL – 8.70 (5.92)

reachability cuts (none, only forward, only backward, and both) are listed in the according rows of the table. Comparing the values for the reachability cuts, we can observe that they have a negative effect for crowds5-6. However, they speed up the solution of leader4-5 by a factor of 103 . The same tendency can be observed for the other crowds and leader instances. The addition of backward cuts to crowds5-6 reduces the running time to about one third, and they typically decrease the times for most of the instances. Since the SCC cuts are even less effective, we only give the minimal value of the three cases (with SCC input, SCC output, and both). Fig. 3 shows the size of the MCS of crowds5-6 for different values of λ (red solid lines), comparing it with the size of heuristically computed critical subsystems using the local search of [10] (blue dotted lines). For λ ≥ 0.23, we could only compute an upper bound (within 8 % from the optimal value) on the size of the MCS with a timeout of 2 hours. Also the local search tool

# states (min) # transitions (min) # states (heur) # transitions (heur)

7000 6000 5000 4000 3000 2000 1000 0 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

λ Figure 3: Size of the MCS and heuristically determined critical subsystems for crowds5-6 and different values of λ

leader4-5

crowds5-6

Table 3: Runtimes for crowds5-6 and leader4-5 with and without redundant constraints using C PLEX as solver no SCC cuts with SCC cuts Reach – → ← ↔ – → ← ↔ 302.03 367.49 103.20 149.73 301.87 342.07 100.26 138.36 none (38.39) (44.70) (23.52) (26.07) (38.64) (42.76) (23.52) (25.20) 656.13 1292.59 651.47 966.57 634.40 833.37 646.64 925.95 fwd (120.04) (148.82) (112.47) (127.94) (118.22) (108.13) (111.18) (125.52) 4043.93 3613.96 770.90 1070.50 3911.81 3603.49 756.28 1074.36 bwd (384.74) (358.49) (121.02) (130.45) (375.48) (358.25) (119.90) (130.72) 2107.84 1403.44 5972.98 2191.83 1986.37 1379.78 5925.31 2210.68 both (251.41) (185.34) (546.83) (281.07) (238.58) (183.38) (542.18) (284.51) none – TL – – TL – – TL – – TL – – TL – – TL – – TL – – TL – 284.04 254.56 286.83 261.53 294.71 259.98 285.33 251.69 fwd (40.02) (35.97) (40.85) (36.46) (41.15) (36.21) (40.57) (35.80) 6.30 6.29 6.27 6.10 5.89 5.73 5.78 5.95 bwd (3.73) (3.69) (3.72) (3.71) (3.65) (3.66) (3.67) (3.69) 4.46 4.06 4.34 4.56 4.17 4.41 4.10 4.39 both (2.77) (2.80) (2.83) (2.91) (2.77) (2.83) (2.84) (2.90)

ran into a timeout for λ ≥ 0.35, however, without yielding a critical subsystem. 5. Conclusion In this paper we used MILP solver to compute state-minimal critical subsystems for DTMCs. By adding redundant constraints, which tighten the LP-relaxation of the MILP, the solution process can be speeded up significantly. A topic for future research is to analyze the theoretical complexity of computing MCSs for DTMCs. We conjecture that this problem is NP-complete. We also plan to integrate the MILP approach into the hierarchical counterexample generation tool described in [10]. References [1] Bustan, D., Rubin, S., Vardi, M.Y.: Verifying ω-regular properties of Markov chains. In: Int’l Conf. on Computer Aided Verification (CAV). Volume 3114 of Lecture Notes in Computer Science, Springer (2004) 189–201 [2] Baier, C., Katoen, J.P.: Principles of Model Checking. The MIT Press (2008) [3] Hermanns, H., Wachter, B., Zhang, L.: Probabilistic CEGAR. In: Int’l Conf. on Computer Aided Verification (CAV). Volume 5123 of Lecture Notes in Computer Science, Springer (2008) 162–175 [4] Chadha, R., Viswanathan, M.: A counterexample-guided abstraction-refinement framework for Markov decision processes. ACM Trans. on Computational Logic 12(1) (2010) 1–45 [5] Aljazzar, H., Leue, S.: Directed explicit state-space search in the generation of counterexamples for stochastic model checking. IEEE Trans. on Software Engineering 36(1) (2010) 37–60 [6] Han, T., Katoen, J.P., Damman, B.: Counterexample generation in probabilistic model checking. IEEE Trans. on Software Engineering 35(2) (2009) 241–257

[7] Wimmer, R., Braitling, B., Becker, B.: Counterexample generation for discrete-time Markov chains using bounded model checking. In: Int’l Conf. on Verification, Model Checking, and Abstract Interpretation (VMCAI). Volume 5403 of Lecture Notes in Computer Science, Springer (2009) 366–380 [8] Andr´es, M.E., D’Argenio, P., van Rossum, P.: Significant diagnostic counterexamples in probabilistic model checking. In: Haifa Verification Conference. Volume 5394 of Lecture Notes in Computer Science, Springer (2008) 129–148 [9] G¨unther, M., Schuster, J., Siegle, M.: Symbolic calculation of k-shortest paths and related measures with the stochastic process algebra tool C ASPA. In: Int’l Workshop on Dynamic Aspects in Dependability Models for Fault-Tolerant Systems (DYADEM-FTS), ACM Press (2010) 13–18 ´ [10] Jansen, N., Abrah´ am, E., Katelaan, J., Wimmer, R., Katoen, J.P., Becker, B.: Hierarchical counterexamples for discrete-time Markov chains. In: Int’l Symp. on Automated Technology for Verification and Analysis (ATVA). Volume 6996 of Lecture Notes in Computer Science, Springer (2011) 443–452 [11] Kattenbelt, M., Huth, M.: Verification and refutation of probabilistic specifications via games. In: IARCS Annual Conf. on Foundations of Software Technology and Theoretical Computer Science (FSTTCS). Volume 4 of LIPIcs, Schloss Dagstuhl – Leibniz-Zentrum f¨ur Informatik (2009) 251–262 [12] Schmalz, M., Varacca, D., V¨olzer, H.: Counterexamples in probabilistic LTL model checking for Markov chains. In: Int’l Conf. on Concurrency Theory. Volume 5710 of Lecture Notes in Computer Science, Springer (2009) 587–602 [13] Fecher, H., Huth, M., Piterman, N., Wagner, D.: PCTL model checking of Markov chains: Truth and falsity as winning strategies in games. Performance Evaluation 67(9) (2010) 858–872 [14] Schrijver, A.: Theory of linear and integer programming. Wiley (1986) [15] Kwiatkowska, M.Z., Norman, G., Parker, D.: P RISM 4.0: Verification of probabilistic real-time systems. In: Int’l Conf. on Computer Aided Verification (CAV). Volume 6806 of Lecture Notes in Computer Science, Springer (2011) 585–591 [16] Reiter, M.K., Rubin, A.D.: Crowds: Anonymity for web transactions. ACM Trans. on Information and System Security 1(1) (1998) 66–92 [17] Itai, A., Rodeh, M.: Symmetry breaking in distributed networks. Information and Computation 88(1) (1990) 60–87