Parallel NuSMV: A NuSMV Extension for the Verification of Complex Embedded Systems Orlando Ferrante, Luca Benvenuti, Leonardo Mangeruca, Christos Sofronis, and Alberto Ferrari ALES S.r.l., Rome, Italy
[email protected]
Abstract. In this paper we present Parallel NuSMV, a tool based on the NuSMV model checker that integrates the ManySAT parallel SAT solver. The PNuSMV is part of the FormalSpecs Verifier framework for the formal verification of Simulink/Stateflow models. The experiments we performed show that the use of a parallel SAT solver allows for an average speedup of an order of magnitude or more on industry-level size models. The main contributions of the papers are (1) the description of the PNuSMV model checker (2) the description of the verification time speedup w.r.t. the NuSMV tool for the verification of industrial-sized embedded systems and (3) the integration of the tool in the FormalSpecs Verifier framework for the verification of Simulink/Stateflow models with the application to a cruise control case study. Keywords: model checking, embedded systems, contract-based design, formal verification.
1
Introduction
Model checking has reached a high maturity level that allows applying this technique to the verification of complex embedded systems. Several techniques and tools have been proposed to tackle industrial-sized models. In [1] the authors describe the verification of a Flight Control System modeled in MATLAB Simulink using the NuSMV model checker. In [2] the verification of avionics embedded software is performed using three different model checkers (namely NuSMV [3], SAL [4] and PROVER [5]). In addition, model checkers have been successfully applied to the verification of software using both static and dynamic analysis [6] [7] [8]. During last decades several techniques have been developed in order to tackle the state explosion problem that limit the application of formal methods. The use of binary decision diagrams allowed the application of model checking to industrial case studies [9]. Bounded model checking [10] introduced the use of SAT solvers in the context of symbolic model checking and provided the basis for its extension to the unbounded case. During last decade increased research and industrial efforts have been spent on the area of Satisfiability Modulo Theories (SMT) [11] trying to improve formal verification tools efficiency exploiting the integration of SAT-based reasoning methods with specific theories F. Ortmeier and P. Daniel (Eds.): SAFECOMP 2012 Workshops, LNCS 7613, pp. 409–416, 2012. c Springer-Verlag Berlin Heidelberg 2012
410
O. Ferrante et al.
obtaining promising results in particular in the field of software model checking. However, the application of such theories to complex industrial embedded systems that usually expose nonlinear dynamics and complex numeric control algorithms is still an open point. In this paper we present a tool called Parallel NuSMV (PNuSMV) that is a modified version of the NuSMV model checker that integrates the ManySAT parallel SAT solver [12]. The PNuSMV is part of the FormalSpecs Verifier framework for the formal verification of Simulink/Stateflow models. The experimental results have been collected using a rich set of industrial use cases and they show the existence of an exponential speedup of the verification time when the bounded model checking technique is used to check model’s invariants. The PNuSMV allows for exploiting last advances in SAT solver techinques with the strength of the FormalSpecs Verifier environment and the NuSMV model checking tool. In this paper we present the application of PNuSMV to a cruise control system and an additional set of logic model of industrial size. The main contributions of the papers are (1) the description of the PNuSMV model checker (2) the description of experimental results that shows the exponential speedup for the verification of invariant properties of industrial sized embedded systems compared to the NuSMV model checker using the bounded-model checking technique and (3) the integration of the tool with the FormalSpecs Verifier framework for the verification of Simulink/Stateflow models and the application to a cruise control case study.
2
FormalSpecs Verifier Verification Framework
The PNuSMV tool is part of the FormalSpecs Verifier (FSV) framework for the verification of discrete systems for the MATLAB Simulink environment [13]. The framework supports several operative modes. In this paper we focus on the capability of verifying properties described as invariant or temporal logic formulae. The FSV tool can be seen as a translator from a Simulink model and specification to the NuSMV tool native language. The transformation process produces a semantically equivalent NuSMV representation of the input model taking into account the non-determinism resolution that may be introduced during the transformation step. In Figure 1 the flow is described in details. As a first step the Simulink textual file is parsed. Then the parsed Simulink model is processed generating a semantically equivalent NuSMV model that is used to generate the concrete NuSMV artifact with a model to text step. The technology used to perform the model transformation step is an internally developed Java embodiment of the OMG Query/View/Transformation (QVT) language called JQVT. The JQVT library aims at providing an industry-level operational implementation of the QVT language. It supports the definition of QVT mappings and the definition of mappings inheritance, disjunction and merging. JQVT allows capturing the mapping relation that links a source model element to a target model element and it supports the resolve and resolveIn operators to retrieve the set of mapping source model elements from a given mapped target model element. JQVT does not support the entire
A NuSMV Extension for the Verification of Complex Embedded Systems
411
Fig. 1. FormalSpecs Verifier transformation flow
QVT specification. However, it has been extensively used as translation infrastructure of different tools for the translation of industry-level sized models [14].
3
Parallel NuSMV Tool
The Parallel NuSMV (PNuSMV) tool is a modification of the open source NuSMV2 symbolic model checker. The structure of the tool is represented in Figure 2.
Fig. 2. Parallel NuSMV layered structure
The top layer is an unmodified version of the NuSMV 2.5.2 open source model checker that is a state-of-the art tool for the formal verification of discrete systems. NuSMV2 supports several verification techniques such as binary decision diagrams and SAT-based bounded model checking using different SAT solvers (such as Minisat [15] and Z-Chaff [16]. A new component (the SAT Solver Controller) has been implemented as intermediate layer between the model checker and the underline SAT solver. The role of the controller is to correctly support the instantiation of (possibly several) SAT solvers providing a common interface to the NuSMV2. Currently we successfully integrated the MiniSat v2.2.0 and the ManySAT 2.0. The latter is the last iteration of the parallel SAT solver that
412
O. Ferrante et al.
won the SAT-Race 2008 and SAT-Competion 2009. The availability of multicore platforms allows for an efficient exploitation of the parallel nature of the solver easily obtaining an average speed up of an order of magnitude for several industrial level models as described in Section 5.
4
Cruise Control Model
To show the performance of the tool using a concrete application we describe a cruise control system modeled using the MATLAB Simulink and translated with the FormalSpecs Verifier tool. Cruise control is the term used to describe a control system that regulates the speed of an automobile. The basic operation of a cruise controller is to sense the speed of the vehicle, compare this speed to a desired reference, and then accelerate or decelerate the car as required. A simple control algorithm for controlling the speed is to use a ”proportional plus integral” feedback based on the error between the current and the desired speed. The model of the truck is based of a force balance for the body, see Figure 3. For a detailed description of the example please refer to [17]. Let v be the speed of the truck, m the total mass , FT the traction force related to the wheels, and Fd the force related to additional elements (such as gravity and aerodynamic drag). The mathematical model
Fig. 3. Cruise control model and Simulink model
of the system is given by the equation mdv/dt = FT − Fd where m = 3450kg and FT is the force of the engine. Force Fd is composed by gravity (Fg ), rolling friction (Fr ) and aerodynamic drag (Fa ) forces. Fg = mg sin(a) with a road slope and g gravitational constant. Rolling friction is Fr = mgCr sign(v) with Cr friction coefficient. Finally Fa = 1/2ρCx A(v + ω)2 where ρ is air density, Cx is the aerodynamic drag coefficient, A is the area of the truck and ω models the wind gusts. In our model the values of the equations parameters are ρ = 1.228kg/m3, Cx = 0.55 and A = 2.4m2 . The control algorithm regulates the traction force FT on the basis of the error e = vREF − v between the desired speed vREF and the current speed v of the car. The algorithm k consits of a proportional-integral control as follows: FT (kT ) = kp e(kT ) + kI h=0 e(hT ) where T = 40ms and kP , kI are the proportional and integral gains. The cruise control system has been modeled in MATLAB Simulink (Fig. 3) implementing a conservative discrete-time abstraction of the plant. The model
A NuSMV Extension for the Verification of Complex Embedded Systems
413
has two inputs of type signed 32 bit integers representing the road slope percentage value and the wind gusts value. The model represents the system in steady state with cruise speed of 22m/s. The specification of the property to be checked has been captured using the FormalSpecs Verifier properties toolbox following the contract-based methodology ([18], [19], [20]) that allows for the specification of requirements in terms of contracts C = (A, G) where A is the assumption and G the guarantee (or promise). Intuitively a contract is a requirement of the form A → G where the promise represents the set of possible system behaviors under the hypothesis that the environment behaves as described in the assumption (i.e. A represents the set of acceptable environments). The contract-based theory and its application to the verification of complex distributed embedded systems is investigated by the authors in the context of the SPRINT EU project [21]. In the case of the cruise control system the assumption has been formalized as the conjunction of the following assertions: the road slope is constrained to assume value in the set {−8, −4, 0, 4, 8}, the wind gusts value is constrained to be in the set {−15, 0, 15} m/s) and its derivative can have a maximum absolute value of 15 m/s (to take into account that the wind gust cannot change its value arbitrarily faster). The contract promise requires that the effective speed must be within 95% and 105% of the reference speed value.
5
Experimental Results
In this section we describe the performance results obtained executing the PNuSMV 1.0 and NuSMV 2.5 tools. The host machines used for the execution of the experiments are an Intel iCore
[email protected] Ghz with 8 GB RAM platform hosting a Linux 64 bit Ubuntu 10.04 operating system (platform A) and an Intel(R) Xeon(R) CPU X5550 @ 2.67GHz with 50 GB RAM platform hosting an Ubuntu Linux 10.04 64 bit operating system (platform B). 5.1
Cruise Control Model
For the cruise control we developed several experiments trying to exercise the verification tools in different ways. The model has been designed to falsify the property previously described after 33 execution steps. The translated NuSMV model has size 88 bits that is small compared to the size of the other models used in another set of experiments (thousands of bits). However, the model represents a good benchmark for automotive applications models that usually contains 32bit signals, complex arithmetic operators and multiple feedback control loops. In addition, the model is hard to verify and the NuSMV tool is not capable of finding a counter-example even after several days of computation and it represents an interesting benchmark for the quantitative evaluation of PNuSMV performance speedup. Bounded Model Checking Verification with Incremental Bounds as a first experiment we execute a bounded model checking verification of the property using different bound lengths. The verification has been performed using the
414
O. Ferrante et al. Table 1. First experiment results Steps
PNuSMV (s)
NuSMV (s)
AV. SP. Plat
8
6.53 ± 0.65
34.47 ± 7.01
5.28
A
15
40.02 ± 5.92
856.41 ± 275.07
21.19
A
20
121.86 ± 119.72 6220.72 ± 2127.13 51.05
B
25
516.89 ± 118.37 49149.79 ± 219.51 95.09
B
33
5446.32 ± 456.08
B
N/A
N/A
check_invar_bmc_inc command with the forward strategy. The average execution time of each set of runs is summarized in Table 1. For each length bound a set of executions have been performed using both NuSMV and PNuSMV and the average value and standard deviation of execution times have been computed. Collected data give us the opportunity to propose some comments. The speedup factor (computed as NuSMV execution time/PNuSMV execution time) is always greater than one and it increases with the dimension of the bound length. This is in line with the ManySAT solver expected performances. In particular we noticed that the PNuSMV fully exploits the available CPUs taking advantage of the parallel nature of the SAT solver. A noticeable drawback of the use of the ManySAT solver is the increasing consumption of memory that, however, did not explode exponentially making the approach usable for industrial applications. Bounded Model Checking Verification with Fixed Bound Value as a second experiment we performed a bounded model checking verification of the property trying to reach its violation. The PNuSMV software found a valid counter example in average time of 5446 seconds (approximately 1 hour and 30 minutes). The same model has been processed using the NuSMV software but a valid counter example has not been found within 11000 minutes (approximately 7 days) of computation (considered as a time-out limit for our experiments). In Figure 4 we represented the execution time for both NuSMV and PNuSMV and we extrapolate an approximate exponential trend function between the number K of BMC step performed and the average execution time. We can notice how the speedup gain is approximately of the form aebK with b 0.16. 5.2
Additional Experiments
The cruise control model is an open-source model developed for the evaluation of the PNuSMV performance for automotive domain applications and we collected promising results in terms of verification speedup. In order to better evaluate the tool performance for a larger set of applications we performed additional experiments using another set of models based on synthetic logic systems. These models in contrast with the cruise control contains mainly logical operators and the reduced number of arithmetic blocks allows for the efficient verification of
A NuSMV Extension for the Verification of Complex Embedded Systems
415
Fig. 4. PNuSMV and NuSMV performance graph Table 2. logic-based tests execution results Model PNuSMV (s) NuSMV (s) AV. SP. I
440 ± 75
3094 ± 925
7
II
639 ± 208
14074 ± 12629
20
thousands of bit sized models in few hours (in contrast with the cruise control that is two order of magnitudes smaller in size but it requires two orders of magnitude more time to verify the property). In our experiments, two models were analyzed both of the size of thousands of bit and we performed a BMC incremental verification of an invariant property falsified in fixed amount of steps using both NuSMV and PNuSMV. The value of the speedup factors are summarized in Table 2 (all experiments have been executed on platform B). For this class of models we noticed a significant variability of the speedup factor. This is in line with the reported behavior of the ManySAT engine w.r.t. the reproducibility of the performances. As a final remark let us observe that the average speedup value grows with a factor of 3 (from 7 to 20) with an increase of the size of the model. This is in line with what we experienced in the first set of experiments.
6
Conclusions and Future Works
In this paper we described PNuSMV a tool that integrates the NuSMV2 open source model checker with the ManySAT 2.0 parallel SAT solver. The experimental results report a promising speedup for both control-based models, such as the cruise control model described in this paper, and logic-based models. Several extensions of this work are possible. At first we aim at supporting different parallel SAT solvers to analyze the performances of the different back-ends. In addition we are working at a deeper integration of the ManySAT solver with the NuSMV engine in order to obtain higher performance gains. Finally, we want to apply the PNuSMV to additional industrial models in order to estimate the performance gain for a broader set of application scenarios.
416
O. Ferrante et al.
Acknowledgment. The authors would like to acknowledge the support of the SPRINT EU project (grant agreement no: 257909).
References 1. Miller, S., Anderson, E., Wagner, L., Whalen, M., Heimdahl, M.: Formal verification of flight critical software. In: Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, pp. 15–18 (2005) 2. Miller, S., Whalen, M., Cofer, D.: Software model checking takes off. Communications of the ACM 53(2), 58–64 (2010) 3. Cimatti, A., Clarke, E., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., Tacchella, A.: NuSMV 2: An OpenSource Tool for Symbolic Model Checking. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 359–364. Springer, Heidelberg (2002) 4. http://sal.csl.sri.com/ 5. http://www.prover.com/ 6. Ball, T., Levin, V., Rajamani, S.K.: A decade of software model checking with slam 7. Ball, T., Cook, B., Levin, V., Rajamani, S.K.: SLAM and Static Driver Verifier: Technology Transfer of Formal Methods inside Microsoft. In: Boiten, E.A., Derrick, J., Smith, G.P. (eds.) IFM 2004. LNCS, vol. 2999, pp. 1–20. Springer, Heidelberg (2004) 8. Godefroid, P.: Compositional dynamic test generation (extended abstract) 9. Burch, J.R., Clarke, E.M., Mcmillan, K.L., Dill, D.L., Hwang, L.J.: Symbolic model checking: 10 20 states and beyond (1990) 10. Biere, A., Cimatti, A., Clarke, E., Zhu, Y.: Symbolic Model Checking without BDDs. In: Cleaveland, W.R. (ed.) TACAS 1999. LNCS, vol. 1579, pp. 193–207. Springer, Heidelberg (1999) 11. Barrett, C., Sebastiani, R., Seshia, S., Tinelli, C.: Satisfiability modulo theories. In: Handbook of Satisfiability, vol. 4 (2009) 12. Hamadi, Y., Sais, L.: Manysat: a parallel sat solver. Journal on Satisfiability, Boolean Modeling and Computation, JSAT (2009) 13. http://www.mathworks.com/products/simulink/ 14. Ferrari, A., Mangeruca, L., Ferrante, O., Mignogna, A.: Desyreml: a sysml profile for heterogeneous embedded systems. In: Embedded Real Time Software and Systems, ERTS (2012) 15. Een, N., S¨ orensson, N.: An extensible sat-solver (ver 1.2) (2003) 16. Herbstritt, M.: zchaff: Modifications and extensions (2001) 17. Murray, R.M., et al.: Feedback Systems An Introduction for Scientists and Engineers. Princenton University Press (2009) 18. Benvenuti, L., Ferrari, A., Mangeruca, L., Mazzi, E., Passerone, R., Sofronis, C.: A contract-based formalism for the specification of heterogeneous systems (invited). In: FDL, pp. 142–147. IEEE (2008) 19. Ferrante, O., Codella, G., Sofronis, C., Mangeruca, L., Ferrari, A.: Verify contractbased designed discrete systems by simulation. In: INCOSE, EuSEC (2010) 20. Ferrante, O., Mignogna, A., Sofronis, C., Mangeruca, L., Ferrari, A.: Contract based design chain integration: An automotive domain case study. In: Applied Simulation and Modelling. ACTA Press (2011) 21. http://www.sprint-iot.eu/
