Region Logic: local reasoning for Java programs and its automation. Stan Rosenberg1 Computer Science Department Stevens Institute of Technology Hoboken, NJ 07030, USA
[email protected] Advisor: David A. Naumann Committee Members: Anindya Banerjee, IMDEA Software, Madrid, Spain Clark W. Barrett, Computer Science Department, New York University David A. Naumann, Computer Science Department, Stevens Insitute of Technology Antonio Nicolosi, Computer Science Department, Stevens Insitute of Technology Charles L. Suffel, Computer Science Department, Stevens Insitute of Technology August 15, 2011
1 Supported by Stanley Fellowship and US NSF awards CNS-0627338, CRI-0708330, CCF0429894.
REGION LOGIC: LOCAL REASONING FOR JAVA PROGRAMS AND ITS AUTOMATION by Stan Rosenberg A DISSERTATION Submitted to the Faculty of the Stevens Institute of Technology in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY
Stan Rosenberg, Candidate ADVISORY COMMITTEE David A. Naumann, Chairman
Date
Anindya Banerjee
Date
Clark W. Barrett
Date
Antonio Nicolosi
Date
Charles L. Suffel
Date
STEVENS INSTITUTE OF TECHNOLOGY Castle Point on Hudson Hoboken, NJ 07030 2011
ii ABSTRACT Shared mutable objects are a cornerstone of the object-oriented paradigm. The ability to share mutable data eliminates unnecessary cloning and gives rise to efficient data structures. Yet, formal reasoning about partial correctness of object-oriented programs is notoriously difficult due to the very same features, viz., sharing and mutable objects. The core problem is aliasing, and one of the contributions of this thesis is a program logic designed to control aliasing through explicit use of effects and disjointedness assertions. We propose a straightforward adaptation of Hoare logic to reason about (sequential) Java programs. The logic employs regions (sets of references) in a novel way, by using them in ghost state, effects and assertions. The aptly named—region logic—embodies “local reasoning” as witnessed by separation logic, without resorting to non-standard semantics or higher-order constructs. Region logic is formalized (and proven sound) with respect to a core subset of Java. Several illustrative examples including subject/observer and composite design patterns are specified and proven partially correct. The assertion language of region logic subsumes boolean algebra and includes (function) image expressions. Full assertion language is quite expressive, e.g., assertions can be quantified, however, its restriction to quantifier-free (QF) assertions yields a decidable theory. Our thesis maintains that the logic is amenable to automation. To that end we implement an automated verifier for region logic; the verifier computes verification conditions which are first-order formulas. The verifier is used to specify and verify a suite of programs including those aforementioned. We also study, i.e., formalize and prove correct, decision procedures for QF assertions. We implement a semi-decision procedure integrated with a (satisfiability modulo theories) solver. To test its feasibility, we compare the implementation with an axiomatization based on quantified formulas; preliminary results are very encouraging. For a restricted language, we give an NP-complete decision procedure and prove its correctness.
iii
In honor of Professor Bloom. His spirit continues to inspire me and remind me that good laughter is one of sufficient conditions for good science.
iv Acknowledgments A great many people have inspired, coached, humored and believed in me throughout my years at Stevens. It is not an exaggeration to say that this thesis would not have been possible without them. Professor Suffel persuaded me to enroll into the Ph.D. program, adding “you have the (knowledge) disease”. Our acquaintance can be traced to lectures on Automata theory which he taught, and which I studied. His lectures utterly fascinated me; it felt like magic was transpiring on the chalkboard except every “trick” was explained with stunning clarity. Many of his lectures would invoke the question “And how do we prove this?”, which as you guessed would be answered “By induction!”. Since that time I learned to appreciate induction and even practiced it in this very thesis. Without Professor Suffel’s encourangement I would most likely not have ended up pursuing Ph.D. The brunt of the coaching work has been endured by none other than my academic father. Without Dave’s guidance this work would not have seen the light of day. I am extremely fortunate to have had Dave as my advisor. He taught me how to think like a scientist! For that and an infinity of other things I shall remain indebted to him. They say that good fortune comes in twos. Anindya has been my teacher and collaborator. He never seemed to mind hearing or reading many of my half-baked ideas; it amazes me how thorough he is. With Anindya time is a mere formality; he never seems to be short on it, at least when it comes to his collaborators and students. Thanks to Anindya I need not use a spellcheker because I can always count on him to check every single word and more importantly every technical detail. (I mean it in the best possible way.) For all these things, I am immensely grateful to Anindya. I thank each and every member of the advisory committee. I am especially fortunate to have Clark who played, and continues to play, a pivotal role in advancing the state-ofthe-art of SMT solving. Clark is also an excellent teacher; I thank him for a wonderful introduction to first-order logic. Antonio’s external expertise in security will undoubtedly shed new light onto this work. With my deepest regrets, Professor Bloom is no longer with us. He was on my advisory committee and also my mentor. His true genius and sense of humor will always remain in my memory. I thank Mike Barnett and Rustan Leino for their help with answering Boogie and Dafny related questions. I thank the developers of Z3, namely Leonardo de Moura and Nikolaj Bjørner. They were extremely instrumental in answering countless of my Z3 related questions and responding with surprising agility to my bug reports. I especially thank Leonardo for providing feedback on Sect. 4.11 and Sect. 4.12 of this thesis. I thank the committee of Stanley Fellowship for awarding me the fellowship for two consecutive years. I thank Professor Duggan for serving as my interim advisor. I would also like to thank my fellow graduate students at Stevens for lively academic as well as social discussions and acitivities: Brian Borowski, Andrey Chudnov, Tang Chunyu, Pablo Garralda, Ye Wu, Zhiqiang Yang. Thanks to Yeting Ge and his (former) advisor Clark for welcoming me at NYU and for answering my questions regarding CVC3. Thanks to Sherry Dorso and Dawn Garcia for much help with administrative issues. I thank my wife who has supported me throughout in every way possible. She always believed in me even during the times when I didn’t.
Contents 1 Introduction 1.1 Overview . . . 1.2 Thesis . . . . . 1.3 Contributions . 1.4 Reader’s Guide
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
10 10 15 16 17
2 Region Logic 2.1 Region Logic in a Composite Nutshell 2.2 Programming Language . . . . . . . . 2.2.1 Syntax . . . . . . . . . . . . . . 2.2.2 Semantics . . . . . . . . . . . . 2.3 Assertion Language . . . . . . . . . . . 2.3.1 Syntax . . . . . . . . . . . . . . 2.3.2 Semantics . . . . . . . . . . . . 2.4 Effects . . . . . . . . . . . . . . . . . . 2.5 Framing and Separators . . . . . . . . 2.5.1 Framing . . . . . . . . . . . . . 2.5.2 Separators . . . . . . . . . . . . 2.5.3 Immunity . . . . . . . . . . . . 2.6 Program Correctness . . . . . . . . . . 2.7 Proof Rules in Action . . . . . . . . . 2.8 Related Work . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
18 18 22 22 25 26 27 28 29 30 30 32 32 33 34 37
3 Verifier for Region Logic 3.1 Background . . . . . . . . . 3.2 Meet Verl . . . . . . . . . . 3.2.1 Programs . . . . . . 3.2.2 Commands . . . . . 3.2.3 Expressions . . . . . 3.2.4 Functions . . . . . . 3.3 Intermediate Representation 3.3.1 Prelude . . . . . . . 3.3.2 Heap . . . . . . . . . 3.3.3 Types . . . . . . . . 3.3.4 Expressions . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
39 39 41 44 47 51 53 56 56 57 59 59
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . . . . . . . . .
. . . .
. . . . . . . . . . .
. . . .
. . . . . . . . . . .
. . . .
. . . . . . . . . . .
. . . .
. . . . . . . . . . . v
. . . . . . . . . . .
vi . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
62 63 65 67 68 70 74
4 Deciding region assertions 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 4.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . 4.2.1 Syntax . . . . . . . . . . . . . . . . . . . . . 4.2.2 Semantics . . . . . . . . . . . . . . . . . . . 4.3 RL-Tableaux Calculus . . . . . . . . . . . . . . . . 4.4 Examples of RL-tableaux . . . . . . . . . . . . . . . 4.4.1 Constraining img-rules . . . . . . . . . . . . 4.5 Correctness of RL-tableaux . . . . . . . . . . . . . 4.5.1 Systematic Procedure . . . . . . . . . . . . 4.6 Towards terminating tableaux: reducing RL to RLE 4.7 Towards terminating tableaux: restricted-RL . . . 4.8 Tableaux Calculus for restricted-RLE . . . . . . . . 4.9 Correctness of restricted-RLE tableaux . . . . . . . 4.10 Complexity of restricted-RL . . . . . . . . . . . . . 4.11 Implementation . . . . . . . . . . . . . . . . . . . . 4.12 Evaluation . . . . . . . . . . . . . . . . . . . . . . . 4.13 Extensions of RL-tableaux . . . . . . . . . . . . . . 4.14 Related Work . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
79 79 81 81 82 86 91 96 97 103 105 110 113 116 124 128 133 136 137
3.4
3.3.5 Functions . . . . . . . . 3.3.6 Commands . . . . . . . 3.3.7 Methods and Effects . . 3.3.8 Loop Effects . . . . . . 3.3.9 Localized Framing . . . 3.3.10 Computing Read Effects Related Work . . . . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
5 Conclusion 143 5.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Bibliography
146
List of Figures 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15
Composite design pattern: specifications and implementation. . . . . . Core programming language. . . . . . . . . . . . . . . . . . . . . . . . Typing rules for commands. . . . . . . . . . . . . . . . . . . . . . . . . Typing rules for expressions. . . . . . . . . . . . . . . . . . . . . . . . . Abbreviations used in semantic definitions. . . . . . . . . . . . . . . . Semantics of region expressions (G). . . . . . . . . . . . . . . . . . . . Region Assertion Language (RAL). . . . . . . . . . . . . . . . . . . . . Typing rules for assertions. . . . . . . . . . . . . . . . . . . . . . . . . Semantics of RAL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selected sub-effect rules. We write ≶ to abbreviate two inclusion rules. Framing of expressions and atomic assertions. . . . . . . . . . . . . . . Inductive definition of the framing judgement. . . . . . . . . . . . . . . Selected correctness rules and axioms for commands. . . . . . . . . . . Selected structural rules. . . . . . . . . . . . . . . . . . . . . . . . . . . Selected derived rules. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
19 22 24 24 25 26 27 28 28 30 31 32 34 35 35
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19
Listing of example1.rl. . . . . . . . . . . . . . . . . . . . . . . . . . Listing of example2.rl. . . . . . . . . . . . . . . . . . . . . . . . . . Example of write and freshness effects in method specification. . . . Example specifying the swinging pivots restriction. . . . . . . . . . . Grammar of Verl statements. . . . . . . . . . . . . . . . . . . . . . . Example illustrating loop invariants and reachability. . . . . . . . . . Code snippet from list.rl. . . . . . . . . . . . . . . . . . . . . . . . Example illustrating why custom loop effects are sometimes needed. Grammar of Verl expressions. . . . . . . . . . . . . . . . . . . . . . Resolution of binary operators for type rgn. . . . . . . . . . . . . . . Using axioms to define a function. . . . . . . . . . . . . . . . . . . . Region axioms declared in the prelude. . . . . . . . . . . . . . . . . . Definedness conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . Well-formedness of Verl expression E . . . . . . . . . . . . . . . . . . Helper functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heap agreement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frame axiom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Read effects of quantifier-free Verl expressions. . . . . . . . . . . . . Read effects of quantified Verl expressions. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
42 43 46 47 48 49 49 51 52 53 55 61 62 63 66 69 69 70 71
vii
. . . . . . . . . . . . . . . . . . .
viii 3.20 Cell specified in VeriCool. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.21 Cell specified in Verl. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.22 Cell specified in Dafny. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15
RL-tableaux rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Remaining RL-tableaux rules. . . . . . . . . . . . . . . . . . . . . . . . . RL-tableau proving r ∩ (s − t) = (r ∩ s) − (r ∩ t) is valid. . . . . . . . . RL-tableau proving (r ∪ s)‘f = r ‘f ∪ s ‘f is valid. . . . . . . . . . . . . . Left: RL-tableau for u ∈ r ∩ s ∧ u 6∈ s. Right: RL-tableau for u ∈ r ∩ s. Left: RL-tableau for u ∈ r ∧ r = r ‘f . Right: RL-tableau for r ( r ‘f . . . Systematic procedure for Φ. . . . . . . . . . . . . . . . . . . . . . . . . . Definition of the reduction function ρe : RL → RLE . . . . . . . . . . . . . restricted-RLE tableaux rules. . . . . . . . . . . . . . . . . . . . . . . . . img rules for restricted-RLE tableaux. . . . . . . . . . . . . . . . . . . . . Examples of restricted-RLE tableaux. . . . . . . . . . . . . . . . . . . . . Non-determinstic decision procedure. . . . . . . . . . . . . . . . . . . . . Valid RL benchmarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Invalid RL benchmarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . Tableau rules for read2 and img2. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
76 77 78 87 88 92 93 94 95 104 107 114 115 116 127 134 135 137
Chapter 1
Introduction The most important property of a program is whether it accomplishes the intentions of its user. An axiomatic basis for computer programming C. A. R. Hoare
We begin with a brief overview of formal methods. Then, we focus on the problem of formally reasoning about functional correctness of Java-like programs. Our starting point is Hoare logic. We illustrate why the classical Assignment rule is unsound in the presence of heap updates. We introduce the notion of “local reasoning” which is at the core of region logic. Subsequently, we present the main thesis of this work and describe our approach to validate it. We conclude with an outline for the rest of the document.
1.1
Overview
Impetus. Software is infested with bugs—pesky human errors, especially in a form of flawed mathematical reasoning, which often times may have grave consequences. Over the years, software engineering methods have been improving, witnessed by safer mainstream general-purpose programming languages, e.g., Java, augmented with static analyzers, e.g., FindBugs [AHM+ 08], Jlint [Art01]. However, one element remains unchanged—software is implemented by human beings, more often than not, relying on their intuition rather than formal methods. Formal methods, in this context and onwards, refers to an approach that uses mathematical logic to express specifications of programs’ intended behavior and to reason about, i.e., prove, their correctness. This methodology had been in (mostly academic) use for several decades, but it has been receiving more widespread attention recently; e.g., Jones, O’Hearn, Woodcock discuss Hoare’s grand verification challenge [JOW06], Jackson discusses creating (mathematical) models of software and automatically checking them [Jac06]. From finding bugs to proving their absence. Documentation and debugging should be inextricable ingredients of any serious software development process. However, docu10
11 mentation tends to be imprecise (typically being expressed in an informal language), and debugging can help determine only some of the pre-existing faults, and even introduce new ones (e.g., inside debug code). Nevertheless, documentation serves to specify a program’s behavior, and debugging is but an unsophisticated attempt at verifying that the program behaves as specified. There are of course more sophisticated verification techniques, and we briefly review some of them. Note, successful verification establishes correctness with respect to specification(s). It is precisely this notion of correctness that we are after in this work. Software testing has been practiced for generations. It has become more sophisticated over the years, and disciplines like regression and unit testing are the standard weapons in quality assurance engineer’s (i.e., tester’s) arsenal. To be effective, testing typically requires “good coverage”—sufficient number of test cases to cover all possible input. (Of course, for all intents and purposes, input is infinite.) Testing may also employ runtime assertions—conditions which must hold during execution. For example, assert x >= 0 would raise an error in those executions where x is negative. However, the dictum of Edsger W. Dijkstra [EWD303]: program testing can be used very effectively to show the presence of bugs but never to show their absence remains true to this day; the infinite number of possible states (due to input) renders exhaustive testing generally infeasible. Consequently, testing can be seen, informally, as showing probable correctness; e.g., a program is 20% correct if the test cases cover this percentage of all possible states. (Of course, in general coverage may be difficult to quantify.) Furthermore, specifications from which test cases are generated are typically informal and therefore potentially ambiguous. Owing much to the great success of a class of automated theorem provers known as SMT (satisfiability modulo theories) solvers, state-of-the-art testing techniques can achieve far greater coverage. For example, recent advances in testing employ white-box fuzz testing; while traditional fuzz testing attempts to randomly mutate well-formed input in order to generate new test input, white-box fuzz testing employs symbolic execution and constraint solving in order to achieve high coverage by exercising only feasible paths. A program is typically executed on a given input; symbolic path constraints are learned; new test input is generated by negating a path constraint and solving it for satisfiability. Two exemplary tools that perform white-box fuzz testing are SAGE [GLM08] and Pex [dHT08]. SAGE targets x86 binary code and uses bit-precise arithmetic reasoning; it has been used to detect several bugs in large software packages (e.g., media players that shipped with Windows 7). Pex works on .NET code; it is used primarily for generating parameterized unit tests. The tool relies on Z3 [dMB08b] (SMT solver) to reason about path constraints. (Bjørner and de Moura wrote a great paper [dMB09b] discussing many applications of SMT solvers, including the above.) We note that testing is usually a combination of static (compile-time) and dynamic (run-time) techniques. In this work, we shall focus on static (formal) techniques only. Program analysis. While testing cannot in general prove the absence of errors, program analysis can prove the absence of some categories of errors. Type checking ensures
12 that all expressions and statements evaluate to the correct type, thereby establishing type safety—any evaluation of a well-typed program during execution will not result in an error due to operands having incompatible types. Some examples of type errors in Java are: casting an integer expression into a reference expression, performing arithmetic on a reference, accessing an undefined field, etc. (Java’s extended type system, i.e., javac also checks for undefinedness errors due to un-initialization—no variable can be used before its initialization.) Although type safety guarantees that there are no field undefined errors, the statement x.f = y may be a different source of error—null dereference—one that is difficult to check statically since it depends on the program context, and assumptions about the input. Type checking as described is a form of context-free program analysis. More sophisticated type checkers or static analyzers perform a context-sensitive program analysis. Some of the typical properties they attempt to verify include the absence of null dereference [LYC+ 08, HP07] and whether x may/must alias y [FYD+ 08, SB06]. Due to undecidability such analyses typically act conservatively—over-approximating the results to stay tractable yet sound. In some cases, additional type annotations may improve the results of an analysis; e.g., a NotNull type annotation can help prove the absence of null dereference [FL03]. (Formal) Program verification. Program verification is concerned with verifying that a program is correct with respect to formal specification(s), generally known as functional correctness. Herein, specifications (specs) are expressed in first-order logic. The specs are further classified into pre- and post-conditions. A precondition states the (input) assumptions—what logical assertions must hold before a program is executed, i.e., in initial state; postcondition states the (output) assertions—what assertions must hold after the program execution, i.e., in final state. Since a program is typically decomposed into modules (methods in Java), functional correctness of the entire program is reduced to functional correctness of each module. The specs serve as a formal contract between a caller and a callee: the former establishes the precondition, the latter establishes the postcondition. To verify that a program is partially correct means to prove that for any initial state that satisfies the precondition, if the program terminates, the final (i.e., terminal) state must satisfy the postcondition, and it must be an error-free state. (The latter condition is known as error-avoiding semantics; by error-free we mean a state not due to null dereference or any other avoidable error.) An executing program may fail to terminate for a host of reasons, be it an infinite loop or an implementation-dependent reason, e.g., out of resources, hardware failure, etc. When functional termination, i.e., loop or function termination, can be proven, the program is said to be totally correct. Neither total correctness nor partial correctness is decidable. (The former can be reduced to the Halting problem; the latter is due to undecidability of first-order logic.) Of course undecidability does not preclude development of formal systems to reason about correctness. Furthermore, great strides have been made towards deciding correctness on both fronts. In this thesis, we restrict ourselves to proving partial correctness. A classical correctness statement assumes the form of { P } C { Q }, where P , C , Q are precondition, command and postcondition, respectively. (For historical reasons, a correctness statement is also known as a Hoare-triple; P , Q are assertions or state predicates; command is to mean any program statement(s), including the whole program.)
13 Axiomatic reasoning. We assume the reader is familiar with Hoare logic [Hoa69] and refer to the following foundational rules: Assignment, Sequence and Consequence, { P /x →E } x := E { P } P ⇒ P0
{ P } C1 { Q }
{ Q } C2 { R }
{ P } C1 ; C2 { R } { P0 } C { Q0 }
Q0 ⇒ Q
{P } C {Q } Consider the following (simple) correctness statement: { x + 1 > 0 ∧ y < 0 } x := x + 1 { x > 0 ∧ y < 0 }
(1.1)
It is derivable using the Assignment rule: substitute x + 1 for x in the postcondition. There is, however, another means to derive the above correctness statement, using Hoare’s Invariance rule [Apt81] (cf. Reynold’s Constancy rule [Rey81]): {P } C {Q }
C does not modify FV(R)
{P ∧ R} C {Q ∧ R} Observe, the antecedent has a side condition: C does not modify any free variables of R. More generally, the Invariance rule allows one to conjoin any state predicate R over an existing triple—provided R does not “interfere” with C . In simple imperative languages, “no interference” can be expressed with the above side condition, i.e., checking free variables. Let’s see how we can use Invariance for simple imperative programs. We can derive (1.1) in two steps: 1. { x + 1 > 0 } x := x + 1 { x > 0 } by Assignment 2. { x + 1 > 0 ∧ y < 0 } x := x + 1 { x > 0 ∧ y < 0 } by Invariance, conjoining y < 0 to pre- and postconditions. (Note, the side condition is satisfied: y is not modified by the command.) The derivation using Invariance takes two steps, and on that basis alone, it may appear less useful than the above one step using Assignment. Enter shared mutable objects. Shared mutable objects. The previous example considered an update to a variable, i.e., local store. Updates to shared objects present a reasoning challenge. Let us change the previous example in (1.1) to use (Java) fields: { z + 1 > 0 ∧ y.f < 0 } x .f := z + 1 { x .f > 0 ∧ y.f < 0 } where x .f , y.f refer to integer-valued fields. Note that in accordance with the Assignment rule, the precondition was obtained by substituting x .f by z + 1 in the postcondition. However, the above correctness statement is patently invalid. There exists an initial state such that: x = y and the precondition are satisfied, thereby resulting in the final state which does not satisfy the postcondition, viz., there is no integer such that x .f > 0 ∧ x .f < 0. The
14 problem is well-known, viz. aliasing—x and y may point to the same object. (Incidentally, the problem of computing precise alias information is undecidable [Ram94].) To cope with aliasing, Assignment can be adapted by Morris’ component substitution [Mor82]. Another approach is to represent the object heap using “component-asarray” model [Bur72] wherein fields are represented by one-dimensional arrays. E.g., the field update x .f := z + 1 is encoded by the array update f := f | x → z + 1. Using the component-as-array encoding of the heap, we would obtain the resulting correctness statement: { f 0 [x ] > 0 ∧ f 0 [y] < 0 } f := f 0 { f [x ] > 0 ∧ f [y] < 0 } where f 0 = b f | x → z + 1), i.e., f updated at x with z + 1. Both approaches restore the soundness of Assignment but expose the global nature of the heap—update of x .f may potentially invalidate all other (logical) assertions that depend on E .f (where E is any arbitrary reference expression, e.g., y.g.h). In general, many case splits may be required to ascertain whether or not E .f was influenced by any update. (In fact, in worst-case the number of case splits is exponential in the size of E .) Local reasoning. As explained above, an explicit heap representation makes reasoning rather hard (at least for mere mortals). There is, however, a more elegant means to reason about the heap. The key idea is to derive only “local” facts about updates, i.e., what is directly changed, and use a suitable adapation of Invariance rule to recover “global” facts, i.e., what is independent of the updates. The term “local reasoning”—coined by O’Hearn, et al. [ORY01]—denotes this paradigm. Informally, Invariance is now stated like so, {P } C {Q }
C does not interfere with R
{P ∧ R} C {Q ∧ R} A local incarnation of Assignment would derive the following: { x 6= null } x .f := z + 1 { x .f = z + 1 } (In observance of error-avoiding semantics, we need x 6= null in the precondition.) Next, we would like to use the above Invariance to conjoin y.f < 0, but observe that the informal rule cannot be invoked because its side condition is not satisfied: y.f inteferes with x .f . However, if x and y were known to be disjoint (i.e., x 6= y), then it would be sound to conjoin any assertion about y.f to the pre- and postconditions of x .f := z + 1. Thus, the above Invariance rule can be formulated so that { x 6= null ∧ x 6= y ∧ y.f < 0 } x .f := z + 1 { x .f = z + 1 ∧ y.f < 0 } is derivable. (Subsequently, conjoin z +1 > 0 and use Consequence to show x .f > 0 in the postcondition.) But what about the general case? A sequence (of commands) may update an arbitrary pool of objects. (For example, consider a simple loop which traverses a linkedlist.) Moreover, an assertion may also depend on any pool of objects; e.g., the assertion of the form ∀x : K · x .f 6= null ⇒ . . . depends on all objects of type K whose f field is non-null, and possibly more. Suppose we can express the following: (a) pools of objects,
15 (b) effects of commands, (c) effects of assertions, (d) separation (i.e., non-interference) of command effects from assertion effects. Then, we can formulate Invariance for shared mutable objects!
1.2
Thesis
Region logic has all the above. Pools of objects are expressible using region expressions which are essentially boolean algebra with (function) image expressions; effects of commands and assertions are expressible using state-dependent region expressions; separation is computable and yields (set) disjointedness assertions. All this machinery comes together in the formulation of Invariance which is known as the Frame rule in region logic. Our work is roughly partitioned into two parts: formal methodology and automated verification. Our thesis maintains that region logic [BNR08b] is not only an effective formal methodology but can also be reasonably automated. Region logic yields an effective formal methodology for reasoning about Java-like programs. Region logic is amenable to automation in an SMT-based framework. Formal methodology. We have formalized region logic and proved it sound [BNR08b]. Several features of region logic make it an effective formal methodology. The assertion language is simple yet expressive. The proof rules consist of “local axioms” which are empowered by the Frame rule; the semantics of proof rules is standard. We used the logic to specify and to prove several benchmark problems: list copy, list reverse, subject/observer, including a recent challenge problem: composite [LLM07]. Automated verification. We have implemented a verifier for region logic, Verl. The verifier sits atop Boogie2, Z3 toolchain. A program specified in region logic is translated to Boogie2 [Lei10b] which then computes efficient verification conditions (VCs) based on weakest-preconditions. These verification conditions are first-order formulas which serve as input into an SMT solver, typically Z3 [dMB08b] but other solvers permitting, e.g., CVC3 [BT07]. A novel feature of Verl is the use of C preserves P annotations, where C is a code block and P is any assertion. Intuitively, these annotations assert that P is invariant of C and instruct Verl to appeal to local reasoning to establish P . These annotations were inspired while trying to verify the Composite design pattern. Other verifiers [SJPS08, Lei08] rely on frame axioms which in our experience were not sufficient to verify Composite. Owing to a great many advances in SMT solving, verification conditions can be often times validated in a fully automatic way. VCs typically consist of formulas combining different theories, e.g., theory of equality, theory of arrays, theory of partial orders, and in our case, theory of regions. What makes an SMT solver rather attractive over other automated theorem provers (ATPs), e.g., resolution-style provers, is its remarkable ability to combine efficient decision procedures for respective theories into a decision procedure for the union theory.
16 To decide quantifier-free theory of regions, we developed tableau-based decision procedures that seem to integrate well into an SMT-based framework. We proved these decision procedures sound and complete. We also implemented a semi-decision procedure as a theory extension in Z3. A preliminary evaluation with respect to synthetic benchmarks yielded encouraging results.
1.3
Contributions
In this section we highlight key contributions of the thesis. We establish region logic—a formal methodology to reason about partial correctness of sequential Java-like programs. We apply region logic to specify and prove correct several benchmark programs including a recent challenge problem. We prove region logic sound. We implement Verl, the first automated verifier for region logic. We use Verl to specify and verify the following benchmark problems: list copy, list reverse, subject/observer, composite. The distribution [Verl] contains the binaries, as well as the sources and build instructions, and verified examples: list.rl, subject observer.rl, composite.rl, composite clients.rl. We formalize a semi-decision procedure for quantifier-free region assertions as well as a decision procedure for a restricted language. We prove both procedures sound and complete; the decision procedure has been shown NP-complete. The semi-decision procedure is integrated with an SMT solver; it is implemented as a theory extension in Z3. A preliminary evaluation compares the semi-decision procedure with an axiomatic encoding based on quantifiers, using synthetic benchmarks. In summary, this work makes the following contributions: Region Logic. In a conference paper [BNR08b] we introduce region logic and prove it sound with respect to a core subset of Java. Our formalization is based on a denotational semantics of CoreJML [LNR06] which we mechanize in PVS, a higher-order logic theorem prover. Feasibility study. We first verify (by hand) several programs, e.g., subject/observer in [BNR08b], serving to illustrate feasibility of local reasoning as embodied by region logic. Subsequently, in a conference paper [RBN10] we introduce a solution for the Composite challenge problem based on region logic. In that paper [RBN10] we specify a representative implementation of the Composite design patterns and several client programs. We then verify the specifications using an automated verifier Verl whose first release accompanies the paper. Decision procedures. We formalize decision procedures for quantifier-free region assertions. We also implement a semi-decision procedure integrated with Z3 and conduct a preliminary performance evaluation. Automated Verifier. We implement Verl, the first automated verifier for region logic. We use Verl to specify and verify several benchmark programs.
17 Other work. Our preliminary work in information-flow control served as impetus for region logic. Although this work is not part of the thesis, previously, we investigated how to use relational specs to specify and verify expressive declassification policies [BNR07]. Subsequently, we formalized the end-to-end semantics of such relational specifications [BNR08a].
1.4
Reader’s Guide
In Chapter 2 we describe core features of region logic. We begin with an example from our recent work [RBN10] which is concerned with the specification and verification of the Composite design pattern. Subsequently, we formalize: region assertions, effects, and a subset of the proof rules. To illustrate the proof rules, we study a small example program and prove its correctness. We conclude the chapter with related work. In Chapter 3 we delve into the details of the verifier. Sect. 3.1 reviews weakestpreconditions and verification conditions. Sect. 3.2 examines the syntax of Verl accompanied by several examples which illustrate Verl’s features. Sect. 3.3 describes the translation of Verl to Boogie2. Sect. 3.3.9 and Sect. 3.3.10 describe how C preserves P annotations are translated and how read effects of expressions are automatically computed. In, Sect. 3.4 we study related work. In Chapter 4 we study decision procedures for region assertions. We investigate two tableau-based procedures for deciding quantifier-free region assertions. We prove both procedures sound and complete. The procedure for the full language is non-terminating, and hence it is only a semi-decision procedure. For a restricted language, we exhibit a (terminating) decision procedure and show that it is NP-complete. We describe a prototype implementation of the semi-decision procedure which is integrated with an SMT solver, namely Z3. The complete outline of Ch. 4 is given at the end of Sect. 4.1. Chapter 5 concludes this thesis with a discussion of results and future work.
Chapter 2
Region Logic We begin our introduction to region logic by considering a simple illustrative implementation of the Composite design pattern accompanied by specifications in region logic. We introduce key features of region logic as we explain the specifications. Composite is challenging to specify (and verify); it has been posed as a challenge problem in [ea08]. We studied Composite in [RBN10] using region logic and gave a solution which was specified and verified using Verl. The complete specifications suitable for automated verification is detailed in [RBN10], and the source files are available in [Verl]. After the informal introduction we proceed to formalize key features of region logic. Sect. 2.2 and Sect. 2.3 detail the programming language and the assertion language for which region logic has been formalized. Sect. 2.4 and Sect. 2.5.1 formalize effects and framing, i.e., derivation of read effects and computation of certain disjointedness assertions. Sect. 2.7 walks through a correctness proof of a tiny program. Sect. 2.8 describes related work. For complete formal details, e.g., proof of soundness, we refer the reader to [BNR08b] and an upcoming journal version [BNR11] which generalizes and simplifies some of the features.
2.1
Region Logic in a Composite Nutshell
Implementation. Fig. 2.1 depicts a simple implementation and specifications of the Composite design pattern. The pattern centers on a collection of mutable data objects organized as a tree. Class Comp is used to represent leaf nodes as well as internal nodes of the tree. Field parent contains an immediate ancestor (if any) of the current object (self ) and field total contains a count of all descendants including self . Field children is a sequence of objects. We use a mathematical sequence for simplicity, to avoid the distraction of heap-allocated arrays. Addition of an element to a sequence is performed using the + operation. Sequence membership is written as o ∈ p.children, which is a shorthand for ∃i : int · 0 ≤ i < len(p.children) ∧ o = p.children[i ]. Specification of add . Public method add inserts an existing composite into the children of self and then invokes private method addToTotal which repairs the total of self and all of its ancestors (if any). As usual, requires and ensures clauses express pre- and postconditions. The
18
19
ok (o) :
o.total = 1 + (sum i ; 0 ≤ i < len(o.children) | o.children[i ].total )
I0 : I 1(r : rgn) : I2 : I3 : I (r : rgn) :
∀o ∀o ∀p ∀o
: Comp · o.total ≥ 1 : Comp ∈ alloc − r · ok (o) : Comp, o : Comp · o ∈ p.children ⇔ o.parent = p : Comp, i : int, j : int · 0 ≤ i < j < len(o.children) ⇒ o.children[i ] 6= o.children[j ] I 0 ∧ I 1(r ) ∧ I 2 ∧ I 3 and I = b I (emp)
public class Comp { seq children; int total ; // initially total = 1 and children is empty sequence Comp parent; // initially parent = null
void add(Comp c) requires c 6= null ∧ c.parent = null; requires self 6= c ∧ I; ensures c.parent = self ∧ I; effects wr {c}‘parent, {self}‘children; effects wr alloc‘total; { c.parent := self; self.children := [c] + self.children; self.addToTotal(c.total); } int getTotal() requires I; ensures result = self.total ∧ I; { result := self.total; }
void addToTotal(int t) requires t ≥ 1; requires self.total + t = 1 + sum i; 0 ≤ i < len(self.children) | self.children[i].total; requires I({self}); ensures I; effects wr alloc‘total; { Comp p; p := self; while (p 6= null) { p.total := p.total + t; p := p.parent; } }
Figure 2.1: Composite design pattern: specifications and implementation.
20 effects clause expresses write effects, that is, what fields (of objects in add ’s pre-state, i.e., state which satisfies the preconditions) may be written. We list write effects following keyword wr. Write effects are also known as frame condition typically specified using a modifies clause. We prefer the former terminology because it fits nicely with read effects. Read effects are the dual of write effects. Whereas write effects express a footprint of a command, read effects express a frame of an assertion, that is, variables and fields whose modification may cause a change in the assertion’s denotation. A read effect rd G ‘f of an assertion says that the meaning of the assertion can vary with updates to f fields of G-objects, i.e., it depends on those fields. A region is a set of allocated references of any type; region expressions, G, have type rgn and can occur in assertions, effects and ghost updates. We allow a Java program to declare variables (and fields) of type rgn and be annotated with ghost updates of type rgn disjoint from all other types. (Technically, our programs here and throughout use a concrete syntax which differs from Java; e.g., we use := instead of = for updates. Since such differences are only syntactic and not semantic we shall refer to these programs as Java or Java-like programs.) As standard, a ghost field/variable cannot be assigned to a program field/variable; ghosts cannot be used inside guards of if or while, and therefore they cannot influence control-flow. The region expression emp denotes the empty region, whereas alloc denotes the set of all allocated references. Singleton regions are constructed with {E }; if the expression E denotes an allocated reference, then {E } denotes a singleton set containing the reference; otherwise, {E } denotes the empty set. Region expressions of the form G ‘f (read “G’s image under f ”) when used for their r -values are restricted to fields f of reference type or of type rgn. If f is a field of reference type then the r -value of G ‘f is the set of allocated references v such that v = o.f for some o ∈ G. However, if f : rgn then the r -value of G ‘f denotes the union of the f -images. In effects, a use of G ‘f refers to its l -value, and then f can have any type (cf. wr {c}‘parent, wr alloc‘total in Fig. 2.1 ). The assertions G ⊆ G 0 and G # G 0 say, respectively, that region G is a subset of G 0 and G ∩G 0 ⊆ emp. In particular, G ‘f ⊆ G says that G is closed under f . Assertions of the form G ‘f ⊆ G (read: “G is f -closed”) is a distinguishing feature of region logic—f -closure is a form of reachability. The assertion G ‘f ⊆ G says that if a reference, o, belongs to G, then everything reachable from o by following f must be in G, i.e., if p = o.f . . . f and p 6= null, then p ∈ G.) More concretely, the assertion root ∈ G ∧ G ‘f ⊆ G says: all (allocated) nodes reachable from root by iterating f are contained in G. Of course, G may not be the smallest such region, i.e., it may contain nodes unreachable from root. In quantified assertions such as I 0 (see Fig. 2.1), the bound variable ranges over allocated (i.e., non-null) references only. The default range is alloc, i.e., the region of all allocated references, but any smaller range can be specified, i.e., by a region expression, as a bound. (The semantics is instrumented so that newly allocated objects are automatically added to alloc. A command that allocates must report effect wr alloc.) Thus in I 2, both p and o range over alloc whereas the o in I 1(r ) ranges over all objects in the region alloc−r . Here ’−’ denotes set subtraction. Write I 1 for I 1(emp) and I for I (emp). Leaving aside condition I , the specification of add says: given an initial state where c is an allocated component distinct from self and has no ancestors (c.parent = null), a final state is one in which c’s parent is self . Furthermore the following updates (but no
21 other) are licensed by the write effects: the parent field of c, the children field of self , and the total field of any allocated component. Condition I is intended to be invariant in the sense that it holds in all client-visible states; so it appears as both pre- and postcondition of add and getTotal . (Region logic does not adhere to any specific invariant methodology. For examples of invariant methodologies used in related work see Sect. 2.8.) The conjunct I 0 says every component’s total is positive; I 1(r ) says every component except those in r has as total one more than the sum of its children’s total ; I 2 says that p is o’s unique parent iff o is p’s child; I 3 says that children does not contain any duplicates. In conjunction with the invariant, I , the specification of add says that c was added to children of self : initially, c.parent = null and I 2 together entail c 6∈ self .children; finally, c.parent = self and I 2 together entail c ∈ self .children. Proof system by example. The proof system of region logic features “local rules” empowered by the Frame rule which we shall see soon. Let’s consider proving a part of the specification needed in the proof of add , in particular, establishing the assertion I 1({self }) which is required, as a conjunct of I ({self }), immediately before the invocation of addToTotal . From the local specification (“small axiom”) for c.parent := self we obtain, { c 6= null } c.parent := self { c.parent = self } [wr {c}‘parent] It says: if c is allocated in the pre-state, then in the post-state, c.parent = self and the write effect {c}‘parent licenses the update. (The above is an instance of FieldUpd rule in Sect. 2.6.) Observe locality at play: the rule refers only to the immediate state of the assignment at hand: c, self and {c}‘parent; the write effect specifies only the location which is pertinent to the field update. Intuitively, one can deduce that an assertion that does not “depend” on the write effect must be preserved by the field update: in this case, the truth of I 1 is unaffected by the update of c.parent because I 1 does not read the parent field of any object in alloc. Consequently, we can conjoin I 1 to the pre- and postconditions. Then by the standard rule of Consequence we can weaken the postcondition to obtain I 1({self }). For the next command that updates self .children, I 1({self }) holds in the preand postcondition because the write effect is {self }‘children, whereas I 1({self }) reads the children field of all objects in alloc except self . The above informal discourse is justified by the Frame rule of region logic, Frame
` { P } C { P 0 } [ε]
P ` δ frm Q
P ⇒ δ ·/. ε
0
` { P ∧ Q } C { P ∧ Q } [ε] read: Q is preserved by C under precondition P if ε, the write effects of C , is separate from δ, the read effects of Q. The framing judgement P ` δ frm Q asserts δ are at least the read effects of Q. (We use syntax-driven analysis for read effects of atomic assertions, and an inductive definition of this judgement for all other formulas, see Sect. 2.5.1.) The antecedent P ⇒ δ ·/. ε asserts that the precondition may be assumed to prove that the read effects are separate from the write effects. We call δ ·/. ε a separator. The function ·/. computes a conjunction R of disjointedness formulas such that in R-states, writes allowed by ε cannot falsify a formula framed by δ. (Framing judgements in conjunction with separators
22 x , y, r ∈ VarName
f , g ∈ FieldName
K ∈ DeclaredClassNames
T ::= bool | K | rgn E ::= x | null | true | false | E = E G ::= x | {E } | G ‘f | emp | alloc | G ⊗ G
where ⊗ is in {∪, ∩, −}
F ::= E | G C ::= x := F | x := new K | x := y.f | x .f := F | if x then C else C | while x do C | C ; C | var x : T in C end
Figure 2.2: Core programming language. formalize the notion of (in)dependence—whether or not an assertion may depend on write effects.) Above, the read effects of I 1 can be derived to be rd alloc, alloc‘children, alloc‘total . Note, they do not refer to parent, hence they are separated from wr {c}‘parent; thus I 1 can be conjoined by Frame. (We needed to discharge rd alloc, alloc‘children, alloc‘total ·/. wr {c}‘parent which evaluates to true.) Read effects of I 1({self }) are rd alloc, self , (alloc − {self })‘children, (alloc − {self })‘total These are separated from wr {self }‘children because alloc − {self } is disjoint from {self }. So I 1({self }) can be conjoined by Frame. (We needed to discharge rd alloc, self , (alloc − {self })‘children, (alloc − {self })‘total ·/. wr {self }‘children which also evaluates to true.)
2.2
Programming Language
We formalize the syntax and the semantics of a core (sequential) programming language which can be seen as a subset of Java; it contains enough constructs to express many interesting programs but shuns many advanced features of Java in order to keep formalization tractable. The programming language is based on CoreJML—a more ambitious subset of Java which includes exceptions, interfaces, visibility, etc.—previously formalized in a higher-order theorem prover [LNR06].
2.2.1
Syntax
A program consists of a command C in the context of some class declarations. The grammar for commands and expressions is in Fig. 2.2. A class declaration class K { T f } introduces a reference type (named) K ; values of this type are null and references to mutable objects with typed fields f : T . (Here and throughout, identifiers with an overline range over lists. Technically, null is an expression in the language whereas nil is a semantic value denoted by that expression. For convenience, we may tacitly use null for both, when the context should suffice to disambiguate this use.)
23 In addition to bool and reference types, there is a distinguished type rgn whose values range over finite sets of allocated references of any type, i.e., excluding null. (Intuitively, rgn is like a Java collection of type Set.) Region expressions (G in Fig. 2.2) include standard set operations, e.g., set difference denoted by (−), as well as {E } which denotes a singleton region when E is non-null and the empty region otherwise, G ‘f which denotes a region obtained by taking the image of f under G. (In case of G ‘f , f must be a reference or a region field.) Region expressions cannot influence control flow or the value of non-region fields/variables, and as such they can only serve as ghosts or auxiliaries (i.e., those that instrument the program, or sometimes just its assertions) to facilitate reasoning. Such use of auxiliary state dates back to the 1970s [Rey81, OG76]. We do not distinguish in our syntax between ghost fields (and variables), and ordinary ones (cf., JML uses special comment notation [LPC+ 08]; e.g., //@ ghost rgn nodes; declares a ghost variable/field in JML). Ordinary, i.e., Java-like, expressions (E in Fig. 2.2) do not depend on the heap: y.f is not an expression but rather part of the primitive command x := y.f (field access) for reading a field, or y.f := x (field update) for updating a field. Furthermore, only a single field dereference as in y.f is admitted by the grammar. (Thus, y.f .g := x is not a well-formed command.) The restrictions were judiciously chosen to simplify proof rules for reasoning about updates. Notice, the restriction of a single field dereference is by no means a handicap of the programming language. A source-to-source translator could readily rewrite Java code y := x .f .g into (semantically equivalent) code z := x .f ; y := z .g, where z is fresh. (A similar source-to-source translation is used by Parkinson to obtain a core programming language, called Inner MJ, used to formalize separation logic [Par05, Fig. 3.2].) The programming language in Fig. 2.2 is almost identical to the one in [BNR08b]. However, there are a couple of notable differences. For ordinary expressions, we have replaced the int type with the bool type. (This streamlining was performed in order to keep the expressions consistent with the assertion language for which we study decision procedures in Ch. 4.) For region expressions, we permit G ‘f so that x := G ‘f is an admissable assignment command. Typing. A program is typed relative to an ambient class table which comprises a wellformed collection of class declarations. There is no general (class) inheritance mechanism. However, a well-formed class table contains a class declaration for the distinguished name Object. The subtype relation ≤ is the smallest reflexive relation such that K ≤ Object for all class names K . (We also lift ≤ to bool and rgn, so that bool ≤ bool, rgn ≤ rgn in order to simplify typing rules.) We write fields(K ) for the field declarations f : T of class K . We assume that field names are globally unique so that type checking of G ‘f yields no ambiguity. Thus, f : T is to mean that there is some unique class K which declares the field f of type T . A context, Γ, assigns types to variable names. Every well-formed context Γ must at least have the mapping Γ(alloc) = rgn. The judgement Γ ` F : T says that region or ordinary expression F has type T in context Γ. (When the context is clear, we shall abbreviate as F : T .) Similarly, Γ ` C says that C is a well-formed command in context Γ. The typing rules for commands and expressions are given in Fig. 2.3 and Fig. 2.4;
24
Γ`F: T
K ≤ Γ(x ) Γ ` x := new K
Γ ` x := F F : T0
(f : T ) ∈ fields(K )
x: K
T ≤ Γ(x )
T0 ≤ T
Γ ` x .f := F (f : T ) ∈ fields(K )
y: K
T ≤ Γ(x )
Γ, x : T ` C
Γ ` x := y.f Γ ` C1
Γ ` var x : T in C end
Γ ` E : bool
Γ ` C2
Γ ` C1 ; C2
Γ ` C1
Γ ` C2
Γ ` E : bool
Γ ` if E then C1 else C2
Γ`C
Γ ` while E do C
Figure 2.3: Typing rules for commands.
Γ(x ) = T
T is in {bool, K , rgn} Γ`x: T
Γ ` null : K Γ ` E1 : T
Γ ` true : bool
Γ ` emp : rgn
Γ ` E2 : T 0
Γ ` false : bool
Γ ` alloc : rgn
T ≤ T 0 or T 0 ≤ T
T , T 0 is in {bool, K }
Γ ` E1 = E2 : bool Γ ` G : rgn
(f : T ) ∈ fields(K )
T is in {K 0 , rgn}
Γ ` G ‘f : rgn Γ ` G1 : rgn
Γ ` G2 : rgn
Γ`E: K Γ ` {E } : rgn
⊗ is in {∪, ∩, −}
Γ ` G1 ⊗ G2 : rgn Figure 2.4: Typing rules for expressions.
25 Abbreviation
Expanded form
Explanation
σ(x ) σ(x .f ) σ(o.f ) σ−x alloc(σ) type(o, σ) update(σ, x , v ) extend(σ, x , v ) update(σ, o.f , v )
s(x ) h(s(x ))(f ) h(o)(f ) (r , h, s 0 ) dom(r ) r (o) (r , h, s 0 ) (r , h, s 0 ) (r , h 0 , s)
variable lookup field of object referenced by variable x field of reference value o s 0 is s with x removed from its domain set of all allocated references type of an allocated reference s 0 overrides s to map x to v s 0 extends s to map x to value v , for x 6∈ dom(s) h 0 overrides h to map field f of o to v .
Figure 2.5: Abbreviations used in semantic definitions. T ranges over all types unless explicitly constrained in the antecedent. Most of the rules are straightforward. The rule for singleton region {E } enforces that E is of reference type. The rule for image expression G ‘f checks that f is declared in some class K , and f has type rgn or reference. Here and throughout, rules are only permitted to be instantiated when the consequent as well as the antecedent are well-formed. For example, the rule for the local variable block command cannot be instantiated when x is in dom(Γ); the comma in Γ, x : T denotes the union of disjoint partial functions.
2.2.2
Semantics
We use a denotational semantics where commands denote deterministic state transformers, which fits well with pre/post specifications. The details are adapted and simplified from a machine-checked semantics of CoreJML encoded in PVS and including hooks to add types like rgn and operations on ghost state [LNR06]. We assume given the sets Ref, {nil}, B which are pairwise disjoint. The values denoted by a reference type K include nil as well as all allocated objects of type K . The values of type rgn are included in the set 2Ref ; they are finite sets of allocated references of any type K . The values of type B are true and false. A state for context Γ has the form (r , h, s) where: r is a ref context, i.e., a partial function mapping the allocated references to their types; h is a heap that maps each allocated reference to its object state (i.e., map from field names to values); and s is a store that maps each variable x in Γ to its value. Throughout the paper, states are assumed to be well-typed and self-contained (i.e., no dangling references). The values of type K in state (r , h, s) are all o ∈ dom(r ) such that r (o) = K , together with the distinguished value nil. The values of type rgn are all finite subsets of dom(r ). The semantics is parameterized on the allocator, i.e., a deterministic function of the state that yields fresh references but is otherwise arbitrary. Following separation logic correctness judgements specify error-free partial correctness. So we use a denotational semantics in which [[Γ ` C ]]σ, for Γ-state σ, is either t (fault), ⊥ (divergence), or a Γ-state σ 0 (normal termination). The only possible fault is due to null dereference. The compound commands like sequence and loops are strict in t as well as in ⊥. The semantics of loops is given by fixpoint as usual, where we order outcomes by
26 [[x ]]σ = σ(x ) [[G1 ∪ G2 ]]σ = [[G1 ]]σ ∪ [[G2 ]]σ [[{E }]]σ = { [[E ]]σ } if [[E ]]σ 6= nil, else ∅ [[G1 ∩ G2 ]]σ = [[G1 ]]σ ∩ [[G2 ]]σ [[alloc]]σ = alloc(σ) [[emp]]σ = ∅ [[G1 − G2 ]]σ = [[G1 ]]σ \ [[G2 ]]σ [ {σ(o.f )} \ {nil} if f : K , o∈[[G]]σ∧f ∈fields(o,σ) [[G ‘f ]] = [ σ(o.f ) otherwise f : rgn o∈[[G]]σ∧f ∈fields(o,σ)
Figure 2.6: Semantics of region expressions (G). ⊥ ≤ t and ⊥ ≤ σ for any state σ (but distinct states are incomparable and not comparable with t). The semantic definitions use a number of abbreviations defined in Fig. 2.5. In these abbreviations, we assume σ is (r , h, s). The metavariables x , y, z range over variable names, whereas we use o, p, q for elements of Ref. We write fields(o, σ) for fields(type(o, σ)). Semantics of commands and ordinary expressions are straightforward and omitted. However, as one would expect, x .f := E faults if x is null. Also note that [[E ]]σ depends on the store but not the heap and is always a value (of appropriate type), never t or ⊥. Fig. 2.6 contains the semantics of region expressions. Thus, a region variable x evaluates to a corresponding region in the store. Singleton region {E } is either the emptyset or a singleton set; alloc is the set of all allocated references in the current heap. For image expression G ‘f , owing to typing in Fig. 2.4, we must consider two cases. For every o in G, when f is a region field, we form the union of all regions denoted by o.f such that f is defined for o; when f is a reference field, we form the union of the singleton sets {o.f } such that f is defined for o and subtract {nil} because regions contain only allocated references.
2.3
Assertion Language
We describe the syntax and the semantics of the region assertion language. The assertion language is like the one in [BNR08b] but slightly different. Here, region expressions G admit image expressions of arbitrary depth, e.g., G ‘f ‘g, and we have an instance-of predicate. Note, while Verl’s assertion language admits field access expressions of arbitrary depth, e.g., x .f .g, the region assertion language does not; instead, field access expression is restricted to one-step dereference and must occur inside an assertion of the form x .f = F . Formalization of unrestricted field access expressions adds a complication due to undefinedness, namely what semantic value should be denoted by E .f when E denotes the null reference. Typically, undefinedness leads to three-valued semantics which complicates matters: to use SMT solvers we need to translate to two-valued semantics; soundness of the Frame rule and underlying machinery would need to be checked for the three-valued case whereas in the two-valued case it has previously been established in [BNR08b].
27
2.3.1
Syntax
The grammar of the region assertion language (RAL) is given in Fig. 2.7. The expression syntax, i.e., syntax categories E , G, F , is exactly the same as in Fig. 2.2. (Thus, RAL expressions are in one-to-one correspondence with the expressions in the programming language.) x ∈ VarName
f ∈ FieldName
K ∈ ClassName
E
::= x | null | true | false
G
::= x | {E } | G ‘f | emp | alloc | G ⊗ G
F
::= E | G
P
::= E = E | x .f = F | G ⊆ G | G # G | type(K , x ) |
where ⊗ is in {∪, ∩, −}
(∀x : K ∈ G · P ) | P ∧ P | ¬P Figure 2.7: Region Assertion Language (RAL).
The atomic (assertion) formulas denoted by P contain: E = E —equality of references (and booleans), x .f = F —field access equality, G ⊆ G—region inclusion, G # G— region disjointedness, type(K , x )—instance-of predicate. The instance-of predicate can be used with both reference and region variables. In case of the latter, type(K , x ) says that the region denoted by x is constrained to consist of allocated references of type K 0 such that K 0 ≤ K . Thus, type(K , x ) can be used to enforce type constraints for otherwise untyped regions. Quantification is over reference types only, where a bounding region is required as in (∀x : K ∈ alloc · P ). Quantification is over allocated objects as is usual [PdB05] and important for certain global invariants. When the bound is alloc, as above, we drop it for convenience and write, ∀x : K · P . The semantics is two-valued and classical. For example, ∃x : K ∈ G · P is obtained from ∀ by DeMorgan. We shall rely on the following syntax sugar: E ∈ G, to mean E 6= null ∧ {E } ⊆ G; x .f ∈ G to mean x .f 6= null ∧ {x }‘f ⊆ G; G1 = G2 , to mean G1 ⊆ G2 ∧ G2 ⊆ G1 . Typing. Assertions are typed according to the rules in Fig. 2.8 and Fig. 2.4 for region expressions. (As before we omit typing rules for ordinary expressions.) Equality of two ordinary expressions is well-formed if both expressions are of the same type or in case of references, one is a subclass of the other; field access equality is typed similarly except we check that the field is defined. The instance-of predicate is well-formed if the first argument is a class name and the second is a variable of either reference or region type. The rule for quantified assertion requires that x is of type Ref and x neither occurs in Γ (recall, Γ, x : K denotes a union of disjoint partial functions), nor in G. (We require that the quantification bound G be free of x in order to facilitate frame derivation, see Sect. 2.5.1 and Fig. 2.12.)
28
Γ ` E2 : T 0
Γ ` E1 : T
T ≤ T 0 or T 0 ≤ T
T , T 0 is in {bool, K }
Γ ` E1 = E2 T ≤ T 0 or T 0 ≤ T
(f : T ) ∈ fields(Γ(x ))
Γ ` F : T0
Γ ` x .f = F Γ ` G1 : rgn
Γ ` G2 : rgn
Γ ` G1 ⊆ G2 Γ`x :T
Γ ` G2 : rgn
Γ ` G1 # G2
T is in {K 0 , rgn}
Γ ` type(K , x ) Γ ` P1
Γ ` G1 : rgn
Γ, x : K ` P
Γ ` G : rgn
Γ ` ∀x : K ∈ G · P Γ ` P2
Γ ` P1 ∧ P2
Γ`P Γ ` ¬P
Figure 2.8: Typing rules for assertions.
2.3.2
Semantics
The semantics of a well-formed assertion, P , typed in Γ, i.e., Γ ` P , is defined in terms of a satisfaction relation, σ |=Γ P , read: Γ-state σ satisfies P . In most cases we elide Γ since it is unchanged throughout. P is called valid iff it is satisfied by all Γ-states. Fig. 2.9 defines the satisfaction relation. The semantics of region expressions was given in Fig. 2.6.
σ σ σ σ
|= E1 = E2 |= x .f = F |= G1 ⊆ G2 |= G1 # G2
iff iff iff iff
σ |=Γ type(K , x )
iff
σ |= P1 ∧ P2 σ |= ¬P σ |=Γ ∀x : K ∈G · P
iff iff iff
[[E1 ]]σ = [[E2 ]]σ σ(x ) 6= nil and σ(x .f ) = [[F ]]σ [[G1 ]]σ ⊆ [[G2 ]]σ [[G1 ]]σ ∩ [[G2 ]]σ = ∅ ( type(o, σ) ≤ K for all o ∈ σ(x ) if Γ(x ) = rgn, type(σ(x ), σ) ≤ K otherwise σ |= P1 and σ |= P2 σ 6|= P extend(σ, x , o) |=Γ,x :K P for all o in [[G]]σ with type(o, σ) ≤ K Figure 2.9: Semantics of RAL.
29
2.4
Effects
An effect set is a comma-separated list ε of effects, ε, with grammar ε ::== rd x | rd G ‘f | wr x | wr G ‘f | rd alloc | wr alloc | fr G The idea is that rd x allows variable x to be read, rd G ‘f allows read of the f field of objects in G, wr x allows update of variable x , wr G ‘f allows update of the f field of objects in G. Recall that alloc is automatically updated by the allocator so that it denotes the set of all allocated references in the current state. Thus, wr alloc allows allocation and rd alloc allows dependence on the set of allocated objects. Finally, fr G says that all elements of G in the final state are freshly allocated. Freshness is used to mask updates to fresh objects in sequences. For example, consider the sequence x := new Node; x .val := false. By itself, the field update has effect wr {x }‘val . But in the pre-state of the sequence, {x } cannot possibly contain the updated object. Indeed, no pre-existing object is updated. In the proof rules, the effect of x := new Node includes fr {x } which by the sequence rule (Seq) in Fig. 2.13 annihilates the write effect. Technicalities. Effects must be well-formed (wf) for the context Γ in which they occur: rd x and wr x are wf if x ∈ dom(Γ); rd G ‘f , wr G ‘f , and fr G are wf if G is wf in Γ. We say σ 0 extends σ provided alloc(σ) ⊆ alloc(σ 0 ) and type(o, σ) = type(o, σ 0 ) for all o ∈ alloc(σ). The semantics has the property that σ 0 extends σ whenever σ 0 = [[C ]]σ. Definition 2.1 (allows transition) Let effect set ε be well-formed in Γ and let σ, σ 0 be Γ-states. We say ε allows transition from σ to σ 0 , written σ → σ 0 |= ε, iff σ 0 extends σ and the following all hold: (a) for every y in dom(Γ) we have either σ(y) = σ 0 (y) or wr y is in ε (b) for every o ∈ alloc(σ) and every f ∈ fields(o, σ), either σ(o.f ) = σ 0 (o.f ) or there is wr G ‘f in ε such that o ∈ [[G]]σ (c) if alloc(σ 0 ) 6= alloc(σ) then wr alloc is in ε. (d) for each fr G in ε, [[G]]σ 0 ⊆ alloc(σ 0 ) − alloc(σ). Definition 2.2 (agreement on read effects) Let ε be an effect set and σ, σ 0 be states such that σ 0 extends σ. Say that σ and σ 0 agree on ε, written σ ∼ε σ 0 , provided the following hold: (a) for all rd x in ε, we have σ(x ) = σ 0 (x ) (b) if rd alloc in ε then alloc(σ) = alloc(σ 0 ) (c) for all rd G ‘f in ε, for all o ∈ [[G]]σ with f ∈ fields(o, σ), we have σ(o.f ) = σ 0 (o.f )
30
G1 ⊆ G2 ` wr G1 ‘f ≤ wr G2 ‘f
G1 ⊆ G2 ` rd G1 ‘f ≤ rd G2 ‘f
true ` wr G1 ‘f , wr G2 ‘f ≶ wr (G1 ∪G2 )‘f P ` ε1 ≤ ε2 P ` ε2 ≤ ε3 P ` ε1 ≤ ε3
true ` ε ≤ ε, ε
true ` rd G1 ‘f , rd G2 ‘f ≶ rd (G1 ∪G2 )‘f P0 ⇒ P P ` ε ≤ ε0 P 0 ` ε ≤ ε0
P ` ε1 ≤ ε2 P ` ε1 , ε ≤ ε2 , ε
Figure 2.10: Selected sub-effect rules. We write ≶ to abbreviate two inclusion rules. For Def. 2.2(c), note that because σ 0 extends σ, we have o ∈ alloc(σ 0 ) and type(o, σ) = type(o, σ 0 ), hence f ∈ fields(o, σ 0 ). But it need not be the case that o ∈ [[G]]σ 0 . Often, we need to subsume an effect by a weaker one when the effect refers to a local variable in a different context. An effect set ε, ε, with ε added to ε, allows at least the effects allowed by ε. In the case of an effect like wr G ‘f there is also the possibility of more liberal effect wr H ‘f in case G ⊆ H . Since regions can be state-dependent, inclusions like the above are state-dependent, so we use a judgement P ` ε ≤ ε0 to express that the writes/reads in a “greater” effect are more permissive. Subsumption for freshness effects is treated separately, since such effects are interpreted in the post-state. Rules for subeffecting are defined in Fig. 2.10. Note that the relation is reflexive, since in the weakening rule ` ε ≤ ε, ε one may choose ε to be some element of the set ε. The following two lemmas capture the semantics of sub-effecting. Lemma 2.3 (write sub-effect) Suppose P ` ε1 ≤ ε2 and ε1 allows transition from σ to σ 0 . If σ |= P then ε2 allows transition from σ to σ 0 . Lemma 2.4 (read sub-effect) Suppose P ` ε1 ≤ ε2 and σ and σ 0 agree on ε2 . If σ |= P then σ, σ 0 agree on ε1 .
2.5
Framing and Separators
This section defines a judgement, P ` ε frm P 0 , that says the truth or falsity of predicate P 0 depends only on the state read according to ε, i.e., ε covers the footprint of P 0 in P states. This is one of the two critical ingredients in the Frame rule (Sect. 2.6). The other is the notion of separator, which applies to the read effects of a formula and the write effects of a command. Their separator is a conjunction of region disjointedness formulas sufficient to ensure that in any transition from state σ to σ 0 allowed by the write effects, σ, σ 0 agree on the read effects.
2.5.1
Framing
Fig. 2.11 defines a syntax-directed analysis that computes a precise “footprint” of ordinary expressions, region expressions and atomic assertions. Intuitively, the footprint is all reads needed to evaluate a given expression or atomic assertion whereas the frame of an assertion is an over-approximation of these requisite reads.
31 For an ordinary expression E , define ftpt(E ) as: ftpt(x ) = {rd x }, where c ∈ {null, true, false}.
ftpt(c) = ∅,
For a region expression G, define ftpt(G) as: ftpt(x ) = {rd x } ftpt(emp) = ∅ ftpt(alloc) = {rd alloc} ftpt({E }) = ftpt(E ) ftpt(G ‘f ) = ftpt(G) ∪ {rd G ‘f } ftpt(G1 ⊗ G2 ) = ftpt(G1 ) ∪ ftpt(G2 ) where ⊗ ∈ {∪, ∩, −}. For atomic assertion P , define ftpt(P ) as: ftpt(E1 = E2 ) = ftpt(E1 ) ∪ ftpt(E2 ) ftpt(x .f = F ) = {rd x , rd {x }‘f } ∪ ftpt(F ) ftpt(G1 ⊆ G2 ) = ftpt(G1 ) ∪ ftpt(G2 ) ftpt(G1 # G2 ) = ftpt(G1 ) ∪ ftpt(G2 ) ftpt(type(K , x )) = {rd x } Figure 2.11: Framing of expressions and atomic assertions. Fig. 2.12 specifies the judgement P ` ε frm P 0 ; intuitively, it says that in P -states, evaluation of P 0 reads at most ε. Note that the absence of hypothesis as in FrmFtpt simply means that P is taken to be true. The rule FrmAnd for framing a conjunction, P1 ∧ P2 with ε allows P1 to be used as hypothesis in showing that ε frames P2 . This is sound because in a state where P1 is false, the conjunction’s value is independent from the value of P2 . It is very helpful in subsuming local effects by more global effects. For example, suppose ε = rd o, rd p, rd r , rd r ‘nxt and we wish to establish that ε frames the formula o ∈ r ∧ p = o.nxt. It is clear that ε frames o ∈ r . But the frame of p = o.nxt must include {o}‘nxt, and this is missing from ε. However, because of o ∈ r we have rd {o}‘nxt ≤ rd r ‘nxt using the second rule in Fig. 2.10. Note that ∧ is commutative —it has standard semantics. The rule for ∧ can be used for either conjunct, owing to the FrmEq rule which allows use of any validity P1 ⇔ P2 . To frame a quantification ∀x : K ∈ G · P 0 in context P , observe that because P 0 might refer to x , we are likely to need rd x , rd {x }‘f and x .g ‘g (i.e., the read effects of the pivot field x .g) to frame P 0 . The frame of the quantification cannot mention x . However, read effects rd {x }‘f may be subsumed by rd G ‘f because x ∈ G. Similarly if we are able to establish that for all x the pivot expressions x .g are all bounded by the region H , the effect x .g ‘g can be subsumed by H ‘g. The first rule in Fig. 2.12 for quantification of a reference variable (x : K ) applies when there are no pivot regions, the second when P 0 uses only a single pivot region, x .g. The generalization to multiple pivots is straightforward but notationally messy. Lemma 2.5 (footprint agreement) For any states, σ, σ 0 , for any expression F , suppose that σ, σ 0 agree on ftpt(F ). Then [[F ]]σ = [[F ]]σ 0 . Lemma 2.6 (frame agreement) For any σ, σ 0 , any predicates P , P 0 , and any set of effects ε, suppose P ` ε frm P 0 and σ |= P and σ ∼ε σ 0 . Then σ |= P 0 iff σ 0 |= P 0 .
32
FrmSub
FrmFtpt
P ` ε1 frm P 0
P is atomic ` ftpt(P ) frm P
Q ` ε2 frm P
Q ⇒P
0
FrmEq
FrmNeg
P ` ε frm P 0
P ` ε frm ¬P
P ` ε1 ≤ ε2
P1 ⇔ P2 0
P ` ε frm P1
P ` ε frm P2
FrmAnd
P ` ε frm P1
P ∧ P1 ` ε frm P2
P ` ε frm P1 ∧ P2 Frm∀
ftpt(G) ⊆ ε
P ∧ x ∈ G ` ε, rd x , rd {x }‘f frm P 0
P ` ε, rd G ‘f frm ∀x : K ∈ G · P 0 Frm∀ pivot
P ⇒ ∀x ∈ G · x .g ⊆ H
ftpt(G) ⊆ ε ftpt(H ) ⊆ ε P ∧ x ∈ G ` ε, rd x , rd {x }‘f , rd x .g ‘g frm P 0
P ` ε, rd G ‘f , rd H ‘g frm ∀x : K ∈ G · P 0 Figure 2.12: Inductive definition of the framing judgement.
2.5.2
Separators
Given effect sets εr and εw , we define the separator formula εr ·/. εw to be a conjunction of certain disjointednesses. In a state where εr ·/. εw holds, nothing that the read effects in εr allow to be read can be written according to the write effects in εw . Note that εw (resp. εr ) may contain read (resp. write) effects but these do not influence the separator. Definition 2.7 (separator) Define separator εr ·/. εw by recursion on the effect sets: rd G1 ‘f ·/. wr G2 ‘g rd y ·/. wr x rd alloc ·/. wr alloc ε ·/. ε0 (ε, ε) ·/. ε1 ε ·/. (ε0 , ε)
= = = = = =
if f ≡ g then G1 # G2 else true if x ≡ y then false else true false true otherwise (for all other single effects) (ε ·/. ε1 ) ∧ (ε ·/. ε1 ) (ε ·/. ε0 ) ∧ (ε ·/. ε)
Lemma 2.8 (separator agreement) Consider any effect sets ε1 and ε2 . Suppose σ → σ 0 |= ε2 and σ |= ε1 ·/. ε2 . Then σ ∼ε1 σ 0 .
2.5.3
Immunity
Consider the sequence y := x .left; y.item := false. The individual effects, write of y and write of {y}‘item, cannot just be unioned to give the effect of the sequence, because write
33 effects are interpreted in the pre-states (Def. 2.1(b)). The y in wr {y}‘item may not be the same y as in the pre-state of the entire composition. The proof rule for sequential composition in Fig 2.13 must therefore ensure that the effect of the field update is immune from (or does not interfere with) the effect of the assignment. The effect wr {y}‘item can be subsumed by the effect wr alloc‘item. The footprint of the region alloc in the write effect is separate from the footprint of y and this permits the combined write effects to be wr y, wr alloc‘item. In Sect. 2.7 we revisit the above sequence of commands; in that setting we show that wr {y}‘item can be subsumed by an effect more precise than wr alloc‘item, namely wr r1 ‘item where r1 happens to be a ghost variable occurring in the precondition. Definition 2.9 (P /ε-immune) Region expression G is said to be P /ε-immune provided P ⇒ ftpt(G) ·/. ε is valid. Effect set ε2 is P /ε1 -immune provided that for all G, f such that wr G ‘f occurs in ε2 , it is the case that G is P /ε1 -immune. For example, alloc is P /ε-immune provided wr alloc is not in ε. Also, wr x is true/wr x -immune (vacuously), but wr {x }‘f is not true/wr x -immune because ftpt({x }) ·/. wr x = false by Def. 2.7. The key property of immunity is that if { P } C1 { P1 } [ε1 ] and { P1 } C2 { P 0 } [ε2 ] are valid, and ε2 is P /ε1 -immune, then ε1 , ε2 is a valid effect for the sequence C1 ; C2 . The following lemma is used to establish the soundness of the Seq rule. Lemma 2.10 Let G be P /ε-immune. Then [[G]]σ = [[G]]σ 0 for any σ, σ 0 such that σ → σ 0 |= ε and σ |= P .
2.6
Program Correctness
A correctness statement takes the form { P } C { P 0 } [ε]. The intended meaning is that from any initial state that satisfies P , C does not fault (terminate with error), and if it terminates then the final state satisfies P 0 . Moreover any allocation and update effects are allowed by ε (Def. 2.1). The statement is well-formed in Γ provided that P , P 0 , C , and ε are well-formed in Γ. The notation `Γ is used for provability of statements that are well-formed in Γ, so the proof system derives judgements of the form `Γ { P } C { P 0 } [ε]. The semantics is used to define valid correctness statements, for which we use notation |=Γ . Definition 2.11 (validity) For state transformer ϕ of type Γ, define ϕ |=Γ { P } − { P 0 }[ε] iff for all Γ-states σ, σ 0 such that σ |= P we have ϕ(σ) 6= t and if ϕ(σ) = σ 0 then σ 0 |= P 0 and σ → σ 0 |= ε. Let { P } C { P 0 } [ε] be well-formed in Γ. The correctness statement is valid, written Γ |= { P } C { P 0 } [ε], if and only if [[Γ ` C ]] |=Γ { P } − { P 0 } [ε]. Fig. 2.13 gives selected syntax-directed proof rules and axioms. In axiom FieldUpd, one step of dereferencing is allowed since F in the rule can be of the form x .f in the case that f : rgn. But if we allowed command x .f := y.f .g, the rule would yield postcondition x .f = y.f .g which is unsound due to possible sharing.
34
fields(K ) = f : T Alloc
` { true } x := new K { x is K ∧ x .f = default(T ) } [wr x , wr alloc, fr {x }] y 6≡ x ` { x = y } x := F { x = (F /x →y) } [wr x ]
Assign
z 6≡ x ` { y 6= null ∧ z = y } x := y.f { x = z .f } [wr x ]
FieldAcc
` { x 6= null ∧ y = F } x .f := F { x .f = y } [wr {x }‘f ]
FieldUpd
` { P } C1 { P1 } [ε1 , fr G] ε1 is fr -free ε2 is P /ε1 -immune ` { P1 } C2 { P } [ε2 , wr G ‘f ] G is P1 /(ε2 , wr G ‘f )-immune P1 ⇒ G i ⊆ G for every wr G i ‘f i ` { P } C1 ; C2 { P 0 } [ε1 , ε2 , fr G] 0
Seq
`Γ,x :T { P ∧ x = default(T ) } C { P 0 } [wr x , ε] Var
`Γ { P } var x : T in C end { P 0 } [ε]
Figure 2.13: Selected correctness rules and axioms for commands. Fig. 2.14 gives selected structural rules. NoUpdate illustrates how assertional reasoning can be used to eliminate effects. Fig. 2.15 gives selected derived rules. These rules are convenient for dealing with less general cases. SimpleSeq is for cases where there are no writes to fresh objects. Theorem 2.12 (Soundness) If ` { P } C { P 0 } [ε] then |= { P } C { P 0 } [ε], for any C , P , P 0 , ε.
2.7
Proof Rules in Action
We give a step-by-step correctness proof of a command acting on variable x of type Node. A Node has three fields: item of type bool and left, right of type Node. The command sets the item field of x ’s left node to false. The precondition is P ∧ Q and the postcondition is Q where P closed Q
= b = b = b
x 6= null ∧ x .left ∈ r1 ∧ x .right ∈ r2 ∧ r1 # r2 ∧ closed r1 ‘left ⊆ r1 ∧ r1 ‘right ⊆ r1 ∧ r2 ‘left ⊆ r2 ∧ r2 ‘right ⊆ r2 ∀x : Node ∈ r2 · x .item 6= false
The specification uses two region variables, r1 and r2 . Precondition P expresses that x is non-null and the object denoted by x .left is in r1 (and x .right is in r2 ). (Recall that x .left ∈ r1 is sugar for x .left 6= null ∧ {x }‘left ⊆ r1 .) Furthermore, regions r1 , r2 are disjoint and closed under left and right, respectively. In summary, the precondition P states that r1 contains (at least) all nodes reachable from x by following the left field, and r2 contains (at least) all nodes reachable from x by following the right field.
35
P ` εQ frm Q P ⇒ εQ ·/. εC ` { P } C { P 0 } [εC ] 0 ` { P ∧ Q } C { P ∧ Q } [εC ]
Frame
SubEff
`Γ { P } C { P 0 } [ε]
P ` ε ≤ ε0 ` { P } C { P 0 } [ε] ` { P } C { P 0 } [ε0 ] Conseq
Context
` { P1 } C { P10 } [ε] P2 ⇒ P1 ` { P2 } C { P20 } [ε]
` { P } C { P 0 } [wr {x }‘f , ε] NoUpdate
`Γ,x :T { P } C { P 0 } [ε] P10 ⇒ P20
rd x ·/. ε rd y ·/. ε 0 ` { P } C { P } [ε]
P ∨ P 0 ⇒ x .f = y
Figure 2.14: Selected structural rules.
x∈ / Vars(F ) SimpleAssign
` { true } x := F { x = F } [wr x ] x 6≡ y
SimpleFieldAcc
` { y 6= null } x := y.f { x = y.f } [wr x ]
` { P } C1 { P1 } [ε1 ] ` { P1 } C2 { P 0 } [ε2 ] ε1 is fr -free ε2 is P /ε1 -immune SimpleSeq
` { P } C1 ; C2 { P 0 } [ε1 , ε2 ] Figure 2.15: Selected derived rules.
The formula Q plays the role of an invariant. It says that for any node o in r2 , o’s item field holds the value true. Since the command writes to the item field, we show that its write effect is at most the item field of objects residing in r1 , denoted by wr r1 ‘item. Thus, we shall derive the following correctness statement: { P ∧ Q } var y : Node in y := x .left; y.item := false end { Q } [wr r1 ‘item]
(2.1)
Indeed a stronger postcondition can be derived but the one above shall suffice for our expository purposes. Our derivation will make frequent use of Frame in Fig. 2.14. Therefore, let’s derive the frames of P and Q first. For P , we obtain the following frame using ftpt, defined in Fig. 2.11, and the rule FrmFtpt in Fig. 2.12 to derive frames of atomic assertions; then, using FrmAnd and FrmNeg in Fig. 2.12 for conjunction and negation. ` rd x , r1 , r2 , {x }‘left, r1 ‘left, r2 ‘left, {x }‘right, r1 ‘right, r2 ‘right frm P
(εP )
(2.2)
Note that the above frame is derived under the hypothesis true. (By convention, absence of hypothesis to the left of the turnstile denotes true.) However, the antecedent of FrmAnd
36 requires that we use the left conjunct as hypothesis, i.e., P1 ` ε frm P2 in Fig. 2.12. Owing to FrmSub, we can drop P1 , i.e., it suffices to derive ` ε frm P2 . Similarly, we can derive the following frame for Q using Frm∀. ` rd r2 , r2 ‘item frm Q
(εQ )
(2.3)
Let’s now derive the correctness statement in (2.1) By SimpleFieldAcc in Fig. 2.15 we obtain: { x 6= null } y := x .left { y = x .left } [wr y] By Def. 2.7, we can see that both εP ·/. wr y and εQ ·/. wr y evalute to true, where εP is in (2.2), εQ is in (2.3). Therefore, by Frame we can conjoin P and subsequently Q to the above thereby obtaining: { x 6= null ∧ P ∧ Q } y := x .left { y = x .left ∧ P ∧ Q } [wr y] Since P ⇒ x 6= null, we can rewrite the precondition using the standard rule of consequence, i.e., Conseq. We obtain: { P ∧ Q } y := x .left { y = x .left ∧ P ∧ Q } [wr y]
(2.4)
For the second update, we use FieldUpd to obtain: { y 6= null } y.item := false { y.item = false } [wr {y}‘item] Next, we conjoin P by Frame since εP ·/. wr {y}‘item = true. { y 6= null ∧ P } y.item := false { y.item = false ∧ P } [wr {y}‘item] We would like to conjoin Q next. However, εQ ·/. wr {y}‘item = r2 # {y}. Our strategy is thus to use the postcondition of (2.4) in order to establish r2 # {y}. Indeed, P ⇒ x .left ∈ r1 ∧ r1 # r2 , whence y = x .left ∧ P ⇒ r2 # {y}. Therefore, using Frame, conjoin y = x .left to the above correctness statement; it is easy to see that ` rd x , y, {x }‘left frm y = x .left is derivable, and the read effects are separated from wr {y}‘item. { y 6= null ∧ y = x .left ∧ P } y.item := false { y.item = false ∧ y = x .left ∧ P } [wr {y}‘item] Because y = x .left ∧ P ⇒ r2 # {y} is a validity, we can now use Frame to conjoin Q: { y 6= null ∧ y = x .left ∧ P ∧ Q } y.item := false { Q } [wr {y}‘item]
(2.5)
Note how the precondition helped establish the validity of the separation condition of Frame. (Above, we tacitly used Conseq to weaken the postcondition by dropping the other conjuncts.) We need a bit more work before we can use SimpleSeq in order to compose (2.4) with the above. First, observe that y = x .left ∧ P ⇒ y 6= null, whence by Conseq the postcondition in (2.4) can be weakened to obtain: { P ∧ Q } y := x .left { y 6= null ∧ y = x .left ∧ P ∧ Q } [wr y]
(2.6)
37 However, the immunity condition in SimpleSeq is not satisfied; that is, wr {y}‘item is not immune from wr y. Therefore, we cannot sequence (2.6) with (2.5). (Of course if we drop write effects and consider classical Hoare-triples [Hoa69], then sequencing is permitted in this case.) To satisfy the immunity condition, we must subsume wr {y}‘item by some other effect whose expression is free of y. (In other words, the write to the item field must be recorded by some other expression whose value is independent of intermediate states.) We can see that y = x .left ∧ P ⇒ {y} ⊆ r1 , i.e., the precondition of (2.5) entails {y} ⊆ r1 which permits us to subsume wr {y}‘item by wr r1 ‘item using the first rule in Fig. 2.10. Therefore, by applying SubEff (in Fig. 2.14) to the correctness statement in (2.5), we obtain: { y 6= null ∧ y = x .left ∧ P ∧ Q } y.item := false { Q } [wr r1 ‘item] (2.7) Finally, we can sequence (2.6) with (2.7) using SimpleSeq to obtain: { P ∧ Q } y := x .left; y.item := false { Q } [wr y, wr r1 ‘item] The rule for local blocks lets us remove the effect wr y and conclude the proof of (2.1).
2.8
Related Work
Burstall observed that by explicitly considering disjointedness in the logic (using an interpreted predicate) reasoning about shared mutable structures could be simplified [Bur72]. Reynolds [Rey00], Ishtiaq and O’Hearn [IO01] proposed a different approach. They added a new logical connective, ?, to express disjointedness. It can be seen as another form of conjunction, P ? Q, whose denotation is: P and Q hold in disjoint parts of the heap, i.e., H1 |= P , H2 |= Q, where H1 , H2 partition the heap. (Cf., P ∧ Q which means P and Q hold in the same heap.) Subsequently, O’Hearn, et al. developed an extension of Hoare logic for a C-like programming language [ORY01]. Therein, they introduced local reasoning and “small axioms”. The logic employs local reasoning with the following Frame rule: {P } C {Q } {P ? R} C {Q ? R} where, as before C must not modify R’s free variables. The rule is deceptively similar to the original Invariance rule, the only difference being ? instead of ∧. It is quite remarkable that this formulation of Frame suffices to establish local reasoning! However, there is one catch: the semantics of correctness statements is not standard. A specification, { P } C { Q } must be tight; that is, P must mention the (entire) footprint of C . Consequently, explicit read/write effects as used in region logic are not needed. (Intuitively, in Frame, the semantics of tight specs abstracts the write effects of C , and the semantics of ? abstracts the read effects of R and the separator.) Separation logic for Java was developed by Parkinson [Par05, PB05]. The use of regions in region logic draws upon previous work by Kassios [Kas06]. Kassios was first to formalize the notion of dynamic framing which addresses the “frame problem” [BMR95] in the context of program states. (Historically, the frame problem was first formulated by McCarthy, in study of artificial intelligence, and roughly asked: how
38 to specify in logic those parts of the state which are unaffected by an operation.) Kassios uses location sets specified by mutable ghost fields, to express footprints of methods and of object invariants, both termed dynamic frames since the interpretation of a frame is statedependent. He defines disjointedness of frames and shows how invariants can be proven to be preserved using these disjointednesses. Note that his work resulted in a metatheory and not a proof system at the level of Hoare-triples. His formalization of framing is declarative, i.e., directly in terms of metatheory; e.g., f frames E , which says f is a dynamic frame of E , is defined using (higher-order) quantification over all states. Region logic, on the other hand, provides a proof system for deriving frames. Ownership types, or more generally, ownership methodology refers to a methodology which guarantees encapsulation for some object structures, e.g., linked-lists, and thus facilitates modular reasoning [CD02]. Under this methodology, each object has an owner: the owner is either another object or a predefined constant. The set of objects having the same owner is called a representation (rep), of the owner. E.g., in a linked-list, the object of type List may be the owner, and its rep is the set of all objects of type Node reachable from the root node. Encapsulation is established by disallowing external references into reps: if an object x references object y, then exactly one of the following must hold: (a) x and y have the same owner; (b) y is owned by x . Consequently, one can reason modularly, i.e., in terms of reps; e.g., c.m()’s invariant is preserved over updates of c 0 so long that c.m() depends only on the reps of c and c 6= c 0 , as distinct objects has disjoint reps. Boogie methodology [BDF+ 04, LM04, NB04] facilitates reasoning about object invariants. The methodology is based on ownership, i.e., it enforces an ownership hierarchy with respect to fields of objects. A field can be declared rep which is to mean that the field refers to a representation object. An invariant of object o may depend only on the fields of o and the fields of objects transitively owned by o. (This restriction is checked syntactically; e.g., if an invariant mentions this.f . . . g.h, then f , . . . , g must be declared as rep fields.) Each object maintains a ghost field inv . The methodology enforces that the invariant of object o holds whenever o.inv is valid. It does so through instrumentation. For example, updates to inv are guarded by special operations pack and unpack; field updates are guarded by condition that the receiver’s inv is not valid. While there are many proposed methodologies based on ownership, ownership is only suitable for specifying restricted object topologies. The Iterator design pattern is challenging for some ownership disciplines, but it is admitted by Universe types [MPH99, MR07] where reps can have external read-only references; it’s also admitted by using dynamic aliases [CD02]. The Composite design pattern (Fig. 2.1), however, is not compatible with ownership disciplines since a client requires access to an arbitrary node of a composite tree. Indeed, Composite was recognized as a challenge problem by the community [ea08]. Summers and Drossopoulou [SD10] propose a methodology for (a) specifying object invariant semantics, i.e., which invariant(s) must hold and at what (program) location; (b) verifying preservation of invariants by computing an upper approximation on the set of objects for which an invariant may get invalidated and asserting the invariant holds for this set, thereby establishing that the invariant holds for all objects. The methodology is applied to the Composite problem by specifying and verifying an implementation of add which is nearly identical, (but weaker, e.g., no effects are specified and postcondition “forgets” that c was added) to the specification depicted in Fig. 2.1.
Chapter 3
Verifier for Region Logic The program and the correctness proof grow hand in hand. Turing award lecture Edsger W. Dijkstra
In Ch. 2 we showed how to use Hoare-style rules to prove correctness of programs specified in region logic. This method of proof is deductive for it relies on finding a proof derivation. An alternative is the inductive assertion method which typically relies on generating verification conditions (VCs) whose validity entails correctness of a putative program. This chapter describes a verifier for region logic programs which is built atop an existing VC-based platform.
3.1
Background
A typical VC-based architecture is structured as follows. A given source program whose behavior is specified using pre-/postconditions and possibly other specification statements, e.g., loop invariants, is translated to a program in a language that is like Dijkstra’s guarded commands [Dij76]. In this (intermediate) language each command has a succinct definition of what its weakest precondition [Dij76] should be. Suppose we have a sequence of guarded commands, to wit body, we want verified with respect to its preconditions denoted by P and its postconditions denoted by Q. We can compute the weakest precondition of body relative to Q, namely wp(body, Q). Then, to verify body, it suffices to check the validity of VC = b ∀x · P ⇒ wp(body, Q), where x are variables which occur free in P or wp(body, Q). VC is a verification condition; assuming that expressions are first-order, VC is a first-order sentence. To check the validitiy of VC , we typically use an SMT solver by asking if the formula P ∧ ¬wp(body, Q)/x →c is unsatisfiable, where c denote fresh constant symbols which are substituted for x (in the existential closure). (Conceivably, a resolution-based prover can be used. However, SMT solvers fare a lot better because typical VCs are comprised of theories such as arrays, equality, partial orders, linear arithmetic, etc., most of which can be decided by a dedicated decision procedure and then combined into an efficient decision procedure for the union theory.) 39
40 The above description of VCs is simplified. We need to mention two additional details which are found in almost all VCs, namely background predicate and labelled expressions. A background predicate, PBG , encodes the semantics of a source language, e.g., types, heap allocation, field definedness, etc., and possibly additional axioms. Thus, in the above, we would prepend PBG as antecedent; i.e., PBG ⇒ ∀x · P ⇒ wp(body, Q) is the actual verification condition. Labelled expressions [LMS05] are used to track locations in the source for assertions that may be violated. That is, if a VC is either unprovable or provably invalid, then owing to labelled expressions it is possible to recover source locations of assertions which may not hold; even a path (through the source code) denoting a possible counterexample can be extracted. In order to further illustrate how VCs are obtained, we describe a simple language of the following commands: assert P
assume P
havoc x
x := E
C1 ; C2
C1 [] C2
The weakest-precondition of each command is defined below. wp(assert P , Q) = P ∧ Q wp(assume P , Q) = P ⇒ Q wp(havoc x , Q) = ∀x · Q wp(x := E , Q) = Q/x →E wp(C1 ; C2 , Q) = wp(C1 , wp(C2 , Q)) wp(C1 [] C2 , Q) = wp(C1 , Q) ∧ wp(C2 , Q) Intuitively, havoc x assigns an arbitrary value to x ; x := E assigns an expression E to x ; C1 ; C2 composes commands together, i.e., execute C1 , followed by C2 ; C1 [] C2 is a choice command meaning that either C1 or C2 can be executed, i.e., non-deterministic composition. Remarkably, the above commands can encode sophisticated source programs having structured control flow. An if-statement, if (E ) then C1 else C2 is translated to assume E ; C1 [] assume ¬E ; C2 A while loop annotated with loop invariant, while (E ) invariant P ; C is translated to assert P ; havoc x ; assume P ; (assume E ; C ; assert P ; assume false [] assume ¬E ) where x are variables that may be modified, i.e., LHS of assignment command, in C . The above embodies the inductive assertion method. That is, to show that P is invariant, as a base case we show that it holds before the executing of the loop. Subsequently, we assume that P holds at the beginning of an arbitrary loop iteration. (Inuitively, havoc x chooses an arbitrary loop iteration by havocing all variables that may be modified by C .) As an inductive step, we show that P is established by the loop iteration. Note, in case the loop guard is false P continues to hold assuming that expressions are free from side effects; also note that assume false essentially says that the remainder of the loop iteration path need not be checked. Preconditions and postconditions are translated to assume and assert commands, respectively. Even the Java heap can be encoded by using maps (also known as arrays) manipulated by read, write in conjunction with the above assignment command. E.g., under the “component-as-array” model [Bur72], fields are represeted by one-dimensional arrays
41 so that x .f translates to read(f , x ) and x .f := E translates to f := write(f , x , E ). In fact, an entire Java-like language, assuming only structured control flow, can be translated to the above guarded commands [Lei06]. Unstructured control flow, e.g., break, goto, requires additional machinery [BL05]. Furthermore, the above weakest preconditions rules, specifically wp(x := E , Q) = Q/x →E and wp(C1 [] C2 , Q) = wp(C1 , Q) ∧ wp(C2 , Q) can yield exponentially large VCs; the culprit is the substitution in the case of assignment command which is even more pronounced because of duplicate Q in the case of choice command. To generate VCs efficiently, a program is first translated into an assignment-free form, i.e., x := E is replaced by assume x 0 = E , for some fresh x 0 . Flanagan and Saxe show how to generate VCs whose size is at worst quadratic [FS01].
3.2
Meet Verl
Verl is a VErifier for Region Logic. Roughly speaking, Verl translates a program specified in region logic to the intermediate specification language Boogie2 [Lei10b]. (Details of this translation are described in Sect. 3.3.) A Verl program typically consists of a program implemented in a core Java-like language together with specifications, i.e., pre/postconditions, effects, specified in accordance with the semantics of region logic. Verl programs are not executable. Speaking facetiously, a verified program is not that interesting to execute. For our purposes, Verl is a primary vechicle for exploring automated verification of programs specified in region logic. Verl is derived from Dafny [Lei10a, Lei08]. (The author implementeed Verl by adapting the source code of an early public release of Dafny, available here [Daf].) Verl inherits many of the features of Dafny’s programming language while adapting the specification language to the semantics of region logic. Verl’s specification language supports the full generality of region assertions and effects as well as “localized framing”—code blocks annotated with (heap-dependent) expressions whose value is to be proven preserved by essentially appealing to the Frame rule of region logic. In general, Verl supports many convenient features of Dafny which are not in region logic, i.e., have not been formalized. Some of these features are sequences, pure functions in specifications, specification statements, e.g., foreach known as “bulk updates” in Dafny. Examples. Fig. 3.1 illustrates a simple Verl program and its specification. (The program as shown can be verified using the command: verl −v example1.rl.) The program should be familiar—we have proven correct an almost identical program in Sect. 2.7. Line 1 declares the fields of class Node. Line 3 declares a method which takes a node x and two (ghost) parameters r, s. (Verl does not have ghost annotation at the moment; thus all parameters are ordinary.) The method merely sets the value of the left child to zero. Lines 4–6 specify the preconditions. Line 7 specifies the postcondition. Line 8 specifies the write effect. Line 4 says the following in region logic x 6= null ∧ x .left ∈ r ∧ x .right ∈ s. Line 5 says that the regions are closed and disjoint. That is, r ‘left ⊆ r and s ‘right ⊆ s and r # s. Note the concrete syntax—grave accent symbol ` denotes the image operation; RefType ::= Id | object
RefType ranges over class names introduced by class declarations as well as the special class name object. There is no mechanism for inheritance, except that every class is a subclass of object. Id derives any string (except reserved keywords) comprising alphanumeric characters; some characters are reserved, e.g., #, $ cannot occur in identifiers. The type rgn denotes sets of allocated references of any RefType. (In accordance with region logic, we use variables and fields of type rgn as if they were ghost; recall, a ghost variable/field cannot influence control flow and cannot be assigned to a non-ghost variable/field. Presently, Verl does not have a ghost annotation.) The type seq < Type > denotes sequences of elements of the given type. Sequences are purely functional entities and should not be confused with Java arrays which suffer from the problem of aliasing. Classes.
A class can declare fields, methods, functions and axioms. Class Member
::= class Id { Members } ::= Field | Method | Function | Axiom
Class members cannot be overloaded; i.e., each member declaration must have a unique name within an enclosing class. Fields.
A field or multiple fields can be declared using the following notation. Field ::= var IdType { , IdType } ; IdType ::= Id : Type
Verl requires that the field names are globally unique. That is, a field f cannot be declared in more than one class. This restriction is used to disambiguate image expressions such as G ‘f . Methods. A method is perhaps the most interesting class member because a Verl program is correct only if all of its specified methods (in all of the enclosing classes) are correct. ::= ( method | constructor ) Id Formals [ returns Formals ] ( ; { MethodSpec } | { MethodSpec } BlockStmt ) Formals ::= ”(” [ IdType { , IdType } ] ”)” MethodSpec ::= requires Expr ; | ensures Expr ; | WrFrSpec ; WrFrSpec ::= wr LocSet { , LocSet } | fr Expr { , Expr } LocSet ::= Id | ”(” Expr , Id { , Id } ”)” Method
45 A constructor is like any other method, except that it is declared by using the constructor keyword. The type checker ensures that a constructor cannot be invoked directly, and that the Id must match the name of the enclosing class. Verl automatically generates a default constructor (i.e., no explicit in-parameters) to ensure that allocated objects have their fields initialized with default values, e.g., null in case of RefType. (The default constructor is generated even if the user provides one; the user constructor is instrumented to invoke the generated constructor.) Every method receives the implicit this in-parameter, i.e., reference to an instance of the enclosing class. As in Dafny, the in-parameters are immutable. (This is not a serious restriction; in-parameters can be copied into locals.) A method can have any number of out-parameters which are declared in the returns clause. The requires and ensures clauses correspond to preconditions and postconditions, respectively. The type checker ensures that Expr is a predicate, i.e., has type bool. Multiple clauses are equivalent to a single clause of conjuncts; i.e., requires P 1; . . . ; requires P n is equivalent to requires P 1 && ... && P n, mutatis mutandis for ensures. A get method which returns the value in field val can be specified like so: method get() returns (res: int) ensures res == val; { res := val; } The absence of requires above is equivalent to the explicit requires true. The production WrFrSpec derives write and freshness effects; the former is declared with the keyword wr, the latter with fr. Recall, there are two types of write effects in region logic, namely wr x which specifies that a variable x is written and wr G ‘f which specifies that a set of f fields are written. There is also the special effect wr alloc which denotes allocation. Freshness effects help pin down fresh objects; a freshness effect specifies a region which must contain only fresh objects, i.e., objects which were allocated by the method. In Verl we use the parenthetical notation to avoid parsing ambiguities. Thus, wr r ‘f is specified using wr (r, f). As a convenience feature, a list of fields can be specified in Verl. E.g., wr (r, f, g, h) is desugared to wr (r, f), wr (r, g), wr (r, h). For convenience, Verl also permits specifying wr (x, f) where x is a variable of RefType; this specification is desugared to wr ({x}, f). Variables occurring in write effects are scoped over the in-parameters; variables occurring in freshness effects are scoped over the out-parameters, with the exception of the implicit this in-parameter. While WrFrSpec is optional, it is an obligation of a method to license all of its heap updates. That is, heap updates to pre-existing objects must be accounted for by specified write effects. For example, the following method is incorrect, i.e., it does not verify, because it does not account for the update. Specifying wr (this, f) would fix the problem. method update() { this.f := null; } On the other hand, the following example is correct because it modifies the field of a fresh object. method update()
46
0 1 2 3 4 5 6 7
method init() requires O == emp; ensures cached.val == −1; wr (this, O), (this, cached), (cached, val); wr alloc; fr O; { var tmp: K;
8
if (cached == null) { tmp := new K(); cached := tmp; O := O + {tmp}; } cached.val := −1;
9 10 11 12 13 14 15
} Figure 3.3: Example of write and freshness effects in method specification. wr alloc; { var x := new K(); x.f := null; } Write effects of the form wr x are permitted in the syntax but are otherwise ignored; Verl issues a warning when such effects are specified. In fact, because the in-parameters are immutable, a method cannot write x. Fig. 3.3 contains a somewhat contrived example which illustrates how different effects are specified. Assume that the class where init is declared also declares the field O of type rgn and the field cached of type K. The precondition requires that O is initially empty. The postcondition ensures that cached.val has been properly initialized. Since a priori it is unknown whether cached has already been initialized, we must account for possible updates to the fields O, cached; plus the update to the field val, which is always executed. Thus, Line 3 specifies these write effects. (Note that implicit references to this are automatically resolved so that wr (cached, val) is desugared to wr ({this.cached}, val).) Because an allocation takes place we must specify wr alloc. Finally, we can prove that in the post-state, the region denoted by this.O contains only fresh objects; hence Line 5 specifies the freshness effect. Fig. 3.4 illustrates another use of freshness effects. The nodes of a linked-list are maintained by contents. Thus, when the method prepend creates a fresh node and makes it the new head of the list, contents must be updated accordingly. However, the updated contents differs from the old contents only by fresh objects. Consequently, Line 5 specifies this fact. The example in Fig. 3.4 expresses what is previously known as the swinging pivots restriction [LN02, Kas06]. In [LN02], Leino and Nelson say this about swinging pivots restriction: a procedure specified to modify a pivot field is allowed to change it only to null or to a value newly allocated within the procedure. In Fig. 3.4, contents plays the role of a
47
0 1
class List { var hd: Node; var contents: rgn;
2
method prepend(x: int) wr alloc, (this, contents); fr contents − old(contents); { var hd: Node := new Node(x); hd.nxt := hd;
3 4 5 6 7 8
contents := contents + {hd};
9
}
10 11
} Figure 3.4: Example specifying the swinging pivots restriction. pivot field. While in the context of [LN02] swinging pivots is an actual restriction imposed by methdology, in other work [Kas06, SJPS08] it is used merely as a specification idiom.
3.2.2
Commands
Fig. 3.5 illustrates the grammar of commands/statements. VarDeclStmts derives variables declarations. Variables are declared using the keyword var. Optionally, a declared variable can be initialized, e.g., var x: int := 0. Verl does not preclude uses of un-initialized variables. For example, the following code type checks but does not verify. class K { var f: int; method foo() { var x: K; x.f := 0; } } Because x can assume any value of type K, the expression x.f is undefined when x == null. To that effect, the translation of the above would generate a well-definedness condition, namely assert x != null. Thus, if we attempted to verify the above code, we would receive an error message saying that x.f must be well-defined. AssignStmt derives assignment statements. RHS of an assignment is either a sideeffect free expression or allocation denoted by new. Allocation creates a fresh object of the given RefType and invokes a constructor. E.g., x := new K() creates a fresh object of type K, calls the constructor (declared in K or otherwise the default one) and assigns the object to x. AssertStmt and AssumeStmt are typical specification statements. The former yields a proof obligation while the latter yields an assumption. Note that assume is potentially unsound, e.g., anything is provable from assume x != x. Therefore, assume usually does not show up in verified code unless there is some formal justification for it. Both specification statements, however, can be used for debugging. Furthermore, assert statements can also be used to guide the prover. That is, a carefully crafted assertion can lead to useful quantifier
48
Stmt VarDeclStmts IdentTypeRhs OneStmt
::= ::= ::= ::=
AssignStmt AssignRhs AssertStmt AssumeStmt CallStmt IfStmt WhileStmt ForeachStmt BlockStmt
::= ::= ::= ::= ::= ::= ::= ::= ::=
VarDeclStmts | OneStmt | BlockStmt var IdentTypeRhs { , IdentTypeRhs } ; Id [ : Type ] [ := AssignRhs ] AssignStmt | AssertStmt | AssumeStmt | CallStmt | IfStmt | WhileStmt | ForeachStmt | break ; | return ; SelectExpr := AssignRhs ; (new RefType ”(” [ Exprs ] ”)”| Expr ) assert Expr ; assume Expr ; call (SelectExpr ; | Id { , Id } := SelectExpr ; ) if Guard BlockStmt [ else (IfStmt | BlockStmt) ] while Guard { invariant Expr ; | WrFrSpec ; } BlockStmt foreach ”(” Id [ : Type ] in Expr ”)” ”{” AssignStmt ”}” [ preserves Exprs ] ”{” { Stmt } ”}”
Figure 3.5: Grammar of Verl statements. instantiations. (Essentially, the expression inside the assertion may yield new terms that would not otherwise occur in verification conditions. See [HP09] for an interesting recent work which uses assert in this way.) CallStmt derives method calls which begin with the call keyword. Since a method may have several out-parameters, the caller must specify the corresponding targets. For example, call x, y, z := foo() invokes foo and simultaneously assigns the three out-parameters to x, y, z, in the order of appearance. WhileStmt derives while loops. Any non-trivial reasoning about loops must involve loop invariants. Loop invariants are specified using the invariant annotation. (As with pre- and postconditions, multiple invariant clauses are equivalent to a single clause with conjoined predicates.) For example, the code in Fig. 3.6 sets each node of an acyclic linked-list to 0. Let’s assume that root != null. Then, the first loop invariant says that r contains root, and if we take any node in r except cur, then its nxt node is also in r. Therefore, if the loop terminates, then cur == null and r−{cur} is equivalent to r; thus, the first assert statement holds. It turns out that the second assert statement holds as well, owing to the other loop invariants. This assert statement says that r is the smallest possible region which contains all nodes reachable from root by nxt. (The code in Fig. 3.6 does indeed verify in Verl.) It is quite remarkable that we do not need a recursive definition of reachability; region closure together with quantification over regions does the trick. We can also specify loop effects, i.e., write and freshness effects. By default, if no loop effects are specified, Verl uses the effects of the enclosing method. If a loop modifies the heap, then we need to relate the pre-heap, i.e., before loop execution, to the post-heap. Consider the loop in the above example which iterates over a linked-list. We can see that
49
var cur: K := root, r: rgn := {root}; while (cur != null) invariant root in r && (r−{cur})`nxt o.root == o) } The predicate J4 (declared in Comp) is used in the verification of the Composite pattern (see composite.rl in [Verl]). The universal quantification ranges over all allocated objects of type Comp. Thus, the following code snippet is incorrect. assume Comp.J4(); x := new Comp(); x.root := null; assert Comp.J4(); The call to the default constructor initializes every field of x to its default value (null for RefType). Thus, x.parent == null. Therefore, the assertion on the last line does not hold because x is an allocated object of type Comp which violates J4. Axioms. Although Verl does not support recursive functions directly, we can use axioms to constrain the interpretations of un-bodied functions. The grammar of axioms is given below. axiom [ string ] Expr ; The optional string can be used to assign a suggestive name or description. Fig. 3.11 illustrates how to use axioms to define the List predicate. (The List predicate is declared in class Defs which contains definitions used in the verification of the subject/observer pattern, see subject observer.rl in [Verl].) List is declared as an un-bodied predicate. The axioms capture the intended meaning, namely List(o, r) is true iff r denotes the region comprising the objects of type Observer which are reached from o by accessing nxt any number of times. Recall that regions are finite sets, hence the axiomatization in Fig. 3.11 is consistent.
56 Note that any expression can be used inside an axiom. The axioms in Fig. 3.11 use quantifiers over regions. Axioms and un-bodied functions should be used sparingly because they can potentially introduce unsoundness.
3.3
Intermediate Representation
The Boogie2 verification platform consists of the procedural specification language Boogie2 [Lei10b], a finely tuned verification condition generator (VCGen) and an SMT solver, typically Z3 [dMB08b] but SMT-LIB [BST10] can also be targeted. Given a specified Boogie2 program, and a list of procedures to verify, VCGen computes the weakest precondition of each specified procedure relative to procedure’s implementation and postconditions. The VCs are handed off to an SMT solver together with the background predicate which encodes the semantics of Boogie2 as well as user-defined axioms which typically encode the semantics of the source language, i.e., Verl in our case. A Boogie2 program may consist of logical declarations and definitions, procedures, as well as specifications thereof. Logical definitions may consist of variables, constants, function symbols and axioms. Procedure implementation can use ordinary assignment, control-flow commands such as while, typically annotated with loop invariants, if -thenelse, return and goto, procedure call commands, as well as special commands: assume, assert, havoc. Procedure specifications consist of pre-/postconditions and write effects of global variables. Specifications can be two-state, allowing a postcondition to refer to the pre-state by way of old; e.g., old(x) == x equates the values of x in the pre- and post-states. Boogie2 is equipped with the built-in, i.e., interpreted, types: bool, int, as well as map types; map types have the standard operations, namely read, e.g., a[i ], and write, e.g., a[i := 0] which denotes a new map obtained by writing 0 at index i . Boogie2 is also equipped with type constructors which are used to declare un-interpreted types. The full type system [LR10] is quite expressive, e.g., it features polymorphic map types. In the sequel we assume that the reader is familiar with Boogie2. We now describe how various Verl features are translated to Boogie2.
3.3.1
Prelude
Boogie2 declarations and axioms which are common to all Verl programs are contained in a prelude. Current distribution of Verl contains two preludes corresponding to two different encodings of regions. (The prelude input file can be specified with the VERL PRELUDE option.) In one encoding, Verl’s rgn type traslates to the map of type ref → bool, type rgn = [ref]bool; In the other encoding, rgn translates to an un-interpreted type. type rgn; The former encoding essentially corresponds to Dafny’s encoding of sets of references. The purpose of the latter encoding is to facilitate integration with a decision procedure for regions. Besides the above type declarations, the two preludes differ only in how region membership is translated. In the case of map type, membership is translated to map read, e.g., x in r is translated to r[x]. In the other case, we use an un-interpreted predicate,
57 function In(ref, rgn) returns (bool); so that in the above example we obtain In(x, r). Without a loss of generality, we assume that in the sequel rgn translates to the un-interpreted type. Note that special characters like $, # cannot occur in Verl identifiers, but can occur in Boogie2 identifiers. We make use of this fact to establish a unique naming scheme. E.g., names of Boogie2 functions encoding regions are always prefixed by Rgn# so that the above predicate is actually named Rgn#In in the prelude; some terms are prefixed by $ to avoid collision with Verl identifiers. To streamline the presentation, we drop these name prefixes.
3.3.2
Heap
The heap is essentially a pair comprising these two global variables: var Heap: HeapType where IsHeap(Heap, Alloc); var Alloc: rgn where IsHeap(Heap, Alloc); As in Dafny, Heap is a two-dimensional map which maps any (ref, Field) pair to its corresponding value. Our encoding of allocated objects differs from Dafny which uses a boolean ghost field alloc. We use the global variable Alloc to denote the current set of allocated objects. Using this representation, we can reason about allocation using pure region expressions. The following prelude declarations are used to complete the above declarations. type ref; const unique null: ref; type Field T; type HeapType = [ref, Field T]T; function IsHeap(h: HeapType, alloc: rgn) returns (bool); References and fields are encoded using un-interpreted types. The null reference is encoded by a unique constant. HeapType is a type synonym for a map parametric in its range type. In the above, the where clause maintains the assumption IsHeap(Heap, Alloc) in every context where the variables Heap and Alloc occur. The predicate IsHeap is used to distinguish the representation of Verl’s heap from other map types that could potentially occur in Boogie2 code. That is, whenever IsHeap(h, alloc) holds, the pair h, alloc encodes an actual Verl heap. The following axiom says that the null reference is not allocated. axiom (forall h: HeapType, alloc: rgn:: { IsHeap(h, alloc) } IsHeap(h, alloc) ==> !In(null, alloc); (The expressions in curly braces are patterns, i.e., triggers, used to constrain quantifier instantiations.) Classes. Each class K declared in Verl, i.e., class K is denoted by a unique Boogie2 constant: const unique class.K: ClassName; The type ClassName is un-interpreted. The prelude declares a unique constant corresponding to the class object:
58 type ClassName; const unique class.object: ClassName; Fields. Each field f declared in class K , i.e., var f: T is denoted by a unique Boogie2 constant: const unique K.f: Field T’ where T’ is the translation of Verl’s type T according to the following: tr[[bool]] = bool tr[[int]] = int tr[[Id ]] = ref tr[[object]] = ref tr[[rgn]] = rgn tr[[seq < T >]] = Seq tr[[T ]] In the sequel, we use the Verl-to-Boogie2 translation function tr[[·]]; for brevity, we shall also elide from our discussion the encoding of sequences. Observe that the translation of RefType erases the actual class name. The class name is encoded using Type predicate which we explain in the next section. For each field f : T declared in class K where T is RefType, Verl generates an allocation axiom which says that f has the appropriate type and its value is either the null reference or some allocated reference. axiom (forall h: HeapType, alloc: rgn, o: ref :: { IsHeap(h, alloc), h[o, K.f] } IsHeap(h, alloc) && In(o, alloc) && TypeOf(o) h[o, K.f] == null || (In(h[o, K.f], alloc) && Type(class.T, h[o, K.f]))); The predicate IsHeap(h, alloc) in the antecedent ensures that the pair h, alloc denotes an actual Verl heap; the following predicates in the antecedent say that o is an allocated reference whose runtime type is a subtype of K , i.e., field access o.f is defined. The consequent says that o.f is either null or an allocated reference of any type compatiable with T , i.e., any subtype of T . Allocation axioms together with the encoding of local RefType variables ensure that every (well-defined) RefType expression denotes either the null reference or some allocated reference of compatible type. Verl also generates an allocation axiom for fields of type rgn and seq < RefType >. We illustrate the allocation axiom for f : rgn declared in class K ; a similar axiom is used for sequences of references. axiom (forall h: HeapType, alloc: rgn, o: ref :: { IsHeap(h, alloc), h[o, K.f] } IsHeap(h, alloc) && In(o, alloc) && TypeOf(o) IsRegion(h, alloc, K.f)); The predicate symbol IsRegion is akin to IsHeap; whenever IsRegion(h, alloc, r) holds, the region denoted by r is well-formed, i.e., comprised of allocated references, with respect to the heap denoted by h, alloc. Every use of region field or variable is essentially guarded by IsRegion.
59
3.3.3
Types
Verl’s instance-of predicate type translates to one of the following functions defined in the prelude. function Type(ClassName, ref) returns (bool); function Type(ClassName, rgn) returns (bool); depending on whether E in type(K, E) is of RefType or rgn type, respectively. (Boogie2 does not support function overloading, the actual predicates are named differently in the prelude. However, for our presentation the types will serve to disambiguate the functions.) Boogie2 has a built-in predicate symbol // following conjunct is generated only if rd alloc is in εF Equal(alloc’, alloc) && (forall p: ref :: In(p, alloc’) ==> Agree(h’, h, p, εF )) ==> K.F(h’, alloc’, this, o) == K.F(h, alloc, this, o)); Figure 3.17: Frame axiom. computation is described in Sect. 3.3.10. Lines 1–4 encode the assumptions about the two heaps and the actual parameters, i.e., this, o. Lines 6–7 encode heap agreement in the same manner as the encoding of preserves. Note the absence of stack agreement in this case; in the case of preserves the actual parameters are local variables hence the need for stack agreement. Finally, Line 8 is the goal of this axiom, namely concluding that F applied to this, o produces the same value in the two heaps. Verl can generate frame axioms like the above at the request of the user. (The function attribute annotation :framing does the trick.) In our experience, preserves clauses are more informative since they delineate code which must preserve some expression. Also in our experience, frame axioms tend to increase (SMT solving) complexity because of quantification over heaps; in fact, this was the motivation for preserves clauses. On the other hand, preserves clauses are currently manual annotations. Thus, a combination approach which uses frame axioms with manual and inferred preserves clauses is perhaps most robust.
70
R(this) = rd this R(alloc) = rd alloc R(Id ) = rd Id R(emp) = R({}) = R(null) = R(true) = R(false) = R(n) = , where n ∈ Nat R({E }) = R(type(K , E )) = R(!E ) = R(−E ) = R(E ) R(∼E ) = rd alloc · R(E ) R(E ‘f ) = R(E ) · rd E ‘f R(E .f ) = R(E ) · rd {E }‘f R(E1 ⊕ E2 ) = R(E1 ) · R(E2 ), ⊕ ∈ {+, −, ∗, /, %, &&, ||, ==, !=, , =, #, in, !in} R(F (E )) = R(E ) · (εF − rd x )/x →E R(E1 .F (E2 )) = R(E1 ) · R(E2 ) · (εF − rd x )/x →E2 /this→E1 Figure 3.18: Read effects of quantifier-free Verl expressions.
3.3.10
Computing Read Effects
In Sect. 3.3.9 we saw how preserves clauses and frame axioms are encoded. Both approaches rely on a sound formulation of read effects of the expression(s) at hand. In region logic, read effects of non-atomic expressions are derived using a system of framing rules (Fig. 2.12) in conjunction with subeffect rules (Fig. 2.10). Using these rules we can derive very precise read effects. However, the rules are deductive and therefore unsuitable for complete automation. (E.g., the antecedent of FrmEq in Fig. 2.12 requires a proof of equivalence of two arbitrary region assertions.) The approach taken in Verl is to trade precision for automation. Read effects of Verl expressions are automatically computed by a syntax-directed analysis. Since Verl expressions include function applications, i.e., F (E ), we need read effects of functions. That is, we must first compute read effects of F ’s body. Recall that Verl functions cannot be recursive. Internally, Verl constructs a function call-graph—a directed graph whose vertices comprise all declared functions and whose edges denote the callercallee relation. That is, there is an edge (F , G) iff the body of F contains an application of G. Verl checks that the function call-graph, is acyclic. Since the graph is acyclic, we can sort its nodes using reverse topological order. Thus, we first compute read effects of “leaf” functions (corresponding to nodes in the call graph whose out-degree is zero), then their immediate caller and so on. Consequently, whenever the call-graph has a path between F and G, G’s read effects are computed before F ’s read effects. Assume that we have an arbitrary Verl function F of one (explicit) argument x. function F(x: T) { body } Let R be a function which computes read effects of any Verl expression E . Then, read effects of F are computed by R(body). We let εF denote these read effects, i.e., εF = R(body). Internally, Verl records computed read effects of functions; effectively we obtain: function F(x: T) εF ; { body } In case of un-bodied functions, εF is tantamount to the user-specified read effects. We are now ready to explain how R is implemented. Fig. 3.18 and Fig. 3.19 illustrate how read effects of Verl expressions are computed. Given an arbitrary Verl expression E ,
71
R(Q x : T :: E ) = substl (R(E ), x , alloc) T ∈ {int, bool} R(Q x : T in B :: E ) = R(B ) · substl (R(E ), x , B ) T ∈ RefType or T = rgn substl (ε · ε, x , B ) = subst(ε, x , B ) · substl (ε, x , B ) ( if Id = x , subst(rd alloc, x , B ) = rd alloc subst(rd Id , x , B ) = rd Id otherwise ( substl (, x , B ) =
subst(rd E ‘f , x , B ) =
rd subst(E , x , B )‘f rd alloc · rd subst(E , x , B )‘f
if subst(E , x , B ) 6= alloc, otherwise
subst(alloc, x , B ) = alloc subst(emp, x , B ) = emp subst({this}, x , B ) = {this} subst({null}, x , B ) = {null} ( ( B if Id = x , B if Id = x , subst(Id , x , B ) = subst({Id }, x , B ) = Id otherwise {Id } otherwise subst(Id .f , x , B ) = subst({Id }‘f , x , B ) subst(E .f .g, x , B ) = subst(E .f , x , B )‘g subst(E ‘f , x , B ) = subst(E , x , B )‘f subst({E .f }, x , B ) = subst({E }‘f , x , B ) ( subst(F (E ), x , B ) if subst(F (E ), x , B ) 6= F (E ) subst({F (E )}, x , B ) = {F((E )} otherwise
subst(E1 .F (E2 ), x , B ) if subst(E1 .F (E2 ), x , B ) 6= E1 .F (E2 ) subst({E1 .F (E2 )}, x , B ) = {E1 .F (E2 )} otherwise ( alloc if x occurs free in E , subst(F (E ), x , B ) = F (E ( ) otherwise
alloc if x occurs free in E1 subst(E1 .F (E2 ), x , B ) = E .F (E2 ) otherwise 1 subst(E1 , x , B ) ⊕ subst(E2 , x , B ) subst(E1 ⊕ E2 , x , B ) = subst(E1 , x , B ) − E2 subst(E1 , x , B )
or E2 ,
if ¬(E1 , E2 : rgn and ⊕ = −) else if x is not free in E2 otherwise
Figure 3.19: Read effects of quantified Verl expressions.
72 R(E ) computes read effects of E . R is defined by cases on E ’s concrete syntax. For brevity the notation is streamlined, e.g., E ‘f instead of E`f, rd E ‘f instead of rd (E, f), etc.; instead of the set notation for effects as in Fig. 2.11, here it is convenient to use lists; thus, · denotes catenation of effects; denotes the empty list of effects. Fig. 3.18 deals with read effects of quantifier-free expressions. For constants and unary expressions it is defined like ftpt (Fig. 2.11). For binary expressions we use an overapproximation. (Cf. FrmAnd in Fig. 2.12 whose antecedent uses left conjunct as an assumption to derive read effects of the right conjunct.) The last two cases in Fig. 3.18 deal with function applications. To compute read effects of F (E ), we compute read effects of E to which we add read effects of F ’s body wherein the formal argument x is substituted with E . However, before substitution we must remove rd x , hence εF − rd x . (The notation ε/x →y means replace every occurrence of x in ε with y.) Note the case R(E .f ). In this case, R computes R(E ) · rd {E }‘f . For example, if an un-bodied function F specifies an effect of the form rd x .f . . . g ‘h, then in the process of computing read effects of some application, say F (o), R rewrites the effect to the form rd {x }‘f ‘ · · · ‘g ‘h. (The two expressions are equivalent assuming that x .f . . . g is welldefined.) Example 3.1 Consider the following Verl function declared in class K . static function F(x: K): rgn requires x != null; { x.f } Its read effects are computed by R(x .f ) = rd x · rd {x }‘f . Let’s compute read effects of a function application, say K .F (o). Thus, R(K .F (o)) = rd o · (rd x · rd {x }‘f − rd x )/x →o, whence R(K .F (o)) = rd o · rd {o}‘f . Fig. 3.19 deals with quantified Verl expressions. In the first case, Q stands for either forall or exists. In the second case, Q can also be pred if x is of RefType; B is any Verl expression of type rgn which denotes the range of quantification. In both cases E is any Verl expression. The two cases cover all possible quantified Verl expressions with one bound variable (excluding quantification over sequences). To compute effects of a quantified expression, we first compute read effects of its body and then use subst to eliminate x from these read effects by over-approximation. Intuitively, subst replaces a region expression E where x occurs free with another expression E 0 such that E ⊆ E 0 is a validity and x does not occur in E 0 . The function substl merely applies subst for every read effect in R(E ). The first two lines in the definition of subst pertain to read effect expressions; the rest pertain to all other expressions that can occur within read effects. (Internally, in Verl’s AST effects are expressions.) As a further remark on implementation, Verl removes duplicate read effects and normalizes read effects so that rd E1 ‘f · rd E2 ‘f is collapsed into rd (E1 ∪ E2 )‘f ; the two expressions are equivalent but the latter is more compact. We explain some interesting cases. Because subst may replace an expression where x occurs free by alloc, we must account for rd alloc; the case subst(rd E ‘f , x , B ) ensures that rd alloc is added to the list. In the case subst(Id , x , B ), if Id = x , then we replace x with its bound expression, namely B . (B is ensured to be free of x by the type checker.) In case of function applications, say subst(F (E ), x , B ), we replace x by alloc if x is free in E .
73 In this case, replacing x with B may not be sound because F could be an arbitrary function. E.g., if F ’s range type is rgn, then it could return an arbitrary region, specifically one that is not included in B ; if F ’s range type is RefType, then it could return a reference which is not contained in B . In case of binary region expressions, we single out region difference, i.e., subst(E1 − E2 , x , B ). If x is free in E2 , then we replace E1 − E2 by subst(E1 , x , B ). It is unsound to replace E2 by B because in general the following may not hold E1 − E2 ⊆ subst(E1 , x , B ) − B . The following properties of subst can be shown: given an arbitrary E of type rgn, subst(E , x , B ) is of type rgn; x does not occur in subst(E , x , B ); E ⊆ subst(E , x , B ). Thus, for any rd E ‘f occurring in read effects of the body of a quantifier, we obtain rd E ‘f ≤ rd subst(E , x , B )‘f by subeffect rule in Fig. 2.10. That is, rd subst(E , x , B )‘f subsumes rd E ‘f . Example 3.2 To illustrate how R works on quantified expressions let’s show the following, where F is the function from Example 3.1. R(forall o : K in B :: K .F (o)
