Formal Semantics and Interpreters in a Principles of Programming Languages Course Kim B. Bruce∗ Williams College
[email protected]
Abstract Most junior-senior level programming languages courses approach the subject either from the point-of-view of principles (concepts) of programming languages or from the perspective of understanding languages through writing progressively more complex interpreters. In this paper we show how to use formal semantics in a series of interpreter assignments in a principles or conceptsbased course. The interpreter assignments make the semantics more concrete for students while providing a deeper understanding of concepts. 1
Introduction
It is unfortunate that in most computing curricula, the uses of theory and formalisms are limited to the algorithms and theory of computation courses. In fact, as indicated in the “Logic in Computer Science” panel [2] in SIGCSE ‘97, logic and logic-related formalisms can play important parts of a wide variety of courses. In this paper we explain how logic-based ideas involving formal operational semantics and type-checking rules can play an important role in upper-level principles of programming languages courses. Modern junior-senior level programming languages courses these days come in two basic flavors: • Concepts or principles of programming languages [10, 8, 9, 5, 11], • Understanding languages through interpreters [7, 4]. ∗ This paper was written while the author was a Visiting Professor at the Department of Computer Science, Princeton University. Partially supported by NSF grant CCR-9870253.
Two decades ago, a third variant to these approaches was popular, one based on a survey of programming languages. This approach, often denigrated as the “if it’s Tuesday it must be APL” approach, has largely died out. The interpreter-based approach uses interpreters to explain the run-time behavior of programs in different languages. Students learn the differences between, for example, different parameter passing conventions by seeing the differences in interpreters for implementing those conventions. As [4] put it, the “goal is to give students a deep, hands-on understanding of the essential concepts of programming languages, using Scheme as an executable meta-language.” Virtually all of these courses use Scheme or LISP for this purpose, though other functional languages such as ML would seem likely to work as well, and languages of other styles (such as logic-based) might also be effective. The principles-based approach to this course tends to pay much more attention to evaluating the features of programming languages, while continuing to discuss run-time behavior in a more descriptive fashion than with the interpreter-based approach. As [10] put it, this course “describe(s) the fundamental concepts of programming languages by defining the design issues of the various language constructs, examining the design choices for these constructs in some of the most common languages, and critically comparing the design alternatives.” Thus this style of course places more emphasis on the analysis of programming languages from the point of view of the user, rather than an implementer. The description of the programming languages course in Curricula ‘91 [3] reflects this approach, and it appears that this style of course is much more common. In the junior-level programming languages course at Williams College, we have tended to follow the principles or concepts-based approach to the material, as we have felt that the ability to evaluate the strengths and weaknesses of particular language constructs and languages is an important outcome of the course. However, the course also includes a substantial amount of material on functional, object-oriented, and (until recently) logic programming languages, on constructs supporting parallel computing, on the run-time behavior of pro-
gramming languages, and on the formal semantics of programming languages. A problem that arises with the concepts-based course compared to the interpreter-based approach is that students generally only have the time to write relatively small, toy programs in new paradigms, making it difficult to really understand the strengths and weaknesses of programming in these styles. Moreover, it is often difficult for students to gain a deep understanding either of run-time behavior of language constructs or of formal semantics. Descriptions of the former often seem too vague, while the latter often seem too precise, formal, and removed from students’ intuitions. Students taking a course using the interpreter-based approach gain deeper insight into run-time behavior because they must program it, while gaining substantial experience at least with the functional language (typically Scheme) which is used to implement the interpreters. On the other hand they may not gain much experience or understanding of the surface language features of languages. Surprisingly, it is also often the case that students in these courses have little or no experience with the use of formal semantics in specifying the behavior of programming constructs. The interpreters themselves take the place of more abstract semantic specifications. However these courses often replace the exposure to new languages and paradigms by learning to simulate their features via an interpreter. While this provides greater insight into how these features are implemented, students often lack experience in learning how the use of new languages can change the programming process, and the variety of ways in which concepts may be incorporated into features of real languages. As a result they may not have the experience to evaluate the strengths and weaknesses of languages in terms of their support of key features like information hiding, modularity, etc. In this paper we describe how we blend these two approaches by adding a series of interpreter assignments to a concepts-based course. Benefits include deeper understanding of the run-time behavior of programming languages, appreciation of the value of formalisms in describing programming languages, and more substantial experience in using functional languages. 2
Formal Specification of Programming Languages
Nearly all courses in programming languages include material on the use of formal grammars in specifying the syntax of programming languages and their use in building parsers. Yet few spend comparable time on specifying the formal semantics of programming languages and their use in building interpreters. Some coverage of formal semantics (denotational, operational, and/or axiomatic) has been creeping into courses in the last few years, probably influenced by the topics specified in Curricula ‘91 [3]. However, these topics are typically in one of the last chapters of the text and are often skipped or rushed through by instructors. We feel that the use of formal semantics should play
a more central role in concepts-based approaches to the programming languages course. A formal operational semantics can be considered a formal specification of an interpreter, as well as the language. Moreover, implementing such an interpreter can provide a substantial exercise in the use of a functional programming language. It is well-known that a context-free grammar can be used as the basis for a parser for a programming language. The context-free grammar can be used to structure a top-down recursive descent parser or can be used with tools such as LEX and YACC to automatically derive tables for a table-driven parser. It is not as well known that a formal semantics can play a similar role for interpreting the syntax trees which are the output of a parser. The operational semantics of a programming language specifies exactly how to execute or evaluate a program. For example in the “natural semantics” style [6] of operational semantics an expression of the form E1 + E2 us evaluated by first evaluating E1 to a value v1 , evaluating E2 to value v2 , and then returning the sum of v1 and v2 . We can write this more formally using the notation E ↓ v to indicate that the expression E, when fully evaluated, gives the result v. The rule for evaluating sums can be written: Sum
E1 ↓ v1 , E2 ↓ v2 E1 + E2 ↓ v1 + v2
This notation is similar to that used to express mathematical proof rules. The hypothesis is written above the line, while the conclusion is below. The above rule states that if E1 evaluates to v1 and E2 evaluates to v2 , then the term E1 + E2 evaluates to v1 + v2 . While other styles of operational semantics, or even a denotational semantics could have been used as the basis for the implementation of an interpreter, we have found the “natural semantics” to be a relatively easy form of semantics for students to understand and use. In the next section we discuss how this formal semantics can be made the basis of a series of assignments for students to write interpreters. Later we discuss how a similar formal notation for type-checking rules can provide the basis for a type-checker. 3
Interpreter Assignments
The programming languages class at Williams begins with an introduction to functional programming languages, with ML used as the main example. The course then proceeds with a relatively standard discussion of concepts (see the first 8 chapters of [8], for instance). Because the students have developed basic competence in programming in ML first, they can use ML to implement the interpreter projects in parallel with the discussion of the concepts. (Other functional languages like Haskell, Miranda, Scheme, or LISP would work as well, as would logic-based languages like Prolog.)
The interpreter projects are a part of the usual weekly homework assignments, which are a mix of programming and analytical questions. In contrast to interpreterbased approaches to programming languages, we spend only 15 to 20 minutes per week in class discussing the projects. It is used as a supplementary exercise to class discussions. We provide students with an ML program which parses a program from a very simple functional language called PCF1 , to an abstract syntax tree form. The syntax of PCF is given by the following contextfree grammar: e ::= x | n | true | false | succ | pred | iszero | if e then e else e | | (fn x => e) | (e e) | rec x => e PCF includes variables (x ), numbers (n), boolean values (true and false), successor (succ) and predecessor (pred ) functions, a function returning whether the argument is zero (iszero), an “if-then-else” expression, userdefined functions (fn x ⇒ e), function applications (e e), and recursively defined terms (rec x ⇒ e). We will ignore the complications of recursion for this paper, but support for recursion is an interesting topic that our students deal with in their interpreters. (Besides, without recursion, this language is extremely limited in expressiveness.) Abstract syntax trees produced by the parser are represented by an ML datatype definition: datatype term = AST_ID of string | AST_NUM of int | AST_BOOL of bool| AST_SUCC | AST_PRED | AST_ISZERO | AST_ERROR | AST_IF of (term * term * term) | AST_FUN of (string * term) | AST_APP of (term * term) | AST_REC of (string *term) For those not familiar with ML datatype definitions, term is defined as a union of types, each of which has an associated tag (e.g., AST ID). Elements of the datatype are written by applying the tag to values of the associated type. Thus AST ID(‘‘counter’’) is a value from the first variant of the data type term. The AST prefix of each expression stands for abstract syntax tree. Each of these expressions can be seen as a tagged term representing the corresponding syntactic category. Thus an if-then-else expression is represented by a term, AST IF(test,yesVal,noVal), where the three arguments are abstract representations of the boolean expression and the expressions to be evaluated depending on whether the boolean evaluates to true or false. Students are to write programs to interpret these expressions using a formal specification of the semantics of the language. The interpreter transforms elements from the datatype term into “values”. Values are those terms which can no longer be reduced using the semantic 1 PCF, designed by Dana Scott, stands for Programming Computable Functions.
rules. Thus terms of the form: AST ID name, AST NUM n, AST BOOL n, AST SUCC, AST PRED, AST ISZERO, AST ERR, and AST FUN (vble,exp) are values, while AST IF(test,yesVal,noVal) and AST APP(func,arg) are not values since there is further computation which may be performed. The value AST ERR only comes into play when there is a type error in the formation of a term. The rules for the operational semantics are given in terms of the source language, PCF, with students being expected to be able to apply them to the (parsed) expressions from type term. As an example, here are the two rules used to interpret well-formed if-then-else expressions: If true
test ↓ true yesVal ↓ v if test then yesVal else noVal ↓ v
If false
test ↓ false noVal ↓ v if test then yesVal else noVal ↓ v
The first rule states that the value of an if-then-else expression is v, if the boolean guard bexp evaluates to true and the expression in the “then” portion evaluates to v. (Note that noVal is not evaluated.) The second rule is similar, explaining how to obtain the result if the boolean guard is false. It is quite easy to transform rules of this form into an ML function which transforms ML expressions of type term into values: fun | | | | | | | |
|
interp interp interp interp
(AST_NUM(n)) = AST_NUM (n) (AST_ID(id)) = AST_ID(id) (AST_BOOL(bval)) = AST_BOOL(bval) (AST_FUN(param,body)) = AST_FUN(param,body) interp (AST_SUCC) = AST_SUCC interp (AST_PRED) = AST_PRED interp (AST_ISZERO) = AST_ISZERO interp (AST_ERROR) = AST_ERROR interp (AST_IF(test,yesVal,noVal)) = let val testval = interp test in if testval = AST_BOOL(true) then interp yesVal else if testval = AST_BOOL(false) then interp noVal else AST_ERROR end ...
Because the first 8 expressions are values, the interpreter simply returns the expressions. However, we can see that when if-then-else expressions are interpreted, the guard test is first evaluated. If it returns the boolean value true than the yesVal is interpreted, while if it returns false, the noVal is interpreted, and otherwise it returns the error term. Notice how this code can easily be read off from the two semantic rules for if-then-else. Start with the left
side of the conclusion (in this case if-then-else), determine if the hypothesis is applicable (in the first rule, if test returns true, in the second, if test returns false), and then perform the indicated computation. Because there were two rules for if-then-else, the code branched on the possible outcomes. We could have given a third rule corresponding to the error result, but we used the default that if no rule was applicable then the result was error. Notice that the interpretation of if-then-else expressions results in at most one of the arms of the expression to be evaluated. Discussion of this leads naturally to a valuable discussion of how if-then-else expressions generally must be evaluated lazily, and what the consequences might be (e.g., in the presence of recursion) if instead both branches of the conditional were evaluated every time. This first interpreter evaluates function application by replacing all occurrences of formal parameters by the corresponding actual parameters. We introduce the notation e[x:=v] to denote replacing all free occurrences of the identifier x by the value v. Students are asked to write a substitution function which takes a term e, identifier name x, and value to be substituted v, and return e[x:=v]. With this notation, function application is specified in the natural semantics by F cnApp
e1 ↓ (f n x ⇒ e3 ) e2 ↓ v1 e3 [x: = v1 ] ↓ v (e1 e2 ) ↓ v
In the above rule, the first argument is evaluated until it results in a function form, then the actual parameter is evaluated and substituted for the formal parameter. Finally the new body of the function is evaluated. (Simpler rules apply when the first argument evaluates to a pre-defined function like succ.) Later we note that while this works fine for simple examples, there is a need for a more subtle substitution process to ensure that identifier names occurring in the actual parameter do not get captured when inserted into the scope of formal parameters with the same name. Most of the students find this assignment somewhat of a challenge, but they are impressed that their final program is both quite readable and only about a page long. They also begin to see the value of a careful semantic specification of the language, and how that specification can direct the design of an interpreter. Many of them also discover through trial and error (mainly error) that minor deviations from the formal semantics can result in faulty run-time behavior. Later in the term, when discussing memory management, we extend the interpreter assignment so that programs can be interpreted with respect to an environment which keeps track of the meaning of identifiers (since PCF does not include imperative features, we do not need a separate memory abstraction). The environment can be represented either by a function from String to term which takes identifier names to their current values, or by a list of pairs of identifiers and their values.
The new interpreter takes both a term and an environment as arguments, returning the value of the term in that environment. In this interpreter, the value of an identifier is found simply by applying the environment to the identifier name: (x, ρ) ↓ ρ(x) In PCF, as in most functional languages, functions may be passed as parameters. Thus a function may be applied in an environment completely disjoint from the one in which it was defined. To be able to interpret free identifiers in the function, we must carry around the defining environment using a closure, which packages together the code of the function with the environment in which it is defined. (f n x => e, ρ) ↓ closure(x, e, ρ) A function is evaluated to a closure which has three parts: the name of the formal parameter, the function body, and the environment at the point at which it is defined. Retaining the environment enables the lookup later of any free identifiers in the function body. We see this by examining the semantic rule for function application in the new interpreter. (f, ρ) ↓ closure(x, body, ρf ), (arg, ρ) ↓ v ′ , (body, ρf [x: = v ′ ]) ↓ v F uncAppl ((f arg), ρ) ↓ v This rule indicates that if a function evaluates to a closure, then we should evaluate the actual parameter, update the environment from the closure to update the value of the formal parameter to be the value of the actual parameter, and then evaluate the function body in the updated environment from the closure. Using the (updated) environment in the closure ensures that if, say, the identifier x had value 7 at the time the function was defined, then any occurrences of x in the function body will have value 7 when the function is defined, even if a different identifier x is in scope at the call site. (Recall that this language does not have assignments or other side-effects, so x is essentially a constant.) Once we have this basic framework, it is easy to give assignments asking students how to modify the interpreter to specify call-by-name rather than call-by-value. Similarly we can ask how to modify it so that it supports dynamic versus static scope. Another relatively straightforward extension is to add assignment statements to the language (though this requires the introduction of a new state or memory component as an argument to the interpreter). It has been our experience that students find writing these interpreters, each of which is short (one to two pages of code), to be quite a valuable experience in developping a much better understanding of basic concepts in programming languages. For example, here is a question which appeared on the homework assignment following the one where the environment-based interpreter was due.
In class we discussed the use of a run-time stack of activation records for procedure calls in block-structured languages. In the interpreter you wrote last week there was no explicit use of a run-time stack. Your (recursive) interpreter for PCF provides similar support for activation records (though PCF is simpler than block-structured languages because there are no local variables and only one parameter for each function call). Explain how your interpreter provides a stacklike support for activation records by explaining how the environment changes during the evaluation of the following expression: ((if is_zero(((fn y => pred y)1)) then succ else pred) ((fn y => (pred(pred y)))8)) 4
A Variation: Type-Checkers
A simple variation on the interpreter exercises (which I sometimes give as one question on a take-home exam) is to ask the students to take a set of formal type-checking rules for an explicitly-typed version of PCF, and write a type-checker for the language. Type checking rules are written with respect to a static environment E which specifies the types of all free variables. Thus if test and count are free variables then E = {test: bool, count: int} indicates that test has type bool, while count has type int. The notation E ⊢ e: T indicates that expression e has type T in the context in which all the typing assertions in E hold. The following are a few typical type-checking rules: Tp Fcn App
E ⊢ e1 : A → B E ⊢ e2 : A E ⊢ (e1 e2 ): B
Tp Fcn Def
E ∪ {x: A} ⊢ e: B E ⊢ f n(x: A) ⇒ e: A → B
Like the rules for the natural semantics, type rules are written with the conclusion under the horizontal bar and the hypotheses above. Thus the first rule indicates that if e1 has type A → B and e2 has type A then (e1 e2 ) has type B. The second rule is a bit more subtle. It indicates that f n(x: A) ⇒ e (note the addition of a type declaration on the formal parameter) has type A → B if, under the assumption that x has type A, the body has type B. The form of an ML program to do the type-checking is similar to that of the interpreters, though different in details. A more ambitious project might involve type inference rather than type-checking. Having specified type-checking rules as well as the semantics affords the opportunity to discuss what it means for a language to be type-safe. A more advanced class might even go so far as to prove the type-safety of the interpreter with respect to the type-checking rules (though we do not go that far in our course).
5
Conclusion
The implementation of a series of interpreters has been a very useful project for students taking a principles of programming languages course, as it allows them to gain some of the benefits of an interpreter-based course, while retaining the benefits of the concepts-based approach. The specific benefits include: 1. Deeper understanding of the meaning and implementation of programming language features, including parameter-passing and scoping mechanisms. 2. Understanding the important role of formal specifications of programming languages, including syntax, semantics, and type-checking rules. 3. More significant experience in programming in a functional programming language. The design of this project is similar to, though less extensive than, those found in interpreter-based courses such as [4] or even [1].2 . We have also found other interpreter assignments in concepts-based courses. What is most different in our projects from a pedagogical point of view is the use of formal semantics as a specification of the interpreter. The use of formal specifications provides a valuable aid for the precise understanding of what may otherwise be unclear or vague concepts. While we have not explicitly used traditional formal logic in these assignments, the notation and ideas for both semantics and type-checking are borrowed directly from that of mathematical logic. Clearly the concerns of the formalism are also tied directly to the concern for syntax, proof rules, and the semantics of first-order logics. Thus students familiar with these ideas from either mathematics or other computing courses will find this a very natural application of these ideas in the understanding of programming languages. Similar notation is used, for example, in specifying reasoning about programs using Hoare triples. We have successfully used these assignments over a series of several years in a Junior-Senior level course in Principles of Programming Languages at Williams College. In the fall of 1998, the assignments were also used in a somewhat more advanced senior-level/beginning graduate level course at Princeton University. In each case, students generally were very successful in completing the assignments and learning the underlying concepts. A web page with the assignments and access to the materials provided to students can be found at http://www.cs.williams.edu/∼kim/interps. References [1] H. Abelson and G.J. Sussman. Structure and Interpretation of Computer Programs. MIT Press, 1985. 2 Special thanks to Jon Riecke for suggesting this project and writing the parser.
[2] K.B. Bruce, P.G. Kolaitis, D.M. Leivant, and M.Y. Vardi. Panel – logic in the computer science curriculum. In Proc. 29th ACM Symp. on Computer Science Education, pages 376–377, 1998. [3] ACM/IEEE-CS Joint Curriculum Task Force. Computing Curricula 1991. ACM Press, 1991. [4] Daniel P. Friedman, Mitchell Wand, and Christopher Haynes. Essentials of Programming Languages. MIT Press, 1992. [5] Carlo Ghezzi and Mehdi Jazayeri. Programming Language Concepts, 3rd edition. Wiley, 1998. [6] Carl A. Gunter. Semantics of Programming Languages: Structures and Techniques. MIT Press, 1992. [7] Samuel N. Kamin. Programming Languages. Addison-Wesley, 1990. [8] Kenneth C. Louden. Programming Languages: Principles and Practice. PWS, 1993. [9] Terrence W. Pratt and Marvin V. Zelkowitz. Programming Languages: Design and Implementation, 3rd edition. Prentice-Hall, 1996. [10] Robert W. Sebesta. Concepts of Programming Languages, 3rd edition. Addison-Wesley, 1998. [11] Ravi Sethi. Programming Languages: Concepts and Constructs, 2nd edition. Addison-Wesley, 1996.
