Formal and Informal Speci cations of a Secure System ... - CiteSeerX

Formal and Informal Speci cations of a Secure System Component: rst results in a comparative study J. S. Fitzgerald1, T. M. Brookes2, M. A. Green2, P. G. Larsen3 1 Centre for Software Reliability, University of Newcastle upon Tyne, NE1 7RU, UK 2 British Aerospace (Systems and Equipment) PLC, Plymouth, UK 3 IFAD (The Institute of Applied Computer Science), Odense, Denmark

Abstract. This paper presents initial results in a comparative study

of formal and conventional techniques in the design of a secure system component: a trusted gateway. The operation of a trusted gateway is brie y introduced. The industrial context of its development is described, as is the form of the experiment. So far, part-formal and conventional design speci cations have been produced for the trusted gateway from a common informal requirements document. As part of this process, queries have been raised against the informal requirements. These have been carefully logged, and form the subject of a preliminary analysis presented here. These rst results suggest that the use of a formal speci cation language (in this case VDM-SL) leads to an an increased number of queries, and a bias in the speci er's concerns towards data rather than design issues.

1 Introduction British Aerospace (Systems and Equipment) (BASE) produces a number of security-critical systems. The use of formal methods in the development of these systems is not mandatory, but to attain the higher evaluation levels [SEC91], certain aspects of the design should be modelled formally. For example, there is a requirement at the E5 level of assurance for the security policy enforced by the product to be described formally. At E6, formal development with proofs is required. As part of its process improvement programme, BASE are conducting an experiment to develop a trusted gateway (network guard) to a high level of evaluation using CASE technology. Conformance between the implementation and security policy is demonstrated by showing how the simple security policy described in the customer requirement is embodied in each stage of the implementation process and by demonstrating that tests for the security policy can be passed by the implementation at each stage of development. It should be stressed that this is an experimental development: there is no intention of attempting to gain an evaluation certi cate with the programme as described. A full development process with all required documentation would be needed for this.

The gateway's limited functionality makes it ideal for testing formal methods against current design practices. Following an initial case study at Newcastle University, BASE was awarded a contract under the European Systems and Software Initiative to undertake an experimental development of a trusted gateway using formal and conventional techniques in parallel. Comparing the two development processes will yield insights into the costs and bene ts of applying formal techniques in design of such security-critical components. The collaborators in the study are BASE, the University of Newcastle upon Tyne and the Institute of Applied Computer Science (IFAD), Denmark. In this paper, we give a brief description of the gateway's functionality (Section 2) and the form taken by the experimental parallel development, which is reaching the end of its rst phase (system speci cation, see Section 3) at the time of writing. More detail on the structure of the project can be found in [BGFL94]. Initial results of the requirements analysis process are presented in Section 4.

2 Trusted Gateways A trusted gateway is a device located in the communications path between systems of dierent security levels. It sorts incoming messages into dierent classi cations and ensures that they are sent to a destination of the correct security level (Figure 1). For the purposes of our experiment at BASE, the trusted gateway being developed is a simple device consisting of a single input and two outputs. Messages are read into the gateway, are analysed for the presence of character strings which indicate the message classi cation and then written to the appropriate (high or low security) output port.

High Security System High Security System

Trusted Gateway Low Security System

Fig. 1. System Function of the Trusted Gateway A number of exception conditions must be considered in the design. For example: { what does the system do with any characters which are read in but do not form a part of a message? { what happens if the message is longer than a pre-de ned number of characters?

Formal speci cation techniques allow the system to be modelled at various levels of abstraction, aiding the detection of these exception conditions and analysis of the proposed system's behaviour when they arise.

3 Parallel formal and conventional development of a trusted gateway 3.1 Objectives

The primary objective is to determine whether and where the system design process from requirements capture to software development and test can be improved by introducing formal methods. Parameters used to make this judgment include the eort expended in the design and evaluation processes and the number of customer requirements which are satis ed on the rst design iteration. An important constraint on the use of formal techniques is that it should be in the context of an existing design methodology already certi ed to ISO9000.

3.2 The Experiment

The experiment consists of the parallel development of a trusted gateway by two separate design teams. Independence is enforced by ensuring that the teams work in physically separate areas and are instructed not to discuss the project with each other. One team uses the standard BASE development methodology, Ward and Mellor [WM85], supported by the Teamwork Computer Aided System/Software Engineering tool set4. The other team follows a similar design methodology, but uses formal speci cation to describe at least the function enforcing the security policy implemented by the gateway. A central authority acts as the customer, keeping records of the queries, problems and progress made on the project. The members of the design teams have similar backgrounds and are estimated by BASE to be equally quali ed.

The Design Process The experiment follows the design process from the cus-

tomer requirement through system design to development and implementation. The three main development phases, as they aect the trusted gateway, are described below. At the conclusion of each stage, a formal design review is held on each speci cation, and a separate comparative review is held by the central authority. No feedback is performed, but exposed de ciencies are recorded in the experiment design log. The phases of the project embody a distinction, common in aerospace systems development, between systems development and software development activities. The system design phase (Phase One) is essentially independent of the implementation medium. It is conducted by engineers who are systems rather than computer specialists. The system design is subsequently passed to a teams of specialist software and hardware engineers for realisation. 4 Teamwork is a Registered Trademark of Cadre Technology Inc.

Phase One: System Design Phase One commences with the initial customer requirement arriving with the design teams. This document consists of a collectn of requirements in English covering many aspects of the system, at various levels of abstraction. The following are examples:

3.1.3 The system shall be written in a high level language :::

3.1.11 The maximum number of characters in a block of text delineated by valid start and stop sequences shall be 10,000 characters :::

3.8.1 A facility to detect the removal of external power shall be provided The phase ends with the production of the System Speci cation, which will form the basis of the software design, and the System Test Plan, which will be used as the nal customer acceptance document. Clearly this phase involves a great deal of negotiation between the design teams and customer. Queries on the requirements are submitted to the customer and responded to in writing. As queries are received, they are logged against the requirement using the RTM requirement traceability tool5, and clarifying text is added to the requirement where needed. This logging process allows us to trace the evolution of the nal system speci cation from the initial customer requirements, despite the divergence in design due to the teams' diering interpretations of the original requirements and dierent queries raised. The number of requests and their complexity are used as a measure of how well the requirements capture process forces the system designer to clarify the requirement with the customer (see Section 4.2). At the time of writing, the system speci cation has been completed in both development paths. Following the review of each of the documents produced, a change in the requirements, unanticipated by the design teams, will be introduced to assess the eort required to implement the change. Phase Two: Software Design The second phase commences with the submission of the system documentation to the software department for design. Two new independent design teams will be employed, one using the conventional speci cation and one the formal speci cation. The customer in this case is represented in each path by the system engineer who produced the system documentation. Again, queries and responses on the documents will be logged for later analysis. A subset of the required BASE documentation, sucient for software implementation, will be produced during the second phase. The primary documents to be produced are a Software Speci cation and Test Plan. Phase Three: Implementation and Testing In the third phase, the software design documentation will be used by a third team of engineers to produce prototype code. The special-purpose hardware required for a secure implementation 5 RTM is a Registered Trademark of GEC Marconi, Addlestone, Surrey

will not be available, so the code will be run on a suitable computer, with common interface software for the two implementations. These will be tested and debugged on the basis of the teams' own test speci cations. Final tests will include running each team's test plan on the other's implementation. The customer will perform nal acceptance testing.

Development Techniques and Tools The formal speci cation path will use

VDM-SL in conformance with the ISO/BSI Draft Standard [ISO93], supported by the IFAD VDM-SL Toolbox [LL91], which provides syntax and type checking, execution and pretty-printing facilities for VDM-SL. The extent to which formal speci cation is used is largely up to the engineers following the formal methods path. The only \given" is that the security policy implemented by the gateway should be described formally to meet the E5 assurance requirement. The mixture of levels of abstraction in the informal user requirements also makes it unlikely that the engineer will model all the details, since that would compromise the level of abstraction of the formal model, for comparatively small gains in coverage. In the later stages of development, more concrete models will be produced, but we do not anticipate the production of mathematical data rei cation or operation modelling arguments. Since the project is of modest size and resources, we have not stressed formal proof at any point. However, we expect informal re nement arguments to be formulated in the later stages, to show conformance with the formal description of the security-enforcing function.

Training and Support A one-week course covering VDM-SL and the IFAD Toolbox was provided to the engineer who was to prepare the requirements speci cation, as well as for the engineers involved in the design, monitoring and quality assurance activities. During the development, engineers using VDMSL had access to consultants who are expert in the speci cation language and tools. These consultants, from Newcastle University and IFAD, also participate in the design reviews. Both teams have equal access to support for the design methodology and associated tools.

4 Initial Results Ultimately, the following points will be used to compare the two design processes: 1. The eort required to perform the design task; 2. The number and complexity of the queries raised against the requirement speci cation; 3. The number and seriousness of the de ciencies identi ed during the design reviews; 4. The number of identi ed inconsistencies detected within the customer requirement;

5. A comparison of the compliance of the system the original speci cation; 6. After implementation, the performance of the system and tests in terms of code size, speed, accuracy and test coverage will be made. Work on the system speci cation is now complete, but has not progressed to the test plan or subsequent review. We therefore address the rst two points in Sections 4.1 and 4.2.

4.1 Design Eort A comparison of person-hours shows that the initial system design took approximately 25% more eort using formal speci cation than using the BASE methodology alone. This comparison does include the additional training in VDM-SL and the IFAD Toolbox. The engineer considered that the experience and skill he had gained by the end of the design process would have saved most of the excess design time. Some element of overhead is nevertheless expected: more work is required to analyse system requirements to a level of rigour sucient to permit formal modelling. This additional overhead in the early stages should be oset by the better understanding of the problem obtained by the designer which will lead to fewer problems being encountered in later phases of the programme. Future results will con rm or deny this expectation.

4.2 Analysis of queries raised against requirements Queries raised by the two system design teams were logged. A comparison of the records is informative. For example, the following questions were recorded from the formal development path against one of the clauses shown in Section 3.2 above (\The maximum number of characters in a block of text delineated by valid start and stop sequences shall be 10,000 characters"): 3.1.11 Does the maximum size of the message, of 10,000 characters, include the start and end functions? 3.1.11 What do you as the customer want to happen if a message is received that has more than 10,000 characters? It is worth noting that no questions were raised against this clause by the developer following the conventional path. Dierences between the engineers following the two design paths (their past experience, for example) is a factor not included in the results presented below. They should therefore be treated with some caution. However, some clear trends have emerged. The engineer using the standard design methodology submitted approximately 40 questions during the design phase. In comparison, the engineer following the formal design path submitted approximately 60 questions. A major topic of interest is the subject matter of the queries, so the questions were given to ve other engineers drawn from the systems, software and quality departments who were each asked to assign each query to one of the following categories:

Function: clari cation of the function the system has to perform; Data: clari cation of data, data types or data structures; Exceptions: to determine desired system behaviour in a particular condition; Design Constraints: to clarify design constraints. The source of the question (i.e. which design path it originated from) was not revealed to the engineers performing the categorization. The questions were of a uniform complexity, and the engineers' classi cations were averaged to give the results shown in Figure 2. The numbers in the table have been normalized over the total number of questions raised to show the percentage of questions which occurred in each category. These data are shown graphically in Figure 3. The distribution of the questions within each category by source is shown in Figure 4.

Function Data Exception Conditions Design Constraints

Conventional Path Formal Design 60 42 10 31 8 14 22 14

Fig. 2. Distribution of questions across categories for each source There are signi cant dierences in the pattern of the two responses. The formal methods approach stresses data modelling more than the conventional approach (by a ratio of almost 3:1 in Figure 4). Exception conditions omitted or unclear from the customer's requirements speci cation are also apparently identi ed. There is much less emphasis on identifying design constraints, which is seen as a very positive attribute at this early stage in the development cycle. The majority of the constraints are to do with actual implementation rather than identifying the logical equivalent of the required system. Such queries more properly belong further along the development cycle. It should still be stressed that some of the dierences in these results are caused by the dierences in the engineers performing the tasks. Much more data (collected on several parallel activities) will be needed before this bias can be eliminated entirely.

4.3 The use of formal speci cation Teamwork provides a diagrammatic notation for the functional decomposition of the system, which the engineer used as a basis for deciding what parts of the system to specify formally. He has successfully speci ed the input analysis component of the system as well as the security-enforcing function.

% Questions in category

80 70

Formal 60

BASE

50 40 30 20 10 0 Function

Data

Exception

Des. Con.

Fig. 3. Distribution of questions across categories for each source

Formal

BASE

% Questions in category

100 80 60 40 20 0 Function

Data

Exception

Des.Con

Fig. 4. Percentage of questions in each category by source

Having to write the speci cation in a formal notation did not pose a major problem for the engineer. The syntax of the language was checked quickly by the tool and the correctness of the speci cation with respect to informal requirements was validated by executing the speci cation on simple test cases (cf. the notion of validation conjecture in [BFL+ 94]) and using the Toolbox's facilities for test suite construction and coverage analysis. A tendency was noted towards the use of familiar constructions instead of the most appropriate or concise form. This problem would reduce in time as the language and its use become more familiar to the engineer. External consultants aid this process by giving the engineer feedback on the style of speci cation being employed. 1.0 state ALPHA of .1 asciicharacters : char .2 somf : char .3 body : char .4 eomf : char .5 ValidMessage : char .6 inv mk-ALPHA (- somf body eomf ValidMessage ) 4 .7 ((((len somf ) + (len body ) + (len eomf ))  (10000)) ^ .8 (((len somf )  (3)) ^ ((len somf )  (10))) ^ .9 (((len eomf )  (3)) ^ ((len eomf )  (10))) ^ .10 (ValidMessage = conc ([somf body eomf ])) ^ .11 (len ValidMessage  10000) ^ .12 (: occurrence (somf body ) ^ : occurrence (eomf body ))) .13 init ALPHA 4 s = mk-ALPHA ( ) .14 end ;

;

;

;

;

;

;

;

:::

Fig. 5. Part of the formal speci cation of input analysis Figure 5 shows part of the engineer's speci cation of input analysis expressing requirement 3.1.11, including the fact that the start and end strings are included in the 10000 character limit (Line 1.7), following the answer to the query raised against the requirement.

4.4 Use of Tools Without prior experience, engineers in both development paths found that they were unsure what they were actually trying to achieve using the requirement analysis and traceability tool (RTM) in particular. This problem indicates that training in this area needed to concern itself more with the principles and the methodology rather than the mechanics of using the tool which was seen to be acquired more quickly. All the students on the VDM-SL course attained reasonable facility with the IFAD Toolbox, even though it involved the use of an unfamiliar editor (Emacs).

The engineer in the formal design path suggested improvements to the manual, provision of worked examples illustrating the use of language constructs and more informative error messages.

5 Conclusions The Trusted Gateway experiment is not intended to show the absolute worth (or otherwise) of formal methods. Rather, it is hoped to add something to the public body of evidence on the costs and bene ts of applying these techniques, as well as uncovering lessons which can be learned in conducting future experiments of this nature. Experiments involving human beings are, of course, subject to many factors, such as the backgrounds and attitudes of those involved, which aect our ability to draw conclusions from them. For this reason, we have sought only to identify gross features in the data obtained so far, and are taking the opportunity to report early in the experiment in order to elicit opinions on the form and content of the experiment. From the work to date, BASE have concluded that formal speci cation should be used where a precise de nition of data and function is required (where a simple function is needed but it is vital that it be implemented correctly), e.g. the kernel of the trusted gateway which forms the basis for the security enforcing function; and where complex functionality is involved, e.g. the input processing in the Trusted Gateway where the data to be treated is made of several components and there are many exception cases to be considered. The use of formal speci cation was not considered advantageous where the required function can be written down easily. An example is the requirement that the memory be purged after the message has been output. Other functions which do not bene t are the initialisation where data is merely read from one source and stored in another for future use. Boswell's description of the formalization of a security policy model in Z [Bos93] shows similarities with our approach. In particular, we note the same \tension between abstraction and `completeness' " in determining how much of the customer requirement is to be written in formal notation. It is also notable that our approach addresses integration of formal techniques into an existing development process in the way suggested in his conclusions: by using an informal technique for recording detailed requirements in conjunction with formal description of abstract requirements. The preliminary results of the experiment suggest that the use of formal speci cation makes an appreciable dierence in the early stages of the development process. Whether the same modest application of formal techniques can help in implementation is our next area of study. Acknowledgments The authors acknowledge the support of the British Aerospace Dependable Computing Systems Centre at the University of Newcastle upon Tyne, and the European Commission (ESSI Grant 10670) for supporting the experiment. JSF gladly acknowledges the support of the United Kingdom EPSRC under the terms of a Research Fellowship.

References [BFL+ 94] J. C. Bicarregui, J. S. Fitzgerald, P. A. Lindsay, R. Moore, and B. Ritchie. Proof in VDM: A Practitioner's Guide. FACIT. Springer-Verlag, 1994. ISBN 3-540-19813-X. [BGFL94] T. M. Brookes, M. A. Green, J. S. Fitzgerald, and P. G. Larsen. A Comparison of the Conventional and Formal Design of a Secure System Component. FACS Europe, March 1994. [Bos93] T. Boswell. Speci cation and Validation of a Security Policy Model. In J. C. P. Woodcock and P. G. Larsen, editors, FME'93: Industrial-Strength Formal Methods, volume 670 of Lecture Notes in Computer Science. SpringerVerlag, 1993. [ISO93] ISO. Document Number ISO/IEC JTC1/SC22/WG19/N-20 Information Technology Programming Languages { VDM-SL First Committee Draft Standard CD 13817-1, November 1993. [LL91] P. G. Larsen and P. B. Lassen. An Executable Subset of Meta-IV with Loose Speci cation. In VDM '91 - Formal Software Development Methods. SpringerVerlag, October 1991. [SEC91] Oce for Ocial Publications of the European Community. Information Technology Security Evaluation Criteria, June 1991. [WM85] P.T. Ward and S.J. Mellor. Structured Development for Real Time Systems, volume 1,2,3. Yourdon Press, Prentice-Hall, Englewood Clis, NJ, 1985.

This article was processed using the LaTEX macro package with LLNCS style