Design and Development of Dependability Case ... - CiteSeerX

Design and Development of Dependability Case Architecture during System Development

Georgios Despotou, Department of Computer Science, University of York, UK Tim Kelly, Department of Computer Science, University of York, UK Abstract A dependability case communicates an argument that a system will acceptably operate with respect to the stakeholders’ envisioned dependability requirements. We describe the use of the Goal Structuring Notation (an argumentation notation and methodology) for dependability case development to establish arguments supporting claims of achievement of the dependability requirements. At each stage of the evolution of the safety case, the dependability argument is expressed in terms of what is known about the system being developed, providing at the same time feedback to the design process. As design knowledge increases during the project, the goals (and the corresponding arguments) can be expressed in increasingly tangible and specific terms. A case can consist of a number of self-contained GSN argument modules each of which reasons about an individual aspect of the system. The resulting dependability case architecture is structured according to how the argument modules are associated during the evolution of the case. This paper presents how a dependability case can be architected throughout the lifecycle of a system, using GSN contracts to capture the associations between the argument modules of the case. Furthermore, the paper introduces the dependability profile, a concept used to facilitate structuring the dependability arguments. The dependability profile is created during the analysis of the system and describes acceptable behaviour of the elements in the system model. The paper uses the UK defence architectural framework MODAF to illustrate the links between system modelling and dependability case architecture. Introduction Similarly to a safety case, a dependability case is a device for communicating ideas and information, usually to a third party. In the example of the safety case this can be a regulator or certification authority. Although dependability cases are not a requirement in the same manner as safety cases – For example the UK Defence Standard 00-56 (ref 1) explicitly asks for a safety case to accompany a system – constructing an argument about the dependability properties of a system, such as safety availability and performance, contributes in establishing assurance about the overall dependability behaviour of a system. The Goal Structuring Notation is used in a dependability case to structure the argument about acceptable fulfilment of the system’s dependability goals. GSN goals are specific claims that a system has achieved a particular requirement (ref 2). Being able to explicitly represent and associate all the elements of an argument, GSN helps to articulate post-conditions for the initially identified requirements of the system in question. At each stage of the evolution of the safety case, the dependability argument is expressed in terms of what is known about the system being developed. For example, at the early stages of project development the dependability argument is related to the highlevel objectives as conceived by the stakeholders in the concept of operations (CONOPS), whereas in later stages is related to detailed behaviour of system components or subsystems. GSN allows the creation of modules, which are self-contained arguments, providing a number of benefits to the dependability case (ref 3). Argument modules can focus on a number of different aspects of the system development including not only arguments regarding the system as artefact (product arguments), but also arguments regarding the development process (process arguments). The final dependability case is composed by combining the appropriate argument modules during system development. Development of the Dependability Case Evolving in parallel to the system, goals are decomposed until they can be directly supported by evidence collected during the development and testing phases of the system. Previous work from the authors proposes three methodologies which collaborate during the evolution of the dependability case. The participant methodologies collaborate during the evolution of the dependability argument, which itself is based on the Goal Structuring Notation (GSN) methodology and notation. Similarly to the twin peaks paradigm (ref 4), evolution of the system and the argument is an iterative process, which involves making decisions about the architecture and the design of the system that need to be justified and documented. Interaction between the argument and the design process exists during the evolution of the system. The evolving argument should serve to evaluate the design’s fitness to satisfy the stated dependability goals. A design that directly addresses the stated goals will result in a strong argument.

If the stakeholders involved in the development of the argument deem that the argument is not satisfactory, changes to the design will have to be made. The argument of a dependability case can constitute the interface between evolution of requirements and design, providing a systematic way of documenting, tracing and reasoning about decisions. System goals

GSN

Feedback

Trade-off argument

System requirements & associated context

TOM

Alternative decisions in design/argument evolution

Tolerability of requirements (bounds)

Design models

Requirements DDA Space

FANDA

Elicitation of requirements

Description of system

Design Space

Figure 1 – Methods Supporting Argument Based System Evolution. Figure1 presents the collaboration between the methodologies to support argument based co-evolution of design and requirements, by showing the flow of information between them, during system evolution. The methodologies participating in the evolution are the following: •

Dependability Deviation Analysis (DDA) (ref 5): DDA provides a method for the systematic analysis of the design identifying the effect of deviations on the normal operation of the system. In the context of dependability cases, the purpose of DDA is to elicit the required and acceptable behaviour of the system with respect to dependability and to clearly identify associations between failures from the viewpoint of each dependability attribute. The identified behaviour constitutes the basis for defining the GSN goals in the dependability case. Moreover DDA assists in the identification of concerns. Concerns substantiate each of the dependability attributes. The stakeholders of the system recognise a concern because it principally affects their interests regarding the envisioned operation of a system. Finally DDA outputs the failure conditions map, which constitutes a graphical representation of how a failure to satisfy a particular aspect of a dependability attribute can affect the system’s behaviour.

•

Factor ANalysis and Decision Alternatives (FANDA (ref 6): FANDA is a design rationale method that complements GSN in supporting the development of arguments. The method can be thought of as a catalyst between the argument and the design. It assists analysts to examine the system goals, record and manage rationale, and elicit candidate design decisions, which will take place during the system lifecycle. FANDA produces a number of ‘meta’ arguments about how the proposed design alternatives satisfy the dependability goals of the case.

•

Trade-Off Method (TOM) (ref 6): TOM represents a methodical way of establishing bounds of acceptability. These (bounds) constitute a space of admissible requirements in which tradeoffs can take place and conflicts may be resolved. TOM ultimately creates arguments of preference between candidate decisions in the evolution of the system.

The starting point is considered to be the definition of the overall concept system operation – e.g. the definition of the concept of operations (CONOPS) and operational scenarios and roles of a system. This information constitutes the high level model of the system. The models are fed into Dependability Deviation Analysis (DDA) during which they are probed using possible dependability deviations. During DDA analysts identify the dependability attributes of interest regarding the behaviour of the system, and elicit dependability requirements for the system elements identified in the models. The

GSN method is used to define an argument about the overall required dependability behaviour of the system. The goals are used by FANDA to elicit design rationale and identify the candidate design alternatives that satisfy the goals. The identified design alternatives are fed (along with the GSN goals) to the trade-off method, which will facilitate selection of the most suitable alternative. Following the selection of the most suitable alternative, the design space is updated with the models that reflect the design decisions. The new models or system elements can be analysed using DDA, thus starting a new iteration during the evolution of the argument and the system.

Availability of Evidence during the System Lifecycle The argument of a case is that which communicates the relationship between the evidence and goals. Both argument and evidence are crucial elements of the safety case that must go hand-in-hand. An argument without supporting evidence is unfounded, and therefore unconvincing. Evidence without an argument is unexplained; it can be unclear that safety objectives have been satisfied. During dependability case development, using FANDA, participants enter an incremental process of establishing a collection of design rationale ‘micro-arguments’, regarding the association of candidate design decisions with the goals. Identification (of sources) and collection of evidence is an essential aspect of the FANDA process in order to support the arguments about fulfilment of the dependability requirements from the candidate alternatives. However, availability of evidence varies according to stage of the system lifecycle. At the early stages of the lifecycle, availability of evidence is more limited compared to later stages. Booch (ref 7) suggests that defining a system architecture involves “…suboptimal decisions made partly in the dark”. The effectiveness of a design decision with respect to its dependability properties may not be confidently assured until later stages of the system development, which include testing and analysis of the proposed option in more detail. Accompanying design rationale specification, FANDA is also used to identify availability and plan the collection process of evidence that will constitute the basis on which design rationale is stated. Collection of evidence does not only include evidence that will support the position by the arguments representing design rationale. An important part of the collection process is the identification of evidence that may negatively affect the argument; in other words evidence that could undermine the position communicated by an argument. This concept is similar to the requirement of Defence Standard 00-56 (ref 1), which asks for the identification of possible counter evidence that could have a negative impact on the creation of a safety case – and hence certification of the system. Use of GSN Contracts to Structure the Dependability Case Using GSN, a case can be partitioned into modules, each of which constitutes an individual self contained argument. The overall case is defined by aggregating and documenting the relationships between the participating modules; this approach provides a number of benefits (ref 8): 1. 2. 3. 4.

High cohesion: Focus on the particular aspect the argument addresses. Low coupling: Associations between arguments are defined at module level and not at the goal level. Defined and documented interfaces: Interfaces are captured documented and explained using a contract module. Information Hiding: Information about details how a goal is argued may be hidden focusing on the high level requirements described what is argued in a module.

Figure 2 shows how contracts can be used to structure a case. The top level module contains the overall claims about the system. For example, overall claims about the achieved safety levels of the system (safety), or the dependability behaviour of the system (dependability case). Development of the argument contained in the top level module can result into goals that are supported by arguments in other modules (orange and blue goals). Moreover goals within an argument module can provide support to more than one contract; in Figure 2 the bottom module provides support to two contracts. Use of contracts provides flexibility in structuring a case. However flexibility in structuring the case is achieved at the expense of cohesion between the modules. Balancing flexibility and coupling between argument modules is essential to preserve the complexity of the case to reasonable levels. A contract is not merely a collation of the interfaces between the modules; instead they are arguments themselves, arguing about the way in which the parent goal is decomposed.

Two GSN goals of the argument (orange & blue) are supported by arguments in other modules. The goals are solved by contract

GSN contracts document the interfaces between the modules providing support and the module of the parent goal.

GSN contracts are a type of argument themselves, being able not only document but argue about the decomposition

Some of the goals in certain argument (GSN) modules can support goals in other modules. The visibility of a goal, supporting a parent goal in a different module solved by contract needs to be set to public. The supporting goal does not have to be the top level goal of the module. This gives flexibility in structuring the case. Some goals may provide support to more than one line of argument

Figure 2 – Use of GSN Contracts in Structuring a Case Architecting the Dependability Case GSN modules allow the overall dependability case to be represented as a series of interrelated modularised arguments. This can result to a number of different ways according to which the case can be partitioned. Figure 3 shows a possible case architecture in which the modules are structured in line with the dependability attributes. The argument modules are stated in the context of an overall tradeoff argument identifying the conflicts between dependability requirements (specified as goals). The high level dependability argument (called dependability specification argument in this example) is used to identify the overall required system dependability behaviour, which is supported by arguments regarding each of the dependability attributes identified in the high level argument.

Figure 3 – Case Partitioning Based on Attributes

The argument modules about the dependability attributes are stated in the context of a module arguing about the conflicts and trade-offs between the attributes. Although this architecture for a dependability case is appealing there are certain problems that make its use an inappropriate way for structuring the case. Dependability attributes are interrelated; a requirement stated from a perspective of an attribute, the satisfaction of which is claimed (by a goal) at the high level argument may be supported by claims stated from the perspectives of other dependability attributes. Making references between the argument (attribute) modules in Figure 3 will result in complicated and unclear case difficult to be reviewed. In our approach, we structure the dependability case of a system based on the models that represent it (the system). In particular we align the dependability case with MODAF products. MODAF is a defence architectural framework, and is the UK equivalent of the DoD DODAF. In general, defence architectural frameworks are optimised to model systems that are increasingly described as Systems of Systems (SoS) (ref 9). One such example of SoS is the Network Centric Warfare paradigm. The MoD and DoD specification do not provide a definite way for developing the products. However, the DoD deskbook provides guidelines for a data-centric approach, called the activity Based Methodology (ABM) (ref 10). Figure 4 presents the sequence in which the products are constructed following the ABM. ABM is used to maintain consistency between the DAF products, creating integrated architectures by adding detail to existing models, the elements of which can be mapped onto the new models. OV-1 OV-5

OV-6

OV-2

OV-4

OV-7

Operational View

OV-3

SV-4

Systems View

SV-10 SV-1 SV-5 SV-2

SV-

SV-7

SV-6 SV-3

Figure 4 – Evolution of DODAF using the Activity Based Methodology (ABM) An important aspect of the ABM methodology is that during the evolution of the system, the developers follow a set of well specified steps incorporating traceability during development of the models. The addition of detail in each step, results to the evolution of the overall system by defining new models. DAF models using the ABM, define a hierarchy that can be used to trace dependability considerations from the high level concept of operations (CONOPS), to individual systems and functions that constitute the operation of the system. Our proposed dependability case architecture is organised around the hierarchy presented in Figure 4. Figure 5 illustrates the resulting architecture of the dependability case employing GSN contracts for its composition. The case consists of four different types of argument modules: •

High level dependability argument: Argument about the satisfaction of the high level dependability requirements, of primary interest to the system stakeholders

•

Operational view product arguments: Arguments about the satisfaction of dependability requirements of each operational level product. These requirements are derived according to

how the operation of the SoS may affect the dependability attributes of interest to the stakeholders. •

System view product arguments: Arguments about the satisfaction of dependability requirements of each system level product. These requirements are derived according to how the systems may affect the operation of the SoS.

•

Trade-off arguments: Evolving the argument in parallel to the system entails making design decisions out of a pool of specified candidate decisions. It is inevitable, especially in SoS, that selection of an alternative results in trade-offs as some designs may be good in satisfying different goals. The trade-off arguments, product of TOM, provide context to the dependability case, providing an argument that the selected alternative is the most suitable.

Dependability goals stated in the context of bounds (elicited in TOM). The goals are analysed in FANDA identifying factors. Proposed decisions are provided to TOM

High level dependability argument

TOM Argument

The contract modules are defined in the context of the decision based on which the respective MODAF views were created. Analysing the resulting models from the decision in DDA the systems that can contribute in manifesting a concerns are identified and their profile is created. The contract captures these associations between the goals of the parent module and the goals that support it.