Using Scenarios to Identify and Trade- off Dependability Objectives in ...

Using Scenarios to Identify and Tradeoff Dependability Objectives in Design George Despotou, John McDermid, Tim Kelly

Overview z Dependability & Dependability Cases z Acceptable Dependability z The Battlefield Communications System z Scenario based elicitation z Selection of Design Alternatives z Summary

2

Dependability z Range of definitions „ Concerned with undesirable consequences of behaviour „ Consists of a number of attributes ‹Safety, security, maintainability, availability etc.

z Attributes resemble ‘non-functional’ requirements „ Cannot effectively separate concerns „ Satisfaction of a dependability attribute can result in addition of further functionality which affects other attributes ‹E.g. fault recovery for safety

z Attributes of Dependability „ Overlap „ Ranked differently and subjectively by the stakeholders 3

Dependability Cases z A clear and defensible argument that a system is acceptably dependable in a given operational context „ Based on proven concepts of safety case development „ Our approach is based on the Goal Structuring Notation

z Each attribute requires different types of evidence/ argument, and different expertise to evaluate „ Attributes can be interrelated ‹In conflict or in harmony

z Clarity of the Argument „ Need separation of concerns „ Systematic approach to addressing different attributes 4

Modular Approach to Dependability Cases z Separate attribute concerns, so far as possible „ Modular ‘case’ approach

z Arguments interrelated through trade-off argument Spinal

Dependability Specification Argument

Spinal

Safety Argument

Spinal

Security Argument

Context

Trade-off Argument

5

Spinal

Performance Argument

Acceptability of Dependability Attributes z Requirements may be in conflict „ Conflicts result in trade-offs „ Least-worse compromise

z Flexibility in expression of dependability requirements „ Identification of the acceptable region for (goal) trade-offs „ Impact on the operation of the system

z Introduction of bounds in GSN „ Specification of target and limit values for (dependability) goals

z Elicitation of goals using scenarios „ Capture of operational context „ Consequences of deviation from target operation 6

Acceptable Dependability - Elicitation describes/context of

is justified

Bounds Rationale

provides

based

Dependability wrt Goals

Deviation

from

Consequences

in terms of

is required

Tolerance based

Objective

achieved

Utility/ Benefit

fulfils

Scenario Scenario

7

Criticality wrt

Assets

The Battlefield Communications System (BCS) z A collection of subsystems communicating and providing data interfaces to other applications z Various sizes „ Can be whole vehicles (central routers and HQ systems) „ Fitted into vehicles, aircraft, ships „ Small devices carried by troops

z Various ways of communicating „ Microwave links, RF links, cables

8

BCS Operation zMany roles „ Non-operational (e.g. maintenance) „ Peacetime operation „ Wartime operation „ Training etc.

zEach role can have different scenarios of use „ Reconnaissance of battlefield „ Reconnaissance of enemy units „ Search and Rescue „ Disassembly and transportation of equipment „ Each scenarios consists of a set of functions provided by the system 9

Top Level Dependability Argument GD System is acceptably dependable

SGD Argument over anticipated scenarios

zExample dependability argument

S_Roles System's operational role

„ First need to consider what the system will be used for (Argument over anticipated scenarios)

Scenarios System scenarios

zScenarios enable: G_SR

G_BR

System is acceptably dependable for a 'Search and Rescue' mission

System is acceptably dependable for routine battlefield reconnaissance

10

„Identification of context „Understanding of attributes (shown later)

Scenarios - Overview z Scenarios allow the stakeholders to describe the envisaged use of a system „ Operational objective „ Constraints „ Rationale

z Patterns for scenario definition „ capture the target requirements and their rationale „ Importance for each scenario identified and used in trade-off

z Two types of scenarios „ Super-scenarios: Based on concept of operations (CONOPS) ‹ e.g. Rescuing requires instant deployment in order to increase levels of survival

„ Micro-scenarios: Within super-scenarios identify scenarios that challenge achievement of dependability attribute ‹ e.g. Possible failure of transmission to SR team (availability) 11

Super Scenarios - Steps 1. State stakeholders and system entities participating „ Rescue team, people/entities to be collected, commander

2. Describe the end objective of the scenario „ The deployment of a team within a 100km radius to find, recover and bring back people and/or the object(s) being rescued

3. Identify utility of the scenario „ Saving life(lives) or collecting a valuable object

4. Describe functions required in the scenario „ Deploy and start-up system, update digital map,command etc.

5. Identify constraints on the scenario and functions „ Team should be sent ASAP to maximise chances for survival „ Communications should not reveal the location of the team

12

Top Level Dependability Argument (2) G_SR System is acceptably dependable for a 'Search and Rescue' mission

Sc_Attr Attributes of concern in scenarios

S_G_SR Argument over attributes of concern

G_Att_Sec

G_Att_Avail

G_Att_Safe

G_Att_Perf

System is acceptably secure for SR mission

System is acceptably available for SR mission

System is acceptably safe for SR mission

System is deployed in acceptably timely manner in SR mission

C_AvailRisk Causes of communications loss

SAvail Argument over situations of service disruption

use of micro-scenarios 13

SSafe Argument over situations leading to loss of life

C_SafeRisk People at risk

micro Scenarios - Steps 1. Identify stimulus „ What happens when the main network router fails during the SR mission?

2. Identify operational context of stimulus „ SR team must be able to communicate if necessary (e.g. in danger, for pickup)

3. Categorise micro scenario under a super-scenario according to operational context „ All critical services during the mission should be available

4. Identify appropriate response „ Discovery of a different source providing functionality

5. State impact of the scenario on system „ Mechanism to detect and repair unavailability „ Additional systems required to perform this function

14

G_Att_Avail Required services/functions for SR mission are available

SAvail Argument over situations of service disruption

Dependability Argument (Target and Limit) C_AvailRisk Causes of communication loss

SSafe

People at risk

Argument over situations leading to loss of life

TimelyDeployment SRScenario

SysFuncAlloc

Definition of SR mission critical services

Systems participating during the SR mission

SSerAvail Argument over availability of critical services

PickLocServcAvail 'Pick up location' services are acceptably available

System is acceptably safe for SR mission

C_SafeRisk

SerAvail Disruption of critical services has been addressed

G_Att_Safe

Situation of not deploying the system in timely manner has been addressed

STimelyDeployment TargetPickLocAvail

TargetSysDeploy

Sevice should be 100% available for the duraiton of the mission

Operational within 1hr from notification of mission

LimitPickLocAvail

LimitSysDeploy

Service should not be disrupted for more than 5 mins

Operational within 2hrs from notificaiton

15

Argument over deployment (time) characteristics of the involved systems

SysDeploy All involved systems are operational in a timely manner

Design and Dependability Goals z Design options featuring different characteristics have different impact on goals z Two design options: „ Design A: Replicate systems providing functionality for the SR mission ‹Consequences: ÖPositive impact on availability ÖNegative impact on deployment time

„ Design B: Replicate systems with critical functions and have as ‘hot standby’ the rest ‹Consequences: ÖPositive impact on deployment time (less components) ÖNegative impact on availability (of non-critical functions though) 16

Design and Dependability Goals (2) z In this case satisfaction of both deployment time and availability goals is possible (within targets and limits given) for both design options

z Assumption for Design B: standby components can be operational within 5 mins – i.e. max of 5 mins ‘unavailability’ z Therefore forced to decide between: „ Design A: Better availability than B, Worse deployment time than B „ Design B: Worse (but still potentially acceptable) availability than A, Better deployment time than A

z Trade-off argument required to resolve: „ Ranking of attributes, acceptability of compromise w.r.t scenario

z Chosen option in this example: Design B „ More timely deployment (close to deployment time target) considered to be higher priority than improving availability (close to availability limit) 17

Trade-off Argument Pattern C1

C2 System Definition

C3

G1

Chosen Design Option

{Design Option} is most acceptable choice for {System} operating in {Scenario} amongst the alternatives available

C4 Alternative Design Options

Scenario Definition

G3 G2 {Design Option} is the optimal choice for {System} operating in {Scenario} amongst the tolerable alternatives

{Design Option} is a tolerable choice for {System} operating in {Scenario}

C7 Tolerable Design Alternatives (Subset of C4)

C8 S2 Argument by appeal to dependability goals identfiied for scenario, est. satisfaction of these goals by design option, and consequent impact on scenario objectives

C5

S1 Argument over dependability goals identified for scenario

Dependability goals identified for scenario (n of)

n C6 Limit identified for {Dependability Objective}

G5

G4

{Design Option} offers best utility for {System} operating in {Scenario} with regard to estimated satisfaction of ranked dependability objectives

{Design Option} will result in achievement against {Dependability Goals} that is equal or better than identified {Limit}

18

Estimated satisfaction of dependability goals possible with Tolerable Design Alternatives

C9 Dependability objective ranking as determined through consideration of overall impact on scenario

Trade-off Argument Instance C1.1

C2.1 BCS System Concept

Definition of Design B

G1.1 Design B is most acceptable choice for BCS operating in SR mission amongst the alternatives available

C3.1 Definition of Search and Rescue Mission

C4.1 Alternative Design Options - only Design A

G3.1 G2.1

Design B is the optimal choice for BCS operating in SR Mission amongst the tolerable alternatives

Design B is a tolerable choice for BCS operating in SR Mission

C7.1 In this case both Design A and B are tolerable alternatives C8.1

S2.1 Argument by appeal to dependability goals identfiied for scenario, est. satisfaction of these goals by design option, and consequent impact on scenario objectives

C5.1

S1.1 Argument over dependability goals identified for SR Mission

Dependability goals identified for SR Mission: Deployment Time and Availability

Estimated satisfaction of dependability goals: Design A = High Availability, Long Deployment Time, Design B = Low Availability, Short Deployment Time C9.1

C6.1

G4.1

Availability goal limit is 5 mins

Design B Availability is equal to identified limit

G5.1 Design B offers best utility for BCS operating in SR Mission with regard to estimated satisfaction of ranked dependability objectives

C6.2

G4.2

Deployment Time goal limit is 2 hours

Design B Deployment Time is better than identified limit

G6.1 Shorter deployment time of Design B is more favourable than higher availability of Design A when considered against the Search and Rescue scenario objectives

19

Deployment Time ranked more highly than Availability when considered against SR Mission objectives

Design and Dependability Goals (3) z Argument and design evolve in parallel „ Argument used to evaluate evolving design „ Design used to structure evolving argument

z Target and (more importantly) limits cannot always be satisfied „ Results in: ‹Re-evaluation of the requirements ‹Re-evaluation of the design

z Example given was relatively straightforward „ More difficult when there are many goals ‹Need of a structured trade-off process

20

Summary z Conflicts between dependability attributes result in trade-offs z Stakeholders need to be able to trade-off requirements „ Scenarios provide context „ Elicit targets & limits for dependability goals

z Selection based on operational consequences of compromise z Ongoing work: „ Scenario Deviation Analysis ‹Based on principles from FHA and PSSA ÖAnalysis based on use of guidewords (early, late etc.)

„ Parallel evolution (and interactions) of argument and design „ Definition of GSN argument patterns for justification of trade-offs 21