Resilient Design for Complex Engineered Systems in the Early Design

AN ABSTRACT OF THE DISSERTATION OF Elham Keshavarzi for the degree of Doctor of Philosophy in Mechanical Engineering presented on February 27, 2018. Title: Resilient Design for Complex Engineered Systems in the Early Design Phase

Abstract approved: ______________________________________________________ Christopher Hoyle

There are many uncertainties that complex engineered systems will face throughout their lifecycles due to changes in internal and external conditions. It is desirable for complex engineered systems to be resilient against various uncertainties. The ability to overcome such uncertainties should be embedded into the system from the beginning. However, there is a lack of applicable methodology that shows how to design a resilient system from the early design phase. This PhD dissertation introduces a new framework to apply resiliency techniques into the system from the beginning in early design stage to enable the system recovers from failures caused by internal or external uncertainties. The first step of this research is to develop an initial functional model in early design stage and simulate the failure behavior of the system. The unique scenarios provide the information on the final behavior of the system assigned to each injected failure. A cost-risk model is developed to compare resiliency of different functional models in design space, and produce a preference ranking. The optimal design is selected based

upon the cost-risk objective. The proposed framework is implemented for the design of a monopropellant propulsion system. The second part of research addressed the validation of the result generated by a selected model in the early design stage. The challenge in the early design stage is that the real system or prototype is not manufactured and there is no or inadequate information available from complete system’s behavior. Several methods are introduced. In the first method, expert knowledge is quantified based on Failure Modes and Effect Analysis to validate the result generated by a selected model in the early design stage. Another method is proposed based on non-parametric comparison of observed data and simulated data for one or more subsystems that currently exist. In addition, a local sensitivity analysis is developed to guide the designers to focus on sensitive parts of the system in further design stages, particularly when mapping the functional model to a component model. The third part of this PhD dissertation addresses the weakness of traditional design methods to cope with external uncertainty throughout the lifecycle of the system. In a traditional design approach, a pre-defined set of requirements based on market studies or user needs is established, and then the optimal design is selected to satisfy the requirements. The traditional approach is inadequate to respond to unexpected changes in initial requirements or operating conditions. To resolve this issue, a new dynamic design framework is developed to make the system resilient against the external uncertainties through the life cycle of the system. In the proposed approach, the flexible design features are utilized as control variables. By adjusting the design features to the optimal forecast obtained from the Kalman filter, the system overcomes external uncertainties. The proposed method is applied to design a dynamic satellite system.

©Copyright by Elham Keshavarzi February 27, 2018 All Rights Reserved

Resilient Design for Complex Engineered Systems in the Early Design Phase by Elham Keshavarzi

A DISSERTATION

submitted to

Oregon State University

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

Presented February 27, 2018 Commencement June 2018

Doctor of Philosophy dissertation of Elham Keshavarzi presented on February 27, 2018

APPROVED:

Major Professor, representing Mechanical Engineering

Head of the School of Mechanical, Industrial, and Manufacturing Engineering

Dean of the Graduate School

I understand that my dissertation will become part of the permanent collection of Oregon State University libraries. My signature below authorizes release of my dissertation to any reader upon request.

Elham Keshavarzi, Author

ACKNOWLEDGEMENTS

This dissertation marks the end of a long journey of intensive study. Two people stand out as having the greatest impact on my success and well-being: My advisor, Dr. Christopher Hoyle, for maintaining a professional demeanor. He is a wonderful human being and an amazing advisor. He is a true gentleman. His calm, wisdom and knowledge were both a model and a medicine for me. He is the nicest person that I have ever met in my life. Being modest in addition to possessing a great set of knowledge, advising techniques and experience, makes him even more stunning advisor. I’m so honored to know him and work for him for 4 years. I would like to acknowledge Prof. Irem Tumer, the most prestigious person I have met in the field of engineering. She provided a role model for being an intelligent and extraordinary female professional in a field dominated by men. She always supported me and showed me the way. She encouraged me to continue, especially when I was disappointed or exhausted. She is a perfect strong leader that everyone can rely on. I am so grateful for her. I would also like to thank my lovely parents, Nasrin and Samad Keshavarzi, my kind husband, and my best friends: Danielle Jackson, Rachel Barton, and Ken Holt who believed in me and supported me. Thank you all!

CONTRIBUTION OF AUTHORS

For the manuscript: Resilient Design for Complex Engineered Systems in the Early Design Phase: Framework and Case Study of a Monopropellant Propulsion System Co-authors: Peter Berg, Kai Goebel, Irem Y. Tumer, Christopher Hoyle For the manuscript: Validating System Fault Behavior Modeled in early Stage design Co-authors: Kai Goebel, Irem Y. Tumer, Christopher Hoyle For the manuscript: Resilient Design Applying Functional Models and Cost-Risk Analysis in Early Design Stage Co-authors: Matthew McIntire, Kai Goebel, Irem Y. Tumer, Christopher Hoyle For the manuscript: Dynamic Design Approach Using the Kalman Filter for Uncertainty Management Co-authors: Matthew McIntire, Christopher Hoyle

TABLE OF CONTENTS Page : Introduction ........................................................................................... 1 1.1

Motivation ................................................................................................................. 5

1.2

Overview ................................................................................................................... 7

1.2.1

Contribution of Objective 1 .............................................................................. 7

1.2.2


1.2.3


: Resilient Design Applying Functional Models and Cost-Risk Analysis in Early Design Stage.................................................................................................. 10 2.1

Introduction ............................................................................................................. 10

2.2

Resilient Design ...................................................................................................... 13

2.3

Failure Analysis and Functional Models ................................................................ 15

2.4

System Representation by Functional Model ......................................................... 18

2.5

Resilient Scoring Function ...................................................................................... 21

2.6

Generate New Designs ............................................................................................ 24

2.6.1

Redundancy and Health Management ............................................................ 24

2.6.2

Change the Order of Functions ....................................................................... 25

2.6.3

Using Unused Flows as Inputs ........................................................................ 25

2.6.4

Combining/Splitting Functions ....................................................................... 25

2.7

Case Study: Monopropellant Propulsion System.................................................... 26

2.7.1

Functions Definition ....................................................................................... 30

2.7.2

Flows Definition ............................................................................................. 30

2.7.3

Modes Definition ............................................................................................ 31

2.7.4

Conditions Definition...................................................................................... 32

2.7.5

Alternative System Designs ............................................................................ 33

2.8

Results ..................................................................................................................... 35

2.9

Conclusions ............................................................................................................. 38

: Validation of Model Simulation Result in Early Design Phase.......... 39 3.1

Introduction ............................................................................................................. 39

3.2

Model Validation Methods ..................................................................................... 41

TABLE OF CONTENTS (Continued) Page 3.2.1

Adequate True Data to Validate Model .......................................................... 43

3.2.2

Little Data to Validate Model ......................................................................... 48

3.2.3

No Data to Validate Model ............................................................................. 50

3.3

Model Validation in the Early Design Stage .......................................................... 52

3.3.1

Apply Expert Knowledge to Validate Functional Model................................ 54

3.3.2

Apply Subsystem Data to Validate Functional Model .................................... 60

3.3.3

Local Sensitivity Analysis of Selected Functional Model .............................. 63

3.4

Conclusion .............................................................................................................. 67

: Dynamic Design Approach Using Kalman Filter for Uncertainty Management 69 4.1

Introduction ............................................................................................................. 69

4.2

Resilient Design Against External Events .............................................................. 71

4.3

Uncertainty in Complex Engineered Systems ........................................................ 73

4.3.1

Statistical (Aleatory) Uncertainty ................................................................... 73

4.3.2

Time-Resolved (Epistemic) Uncertainty ........................................................ 73

4.4

Design and State of Interest .................................................................................... 76

4.4.1

Class (0): Unconstrained Design..................................................................... 76

4.4.2

Class (1): Predictive Design ............................................................................ 77

4.4.3

Class (2): Reactive Design .............................................................................. 77

4.4.4

Class (3): Dynamic Design ............................................................................. 78

4.5

Dynamic Design Approach for Resilient Design .................................................... 78

4.5.1

Kalman Filter .................................................................................................. 80

4.5.2

Design Selection ............................................................................................. 84

4.6

Case Study: Resilient Design for Communication Satellite System ....................... 86

4.6.1

Demand Prediction with Kalman Filter .......................................................... 87

4.6.2

Design Optimization ....................................................................................... 90

4.7

Results ..................................................................................................................... 95

4.8

Conclusion .............................................................................................................. 96

: Conclusions......................................................................................... 98 5.1

Accomplishments and Contributions .................................................................... 100

TABLE OF CONTENTS (Continued) Page 5.2

Future Work .......................................................................................................... 101

5.3

Intellectual Merit ................................................................................................... 102

5.4

Broader Impact...................................................................................................... 103

5.5

Research Funding.................................................................................................. 103

Bibliography ............................................................................................................. 104 Appendices ................................................................................................................ 116 APPENDIX A-1: Functional Definition in Python for Monopropellant system .............. 116 Appendix A-2: Graphical Representation of Monopropellant Functional Model in Spyder .......................................................................................................................................... 117 Appendix A-3: Modes Definition in Python for Monopropellant System ........................ 119 Appendix B: R Code for Chi-Squared Test to Compare FMEA Resaul and Simulation . 127 Appendix C: R Code for Chi-Squared Test to Compare Observed and Simulation ......... 128 Appendix D: Python Code to Find Sensitive Functions ................................................... 129 Appendix E: Kalman Filter Code in Matlab to Forecast Uncertain Event........................ 130

LIST OF FIGURES Figure

Page

Figure 1.1: Design changes cost increasing by design stage ........................................ 2 Figure 1.2: Examples of resilient systems .................................................................... 6 Figure 2.1: Proposed framework for resilient design in early design stage ................ 12 Figure 2.2: Graphical representation of a functional model ....................................... 19 Figure 2.3: General idea to design a monopropellant propulsion system ................... 27 Figure 2.4: End states for a monopropellant propulsion system ................................. 28 Figure 2.5: Basic functional model developed for monopropellant system ............... 29 Figure 2.6: Classification of fault scenarios for one monopropellant design ............. 34 Figure 2.7: Failure simulation results for candidate monopropellant designs ............ 36 Figure 3.1: Selected functional model for monopropellant propulsion system .......... 53 Figure 3.2: A monopropellant propulsion system schematic [114] ............................ 56 Figure 3.3: Parallel redundant regulator and isolation valve [114] ............................ 58 Figure 3.4: Simulation scenarios for selected monopropellant design ....................... 62 Figure 3.5: Observed scenarios for gas regulator in spacecraft engine ...................... 62 Figure 3.6: Most frequent words in simulated failure scenarios ................................. 65 Figure 3.7: Sensitive parts in the monopropellant functional model ......................... 66 Figure 4.1: Iridium and Globalstar satellite systems failed due to market changes ... 70 Figure 4.2: Classification of relationship between Design and SoI ............................ 76 Figure 4.3: Aleatory and epistemic uncertainties in design process ........................... 79 Figure 4.4: Generalized framework of dynamic design approach .............................. 80 Figure 4.5: The ongoing discrete Kalman filter cycle [158] ....................................... 81 Figure 4.6: Steps to obtain optimal state estimation through Kalman filter ............... 84 Figure 4.7: Detailed framework of dynamic design approach .................................... 85

LIST OF TABLES Table

Page

Table 2-1: Generated designs for monopropellant system ......................................... 35 Table 2-2: Scoring function for monopropellant system designs [Millions $] ........... 37 Table 3-1: Model validation methods and assumptions ............................................. 43 Table 3-2: FMEA for monopropellant propulsion system .......................................... 57 Table 3-3: Proportions of each category for expert knowledge and simulation ......... 59 Table 3-4: Observed and simulated proportions for regulating gas ............................ 63 Table 3-5:Methods to validate model selected in the early design stage .................... 67 Table 4-1: Demand forecast for communication satellite system ............................... 88 Table 4-2: Kalman filter assumptions for the satellite system .................................... 89 Table 4-3: Demand estimation for satellite system using Kalman filter .................... 90 Table 4-4: LEO constellation design variables ........................................................... 91 Table 4-5: Example of exhaustive search values for optimal design of satellite system ..................................................................................................................................... 95

1

: INTRODUCTION

Complex engineered systems contain an ever-increasing number of interactions between components and subsystems. Complex interactions can cause unpredictable behavior of the system and undesired consequences. Such systems are high in cost and complexity, and often incorporate highly sophisticated materials, components, design, and technologies. There are many uncertainties such systems will face throughout their lifecycles due to changes in internal and external conditions. It is desirable for complex engineered systems to be resilient against various uncertainties. The ability to overcome such uncertainties should be embedded into the system from the beginning, in early design phase. Conceptual design is an early phase of the design process. In early design phase, the broad outlines of function and form of the system are articulated. It involves problem definition, brainstorming, feasibility study and establishing design requirements. Conceptual design includes the designing functions, interactions, and attributes of the system and determining how to meet the requirements. Common artifact of the conceptual design is developing a model of the system. The preliminary design fills a gap between conceptual design and detailed design, particularly when the conceptual design does not provide sufficient detail in the early

2

design phase is not sufficient for full evaluation. In the preliminary phase, the general structure of the system is defined, and designers provide figures, layouts and flowcharts of the system. In fact, in the preliminary design, a general framework of the system is developed. In further steps of design process, the general framework id the guide to build the system. The ‘whats’ initiate the conceptual design and ‘hows’ from the conceptual design drive the preliminary design phase. In the detailed design phase, may include the material study to decide what materials to use to build the system. This phase provides more details related to each aspect of the system with solid modeling, drawing, specifications and different analysis of the system. In production phase, designers decide about operating parameters, test requirements, materials requirements, reliability and maintenance, external surface treatment and packaging. Computer-aided design (CAD) is a common tool in the production phase to help the designers optimize the design by reducing the cost of manufacturing without hindering the quality of the design. As we go forward through the design process, it is more expensive to make changes in the system design (Figure 1.1) [1].

Figure 1.1: Design changes cost increasing by design stage

3

However, there is a lack of applicable methodology that shows how to design a resilient system from the early design phase. The first part of this research develops a methodology using functions, behaviors, and design structure, as the only information available at the early stages of the design, to guide the designers toward a resilient design. Applying the proposed approach, the ability to recover from failures would be embedded into the system from the beginning. For this part of research, our developed Inherent Behavioral Functional Model (IBFM) tool is applied to simulate the failure behavior for an initial functional model of the system. The simulated failure scenarios provide information on the unique failure propagation paths and the end state, or final behavior, of the system assigned to each failure. Each failure path is caused by injecting one or multiple simultaneous faults into the functional model. Within this framework, we generate a population of functional models, and evaluate its potential failure scenarios. We also develop a cost-risk model to compare the resiliency of different designs, and produce a preference ranking. The optimal design is selected based upon a cost-risk objective. The risk is calculated based on the chance of having an undesired end behavior for each design among all the unique failure scenarios, and a consequential cost is assigned to each failure to quantify the cost-risk for a given design. We implement and demonstrate the proposed method on the design of a resilient monopropellant propulsion system. The second part of the research addresses the validation of the result of the first part. The fault simulation analysis shows that the selected functional model has the optimum cost-risk trade off among different functional models in the design space. The challenge is how the designers can gain the confidence that the selected model is performing as

4

expected, and reflects reality. The focus of this part of research is to propose methods to validate the simulation result in the early design stage when the system or prototype is not yet manufactured. Firstly, a method is presented showing how to quantify the expert knowledge to validate the model results. Second, a non-parametric method is proposed to validate the model using a data from subsystems. Third, a local sensitivity analysis is developed that helps the designers to focus on sensitive parts of the system in later design stages, particularly when mapping the functional model to a component model. We apply the proposed method to validate the output of failure analysis for the monopropellant propulsion system design. There are many uncertainties that complex engineered systems will face throughout their lifecycle due to changes in external conditions, such as technology readiness, market conditions, customer preferences or environmental changes. These states of interest affect the success of the system design with respect to the main objectives and application of the system, and are generally uncertain over the lifecycle of the system. To address such uncertainties, we propose a resilient dynamic design approach for engineered systems. We utilize Kalman filter equations to model the system operating state. Then, based upon the modeled states, the optimal change in the design of the system is achieved to respond to the new states. In the proposed approach, the dynamic design features of the system play the role of control variables, and by adjusting the design features to optimal estimation of uncertain parameters, the system overcomes the external uncertainties. The methodology is applied to design a resilient communication satellite system with ability of coping with market uncertainty (demand for the system services).

5

1.1

MOTIVATION An engineered system is defined as an assemblage of sub-systems,

hardware/software components, and people, designed to perform a set of tasks to satisfy specified functional requirements and constraints. The traditional approach for designing an engineered system is to establish a pre-defined set of requirements based on market studies and best estimate extrapolations of the current state, and then find the optimal design to satisfy the requirements. The traditional approach is inadequate to respond to changes in initial requirements or changes in operating conditions. This can lead to technical or economic failure if the system is faced with significantly different conditions than the ones predicted. This research addresses several of the weaknesses that traditional methods face in both complex system design and resilient design. The first step in developing a complex system is modeling and representing the characterization of the system, and simulation of the faulty behavior. Many complex engineered systems are represented by their component model; however, their complexity is dependent on the number of different components and interaction between the components. Also, when designing a new system, or in the early design phase, typically the set of components to build the system is not selected. On the other hand, design strategies used for advancing reliability and robustness are implemented for the purpose of advancing resilience in a system. However, there is meaningful difference between these concepts. Reliability is the ability of the system to function without failures for a specific period of time. Fault Tree Analysis (FTA), Failure Modes and Effects Analysis (FMEA), and event tree analysis are known techniques to study reliability. Practice has shown that the main

6

issue of these techniques is they require component-level detail throughout the system, at which point a designer may have limited ability to influence the design. Robustness is the state where the system performance is minimum sensitive to changes caused by internal or external factors of the system. However, the factors causing variability are not completely known and predictable when designing a system. A resilient system is a system with the ability to recover from a fault to maintain the desired level of performance. Figure 1.2 shows examples of resilient complex engineered systems (images from google).

Figure 1.2: Examples of resilient systems

Another issue with current design methods is a significant lack of research on strategies to validate a developed model in the early design stage, when the historical data of the complete system or data from a system prototype is not generally accessible.

7

The method presented in this research will address the issues with current design methods and direct a model-based resilient approach. The proposed approach is focused on applying resiliency techniques into the system from the beginning in the early design stage to enable the system to recover from failures caused by internal or external events through the lifecycle of the system. 1.2

OVERVIEW This PhD research can be divided into three major objectives describes in following

sessions. 1.2.1

Contribution of Objective 1

The main purpose of this part of the research is to present a framework to compare the resiliency of different designs during the conceptual design, when information about detailed implementation is unavailable. The Inherent Behavioral Functional Model (IBFM) tool is applied to simulate the failure scenarios for an initial functional model. The simulated failure scenarios provide us the information on the unique failure propagation paths and the end state/final behavior of the system assigned to each failure. Each failure path is caused by injecting one, or multiple, simultaneous faults into the functional model. Within this framework, we generate a population of functional models from a baseline seed model, and evaluate its potential failure scenarios. We also develop a cost-risk model to compare resiliency score of different designs, and produce a preference ranking, based upon the cost-risk objective. The risk is calculated based on the proportion of having a particular undesired end state for each design, and a consequential cost is assigned to each failure to quantify the cost-risk for a given design.

8

We implement the proposed method on the design of a resilient monopropellant propulsion system. 1.2.2


In the second part of this research, a method is proposed to validate the result from Objective 1. The idea is to validate the simulation generated by a selected model in the early design stage when the system or prototype is not manufactured yet and there is little or no information available from the complete system’s behavior. A method is proposed to quantify expert knowledge utilizing FMEA to enable the designers of the system validate the result obtained in the early design phase. Also, a non-parametric technique is applied to test if the observed behavior of one or more subsystems which currently exist, and the simulated model (obtained from the objective 1) come from the same distribution. Non-parametric or distribution-free techniques are methods to test a statistical hypothesis, they don’t need any probability distribution assumptions despite regular statistical methods. The Kolmogorov–Smirnov test, usually called K–S test is a nonparametric test to study test hypothesis related to one or two samples of continuous data. To study equality of discrete or categorical samples, chi-squared test is a well-known non-parametric method. In addition, a local sensitivity analysis is developed that helps the designers to focus on sensitive parts of the system in further design stages, particularly when mapping the functional model to a component model. We apply the proposed methods to validate the output of failure simulation for the monopropellant propulsion system design.

9

1.2.3


For a complex engineered system, it is not enough to be only resilient against internal system failures. There are uncertainties the system will face throughout the lifecycle due to changes in the external conditions, such as environmental changes, or market conditions. To address such uncertainties, we propose a lifecycle resilient design approach for engineered systems. We utilize a Kalman filter approach to model the uncertain future states of interest. Then, based upon the modeled states, the optimal change in the design of the system is achieved to respond to the new states. This resilient method is applicable in systems when the ability to change is embedded in the system design. A design framework is proposed encompassing a set of definitions, metrics, and methodology. A case study of a communication satellite system is presented to illustrate the features of the approach.

10

: RESILIENT DESIGN APPLYING FUNCTIONAL MODELS AND COST-RISK ANALYSIS IN EARLY DESIGN STAGE

2.1

INTRODUCTION

The number of parts in complex engineered systems is increasing rapidly. For instance, the components only in an integrated system is expected to be double every year as Gordon Moore predicted properly. Therefore, the interaction between various parts of the system, and between the system and external factors gets more complicated as technology and market demand is changing. The complicated interactions cause different type of uncertainties, and unexpected uncertainties can cause undesired behavior of the system or even catastrophic failures. An important concept in designing complex engineered systems, is to ensure that the behavior of the system in undesired and uncertain situations is determined early in the design phase, prior to the manufacturing and operational life of the system. It requires to conduct a fault behavior study in the early design stage to find improvement strategies and make changes into the design to enable the system cope with the uncertainties. To study the fault behavior of a system in the early design stage, modeling and simulation of the fault scenarios are the necessary steps. In many complex engineered systems, the characterization of the system is represented using a component model. However, when developing a new design, or in the early design phase, there is no component-level model available, and typically the set of components is not selected.

11

Because of this, system properties are studied using functional models. Developing a functional model of a complex engineered system is an effective way to study the system’s behavior, and to achieve a reliable fault analysis of the system. An overall description of the design methodology is provided as follows. A functional model is the first phase of the overall design methodology. Based on the system requirements and user knowledge, functions and flows are defined. Each function can have different modes: nominal, degraded, and failed. Function, flow, and mode definitions are fed to the IBFM tool, and the simulation runs to produce all system fault scenarios. The unique fault scenarios provide the proportions of the unique scenarios ended up a particular undesired final behavior. Applying the cost-risk model, the designer evaluates the cost of a design versus the resiliency or the ability of the design to recover from a failure. If the design fulfills the requirements, the process ends. Otherwise, a new design is generated by modifying the functional model. The program runs until the algorithm achieves the design with the lowest cost and highest resilience. Figure 2.1 shows the flowchart for the presented method.

12

Domain Functions, Flows, Modes and Conditions

User Knowledge

Alternative Functional Topology Generator

No

Functional Model for Conceptual Design

Python Input File

Fault Scenarios Generation Software

IBFM Toolkit

Resilience Scoring Function

Cost Model to Balance Design and Risk Cost

Does the Design Meet the Termination Condition ?

Yes

End Green: User Blue: Software

Figure 2.1: Proposed framework for resilient design in early design stage

This chapter provides an applicable framework to develop a resilient design. It illustrates how to develop a functional model, simulate the fault scenarios, and implement the cost-risk analysis to assess the resilient design. The material of this chapter is organized as follows: Section 2.2 proposes a clear and consistent definition of resilient design, and carefully disentangles it from reliability and robust design. Section 2.3 provides the function failure logic and behavior rule implementation. Section 2.4 illustrates system representation applying the functional models. Section

13

2.5 presents the scoring function. Section 2.6 presents generating different designs. Section 2.7 provides a case study of an electrical power system design, to illustrate the application of our method. Section 2.8 is the result summary and finally, section 2.9 provides conclusion and summary. 2.2

RESILIENT DESIGN Resilience can be defined as a system’s ability to recover from a fault to maintain

the desired level of performance. The ability of an engineered system to recover from failure must be designed into the system [1]. Design strategies used for advancing reliability and robustness can be implemented for the purpose of advancing resilience in a system; however, there is meaningful difference between these concepts. Reliability is the ability of a system to perform nominally for a specific period of time. In fact, reliability is the probability of success or availability of a component/system [2]. Reliability concentrates more on “why and how” components or systems may fail or have failed. Fault Tree Analysis (FTA), Failure Modes and Effects Analysis (FMEA), and event tree analysis are known techniques to study reliability [3]. Practice has shown that the main issue of these techniques is the requirement of the component-level detail of the system from the beginning of the design process. Robustness is where the system’s performance is minimum sensitive to internal and external factors that cause variability [4, 5, 6]. However, the factors causing variability are not completely known and predictable when designing a system. Therefore, the ability to overcome such uncertainties should be embedded into the system.

14

Uncertainty is the lack of ability to determine something precisely [7, 8]. Uncertainty management is an important concept in the design process of complex engineered systems. Reducing uncertainty when designing a system results in more efficient system with less cost. Recovery from the effects of uncertain events is an alternative to reducing uncertainty. There are different ways to recover from uncertain events, including flexibility, Monitoring and Automated Contingency Management (ACM), and resiliency. Flexibility is the ability of a system to respond to changes in initial requirements and objectives, after it starts operating, in a timely and cost-efficient way [9-12]. An ACM system adapts automatically and allows some degradation in the system performance when failure occurs with the goal of still achieving the mission [13]. For example, taking photos with less resolution. Resilient Design enables complex engineered systems to recover from uncertain event occurrences [14]. A resilient system monitors the performance consistently and is able to keep an acceptable level of functionality in case undesired internal or external events occurs [15]. Yodo et al. provide a literature survey of existing studies in engineering resilience from a system design perspective, with the focus on engineering resilience metrics and the strategies to quantify those metrics [16]. However, there is a lack of applicable methodology that shows how to design a resilient complex engineered system from the early design phase. To develop a resilient design, fault analysis is a necessary step. In most existing fault analysis methods, system designers need a precise model of system components

15

to be able to analyze fault behavior in a complex system. However, in the early design stages, specific set of components have not yet been selected and such detailed models of the complete system is not available. Because of this lack of methods to manage risks in early stages of design, the idea of representing the complex system by only its intended functionality has been developed. The following section discusses failure analysis using functional models. 2.3

FAILURE ANALYSIS AND FUNCTIONAL MODELS An engineered system is defined as an assemblage of sub-systems,

hardware/software components, and people designed to perform a set of tasks to satisfy specified functional requirements and constraints [17]. The traditional approach for designing an engineered system is to establish a pre-defined set of requirements based on market studies and best estimate extrapolations of the current state and then find the optimal design to satisfy the requirements [18]. However, these approaches are inadequate to respond to changes in initial requirements and uncertain events. This can lead to failure if the system is faced with significantly different conditions than the ones predicted. As a system becomes more complex, the uncertainty in the operating conditions increases. In such a system, implementing a precise failure analysis in early design stage is vital to study system fault behavior. There are different types of failure analysis techniques for complex systems [1927], like probabilistic methods [28, 29], or reliability techniques [30, 31], or approaches based on observations analysis [32]. Studies have shown that the early design stage is

16

the best time to catch potential failures and anomalies [33]. However, in the early design phase, decisions about the specific set of components is not made, and a component model is not available. The idea of using functional model, instead of component model, to design a complex system is of increasing interest. A functional model in systems engineering is a structured representation of the functions required to meet system requirements. The goal of developing a functional model is to describe the system behavior and determine vulnerable parts of the design, resulting in potential system improvement, particularly helpful in the early design stage when the detailed component model of the system is not available. Methods have been developed to combine functional modeling with failure analysis. Stone and Tumer developed the Function Failure Design Method (FFDM) which was the bridge between failure analysis and functional modeling. They applied functional models to represent the system design and identify potential failure states for each function [34, 35]. Grantham et al., estimated the failure likelihood for each function in the system. Lough et al., classified functions to high-risk to low-risk based on the consequence of failures [36, 37]. The idea of providing function-based failure analysis provided some improvements in the design of complex engineered systems; however, these methods limit the designers of the system to considering only one single-fault impact analysis at a time. To overcome this restriction and to enable the designer to consider multiple function failures effect, the Function Failure Identification and Propagation (FFIP)

17

method was presented [38-47]. The FFIP method identifies failure propagation paths by defining states of function health. This approach uses a separate behavioral model simulation to show failure propagation paths and failure effects. Typically, the modeling language Modelica is applied to simulate failure scenarios [48, 49]. However, system models cannot be automatically constructed from a description of the functional structure of a system and therefore it may not be useful in FFIP where multiple designs are investigated. McIntire et al. have created the Inherent Behavior of Functional Models (IBFM), a new function failure simulation which produces failure behavior of the system applying functions, modes, flows between the functions and conditions for transition between the functions. With this tool, designers can create a functional model of the system in the early design stage and simulate the failure propagation paths for the system without developing a separate behavioral model [50]. An open access IBFM tool in Python has been developed by our group at Oregon State University to simulate the behavior of the system as a hierarchical state machine. In the IBFM tool, one avoids equation-based physical behavioral modeling in favor of discrete event based modeling using a hierarchical state machine. This allows to study the failure behavior of the system even without complete knowledge of specifics of the system. It also produces very fast simulation of a huge number of failure scenarios because of no time-based differential equation solvers [50].

18

In the presented framework, we apply the IBFM tool to simulate the fault scenarios for different design topologies. The following section describes the functional models in detail. 2.4

SYSTEM REPRESENTATION BY FUNCTIONAL MODEL The first step in the proposed method is to study the requirements and expectations

from the system and define the functions and flow. We use a graph to represent the system functionality and its interaction with the environment. We implement the method in Python, because Python is free access and provides other tools like NetworkX which can be utilized to represent the functional model graphically. Models like Python graph objects provide the designers the ability to develop functional models with complicated structure [51]. Each graph edge (arrow) represents a flow of material, energy, or information within the system and each graph node (rectangle) represents a function that acts on the flows intersecting it. Figure 2.2 provides an example of a graph representation of a functional model consisting of a single internal function and three functions.

19

Figure 2.2: Graphical representation of a functional model

In this paper, a flow is defined similar to a bond used in the bond graph approach [50]. As in a bond graph, the flow has a direction, and uses two variables to define the flow: an effort variable, and a rate variable. For example, a liquid flow can be modeled as either a liquid pressure (effort) or a liquid volume flow rate (rate). The function at one end of the flow controls the effort state, while the function at the other end controls the flow rate. In the IBFM approach, effort and rate variables take qualitative values, such as Zero, Low, Nominal, and High. Each function consists of a set of modes. Mode definitions show the different levels of functionality for a function, which are usually categorized as operational, degraded and failed modes. Conditions determine how the flows go from one function to another and basically provide the conditions to generate the failure paths. Conditions define the transition between modes, e.g. the transition from an operational to degraded state, or from a degraded to failed state. In the IBFM approach, all modes and conditions are qualitative, rather than quantitative. The conditions that regulate a function transitioning from one mode to another mode are also flow specific. For example, the

20

operational mode of the function “Regulate Gas Pressure” has a “Gas High-Pressure” condition that leads to a failed mode. The “Gas High-Pressure” condition is only used by functions that have a “Gas” flow. Functions, flows, modes, and conditions are defined to construct a functional model to be fed to the IBFM tool to simulate all system fault scenarios. The Inherent Behavior of Functional Models (IBFM) module in Python [50] is applied to simulate the fault scenarios. The IBFM tool is hosted on the Oregon State University

Design

Engineering

Lab

GitHub

site

at

https://github.com/DesignEngrLab/IBFM. The repository contains the module ibfm.py, which includes classes defining each component that make up the program structure: Experiment, Model, Function, Flow, Mode, and Condition. It also contains the user guide and example scripts which run example models demonstrating the simulator [50]. The IBFM tool simulates the functional effects of single and multiple simultaneous faults. An exhaustive list of scenarios comprising fault paths is constructed by single fault, or two or more simultaneous faults. The probability of the initiating events for all faults to each function is assumed to be equal in this work. This assumption is based on the fact that the faults are applied to functions and not components, and therefore a method for determining faults at the functional level is required. The proposed method is compatible with assigning unequal probabilities to initiating faults; however, we have assumed equal probabilities in this work. The simulation of each fault scenario begins at that nominal model state, and then changes the mode of one or more functions to a degraded or failed mode. The end state of each scenario is recorded for further analysis. The number of unique paths that have

21

a particular undesired end state is used in the cost-risk model described in the following section. 2.5

RESILIENT SCORING FUNCTION In this part, a cost-risk model is proposed to evaluate the resiliency of different

designs for a complex engineered system. The idea of this cost-risk model is rooted in the risk-based utility theory [52]. This model studies the tradeoffs between the costs of designing a system that can recover from a failure, versus the cost of designing a system without a recovery while accepting the inherent risk of failure. The key element is that the “cost of risk” can be quantified, such that the trade-off between adding system cost versus accepting risk can be made. The proposed model is composed of three cost elements: the baseline cost of the design, the cost of mitigation, and the cost of risk, given by Equation 2.1. Min (𝐶𝐷 + 𝐶𝑂 ) + 𝐶𝑀 + ∑𝑁 𝑖=1 𝐶𝑅,𝑖 𝑃𝑅,𝑖 (1 − 𝑃𝑀,𝑖 ) CD: Baseline cost of the design CO: Operation cost CM: Cost of mitigation CR: Consequential cost of the risk PR,i : Proportion of scenarios with undesired end state i PM,i : Probability of mitigation 1 − 𝑃𝑀,𝑖 : Probability of mitigation failure N: Number of undesirable end states

(2.1)

22

CD is the cost of design when there are no strategies to recover from failure embedded in the design. CM, represents the cost of changing the design to be able to recover from a particular failure. There are different strategies to change a functional model to design a more resilient system. In the next section, we describe four ways. Independent of the quality of performance, there is a certain cost to operate a system, defined as operation cost, CO. With respect to defining the “cost of risk”, we define a risk as a triplet of its impact or consequence, its probability of fault occurrence, and it’s probably of being mitigated. In Equation 2.1, CR is the impact or consequential cost of the risk; in our approach, it is quantified as the cost of having a failure and not recovering from it (in units of dollars). Secondly, PR is the proportion of scenarios with a particular undesired end and is quantified using the results of the IBFM fault simulation. It is quantified as the number of unique scenarios with a particular undesired end state (or fault) divided by total number of unique scenarios. Lastly, PM is the probability that a resilient design recovers from a failure due to a mitigation action (a mitigation action is assumed to have a cost of CM). Probability of mitigation is also calculated from the IBFM simulation result by adding the mitigation action as a function to the functional model and rerunning the simulation. N is the number of system end states or final behavior that the system is designed to consider. Except for the nominal scenario when all functions perform nominally, other end states are fault scenarios and undesired. However, an undesired end state does not imply that a safety hazard is present, it may just reflect an end state that prevents the system from nominal operation or from completing a mission. For instance, when designing a car, having a flat tire is undesirable, but may not be

23

classified as a safety hazard; however, an engine fire is both an undesirable state and a safety hazard. Applying the cost-risk model, the designer specifies the tradeoff between the resiliency and cost of the design; the cost-risk model is treated as an objective function in an optimization framework. In the optimization formulation, the objective is to minimize total system cost (i.e., Equation 2.1), subject to any system-level constraints. Treating the search for the best system-level design as an optimization problem necessitates identifying a termination condition for the search. In application, the search would most likely be done by a heuristic or stochastic search method, given the discrete nature of the functional model representation of the system. Since these methods cannot be guaranteed to converge to the optimum in finite time, the search must be ended based on a heuristic. Termination Conditions for the proposed framework in Figure 2.1 can be defined as: •

An upper limit on the number of evaluations (designs) is reached.

• An upper limit on the time of evaluations of the fitness function (cost-risk model) is reached. •

The chance of achieving significant changes is very low.

If the design meets one or more of the termination conditions, it gets selected, otherwise a new design is generated and evaluated.

24

2.6

GENERATE NEW DESIGNS As noted in previous part, the cost-risk approach is implemented as a search

problem, in which the objective is to find the lowest cost design. An issue to address is how to change/modify the design in a way to produce other feasible designs in the search process. The challenge is identifying alternate functional models which are technically feasible, i.e., functional models which preserve the key system functions and the associated flows. Therefore, a method is needed to ensure that functional models evaluated in the search process are technically feasible. Graph grammars [50] could be implemented to address this issue and are compatible with the cost-risk approach. In this work, we define design strategies to generate new design based upon a few simple rules used by designers of aerospace systems. The design modification rules can be placed into four categories as follows: 2.6.1

Redundancy and Health Management

In this technique, redundancy or health management is added to the functional model. The redundancy could be the addition of a redundant function, or it could be added in the form of partial redundancy. In the case of partial redundancy, we may be able to fulfill the needed functionality using secondary functions. For example, we may be able to use a pressure sensing function to also indirectly fulfill a flow rate sensing function in the case that the flow rate sensing function is faulted. Adding redundancy or health management will affect CM and PM in Equation 2.1. The tradeoff will in general be one in which mitigation cost is added to increase the probability of mitigation (and thus reduce the cost of risk).

25

2.6.2

Change the Order of Functions

Changing the order of functions is another way to generate a new design at the conceptual level of the design. This change affects the proportion of scenarios with undesired end state 𝑖, PR,i , by focusing on fault avoidance; i.e., arranging the functions is such a way that fault propagation path is changed and thus the proportion of scenarios with a particular undesired end state is changed. 2.6.3

Using Unused Flows as Inputs

In this method, any flows that transfer material or energy to the environment (i.e., are not used by the system directly) are utilized as additional inputs for other functions where applicable. This improves the fault avoidance aspect of the design. For example, one could use waste heat to supplement a heating function as a mitigation function (CM and PM), or one could lower the cost of design of a heating function (CD) by coupling waste heat from a different function with the heating function. 2.6.4

Combining/Splitting Functions

Two or more functions can be combined (or split) to make a new design and potentially improve the fault avoidance of a system. This could affect both CD and PR. whether combining two or more functions into a single function (or splitting a single function into two or more functions) improves or worsens the cost-risk objective function must be determined by the results of an IBFM simulation. Combining functions may lead to elimination of a fault path (thus reducing PM), or may make the new combined function more likely to fail (thus increasing PM).

26

We can generate an infinite number of functional models for a design, however a limited number is technically feasible. The following section presents a case study using a monopropellant propulsion system design to show how the methodology can be applied in a system design problem. 2.7

CASE STUDY: MONOPROPELLANT PROPULSION SYSTEM A monopropellant propulsion system refers to a chemical propulsion fuel which

does not require a separate oxidizer and thus can be used in space. Monopropellant designs are typically used in the aerospace industry because they make the engine lighter, less expensive, and more reliable. In this case study, the monopropellant is hydrogen peroxide (𝐻2 𝑂2). It is important when designing a monopropellant system to consider environmental conditions in space. With no gravity assistance, the system should be able to push the propellant towards the catalyst. The concept for this system design is to apply expanded gas to push the monopropellant over a catalyst and produce thrust. This system can be divided into three main subsystems: gas, propellant, and catalyst. When there is a command for a change of the spacecraft velocity, the inert gas is heated to expand. The expanded gas is fed through regulation and control functions to reach the right quantity, pressure, and temperature. The expanded gas places pressure on the propellant (hydrogen peroxide) and guides it to the catalyst. When the propellant passes the catalyst, combustion occurs and changes the velocity of the spacecraft. Figure 2.3 presents the general idea of a monopropellant propulsion system.

27

Figure 2.3: General idea to design a monopropellant propulsion system

When the thrust is commanded, if all functions are nominal, the system performs as expected; however, there are scenarios in which something can go wrong with one or more functions and the result is not as expected. This would cause the engine to provide too much, too little or no thrust. In practice, it means the aerospace system passes the location, or does not reach it. In rare catastrophic failures, the system might explode (i.e. loss of system). The final behavior or end states of a monopropellant propulsion system can classified to 5 groups: 

Mission Accomplished (Desired)



Too Much Thrust (Undesired)



Too Little Thrust (Undesired)



No Thrust (Undesired)



Loss of the System (Undesired)

28

Figure 2.4: End states for a monopropellant propulsion system

To apply IBFM tool, the first step is to define the functions, flows, modes and conditions in Python. The advantage of defining the functions and flows in Python is that our NetworkX tool can be used to represent the functional model graphically. Figure 2.5 represents our developed functional model for the monopropellant system. The following session describes how to develop a functional model and simulate fault scenarios.

29

Figure 2.5: Basic functional model developed for monopropellant system

30

2.7.1

Functions Definition

The monopropellant propulsion system shown in Figure 2.3 contains 28 functions and 129 modes. One example of how to define a function for monopropellant propulsion system is as follows: function ImportHeat mode 1 Operational NominalHeatSource mode 2 Degraded LowHeatSource mode 3 Degraded HighHeatSource mode 4 Failed NoHeatSource The first line contains the keyword function, followed by the desired name of the function. The indented lines contain all of the modes of the function. Each line describing a mode begins with the keyword mode, followed by a unique withinfunction alphanumeric identifier, followed by the function health associated with the mode, followed by the name of the mode. Available mode health states are Operational, Degraded, and Failed modes. Failed modes are the ways, in which the system might fail. A single mode in each function definition may be followed by the keyword default, which assigns that mode to be the initial mode of the function at the beginning of simulation. 2.7.2

Flows Definition

Flows are defined in a single line. The right way to define a flow is to mention the keyword flow, then the type of flow, followed by the name of the category of the flow

31

which can be Material, Energy, or Signal. All flows are derived from one of these three categories of flows. flow Heat Energy 2.7.3

Modes Definition

Modes definition is more complicated, as all of the mode’s behaviors must be explicitly described. Operational, degraded, and failed modes are required to be defined for each function. A simple mode definition example is the nominal gas source mode: mode NominalGasSource InertGas output effort = Nominal The first line consists of the appropriate keyword, in this case mode, followed by the desired name of the mode. Each indented line consists of a single assignment statement. The expression to the left of the assignment operator = is evaluated to determine the flow variable being assigned to. The expression on the right of the assignment operator determines the state of the flow variable. Every flow in the statement must be referred to using three words: the flow type name, its direction, either input or output, and its variable, either effort or rate. More complex behaviors may be defined by using operators. A single unary operator is used in the definition of the “NoGasSource” mode: mode NoGasSource InertGas output effort = Zero

32

This mode definition makes use of a constant state. Available states are Zero, Low, Nominal, High, and Highest. The definition of the drifting low pressure sensing mode uses two unary operators: mode DriftingLowPressureSensing import NominalConductingRegulatedGas SignalDesiredPressure output effort = RegulatedGas input effort - The first one, the keyword import, copies all of the statements from the definition of the mode directly following the keyword. In this case, the two statements from the “NominalConductingRegulatedGas”

mode

are

copied

into

“DriftingLowPressureSensing”. The second one, the decrement operator --, decreases the value of the state by one qualitative level. 2.7.4

Conditions Definition

Condition definitions are similar to mode definitions. They name the condition being defined, and explicitly describe the behavior, but they only include a single behavior statement. Rather than being an assignment, the statement is a logical test. For example, the condition to test for a function being exposed to high temperature is: condition HighTemperature Heat output effort > Nominal Logical operators may be combined to form more complex tests. All binary operators are evaluated from left to right.

33

Running the IBFM tool produces all fault scenarios and tabulates the unique scenarios terminating with particular undesired end states. The final behavior of the system as shown in Figure 2.4 is categorized into different main end states. The designers of the system decide what end states the system failures could cause. In this case study, undesired end states are: •

Pass the Mission Location

•

Do Not Reach to Mission Location

•

No System Movement when Needed

•

Loss of the System

2.7.5

Alternative System Designs

The baseline design, shown in Figure 2.3, represents a functional model developed for a monopropellant system in the early design phase. In this design, the desired final behavior is the mission accomplishment, in which all gas, propellant, and catalyst functions operate nominally. Any improvements in the functional model influence the final behavior of the system. In other words, the baseline model is not designed to be resilient against failures. Therefore, different system topologies with focusing on improvement the resilient of the system can be investigated. Improvement strategies include applying different levels of redundancy, reconfigurability or integrated health management sensors in the system design. For each design, the fault scenarios can be classified into five end states. Figure 2.6 represents the Pie Chart for a candidate design for monopropellant propulsion system.

34

Figure 2.6: Classification of fault scenarios for one monopropellant design

In this case study, we simulate the failure behavior of the initial design and compare it to six alternative functional models representing different designs for the monopropellant propulsion system. These alternative designs represent different level of resilient. In each alternative model, changes are applied in the functional model to improve the resilient of the design against some particular failures. The first modification of the baseline design includes a redundant gas rate sensing function. In this design, if no signal is coming from the primary sensing function, a secondary sensing function can supply the required information. The second modification of the initial design is an identical system with redundant gas pressure sensing. The third modification of the initial design combines the adjusting pressure and rate into one function. The fourth design uses output heat to expand inert gas (by contrast, in the baseline, the heat is exported from the system and disappears into space). The fifth design incorporates the redundant sensing for propellant. The sixth design

35

applies the output thrust heat to preheat the propellant. In this way, the propellant plays the role of a cooling system. The following section presents the results. 2.8

RESULTS Six candidate designs for the monopropellant propulsion system are examined. In

each design, the functional model has been modified to be more resilient to a particular failure or undesired end state. Table 2.1 shows all the designs generated by the functional model modifications. Table 2-1: Generated designs for monopropellant system Basic Design

The original unaltered system model

Design 1

Redundant gas rate sensing added

Design 2

Redundant gas pressure sensing added

Design 3

Combine adjusting gas pressure and rate into one function

Design 4

Use output thrust heat to expand inert gas

Design 5

Redundant propellant pressure sensing

Design 6

Use output thrust heat to expand inert gas and preheat propellant

All failure scenarios have been simulated using the IBFM tool. The amount of CPU time required to simulate each scenario increases as the complexity and the number of faults injected to the model increases. For instance, the time of simulation for injecting two simultaneous faults is less than the time of simulation for injecting three simultaneous faults. In this study, we investigate injecting up to 3 faults (all possible three failures) in each one of the functional models. The classified simulated scenarios for the seed and the six design topologies is shown in Figure 2.7. In this histogram, the number of successful and failed scenarios simulated for each design is demonstrated.

36

Figure 2.7: Failure simulation results for candidate monopropellant designs

The proportion of scenarios for each undesired end state is applied to the cost-risk model of Equation 2.1. The mitigation cost CM is the cost of changing the basic design to make it more resilient to a particular failure or undesired end state. The operation cost CO for all candidate designs is assumed to remain the same because on-ground systems cost is the main operation cost for a monopropellant propulsion system. It’s assumed that regardless of how the system is performing, there is an on-ground system to monitor, control and run the spacecraft. For other design studies, the operating cost may vary by design concept. It is assumed that the aircraft completes three missions per year, and for each mission the operation cost is $500 million. The number of failed scenarios versus the total number of scenarios for each design defines the proportion scenarios with that particular undesired end state, PR. The probability of the mitigation failure, is the probability that the mitigated part does not work when needed and the system ends up an undesired final behavior. These probabilities are quantified based on the IBFM simulation and the costs, CR, are

37

estimated based on the data provided by Miller [53]. The total cost reflects the tradeoff between designing a resilient system and the cost of doing so. The lower the total cost, the higher the rank of the design would be. The results of cost-risk optimization for the monopropellant propulsion system are shown in Table 2.2. In the table, we tabulate all the elements of Equation 2.1. Since there were only seven designs to consider, an optimization algorithm was not used; however, an evolutionary or other stochastic algorithm could be used to search larger design spaces. Cost numbers are estimated based on the space system cost models and data provided by Miller et al. [53]. Table 2-2: Scoring function for monopropellant system designs [Millions $] Cost-Risk Model Cost of Basic Design CD

Basic Design

Design 1

Design 2

Design 3

Design 4

Design 5

Design 6

100

100

100

95

90

100

80

0

10

10

0

0

20

0

Operation Cost CO

1500

1500

1500

1500

1500

1500

1500

Cost of Risk 𝑪𝑹 𝑷𝑹 (𝟏 − 𝑷𝑴 )

45.50

32.20

30.75

34.90

40.70

21.30

35.20

Total Cost

1645.50

1642.25

1640.75

1629.90

1630.70

1641.30

1615.20

Resilient Score

Rank 7

Rank 6

Rank 4

Rank2

Rank 3

Rank 5

Rank 1

Mitigation Cost CM

As shown the table, Design 6 has the optimal trade-off between resilience and cost. In this design the propellant acts as a cooling system. The heat produced by propulsion is used to preheat the propellant and expand the inert gas. Utilizing this source of energy helps the successful combustion process. This design was selected because it had the lowest value of the cost-risk objective function. The next best design is Design 3, which combines the gas pressure and rate control into a single function. This may indicate

38

that since both functions are critical to operation, and a failure of either function is highly detrimental to the system, it is better to address them with a single function with a single failure rate. 2.9

CONCLUSIONS This chapter presented a framework for resilient design based upon the evaluation

of different designs during the early design stage. We created a framework to present an initial functional model of the system and simulate the fault behavior of the system. We showed how to generate different designs using a functional model. To produce a preference ranking of designs, we developed a cost risk-model to generate a resilient score for each design. The central idea is that each design is evaluated based upon the trade-off between the cost of risk and embedding resiliency into the design of the system. The design implications of using this framework for the development of resilient engineered systems were discussed, with the focus on the functional model development, failure simulation, and cost-risk analysis to rank different designs and choose the most resilient one. The method was applied to design a resilient monopropellant propulsion system. The results show that the presented method is a practical design framework to develop resilient complex engineered systems in the early design stage, when the complete knowledge of the system specifics and characterizations is not available.

39

: VALIDATION OF MODEL SIMULATION RESULT IN EARLY DESIGN PHASE

3.1

INTRODUCTION In the design process, studying the behavior of the system prior to manufacturing

plays a key role to reduce cost and enhance the efficiency of the system during its lifecycle. To study the behavior of the system in the early design phase, it is required to model the characterization of the system and simulate the system’s behavior. In traditional design methods, designers start with developing a component model of the system; however, this is not an efficient design methodology. The main weakness of the traditional design method is using a component model to represent the system in the early design stage. This causes the complexity of the model to depend on the number of the different components, as well as interaction between the components. In addition, in the early design phase, typically the set of components is not available. We addressed this problem in the first part of the research [54]. We proposed a design method using functional models. In the proposed method, a population of functional models is generated and the potential failure scenarios are simulated; finally, applying a cost-risk model, the most resilient design is selected. The open question is how do we make sure that the fault behavior of the selected functional model is correct? The approach used in this research to answer this question is model validation. In discussing model validation, we will also discuss the related concepts of model selection and model calibration. The reason for this is to understand the statistical

40

methods associated with these related concepts to determine if they have applicability for our model calibration problem. Model validation studies the validity of the model to represent the real system’s behavior [55-61]. There are many approaches to validate a computer model or mathematical model; however, there is not an applicable method to show how a designer can validate a model developed in the early design stage when the real system or prototype is not available. It is difficult to validate a model in early design since very little is known at this time. Validation at this phase must consider not only traditional model validation, but also suggestions for further design steps. The objective of this chapter is to propose methods to validate the simulation generated by a functional model developed in the early design stage, when the complete system or prototype is not available. Specifically, we are interested in validation of the failure behavior simulation produced by a selected functional model in the early design phase to ensure that it is a good representation of the real system. The material of the chapter is organized as follows: Section 3.2 investigates existing methods and discusses the limitations of each method and the reasons why existing methods are not applicable to validate the fault behavior of a selected model in early design stage. Section 3.3 provides methods to validate a functional model selected in early design stage, and applies the proposed methods in a case study of functional model validation for the monopropellant propulsion system design. Finally, section 3.4 provides conclusion and summary.

41

3.2

MODEL VALIDATION METHODS In this section, we investigate strategies to validate models based on the available

data. By looking at the literature in model validation, we gain insights into approaches that can be applied or adapted for our problem. Models are tools to represent and study a real system. Popular models to represent a system are mathematical, statistical, computer, component, functional or network models. Model validation is a necessary step to make sure that the model simulation reflects the real behavior of the system. Later in this chapter, we discuss the limitations of model validation in the early design phase when there is no or little data available. It is shown that existing validation methods are based upon strong assumptions, and therefore they are not always applicable to validate fault behavior of a functional model in the early design stage, when little or inadequate information is available from the real system behavior. Different approaches to assessing model validity are described. An engineered system is built for a specific set of purposes or objectives, such as improving the sustainability, safety, reliability, performance etc., while managing the cost of the system; this is a challenge that engineers face from the beginning of the design process. The first step in designing such systems is to develop a model and simulate the behavior and characteristics of the system to enable the designers to study the system, and decide upon strategies to improve the design features regarding the objectives. Model validation is a necessary step to ensure that information obtained from the model is within an acceptable range of accuracy consistent with reality [6265]. There are many approaches that have been used to validate a model. We classify the approaches into three groups based on the amount of available true data to validate

42

a model. For the aim of this research, when there are enough data available to assume a probability distribution, the classification is called adequate data. When there are some true data available to validate the model, but not enough to assigning a probability distribution, it is called little data. When there is no data and the only available source to validate a model is expert knowledge, it is a no data situation. Table 3.1 represents the classification of the model validation methods based on the available data. In the first class, enough true data is available to validate the model under study. For this class, standard statistical methods can be used to test if the model under review produces the result similar to the true data by comparing the true data and output generated by the model. True data can be obtained from another valid model, historical data, prototype, or a real system. In the second class, there is not enough true data available to validate the whole model; however, observations from some subsystems are available. In this class, a prototype or real system is not manufactured or historical data of the complete system behavior is not available; however, observations from some subsystems can be applied to validate parts of the model. If the observations and model output are discrete values, the chi-squared test is the tool to test the validity; otherwise, for continuous values, the K-S test is the appropriate tool to test the similarity of the observed data from subsystems and output from the model. In the third class, there is no data available to validate the model. In this case, the only source to validate the model is expert knowledge. Methods like Failure Modes and Effect Analysis (FMEA) or evidence theory are applied to quantify the expert knowledge and validate the model. In regard to the objective of this chapter to validate the output from the model developed in early

43

design stage, class 2 or 3 in Table 3-1 is typically the case, because in the early design stage no or little data from the complete is available. Table 3-1: Model validation methods and assumptions Class

Available Data

Source

Tools to Test Validity

1

Adequate Data

Historical Data,

Standard statistical Methods, Sensitivity Analysis

Another Valid Model, Real system/Prototype

2

3

Little Data

Subsystems

No Data

Experts knowledge

Chi-Squared for Discrete Samples K-S Test for Continuous Samples FMEA, Evidence Theory

The following sections provide more details on each class. 3.2.1

Adequate True Data to Validate Model

Adequate true data can be utilized to select, calibrate and validate a model with familiar statistical methods. True data might be obtained from a real system or prototype observations, or a valid model. Having enough true data is desired to develop, select, test or validate a model, however most of the time in complex engineered system design, this is not the case. In the next subsections, we will discuss techniques for model selection and model calibration, in addition to model validation, to understand how adequate data can be used in the modeling process. 3.2.1.1 Model Selection In previous chapter, we developed a cost-risk function to select the optimal design among six candidate models for a mono-propellant propulsion system, because there

44

was no data available from the real system behavior to compare the models. If enough data were available from the real system’s behavior, cross-validation method could be applied to select the model with closest behavior to the real system. Cross-validation is a technique to select the model that has lowest Mean Square Error (MSE) with respect to the true data. It is applicable when there are different models and the goal is to select the one that generates output similar to the true data [66-68]. Another application of the cross-validation method is to build a model when enough true data is available. This has been widely used to build predictive models e.g., sales predictions models. In this application, the dataset is divided to two parts, one part is used to build a predictive model and the second part is used to test the model. More details about cross-validation method can be found in the survey paper [69]. A method to select a model among two candidate models, when true data is available is Likelihood Ratio (LR) test. It is assumed that a proper statistical distribution can be assigned to data generated by each one of the candidate models and there are no unknown parameters to estimate. The hypothesis for the likelihood ratio test specifies if there is a meaningful difference between the parameter obtained from the first model and the parameter from the second model [70]: 𝐻0 ∶ 𝜃1 = 𝜃2 𝐻𝐴 : 𝜃1 ≠ 𝜃2

45

where 𝜃1 is the parameter of the first model and 𝜃2 is the parameter of the second model. Equation 3.1 demonstrates the likelihood ratio test:

𝐿𝑅 =

𝐿(𝜃2 |𝑑) 𝐿(𝜃1 |𝑑)

(3.1)

The true dataset, 𝑑, (obtained from valid model or observation or real system) provides the maximum likelihood estimate of the parameter. The ratio selects the model that is more likely to produce true dataset 𝑑. In other words, the selected model has the parameter with smaller difference from maximum likelihood estimator [71-74]. Bayes Factor is a popular method to decide between two models when there is true data, 𝑑. This method has been used in a wide range of fields [75-79]. It is assumed that a proper statistical distribution can be assigned to data generated by each one of the candidate models. Consider 𝑀1 and 𝑀2 are two candidate models and we are interested in selecting the one that is more likely to produce given true data d. The Bayes factor 𝐵12 illustrated in Equation 3.2 is to decide between model 𝑀1 and 𝑀2 [80]:

𝐵12 =

𝑃(𝑀1 |𝑑 )/𝑃(𝑀2 |𝑑 ) 𝑃(𝑀1 )/𝑃(𝑀2 )

(3.2)

Where 𝑃(𝑀𝑖 ) is the prior probability distribution of model 𝑀𝑖 and 𝑃(𝑀𝑖 |𝑑) is the posterior probability distribution of the model 𝑀𝑖 generates the data. When the Bayes factor is larger than 1, model 𝑀1 is selected, and when the Bayes factor is less than 1, model 𝑀2 is selected. Bayes factor equal to 1 does not provide any evidence to select one model over the other.

46

3.2.1.2 Model Calibration True data can be used to calibrate the model. Calibration is widely used in computer-based models. In the calibration technique, some particular model parameters are considered fixed and unknown and using the true data, the best match of the parameters is found. The limitation of the calibration technique is that in practice model parameters are not fixed and they change in each run of simulations. Xiong et al. [81], address the limitation of using true data to calibrate a model assuming the parameters are fixed. They proposed a Maximum Likelihood Estimation (MLE) method to assign a distribution to random changing parameters of the model and to calibrate those parameters utilizing true data. More details can be found in the survey paper [81]. 3.2.1.3 Model Validation When there is a model developed to represent the real system’s behavior and we are interested in validating the model using true data, many standard statistical tests can be applied. In this case, the validation phase focuses on comparing the true data, with the corresponding elements of the model simulation to determine whether the differences are acceptable. If the result shows that the model simulation is different from the true data, changes should be made to the model. Making changes in the model has to be continued until designers make sure that the model is a good representation of the real system, and the difference between the simulation and true data is not significant.

47

Another tool to validate a model using true data is sensitivity analysis. In this technique, designers evaluate the confidence in a model based on the relationship between inputs and outputs of the model simulation. This requires studying the effect of each input on the output variation. The same input-output relationship is expected to occur in the real system [82-88]. Sensitivity analysis requires repeating the simulation many times; therefore, it can be expensive when the simulation takes a long time to run or the model has too many input parameters. Another limitation of sensitivity analysis is the assumption of the independent input parameters. Some sensitivity analysis techniques assume independency between model inputs; however, in designing complex engineered systems, designers have to pay extra attention to the model inputs, since they can be highly dependent. Dependent inputs shows greater variation in the model output compared to an independent parameter. Regression is a simple tool for sensitivity analysis. Regression coefficients are representation of the sensitivity of the response variable (output) to each one of the dependent variables (inputs) [89-91]. The limitation of the regression method is that it assumes a linear relationship between the model output and the input parameters (variables). Also, it is assumed that there is only one output for the model. However, in designing complex engineered systems, there could be more than one output or a nonlinear relationship between the input parameters and model output. When there is more than one output, sensitivity analysis can be performed for each output separately but, it is hard to interpret the result in a correct way. In such cases, variance-based sensitivity analysis is recommended.

48

The variance-based, or global, sensitivity analysis SOBOL method is a sensitivity analysis method based on probabilistic framework. It breaks the variation of the model output into fractions, each fraction is assigned to one model input [92-95]. The local sensitivity is a method to complement the SOBOL analysis. In this technique, the fractions are produced by taking partial derivatives of the output regards to each input 𝜕𝑌

|𝜕𝑋 | [96]. 𝑖

3.2.2

Little Data to Validate Model

When there is little true data to validate a model, assigning a probability distribution is not reasonable. In this case, non-parametric tests are better strategies to compare the data generated by model to the true data observed from real system or prototype or a valid model, because non-parametric tests are distribution-free, it means they are not based on probability distribution assumption for the data. If true data and simulated data are categorical, a chi-squared test can be used to validate the model [97, 98]. The null hypothesis is that frequencies for the true data and the simulated data are equal. Under the null hypothesis, the test statistic has approximately a chi-squared distribution with 𝑛 − 1 degrees of freedom. For 𝑛 categories, the test statistic is calculated by Equation 3.6 [99-101]. 𝑇𝑒𝑠𝑡 𝑆𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = ∑𝑛𝑖=1

(𝑂𝑏𝑠𝑒𝑟𝑣𝑒𝑑𝑖 −𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑒𝑑𝑖 )2 𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑒𝑑𝑖

(3.6)

If true data and simulated data are continuous values, and non-categorical, the Kolmogorov–Smirnov test is a well-known way to validate the model [102,103]. The test hypothesis in a Kolmogorov-Smirnov test is:

49

𝐻0 ∶ 𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑎𝑛𝑑 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 𝑎𝑟𝑒 𝑓𝑟𝑜𝑚 𝑡ℎ𝑒 𝑠𝑎𝑚𝑒 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 𝐻𝐴 : 𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑎𝑛𝑑 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 𝑎𝑟𝑒 𝑛𝑜𝑡 𝑓𝑟𝑜𝑚 𝑡ℎ𝑒 𝑠𝑎𝑚𝑒 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 If the test result is 1, the data provides enough evidence to reject the null hypothesis; if the test result is 0, we fail to reject the null hypothesis. Rejecting the null hypothesis means the simulation and true data are not from the same distribution. In this case, the model is not adequate to represent the real system. In the early design stage, there is no information available from the real system’s behavior. The prototype of the system is not manufactured yet and historical data from the complete system does not exist for a new design. In this case, the methods based on statistical distribution assumptions are generally not applicable. The limitations and assumptions of model validation in early design stage are the main motivation for this chapter of the research. In the previous chapter, we developed different designs for a monopropellant propulsion system. Next, we simulated failure behavior for each candidate design and classified the failure scenarios to five groups based on the final behavior of the system. Then, we developed a cost-risk model to compare the candidate designs. Finally, we selected the design with the optimal costrisk trade off. In this chapter, we are interested in validating the result of the previous chapter. Specifically, we are interested in answering the question how we can make sure that the failure behavior simulation of the selected functional model is a good representation of the real system. The main challenge to answer to this question is that there is no real system or prototype manufactured in the early design stage and there is no historical data available from the complete system. Therefore, model validation for

50

the selected model simulation must be built upon data from some subsystems and expert knowledge as the only source of information available in the early design stage. In the following section, we propose methodologies to validate the fault behavior simulated by the selected functional model for the monopropellant propulsion system in the early design stage. 3.2.3

No Data to Validate Model

When there is no true data to validate a model, one strategy is to use expert knowledge to validate the model. The challenge is to quantify the expert knowledge in a meaningful way to be able to compare with the model output. When probability theory and distribution assumptions are not applicable, evidence theory is a good approach. Evidence theory is also called Dempster–Shafer theory because Shafer developed the idea presented by Dempster [104]. In evidence theory, the basic idea to quantify expert knowledge is using Basic Belief Assignment (BBA). 𝐵𝐵𝐴 can be described as the degree of trust in an element. An example of using BBA is provided using the monopropellant orbiter. In the monopropellant system, 𝑋 = {𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5 } where 𝑥1 to 𝑥5 are uncertain elements that we are interested in quantifying using expert knowledge. The elements are mutually exclusive of each other: 𝑥1 defines system behaves nominally, 𝑥2 defines a system with too much thrust, 𝑥3 defines a system with too little thrust, 𝑥4 defines a system with no thrust, and 𝑥5 a system with catastrophic failure. 𝐵𝐵𝐴 quantifies expert knowledge using a mapping function m. The number 𝑚(𝐴) is in range zero to one and illustrates the portion of total expert’s belief in element 𝐴, when 𝐴 𝜖 𝑋 [105-107].

51

When there are two 𝐵𝐵𝐴, 𝑚1 and 𝑚2 , provided by different evidence sources, e.g., different expert knowledge, Dempster’s equation can be utilized to get the combined 𝐵𝐵𝐴 presented in Equation 3.3 [108].

𝑚𝐴 =

∑𝑐 ∩𝑐 =𝐴 𝑚1 (𝑐𝑖 )𝑚2 (𝑐𝑗 ) 𝑖 𝑗 ∑ 1− 𝑐𝑖 ∩𝑐𝑗=∅ 𝑚1 (𝑐𝑖 )𝑚2 (𝑐𝑗 )

(3.3)

where 𝐶𝑖 and 𝐶𝑗 are propositions from knowledge sources 𝑚1 and 𝑚2 . In Equation 3.4, ∑𝑐𝑖 ∩𝑐𝑗=∅ 𝑚1 (𝑐𝑖 )𝑚2 (𝑐𝑗 ) is the conflict among two independent sources of knowledge. Dempster’s rule resolves the conflict by normalization. It is more reasonable to present a range instead of a single number to quantify belief or trust or confidence of experts. In evidence theory, the range is expressed as [𝐵𝑒𝑙(𝐴), 𝑃𝑙(𝐴)]. Equation 3.4 and 3.5 show how to obtain the range [108]. 𝐵𝑒𝑙(𝐴) = ∑𝑐⊂ 𝐴 𝑚(𝑐)

(3.4)

𝑃𝑙(𝐴) = ∑𝑐∩𝐴≠∅ 𝑚(𝑐)

(3.5)

where 𝐵𝑒𝑙(𝐴) is the total degree of belief. It is the summation of 𝐵𝐵𝐴𝑠 for sets including complete proposition 𝐴. Plausibility for 𝐴, 𝑃𝑙(𝐴), is the summation of 𝐵𝐵𝐴𝑠 of propositions that do not have empty intersections with proposition 𝐴. In fact, 𝑃𝑙(𝐴) is the total BBAs that agree with proposition 𝐴 totally or partially [108]. Evidence theory uses the described equations to quantify the degree of belief or trust or confidence of experts to a model output and help decide about the validation of the model.

52

Failure Mode and Effect Analysis (FMEA) [4, 24] is another strategy to bring the ideas of experts take into account. FMEA describes failure of the system as well as causes and effects of the failures. The results of FMEA are used to consider design changes that may be necessary to reduce risk. It is assumed that experts are able to judge the models based on the outputs [109-112]. An example of an FMEA will be provided in section 3.3. 3.3

MODEL VALIDATION IN THE EARLY DESIGN STAGE A functional model is a structured representation of the functions required to meet

system requirements [44]. The purpose of a functional model is to describe the system behavior and determine vulnerable parts of the design, resulting in potential system improvement. In chapter 2, we investigated six candidate designs for a monopropellant propulsion system. Design 6 was selected based on the cost-risk trade off model proposed in Equation 2.1. Figure 3.1 illustrates the selected functional model for the monopropellant propulsion system. In this selected design, the propellant plays the role of cooling material and the extra heat produced from propulsion helps to expand inert gas and preheat propellant for the better propulsion.

Import Catalyst

Contain Inert Gas

Export Byproducts

Catalyst

Byproducts

Thrust Command

Conduct Expanded Gas

Contain Propellant

Pass Propellant Over Catalyst

Sense Gas Rate

Control Gas Rate

Export Heat

Complete Mission

Regulate Gas Pressure

Export Excess Propellant

Regulated Gas

Adjusted Rate Gas

Excess Propellant

Conduct AdjustedRate Propellant to Catalyst

Control Propellant Rate

Thrust

Propellant

Adjusted Rate Propellant


Sense Propellant Rate

Sense Gas Pressure

Desired Propellant Rate

Propellant

Adjusted Rate Gas

Conduct Regulated Gas to Propellant

Conduct AdjustedRate Gas

Regulated Gas

Adjusted Rate Gas

Excess Gas

Export Excess Gas

Desired Gas Pressure

Sense Propellant Pressure

Expanded Gas

Conduct Propellant

Real Propellant Pressure

Regulated Gas

Expanded Gas

Real Gas Pressure

Desired Gas Rate

Propellant

Real GasRate

Control Propellant Pressure

Real Propellant Rate

Propellant

Excess Gas2

Inert Gas

Heat

Heat

Catalyst

Process Signal

Import Propellant

Process Signal

Export Excess Gas 2

Import Gas

Import Heat

Expanded Gas Regulated Gas

Propellant H2O2

Inert Gas

Heat

Needed Temperature

Process Signal

Desired Propellant Pressure

53

Heat

Heat

Figure 3.1: Selected functional model for monopropellant propulsion system

54

The question is how to validate the fault behavior simulation generated by the selected model. To answer this question, we apply ideas from model validation methods using expert knowledge, historical data from sub-systems and sensitivity analysis to develop three practical methodologies to guide the designers to validate the model developed in the early design stage. To apply expert knowledge to validate the model in early design stage, we propose a FMEA based method to quantify expert knowledge and validate the model selected in early design. Existing validation methods based on historical data are not directly applicable, since it is assumed that historical data of real system behavior are not available; therefore, we made modifications to the idea to enable the designers to validate the fault behavior generated by the selected functional model in the early design stage utilizing the observed data from one or more subsystem/s operating in other systems. For this method, we use a chi-squared test. The sensitivity analysis method is based on the cost-risk model developed in chapter 2 and it guides designers to focus on sensitive parts of the system, specifically in later design stages. These three methodologies are described in detail in the following subsections. 3.3.1

Apply Expert Knowledge to Validate Functional Model

One approach to validate a model is to have experts with deep knowledge to confirm the simulation results and decide about validity of the model. This strategy is commonly used when data (observations) of real system’s behavior is not available. In this method, experts should be involved in the design process to be able to provide the designers effective technical advice and validate the model through analyzing the failure simulation outputs. For example, in the monopropellant propulsion system design (described in Chapter 2), experts from NASA Ames were

55

involved in the design progress. Based on NASA Ames expert knowledge, a FMEA was conducted to validate the result from fault behavior simulated by functional model. Failure modes and effects analysis (FMEA) is a step-by-step approach for identifying all possible failures in a design of the system. Failures are defined as any errors that affect the completion of the mission for the monopropellant propulsion system. The effects analysis part of FMEA addresses the cause and consequences of each failure. Failures are sorted considering the severity of the consequences of the failures and the chance of occurrence. FMEA is not a one-time analysis: it has to be done in the early design phase, as well as in further design steps and in the lifecycle of the system. The main goal of the FMEA in design process and operation control is to provide improvement strategies to reduce the failures, particularly the ones with higher priority rank [113]. In this research we use a FMEA to quantify expert knowledge about fault behavior of the system in the early design stage when there is no information available about complete system’s behavior. The setup to quantify the expert knowledge is exploring their ideas about possible failure scenarios and probabilities of failure. The schematic shown in Figure 3.2 was applied to develop the FMEA table.

56

Figure 3.2: A monopropellant propulsion system schematic [114]

The failures that affect system mission accomplished are divided into four classes, and in each class the severity and the occurrence have been decided by NASA Ames experts. Table 3.4 represents a developed FMEA for the monopropellant propulsion system based on NASA Ames documents and asking their experts about the occurrence of each failure mode and severity. The severity is a number from 1 to 10, where 10 is the highest and 1 is the lowest. The occurrence is a percentage which shows the number of failure occurrences every 100 missions.

57 Table 3-2: FMEA for monopropellant propulsion system Potential Failure Mode Valve IV1 stuck closed Valve IV2 stuck closed Valve IV3 stuck closed Leakage in IV1 Regulator failure Low propellant level Valve IV3 is stuck open Timer Relay K6 fails to disengage Switch S3 failure Pressure sensor on RG fails Pressure Sensor TK failures Pressure Sensor PT failures Temperature Sensor TK failures Temperature Sensor PT failures

Potential Effect of Failure Failure of the system to provide thrust when commanded Failure of the system to provide thrust when commanded Failure of the system to provide thrust when commanded Failure of the system to provide thrust when commanded Failure of the system to provide thrust when commanded Failure of the system to provide thrust when commanded Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off Continued firing after the system has been commanded off

Class

Occur in hundred

Severity

3

9

2

9

4

8

0.1

10

1 1 1 1 1 1 2 2 2 2 2 2 2 2

Abnormal inert gas path pressure

Inadequate gas pressure

3

Abnormal propellant gas path pressure


3

Pressure regulator failure


3

Low propellant due to leakage


3

Low inert gas due to leakage


3

Catastrophic Failure

Explosion or lose control of the system

4

58

NASA Ames experts concluded that regulator and isolation valve number 4 plays a significant role in mission accomplishment, and they decided to improve the design to that shown in Figure 3.5, with a redundant regulator and isolated valve. The system can switch the gas path to the redundant (IV4-RG2) path when a fault in either IV1 or RG occurs [114,115].

Figure 3.3: Parallel redundant regulator and isolation valve [114]

The result obtained from functional model failure simulation is compared to the FMEA. As shown in Table 3.2, effects of failures can be classified into 3 categories: 1) failure

59

of the system to provide thrust when commanded, 2) continued firing after the system has been commanded off, and 3) inadequate gas pressure. These groups are the same as the end states or final behaviors that we defined for the functional model’s failure simulation. The classes of the failures, occurrence, and severity from the FMEA table matches with our developed end states and proportions obtained from classification of failure scenarios. Note that it is assumed that the initiating fault events are equally probable in the IBFM simulation used to generate failure scenarios. For each category of effects from expert knowledge, the summation of the occurrence provides the chance of having that type of failure (Table 3.3). Table 3-3: Proportions of each category for expert knowledge and simulation Expert Knowledge Failure Effect

Simulation Proportions

Undesired End States

Proportions

Failure of the system to provide thrust when commanded

3%

No Thrust

1%

Continued firing after the system has been commanded off

2%

Too much Thrust

4%


4%

Little Thrust

1%

Catastrophic Failure

0.1%

Loss the system

4%

No Failures

90.9%

Mission Accomplished

90%

Table 3.3 provides the proportion of each category based on the simulation and expert knowledge. We are interested to know if there is any significant difference between the simulated proportions and the proportions claimed by experts for different categories. Using a chi-squared test to compare the proportions quantified from expert knowledge and proportions obtained from simulation resulted in a p-value of 0.1266 which shows statistically there is no evidence to reject the null hypothesis, both sets of

60

proportions are similar. Also, our definition of undesired final behavior and the proportions of scenarios end up with different undesired end state obtained from failure simulation using functional model were discussed and approved by a NASA Ames expert on September 28, 2016 at Oregon State University. 3.3.2

Apply Subsystem Data to Validate Functional Model

This validation phase focuses on utilizing available observed data from some subsystems to validate the simulation results when there is no data available from the entire system behavior. There are statistical tests to specify whether the difference between observation and simulation is meaningful or not. There are parametric methods that require a distribution assumption (described in section 3.2). When observed and simulated data are discrete or categorical, and no distribution assumption is made, non-parametric methods are more precise and robust. What we obtain from a selected functional model simulation in the early design stage is the failure scenarios, which can be divided into distinct categories based on the final behavior of the system. In the monopropellant propulsion system, there is no data from the real system available to validate the functional model simulation, and manufacturing a prototype is not technically or financially feasible in the early design phase. However, there are subsystems that are built into other engineered systems. A NASA Ames expert provided us the information from three years of operation of an outer space system with a similar regulating gas subsystem. In each year, the system completes three missions on average. In three years, only one time the regulate gas function had a malfunction

61

and produced lower pressure than what was expected; in other observations, there were no fault behaviors related to the regulating gas. The modes in the functional model for regulating gas are: 

Nominal Pressured Regulating



Low Pressure Regulating



High Pressure Regulating



No Pressure Regulating



No Gas to Regulate

For the aim of this study, we extract the scenarios with “No Gas to Regulate” because this mode is caused by failure of the previous functions. Figures 3.4 and 3.5 show the histograms for the classified scenarios caused by the subsystem regulate gas failure, for the simulated functional model and the real system. The test hypothesis is: 𝐻0 ∶ 𝑇ℎ𝑒 𝑑𝑒𝑛𝑠𝑖𝑡𝑖𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑤𝑜 ℎ𝑖𝑠𝑡𝑜𝑔𝑟𝑎𝑚𝑠 𝑎𝑟𝑒 𝑏𝑖𝑛 𝑏𝑦 𝑏𝑖𝑛 𝑒𝑞𝑢𝑎𝑙 𝐻𝐴 : 𝑇ℎ𝑒 𝑑𝑒𝑛𝑠𝑖𝑡𝑖𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑤𝑜 ℎ𝑖𝑠𝑡𝑜𝑔𝑟𝑎𝑚𝑠 𝑎𝑟𝑒 𝑛𝑜𝑡 𝑏𝑖𝑛 𝑏𝑦 𝑏𝑖𝑛 𝑒𝑞𝑢𝑎𝑙

62

1800 1600 1400 1200 1000 800 600 400 200 0 Nominal Pressure Regulating




Figure 3.4: Simulation scenarios for selected monopropellant design

8 7 6 5 4 3 2 1 0 Nominal Pressure Regulating




Figure 3.5: Observed scenarios for gas regulator in spacecraft engine

As discussed in chapter 2, fault behavior produced by the functional model is categorical. In other words, the simulation only produces discrete outcomes belonging to one category of final behavior and the observations are also categorical. Therefore, a chi-squared test is applied to compare the observed and simulated behavior.

63 Table 3-4: Observed and simulated proportions for regulating gas End State

Simulation

Observations

1633

8


73

1


18

0

No Pressure

16

0

Nominal Regulating

The chi-squared critical value at 95% confidence interval for 3 degrees of freedom is 1.2115. The P-value is 0.7503, which shows the observed proportions are not significantly different from the expected proportions. 3.3.3

Local Sensitivity Analysis of Selected Functional Model

Sensitivity analysis is an unavoidable part of the validation process [39]. Designers apply the sensitivity analysis to study the effect of each model’s input on the model’s output. In this section, a local sensitivity analysis methodology is proposed to find sensitive functions in the selected model for further study in the design process. This approach is based on the mathematical derivative of the cost function with respect to cost of each faulty behavior. The cost-risk function presented in Equation 2.1 is formulated based on the operation, design, mitigation, failure cost, and proportion scenarios for of failure. Changes in the actual functions in selected functional models only affect the proportion of the system fault behavior and consequential cost of failure. Partial derivatives of the cost function relative to 𝐶𝐼 , 𝐶𝑂 and 𝐶𝑀 are constant values, and they do not provide insight about how the function influences the cost function. Equation 3.8 shows the derivative of the cost function to the cost of failure, which reflects the proportion of having faulty behavior 𝑖 among all unique scenarios. 𝑓 = ∑𝑁 𝑖=1 𝐶𝑅,𝑖 𝑃𝑅,𝑖

(3.8)

64

𝜕𝑓 = 𝑃𝑅,1 , 𝜕𝐶𝑅,1

𝜕𝑓 𝜕𝑓 = 𝑃𝑅,2 , … ., = 𝑃𝑅,𝑁 𝜕𝐶𝑅,2 𝜕𝐶𝑅,𝑁

To quantify each partial derivative in Equation 3.8, the classification of failure scenarios based on the final behavior of the system is used. The total number of failure scenarios that ended with a specific behavior divided by the total number of scenarios generated by the model quantifies each of the partial derivatives. For instance, the classification illustrated in Figure 2.6, reflects the chance of occurrence of each final behavior among all unique failure scenarios, and can be used to quantify the partial derivatives of the cost-risk model. To identify the probability of failure of sensitive functions, we develop a search tool in Python to specify the top sensitive functions. The tool searches for the top repeated functions among all the unique failure scenarios. The pseudo code for the search tool to find the sensitive functions is presented as follow: Read the file of unique scenarios Put the file into a list Capitalize all the words in the list Create an empty dictionary for each line in the list remove all the white space for each word in the line if word exists in the dictionary add 1 to the number of counts for this word; else add the word to the dictionary end (for) end (for) For the selected functional model, the developed search tool determines that “Expanded Gas”, and “Excess Gas” are the most frequent words in the failure simulation results (Figure 3.6).

65

“Heating Gas” causes expanded gas and excess gas issues which is the main root for undesired final behavior of the system.

Figure 3.6: Most frequent words in simulated failure scenarios

In the selected design, the extra heat produced from thrust is saved to apply in the process of heating gas. Next, in the preliminary and detailed design phases, designers should investigate the strategies to enable the design to deal with the heat issue. Figure 3.6 shows the parts of selected functional model (red arrows) that mainly cause the undesired final behavior.

Import Catalyst

Contain Inert Gas

Export Byproducts

Catalyst

Byproducts

Thrust Command

Conduct Expanded Gas

Contain Propellant

Pass Propellant Over Catalyst

Sense Gas Rate

Control Gas Rate

Export Heat

Complete Mission

Regulate Gas Pressure

Export Excess Propellant

Regulated Gas

Adjusted Rate Gas

Excess Propellant

Conduct AdjustedRate Propellant to Catalyst

Control Propellant Rate

Thrust

Propellant



Sense Propellant Rate

Sense Gas Pressure

Desired Propellant Rate

Propellant

Adjusted Rate Gas

Conduct Regulated Gas to Propellant

Conduct AdjustedRate Gas

Regulated Gas

Adjusted Rate Gas

Excess Gas

Export Excess Gas

Desired Gas Pressure

Sense Propellant Pressure

Expanded Gas

Conduct Propellant

Real Propellant Pressure

Regulated Gas

Expanded Gas

Real Gas Pressure

Desired Gas Rate

Propellant

Real GasRate

Control Propellant Pressure

Real Propellant Rate

Propellant

Excess Gas2

Inert Gas

Heat

Heat

Catalyst

Process Signal

Import Propellant

Process Signal

Export Excess Gas 2

Import Gas

Import Heat

Expanded Gas Regulated Gas

Propellant H2O2

Inert Gas

Heat

Needed Temperature

Process Signal

Desired Propellant Pressure

66

Heat

Heat

Figure 3.7: Sensitive parts in the monopropellant functional model

67

The results of the local sensitivity analysis should be consulted throughout the design process. Designers should pay more attention to sensitive functions, especially in the phase when mapping the functional model to a component model. The model validation in this way is viewed as an evolutionary process: as we go through the design process, more information is available which can be applied for further validation and implement updates if needed. Table 3.2 provides a summary of proposed methods in this section to validate the results of the selected model in early design stage. Table 3-5: Methods to validate model selected in the early design stage ID

Functional Model Validation Method

Tools

1

Apply expert ideas to validate functional model

Failure Proportions, FMEA

2

Apply observed data for a subsystem to validate functional model

Failure Proportions,

Apply local sensitivity Analysis to find the most sensitive functions

Cost-Risk Model Derivatives,

3

3.4

Chi-Squared Test

Sensitive Functions Search Tool

CONCLUSION The main contribution of this chapter is to provide practical methodologies to

validate the fault behavior produced by a functional model selected in the early design phase. We reviewed the concept of model validation and its value in complex engineered systems. We classified the existing validation methods based on the information available from the real system’s behavior in the early design phase and for each class, we proposed the tools and techniques. We discussed that some existing methods are not applicable directly to validate fault behavior in early design stage since

68

there is no or inadequate information from the real system behavior. We showed that there is a lack of research on methods to validate a model when there is no or inadequate observed data. We proposed strategies to validate the fault behavior generated by a functional model in the early design stage. In the first proposed method, we showed how to quantify the expert’s knowledge using a FMEA technique to decide about the validation of the failure proportions obtained from the simulated failure scenarios. The second proposed approach applies available observed data related to a subsystem of the model, then the non-parametric chi-squared test is utilized to provide evidence of meaningful differences between the model and observation. The third strategy is based on the local sensitivity analysis. In this method we applied our developed tool to find the most sensitive functions and find the variation in the cost model caused by changes in the sensitive functions. Functions with a high effect on the cost model should receive close attention in further steps of design process, especially when mapping the functional model to a component model. We successfully applied the proposed strategies to validate the functional model of a monopropellant propulsion system selected in the early design stage.

69

: DYNAMIC DESIGN APPROACH USING KALMAN FILTER FOR UNCERTAINTY MANAGEMENT

4.1

INTRODUCTION An engineered system is defined as an assemblage of sub-systems,

hardware/software parts, and people designed to perform a set of tasks to satisfy specified functional requirements and constraints [116]. The traditional approach for designing an engineered system is to establish a pre-defined set of requirements based on market studies and best estimate extrapolations of the current state, and then find the optimal design to satisfy the requirements [117]. Chen developed the decisionbased design approach to design engineered system; the authors apply optimization and utility theory to select the best design [118,119]. However, these approaches are inadequate to respond to changes in initial requirements or changes in operating conditions. This can lead to technical or economic failure if the system is faced with significantly different conditions than the ones predicted. The economic troubles of the Iridium [120] and Globalstar [121] ventures illustrate the significance of resiliency of engineered systems in responding to potential uncertainties in demand. Both of these satellite communication systems in Low Earth Orbit (LEO) forecasted a large number of customers based on a certain type of service (telephony along with fax, messaging, etc.). Although both systems were successful technically, they failed economically due to market changes that had taken place between the time of conceptual design and the time they became operational, resulting in losses of roughly $4 and $3.5 billion, respectively. The proximate cause of these

70

failures is that prior forecasts and expectations were confounded; specifically, the market for wireless telephony went to ground-based competitors that arose between the system conception in 1990, and the launch of the ill-fated communication satellites starting in 1998. The satellite systems were too inflexible to be easily downsized or switched to a different type of service or coverage. Attention to such cases illustrates the coupling of design, performance, and economics, and motivates engineers to investigate ways of making engineered systems resilient to uncertainties in future states. In this part of PhD research we introduced a new dynamic design approach which enables complex engineered systems to overcome such uncertainties in the operational phase. Figure 4.1 shows Globalstar and Iridium satellite systems (images from google).

Figure 4.1: Iridium and Globalstar satellite systems failed due to market changes

The main goal of this chapter is to present an approach for system design to manage the uncertainty in a State of Interest (SoI) throughout the lifecycle of the system. The SoI in this work is the state of an uncertain internal or external condition which directly influences the success of the system throughout its lifecycle. The uncertain condition is usually rooted in the operating environment of the system. Examples of a SoI are the

71

state of health of an aerospace or automotive vehicle (internal state), the demand for a goods or services (external state), or the state of technology for system components (external state). These example SoIs share a common attribute in that they are highly related to the success of a given system, yet they can only be forecasted with considerable uncertainty. Oftentimes, the current state of such conditions can only be inferred through indirect measurement, for example aggregating sensor data to estimate the state of health of an aerospace vehicle, or counting the number of users for data services to estimate the actual service demand. Viewing system design as an optimization problem, the SoI will appear in the objective or constraints, and thus has a quantifiable impact on the values of design variables and hence system performance. The material of the chapter is organized as follows: Section 4.2 proposes a clear and consistent definition of flexibility of a design as a tool to enable resilient design, while carefully disentangling it from discussions on flexibility in the design process. Section 4.3 illustrates the different types of uncertainty in engineered systems with a focus on time-resolved uncertainties. Section 4.4 formalizes the design decisionmaking process and section 4.5 provides the Kalman filter approach to making design selections. Section 4.6 provides a case study of a space borne system design, to illustrate the application of the Kalman filter method. Section 4.7 provides summary and conclusion. 4.2

RESILIENT DESIGN AGAINST EXTERNAL EVENTS Developing resilient engineered systems is of increasing interest in the design

community. Resilient design enables engineering systems to recover from uncertain

72

event occurrences [120]. A resilient system is a system with the ability to monitor the performance consistently and keep an acceptable level of performance in case of undesired internal or external event occurs. Flexibility is an effective means to embed resiliency into an engineering system design; however, flexibility is a word rich in ambiguity. While it is increasingly being used in various fields, few attempts have been made to formally define, quantify, and propose ways of achieving flexibility. Saleh investigated 50 definitions of flexibility in different fields, and proposed the clear definition of flexibility as the ability of a system to respond to changes in initial requirements and objectives, after it has been fielded, in a timely and cost effective way [123]. This definition places the emphasis on change after the system starts operating. Flexibility is necessary as a response to uncertainty in use. If we knew exactly how a system would be used for its design lifetime and beyond, then it would be designed into the system from the beginning. Therefore, flexible systems are understood to be those systems that can be changed or adapted in order to meet new or changed requirements [124-127]. The literature on flexibility in engineering design addresses two distinct problems: the flexibility of the design process [128], and the flexibility of the design itself (not the process through which the product or system is designed). This distinction between the flexibility of the process and the flexibility of the design is often forgotten in the literature, and it adds to the confusion in this emerging field [123]. Flexibility is a key property that should be embedded in the designs of engineered systems to enable resilient designs, particularly as they are being designed to cope with time-resolved

73

uncertainties [129]. In the following sections, descriptions of the different types of uncertainty, and their relation to resilient design, are presented. 4.3

UNCERTAINTY IN COMPLEX ENGINEERED SYSTEMS Uncertainty is the inability to specify something with precision [130], and is the

key driver for the need for resilient designs. Uncertainty influences designs, decisions, and behaviors in a variety of fields, from economics to engineering; reducing uncertainty is costly in both time and resources. While there can be many classifications of uncertainty, for the purpose of this work, uncertainty in engineered systems may be classified as statistical and time-resolved [131]. 4.3.1

Statistical (Aleatory) Uncertainty

Statistic uncertainty is inherent variation associated with a physical system or environment under consideration [132]. Statistical uncertainty goes by many names: aleatory uncertainty, variability, irreducible uncertainty, stochastic uncertainty, noise, risk, type “A” uncertainty, and de re [133]. Statistical uncertainties can be differentiated from time-resolved uncertainties by their representation as quantities that can take on values in a known range or distribution. The mathematical representation most commonly used for statistical uncertainty is a probability distribution function. 4.3.2

Time-Resolved (Epistemic) Uncertainty

Time-resolved uncertainty refers to the types of uncertainties that will be resolved or reduced over the system’s lifecycle. These are traditionally classified as epistemic uncertainties. The key feature of epistemic uncertainty is that the fundamental cause is

74

incomplete information or incomplete knowledge of some characteristic of the system or the environment. Quantification of the impacts of time-resolved, or epistemic, uncertainty is inherently difficult, because most of the existing stochastic tools rely on the specification of probability distributions, and thus do not readily apply to epistemic uncertainty [133]. Epistemic uncertainty also goes by these names: reducible uncertainty, subjective uncertainty, model form uncertainty, state of knowledge, type “B” uncertainty, and de dicto [136]. A key feature of time-resolved uncertainties is that the resolution (or reduction) of the uncertainty provides an opportunity for updated design decision-making, if flexibility has been provided in the design. As an example of this process, future demand for a system is an epistemic uncertain variable as sudden changes in the future may occur due to the introduction of a new technology to the market, changes in government policy, or changes in customers’ interests [137]. Future changes in demand, including changes in the type of demand or the number of users, are unknown when a system is being designed; however, when demand is observed, modifications can be made in response to the state of demand (i.e. the state of interest). Whereas much research has gone into quantifying aleatory uncertainty within the probabilistic framework [138], few approaches have been proposed as the most appropriate mathematical representation for epistemic uncertainty, as well as the accompanying decision-making methodology as the uncertainty is resolved. Possibility theory, first introduced by Zadeh [139], has been applied in many areas to model epistemic uncertainty. Structural design problems with fuzzy variables were investigated [140-142]. This approach can be used, but does not directly address the time-related aspect of the uncertainty. Shafer applied evidence theory to quantify

75

uncertainty when both kinds of uncertainties are presented in a system [143]. Scientists proposed using possibility theory to quantify epistemic uncertainties for robust and/or reliable design [144,145]. Bayesian approaches have been investigated to quantify uncertainty in engineering design [146]. Herzig & Paredis applied a Bayesian method for validation and verification of models for engineering design [147]. Bayesian approaches have the benefit of providing a means for both quantifying uncertainties at a given time, and updating those “beliefs” as “evidence” becomes available. The Bayesian approach has been shown to be effective for quantifying time-resolved uncertainties; however, the approach must be extended for use in forecasting future states of interest. Additionally, appropriate prior distributions must be employed, particularly when “evidence” is low. When considering the role of epistemic uncertainty in the resilient design context, there currently is no consensus on the preferred approach to quantifying epistemic uncertainty in the literature, particularly when the entire lifecycle of the product is considered [148]. Further, there is no approach available to leverage the resolution of uncertainty to enable the design to be optimized in the new, lower uncertainty state. Much work has been dedicated to robustness, which refers to the ability to tolerate perturbations that might affect the system’s functional body. Robustness is usually defined as “the ability of a system to resist change without adapting its initial stable configuration” [149]. Robustness, however, does not address recovery of the system; recovery is a feature of a resilient system. The main concern of this chapter is to use design variables to manage the epistemic uncertainty.

76

In this work, we apply the Kalman filter approach to forecast a future SoI, when a source of epistemic uncertainty occurs. Then based upon this modelled state, the optimal change in the design of the system will be achieved to respond to the new state. This resilient design method is applicable to systems when the ability to change is embedded in the system design. 4.4

DESIGN AND STATE OF INTEREST Before developing a design theory to accommodate time-resolved uncertainties, it

is necessary to specify the relationship between the design and SoI. In general, we propose that four relationships between the design and SoI can occur in the design process, as shown in Figure 4.2.

Figure 4.2: Classification of relationship between Design and SoI

4.4.1

Class (0): Unconstrained Design

In this class of design problem, the system design does not influence the SoI, and the SoI has no influence upon the optimal system design. An example is the design of automobile cup holder and the state of health of the automobile engine, which do not

77

have any discernible relationship. This class is not explored further in this research, since there is no relationship to model. 4.4.2

Class (1): Predictive Design

In this class, the system design influences the SoI; however, the SoI does not influence the design selection. An example is the design of an automobile exterior and the state of consumer preferences for the vehicle in a stable market. In this example, it is assumed that preferences (i.e. the SoI) for vehicle designs are stable, but that we can make a vehicle more (or less) preferred by changing its design. Design decisions for Class 1 designs have been modeled extensively using techniques such as Generalized Linear Models or Discrete Choice Analysis (e.g., Chen et al. 2012) to model the SoI as a function of independent design variables. 4.4.3

Class (2): Reactive Design

In this class, the future SoI has an influence upon the system design selection; however, the design does not influence the SoI. An example is the design of a power grid which must be expanded and reconfigured based upon increasing state of demand due to population growth (i.e. an external source of change). In this example, it is assumed that changing the power grid design cannot drive demand, but that changes in demand (i.e. the SoI) drive changes to the grid design. Class 2 design has been studied primarily from the design for reconfigurability/flexibility standpoint, to provide necessary flexibility in the design to modify the system as needed over time.

78

4.4.4

Class (3): Dynamic Design

In this class, the system design influences the SoI, and the SoI influences the design selection. The future SoI is a function of both the current state as well as the selected design. An example is the design of a smart phone and the state of consumer preferences for smart phones. In this example, at a given time period, preferences may be relatively stable, and this class of design may be similar to the Class 1 design in which the design may be altered to make the smart phone more (or less) preferred. Over time, however, preferences may be changing (e.g., size, shape, and weight preferences, need for apps, etc.) and the design may be altered to respond to these SoI changes, similar to Class 2 design. Another example is design for system health management, where one could make component redesigns to improve the system health in times when system health is stable, while also making redesigns to respond to changes in system health. The Class (3) design case is the case which represents the type of resilient design of interest in this research, and will be the focus of methodology development of the next section. 4.5

DYNAMIC DESIGN APPROACH FOR RESILIENT DESIGN The traditional method of designing an engineered system is to design for a pre-

defined set of requirements. It is based on a forecast of system needs, such as the expected number of users, available technology, or a given system health expectation. This can lead to technical or economic failure if the actual conditions or requirements are significantly different than those predicted [117]. As an example of system design challenges, Figure 4.3 illustrates a typical system design problem in which both

79

aleatory and epistemic uncertainties are present in the “Design Phase”. While it is assumed that the aleatory uncertainties remain for the life of the system, epistemic uncertainties, such as operating budgets, demand for the system, or operating environments, may be resolved throughout the life of the system.

Figure 4.3: Aleatory and epistemic uncertainties in design process

In this section, we present an approach to ensure the design is resilient to changes in the operation state as uncertainties are resolved. The key element in our approach is to use the Kalman filter to compute a best estimate of the future state of interest, and then subsequently update the system design to meet this forecasted future states. A generalized framework of our approach is proposed in Figure 4.4.

80

Figure 4.4: Generalized framework of dynamic design approach

In this framework for an operating resilient system, the Kalman Filter is utilized to estimate the SoI (Section 4.5.1). Optimization is then employed to determine how to update the design to best fit the forecasted operational environment (Section 4.5.2). 4.5.1

Kalman Filter

Kalman filter is a popular method to monitor and estimate an uncertain parameter [150-152]. Kalman filter is powerful because it provides optimal estimation of future based on the current information, then as more observations get available it goes back and correct the estimations and provides better estimation of future [153]. Kalman filter has been the subject of extensive research and application. Work has been done to adapt the Kalman filter for nonlinear and complex systems [154-156]. Even if Kalman filter has received lots of attention in the literature, its application in the design process has not been previously explored. In this section, the definition and equations of Kalman filter are discussed, and their relevance to the proposed design process presented. The Kalman filter estimates a process by using a form of feedback control. It estimates the future state of a process and then receives feedback about the actual state

81

in the form of measurements. The time update equations are used to estimate an uncertain parameter. However, measurement equations correct the estimation obtained from time update equation as more observations are available [157]. The cycle to predict and correct the estimation in Kalman filter is shown in Figure 4.5.

Figure 4.5: The ongoing discrete Kalman filter cycle [158]

4.5.1.1 Predict Equation The Kalman filter addresses the problem of estimating the state 𝑠 ∈ 𝑅 𝑛 of a discrete time-controlled process that is governed by a stochastic difference equation [158]. The general predict equation, adapted for design as opposed to control, is: 𝑠𝑡+1 = 𝑓(𝑠𝑡 , 𝑥𝑡 , 𝑤𝑡 ) = 𝐴𝑡 𝑠𝑡 + 𝐵𝑥𝑡 + 𝑤𝑡

(4.1)

where 𝑠𝑡 is the SoI, 𝑥𝑡 is the design vector, and 𝑤𝑡 is a random variable representing the process model error at time 𝑡 (note that nomenclature is changed from conventional control theory nomenclature for consistency with the design process). The matrix 𝐴 relates the state s at time t to the state at time 𝑡 + 1, in the absence of either a driving function or process noise; thus, A is the state evolution model. The matrix B relates the design vector to the SoI. Thus, the B matrix represents the influence of the design on the SoI. In the context of design, 𝑠𝑡 is a vector of variables which represents

82

the current state in which the design is operating, such as the current demand for data service on a satellite system. 4.5.1.2 Measurement Equation When it is not possible to measure the real value of the uncertain SoI variables, s, directly from the process, we can define the relationship between indirect measurements, z, and the real value of the uncertain SoI variables at time 𝑡 [158]. The measurement equation z∈ 𝑅 𝑚 is defined as: 𝑧𝑡 = 𝑓(𝑠𝑡 , 𝑣𝑡 ) = 𝐻𝑡 𝑠𝑡 + 𝑣𝑡

(4.2)

The m×n matrix H relates the SoI s to the indirect measurements z; thus, H represents the measurement model. The random variable 𝑣𝑡 represents the model error at time t. 4.5.1.3 Kalman Filter Computation We define 𝑠̂𝑡− ∈ 𝑅 𝑛 as the prior estimation of uncertain variable 𝑠 at time 𝑡,. 𝑠̂𝑡 ∈ 𝑅 𝑛 is the posterior estimation of the uncertain variable 𝑠 at time 𝑡 after observing the true value of the uncertain variable 𝑠 at state 𝑡 measured by 𝑧𝑡 . Prior and a posterior estimate errors are defined as [158]: 𝑒̂𝑡− = 𝑠̂𝑡 − 𝑠̂𝑡−

(4.3)

𝑒̂𝑡 = 𝑠𝑡 − 𝑠̂𝑡

(4.4)

A prior estimate error covariance is defined as[158]: 𝑃𝑡− = 𝐸[𝑒̂𝑡− 𝑒̂𝑡− 𝑇 ]

(4.5)

83

A posterior estimate error covariance is given by[158]: 𝑃𝑡 = 𝐸[𝑒̂𝑡 𝑒̂𝑡 𝑇 ]

(4.6)

The equation for the posterior state estimate 𝑠̂𝑡 is derived as a linear combination of the prior estimate 𝑠̂𝑡− and a weighted difference between an actual measurement 𝑧𝑡 and a measurement prediction 𝐻𝑡 𝑠̂𝑡− [156]: 𝑠̂𝑡 = 𝑠𝑡− + 𝐾(𝑧𝑡 − 𝐻𝑡 𝑠̂𝑡− )

(4.7)

The difference (𝑧𝑡 − 𝐻𝑡 𝑠̂𝑡− ) in Equation 4.7 is the measurement residual. Measurement residual reflects the difference between the predicted measurement 𝐻𝑡 𝑠̂𝑡− and the true measurement 𝑧𝑡 . A residual of zero means that the two are the same. The 𝑛 × 𝑚 matrix 𝐾 is the Kalman gain, which minimizes the prior estimate error covariance. Equation for the Kalman gain can be found in the literature. One familiar form of the Kalman gain is given as [158]: 𝑃−𝐻 𝑇

𝐾𝑡 = 𝐻 𝑃−𝑡 𝐻 𝑇𝑡+𝑅 𝑡 𝑡

𝑡

𝑡

(4.8)

where Rt is the measurement model error covariance ((𝑣~𝑁(0, 𝑅)). 4.5.1.4. Optimal Estimation We apply the Kalman filter time update equations to obtain the optimal estimation of the future state 𝑠𝑡+1 . The linear Kalman filter time update equations are given by following equations [158]: − 𝑠̂𝑡+1 = 𝐴𝑡 𝑠̂ 𝑡 + 𝐵𝑥𝑡 − 𝑃𝑡+1 = 𝐴𝑡 𝑃𝑡 𝐴𝑇𝑡 + 𝑄𝑡

(4.9) (4.10)

84

where the – sign indicates an a priori estimate, and Qt is the error covariance of the predict model ( 𝑤~𝑁(0, 𝑄)). Estimation of the update equations 3-10 follows the standard approach outlined in Welch & Bishop [158]. Figure 4.6 clarifies steps to obtain the optimal estimation of future states using the Kalman filter.

Figure 4.6: Steps to obtain optimal state estimation through Kalman filter

4.5.2

Design Selection

The Kalman filter approach treats the design process as a stochastic control problem, in which the design is used to manage the uncertainty throughout the lifecycle of the system. The State of Interest (SoI) is used to describe particular states of an uncertain parameter that may change over time. Given the Kalman filter estimation of the SoI, an optimization model is developed to select the best design for future time steps. The SoI may be appear in the optimization model within the objective function or a constraint function (or both). In the proposed approach, the SoI is assumed to be observed in discrete time. We select the design variable values, using optimization, based on the forecasted SoI in future time period 𝑡. The complete design process is shown in Figure 4.7.

85

At time t; Measure the SoI

Before time t; Estimate the SoI for t

Correct the previous estimation (Applying Kalman Filter)

Estimate the SoI for t+1 (through Kalman filter)

Select the best design (through solving optimization)

Yes Change the design

Is the design optimal?

No

Change control (design) inputs

Figure 4.7: Detailed framework of dynamic design approach

A key challenge in the optimization part is that for the general Class 3 design case, the SoI forecast, 𝑠𝑡+1 , is a function of the optimized design 𝑥𝑡 for time period t. However, the SoI forecast is required to quantify the objective function of the design optimization problem needed for design selection. Thus, the state estimation of Equation 4.9 is coupled with the design selection problem; an efficient means of solving this coupled set of equations must be identified. In the following section, we apply our approach to optimize the design of a communication satellite constellation under uncertain demand for the system service. We employ a decoupled approach to solving the problem to alleviate the coupling issue previously discussed.

86

4.6

CASE

STUDY:

RESILIENT

DESIGN

FOR

COMMUNICATION

SATELLITE SYSTEM In this section, we apply our methodology for designing a LEO communication satellite constellation. The methodology allows prediction of the total system capacity versus total lifecycle cost for a given communication quality. The design vector includes orbital altitude,ℎ, the satellite transmitter power, 𝑃𝑇 , the antenna size,𝐴𝐷 , and the number of satellites, 𝑁𝑡 . As has been described previously, both Iridium and Globalstar met or exceeded given technical requirements, yet they both were economic failures. It appears that the reasons for system success or failure is not strictly determined by adherence to design phase requirements, but rather in the system design over the entire lifecycle. The system design must respond to the combination of aleatory and epistemic uncertainties that ultimately determine the success of the system [159]. A communication satellite constellation is a group of satellites that respond to a specific global data demand, such as phones or pagers [160]. The greatest epistemic uncertainty that a communication satellite constellation faces is the number of users and hence demand for data [161]. In a new approach, de Weck et al. [162] suggests designing reconfigurable satellite systems by adding the ability to change the satellites’ orbits. Based on the orbital change concept, a case study is developed to show the application of Kalman filter to quantify demand (number of users) when a sudden unexpected change occurs.

87

4.6.1

Demand Prediction with Kalman Filter

It is assumed that the initial required capacity for a new communication satellite constellation is initially 50,000 channels and it increases at an uncertain rate during the 15 years of the system lifetime, 𝑇𝑠𝑦𝑠 . The needed channel capacity, 𝑁𝑐ℎ , is calculated based on market studies and best guess extrapolations of current demand. The two quantities that have to be estimated are the expected global number of number of users, 𝑁𝑢𝑠𝑒𝑟 and the average use of the system’s service by each user, 𝐴𝑢𝑠𝑒𝑟 [min/month]. The number of channels that the satellite system needs to satisfy this demand is obtained from Equation 4.11 [163]: 𝑁𝑐ℎ =

𝑁𝑢𝑠𝑒𝑟 . 𝐴𝑢𝑠𝑒𝑟 𝑈𝑠 .

365 . 12

24 . 60

(4.11)

Where 𝑈𝑠 is the global system utilization, a fraction between 0 and 1. For a new satellite constellation, we assume a global subscriber base of 𝑁𝑢𝑠𝑒𝑟 = 2 × 106 , and an average monthly user activity of 𝐴𝑢𝑠𝑒𝑟 = 110 [min/month]. Global system utilization,𝑈𝑠 is typically around 0.1 according to Lutz this number captures the fact that only a fraction of satellites is over inhabited, revenue-generating terrain at any given time [164]. Substituting these values in Equation 4.11 results in a constellation capacity of 𝑁𝑐ℎ = 50,228. It is assumed that we initially launch the constellation consisting of four satellites to the circular orbit of altitude 800 [km], antenna size 1.5 [m], and transmitter power 200 [W] to meet the near-term demand for data.

88

To formulate an objective function for the design, we use total life cycle cost (LCC). LCC includes satellite manufacture, validation testing, launch, deployment, ground station operations, and replenishment. For the proposed initial design, the LCC is approximately $124 million [162]. Because demand is uncertain, there might be significant excess capacity in the system, as in the Iridium and Globalstar cases. In these cases, the revenue stream may be insufficient to recuperate the initial investment. If the actual market is only 10,000 users instead of 100,000, the capacity must be viewed as a wasted investment. Therefore, it is critical that the system be flexible to allow system reconfiguration to ensure positive revenue. While we have selected an initial design (i.e. t = 0), one would want the ability to adjust the capacity of the system to the evolving demand in future time periods. We make estimations of future demand containing interval uncertainty; future demand intervals are estimated using a known methodology, such as evidence [165], and are shown in Table 4.1. Table 4-1: Demand forecast for communication satellite system Year

Predicted Demand

Observed Demand

Difference

1

50000-60000

55,800

800

2

60000-80000

67,900

-2100

3

80000-85000

179,150

96,650

4

85000-90000

?

?

In this problem, the actual observed demand remains within the predicted intervals of Table 4.1 in the first two years, while a sharp unexpected increase in demand occurs in year 3.

89

The Kalman filter is now applied to quantify the future state of demand in year 4 (i.e. t = 4). In this case study, we attempt to estimate the SoI which is the demand, quantified as number of channels 𝑁𝑐ℎ , for the next year. For the measurement model, we estimate the demand as a function of users using Equation 4.11. However, since Equation 4.11 is an estimate, we assume there is an error 10% measurement noise, 𝑅 = 10% (i.e. number of users cannot be measured exactly). Also, we assume that the process is governed by the linear difference and SoI linearly increases from year to year based upon the demand model, leading to a state evolution term = 1.2 . The noisy measurement 𝑧 is the number of users that is related to SoI through Equation 4.11, and 𝐻 is assumed to be constant in the measurement equation. Presuming a small process variance, we let 𝑄 = 1𝑒 − 5. An initial value for 𝑃𝑡− can be computed as the average of previous errors. Table 4.2 presents the assumptions that are applied in this simplified case study. Table 4-2: Kalman filter assumptions for the satellite system Parameter

Assumption

Quantity

R

Measurement model error

10%

A

SoI increases linearly

1.2

x

SoI (Demand for the system)

#Channels

H

Number of users and activity per user are measurable

Constant from equation (4.11)

Q

Process variance

1e-5

𝑷− 𝒕

Prior error

Previous errors average

Applying Equations 4.3 to 4.10, we estimate the demand (SoI) for the next time step, 𝑡 + 1 = 4, as 188,815 channels, shown in Table 4.3.

90 Table 4-3: Demand estimation for satellite system using Kalman filter Measurement Update Prior estimate error

𝑃3−

Time Update −650

Prior estimate error covariance

covariance for 𝒕 = 𝟑 Kalman filter

𝑃4−

65.0067

𝑠̂4−

188,815

for 𝑡 = 4 𝐾3

1.1

Demand estimation for 𝑡 = 4

As the following section describes, the optimal estimation of needed channels for year 𝑡 = 4 enables the ability to choose the best design to meet the new market demand. 4.6.2

Design Optimization

As shown in the previous section, the output of the Kalman filter is the optimal estimation of demand for the next time steps. While the Kalman filter provides an estimate of the future states, it does not provide the optimal design for the future state. The design vector for the LEO communication constellation includes the orbital altitude, ℎ, the satellite transmitter power, 𝑃𝑇 , the antenna size, 𝐴𝐷 , and the number of satellites, 𝑁𝑡 . It is assumed that some parameters like the allocated frequencies and bandwidth are fixed; this is a realistic assumption since frequencies and bandwidth are not under the control of the system designers. Equation 4.12 represents the design vector. 𝑥𝑡 = [ℎ𝑡 , 𝑁𝑡 , 𝐴𝐷 , 𝑃𝑇 ]

(4.12)

Each feasible vector x defines a particular design for the communication satellite constellation. Table 4.4 shows the bounds for design variables in design vector 𝑥.

91

Table 4-4: LEO constellation design variables Variable

Name

Lower Limit

Upper Limit

Unit

𝒉𝒕

Altitude at time t

400

2000

km

𝑷𝒔

Satellites transmission power

200

1800

W

𝑨𝑫

Antenna size

0.5

5.5

m

𝑵𝒕

Number of satellites at time t

4

112

-

An important constraint to consider in the system design is the total designed capacity of the communication satellite system, 𝐶𝑎𝑝𝑇𝑜𝑡 , to ensure that demand is met. The total capacity is a function of the power received, PR, as well at the number of satellites, Nt, and hence is a function of the four design variables. Equation 4.13 shows that capacity must be greater than or equal to demand (in units of minutes) at a time t: 𝑇

𝑠𝑦𝑠 𝐶𝑎𝑝𝑇𝑜𝑡 (𝑃𝑅 , 𝑁𝑡 ) ≥ ∑𝑦=1 [𝐴𝑐ℎ 𝑁𝑐ℎ ]

(4.13)

Where demand is quantified with 𝐴𝑐ℎ , the annual average use of the system per channel [min] and 𝑁𝑐ℎ , the number of channels to meet demand (i.e. the SoI). The received power 𝑃𝑅 is a function of the antenna size, altitude, and the transmitted power. The Friis Transmission Equation 4.14 presents the relationship [166]: 𝜆

2

1 2

𝑃𝑅 = 𝑃𝑇 × 𝐺𝑇 × 𝐺𝑅 × (4𝜋) × (ℎ) × 𝐴𝐷 2 𝑃𝑅 : Power received [watts] 𝑃𝑇 : Power transmitted [watts] 𝐺𝑇 : Gain of transmit antenna (constant)

(4.14)

92

𝐺𝑅 : Gain of receive antenna (constant) 𝜆 : Wavelength 𝐴𝐷 : Antenna size [m] Essentially, this equation demonstrates that the strength of the received signal is a function of the strength of the original transmitted signal, the antenna size, the wavelength corresponding to the frequency of operation, and the altitude of satellite. Equation 4.15 is the model for the system objective function. The objective function 𝐽 captures all the metrics by which the goodness of a particular design can be evaluated. The objective function contains the lifecycle cost, the cost of changing the satellite system design, and the profit obtained from the service that the system can provide. 𝑇

𝑇

𝑇

𝑠𝑦𝑠 𝑠𝑦𝑠 𝑠𝑦𝑠 𝐶𝑜𝑠𝑡 𝑀𝑜𝑑𝑒𝑙 = min(∑𝑦=1 𝐿𝐶𝐶 + ∑𝑦=1 𝐷𝐶𝐶 − ∑𝑦=1 (𝐶𝑃𝐹 × 𝑁𝑐ℎ ))

(4.15)

𝐿𝐶𝐶 : Lifecycle cost [$] 𝐷𝐶𝐶 ∶ Design change cost [$] 𝐶𝑃𝐹 : Cost per function [$/min] to the user 𝑁𝑐ℎ : Number of channels T: System lifetime The lifecycle cost is the sum of the research & development, test and assembly cost, the manufacturing cost, the launch cost, the space and ground operation cost and the replacement costs. The system lifetime, 𝑇𝑠𝑦𝑠 , is the total of years that we expect the system provides the service; in this study, we assume system lifetime is 15 years. The

93

cost per function (𝐶𝑃𝐹) is the cost for service to users and the number of channels is the estimated SoI; the product of 𝐶𝑃𝐹 and 𝑁𝑐ℎ represents profit. The summation of the costs in the objective function should be recovered by the profit obtained from the users. A primary main element of the objective function is the cost of changing the system design, given by Equation 4.17. Design change cost, 𝐷𝐶𝐶, (from time 𝑖 to time 𝑗) includes costs of adding new satellites, the cost of re-orbiting existing satellites, changing the antenna size and power transmitter: 𝑦

𝐷𝐶𝐶 = ∑1 (𝑁𝑖 𝐶𝑖𝑗 )+(𝑁𝑗 − 𝑁𝑖 )𝐶𝑛𝑒𝑤 + 𝐶𝐴

(4.16)

𝑖 : Time step in which design decision is made 𝑗 : Future time step in which a design decision is implemented 𝑁𝑖 , 𝑁𝑗 : The number of satellites at times 𝑖 and j 𝐶𝑖𝑗 : Cost of changing the orbit between altitudes ℎ𝑖 and ℎ𝑗 𝐶𝑛𝑒𝑤 ∶ Total cost of adding a new satellite to the constellation 𝐶𝐴 : Cost of changing the antenna size The cost of adding a new satellite, 𝐶𝑛𝑒𝑤 , includes satellite manufacturing, validation testing, launch, deployment, ground station operations, and replenishment. The new satellite costs are determined from a cost model, SVLCM [159], which provides a rough order-of-magnitude cost estimate based on the dry mass of the spacecraft. The model gives estimates of the development and production costs of unmanned Earth-orbiting spacecraft. The SVLCM is a top-level model derived from

94

the NASA/Air Force Cost Model (NAFCOM) database [159]. The re-orbiting cost, 𝐶𝑖𝑗 , contains the cost of delay, fuel, and propulsion: 𝐶𝑖𝑗 = |(𝑁𝑗 − 𝑁𝑖 ) ∆𝑣𝑖𝑗 |(𝐶𝐹 + 𝐶𝑃 + 𝐶𝐷 )

(4.17)

𝐶𝑖𝑗 : Cost of changing the orbit between altitudes ℎ𝑖 and ℎ𝑗 𝑁𝑖 , 𝑁𝑗 : The number of satellites at times 𝑖 and j ∆𝑣𝑖𝑗 : Velocity change between altitudes ℎ𝑖 and ℎ𝑗 𝐶𝐹 : Cost of fuel to move the satellite from ℎ𝑖 to ℎ𝑗 𝐶𝑃 : Cost of propulsion to move the satellite from ℎ𝑖 to ℎ𝑗 𝐶𝐷 : Cost of delay in data transferring during the orbit reconfiguration The cost of orbital change is a function of excessive fuel that satellites carry. According to data from NASA, $15,800 is the average cost of launching a kilogram of mass to Low Earth Orbit (LEO) [159]. The cost of fuel to move from ℎ𝑖 to ℎ𝑗 is a function of the required ∆𝑣𝑖𝑗 and the mass of the satellite, 𝑀: 𝐶𝐹 = 𝑓(∆𝑣, 𝑀)

(4.18)

Equation 4.19 presents the Hohmann rule to calculate ∆𝑣𝑖𝑗 [167,168]. 𝜇

2∗ℎ

∆𝑣𝑖𝑗 = √ℎ (√ℎ +ℎ𝑗 − 1) 𝑖

𝑖

𝑗

(4.19)

In this equation, 𝜇 is the standard gravitational parameter of the satellite, obtained by multiplying standard gravity (9.806 𝑚/𝑠 2 ) by the mass of the satellite (and ℎ is altitude).

95

4.7

RESULTS To find the optimal design for the constellation that meets the Kalman filter demand

estimate of 188,815 channels (or 2.37e+12 mins.), it is necessary to formulate an optimization problem: this is accomplished based upon minimizing the objective function of Equation 4.15 subject to the capacity constraint of Equation 4.13. An exhaustive search method is conducted to find the total capacity and total cost for each possible design. Table 4.5 shows a part of exhaustive search method that meets the estimated capacity with less total cost. The costs are based upon the work developed by de Weck [169], Table 4-5: Example of exhaustive search values for optimal design of satellite system h [km]

Antenna [m]

Nsat

PT [Watt]

Total Cost [million $]

Capacity Total [min]

800

0.5

4

200

126.22

3.14E+11

800

1.5

4

600

137.19

4.10E+11

800

2.5

4

1000

143.53

6.15E+11

800

3.5

4

1400

166.19

6.28E+11

800

3.5

4

1800

172.53

9.43E+11

400

0.5

8

200

179.04

5.54E+11

400

1.5

8

600

205.11

1.08E+12

400

2.5

8

1000

213.19

1.63E+12

400

3.5

8

1400

257.23

1.66E+12

400

3.5

8

1800

265.32

2.49E+12

400

0.5

16

200

554.84

3.25E+12

400

1.5

16

600

700.00

5.42E+12

An optimal system design for the next time period, i.e., period 𝑡 + 1 = 4 , is obtained as 𝑥4 = [ℎ4 = 400 𝑘𝑚, 𝑁4 = 8, 𝑃𝑇 = 1800 𝑊, 𝐴𝐷 = 3.5 𝑚]. The new design vector represents changing the current four satellites’ orbits from 800 km to 400

96

km and deploying 4 additional satellites to meet the new demand. Also, it requires the antenna length be increased from 1.5m to 3.5 m, and the power be increased substantially through the addition of significantly more solar panels to the constellation. The initial constellation was designed to fulfil the initial demand of 50,000 channels and up to the 100,000 channels. However, market changes led to an unexpected increase in demand for this constellation service at 𝑡 = 3 (i.e. year 3) and hence higher demand in year 4 as estimated by the Kalman filter. The total cost of redesigning the constellation from initial design to the new configuration is estimated to be $265.32 million. If the initial constellation were designed without flexibility, retiring that constellation and launching 8 new satellites to meet the new market would cost a minimum of $391 million. 4.8

CONCLUSION This chapter presented a resilient design methodology for complex systems. We

reviewed the concept of resilient design and its value in engineered systems. Since engineered systems face a variety of epistemic uncertainties, they should be designed with resiliency to be able to overcome the epistemic uncertainties. With the example of Iridium and Globalstar, we showed that the resiliency of an engineered system subject to epistemic uncertainty can be the difference between financial success and failure of the system. We proposed a resilient dynamic design approach for engineered systems. In this approach, we apply the Kalman filter to handle epistemic uncertainty and obtain the optimal estimation of future states of interest. Based upon this estimation, and through developing an optimization problem, we are able to maintain an optimal design throughout the system lifecycle.

97

We presented a case study of a communication satellite constellation, showing how the Kalman filter approach treats design as a stochastic control problem, demonstrating that four flexible design variables can be changed to keep the system operating optimally according to a user selected objective and constraints. This provides the system designers with a self-updating design, negating the need to re-engineer the system whenever operation conditions fall outside of their expected ranges.

98

: CONCLUSIONS

Complex engineered systems are high in cost and complexity, and often incorporate highly sophisticated materials, components, design, and technologies. There are many uncertainties such systems will face throughout their lifecycles due to changes in internal and external conditions. It is desirable for complex engineered systems to be resilient against various uncertainties. The ability to overcome such uncertainties should be embedded into the system from the beginning. However, there is a lack of applicable methodology that shows how to design a resilient system from the early design phase. The weaknesses of traditional methods in complex system design and resilient design are the main motivations behind this PhD research. In developing a complex system, modeling the characterization of the system, and simulation of the system’s behavior are necessary steps. In traditional design methods, complex engineered systems are represented by their component model. The downside of this approach is that, the complexity of the design will be a function of the number of components and the interaction between them. On the other hand, when developing a new design, or in the early design phase, typically the detailed component level of the complete system is not available. In traditional design practice, reliability and robustness techniques are used for improving resilience in a design. However, as discussed before, there is a meaningful difference between resilient, reliable and robust design.

99

The main issue with reliability and robust techniques is the requirement of the detailed component model. In the early design stage, typically designers can not present such specifics of the complete system or the component model. This research proposed an approach to design resilient complex engineered systems in the early design phase. The focus is on applying resiliency techniques into the system from the beginning in early design stage to enable the system to recover from failures caused by internal or external uncertainties. The first part of this research proposed a methodology using functions, behaviors, and design structure as the only information available at the early stages of the design, to guide the designers toward a resilient design. Applying the proposed approach, the ability to recover from failures can be embedded into the system from the beginning. In the second part, two methods based on non-parametric test and expert knowledge were developed to analyze and validate the simulation generated by a selected model in the early design phase. Also, a local sensitivity analysis strategy was proposed to highlight the sensitive functions of the system which could be useful in further stage of the design particularly when mapping the functional model to a component model. In the third part of research, we developed a new dynamic design approach to make the system resilient against the external uncertainties through the life cycle of the system. In our approached, we applied a Kalman filter to obtain the optimal forecast of the uncertain events and utilized the dynamic design features as control variables. By adjusting the design features to optimal estimation of future states, the system becomes resilient against the external uncertainties.

100

This work merged failure analysis, design techniques, optimization and advanced statistical methods to develop a practical and effective technique for designing resiliency and managing uncertainties through the lifecycle of the system. While traditional approaches for uncertainty detection and management are still applicable, the purpose of this research is to provide the system designers a framework that allow them to examine the resilient of a design against failure in the early design stage and control the design through the lifecycle. 5.1 

ACCOMPLISHMENTS AND CONTRIBUTIONS Formulated a new strategy to design complex systems with the ability to cope with internal uncertainties.



Defined and executed a methodology to design resilient complex engineered system in the early design stage; the methodology includes functional model development, and simulation and cost-risk analysis to select the optimal design in early design phase.



Used a custom Python tool to define all the functions, modes, flows and conditions to create a model.



Simulated all failure scenarios and classified them based on the final behavior.



Created the cost-risk model to optimize the design selection.



Literature survey on existing model validation methods

101



Validated result produced by model in the early design stage when there is no or inadequate data, the methodology include, quantify the expert knowledge using FMEA, applying non-parametric tests for subsystems observed data.



Developed a search tool in Python to find most sensitive functions



Successfully applied the methodology to design a resilient monopropellant propulsion system.



Formulated a new strategy to design complex systems with the ability to cope with external uncertainties.



Forecasted future states of an uncertain event using Kalman filter strategy utilizing Matlab.



Developed a cost model to adjust the design features based on the estimations.



Successfully designed a satellite system with dynamic features such as number of satellites, altitude, power transmitter and antenna size to be able to cope with the satellite system demand uncertainty.

5.2

FUTURE WORK While we have presented the framework to design resilient complex systems, future

work is needed to improve the current approach. A primary issue is that the current approach does not address uncertainty in costs estimation. This could be addressed by quantifying such uncertainties with probability distribution functions and creating a utility function, with the cost-risk equation as the selection criterion. Another issue is

102

that we have only demonstrated this approach on a small design space and on a single system design problem (six candidate design for a monopropellant propulsion system). Graph grammars could be implemented to help generate a large number of design alternatives. Finally, development of a tool for the functional models with user interactions would be ideal to simplify the application for industries. Future work also could be extending the proposed methods to actual system design includes testing the methodology on a wider range of design problems with varying levels of complexity, as well as addressing the difficulties in applying proposed equations like Kalman filter in real design optimization problem. 5.3

INTELLECTUAL MERIT The PhD research provides a general framework to design resident complex

systems. In practice, the proposed framework provides a system-level approach, it shows how to develop a functional model, simulate the fault behavior, generate other designs by improving the initial model, and select the most resilient one that enables the system to recover from the internal or external failures. In fact, the proposed approach makes the system resilient against not only internal system failures but unpredictable events and environmental uncertainties through the lifecycle of the system. In addition, it enables the designers to understand the effect of functional model changes on the mitigation of failures and improvement of the system. Furthermore, the novelty of this research is the combination of design, and control methods to improve the resilient of the system both in the design phase and operation phase. On one hand, functional failure analysis and model checking verification process

103

incrementally minimizes the performance malfunction and, on the other hand, the Kalman filter estimation helps to control the system and adjust the design to cope with undesired events. 5.4

BROADER IMPACT In many areas, such as transportation, finance, and human systems, complex

systems have become ubiquitous in modern society, and touch every aspect of human lives. Such systems are inherently and inevitably hazardous by their own nature. The purpose of this research is to provide a practical modeling approach and tools enabling one to integrate resiliency into early design phase. While it is impossible to eliminate all potential uncertainties, a focus on resilient and dynamic design can manipulate system operations to recover from many undesired situations. As a result, systems are designed to fail gracefully. However, improving the ability to recover from the failure depends on how a system is designed back in the early design stage. The framework developed in this research addresses these issue by enabling the automatic failure simulation for different design topologies, design verification, identifying those designs with a higher rank of resilient design and controlling the operating system by adjusting the design features to the optimal estimation of the future states. 5.5

RESEARCH FUNDING

At the end, I would like to thank organizations that provided me the funding for this research. This research was funded by the National Science Foundation (project number CMMI 1349787) and NASA Grant and Cooperative Agreement NNX15AQ90G.

104

BIBLIOGRAPHY

[1] Ullman, David G. "The mechanical design process." (2003). [2] O'Connor, Patrick, and Andre Kleyner. "Practical reliability engineering." John Wiley & Sons, 2012. [3] Barlow, Richard E. "Engineering reliability." Society for Industrial and Applied Mathematics, 1998. [4] Phadke, Madhan Shridhar. "Quality engineering using robust design." Prentice Hall PTR, 1995. [5] Taguchi, Genichi, Subir Chowdhury, and Shin Taguchi. "Robust engineering." McGraw-Hill Professional, 2000. [6] Beyer, Hans-Georg, and Bernhard Sendhoff. "Robust optimization–a comprehensive survey." Computer methods in applied mechanics and engineering 196.33 (2007): 3190-3218. [7] Hund, Edelgard, D. Luc Massart, and Johanna Smeyers-Verbeke. "Operational definitions of uncertainty." Trac trends in analytical chemistry 20.8 (2001): 394-406. [8] Milliken, Frances J. "Three types of perceived uncertainty about the environment: State, effect, and response uncertainty." Academy of Management review 12.1 (1987): 133-143. [9] Chen, Wei, and Kemper Lewis. "Robust design approach for achieving flexibility in multidisciplinary design." AIAA journal 37.8 (1999): 982-989. [10] Ge, Jianhua, M. J. Roemer, and George Vachtsevanos. "An automated contingency management simulation environment for integrated health management and control." Aerospace Conference, 2004. Proceedings. 2004 IEEE. Vol. 6. IEEE, 2004. [11] Ge, Jianhua, et al. "Automated contingency management design for UAVs." AIAA 1st Intelligent Systems Technical Conference, Chicago, Illinois, AIAA. Vol. 6464. 2004. [12] Keshavarzi, Elham, Matthew McIntire, and Christopher Hoyle. "Dynamic Design Using the Kalman Filter for Flexible Systems with Epistemic Uncertainty." ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2015. [13] Saxena, Abhinav, et al. "Automated Contingency Management for Propulsion Systems." Control Conference (ECC), 2007 European. IEEE, 2007. [14] Fiksel, Joseph. "Designing resilient, sustainable systems." Environmental science & technology 37.23 (2003): 5330-5339. [15] Li, Junxuan, and Zhimin Xi. "Engineering Recoverability: A New Indicator of Design for Engineering Resilience." ASME 2014 International Design Engineering Technical

105

Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2014. [16] Yodo, Nita, and Pingfeng Wang. "Engineering Resilience Quantification and System Design Implications: A Literature Survey." Journal of Mechanical Design 138.11 (2016): 111408. [17] Blanchard, Benjamin S., Wolter J. Fabrycky, and Walter J. Fabrycky. "Systems engineering and analysis." Vol. 4. Englewood Cliffs, NJ: Prentice Hall, 1990. [18] Siddiqi, Afreen, and Olivier L. de Weck. “Modeling Methods and Conceptual Design Principles for Reconfigurable Systems.” Journal of Mechanical Design 130.10 (2008): 101102. [19] Tamilselvan, Prasanna, and Pingfeng Wang. "Failure diagnosis using deep belief learning based health state classification." Reliability Engineering & System Safety 115 (2013): 124-135. [20] Hawkins, P. G., and David J. Woollons. "Failure modes and effects analysis of complex engineering systems using functional models." Artificial intelligence in engineering 12.4 (1998): 375-397. [21] Li, Wenyuan. "Risk assessment of power systems: models, methods, and applications." John Wiley & Sons, 2014. [22] Youn, Byeng Dong. "Advances in reliability-based design optimization and probability analysis." Diss. The University of Iowa, 2001. [23] Standard, Military. "Procedures for performing a failure mode, effects and criticality analysis." MIL-STD-1629, November, AMSC Number N3074 (1980). [24] Vesely, William E., et al. "Fault tree handbook." No. NUREG-0492. Nuclear Regulatory Commission Washington DC, 1981. [25] Zang, T. A., et al. "Needs and Opportunities for Risk-Based Multidisciplinary Design Technologies for Vehicles." NASA TM, July (2002). [26] Backman, B. "Design Innovation and Risk Management: A Structural Designer's Voyage into Uncertainty." ICASE Series on Risk-based Design (2000). [27] Liu, Hu-Chen, Long Liu, and Nan Liu. "Risk evaluation approaches in failure mode and effects analysis: A literature review." Expert systems with applications 40.2 (2013): 828838. [28] Smith, Natasha, and Sankaran Mahadevan. "Probabilistic methods for aerospace system conceptual design." Journal of spacecraft and rockets 40.3 (2003): 411-418. [29] Greenfield, Michael A. "NASA's use of quantitative risk assessment for safety upgrades." Space safety, rescue and quality 1999-2000 (2001): 153-159. [30] Choi, K. "Advances in Reliability-Based Design Optimization and Probability Analysis-PART II." ICASE Series on Risk-based Design (2001).

106

[31] Xi, Zhimin, and Ren-Jye Yang. "Reliability analysis with model uncertainty coupling with parameter and experiment uncertainties: a case study of 2014 verification and validation challenge problem." Journal of Verification, Validation and Uncertainty Quantification 1.1 (2016): 011005. [32] Ericson, Clifton A. "Hazard analysis techniques for system safety." John Wiley & Sons, 2015. [33] Mahadevan, Sankaran, Natasha L. Smith, and Thomas A. Zang. "System risk assessment and allocation in conceptual design." (2003). [34] Kurtoglu, Tolga, and Irem Y. Tumer. "FFIP: A framework for early assessment of functional failures in complex systems." ICED, Cite des Sciences et de L’industrie, Paris, France (2007). [35] Stone, Robert B., Irem Y. Tumer, and Michael E. Stock. "Linking product functionality to historic failures to improve failure analysis in design." Research in Engineering Design 16.1-2 (2005): 96-108. [36] Lough, Katie Grantham, Robert B. Stone, and Irem Tumer. "Implementation procedures for the risk in early design (red) method." J Ind Syst Eng 2.2 (2008): 126-143. [37] Lough, Katie Grantham, Robert Stone, and Irem Y. Tumer. "The risk in early design method." Journal of Engineering Design 20.2 (2009): 155-173. [38] Jensen, David C., Irem Y. Tumer, and Tolga Kurtoglu. "Modeling the propagation of failures in software driven hardware systems to enable risk-informed design." ASME 2008 International Mechanical Engineering Congress and Exposition. American Society of Mechanical Engineers, 2008. [39] Jensen, D., Irem Y. Tumer, and Tolga Kurtoglu. "Design of an electrical power system using a functional failure and flow state logic reasoning methodology." San Diego, CA (2009). [40] Jensen, David, Irem Y. Tumer, and Tolga Kurtoglu. "Flow State Logic (FSL) for analysis of failure propagation in early design." ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2009. [41] Krus, Daniel, and Katie Grantham Lough. "Applying function-based failure propagation in conceptual design." ASME 2007 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2007. [42] Kurtoglu, Tolga, and Irem Y. Tumer. "A graph-based fault identification and propagation framework for functional design of complex systems." Journal of mechanical design 130.5 (2008): 051401. [43] Kurtoglu, Tolga, Irem Y. Tumer, and David C. Jensen. "A functional failure reasoning methodology for evaluation of conceptual system architectures." Research in Engineering Design 21.4 (2010): 209-234.

107

[44] Papakonstantinou, Nikolaos, et al. "Capturing Interactions and Emergent Failure Behavior in Complex Engineered Systems at Multiple Scales." ASME 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2011. [45] Sierla, Seppo, et al. "Early integration of safety to the mechatronic system design process by the functional failure identification and propagation framework." Mechatronics 22.2 (2012): 137-151. [46] Tumer, Irem, and Carol Smidts. "Integrated design-stage failure analysis of softwaredriven hardware systems." IEEE Transactions on Computers 60.8 (2011): 1072-1084. [47] Coatanéa, Eric, et al. "A framework for building dimensionless behavioral models to aid in function-based failure propagation analysis." Journal of Mechanical Design 133.12 (2011): 121001. [48] de Kleer, Johan, et al. "Fault augmented modelica models." The 24th International Workshop on Principles of Diagnosis. 2013. [49] Quoilin, Sylvain, et al. "ThermoCycle: A Modelica library for the simulation of thermodynamic systems." Proceedings of the 10 th International Modelica Conference; March 10-12; 2014; Lund; Sweden. No. 96. Linköping University Electronic Press, 2014. [50] McIntire, Matthew G. From Functional Modeling to Optimization: Risk and Safety in the Design Process for Large-Scale Systems. Diss. 2016. [51] Clauset, Aaron. "Five Lectures on Networks." (2014). [52] Van Bossuyt, Douglas, et al. "Risk attitudes in risk-based design: Considering attitude using utility theory in risk-based design." Artificial Intelligence for Engineering Design, Analysis and Manufacturing 26.04 (2012): 393-406. [53] Miller, David W., Col John Keesee, and Mr Cyrus Jilla. "Space systems cost modeling." (2003). [54] Keshavarzi, Elham, et al. "Resilient System Design Using Cost-Risk Analysis with Functional Models." ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2017. [55] Sargent, Robert G. "Verification and validation of simulation models." Proceedings of the 37th conference on Winter simulation. winter simulation conference, 2005. [56] Sargent, Robert G. "An overview of verification and validation of simulation models." Proceedings of the 19th conference on Winter simulation. ACM, 1987. [57] Sargent, Robert G. "Verifying and validating simulation models." Proceedings of the 28th conference on Winter simulation. IEEE Computer Society, 1996.

108

[58] Sargent, Robert G. "Verification, validation, and accreditation: verification, validation, and accreditation of simulation models." Proceedings of the 32nd conference on Winter simulation. Society for Computer Simulation International, 2000. [59] Sargent, Robert G. "Validation and verification of simulation models." Proceedings of the 36th conference on Winter simulation. Winter Simulation Conference, 2004. [60] Sargent, Robert G. "Verification and validation of simulation models." Journal of simulation 7.1 (2013): 12-24. [61] Kleijnen, Jack PC, and Robert G. Sargent. "A methodology for fitting and validating metamodels in simulation." European Journal of Operational Research 120.1 (2000): 14-29. [62] Sanders, P. "DoD Modeling and Simulation (M&S) Verification, Validation, and Accreditation (VV&A)." No. DODI-5000.61. OFFICE OF THE UNDER SECRETARY OF DEFENSE (ACQUISITION AND TECHNOLOGY) WASHINGT ON DC, 1996. [63] Lewis, Robert O. "Independent verification and validation: A life cycle engineering process for quality software." Vol. 11. John Wiley & Sons, 1992. [64] Law, Averill M. "How to build valid and credible simulation models." Proceedings of the 40th Conference on Winter Simulation. Winter Simulation Conference, 2008. [65] Cook, David A., and James M. Skinner. "How to perform credible verification, validation, and accreditation for modeling and simulation." The Journal of Defense Software Engineering (2005). [66] Smyth, Padhraic. "Model selection for probabilistic clustering using cross-validated likelihood." Statistics and computing 10.1 (2000): 63-72. [67] Kohavi, Ron. "A study of cross-validation and bootstrap for accuracy estimation and model selection." Ijcai. Vol. 14. No. 2. 1995. [68] Stone, Mervyn. "An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion." Journal of the Royal Statistical Society. Series B (Methodological) (1977): 44-47. [69] Arlot, Sylvain, and Alain Celisse. "A survey of cross-validation procedures for model selection." Statistics surveys 4 (2010): 40-79. [70] Vuong, Quang H. "Likelihood ratio tests for model selection and non-nested hypotheses." Econometrica: Journal of the Econometric Society (1989): 307-333. [71] Akaike, Hirotogu. "Information theory and an extension of the maximum likelihood principle." Selected Papers of Hirotugu Akaike. Springer New York, 1998. 199-213. [72] Lee, Sik-Yum, and Hong-Tu Zhu. "Maximum likelihood estimation of nonlinear structural equation models." Psychometrika 67.2 (2002): 189-210.

109

[73] Banerjee, Onureena, Laurent El Ghaoui, and Alexandre d’Aspremont. "Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data." Journal of Machine learning research 9.Mar (2008): 485-516. [74] Bagozzi, Richard P., and Youjae Yi. "On the evaluation of structural equation models." Journal of the academy of marketing science 16.1 (1988): 74-94. [75] Morey, Richard D., Jan-Willem Romeijn, and Jeffrey N. Rouder. "The philosophy of Bayes factors and the quantification of statistical evidence." Journal of Mathematical Psychology 72 (2016): 6-18. [76] Wasserman, Larry. "Bayesian model selection and model averaging." Journal of mathematical psychology 44.1 (2000): 92-107. [77] Raftery, Adrian E. "Bayesian model selection in social research." Sociological methodolo [78] Berger, James O., and Luis R. Pericchi. "The intrinsic Bayes factor for model selection and prediction." Journal of the American Statistical Association 91.433 (1996): 109-122. [79] Berger, James O., et al. "Objective Bayesian methods for model selection: introduction and comparison." Lecture Notes-Monograph Series (2001): 135-207. [80] Minhas, Raj, et al. "Using fault augmented modelica models for diagnostics." Proceedings of the 10 the International Modelica Conference; March 10-12; 2014; Lund; Sweden. No. 96. Linköping University Electronic Press, 2014. [81] Xiong, Ying, et al. "A better understanding of model updating strategies in validating engineering models." Computer methods in applied mechanics and engineering 198.15-16 (2009): 1327-1337. [82] Saltelli, Andrea, Karen Chan, and E. Marian Scott, eds. "Sensitivity analysis." Vol. 1. New York: Wiley, 2000. [83] Wagner, Harvey M. "Global sensitivity analysis." Operations Research 43.6 (1995): 948-969. [84] Saltelli, Andrea, et al. "Global sensitivity analysis: the primer." John Wiley & Sons, 2008. [85] Triantaphyllou, Evangelos, and Alfonso Sánchez. "A sensitivity analysis approach for some deterministic multi‐ criteria decision‐ making methods." Decision Sciences 28.1 (1997): 151-194. [86] Cannavó, Flavio. "Sensitivity analysis for volcanic source modeling quality assessment and model selection." Computers & geosciences 44 (2012): 52-59. [87] Gauchi, J-P., et al. "Metamodeling and global sensitivity analysis for computer models with correlated inputs: A practical approach tested with a 3D light interception computer model." Environmental Modelling & Software 92 (2017): 40-56.

110

[88] Likhachev, D. V. "Model selection in spectroscopic ellipsometry data analysis: Combining an information criteria approach with screening sensitivity analysis." Applied Surface Science 421 (2017): 617-623. [89] Storlie, Curtis B., et al. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models." Reliability Engineering & System Safety 94.11 (2009): 1735-1763. [90] Mangan, Niall M., et al. "Model selection for dynamical systems via sparse regression and information criteria." arXiv preprint arXiv:1701.01773 (2017). [91] Hurvich, Clifford M., and Chih-Ling Tsai. "Regression and time series model selection in small samples." Biometrika 76.2 (1989): 297-307. [92] Saltelli, Andrea, et al. "Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index." Computer Physics Communications 181.2 (2010): 259-270. [93] Marzban, Caren, et al. "Variance-Based Sensitivity Analysis." [94] Saltelli, Andrea. "Making best use of model evaluations to compute sensitivity indices." Computer Physics Communications145.2 (2002): 280-297. [95] Mara, Thierry A., and Stefano Tarantola. "Variance-based sensitivity indices for models with dependent inputs." Reliability Engineering & System Safety 107 (2012): 115-121. [96] Annoni, Paola, Rainer Brüggemann, and Andrea Saltelli. "Partial order investigation of multiple indicator systems using variance-based sensitivity analysis." Environmental Modelling & Software26.7 (2011): 950-958. [97] Conover, William J., and Ronald L. Iman. "Rank transformations as a bridge between parametric and nonparametric statistics." The American Statistician 35.3 (1981): 124-129. [98] Mantel, Nathan. "Chi-square tests with one degree of freedom; extensions of the Mantel-Haenszel procedure." Journal of the American Statistical Association 58.303 (1963): 690-700. [99] Browne, Michael W., and Robert Cudeck. "Alternative ways of assessing model fit." Sage focus editions 154 (1993): 136-136. [100] Steiger, James H. "Structural model evaluation and modification: An interval estimation approach." Multivariate behavioral research 25.2 (1990): 173-180. [101] Schoenfeld, David. "Chi-squared goodness-of-fit tests for the proportional hazards regression model." Biometrika 67.1 (1980): 145-153. [102] Conover, William J. "A Kolmogorov goodness-of-fit test for discontinuous distributions." Journal of the American Statistical Association 67.339 (1972): 591-596.

111

[103] Massey Jr, Frank J. "The Kolmogorov-Smirnov test for goodness of fit." Journal of the American statistical Association 46.253 (1951): 68-78. [104] Shafer, Glenn. intelligence (1992): 330-331.

"Dempster-shafer

theory." Encyclopedia

of

artificial

[105] Sentz, Kari, and Scott Ferson. "Combination of evidence in Dempster-Shafer theory." Vol. 4015. Albuquerque: Sandia National Laboratories, 2002. [106] Zadeh, Lotfi A. "Review of a mathematical theory of evidence." AI magazine 5.3 (1984): 81. [107] Liu, Liping, and Ronald Yager. "Classic works of the Dempster-Shafer theory of belief functions: An introduction." Classic works of the Dempster-Shafer theory of belief functions (2008): 1-34. [108] Bae, Ha-Rok, Ramana Grandhi, and Robert Canfield. "Epistemic Uncertainty Quantification Techniques Including Evidence Theory for Large-Scale Structures." Elsevier (2004): 1101–1112. [109] Stamatis, Dean H. "Failure mode and effect analysis: FMEA from theory to execution." ASQ Quality Press, 2003. [110] Van Leeuwen, J. F., et al. "Risk analysis by FMEA as an element of analytical validation." Journal of pharmaceutical and biomedical analysis 50.5 (2009): 1085-1087. [111] Grunske, Lars, Robert Colvin, and Kirsten Winter. "Probabilistic model-checking support for FMEA." Quantitative Evaluation of Systems, 2007. QEST 2007. Fourth International Conference on the. IEEE, 2007. [112] Goddard, Peter L. "Validating the safety of embedded real-time control systems using FMEA." Reliability and Maintainability Symposium, 1993. Proceedings., Annual. IEEE, 1993. [113] Teng, Sheng-Hsien, and Shin-Yann Ho. "Failure mode and effects analysis: an integrated approach for product design and process control." International journal of quality & reliability management 13.5 (1996): 8-26. [114] Tang, Liang, et al. "Simulation-based design and validation of automated contingency management for propulsion systems." Aerospace Conference, 2007 IEEE. IEEE, 2007. [115] Tang, Liang, et al. "Prognostics in the control loop." Working Notes of 2007 AAAI Fall Symposium: AI for Prognostics. 2007 [116] Suh, Nam P. "Axiomatic design theory for systems." Research in engineering design 10.4 (1998):189-209. [117] Sommerville, Ian, and Pete Sawyer. "Requirements engineering: a good practice guide." John Wiley & Sons, Inc., 1997.

112

[118] Chen, Wei, Christopher Hoyle, and Henk Jan Wassenaar. “Decision-based design: integrating consumer preferences into engineering design.” Springer Science & Business Media, 2012. [119] Chen, Wei, Christopher Hoyle, and Henk Jan Wassenaar. "Decision-based design: an approach for enterprise-driven engineering design." Decision-Based Design. Springer London, 2013. 3-11. [120] Pratt, Stephen R., et al. "An operational and performance overview of the IRIDIUM low earth orbit satellite system." Communications Surveys & Tutorials, IEEE 2.2 (1999): 2-10. [121] Budget, Link. "Satellite systems for personal and broadband communications." (2000). [122] Hollnagel, Erik, David D. Woods, and Nancy Leveson. “Resilience engineering: Concepts and precepts.” Ashgate Publishing, Ltd., 2007. [123] Saleh, Joseph H., Gregory Mark, and Nicole C. Jordan. "Flexibility: A MultiDisciplinary Literature Review and a Research Agenda for Designing Flexible Engineering Systems." Journal of Engineering Design 20.3 (2009): 307–323. [124] Saleh, Joseph H. "Weaving Time into System Architecture: New Perspective on Flexibility, Spacecraft Design Lifetime, and On-Orbit Servicing." Massachusetts Institute of Technology, 2002. Print. [125] Lamassoure, Elisabeth, Joseph H. Saleh, and Daniel E. Hastings. "Space systems flexibility provided by on-orbit servicing: Part 2." Journal of Spacecraft and Rockets 39.4 (2002): 561-570. [126] Saleh, Joseph H., Elisabeth Lamassoure, and Daniel E. Hastings. "Space systems flexibility provided by on-orbit servicing: Part 1." Journal of Spacecraft and Rockets 39.4 (2002): 551-560. [127] John, Jacob, and M. B. Kiran. "Flexible Manufacturing System Modeling: A Petrinet Based Survey Approach." n. pag. Google Scholar. Web. 23 Dec. 2014. [128] Suh, Eun Suk et al. "Flexible Platform Component Design under Uncertainty." Journal of Intelligent Manufacturing 18.1 (2007): 115–126. CrossRef. Web. 26 Dec. 2014. [129] Mark, Gregory T. "Incorporating flexibility into system design: a novel framework and illustrated developments." Massachusetts Institute of Technology, 2005. [130] Antonsson, Erik K., and Kevin N. Otto. "Imprecision in engineering design." Journal of Vibration and Acoustics 117.B (1995): 25-32. [131] Thunnissen, Daniel Pierre. "Propagating and mitigating uncertainty in the design of complex multidisciplinary systems." California Institute of Technology, 2005. [132] DeLaurentis, Daniel A., and Dimitri N. Mavris. "Uncertainty modeling and management in multidisciplinary analysis and synthesis." AIAA paper 422 (2000): 5.

113

[133] Igusa, T., S. G. Buonopane, and B. R. Ellingwood. "Bayesian Analysis of Uncertainty for Structural Engineering Applications. " Structural Safety 24.2 (2002): 165–186. Print [134] Helton, Jon C. "Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty." Journal of Statistical Computation and Simulation 57.1-4 (1997): 376. [135] Chen, Xiaoxiao, Eun-Jae Park, and Dongbin Xiu. "A Flexible Numerical Approach for Quantification of Epistemic Uncertainty." Journal of Computational Physics 240 (2013): 211–224. [136] Beynon, Malcolm, Bruce Curry, and Peter Morgan. "The Dempster–Shafer theory of evidence: an alternative approach to multicriteria decision modelling." Omega 28.1 (2000): 3750. [137] Carvalho, A., and M. Puterman. "How Should a Manager Set Prices When the Demand Function Is Unknown?" Univ. British Columbia, Vancouver, BC, Canada (2004): n. pag. Print [138] Oberkampf WL, Helton JC. "Mathematical representation of uncertainty. Nondeterministic approaches." Forum 2001, Seattle, WA, AIAA-2001-1645 [139] Zadeh, Lotfi A. "Fuzzy sets." Information and control 8.3 (1965): 338-353. [140] Wood LK, Otto NK, Antonsson KE. "Engineering design calculations with fuzzy parameters." Fuzzy Sets Syst 1992;53:1–20. [141] Antonsson EK, Otto NK. "Improving engineering design with fuzzy sets. In: Dubois D, Prade H, Yager RR, editors.Fuzzy information engineering: a guided tour of applications" New York: John Wiley & Sons; 1997. [142] Penmetsa RC, Grandhi RV. "Efficient estimation of structural reliability for problems with uncertain intervals." Comput Struct 2002;80:1103–12. [143] Shafer G. "A mathematical theory of evidence." Princeton, NJ: 1976. [144] Youn, Byeng D., et al. "Integration of possibility-based optimization and robust design for epistemic uncertainty." Journal of mechanical design 129.8 (2007): 876-882. [145] Hofer, Eduard, et al. "An approximate epistemic uncertainty analysis approach in the presence of epistemic and aleatory uncertainties." Reliability Engineering & System Safety 77.3 (2002): 229-238. [146] Aughenbaugh, Jason Matthew, and Christiaan J. Paredis. "The value of using imprecise probabilities in engineering design." Journal of Mechanical Design 128.4 (2006): 969-979. [147] Herzig, Sebastian JI, and Christiaan JJ Paredis. "Bayesian Reasoning Over Models." MoDeVVa@ MoDELS. 2014.

114

[148] Dai, Zhihuang, and Zissimo Mourelatos. "Incorporating Epistemic Uncertainty in Robust Design." Chicago, Illinois: ASME, 2003. Print [149] Fowlkes, William Y., and Clyde M. Creveling. “Engineering methods for robust product design: using Taguchi methods in technology and product development.” AddisonWesley, 1995. [150] Kalman, Rudolph Emil. "A new approach to linear filtering and prediction problems." Journal of Fluids Engineering 82.1 (1960): 35-45. [151] Kleeman, Lindsay. "Understanding and Applying Kalman Filtering." Proceedings of the Second Workshop on Perceptive Systems, Curtin University of Technology, Perth Western Australia (25-26 January 1996). N.p., 1996. [152] Maybeck, Peter S., et al. "Stochastic Models, Estimating, and Control." New Directions in Effective Management 8 (1982): 12-14. [153] Sorenson, Harold W. "Least-squares estimation: from Gauss to Kalman." Spectrum, IEEE 7.7 (1970): 63-68. [154] Jacobs, Oliver Louis Robert. "Introduction to control theory." (1974). [155] Wan, Eric A., and Rudolph Van Der Merwe. "The unscented Kalman filter for nonlinear estimation." Adaptive Systems for Signal Processing, Communications, and Control Symposium 2000. AS-SPCC. The IEEE 2000. IEEE, 2000. [156] Julier, Simon J., and Jeffrey K. Uhlmann. "A new extension of the Kalman filter to nonlinear systems." Int. symp. aerospace/defense sensing, simul. and controls. Vol. 3. No. 26. 1997. [157] Welch, Greg, and Gary Bishop. "An introduction to the Kalman filter." (1995). [158] Welch, Greg, and Gary Bishop. "An introduction to the kalman filter. Department of Computer Science, University of North Carolina." ed: Chapel Hill, NC, unpublished manuscript (2006). [159] Siddiqi, Afreen. "Reconfigurability in Space Systems: Architecting Framework and Case Studies." Massachusetts Institute of Technology, 2006. [160] Wertz, James Richard, David F. Everett, and Jeffery John Puschell, eds. "Space Mission Engineering: The New SMAD". Microcosm Press, 2011. [161] De Weck Olivier, Eckert Claudia M., Clarkson P. John, and others. "A Classification of Uncertainty for Early Product and System Design." Guidelines for a Decision Support Method Adapted to NPD Processes (2007). [162] De Weck, Olivier L., Richard De Neufville, and Mathieu Chaize. "Staged deployment of communications satellite constellations in low earth orbit." Journal of Aerospace Computing, Information, and Communication 1.3 (2004): 119-136.

115

[163] Simon, Marvin K., and Mohamed-Slim Alouini. “Digital communication over fading channels.” Vol. 95. John Wiley & Sons, 2005. [164] Lutz, Erich, Markus Werner, and Axel Jahn. ”Satellite systems for personal and broadband communications.” Springer Science & Business Media, 2012. [165] Agarwal, Harish, et al. "Uncertainty quantification using evidence theory in multidisciplinary design optimization." Reliability Engineering & System Safety 85.1 (2004): 281-294. [166] Richharia, Madhavendra. "Satellite communications principles(Book)." New York: McGraw-Hill, Inc, 1995. (1995).

systems-

Design

[167] Abdelkhalik, Ossama, and Daniele Mortari. "N-impulse orbit transfer using genetic algorithms." Journal of Spacecraft and Rockets 44.2 (2007): 456-460. [168] Belbruno, Edward A., and James K. Miller. "Sun-perturbed Earth-to-Moon transfers with ballistic capture." Journal of Guidance, Control, and Dynamics 16.4 (1993): 770-775. [169] De Weck, Olivier L., Uriel Scialom, and Afreen Siddiqi. "Optimal Reconfiguration of Satellite Constellations with the Auction Algorithm." Acta Astronautica 62.2-3 (2008): 112–130

116

APPENDICES APPENDIX A-1: FUNCTIONAL MONOPROPELLANT SYSTEM @author: keshavarzi CompleteMission ConductAdjustedRateGas ConductAdjustedRatePropellantToCatalyst ConductExpandedGas ConductPropellant ConductRegulatedGasToPropellantTank ContainCatalyst ContainInertGas ContainPropellant ControlGasRate ControlPropellantRate ControlTankTemperatureAndPressure ExportByproducts ExportExcGas ExportExcessGas2 ExportExcessPropellant ExportHeat ImportCatalyst ImportGas ImportHeat ImportPropellant PassPropellantOverCatalyst ProcessSignal RegulateGasPressure SenseGasPressure SenseGasRate SensePropellantRate SensePropellantTankPressure SensePropellantTankTemperature

DEFINITION

IN

PYTHON

FOR

117

APPENDIX A-2: GRAPHICAL REPRESENTATION MONOPROPELLANT FUNCTIONAL MODEL IN SPYDER

OF

@author: keshavarzi #from visualize import plotPgvGraph import networkx as nx from pylab import * #import matplot.python as plt G= nx.DiGraph() pos = {0: (0, 100), 1: (2200, 100), 2: (2400, 200), 3: (3650, 200), 4: (4800, 100), 5: (4800, 0), 6: (3550, 0), 7: (2100, 0), 8: (0, 0), 9: (0, 50)} #pos=nx.spring_layout(G) nx.draw_networkx_nodes(G, pos, node_size=2500, nodelist=[0,1,2,3,4,5,6,7,8,9], node_color='w', alpha=0.2, node_shape='s') nx.draw_networkx_edges(G,pos,edgelist=[(0,1),(1, 2),(2,3),(1,4),(4,5),(5,6),(5,2),(6,7),(7,8),(9,3),(6,9),(9,7),(9,5)],width=1,alpha=0.2,edge_color='b') ''' # Inert GAS G.add_node('ImpGas', function='ImportGas') G.add_node('ContainGas', function='ContainGasInContainter') G.add_node('ConductExcGas', function='ConductExcessGas') G.add_node('ExGas', function='ExportGas') G.add_node('ConductExpGas', function='ConductExpandedGasToSystem') G.add_node('CtrlGasFlow', function='ControlGasFlow') G.add_node('MeasGasPress', function='MeasureGasPressure') G.add_node('RegGasPress', function='RegulateGasPressure') G.add_node('ConductRegGas', function='ConductRegGasToPropallent') G.add_node('importBS1',function='ImportBinarySignal') G.add_edge('ImpGas', 'ContainGas', bond='InertGas') G.add_edge('ContainGas', 'ConductExcGas', bond='ExcessGas') G.add_edge('ConductExcGas','ExGas', bond='ExcessGas') G.add_edge('ContainGas', 'ConductExpGas', bond='ExpandedGas') G.add_edge('ConductExpGas','CtrlGasFlow', bond='ExpandedGas') G.add_edge('CtrlGasFlow','MeasGasPress', bond='ExpandedGas') G.add_edge('CtrlGasFlow','ConductExcGas', bond='ExcessGas') G.add_edge('MeasGasPress', 'RegGasPress', bond='ExpandedGas') G.add_edge('RegGasPress', 'ConductRegGas', bond='RegulatedGas') G.add_edge('importBS1','CtrlGasFlow',bond='Signal') G.add_edge('MeasGasPress','importBS1',bond='Signal') G.add_edge('importBS1','RegGasPress',bond='Signal') G.add_edge('importBS1','ExGas',bond='Signal') ''' labels={} labels[0]=r'$ImportGas$' labels[1]=r'$ContainGas$' labels[2]=r'$ConductExcessGas$' labels[3]=r'$ExportGas$' labels[4]=r'$ConductExpandedGas$'

118

labels[5]=r'$ControlGasFlow$' labels[6]=r'$MeasureGasPressure$' labels[7]=r'$RegulateGasPressure$' labels[8]=r'$ConductRegGasToPropallent$' labels[9]=r'$ImportSignal$'

nx.draw_networkx_labels(G,pos,labels,font_size=9) #nx.draw_networkx_labels(G, pos, font_size=16, font_color='g', font_family='sans-serif') #plt.axis('off') #plt.savefig("labels_and_colors.png") nx.draw_spectral(G) plt.show() plot(G) show()

Functional model representation using our developed tool in Spyder

119

APPENDIX A-3: MODES MONOPROPELLANT SYSTEM

DEFINITION

IN

PYTHON

@author: keshavarzi # ImportGas mode NominalGasSource InertGas output effort = Nominal mode NotCompleteGasSource InertGas output effort = Low mode NoGasSource InertGas output effort = Zero # ImportHeat(SinkHeat) mode NominalHeatSource Heat output effort = Nominal mode LowHeatSource Heat output effort = Low mode HighHeatSource Heat output effort = High mode NoHeatSource Heat output effort = Zero # ContainInertGas mode NominalContaining ExpandedGas output effort = max(Heat input effort, InertGas input effort) mode SlowLeakage ExpandedGas output effort = max(Heat input effort, InertGas input effort) -mode SeveareLeakage ExpandedGas output effort = min(Heat input effort, InertGas input effort) -InertGas input rate = ExpandedGas output rate Heat input rate = ExpandedGas output rate mode NoContaining ExpandedGas output effort = Zero mode NoInertGasToContain InertGas input rate = Zero # ConductExpandedGas mode NominalConductingExpandedGas ExpandedGas input effort = ExpandedGas output effort mode PartialConductingExpandedGas ExpandedGas output effort = ExpandedGas output effort -mode NoConductingExpandedGas ExpandedGas output effort = Zero mode NoExpandedGastoConduct ExpandedGas input rate = Zero # SenseGasRate mode NominalRateSensing import NominalConductingExpandedGas SignalRealGasRate output effort = ExpandedGas input rate mode DriftingLowRateSensing import NominalConductingExpandedGas

FOR

120

SignalRealGasRate output effort = ExpandedGas input rate -mode DriftingHighRateSensing import NominalConductingExpandedGas SignalRealGasRate output effort = ExpandedGas input rate ++ mode NoRateSensing import NominalConductingExpandedGas SignalRealGasRate output effort = Zero mode NoGasToSenseRate ExpandedGas input rate = Zero # ControlGasRate mode NominalControling MinimumEffort in SignalDesiredGasRate ExpandedGas output AdjustedRateGas ExcessGas AdjustedRateGas output effort = Nominal ExcessGas output effort = mode ReleaseLessThanNeeded ExcessGas output effort = min(SignalDesiredGasRate inpput effort, ExpandedGas input effort) -AdjustedRateGas output rate = High AdjustedRateGas output effort = High mode ReleaseMoreThanNeeded ExcessGas output effort = min(SignalDesiredGasRate input effort, ExpandedGas input effort) ++ AdjustedRateGas output rate = Low AdjustedRateGas output effort = Low mode StuckOpen ExpandedGas input rate = ExcessGas output rate AdjustedRateGas output effort = Zero mode StuckClosed ExcessGas output effort = zero AdjustedRateGas output effort = ExpandedGas input effort * Highest ExpandedGas input rate = AdjustedRateGas output rate mode NoExpandedGasToControl ExpandedGas input rate = Zero #ProcessSignal mode NominalSignalSource Signal output effort = Nominal mode ZeroSignalSource Signal output effort = Zero # ExportExceccGas mode NominalExcessGasSink ExcessGas input effort = Nominal mode InsulatedExcessGasSink ExcessGas input rate = Low mode NoExcessGasSink ExcessGas input rate = Zero # ConductAdjustedRateGas mode NominalConductingAdjustedRateGas AdjustedRateGas input effort = AdjustedRateGas output effort mode PartialConductingAdjustedRateGas AdjustedRateGas input effort = AdjustedRateGas output effort --

121

mode NoConductingAdjustedRateGas AdjustedRateGas output effort = Zero mode NoAdjustedRateGasToConduct AdjustedRateGas input rate = Zero # SenseGasPressure mode NominalPressureSensing import NominalConductingAdjustedRateGas SignalRealGasPressure output effort = Nominal mode DriftingLowPressureSensing import NominalConductingAdjustedRateGas AdjustedRateGas input effort = SignalRealGasPressure output effort -mode DriftingHighPressureSensing import NominalConductingAdjustedRateGas AdjustedRateGas input effort = SignalRealGasPressure output effort ++ mode NoPressureSensing import NominalConductingAdjustedRateGas SignalRealGasPressure output effort = Zero mode NoGasToSensePressure AdjustedRateGas input rate = Zero # RegulatingGasPressure mode NominalPressureRegulating RegulatedGas output effort = min(SignalDesiredGasPressure input effort, AdjustedRateGas input effort) mode LowPressureRegulating RegulatedGas output effort = min(SignalDesiredGasPressure input effort, AdjustedRateGas input effort) -mode HighPressureRegulating RegulatedGas output effort = min(SignalDesiredGasPressure input effort, AdjustedRateGas input effort) ++ mode NoPressureRegulating AdjustedRateGas input effort = RegulatedGas output effort mode NoPressureToRegulate optional AdjustedRateGas input rate = Zero optional SignalDesiredGasPressure input effort = Zero RegulatedGas output effort = Zero # ConductRegulatedGasToPropellantTank mode NominalConductingRegulatedGas RegulatedGas input effort = RegulatedGas output effort mode PartialConductingRegulatedGas RegulatedGas input effort = RegulatedGas output effort -mode NoConductingRegulatedGas RegulatedGas output effort = Zero mode NoRegulatedGasToConduct RegulatedGas input rate = Zero #Propellant # ImportPropellant mode NominalPropellantSource Propellant output effort = Nominal mode LowPropellantLevel Propellant output effort = Low

122

mode NoPropellantSource Propellant output effort = Zero # ContainPropellant mode NominalContaining Propellant output effort = Nominal mode SlowLeakage Propellant output effort = Low mode SeveareLeakage Propellant output effort = Lowest mode NoContaining Propellant output effort = Zero mode NoPropellantToContain Propellant input rate = Zero #SensePropellantTankPressure mode NominalPressureSensing import NominalConductingRegulatedGas SignalDesiredPTPres output effort = RegulatedGas input effort mode DriftingLowPressureSensing import NominalConductingRegulatedGas SignalDesiredPTPres output effort = RegulatedGas input effort -mode DriftingHighPressureSensing import NominalConductingRegulatedGas SignalDesiredPTPres output effort = RegulatedGas input effort ++ mode NoPressureSensing import NominalConductingRegulatedGas SignalRealGasPressure output effort = Zero mode NoRegulatedGasToSensePressure RegulatedGas input rate = Zero #SensePropellantTankTemperature mode NominalTemperatureSensing import NominalConductingRegulatedGas NominalEffort output SignalRealPTTemp output effor = RegulatedGas input effort mode DriftingLowTemperatureSensing import NominalConductingRegulatedGas DecreasedEffort input RegulatedGas output SignalRealPTTemp mode DriftingHighTemperatureSensing import NominalConductingRegulatedGas IncreasedEffort input RegulatedGas output SignalRealPTTemp mode NoTemperatureSensing import NominalConductingRegulatedGas ZeroEffort output SignalRealPTTemp mode NoRegulatedGasToSenseTemperature ZeroRate input RegulatedGas #ControlTankTemperatureAndPressure mode NominalControling2 NominalEffort output Propellant ExcessGas2 mode ReleaseLessThanNeeded2 DecreasedMinimumEffort in SignalDesiredPTPres SignalDesiredPTTemp RegulatedGas output ExcessGas2

123

IncreasedMinimumEffort in SignalDesiredPTPres SignalDesiredPTTemp RegulatedGas output Propellant mode ReleaseMoreThanNeeded2 IncreasedMinimumEffort in SignalDesiredPTPres SignalDesiredPTTemp RegulatedGas output ExcessGas2 DecreasedMinimumEffort in SignalDesiredPTPres SignalDesiredPTTemp RegulatedGas output Propellant mode StuckOpen2 EqualRate in RegulatedGas output ExcessGas2 AllZeroRate in Propellant AllZeroEffort output Propellant mode StuckClosed2 IncreasedRate output Propellant AllZeroRate in ExcessGas2 ZeroEffort output ExcessGas2 mode NoRegulatedGasToControl ZeroRate in RegulatedGas #ExportExcessGas2 mode NominalExcessGas2Sink ReflectiveRate in ExcessGas2 mode InsulatedExcessGas2Sink LessReflectiveRate in ExcessGas2 mode NoExcessGas2Sink ZeroRate in ExcessGas2 #ConductPropellant mode NominalPropellantConducting NominalEffort output Propellant EqualRate in Propellant output Propellant mode PartialPropellantConducting DecreasedEffort in Propellant output Propellant mode NoConducting ZeroEffort output Propellant mode NoPropellantToConduct ZeroRate in Propellant ZeroEffort output Propellant #SensePropellantRate mode Operational NominalPropellantRateSensing from NominalPropellantConducting TranslateRateToEffort output Propellant SignalRealPropellantRate mode Degraded DriftingLowPropellantRateSensing from NominalPropellantConducting TranslateDecreasedRateToEffort output Propellant SignalRealPropellantRate mode Degraded DriftingHighPropellantRateSensing from NominalPropellantConducting TranslateIncreasedRateToEffort output Propellant SignalRealPropellantRate mode Failed NoPropellantRateSensing from NominalPropellantConducting ZeroEffort output SignalRealPropellantRate mode Failed NoPropellantToSenseRate ZeroRate in Propellant

124

ZeroEffort output SignalRealPropellantRate ZeroEffort output Propellant #ControlPropellantRate mode NominalPropellantControling MinimumEffort in SignalDesiredPropellantRate Propellant output AdjustedRatePropellant ExcessPropellant mode ReleasePropellantLessThanNeeded DecreasedMinimumEffort in SignalDesiredPropellantRate Propellant output ExcessPropellant IncreasedMinimumEffort in SignalDesiredPropellantRate Propellant output AdjustedRatePropellant mode ReleasePropellantMoreThanNeeded IncreasedMinimumEffort in SignalDesiredPropellantRate Propellant output ExcessPropellant DecreasedMinimumEffort in SignalDesiredPropellantRate Propellant output AdjustedRatePropellant mode PropellantReleaseValveStuckOpen EqualRate in Propellant output ExcessPropellant AllZeroRate in AdjustedRatePropellant AllZeroEffort output AdjustedRatePropellant mode PropellantReleaseValveStuckClosed EqualRate in Propellant output AdjustedRatePropellant IncreasedEffort output AdjustedRatePropellant ZeroEffort output ExcessPropellant ZeroRate in ExcessPropellant mode NoPropellantToControl ZeroRate in Propellant #ExportExcessPropellant mode NominalExcessPropellantSink ReflectiveRate in ExcessPropellant mode InsulatedExcessPropellantSink LessReflectiveRate in ExcessPropellant mode NoExcessPropellantSink ZeroRate in ExcessPropellant #ConductAdjustedRatePropellantToCatalyst mode NominalAdjustedRatePropellantConducting NominalEffort output AdjustedRatePropellant EqualRate in AdjustedRatePropellant output AdjustedRatePropellant mode PartialAdjustedRatePropellantConducting DecreasedEffort in AdjustedRatePropellant output AdjustedRatePropellant mode NoAdjustedRatePropellantConducting ZeroEffort output AdjustedRatePropellant mode NoAdjustedRatePropellantToConduct ZeroRate in AdjustedRatePropellant #### catalyst #ImportCatalyst mode NominalCatalystSource NominalEffort output Catalyst mode InsuficientCatalystSource LowEffort output Catalyst mode NoCatalyst ZeroEffort output Catalyst

125

#ContainCatalyst mode NominalCatalystSource NominalEffort output Catalyst mode InsuficientCatalystSource DecreasedEffort in Catalyst output Catalyst mode NoCatalyst ZeroEffort output Catalyst #PassPropellantOverCatalyst mode NominalContact TranslateRateToEffort output Thrust Heat Byproducts NominalEffort output Thrust Heat Byproducts mode LowContact TranslateDecreasedRateToEffort output Thrust Heat Byproducts DecreasedEffort in Catalyst AdjustedRatePropellant CreatThrustSignal output Thrust mode NoContact ZeroEffort output Heat ZeroEffort output Thrust ZeroEffort output Byproducts mode NoThrustCommandSignal ZeroRate in CreatThrustSignal ZeroEffort output Heat ZeroEffort output Thrust ZeroEffort output Byproducts mode NoCatalystToProduceThrust ZeroRate in Catalyst ZeroEffort output Heat ZeroEffort output Thrust ZeroEffort output Byproducts mode NoPropellantToProduceThrust ZeroRate in AdjustedRatePropellant ZeroEffort output Heat ZeroEffort output Thrust ZeroEffort output Byproducts mode InefficientPropellantToProduceThurst DecreasedEffort in AdjustedRatePropellant output Thrust DecreasedEffort output Thrust DecreasedEffort output Heat DecreasedEffort output Byproducts mode InefficientCatalystToProduceThurst DecreasedEffort in Catalyst output Thrust DecreasedEffort output Thrust DecreasedEffort output Heat DecreasedEffort output Byproducts mode HighRatePropellantMoreThanNeededThurst IncreasedEffort in AdjustedRatePropellant output Thrust IncreasedEffort output Thrust IncreasedEffort output Heat IncreasedEffort output Byproducts mode NonTolerableHighRatePropellantSystemExplosion HighestEffort in AdjustedRatePropellant output Heat Explosion

126

mode EarlyThrustCommand IncreasedEffort in CreatThrustSignal DecreasedEffort output Thrust mode LateThrustCommand DecreasedEffort in CreatThrustSignal IncreasedEffort output Thrust #ExportByproducts mode NominalByproductsSink ReflectiveRate in Byproducts mode InsulatedByproductsSink LessReflectiveRate in Byproducts mode NoByproductsSink ZeroRate in Byproducts #ExportHeat mode NominalHeatSink ReflectiveRate in Heat mode InsulatedHeatSink LessReflectiveRate in Heat mode NoHeatToExport ZeroRate in Heat mode NoHeatExport ZeroEffort output Heat HighestEffort output Explosion #CompleteMission mode MissionAccomplished NominalEffort in Thrust mode PassMissionLocation IncreasedEffort in Thrust mode NotReachToMissionLocation DecreasedEffort in Thrust mode NoMovements ZeroEffort in Thrust mode LossTheSystem HighestEffort in Explosion

127

APPENDIX B: R CODE FOR CHI-SQUARED TEST TO COMPARE FMEA RESAUL AND SIMULATION #Compare FMEA and simulation

128

APPENDIX C: R CODE FOR CHI-SQUARED TEST TO COMPARE OBSERVED AND SIMULATION

129

APPENDIX D: PYTHON CODE TO FIND SENSITIVE FUNCTIONS #find the most repeated in unique failure scenarios import re name = raw_input("Enter file:") my_string = open(name) words = re.findall(r'\w+', my_string) #This finds words in the document from collections import Counter cap_words = [word.upper() for word in words] #capitalizes all the words word_counts = Counter(cap_words) #counts the number each time a word appears print(word_counts) for line in handle: wordList = line.split() for word in wordList: if word in wordDict: wordDict[word] += 1 else: wordDict[word] = 1 print wordDict

130

APPENDIX E: KALMAN FILTER CODE IN MATLAB TO FORECAST UNCERTAIN EVENT function [residuals,xhatOut]=EXTKALMAN(meas,deltat) %This program is executed as a matlab function block in the aro_radmod % Simulink model. There is a file called "aero_radat.m" that contains the % data needed to run the aero_radmod model. The estimated and actual % positions are saved to the workspace and are plotted at the end of % simulation by the program aero_radplot (called from the simulation % automatically.)

%Initialization persistent P ; persistent xhat; if isempty(P) xhat=[0.001;0.1;0.001;400]; P= zeros(4); end % Radar update time is inherited from model workspace %1)Compute Phi, Q (covariance of state equation error) and and R (measurement %equation error) Phi= [1 deltat 0 0;0 1 0 0; 0 0 1 deltat; 0 0 0 1]; Q= diag([0.005 0.005]); R= diag([0.005 0.005]); %2)Propogate the covariance matrix P=Phi*P*Phi' +Q; %3)Propogate the track estimate xhat=Phi*xhat; %4 a) Compute observation estimes Rangehat = sqrt(xhat(1)^2 + xhat(3)^2); Bearinghat = atan2(xhat(3),xhat(1)); %4 b)Compute observation vector y and linearized measurement matrix M

131

yhat=[Rangehat; Bearinghat]; M= [cos(Bearinghat) 0 sin(Bearinghat) 0 -sin(Bearinghat/Rangehat 0 cos(Bearinghat)/Rangehat 0]; %4 c) Compure residual (Estimate Error) residual=meas-yhat; %5)Compute Kalman gain W = P*M'*inv(M*P*M'+R); %6)Update estimate xhat=xhat+W*residual; %7)Update covariance matrix P= (eye(4)-W*M)*P*(eye(4)-W*M)' +W*R*W'; end

Resilient Design for Complex Engineered Systems in the Early Design

Resilient Design for Complex Engineered Systems in the Early Design

Suggest Documents

Complex Systems Design Across Cultures

Can Complex Systems Be Engineered?

Thermo-mechanical Design of Resilient Contact Systems for Wafer ...

The virtual reality for the design of complex systems ...

early warning systems (e-wars) design for early

Introducing systems-oriented design for complex societal contexts in ...

embedded design intelligence in systems for the early phases of design

SoS Architecture - Complex Systems Design & Management ...

Modularity in the Design of Complex Engineering Systems - CiteSeerX

Methods to Exploit Noise in the Design of Complex Systems

Complex Systems Design Across Cultures - Semantic Scholar

MORE EFFICIENT DESIGN OF COMPLEX AIRCRAFT SYSTEMS

dEsIgn and IMplEMEntatIon of sUrvEIllancE systEMs for thE Early ...

resilient interface design for safety-critical

Planning for Bushfire Resilient Urban Design

Case Study: Resilient Backbone Design for IPTV

Resilient Web Design - Amazon AWS

Resilient Web Design - Amazon AWS

Resilient Web Design - Amazon AWS

Resilient Web Design - Amazon AWS

Genetically Engineered Architecture: Design ...

Genetically Engineered Architecture: Design ...

complex building performance analysis in early stages of design - IBPSA

complex building performance analysis in early stages of design