Heritage Technologies in Space Programs ...

Institute of Astronautics Technical University of Munich

Heritage Technologies in Space Programs – Assessment Methodology and Statistical Analysis Rigorosum Dipl.-Ing. Andreas Makoto Hein zur Erlangung des akademischen Grades eines Doktor-Ingenieurs (Dr.-Ing.) am 18.11.2016 Prüfungsvorsitzender: Prof. Dr.-Ing. Mirko Hornung (TUM) 1. Gutachter: Prof. Prof. h.c. Dr. Dr. h.c. Ulrich Walter (TUM) 2. Gutachter: Prof. Edward F. Crawley, Ph.D., (MIT, USA)


Motivation: Why are Heritage Technologies Relevant? Most space programs rely on so-called „heritage technologies“: reuse of proven technologies

Space program A

Space program B

Technology

Technology

Technology

Different outcomes:

MAVEN • •

Within schedule and budget Mission sucess

Mars Observer • •

Schedule and budget overruns Mission failure

 Program success and failure attributed to the use of „heritage technologies“ 2


Heritage Technology State of Knowledge / State of the Art What is heritage technology? No clear definition (Larson & Wertz, 1999) Perceived benefits:

Challenges:

• •

•

• •

Development cost Development schedule (Fallon, 1997), (NASA, 2013), (Coonce, et al., 2009) Programmatic risks (NASA, 2013) Quality and reliability (Kapurch, 2010)

•

Heritage technology issues: ~65% of large NASA programs (GAO, 2010) Issues from literature & interviews:

Profound design modifications Insufficient verification, validation, and testing Component obsolescence Organizational capability obsolescence

State of heritage technology assessment: • Industrial practice: ad-hoc, flown before? • Literature survey: No dedicated methodology

Technology 3


Heritage Technology State of Knowledge / State of the Art What is heritage technology? No clear definition (Larson & Wertz, 1999) technologies Objective 1: Provide a definition and conceptual framework of heritage Perceived benefits:

Challenges:

• •

•

• •

Development cost Development schedule (Fallon, 1997), (NASA, 2013), (Coonce, et al., 2009) Programmatic risks (NASA, 2013) Quality and reliability (Kapurch, 2010)

•

Heritage technology issues: ~65% of large NASA programs (GAO, 2010) Issues from literature & interviews:

Profound design modifications Insufficient verification, validation, and testing Component obsolescence Organizational capability obsolescence


Technology 4



Challenges:

• Heritage technology issues: ~65% of large NASA Development cost and programs (GAO, 2010) schedule (Fallon, 1997), • Issues from literature & interviews: (NASA, 2013), (Coonce, et al., 2009) • Reduction of programmatic Profound design modifications Objective 2: Provide risks (NASA, 2013) empirical evidence for heritage technology benefits Insufficient verification, validation, and • Higher quality and reliability (Kapurch, 2010) testing •

Component obsolescence Organizational capability obsolescence


Technology 5



Challenges:

• Heritage technology issues: ~65% of large NASA Development cost and programs (GAO, 2010) schedule (Fallon, 1997), • Issues from literature & interviews: (NASA, 2013), (Coonce, et al., 2009) • Reduction of programmatic Profound design modifications Objective 2: Provide risks (NASA, 2013) empirical evidence for heritage technology benefits Insufficient verification, validation, and • Higher quality and reliability (Kapurch, 2010) testing •

Component obsolescence Organizational capability obsolescence

State of heritage technology assessment: Objective 3: practice: Enable heritage technology at early stage of development • Industrial ad-hoc, flown before?assessment Technology • Literature survey: No dedicated methodology 6


Main Objectives of PhD Thesis

Objective 1

Provide a definition and conceptual framework of heritage technologies

Objective 2

Provide empirical evidence for heritage technology benefits

Objective 3

Enable the assessment of heritage technologies at an early stage of development with respect to: • A new set of requirements, constraints, and environments • Modifications • Development, manufacturing, and operations capabilities 7


Heritage Technology Definition & Conceptual Framework Heritage technology

has Successful history of verification, validation, testing, and operation

inherits consists of Technological capabilities

Technology

consists of

consists of Design

System

Space Launch System (SLS)

Definition: Heritage technology A heritage technology is a technology that has inherited a successful verification, validation, testing, and operation history, technological capabilities, its design, and optionally artifacts based on the design. 8


Heritage Technology Definition & Conceptual Framework Heritage technology

has Successful history of verification, validation, testing, and operation


inherits consists of Technological capabilities

Technology

consists of

consists of Design

System

How much is inherited to a new system / context?

Definition: Heritage technology A heritage technology is a technology that has inherited a successful verification, validation, testing, and operation history, technological capabilities, its design, and optionally artifacts based on the design. 9



Objective 1


Objective 2


Objective 3



Empirical Evidence of Heritage Technology Benefits: Statistical Research Approach Hypotheses: Programmatic variables Development cost Development duration Cost overrun Schedule overrun

Heritage technology

Sample: Data from 54 space programs: spacecraft and rocket launchers Statistical approach: multiple regression y

𝑦 = 𝛽0 + 𝛽1 𝑥1 + ⋯ + 𝛽𝑘 𝑥𝑘 , 𝑖 = 1, … , 𝑛 Dependent variable (cost, duration, overruns)

Coefficients

Independent variables (heritage variables + control variables) x 11


Statistical Research Approach: Variables

12


Approach and Key Results of the Statistical Analysis Confirmed hypotheses, the effect (decrease/increase), and effect size (how much) Specific Development development cost duration (n=50) (n=48)

Development Development cost overrun duration (n=34) overrun (n=33)

Heritage metric

(design heritage + organizational capability)

Heritage technology variables have the largest effect on development cost and duration of all variables In general, heritage technologies seem to be correctly assessed for space programs, if there is really no relationship with overruns 13



Objective 1


Objective 2


Objective 3



Overview of Heritage Assessment Methodology

Inputs System-level requirements, context model Set of technologies Setconsideration of systems / under Set of systems technologies under/ technologies under consideration consideration System-level TRL Design DSMs Capability list

Outputs Heritage assessment methodology

Compliance issues Modifications

Obsolescense issues

Capability issues Quantitative heritage estimates Downselected systems / technologies

15




Qualitative results: potential issues Heritage assessment methodology

Outputs Compliance issues Modifications

Obsolescense issues


16



Inputs System-level requirements, context model

Outputs Heritage assessment methodology

Set of technologies Setconsideration of systems / under Set of systems technologies under/ technologies under consideration consideration System-level TRL

Compliance issues Modifications

Obsolescense issues

Capability issues

Design DSMs

Quantitative heritage estimates

Capability list

Downselected systems / technologies

Quantitative results

17




Outputs

Compliance assessment

Compliance issues

(Modified) technologies

Modifications

Assess heritage aspects

Calculate heritage metric Heritage values

Heritage-based comparison of options

Obsolescense issues


18




Outputs


Compliance issues


Modifications




Obsolescense issues


19

Institute of Astronautics Technical University of Munich Set of systems / technologies under consideration

Compliance Assessment •

Identify potential compliance issues at an early stage of development:

System-level requirements, context model Valuerelated function Supporting systems

Compliance matrix Requirement / constraint

Requirement / constraint type

Transport >100t to Low Earth Orbit

System-level requirements Value-related function

Yes

Yes

Interface requirements Compatibility with ground station Interfaces with supporting systems equipment

Yes

Yes

Launch pad compatibility

Interfaces with supporting systems

Yes

No

infrastructure Interfaces with supporting systems

Yes

Yes

Transportation compatibility

Context systems Environments

Space Launch Saturn V System

NASA human-ratedness standard

Interfaces with contextual systems

Yes

No

LEO space environment

Environmental constraints Constraints from natural environment

Yes

Yes

20




Outputs


Compliance issues


Modifications




Obsolescense issues


21


Heritage Metric

Input

TRL to VVTO mapping

Interstage section

Instrument section

S-IVB stage

J-2 engine 3rd stage

S-II stage

J-2 engine 2nd stage

S-IC stage

F-1 engine

S-IC stage J-2 engine 2nd stage S-II stage J-2 engine 3rd stage S-IVB stage Instrument section

Original system design DSM

F-1 engine S-IC stage J-2 engine 2nd stage S-II stage J-2 engine 3rd stage S-IVB stage Instrument section Interstage section

Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

Interstage section

New system design DSM

Capability values

Output

VVTO history value [0, 1]

System-level TRL value

F-1 engine

Metric calculation

DSM change degree 𝐷 = 1.0 −

𝑤𝑐 𝑓𝑐 + 𝑤𝑟 𝑓𝑟 𝑤𝑐 + 𝑤𝑟

Arithmetic mean 1 𝐶= 𝑛

𝑛

Choquet integral

Normalized heritage metric value [0, 1]

Aggregated capability value [0, 1] Parameter value

𝐶𝑖 𝑖=1

Design heritage value [0, 1]

Calculation step Input / output 22


Heritage Metric

Input

Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

System-level TRL value (4)

F-1 engine S-IC stage J-2 engine 2nd stage S-II stage J-2 engine 3rd stage S-IVB stage Instrument section


Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

Interstage section



Capability values Capability

Value

RS-25

0.5

5 segment SRB

0.7

Modified Shuttle tank

0.7

…

…

TRL to VVTO mapping

Metric calculation VVTO history value (0.02)




𝑛

Output Choquet integral

Normalized heritage metric value (0.4)

Aggregated capability value (0.7) Parameter value

𝐶𝑖 𝑖=1

Design heritage value (0.62)


Calculation step Input / output 23


Heritage Metric: More Examples

24



Launch-ready

25



Obsolescence

CubeSat component case study Obsolescence + staff loss

26



Modified heritage tank

Ariane 5 hydraulic tank case study

New tank

27




Outputs


Compliance issues


Modifications




Obsolescense issues


28


Heritage-Based Comparison of Options: SLS vs. Saturn V Comparison of two system families: Saturn V variants and SLS variants

J-2X

Space Launch System (SLS) variants

F-1b

Resurrected Saturn V with new technologies

Heritage metric value

•

29


SLS Case Study: Maturing Lead System in System Family Matured lead system

 Design commonality and common capabilities lead to an increase in heritage of system family 30


Heritage-Based Trade-Offs: Performance vs. Heritage Sacrifice heritage for gaining performance?

SLS standard config. + ICPS stage after CDR SLS preliminary design stage

31



Objective 1


Objective 2


Objective 3



Key Findings and Contributions

Statistical Analysis

• Confirmed: Decrease in development cost and development duration • Heritage technology variables: Largest effect on development cost and duration of all variables • Not confirmed: Decrease in schedule and cost overrun (Explanation: too small sample size or really no relationship) • Correct assessment / use of heritage technologies in space programs  benefits > risks

• Prototype methodology • Enables to systematically identify potential heritage technology issues Heritage • Enables to quantify heritage Assessment • Heritage-based comparison of technologies: system families, Methodology Pareto frontier • Applied to 3 different case studies: Different heritage technology concerns 33


Future Work and Applications Sample size

Statistical Analysis

• Larger sample for cost and schedule overruns  Obtain statistically significant results • Explore relationship between heritage technology and mission risk: Increase or decrease?

• Extend scope of methodology to other domains: aeronautics, automotive

Heritage Assessment Methodology • Extend to general technology maturity assessment • Direct application within projects: MIT, NASA MSC, Airbus 34


Scientific Output and Funding Sources at LRT My scientific output: • 6 journal papers • 7 peer-reviewed conference papers, • 12 non-peer reviewed conference papers, • 3 journal papers currently in preparation (Acta Astronautica, Systems Engineering, IEEE Engineering Management), • 31 student term papers (DA, MA, BA, SA), Funding Sources (~400k€, leading acquisition efforts): • Joachim-Lorenz-Stiftung • German Academic Exchange Service (DAAD) • Helmholtz-Community (HGF) • Airbus Defence and Space • MT-Aerospace

35



Objective 1


Objective 2


Objective 3



Back-up 37


Heritage Definitions: Literature Survey “Heritage technologies are proven components that are being modified to meet new requirements” (GAO, 2009) “Heritage technology includes hardware or software subsystems or components with previous flight history that are used as part of a new mission system. The heritage of the component includes not only the previous flight history, but the previous function(s) for which the component was used, the environment in which it was used, and physical, thermal, and data interfaces with other elements of the mission system. Heritage also includes the availability of documentation, support equipment, and personnel experienced in its design, implementation, and operational use.” (Barley et al., 2010) ““Heritage” refers to the original manufacturer’s level of quality and reliability that is built into parts and which has been proven by (1) time in service, (2) number of units in service, (3) mean time between failure performance, and (4) number of use cycles. High-heritage products are from the original supplier, who has maintained the great majority of the original service, design, performance, and manufacturing characteristics. Low heritage products are those that (1) were not built by the original manufacturer; (2) do not have a significant history of test and usage; or (3) have had significant aspects of the original service, design, performance, or manufacturing characteristics altered. An important factor in assessing the heritage of a COTS product is to ensure that the use / application of the product is relevant to the application for which it is now intended. A product that has high heritage in a ground-based application could have a low heritage when placed in a space environment.” (Kapurch, 2010, p.76) 38


Heritage Definitions in the Literature: Shortcomings Missing aspect Technology definition Instance – design distinction Unit of heritage technology Organizational capabilities as part of a heritage technology Evolution of heritage Different types of heritage Heritage technology context

Addressed in this thesis by Technology conceptual model (Section 1.3) Heritage system and proven system design Can be applied to different levels in system hierarchy

Organizational capabilities are considered part of a heritage technology Organizational capabilities Focus on capabilities, verification, validation, testing, and operations (VVTO), and design Context model

39


Key Definitions of Thesis Heritage Heritage refers to something transmitted by or acquired from a predecessor that is considered increasing its quality. Proven A system is called “proven”, if it has a successful history of verification, validation, testing, and operational history. This definition of potentiality is similar to the notion of “iterated ability” from (Vetter, 2015, p.81). Obsolescence The status given to a technology that is no longer available from its original manufacturer. Uninvention The status given to a technology that is in general no longer available.

40


Capability-Related Definitions Capability The attribute of an actor which can perform an action. Specific capability A specific ability is an ability that is bound to specific external circumstances to exist. General capability A general ability is an ability that does not depend on external circumstances to exist. Potentiality “Potentiality” is the power to perform an action in the future given the required ability has been acquired by an actor. Technological capability A technological capability is a capability that is related to the lifecycle phases of a technology.

41


Adapt Methodology to Other Domains / Other Use Cases VVTO history assessment •

TRL – VVTO mapping function  needs to be adapted to new domain

Capability assessment •

Data about resources (personnel etc.) not available  perform best and worst case

analysis Design heritage assessment •

Design heritage: Use DSMs or adapted design heritage categories, e.g. automotive platform as an equivalent of a spacecraft bus

Choquet integral •

Weightings between VVTO history, design heritage, and capability may differ

42


Technology Framework

43


Heritage Metric

Survey: Completeness of metric (n=5)

Basis for function from expert survey (n=13) Input

TRL to VVTO mapping

Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

S-IC stage J-2 engine 2nd stage S-II stage J-2 engine 3rd stage S-IVB stage Instrument section



Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

Interstage section


Capability values

Metric calculation

Output

VVTO history value [0, 1]

System-level TRL value

F-1 engine

Weightings based on expert survey (n=8)




Design heritage value [0, 1]

Choquet integral

Normalized heritage metric value [0, 1]

Aggregated capability value [0, 1]

𝑛

Parameter value 𝐶𝑖

Calculation step

𝑖=1

Adaption of literature (Smaling & de Weck, 2007; Suh et al., 2010)

Adaption of literature (Birkler et al., 1993; Bilbro, 2008)

Input / output 44


Two Cases of Heritage Technology Assessment Heritage system Component 1

System consisting of heritage components Component 2

Component3

Component 1

Component 2

Component3

Nomenclature Changed relationship Inherited relationship Component

Inherited relationship

Component

Changed component

45


Heritage Metric Results: Sample Technologies

46


Heritage Metric Values

Full heritage technology MAVEN (before launch) Mars Observer (before launch) Hydraulic tank (modified heritage technology) CubeSat component (obsolescence) SLS ICPS New hydraulic tank CubeSat component (obsolescence & staff loss)

1 0,85 0,85

1 0,56 0,3

1 1 1

0,1

0,5

0,9

0,3 0,02 0,02

0,2 0,62 0

1 0,7 0,8

0,3

0,2

0

47


Determination of Choquet Integral Weightings Process: 1. Expert evaluation on alternatives with different heritage values 2. Estimation of overall heritage value of alternatives 3. Use of (heuristic) least squares algorithm + linear programming and tweaking for finding

appropriate weightings  do results make sense?

48


Assessment of Different Value Functions Weighted Sum 𝑣 𝑥1 , 𝑥2 , … , 𝑥𝑛 𝑛

=

𝜆𝑖 𝑥𝑖 𝑖=1

Multiplicative 𝑣 𝑥1 , 𝑥2 , … , 𝑥𝑛 𝑛

𝑥𝑖 𝛽𝑖

= 𝑖=1

Choquet Integral 𝐶𝜇 𝑥 𝑛

≔ 𝑖=1

(𝑥𝜏 𝑖 − 𝑥𝜏(𝑖−1) ) 𝜇( 𝜏 𝑖 , … , 𝜏 𝑛 )

If one element is 0, whole metric is not 0 Dependencies between sets of criteria

49


Sample Spacecraft (Interplanetary)

Spacecraft (LEO to GEO)

Launcher

N=22 Viking Orbiter Viking Lander Phoenix Mars Pathfinder Mars Climate Orbiter

N=15 TDRS K LDCM MMS NPP SMAP

N=17 L3S Europa Ariane 1 Ariane 2 Ariane 3 Ariane 4

Mars Global Surveyor

GLAST

Ariane 5

Mars Observer MAVEN Juno MSL Cassini Dawn Kepler MER Voyager Pioneer 11 Magellan NEAR LRO GRAIL LADEE Messenger

Glory GPM WISE OCO-1 OCO-2 HETE 1 HETE 2 QuickBird 1 QuickBird 2

Titan I Titan II Titan III Titan IV Falcon 1 Falcon 9 Saturn I Saturn IB Saturn V Space Shuttle

50


Variable scales: Heritage Design Scale I

Value 1 0.75

0.5

0.25 0

Conditions Component design is used off-the-shelf without modifications or minor modifications. Component design is used with modifications or its subcomponent designs are proven component designs but haven’t been used as a system before. Hence, its architecture is new. Implicitly, the architecture of the system (the way the components interact) has less weight than the components themselves. The component design is partly based on new sub-component designs and has been newly developed. Alternatively, the component design has been subject to major modifications. A majority of the component designs is newly developed and have not been operated in this architecture before. Newly developed component designs, where most of its subcomponent designs are newly developed. “Newly developed” means that a component instance has not yet been operated in its intended environment before.

51


Variable scales: Heritage Design Scale II

Category Conditions 0 new 1 Heritage parts / equipment: The space system consists of heritage parts and equipment, e.g. RTGs, antenna. However, the subsystems are newly composed, i.e. no heritage subsystems are used. 2 Heritage subsystems: One or more subsystem / stages of the space system are existing or an existing subsystem / stage has been modified 3 Heritage bus: The spacecraft bus is already existing but the payload can be newly developed. For launchers, all stages need to be based on existing designs. Modifications to the bus / stages are allowed. 4 Carbon copy: The spacecraft is identical to a predecessor, except small modifications. The bus and the payload are identical or almost identical. “Carbon copies” are usually related with spacecraft and not with launchers as launchers are usually produced series.

52


Variable scales: Technological Capability – Organizational Capability

Category Conditions 0 First time the organization (prime contractor) has developed a space system from a class (launcher, lander, orbiter, rover) for a class of mission (destination: LEO, GEO, Lunar, Mars, outer solar system). Furthermore, if more than 20 years have passed since the last time a space system from the class has been developed by the organization, then the new system is considered a “first”. It is expected that after 20 years, most of the personnel with experience is no longer available. 1 At least one previously developed space system from a class as defined under previous bullet point.

53


Variable scales: Technological Capability – Team Similarity

Category 0

1

Conditions The spacecraft / launcher is developed without the involvement of the team that has worked on predecessors. In case data is not available, it is assumed that teams have changed if the prime contractor or prime investigator has changed. For example, a spacecraft is developed by NASA Ames instead of JPL. It is assumed that teams change when the prime changes. This is a simplification, as the subcontractors may still remain the same. The prime contractor / prime investigator remains the same and there is data available that confirms that the teams have essentially stayed the same.

54


Variable scales: Technological Capability – Program Manager Experience

Category 0 1 2

Conditions No previous experience with program management of interplanetary missions. Experience with one mission and has served in the role of program manager during development before for an interplanetary mission. More than one interplanetary mission managed in the role of program manager during development.

55


Key Results of the Statistical Analysis Confirmed hypotheses, the effect (decrease/increase), and effect size (how much) Specific cost

development

(H3) Design heritage

Development duration

Development cost overrun

Development duration overrun

(H4)

(H1)

(H2)

Decrease*

Decrease*

(maximum savings

(maximum savings

-75% to -85%) Decrease*

-29 to -46 months) Decrease*

(maximum savings

(maximum savings

-526 k$/kg to -1325 k$/kg)

-24 to -36 months)

Team similarity

-

Program manager experience

-

(maximum value) Organizational capability (with experience)

-

-

-

-

-

-

-

-

-

-

*statistically significant at the 0.05 level

 H3 and H4 confirmed but H1, H2 not confirmed (too small sample size?) 56


Specific development cost [k$/kg]

Heritage vs. Specific Development Cost Regression

57


Heritage vs. Specific Development Duration Regression

58


Heritage vs. Relative Cost Overrun Regression

59


Heritage vs. Relative Schedule Overrun Regression

60


Log(Specific development cost) [k$/kg]

Design Heritage – Specific Development Cost

Design heritage 61


Development duration [months]

Design Heritage – Development Duration

Design heritage

62


Relative development cost overrun

Design Heritage – Cost Overrun

Design heritage

63


Relative development schedule overrun

Design Heritage – Schedule Overrun

Design heritage

64


Specific development cost [k$/kg]

Technological Capability: „Firsts“ – Specific Development Cost

Existing capability

Organizational „First“ 65


Development duration [months]

Technological Capability: „Firsts“ – Development Duration

Existing capability



Relative development cost overrun

Technological Capability: „Firsts“ – Cost Overrun

Existing capability



Relative development schedule overrun

Technological Capability: „Firsts“ – Schedule Overrun

Existing capability



Estimation Relationships for Heritage Technologies 𝐻𝑑𝑒𝑠𝑖𝑔𝑛 ∈ [0,1] Combined heritage technology metric:

𝐻 = 0.5 ∗ 𝐻𝑑𝑒𝑠𝑖𝑔𝑛 + 0.5 ∗ 𝐻𝑐𝑎𝑝𝑎𝑏𝑖𝑙𝑖𝑡𝑦

𝐻𝑐𝑎𝑝𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ∈ {0,1}

Relative specific development cost reduction depending on heritage technology degree:

𝑐𝑑𝑒𝑣 = 1 − 0.869 ∙ 𝐻

𝐻 ∈ [0,1]

87% specific development duration cost reduction @H=1 Relative development duration reduction depending on heritage technology degree:

𝐻 ∈ [0,1]  51% development duration reduction @H=1

𝜏𝑑𝑒𝑣 = 1 − 0.507 ∙ 𝐻

69


Heritage Assessment Methodology

Thesis defense Andreas Hein

70


In- Outputs of Methodology Steps Process step Define assessment objectives and depth Define technologies / systems under consideration Compliance assessment

Inputs Initial objectives for heritage technology assessment

Outputs Prioritized objectives and required level of detail for decision-making Prioritized objectives and required level of detail for Set of existing or planned decision-making technologies and systems for further assessment  Set of existing or planned technologies and For each technology / system: systems for further assessment compliance, non-compliance, or  Set of system-level requirements and constraints further information required

VVTO assessment



Heritage-based comparison

 Capability metric value  Design heritage metric value Values for other system / technology evaluation criteria, Set of Pareto-optimal e.g. development cost, utility, etc. technologies

VVTO data for compliant technologies / systems

VVTO classification and extent for technologies / systems to the level of detail required Capability assessment Capability-related data Capability levels for technologies / systems to the level of detail required Design heritage  System architecture of original system, system Impact of modifications on heritage: assessment architecture(s) of modified systems components and architecture,  List of technological capabilities for original capabilities system  modified systems Calculate heritage metric For each system / technology: Heritage metric value for each system / technology  VVTO metric value

systems 71 /


TRL-VVTO Mapping

Mean values from expert survey (NASA engineers): n=10 for specific technology (dotted line); n=13 for generic technology (straight line) 8/15/2017

𝐿 Data fitting with 𝑉 𝑥 = 1 + ex p( − 𝑐(𝑥 − 𝑥0 )) logistic function:


Logistic function parameter L c 𝑥0

Parameter value 0.95 1.5 6.5 72


Survey Results TRL-VVTO Mapping Comparison of Medians for Mission time and purpose given vs. no further information provided (Rösner, 2014, p.135)

73


Design Heritage Assessment: Principle

Change of heritage system (left) and system composed of heritage components (right)

8/15/2017


74


Interstage section

Instrument section

S-IVB stage


S-II stage


S-IC stage

F-1 engine

Design Heritage Assessment: Input DSM – Saturn V

F-1 engine S-IC stage J-2 engine 2nd stage S-II stage J-2 engine 3rd stage S-IVB stage Instrument section Physical connection Mass flow

Interstage section

Energy flow Information flow

8/15/2017


75


Interstage section

Avionics section

DCSS

RL-10BL

5-segment Shuttle booster

Modified Shuttle tank

RS-25

Design Heritage Assessment: Input DSM – SLS

RS-25 Modified Shuttle tank 5-segment Shuttle booster RL-10BL DCSS Avionics section Physical connection Mass flow

Interstage section

Energy flow Information flow

8/15/2017


76


Design Heritage Assessment: Input DSMs The graph-edit similarity metric of two systems 𝐺𝑠𝑦𝑠_0 and 𝐺𝑠𝑦𝑠_1 is defined as

𝑆𝑔𝑒𝑑 𝐺𝑠𝑦𝑠_0 , 𝐺𝑠𝑦𝑠_1 = 1.0 −

𝑓𝑐 =


𝑚 𝑖=1 𝐷𝑐𝑖 𝑆𝑐𝑖

|𝑁0 | + |𝑁1 |

𝐷𝑐 and 𝐷𝑟 are component and relationship change degrees, where

𝑓𝑟 = w are weightings with

𝑛 𝑖=1 𝐷𝑟𝑖 𝑆𝑟𝑖

|𝐸0 | + |𝐸1 |

𝐷𝑐 ∈ [0,1]

𝐷𝑟 ∈ [0,1]

0≤𝑤≤1 8/15/2017


𝑤𝑐 + 𝑤𝑟 = 1 77


Design Heritage Categories to Design Heritage Metric Mapping Design heritage categories mapped to fine-grained design heritage metric Category

0

Description spacecraft

Corresponding Mean value fine-grained metric range min / max 0 – 0.2 0.1730

Standard deviation

New 0.1490 development 1 Heritage components Heritage 0.05 – 0.85 0.5619 0.2447 but no heritage components subsystem but no heritage stage 2 Heritage subsystems One heritage 0.25 – 0.9 0.58 0.1719 stage or modified heritage stages 3 Heritage bus Heritage 0.2 – 0.98 0.6131 0.1841 stages 4 Carbon copy Carbon copy 0.85 – 1 0.9476 0.0908 For a quick design heritage assessment, simply take the mean value that corresponds to the category! 8/15/2017

New development

for Description for launcher


78


Design Heritage Categories to Design Heritage Metric Mapping

79


Choquet Integral Aggregation function for heritage metric (VVTO history, design heritage, capability): 1

𝐶𝜇 (𝑥) = 𝑣1 𝑥1 + 𝑣2 𝑥2 + 𝑣3 𝑥3 − 2 (𝐼12 |𝑥1 − 𝑥2 | + 𝐼13 |𝑥1 − 𝑥3 | + 𝐼23 |𝑥2 − 𝑥3 |) 1

= 0.35𝑥1 + 0.25𝑥2 + 0.4𝑥3 − 2 (0.1|𝑥1 − 𝑥2 | + 0.2|𝑥1 − 𝑥3 | + 0.2|𝑥2 − 𝑥3 |)

𝐼𝑖𝑗 ≔ 𝐾⊆𝑁{𝑖,𝑗 }

𝑣𝑖

Importance index

𝐼𝑖𝑗

Interaction index

𝜇𝑖

Weighting for variable i

𝑛 − 𝑘 − 2 ! 𝑘! [𝜇 𝐾 ∪ 𝑖, 𝑗 − 𝜇 𝐾 ∪ 𝑖 − 𝜇 𝐾 ∪ 𝑗 𝑛−1 ! 𝑛 − 𝑘 − 2 ! 𝑘! 𝑣𝑖 ≔ (𝜇 𝐾 ∪ 𝑖 − 𝜇 𝐾 𝑛!

+𝜇 𝐾

𝐾⊆𝑁\𝑖

8/15/2017


80


Choquet Integral: Visualization

Capability

Heritage Metric Value

VVTO

Design Heritage 81


Choquet Integral: Visualization

Capability

Heritage Metric Value: 0.5

VVTO Design Heritage 82


Choquet Integral: Sensitivity Analysis • • • •

Difference in one TRL level 5, 6, 7, 8: < 11% 20% design heritage error: < 7% 20% capability error: < 13% 40% capability error: < 19%

Errors in three metrics (TRL, design heritage, capability)  < 31% error in heritage metric

83