Bayesian Multi-Source Modeling with Legacy Data

AIAA SciTech Forum 8–12 January 2018, Kissimmee, Florida 2018 AIAA Non-Deterministic Approaches Conference

10.2514/6.2018-1663

Bayesian Multi-Source Modeling with Legacy Data Sayan Ghosh∗, Isaac Asher†, Jesper Kristensen∗, You Ling †, Kevin Ryan ∗, and Liping Wang‡ GE Global Research Center, 1 Research Circle, Niskayuna, New York. 12309, USA.

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Abstract As the complexity of engineering systems increases, the cost of computer simulations, experimentation and physical tests for design and development also gets expensive. Due to the significant resources required by these physical and computational experiments on a new design, only sparse data are generally available and are often not sufficient enough to build accurate meta-models. A common approach to reduce the cost and the time cycle of a design process is to use meta-modeling techniques to build simpler and faster mathematical models to carry out the decision-making process for a new design. If datasets are available from legacy systems which belong to a similar family as the new design, one can leverage that information and knowledge to improve the accuracy of models built for the new system. In this work, a Bayesian multi-source modeling technique has been developed which combines models built on data from legacy systems with sparse data for a new design to improve the predictive capability of meta-model for the new design.

I.

Introduction

Engineering systems are becoming more and more complex as requirements (performance, environmental, cost, etc.) are becoming more stringent. This, has a direct impact on the time and cost to design new systems. A typical industrial approach to manage the time and cost, while achieving the performance goal is to develop a derivative design rather than a completely new and unconventional system. The benefit of designing a derivative design is that the engineers can leverage their knowledge and experience of the existing infrastructure. However, even a new derivative design comes with new technology, parameter settings, and operational conditions, which have not been tested in the legacy systems. Therefore, new experiments are required to understand and design the new derivative design. Due to the complexity and high experimental cost of the system, sparse data are generally available for the new designwhich alone are not sufficient to build an accurate model. To overcome this challenge, the goal of the current work is to use fewer data from the new design and leverage the data available from legacy systems to have a better predictive capability and better inform the design and decision-making process. One of the common approaches for handling multiple sources of data is known as multi-fidelity modeling. Many of the existing methods for generating multi-fidelity surrogate models are based on the idea that the high-fidelity experiments can be approximated as a tuned or corrected functions of low-fidelity models (e.g., [1–10]). This approach was generalized by Toropov [11] with three types of tuning: linear and multiplicative functions, correction factors, and the use of low-fidelity model inputs as tuning parameters. For the first two types, the analyst must specify a functional relationship between the low- and high-fidelity functions. The third type requires that the low-fidelity code(s) possess inputs that can serve as tuning parameters. Kriging and Gaussian process regression have also been proposed for the multi-fidelity analysis problem (e.g., [2,10]). These methods use an auto-regressive model to correct low-fidelity predictions, and the response covariance matrix is modified to account for the multiple data sources. In a typical multi-fidelity modeling scenario, it is assumed that sources of data are available from different fidelities. However, data sources coming from ∗ Research

Engineer, Mechanical Systems, GE Global Research, Niskayuna, New York Engineer, Mechanical Systems, GE Global Research, Niskayuna, New York ‡ Technical Operations Leader, Mechanical Systems, GE Global Research, Niskayuna, New York † Lead

1 of 12 American Institute of Aeronautics and Astronautics Copyright © 2018 by General Electric Co. . Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

different legacy systems may have the same fidelity. Additionally, most of the typical multi-fidelity methods become computationally expensive when more using two or more sources. An alternate approach to multi-fidelity modeling is to use ensemble learning [12]. In ensemble learning, two or more related but different analytical models are combined together to improve the accuracy of predictive analytics. One of the commonly used ensemble learning approaches is Bayesian Model Averaging (BMA)[13]. One of the main advantages of BMA is that it accounts for model uncertainty which may arise due to the modeling method as well sparsity of the data used. However, the main issue of BMA is that all the models are built using same data set. That means, for the current problem addressed in this paper, all the legacy data have to be combined together to form a single database. Different models would need to be built using the single database, which is then combined to form an ensemble model. Marginalizing all the legacy data together can lead to few issues. For example, if discrepancies exist in the data between each legacy system, large data noises will be introduced during the modeling process. Also, models build on each legacy system data can have more information than the models build on combined dataset. In this paper, A Bayesian multi-source modeling approach is developed for legacy data that addresses the aforementioned challenges. The approach is similar to Bayesian Model Averaging where different models are built and combined together to build a better predictive model. However, unlike BMA, the models are built for each legacy data set separately. Then, model validity is evaluated for each model as a function of input space, which is then used to combine the model. The details of the methodology are discussed in Section II. The method is demonstrated on three different numerical experiments in Section III. Finally, the conclusion is given in Section IV.

II.

Methodology

The goal is to build a model for a new design with sparse data, which alone is not sufficient to build an accurate model. However, there is a relatively large number of data available from legacy designs which are not exactly same as the new design, but they belong to a similar family. It is assumed that the performance of the new design will not be exactly same as any individual legacy design, but there will be some similarity in the performance trends. Let’s say n∗ number of data are available for the new design with y∗ = [y1∗ , . . . , yn∗ ∗ ]T as outputs and ∗ x = [x∗1 , . . . , x∗n∗ ]T as inputs. Consider that legacy data are available from p number of legacy systems from a similar family. The input for k th data source is given by xk = [x1k , . . . , xnk k ]T , where nk is the number of data points available in the k th data source. The corresponding output for each data point is given as yk = [y1k , . . . , ynk k ]T . It is assumed that the input variables (design, operational, etc.) and the output variables (performance, cost, etc.) are the same for the new system and all the legacy systems. However it is not required that a subset of xk be collocated with x∗ , i.e. it is not required to have x∗j = xjk for any j and k. Let’s define Mk = ηˆklegacy (x) as a meta-model built on the k th legacy dataset. In a typical Bayesian Model Averaging approach, a model for the new data source is given as fˆ∗ (x) =

p X

wk ηˆklegacy (x)

(1)

k=1

where wi is the model validity associated with each legacy data model and can be estimated by various techniques [14]. However there are two main issues in directly using this approach: • There can be large discrepancies between one or more of the legacy systems and the new system, while having a similar trend. This can lead to an inappropriate allocation of model validity. For example, in Figure 1a legacy-1 has a similar trend to the new system, however due to a large discrepancy it’s validity can be low. • The model validity may also vary in input space. For example, in Figure 1b, the validity of legacy-1 is higher for higher values of x while the validity of legacy 2 is higher for lower values of x. To overcome these challenges, the discrepancy of the legacy data is also included in the legacy model as Mk = ηˆklegacy (x) + δˆklegacy (x). Also, it is assumed that the model validity may vary in the design space. The

2 of 12 American Institute of Aeronautics and Astronautics

20

25

15

20 15

10

10 5 5 0

0

-5 -10 0

-5 0.2

0.4

0.6

0.8

-10 0

1

(a) Lower validity to a better model due to high discrepancy

0.2

0.4

0.6

0.8

1

(b) Variation of validity with input space

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Figure 1: Issues with traditional Bayesian model averaging applied to legacy data

new formulation used in the current work is given as fˆ∗ (x) =

p X

  w ˆk (x) ηˆklegacy (x) + δˆklegacy (x)

(2)

k=1

where δˆklegacy is the discrepancy model of the k th legacy model withPdata from new design, and w ˆk is the p model validity which is a function of the input variables such that k=1 w ˆk (x) for any value of x. In the current work, both ηˆklegacy and δˆklegacy are built using a Bayesian Hybrid Modeling (BHM) approach [15]. The model validity w ˆk is estimated by calculating the legacy model likelihood and predictive uncertainty. II.A.

Bayesian Hybrid Modeling

To model the legacy data and the discrepancy, a Bayesian framework proposed by Kennedy and O’Hangan [15] is used in this work and is referred to as a Bayesian Hybrid Model (BHM). In the original work, BHM is used to combine test/experimental data and simulation data to perform a calibration of a computer model and to quantify the discrepancy of a simulation model. Let y(x) represent the outputs of the experimental process and η(x, θ) represent the outputs of computer simulation where x are the input variables and θ are unobservable model parameters or calibration parameters of the simulation model. According Kennedy and O’Hangan’s framework, the experimental observation of outputs y is expressed as: y(xi ) = η(xi , θ∗ ) + δ(xi ) + (xi ),

for i = 1, . . . , n

(3)

where n is the number of experimental observations, θ∗ are the true values of the calibration parameters, δ is the discrepancy between the calibrated simulator η and the experimental observation, and  are wellcharacterized observation errors. In the current work, the models of the legacy data are built using the same Bayesian philosophy: y ∗ (xi ) = ηklegacy (xi ) + δklegacy (xi ) + k (xi ),

for i = 1, . . . , n∗

(4)

where y ∗ is the output of new data source, ηklegacy (x) is the output of the k th legacy system, δk (x) is the discrepancy between the k th legacy system and the new system, and k are well-characterized observation errors. It should be noted that the calibration parameters are not used in Equation 4 because the legacy data may not contain these parameters. However, if calibration parameters θk are available, it can be included in Equation 4. The output of a legacy system η(x) and it’s discrepancy δ(x) are modeled as Gaussian Processes (GP) as described by Kennedy and O’Hagan [15] and Higdon et al [16]. The model for a given legacy system η(x) is approximated as GP model with a zero mean and covariance matrix given by,

3 of 12 American Institute of Aeronautics and Astronautics


1 1 exp βη |Xi − Xj |2 + I , for i, j = 1, . . . nk ληz ληs

Σηij =

(5)

where X is the vector of design variables, the parameters ληz and ληs characterize the marginal data variance captured by the model and by the residuals, respectively, β characterizes the strength of dependence of the outputs on the design variables. The outputs of new system y(x) is modeled as a GP model: Σyij =


1 1 exp βy |Xi − Xj |2 + I , for i, j = 1, . . . n∗ λyz λys

(6)

The cross covariance matrix to represent correlation between the legacy system outputs and the new system observations is given as Σηy ij =


1 1 exp βy |Xi − Xj |2 + I , for i = 1, . . . n∗ , j = 1, . . . , nk λyz λys

(7)

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

The discrepancy parameter δ is modeled as Σδij =


1 exp βδ |Xi − Xj |2 , for i, j = 1, . . . n∗ λδz

The likelihood of combined data z = (y T , η T ) is then given as   1 T −1 1 exp − D Σ D L(D|ληz , βη , ληs , λδz , βδ ) = 2 |Σ|1/2   ! Σδ 0 0 Σ 0   new where D = (y T , η T , δ T ) and Σ =  0 Σy Σyη  + 0 Σlegacy 0 Σηy Ση The posterior distribution of all the hyper-parameters is given by π(ληz , βη , ληs , λδz , βδ |D) = L(D|ληz , βη , ληs , λδz , βδ )π(ληz )π(βη )π(ληs )π(λδz )π(βδ )

(8)

(9)

(10)

where π(.) on the right-hand side of the equation is the prior distribution of the parameters. The target posterior distribution is evaluated using a Markov Chain Monte Carlo (MCMC) [17] approach. For more details of BHM modeling and its enhancements please refer to [18, 19]. II.B.

Model Validity

Once the ηˆklegacy and δˆklegacy models are built for each legacy dataset, the next step is to estimate the model validity. In this work two metrics are used to estimate model validity. The first one is a likelihood based model validity, while the second one is an uncertainty based model validity. In the likelihood-based model validity, the validity of a legacy model at an available input setting x∗ of the new design is proportional to the probability of the legacy model to predict the output or performance of the new design, as shown in Figure 2. For example, the likelihood of the k th legacy model, Mk at a location x∗ , for which output of new design is known to be y ∗ is given as   1 (y ∗ − µk (x∗ )2 ) likelihood ∗ ∗ wk (x ) ∝ P (y |Mk ) = p exp − (11) 2σk (x∗ )2 2πσk (x∗ )2 where µk (x∗ ) and σk (x∗ ) are the predictive mean and standard deviation of the k th legacy model Mk = ηˆklegacy (x) + δˆklegacy (x) at design specified by x∗ . In the uncertainty-based model validity, the model validity is inversely proportional to the predictive uncertainty of legacy only model ηˆklegacy (x). In other words, the validity of a legacy model Mk reduces as it goes farther from the available legacy data which is indicated by the predictive standard deviation σηlegacy (x) k as shown in Figure 3 and given as wkuncertainty (x) ∝

1 σηlegacy (x) k

4 of 12 American Institute of Aeronautics and Astronautics

(12)

𝑦 𝑤2𝑙𝑖𝑘 (𝑥 ∗) ∝

1 2𝜋𝜎2 𝑥 ∗

2

𝑒

𝑦∗−𝜇2 𝑥 ∗ 2𝜎2 𝑥 ∗ 2

2

𝑀2 (𝑥)

𝑦∗

𝑀1 (𝑥) Data for new design

𝑥

𝑥∗ 𝑤1𝑙𝑖𝑘 (𝑥

∗) ∝

1

2𝜋𝜎1 𝑥 ∗

2

𝑒

𝑦∗−𝜇1 𝑥 ∗ 2𝜎1 𝑥 ∗ 2

2

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Figure 2: Likelihood based model validity 𝜂Ƹ 𝑘 𝑙𝑒𝑔𝑎𝑐𝑦 (𝑥)

𝑦

Predictive uncertainty 𝑙𝑒𝑔𝑎𝑐𝑦 𝜎𝑘 (𝑥)

Legacy data

𝑤𝑘𝑢𝑛𝑐𝑒𝑟𝑡. 𝑥 ∝

1 𝑙𝑒𝑔𝑎𝑐𝑦 𝜎𝜂𝑘 (𝑥)

𝑥 Figure 3: Uncertainty based model validity

The overall model validity of the k th legacy model at a given design point x is then given as wk (x) = wklikelihood (x)wkuncertainty (x) = κ(x) where κ(x) is proportionality factors given as κ(x) = 1/

P (y new |Mk ) σηlegacy (x) k

(13)

P(y new |Mk ) k=1 σηlegacy (x) . k

Pp

(x) is the byproduct of each legacy BHM modeling and can be estimated It must be noted that σηlegacy k directly using the legacy Gaussian Process model ηˆklegacy during the prediction. However, the model likelihood P (y new |Mk ) is only known at design points x∗ where data for new designs are available and need to be interpolated or extrapolate for other design points. In the current work, a Gaussian Process model of P (y|Mk ) is built for each legacy model to estimate a likelihood-based model validity at any location in the design space.

III.

Results

In this section, three different examples are carried out to demonstrate the benefit of Bayesian multisource modeling. The first test case is a mathematical problem, the second test case is an analytical problem for borehole design and the third is an experimental problem to estimate the crack propagation rate of a new aluminum alloy.

5 of 12 American Institute of Aeronautics and Astronautics

III.A.

Exponential Function

The first test case is a two dimensional exponential function reported by Currin et al. [20] which is treated as a true function and is given by Equation 14. The design space of the input variables is defined by xi ∈ [0, 1], for all i = 1, 2.    2300x31 + 1900x22 + 2092x1 + 60 1 (14) fexp (x) = 1 − exp − 2x2 100x31 + 500x21 + 4x1 + 20

1 [fexp (x1 + 0.05, x2 + 0.05) + fexp (x1 + 0.05, max(0, x2 − 0.05))] 4 1 + [fexp (x1 − 0.05, x2 + 0.05) + fexp (x1 − 0.05, max(0, x2 − 0.05))] 4

fL1 (x) =

fL2 (x) = 16.55 − 10.15x1 − 13.24x2 + 2.6x1 x2 + 4.56x21 + 3.77x22

(15)

(16) New Legacy-1 Legacy-2

1 0.9 0.8

20

True Legacy-1 Legacy-2

0.7 0.6

10 x2

y

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Two legacy data sources are used to demonstrate the benefit of multi-source modeling in this example. They are generated using two different lower fidelity functions, and are labeled as legacy-1 and legacy-2. The first legacy source is a lower fidelity model used by Xiong et al. [21] and is given by Equation 15. The second legacy data source is built from a low fidelity function generated using a quadratic equation given by Equation 16. The behavior of the true and the legacy functions are shown in Figure 4a. The legacy-1 function is very similar to the true function in entire design space. The legacy-2 function is only accurate for higher values of x1 and does not capture the non-linear behavior for lower values of x1 .

0.5 0.4

0 1

0.3 0.2 0.1

0.5

0

x2

0

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

x1

x1

(a) Exponential function and legacy functions

(b) Distribution of training data

Figure 4: True and legacy functions and data distribution for exponential function To train the BHM and multi-source models, 10 randomly selected new designs are generated using the true function within the original design space. 100 additional are generated using the true function and are used for validation. For the legacy-1 data source, 50 designs are generated using Equation 15 within the design space of x1 ∈ [0, 0.5] and x2 ∈ [0, 1], while for the legacy-2 data source, 50 designs are generated using Equation 16 within the design space of x1 ∈ [0.5, 1] and x2 ∈ [0, 1]. The selected data points in input space are shown in Figure 4b. In the first step of multi-source modeling, BHM models of individual legacy source and discrepancy are built using the respective data sources. The surface plot for each legacy BHM model, Mk = ηˆklegacy (x) + δˆklegacy (x), is shown in Figure 5. Next, the model validity of each legacy model is calculated for each data point generated from the true function and the BHM models. The contour plot of normalized model validity for both the legacy models is shown in Figure 6. It can be seen that the model validity of the legacy-1 BHM model is relatively low for higher values of x1 while, the validity of the legacy-2 BHM model is relatively low for lower values of

6 of 12 American Institute of Aeronautics and Astronautics

15

y

10

y

20

BHM Model Test-data Legacy-1 data

5 0 1

10

0 1

0.5

x2

BHM Model Test-data Legacy-2 data

0.5 0

1

0.5

0

x2

0

x1

(a) Legacy-1 Source

1

0.5

0

x1

(b) Legacy-2 Soruce

x1 . This is mainly due to the predictive uncertainty of each legacy model. For example, the data for the legacy-2 model are only available in the region of x1 ∈ [0.5, 1], which causes a higher predictive uncertainty in the model in the region of x1 ∈ [0, 0.5]. However, there are regions around lower values of x1 where the validity of legacy-2 model is still better than the legacy-1 model. This is due to the contribution of likelihood term in model validity, i.e. in this region the likelihood of the legacy-2 model to predict the true function overpowers the predictive uncertainty when compared to the legacy-1 model. 1

1

1

1

0.9

0.9

0.9

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.1

0

x2

x2

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Figure 5: Bayesian hybrid model (BHM) of legacy sources for exponential function

0

0

0

0.2

0.4

0.6

0.8

1

0

0

x1

0.2

0.4

0.6

0.8

1

x1

(a) Legacy-1 Source

(b) Legacy-2 Soruce

Figure 6: Model validity of legacy sources for exponential function Once the multi-source model is built, it is validated with the validation data set for the new design. In Figure 7a, the left plot shows the predicted versus actual plot of each legacy BHM model. As seen in the figures, the legacy-1 model has a higher error and higher predictive uncertainty when compared to the legacy2 model. The right plot of Figure 7a shows the predictions of the multi-source model which combines both the legacy models. The multi-source model has better predictive accuracy than the legacy BHM models. The improvement in prediction by multi-source modeling is also shown by the percentage error box plot shown in Figure 7b. The median, third quartile and the maximum percentage error was found to be better for the multi-source model. III.B.

Borehole Problem

The second test problem used to demonstrate multi-source modeling is the Borehole problem [22] where an analytical function is used to calculate the water flow rate through a borehole. The analytical function is

7 of 12 American Institute of Aeronautics and Astronautics

Legacy BHM Model 14

Multi-Source Model

Legacy-1 Legacy-2

14

18

Multi-Source

16 12

12

10

10

8

12

% error

Predicted

Predicted

14

8

6

6

4

4

2

2

10 8 6 4 2 0

2

4

6

8

10 12 14

2

4

6

True

8

10 12 14

Legacy-1

Legacy-2

Multi-Source

True

(a) Actual vs Predicted plot

(b) Percentage error distribution

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Figure 7: Prediction and error on validation data for exponential function problem

given in Equation 17. For the true function, parameter α is given as α = [2π, 1.0, 2.0] and is used to generate data for the new design. Additionally, two legacy data sources are generated using the same function but with different parameter values. For the legacy-1 data source, α = [5, 1.5, 2.0] is used while for the legacy-2 data source, α = [6, 0, 1.8] is used. The definition and ranges of all the variables are given in Table 1. f (x) =

α T (H − Hl )  1 u u α3 LTu ln(r/rw ) α2 + ln(r/r + 2 w )r Kw ) w

Variable rw r Tu Hu Tl Hl L Kw

Tu Tl

Definition Radius of borehole (m) Radius of influence (m) Transmissivity of upper aquifer (m2 /yr) Potentiometric head of upper aquifer (m) Transmissivity of lower aquifer (m2 /yr) Potentiometric head of lower aquifer (m) Length of borehole (m) Hydraulic conductivity of borehole (m/yr)



(17)

Range [0.05, 0.15] [100, 50000] [63070, 115600] [990, 1110] [63.1, 116] [700, 820] [1120, 1680] [9855, 12045]

Table 1: Variable definition and ranges for borehole problem To carry out the multi-source modeling, 23 data points are generated for the new design, 50 data points are generated for the legacy-1 source, and 40 data points are generated for the legacy-2 data source. The distribution of data points in the input design space is shown in Figure 8. One can notice that data is missing in some parts of the design space for each legacy data. This is deliberately done to simulate a scenario typical in real engineering problems where legacy designs are built for different requirements. For validation 200 additional data points were generated using the true function and are predicted using the multi-source model. The left plot of Figure 9a shows the predicted versus actual plot for each legacy BHM model, while the right plot shows the same for the multi-source model. It can be observed that the prediction of the multi-source model is relatively tighter to the 45o line than each legacy model. Figure 9b shows the box plot of percentage error of legacy models and multi-source model for all the validation data. Similar to the previous example, the multi-source model had better predictive capability than each legacy model..

8 of 12 American Institute of Aeronautics and Astronautics

New Legacy-1 0.15 Legacy-2

rw

0.1 4 2

r 104

11 10 9 8 7 1100 1050 1000 110 100 90 80 70 800 750

Tu Hu Tl Hl

1600 1400 1200

L

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

10000 5000

Kw

0.05 0.1

0

2

4

7

104

9 111000

70 90 110

750 8001200 16005000 15000

104

Figure 8: Data distribution of new design and legacy design for borehole problem Legacy BHM Model

Multi-Source Model

200

200 16

180 160

140

140

120

120

Predicted

160

100 80

14 12

% error

180

Predicted

Multi-Source

Legacy-1 Legacy-2

100 80

10 8 6

60

60

40

40

4

20

20

2

0

0 0 0

50

100

True

150

200

0

50

100

150

200

Legacy-1

Legacy-2

Multi-Source

True

(a) Actual vs Predicted plot

(b) Percentage error distribution

Figure 9: Prediction and error on validation data of borehole problem

III.C.

Material Crack Growth Rate Prediction

To demonstrate the multi-source modeling method on an engineering problem, material crack growth rate data of three different aluminum allows are used from a Damage Tolerant Design Handbook [23]. The Al7475 alloy is assumed to be a new design, while Al-7150 and Al-7175 alloys are assumed to be legacy designs. For each of these alloys, crack growth rate (da/dn) data is extracted for various loading frequencies (F req), stress ratios (R), amplitude stress intensity factors (∆K). The data were only extracted for test environment of LAB AIR (Laboratory Air) and L.H.A (Low Humidity Air, < 10%RH) and a material orientation of L-T. For legacy material Al-7150 and Al-7175, 40 and 38 data points are extracted, respectively. For the new design, Al-7475, 24 data points are extracted out of which 10 data points are used for training and 14 data points are used for validation. The extracted data are shown in Figure 10 and the scatter plot of input variables are shown in Figure 11.

9 of 12 American Institute of Aeronautics and Astronautics

Aluminium 7150 Alloy

10-4

Aluminium 7175 Alloy

10-4

10-5

10-5

10-6

10-6 R = 0.1 R = 0.4

R = 0.02 R = 0.08

10-7

10-8

10-7

0

2

4

6

8

10

12

14

16

10-8

18

0

(a) Actual vs Predicted plot

4

6

8

10

12

14

16

(b) Percentage error distribution Aluminium 7475 Alloy

10-4

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

2

10-5

R = 0.1 R = 0.3 R = 0.4

10-6

10-7

4

6

8

10

12

14

16

18

(c) Percentage error distribution

Figure 10: Prediction and error on validation data of borehole problem

20

Freq

10

0.3 R

0.2 0.1

15 10

K

New Legacy-1 Legacy-2

5 0

10

0.1

0.2

0.3

5

10

15

Figure 11: Data distribution of new design and legacy design for material crack growth rate problem To carry out the multi-source modeling, legacy BHM models of crack growth rate and models of model validity are built as a function of frequency, stress ratio, and amplitude stress intensity factor. The multi-

10 of 12 American Institute of Aeronautics and Astronautics

Legacy BHM Model

Multi-Source Model

Legacy-1 Legacy-2

20

Multi-Source

20

50

15

10

40

% error

Predicted

Predicted

15

10

30

20 5

5 10

0

0 0 0

5

10

True

15

20

0

5

10

15

20

Legacy-1

Legacy-2

Multi-Source

True

(a) Actual vs Predicted plot

(b) Percentage error distribution

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

Figure 12: Prediction and error on validation data of material crack growth rate problem

source model is validated with 14 data points from the Al-7475 alloy and were not used for training. The predicted versus actual plot is shown in Figure 12a and the box plot of percentage error is shown in Figure 12b. It is observed that the multi-source modeling method is able to significantly improve the predictive accuracy in this case.

IV.

Conclusion

A Bayesian multi-source modeling approach for legacy data is developed to overcome some of the issues of current multi-fidelity and ensemble modeling techniques. In this approach, models of each legacy system and the corresponding discrepancy with the new design is built using a Bayesian Hybrid Model framework. Models were built for the validity of each legacy system as a function of input space. Then, the ensemble model is used to carry out the predictive analysis for a new design. The method has been demonstrated with two analytical problems and and engineering problem and has been found to improve the accuracy of prediction for all three examples.

References 1 Keane, A. and Nair, P., Computational Approaches for Aerospace Design: The Pursuit of Excellence, John Wiley & Sons, 2005. 2 Kennedy, M., “Predicting the output from a complex computer code when fast approximations are available,” Biometrika, Vol. 87, No. 1, 2000, pp. 1–13. 3 Vitali, R., Haftka, R., and Sankar, B., “Correction response surface approximations for stress intensity factors of a composite stiffened plate,” 39th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference and Exhibit, Structures, Structural Dynamics, and Materials and Co-located Conferences, American Institute of Aeronautics and Astronautics, April 1998. 4 Alexandrov, N. M. and Lewis, R. M., “An Overview of First-Order Model Management for Engineering Optimization,” Optimization and Engineering, Vol. 2, No. 4, Dec. 2001, pp. 413–430. 5 Eldred, M., Giunta, A., and Collis, S., “Second-Order Corrections for Surrogate-Based Optimization with Model Hierarchies,” 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Multidisciplinary Analysis Optimization Conferences, American Institute of Aeronautics and Astronautics, Aug. 2004. 6 Haftka, R. T., “Combining global and local approximations,” AIAA Journal, Vol. 29, No. 9, Sept. 1991, pp. 1523–1525. 7 Umakant, J., Sudhakar, K., Mujumdar, P., and Rao, C., “Customized Regression Model for Improving Low Fidelity Analysis Tool,” 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Multidisciplinary Analysis Optimization Conferences, American Institute of Aeronautics and Astronautics, Sept. 2006. 8 Berci, M., Toropov, V., Hewson, R., and Gaskell, P., “Metamodelling Based on High and Low Fidelity Model Interaction for UAV Gust Performance Optimization,” 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Structures, Structural Dynamics, and Materials and Co-located Conferences, American Institute of Aeronautics and Astronautics, May 2009. 9 Qian, P. Z. G. and Wu, C. F. J., “Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments,” Technometrics, Vol. 50, No. 2, May 2008, pp. 192–204.

11 of 12 American Institute of Aeronautics and Astronautics

Downloaded by You Ling on January 12, 2018 | http://arc.aiaa.org | DOI: 10.2514/6.2018-1663

10 Forrester, A. I. J., S´ obester, A., and Keane, A. J., “Multi-fidelity optimization via surrogate modelling,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 463, No. 2088, 2007, pp. 3251–3269. 11 Toropov, V. V., “Modelling and Approximation Strategies in Optimization: Global and Mid-Range Approximations, Response Surface Methods, Genetic Programming, Low/High Fidelity Models,” Emerging Methods for Multidisciplinary Optimization, edited by J. Blachut and H. Eschenauer, chap. 5, Vol. 425, CISM Courses and Lectures, Springer-Verlag, New York, 2001, pp. 205–256. 12 Opitz, D. and Maclin, R., “Popular ensemble methods: An empirical study,” Journal of Artificial Intelligence Research, Vol. 11, 1999, pp. 169–198. 13 Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, C. T., “Bayesian model averaging: a tutorial,” Statistical science, 1999, pp. 382–401. 14 Zhang, C. and Ma, Y., Ensemble machine learning, Vol. 1, Springer, 2012. 15 Kennedy, M. C. and O’Hagan, A., “Bayesian calibration of computer models,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 63, No. 3, 2001, pp. 425–464. 16 Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A., and Ryne, R. D., “Combining field data and computer simulations for calibration and prediction,” SIAM Journal on Scientific Computing, Vol. 26, No. 2, 2004, pp. 448–466. 17 Hastings, W. K., “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika, Vol. 57, No. 1, 1970, pp. 97–109. 18 Kumar, N. C., Subramaniyan, A. K., and Wang, L., “Improving high-dimensional physics models through Bayesian calibration with uncertain data,” ASME Turbo Expo 2012: Turbine Technical Conference and Exposition, American Society of Mechanical Engineers, 2012, pp. 407–416. 19 Kumar, N. C., Subramaniyan, A. K., Wang, L., and Wiggs, G., “Calibrating transient models with multiple responses using Bayesian inverse techniques,” ASME Turbo Expo 2013: Turbine Technical Conference and Exposition, American Society of Mechanical Engineers, 2013, pp. V07AT28A007–V07AT28A007. 20 Currin, C., “A Bayesian approach to the design and analysis of computer experiments,” Tech. rep., ORNL Oak Ridge National Laboratory (US), 1988. 21 Xiong, S., Qian, P. Z., and Wu, C. J., “Sequential design and analysis of high-accuracy and low-accuracy computer codes,” Technometrics, Vol. 55, No. 1, 2013, pp. 37–46. 22 Morris, M. D., Mitchell, T. J., and Ylvisaker, D., “Bayesian design and analysis of computer experiments: use of derivatives in surface prediction,” Technometrics, Vol. 35, No. 3, 1993, pp. 243–255. 23 Skinn, D., Gallagher, J. P., Berens, A. P., Huber, P., and Smith, J., “Damage Tolerant Design Handbook. Volume 5. Chapter 8.” Tech. rep., Dayton Univ OH Research Inst, 1994.

12 of 12 American Institute of Aeronautics and Astronautics