Bayesian optimal design of step stress accelerated ...

ES20141141，Reliability

Bayesian optimal design of step stress accelerated degradation testing Xiaoyang Li *1, Mohammad Rezvanizaniani2, Zhengzheng Ge3, Mohamed AbuAli2 and Jay Lee2 1. Science and Technology on Reliability and Environmental Engineering Laboratory, Beihang University, Beijing 100191, P.R.China; 2. NSF Industry/University Cooperative Research Center on Intelligent Maintenance Systems (IMS), University of Cincinnati, OH 45221, USA 3. Beijing institute of electronic system engineering, Beijng 100854, P.R.China

Abstract:

This study presents a Bayesian

surface fitting algorithm is chosen. At the end of the

methodology for designing step stress accelerated

paper, a NASA lithium-ion battery dataset was used as

degradation testing (SSADT) and its application to

historical information and the KL divergence oriented

batteries. First, the simulation-based Bayesian design

Bayesian design are compared with maximum

framework for SSADT is presented. Then, by

likelihood theory oriented locally optimal design. The

considering historical data, specific optimal objectives

results show that the proposed method can provide a

oriented

much better testing plan for this engineering

Kullback–Leibler

(KL)

divergence

is

established. A numerical example is discussed to

application.

illustrate the design approach. It is assumed that the

Keywords:

degradation model (or process) follows a drift

theory; KL divergence; degradation; optimal

Brownian motion; the acceleration model follows

design; battery

accelerated testing; Bayesian

Arrhenius equation; and the corresponding parameters follow normal and Gamma prior distributions. Using Markov Chain Monte Carlo (MCMC) method and

1 Introduction Acceleration

degradation

testing

(ADT)

WinBUGS software, the comparison shows that KL

technique is commonly used to obtain

divergence is better than quadratic loss for optimal

degradation data of products over a short time

criterion. Further, the effect of simulation outliers on

period, to help extrapolate lifetime and reliability under usage conditions . Constant

the optimization plan is analyzed and the preferred

stress accelerated degradation testing (CSADT) and SSADT are the two types of ADT that are Manuscript received … *Corresponding author Xiaoyang Li This work was supported by the National Natural Science Foundation of China (stochastic process and relative entropy based Bayesian design for SSADT, 61104182)

1

commonly performed. In CSADT, the samples

important factor that affects the testing design

are divided into different groups and each

plan. Tang et al.[2] proposed an optimal design

group is tested under different constant stress

for SSADT and determined the sample size,

level. For SSADT, all samples are tested with

testing duration and inspection times in

same stress conditions to begin with and the

consideration with the testing cost. By using

stress levels increase with time in a step by

Weiner process and the Gamma process

step way. SSADT has the advantage of smaller

respectively, Liao and Tseng[3] and Tseng[4]

sample size and shorter testing time than

also proposed the optimal design for SSADT

CSADT. Research on SSADT has rapidly

considering testing cost. Li[5] also provided

increased in the past decades[1-3].

optimal solutions for SSADT considering

Traditional optimal design of SSADT is

competing failure causes. According to the

based on a known degradation model and

competing failure rule, Li used the drift

specified values of model parameters. If the

Brownian motion to establish a reliability

deviation between the specified value and the

model and minimized the asymptotic variance

true value is large, an optimal testing plan is

of the estimated 100th percentile of the

usually hard to obtain. Consequently, the plan

competing reliability model under normal

will not provide the most effective testing data.

conditions with a constraint that the total

If prior information or similar data of products

experimental cost is below a pre-determined

is available before testing, the design of

budget. Based on the analysis of the optimal

SSADT using Bayesian method will be more

plans with different budgets, stress levels and

reliable and efficient than using the traditional

stress steps, Li[5] summarized the guidelines

method.

for stress loading principles of SSADT were

Park et al.

[1]

tested designs for SSADT

proposed.

with destructive performance measurements.

Typically, some information such as

Based on the cumulative effects of exposure,

some laboratory testing data and field

they established a degradation model with

operating data for similar products, is usually

lognormal

constant

available prior to the experiments on a specific

degradation rate for the performance. They

product. Therefore, Bayesian methods can

minimized the asymptotic variance of the

play

maximum likelihood estimator of the 100th

Verdinelli[6]

percentile of the lifetime distribution under the

Bayesian experimental designs and indicated

using condition. Then, they determined the

that it is difficult to obtain a posterior

optimal stress levels, the number of units

distribution in a closed form. There are two

measured at each test point, and the stress

ways to solve this problem [7], one is based on

changing time. The cost of testing is also an

simulation and the other is based on

distribution

and

an

important and

role.

Chaloner

and

Clyde[7]

reviewed

the

2

large-sample theory.

specified.

Research on Bayesian accelerated testing

There is limited published literature on

design mostly is based on accelerated life

Bayesian research for ADT plan. Hamada et

testing (ALT). Erkanli and Soyer[8] studied the

al.[11] used a mixed effect model with normal

Bayesian design for constant stress accelerated

and inverse Gamma distributions as the priors

life testing (CSALT) using MCMC method.

for the parameters and proposed the optimal

To avoid computational issues, a curve fitting

design

approach

facilitate

optimization, the objective was to minimize

pre-posterior analysis and find an optimal

the total number of inspections, and the

design.

was

Zhang

adopted

Meeker[9]

degradation testing.

In

this

used

constraint was that the length of credible

large-sample approximation to obtain the

interval should not exceed a given value. Liu

Bayesian criterion. In this method, the general

and Tang[12] presented a Bayesian optimal

equivalence theorem (GET) was used to find

design for CSADT using quadratic loss as the

out an optimal plan for this kind of non-linear

optimality criterion, and used and simulation

problem. Liu and Tang proposed a scheme of

combined with surface fitting as the stochastic

sequential CSALT[10]. In this scheme, testing

optimization

at the highest stress is first conducted to

showed that the Bayesian design approach

quickly acquire failures. Then, based on this

could significantly help with enhancement of

failure data, prior distributions for lower stress

robustness of an ADT plan against the

levels are established using the Bayesian

uncertainty[13]. Shi and Meeker[14] studied the

inference method. They first came up with two

Bayesian optimal design for accelerated

Bayesian inferences, and then categorized into

destructive degradation testing in which the

all-at-one prior-distribution construction (APC)

estimation

and

prior-distribution

distribution quantile at use conditions was the

construction (FSPC). Under their scheme,

optimal objective. They also used nonlinear

using large-sample approximation and failure

regression model to describe degradation

information collected under the highest stress

process

level, the prior of lower stresses was obtained.

distribution was obtained by a large-sample

By choosing the optimality criterion as

approximation.

full

and

to

for

sequential

minimizing the pre-posterior expectation of the asymptotic

posterior

method.

precision

and

the

Their

of

a

comparison

failure

simplified

time

posterior

From the above introduction, it is could

variance at a

be known that the two existed ADT Bayesian

percentile of interest under the normal stress

design are focused on constant stress loading

level, the testing plan can be optimized. In

way. And in [13], the objective is quadratic loss

other words, the remaining sample size and

of special parameter in the accelerated model

testing duration on each stress level can be

which might not be a good measure for the

3

optimality of testing plan. With regarding to

experiment is to maximize the expected utility

[14]

, large-sample approximation was used to

or minimize the expected loss. Since testing

derive the simplified posterior distribution of

plan needs to be designed before sample data

parameter, and that means the sample size is

collected, the expected utility should be

large. But Chaloner and Verdinelli[6] have

integrated over possible outcomes in the

been pointed out that when sample size is

sample space Ω. And with the consideration of

large, the posterior distribution will be driven

the uncertainty of parameter vector θ, the

by the data and will not be sensitive to the

utility also need to be the expectation in the

prior distribution. Consequently, Bayesian

parameter space Θ. Therefore, for any design

design has little impact on testing plan.

η, the expected utility function Λ(η) of the best

Therefore, we investigate a new Bayesian

decision is expressed as[6]:

optimal design for SSADT in which KL

    max  D

 U (, x, θ) (θ x , ) p( x , θ)dθdx

 

divergence will be as the optimality criterion (1)

to entirely measure the distance between prior and

posterior

distribution.

Since

KL

divergence can be also known as measure of the information provided by testing, it is better than quadratic loss which only takes one aspect of testing into consideration. The remaining paper is organized as follows. In Section 2, the framework of Bayesian optimal design for ADT is presented. In Section 3, drift Brownian motion and Arrhenius equation are used as degradation model and acceleration model, respectively, to illustrate the proposed procedures. In Section 4, KL divergence oriented Bayesian design and local optimal design are both applied to battery’s SSADT. Finally, the conclusions are made and the future work is pointed out.

where, π(θ) is the prior distribution of parameter θ; π(θ|x, η) is the posterior distribution under the design η, data x and prior π(θ); U(η, x, θ) is utility function which depends on the data x of a future experiment with design η and parameter θ; p(x|η, θ) is the likelihood function with the condition of design η and parameter vector θ. In most cases, it is difficult to obtain posterior in an analytical closed form. Thus, numerical computation and simulation are the main contents in the Bayesian theory. When straight Monte Carlo simulations are used, the samples need to be drawn from every design and therefore needs large number of iterations. To calculate the posterior means or KL divergence, Gibbs sampling is typically

2 Framework

of

Bayesian

Optimal Design for SSADT

conducted, which also involves a large number of simulations. Thus, Monte Carlo simulations are time-consuming and are

Bayesian testing plan is essentially a Bayesian decision problem. The objective of the

therefore not used in this study. Instead, surface fitting methods will be substituted[8, 12, 4

15]

.

regarding to S, we introduce the stress ratio

2.1 Surface

fitting

based

Bayesian

Optimal Design for SSADT

ξ=(ξ1,…, ξl ,…, ξk) ( l  [0,1] ) to simplify the specified stress values:

The basic optimization steps using surface fitting are presented below and a flow chart is shown in Fig. 1. Draw design ηi from feasible region

l  (sl  smin ) (smax  smin )

(2)

Corresponding to M, the measurement allocation ratio ρ=(ρ1,…, ρl ,…, ρk) ( l  (0,1) ) denotes the number of measurements assigned to the lth stress level:

Simulate θi from the priors

l  ml

Generate data Xi based on θi

m ,  l

l

1

(3)

In this way, our design space involves the stress ratio ξ and the allocation ratio ρ. For the

Calculate expected utility Ui

case of two stress levels (also known as 2-point design), the highest stress level could

Surface fitting based on the pair of (ηi , U(ηi)) or (ηi , L(ηi))

be fixed as smax, i.e. s2= smax and ξ2=1, then only 1  (s1  smin ) (smax  smin ) needs to be

Find the maximum or minimum of the fitted surface

optimized. Similarly, we

only need to

determine ρ1, because of the given total

Fig. 1 Flow chart of basic steps for SSADT Bayesian design Based on Surface Fitting

measurements and 2  1  1 . and

In addition, it is hard to specify the exact

measurement interval is constant in SSADT,

structure of the surface regression model.

the magnitudes of stress levels and the number

Therefore, a non-parametric method such as

of measurements needed to optimize, i.e.,

Kernel smoother is preferred. For comparison,

these two factors are decision variables and

both locally weighted polynomial (e.g., locally

span the decision space of testing design. In

weighted

order to reduce computation time, there is

polynomial regression methods are used in

need

this paper.

When

sample

reparameterize

size

is

stress

given

levels

and

scatterplot

smoothing)

and

measurement numbers. For k-level SSADT, S

2.2 KL Divergence

is a vector of accelerated stress level, S=

Many researchers prefer to use asymptotical

(s1,…, sl ,…,sk), the element sl means the

approximation as the optimization criterion. If

magnitude of the lth stress level,. M is a

asymptotical

measurement vector, M= (m1, … , ml, … , mk),

optimality, then historical information can

the element ml means there are ml degradation

play a little role in parameter estimations. On

data collected on the lth stress level. Here, we

the other hand, asymptotical approximation is

normalize these two vectors, S and M. With

reasonable only when sample size is large, but

approximation

is

used

for

5

this requirement is hard to meet in real world.

posterior distributions, it is hard to get a

In Bayesian theory, KL divergence is

closed form analytical solution. In this

always used as a measure of distance between a

situation, the best choice to compute KL

prior distribution and a posterior distribution.

divergence is the Monte Carlo method.

From the perspective of Shannon information,

Since p( x θ ) is the likelihood function,

it also represents the information gain contributed by an experiment, also known as

it

is

relatively

easy

to

calculate

relative entropy. Based on this fact, to the

Ex ,θ| log p( x θ ) 

process of how to design the optimal SSADT

simulation. However, the marginal likelihood

with respect to maximize KL divergence is

function p(x) presents a significant challenge,

presented in this paper.

and is a topic of ongoing research. MCMC

According to Lindley[16], the information contained in the prior distribution is:

using

Monte

Carlo

sampling method is one of the solutions to solve this problem and can be implemented using a software called WinBUGS[17, 18]. The

I 0    (θ ) log  (θ )dθ Eθ log  (θ )

(4)

The amount of information taken with

(5)

The information provided by experiment η was defined by Lindley[16] as following:

I ( , x, p(θ))  I1 ( x)  I 0 Based

on

Bayesian

Theorem,

be used to estimate the marginal likelihood Ex| log p( x) , based on guidance provided by

posterior distribution is expressed as: I1 ( x)    (θ x) log  (θ x)dθ

following Laplace-Metropolis algorithm will

Ntzoufras[18]. p( x)  (2 )d / 2 Σ

1/ 2

p( x θ ) (θ )

(9)

RML

θ  1/ RML    θg and Σ 

(6) the

g 1

RML

1/  RML  1     θg  θ  θg  θ 

T

g 1

expectation of experimental information can

where, θ and Σ are the posterior mean and

be written as:

posterior variance-covariance matrix of the

Ex ,θ |  I1  I 0 

simulated values, respectively; RML is the

 Ex ,θ | log p( x θ )   Ex|  log p( x)     

(7)

where, p(x) is the marginal likelihood and also

θ , 1

, θRML



from the

posterior distribution. Since KL divergence represents the

a normalizing constant expressed as: p( x)   p( x θ ) (θ )dθ

number of samples

information gained from the experiments, the (8)

Equation (7) is known as KL divergence

optimal problem of Bayesian design for SSADT could be:

and it is the expected utility of our Bayesian optimal design for SSADT. For most of 6

Ex ,θ|  I1  I 0 

max s.t.

Step 5.

Calculate KL divergence Ur based

on Equation (7) for the design ηr;

for a given n S0  S1  S2    Sk  Smax 1  m1  m2    mk  0,  l 1 ml  1

Step 6.

Go to step 1 and repeat step 1-5 for

k

every feasible design in the design

The steps for KL divergence based on surface fitting are shown below: Step 1.

space； Step 7.

Divide design space D= ξ1 × ρ1 into

equal segments in the direction of the

Based on the pair (ηr, Ur), the

surface can be fitted, and Step 8.

The optimal plan for SSADT is

stress ratio ξ1 and the allocation ratio ρ1.

obtained by taking maximum of this

Then, there will be RD simulating point in

surface.

the design space with boundaries; Step 2.

3 Illustration

Draw the design ηr from D (r = 1,…,

RD), and simulate parameter θr from their corresponding prior distributions. Based on the simulated θr, generate degradation data xrq from sampling distribution

In this section, we will narrow down to a specific

degradation

acceleration

model

model based

and on

an some

assumptions. Then, the process of optimal design will be discussed.

pdf ( x θrh ) for R1 times (q = 1,…, R1).

Then use MCMC algorithm to estimate the posterior means θq and Σq for the qth

3.1 Assumptions and models A.

Assumptions Brownian motion is one of the most

powerful stochastic processes in continuous simulated xrq; time and space. It has wide applications in Step 3.

Calculate the marginal likelihood many disciplines such as physics, economics,

p(xrq)

using

Equation

(9)

and communication theory, and reliability theory

Ex log p( x)   q11 log p( xrq )  R1 ;   R

Step 4.

Therefore, it is used herein to fit the

Draw the design ηr from D (r = 1,…,

RD), and simulate parameter θrh from

accelerated degradation process[19]. The following assumptions are made in

their corresponding prior distributions, h

the analysis[3, 20]:

= 1,…, R2. Then, generate degradation

Assumption 1 Degradation is irreversible

increment data xrh from the sampling

Assumption 2 Degradation mechanism will

distribution

p( x θrh )

.

Calculate

not change with stress Assumption 3 Under the normal condition S0

Ex Eθ log p( xrh θrh ,r ) 

 R2 log p( xh θh )  R2 ;  h 1 

via

and

k-level

accelerated

conditions,

s1 2 stress levels. The testing cost is a significant constraining factor which cannot be neglected. It is also important that the choice and sensitivity analysis of prior distribution should be conducted in the Bayesian design and interference. In future, the proposed method should be extended to those applications.

Acknowledgment This work was partially supported by the National Natural Science Foundation of China under Grant NSFC 61104182.

References [1] PARK S-J, YUM B-J, BALAMURALI S. Optimal design of step-stress degradation tests in the case of destructive measurement [J]. Quality Technology & Quantitative Management, 2004, 1(1): 105-24. [2] TANG L-C, YANG G, XIE M. Planning of step-stress accelerated degradation test; proceedings of the Reliability and Maintainability, 2004 Annual Symposium-RAMS, F, 2004 [C]. IEEE. [3] LIAO C-M, TSENG S-T. Optimal design for step-stress accelerated degradation tests [J]. Reliability, IEEE Transactions on, 2006, 55(1): 59-66. [4] TSENG S-T, BALAKRISHNAN N, TSAI C-C. Optimal step-stress accelerated degradation test plan for gamma degradation processes [J]. Reliability, IEEE Transactions on, 2009, 58(4): 611-8. [5] LI X, JIANG T. Optimal design for step-stress accelerated degradation testing

with competing failure modes; proceedings of the Reliability and Maintainability Symposium, 2009 RAMS 2009 Annual, F, 2009 [C]. IEEE. [6] CHALONER K, VERDINELLI I. Bayesian experimental design: A review [J]. Statistical Science, 1995, 10(3): 273-304. [7] CLYDE M A. Experimental Design: Bayesian Designs [M]//SMELSER N J, BALTES P B. International Encyclopedia of the Social & Behavioral Sciences. Oxford; Pergamon. 2001: 5075-81. [8] ERKANLI A, SOYER R. Simulation-based designs for accelerated life tests [J]. Journal of statistical planning and inference, 2000, 90(2): 335-48. [9] ZHANG Y, MEEKER W Q. Bayesian methods for planning accelerated life tests [J]. Technometrics, 2006, 48(1): 49-60. [10] LIU X, TANG L C. A sequential constant‐stress accelerated life testing scheme and its Bayesian inference [J]. Qual Reliab Eng Int, 2009, 25(1): 91-109. [11] HAMADA M S, WILSON A, REESE C S, et al. Bayesian reliability [M]. Springer, 2008. [12] LIU X, TANG L C. A Bayesian optimal design for accelerated degradation tests [J]. Qual Reliab Eng Int, 2010, 26(8): 863-75. [13] SHI Y, MEEKER W Q. Bayesian methods for accelerated destructive degradation test planning [J]. Reliability, IEEE Transactions on, 2012, 61(1): 245-53. [14] SHI Y, MEEKER W Q. Bayesian Methods for Accelerated Destructive Degradation Test Planning [J]. IEEE Trans Reliability, 2012, 61(1): 245-53. [15] M LLER P, PARMIGIANI G. Optimal design via curve fitting of Monte Carlo experiments [J]. Journal of the American Statistical Association, 1995, 90(432): 1322-30.

17

[16] LINDLEY D V. On a measure of the information provided by an experiment [J]. The Annals of Mathematical Statistics, 1956, 986-1005. [17] LUNN D J, THOMAS A, BEST N, et al. WinBUGS-a Bayesian modelling framework: concepts, structure, and extensibility [J]. Statistics and computing, 2000, 10(4): 325-37. [18] NTZOUFRAS I. Bayesian modeling using WinBUGS [M]. John Wiley & Sons, 2011. [19] LIAO H T, ELSAYED E A. Reliability inference for field conditions from accelerated degradation testing [J]. Nav Res Logist, 2006, 53(6): 576-87. [20] LIM H, YUM B-J. Optimal design of accelerated degradation tests based on Wiener process models [J]. Journal of Applied Statistics, 2011, 38(2): 309-25. [21] ROBERT C P. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation (Springer Texts in Statistics) by [J]. 2001, [22] LI X, JIANG T, SUN F. Constant Stress ADT for Superluminescent Diode and Parameter Sensitivity Analysis [J]. Eksploatacja I Niezawodnosc-Maintenance And Reliability, 2010, 46(2): 21-6. [23] SAHA B, GOEBEL K. Battery Data Set [M]. http://ti.arc.nasa.gov/tech/dash/pcoe/progno stic-data-repository/; NASA Ames Prognostics Data Repository. 2007. [24] GE Z, JIANG T, HAN S, et al. Design of accelerated degradation testing with multiple stresses based on D optimality [J]. Systems Engineering and Electronics, 2012, 34(4): 846-53.

Dr. Xiaoyang Li is an Associated Professor in the Science and Technology on Reliability and Environmental Engineering Laboratory at Beihang University (BUAA), where she has been affiliated since 2007.

She

has

Industry/University

been Cooperative

to

NSF

Research

Center on Intelligent Maintenance Systems (IMS), University of Cincinnati, as a visiting professor for one year. Dr. Li received her Ph. D. degree from BUAA in systems engineering of aeronautics and astronautics, in 2007, and a B.S. degree from BUAA in quality and reliability, in 2002. Her research interests focus on life prediction, design of experiment, accelerated testing. E-mail: [email protected]

Mohammad Rezvani is a PhD student in IMS. He received his M.S. degree from Luleå University of Technology, Luleå, Sweden, in Maintenance Management and Engineering, in 2008, and a B.S. degree from Iran Science and Technology University, Tehran, Iran, in Mechanical Engineering, in 2002. He has the deep knowledge in the field of smart battery, electric vehicle, applying Prognostics & Health Management Tools, 18

Reliability

Analysis

and

Preventive

(SSGB) and a Certified Quality Engineer

Maintenance.

(CQE).

E-mail: [email protected]

E-mail: [email protected]

Dr. Zhengzheng Ge is a senior

Prof. Jay Lee is an Ohio

engineer in Beijing institute of electronic

Eminent Scholar and L.W. Scott Alter Chair

system engineering. She obtained her PhD

Professor at the Univ. of Cincinnati and is

degree in 2012, and a B.S. degree from Hebei

founding

University of Technology in automation, in

Foundation

2006.. Her research interests include reliability

Cooperative Research Center (I/UCRC) on

testing, accelerated test modeling and test

Intelligent Maintenance Systems. He received

designing. Her recent work is on optimal

his B.S degree from Taiwan in 1979, a M.S. in

design for accelerated degradation testing.

Mechanical Engineering from the Univ. of

E-mail: [email protected]

Wisconsin-Madison in 1983, a M.S. in

director

of

(NSF)

National

Science

Industry/University

Industrial Management from the State Univ. of New York at Stony Brook in 1987, and D.Sc. in Mechanical Engineering from the George Dr. Mohamed Abuali is CEO

Washington University in 1992. His current

of FORCAM and responsible for leading and

research focuses on dominant innovation

growing FORCAM operations in the Americas.

design

He obtained a Ph.D. in Industrial Engineering

technologies for service and maintenance

through his studies at the Center for Intelligent

automation applications.

Maintenance Systems (IMS) at the University

E-mail: [email protected]

tools

and

smart

infotronics

of Cincinnati in 2010, a M.S. degree from The American University in Cairo in Industrial Engineering in 2005, and a B.S. degree from University of Arizona in systems engineering in 2003. He is also a Project Management Professional (PMP), a Six Sigma Green Belt

19