Case based time series prediction using biased ... - Semantic Scholar

Applied Soft Computing 13 (2013) 2467–2477

Contents lists available at SciVerse ScienceDirect

Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc

Case based time series prediction using biased time warp distance for electrical evoked potential forecasting in visual prostheses Jin Qi a,∗ , Jie Hu a , Yinghong Peng a , Xinyu Chai b , Qiushi Ren c a b c

Institute of Knowledge Based Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai 200240, China Institute of Laser Medicine and Bio-Photonics, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai 200240, China Department of Biomedical Engineering, Peking University, 60 Yannan Yuan, Beijing 100871, China

a r t i c l e

i n f o

Article history: Received 20 October 2011 Received in revised form 8 October 2012 Accepted 25 November 2012 Available online 11 December 2012 Keywords: Time series prediction Electrical evoked potential Temporal similarity measure Distance metric Bias configuration

a b s t r a c t Case based time series prediction (CTSP) is a machine learning technique to predict the future behavior of the current time series by referring similar old cases. To reduce the cost of the visual prostheses research, we devote to the investigation of predictive performance of CTSP in electrical evoked potential (EEP) prediction instead of doing numerous biological experiments. The heart of CTSP for EEP prediction is a similarity measure of training case for target electrical stimulus by using distance metric. As EEP experimental case consists of the stationary electrical stimulation values and time-varying EEP elicited values, this paper proposes a new distance metric which takes the advantage of point-to-point distance’s efficient operation in stationary data and time series distance’s high capability in temporal data, called as biased time warp distance (BTWD). In BTWD metric, stimulation set difference (Diff I) and EEP sequence difference (Diff II) are calculated respectively, and a time-dependent bias configuration is added to reflect the different influences of Diff I and Diff II to the numerical computation of BTWD. Similarity-related adaptation coefficient summation is employed to yield the predictive EEP values at given time point in principle of k nearest neighbors. The proposed predictor using BTWD was empirically tested with data collected from the electrophysiological EEP eliciting experiments. We statistically validated our results by comparing them with other predictor using classical point-to-point distances and time series distances. The empirical results indicated that our proposed method produces superior performance in EEP prediction in terms of predictive accuracy and computational complexity. © 2012 Elsevier B.V. All rights reserved.

1. Introduction 1.1. The feasibility of applying case based time series prediction (CTSP) in visual prostheses As the interests in time series have been explored by the number of domains in which time series occur: medicine, industry, entertainment, finance, computer science, meteorology and in almost every other field of human activity [1], the problem of time series prediction (TSP) has attracted great attention [2–5], in which the historical observation values are collected and analyzed to predict the possible values of future time point. Different from traditional TSP approaches based on neural network (NN) model or regression model, case based TSP (CTSP) allows us to use previous data in predicting current data without completely understanding the underlying mechanism of time series behavior [6], actually, it

∗ Corresponding author. Tel.: +86 21 34206076; fax: +86 21 34206313. E-mail address: [email protected] (J. Qi). 1568-4946/$ – see front matter © 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.asoc.2012.11.047

would be very useful for experts to make their decision and predict the future behavior of the current time series by referring similar old case from past. The purpose of optic nerve visual prosthesis is to deliver the electrical impulses to optic nerve with the penetrating electrode, and the visual perception is then emerged from the visual cortex, therefore, the analysis of responses of the visual cortex after the electrical stimulation to the optic nerve is a key part in optic nerve visual prosthesis [7–10]. To explore the temporal property of responses, electrical evoked potentials (EEPs) are elicited over the rabbit skull when the optic nerve was stimulated with specific electrical stimulation current, as shown in Fig. 1. However, subjected to experiment cost and material restriction, the EEP response data triggered by various stimulations are too sparse and inadequate to be analyzed, and the characteristics of EEP data are time series, non-linear, inherently noisy, non-stationary, and deterministically chaotic [11]. To reduce the cost of the visual prostheses research, it is useful to exploit existing EEP electrophysiological data to predict new EEP response at given time point under new electrical stimulations, e.g., the current intensity, the pulse duration, the frequency

2468

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

Fig. 1. Electrical evoked potentials (EEPs) are elicited when the optic nerve was stimulated with specific electrical stimulation current.

of the stimuli, etc. In our previous works [11,12], we presented an adapted support vector machine (SVM) based TSP model to predict EEP movement but with higher computational complexity. Enlighten by the advantages of CTSP, we devote to the investigation of predictive performance of CTSP to solve the EEP forecasting problem.

1.2. The contribution of this research In this research, we attempt to employ CTSP to undertake EEP prediction task, and each experimental sample is expressed as an EEP experimental case, which involves stationary electrical stimulation values and time-varying EEP elicited values. The contribution of this paper is twofold. On the one hand, we try to construct a new distance function of similarity measure for EEP prediction. For the sake of implementing CTSP, training case should capture the evolution of the observed phenomenon over time [13], but time-varying information in cases will be neglected or the retrieval procedure will require massive calculated quantity when classical point-to-point distance or time series distance is used to measure the similarity of each case. The motivation of this research lies in employing a novel distance metric to calculate the similarity of EEP biological case by referring classic distance metrics, and intending to introduce the bias parameter to reflect the different influences from stationary electrical stimulation values and time series EEP elicited values for new predictive result, so the proposed distance is called biased time warp distance (BTWD). This paper is the first attempt to develop a new similarity measurement in CTSP which takes the advantages of classical point-to-point distance function’s efficient operation in stationary data and time series distance function’s high capability in temporal data. On the other hand, we try to investigate the superiority of BTWD in actual EEP prediction, and the predictive performances of CTSP with BTWD and other TSP approaches are planned to be compared. Initial EEP experimental data are collected from the studies of rabbit’s visual cortex by optic nerve stimulation carried out inside the pia mater, on the pia mater and on the dura mater. In the assessment of predictive performance, we employ 30-time cross-validation strategy by combining hold-out method and leaveone-out cross-validation. Predictive performance is evaluated by predictive results produced on hold-out data. Statistical analysis is employed to find whether or not there are significant differences among comparative predictors in the light of the predictive results. The breakdown of this paper is organized as follows. The next section summarizes the research trends in TSP, and the motivation and originality of this research. Section 3 presents the specification on the BTWD. Section 4 describes the empirical design. Section 5 presents the empirical results and relevant discussions. The last section provides the conclusions and suggestions for further research.

2. Related works 2.1. Model based time series prediction Model based TSP (MTSP) represents a family of TSP methods based on various predictive models. Initially, TSP for business failure prediction has been extensively studied from the views of statistic models in 1960s [14], e.g., discriminant analysis model [15,16] and logistic regression model [17] were applied to predict business failure. Later, moving average model [18] and auto-regressive integrated moving average model [19] have been proposed to forecast the monthly electricity demand, but statistic models only work well when the data dependency is linear [11]. To coincide with non-linear fluctuation path of time series data, the neural network (NN) models have been developed for time series forecasting since the 1990s [20–25], which apply the empirical risk and mean squared error minimization in some way. Chang et al. [2,26,27] argued that the multi-layer perceptron NN model is the most commonly used NN model for TSP. Although NNs work well, they have some inherent drawbacks, for example the problem of multiple local minima, the choice of number of hidden units and the danger of overfitting [28–30]. Another widely used non-linear model for TSP is support vector machine (SVM) model [31,32]. Unlike NN model, SVM implements the structural risk minimization and tries to minimize an upper bound of generalization error instead of minimizing the misclassification error or deviation from correct solution of the training data. Thus, SVM could achieve an optimum network structure, and resulting in better generalization performance than NNs [33–36]. Under this principle, the SVM based TSP for EEP prediction has been presented in our previous works [11,12], but we found that the over-complex SVM model would be computationally intensive. Moreover, MTSP operates training data implicitly in black box and remains difficult to explain the final predictive results.

2.2. Case based time series prediction Compare with MTSP approach, CTSP is believed to be a preeminent method for predicting temporal activity by the following reasons [37]. First CTSP is considered as a non-parametric method which dose not requires any time-stamp data distribution assumption for input temporal case. Second, CTSP is an incremental learning technique that can retain new case without reprocessing to update the previous case base. By contrast, MTSP methods like SVM are batch-oriented, wherein both new and old time series data must be submitted as a single batch to the model. And third, CTSP not only can process data more explicitly in explaining analytic results but also can address better efficiency in predicting dynamic fluctuant problem [38]. Performance of CTSP depends largely on the mechanism of similarity measure. Classical point-to-point distance functions such as Euclidean distance (ED) [28,39–41] and Manhattan distance

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

(MD) [42] have been employed to measure the similarity in the process of case retrieval of CTSP, where the temporal sequence of case is viewed as a series of points in an appropriate multidimensional space. Other distance measures used in time series clustering, as reviewed by Liao [43], could also be used here. However time-varying information embedded in temporal cases has been neglected and its role also has been oversimplified in traditional CTSP. To handle the time series information embedded in training case, temporal dimension reduction approaches such as discrete Fourier transform [44,45], discrete wavelet transform [46], singular value decomposition [47], piecewise linear approximation [48] and symbolic aggregate approximation [49,50] are employed to transfer the two-dimensional time series sequence into one dimension space which consists of a series of points, and then the classical ED metric is utilized to compute the similarity. Later, to achieve a better measurement effect, time series distance metrics have been developed to calculate the similarities among time series sequences without temporal dimension reduction, such as dynamic time warping (DTW) distance [49–55], cosine distance [56–58] and Lp norm [59–61]. Boucheham [62] designed a shape exchange algorithm (SEA) in distance function, which is able to perform a point-to-point matching and render a kind of difference between the two time series. However, Lp norm and SEA have the same weakness as the ED metric in that it difficult to capture the fluctuation trends in time series data [61]. Among the time series distances, DTW is the most popular way, but it dose not satisfy the triangular inequality which is a desirable property to speed up retrieval procedure [61,62]. Another category of time series distance function appropriates for time-stamp data is edit distance, developed by Levenshtein firstly in 1966 [63]. Unlike other time series distances, this distance is defined as the minimum number of edit operations needed to transform a sequence into another [61,64,65]. Following that, the extension methods of edit distance have been proposed, e.g., longest common sub-sequence (LCSS) [66], edit distance with real penalty (ERP) [67], and point-patterns matching (PPM) [68]. Recently, Marteau [60] offered a time wrap edit distance (TWED) method integrating with DTW, Lp-norm and ERP. TWED addresses

2469

the similarity between two time series sequences is measured as the minimum cost sequence of edit operations. So far, these edit distance extension approaches mainly pay attention to temporal dimension, but not to the multi-dimensional space. To support electrical stimulation space and temporal EEP elicited space simultaneously, in this study, we concentrate on introducing a new distance function through referring ED and TWED metrics to figure out the similarity between two cases. Moreover, we intend to introduce the bias parameter to reflect the different influences from stationary electrical stimulation values and time series EEP elicited values for new predictive result, so the proposed distance is called biased time warp distance (BTWD). Finally, we also attempt to provide empirical evidence on the performance of BTWD for EEP forecasting, and the experimental data are collected to compare the predictive performances with other CTSP methods using traditional distances. 3. Case-based time series prediction using biased time wrap distance 3.1. Framework of CTSP using BTWD for EEP prediction The framework of CTSP process using BTWD metric is shown in Fig. 2, where initial cases collected from EEP electrophysiological recording data is constructed, containing several stimulation values and EEP values. Accordingly, there are two types of differences in case comparison problem, that is, stimulation set difference (Diff I) and EEP sequence difference (Diff II). Diff I means the difference of electrical stimulation sets from two comparison cases, and Diff II is the difference between two temporal EEP elicited sequences of cases. Thus, the general BTWD value between two historical cases is composed of Diff I and Diff II. Meanwhile, the bias parameter is added to emphasize the different influences of Diff I and Diff II to the numerical computation of BTWD, whose value will change over time. So a time-dependent bias configuration should be also considered in this paper. Finally, the predictive EEP values for target stimulus input can be generated by similarity-related summation on the foundation of k similar cases.

Fig. 2. Framework of CTSP using BTWD metric for EEP prediction, which consists of three major elements, i.e., case representation, case retrieval and case reuse.

2470

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

3.2. Case retrieval using biased time warp distance 3.2.1. Mechanism of biased time warp distance The heart of CTSP is the similar case searching which guides future prediction and problem solving. The concept of similarity between two cases explained in this way [14]: the less difference between two cases, the more similar they are. In this research, we present the BTWD metric to embody the difference of two EEP ˜ experimental cases. Given two EEP experiment cases A = (A˙ ⊕ A) ˜ which contain stimulation trigger parts, i.e., A˙ and and B = (B˙ ⊕ B), ˜ and B. ˙ B˙ ∈ S ˜ Note that, A, ˙ and EEP elicited response parts, i.e., A B, and S ⊂ Rd with d ≥ 1 embed the multidimensional stimulus vari˜ B˜ ∈ S  × T and S , T ⊂ R embed the temporal EEP elicited able, while A, variable. Then the BTWD value between A and B expressed as d(A, B), can be calculated as follows: ˙ B) ˜ B) ˙ + ˛(t) × diff  (A, ˜ d(A, B) = [1 − ˛(t)] × diff  (A,

(1)

˙ B) ˜ B) ˙ and diff  (A, ˜ denote the Diff where diff  (A,

I and Diff II between A and B, and ˛(t) is a bias configuration function to reflect the different influences of Diff I and Diff II for computational result of d(A, B). To calculate the Diff I, the point-to-point distance function is employed as follows:

diff







˙ B˙ = A,

 m 

1/ı wi × |a˙ i − b˙ i |

ı

(2)

i=1

where ı is a distance adapted coefficient, for ı = 2 we have the Euclidian one, while ı = 1 is the Manhattan distance and ı = ∞ is the Chebychev distance, i.e., Max|a˙ i − b˙ i | [39,69]. a˙ i and b˙ i represent the values of ith stimulus feature of A˙ and B˙ respectively, and wi is the associated weight to this feature. For EEP sequence difference denoted as Diff II, we intend to refer the principle of TWED approach for a couple of reasons [60]: firstly TWED expresses a superior performance in dealing with local time shifting, secondly it satisfies the triangle inequality, an arithmetic property which might be valuable in computing Diff II, finally the effectiveness of TWED has been empirically proved in [60,70,71]. More detailed descriptions on TWED could be obtained ˜ B) ˜ calculation, three from [60,71]. To solve the problem of diff  (A, ˜ delete B, ˜ and match, are introduced, and operations, i.e., delete A, ˜ B) ˜ is measured as the minimum cost sequence of editing diff  (A, ˜ into B, ˜ thus cost operations needed to transform one time series A ˜ B). ˜ As function  and cost minimum operation is added into diff  (A, ˜ B˜ ∈ S  × T and S , T ⊂ R, A ˜ and B˜ can be expressed in the following A, way. ˜ = {˜aj }j=n = {(avj , tj )}j=n , A j=1 j=1

j=n j=n B˜ = {b˜ j }j=1 = {(bvj , tj )}j=1

to facilitate the calculation, we also use ED metric to construct the cost function , instead of Lp-norm metric used in TWED, and then the mathematic formula of  can be expressed as follow:

(˜aj , b˜ j , ) =

 ⎧ ⎪ (˜ a → ) = (avj − avj−1 )2 + (tj − tj−1 )2 ⎪ j ⎪ ⎪ ⎨  (˜a → b˜ ) =

)2

(5)

where  is a non-negative constant penalty parameter, used to be proportional to the (tj − tj−1 ) measure during the edit procedure. 3.2.2. Time-dependent bias configuration function According to previous studies, the movements of EEPs are not random, rather they behave in a highly non-linear and dynamic manner [7,9,11]. With further observation, we can see that the movements of EEP elicited curves are comparatively similar at the initial stage with specific stimulation setting, e.g., different pulse amplitudes but fixed pulse duration or different pulse durations but fixed pulse amplitude. As the elapse of time, the dissimilarities among EEP curves expand increasingly, as shown in Fig. 3. This means that the Diff II plays an increasing role over time in constructing the time wrapping distance. Conversely, the influence of Diff I begins to decrease as time passed. To illustrate the time-varying influences of Diff I and Diff II, we develop a timedependent bias configuration function to determine the value of bias parameter, and the Gaussian function [39] is utilized to build the mathematical formula, expressed in the following way.



˛(t) =

exp(−(t − p)2 + ln(ω))

t≤p

ω

t>p

p ≤ n ∈ N+

(6)

where  is a parameter to control the ascending rate of ˛(t) and  > 0. p is a flexure time point, after that, the trend of ˛(t) movement will level off, and the addition of natural logarithm ln(ω) also means the value of bias for Diff II will continue to increase before reaching a certain upper bound value ω ∈ (0, 1), and the trend of bias value for Diff I is just the opposite. Fig. 4 illustrates the ˛(t) curves for Diff I and Diff II, when p = 50 ms,  = 0.01,  = 0.04,  = 0.002 and  = 0. Due to the introduction of bias function, d(A, B) is also a timedependent function, and its value at time point tj can be expressed as: ˙ B) ˜ tj , B˜ tj ) ˙ + ˛(tj ) × diff  (A d(A, B, tj ) = [1 − ˛(tj )] × diff  (A, ˙ B) ˙ + ˛(tj ) = [1 − ˛(tj )] × diff  (A,

(3)

˜ that is, the EEP value avj where a˜ j is the jth value-time pair of A, elicited at jth time point tj in case A, and there are similar definitions for b˜ j and bvj in case B. n is the total number of time series value.

2

(av − bv ) + (t − t

j j j j j j−1 ⎪ ⎪  ⎪ ⎪ ⎩ (b˜ → ) = (bv − bv )2 + (t − t )2 j j j−1 j j−1

× min

˜ and B˜ are same. In EEP biological experiments, the lengths of A ˜ and B˜ with the same length n can Hence, the difference between A be calculated by the following formula.

 ⎧  (A ⎪ ˜ j−1 , B˜ j ) + (avj − avj−1 )2 + (tj − tj−1 )2 d ⎪ ⎪ ⎪ ⎨  ˜ j−1 , B˜ j−1 ) + d (A

2

(av − bv ) + (t − t

)2

j j j j−1 ⎪ ⎪  ⎪ ⎪ ⎩ d (A˜ j , B˜ j−1 ) + (bv − bv )2 + (t − t )2 j j−1 j j−1

(7)



˜ B) ˜ n , B˜ n ) ˜ = diff  (A diff (A,

= min

⎧  ˜ n−1 ˜ n ˜ diff (A , B ) + (˜an → ) delete A ⎪ ⎨ ⎪ ⎩

˜ n−1 , B˜ n−1 ) + (˜an → b˜ n ) match diff  (A

˜ n , B˜ n−1 ) + (b˜ n diff  (A

→ )

(4)

delete B˜

In formula (4),  is the empty time series set. a˜ j → , a˜ j → b˜ j and ˜ match, and delete B. ˜ The b˜ j →  represent the operations of delete A, ˜ B) ˜ 0 , B˜ 0 ) = 0. Moreover, ˜ is initialized by diff  (A recursion of diff  (A,

3.2.3. Similarity measurement based on biased time warp distance After the value of d(A, B, tj ) has been computed, the similarity between case A and case B can be determined subsequently. Suppose that sim(A, B, tj ) is the similarity between two cases at time point tj , then the formulation of sim(A, B, tj ) can be expressed by the following formula. sim(A, B, tj ) =

1 1 + d(A, B, tj )

(8)

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

2471

Fig. 3. (a) EEPs elicited by stimuli with different pulse amplitudes but fixed pulse duration (0.5 ms). (b) EEPs elicited by stimuli with different pulse durations but fixed pulse amplitude (100 ␮A).

1 λ=0

ω λ=0.002

Bias value

λ=0.002

λ=0.01

λ=0.004

Diff -II when p=50ms λ=0.004 Diff -II when p=50ms λ=0.002 Diff -II when p=50ms λ=0.01 Diff -I when p=50ms λ=0.004 Diff -I when p=50ms λ=0.002 Diff -I when p=50ms λ=0.01 Equal bias λ=0

λ=0.004

λ=0.01

p=50 0

10

20

30

40

50

60

Time point (ms)

Fig. 4. ˛(t) curves for Diff I and Diff II, when p = 50 ms,  = 0.01,  = 0.004,  = 0.002 and  = 0.

For EEP prediction, if case A and case B are defined as the target case and training case, then sim(A, B, tj ) also represents the similarity of case B for target case A at time point tj , which can be abbreviated as sim(B, tj ). The mechanism of similarity measure using BTWD is illustrated in Fig. 5. 3.3. Case reuse for electrical evoked potentials prediction Similarity-related summation is employed to perform the case reuse process on the foundation of k similar cases. After the similarity of each training case in the case base has been calculated, a set of nearest neighbors can be retrieved [14]. The predicted EEP

value is computed by multiplied with a kind of similarity-related adaptation coefficient (AC) and then summed. The value of AC is directly proportional to the similarity between the old and target cases, and the sum of all ACs must be 1. The corresponding mathematical formula of AC under k-NN principle is calculated at first in the following way.

AC(i) tj =

sim(case(i), tj )

k

i=1

(9)

sim(case(i), tj )

where AC(i)|tj and sim(case(i), tj ) are AC value and similarity of case case(i) at time point tj .

Fig. 5. Mechanism of similarity measure using BTWD between target case A and training case B.

2472

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

Subsequently, the predicted EEP value at time point tj , i.e., EEP prediction|tj , can be computed as:

EEP prediction tj =

i=k 



(AC(i) tj ) × (EEP case(i) tj )

(10)

i=1

where EEP case(i)|tj is the actual EEP elicited value of case(i) at time point tj . 4. Empirical design 4.1. Objective and comparative methods Objective of this empirical research is to investigate whether or not the proposed CTSP using BTWD metric can achieve higher EEP predictive performance compare with other CTSP methods using traditional distances. Two chief ways of distance computation in similarity measure of CTSP are point-to-point distance metrics and time series distance metrics. Thus, this study makes two empirical comparisons, i.e., comparison of predictive accuracy between predictors using BTWD and point-to-point distances, and analogous comparison between predictors using BTWD and time series distances. For the former comparison, classical point-to-point distance metrics such as ED and MD are adopted, and for the latter, DTW and TWED metrics are employed to perform the empirical comparison. Additionally, we utilize the distance name directly to represent the corresponding EEP predictor just for convenience. Hence, there are total five comparative EEP prediction methods in this empirical comparison, that is, BTWD, ED, MD, DTW and TWED. Meanwhile, we would like to investigate whether or not there is diversity in computational complexity between BTWD and other time series distance metrics including DTW, TWED, ERP and SEA, because the computational complexity is another important criterion for evaluating time series distance. 4.2. Data and variable As we described above, each training case derived from EEP biological experiment involves trigger part and EEP response part, and we choose nine electrical stimulus features and EEP-time point pairs to express two parts respectively, as listed in Table 1. To guarantee the feasibility of this empirical comparison, totally 96 cases are collected, and each of which involves 500 time points starting from the earliest time point 0.1 ms with 0.1 ms time interval, and we use MATLAB toolkits to implement the proposed algorithm for carrying out the comparisons. 4.3. Parameter optimization of proposed method According to the Formula (6), (7) and (9), there are five parameters should be optimized in the process of BTWD computing, that is, flexure point p, upper bound value ω, constant value , constant

penalty , and nearest neighbor number k. Among them, p and  determine the shape of ˛(t) curve, while ω,  and k impact the efficiency of predictive performance. In this empirical comparison, 50 ms is the upper bound of selected time point, thereby it is a logical choice to use 50 ms as the value of p. Meanwhile, as indicated in Fig. 5, the reasonable range of  value is [0.002, 0.01], here the value of  is 0.004, and by reference to previous studies [14,28,39,41], we set k = 7. For ω and , it is not known beforehand which values of ω and  are the best for one problem. This study prefers a gridsearch on (ω, ) using leave-one-out cross-validation for model parameter selection. Each time one distinct sample is taken out to act as target case, while the rest consist of case base. The predictive EEP values generated on the foundation of case base are compared with its true EEP values from target case to determine the crossvalidation accuracy rate (CVAR). Let the original values of ω and  are {0.5, 0.55, . . ., 0.9} and {0.3, 0.35, . . ., 0.75} empirically and 500 time points are selected, then the grid-search is performed respectively in training set containing 66 experiment cases and test set containing 30 experiment cases to figure out the optimal solution. The CVAR of each (ω, ) option is calculated as follows: 1 CVAR = n

  n 500   1 − ((|ak,t − pk,t |)/ak,t )  500

(11)

k=1 t=1

for traning set n = 66 for test set n = 30 where ak,t and pk,t represent the actual and predictive EEP value at tth time point of kth cross-validation, and n is the number of cases. If (ω, ) option lead to a maximal cross-validation rate estimated for the training set and test set, then it will be considered as the best solution. We selected six candidate (ω, ) pairs in order with highest cross-validation rates in training set and test set respectively, as shown in Fig. 6, and it is easy to see that (0.8, 0.55) and (0.85, 0.5) have higher cross-validation rates both in training set and test set, and we choose (0.85, 0.5) as the final solution in this example. 4.4. Validation technique and evaluation criteria In this empirical example, we use the rabbit EEP experimental case with the most common values of electrical simulation as the target case, illustrated in Table 2. Its corresponding F1–F9 stimulation values are considered as the target stimulus parameters, and total 500 EEP temporal values of F10 are compared with the predictive results produced by comparative predictors on basis of the test set consists thirty cases. We define point predictive accuracy (PPA) and segment predictive accuracy (SPA) as evaluation criteria to evaluate the predictive performance. However, the conventional absolute percentage arithmetic of PPA ignores the evolution of the time series EEP value, so we will add a direction evaluation factor into PPA algorithm, besides single-point absolute percentage. In return, the mathematic expression of new PPA can be described

Table 1 The definition of features of training case. Category

Variable

Feature

Existing value

Value type

Trigger part

F1 F2 F3 F4 F5 F6 F7 F8 F9 F10

Current intensity (␮A) Duration (ms) Electrical frequency (Hz) Stimulus object Current polarity Charge density (␮C/cm2 ) Electrode type Electrode area (mm2 ) Stimulus site (EEPs value (␮V), time point (ms))

90, 250, 320, 380, . . . 0.3, 0.6, 1.2, 1.5, . . . 0.8, 1.3, 1.5, . . . Rabbit, Mice, . . . Cathodic, Anodic 30, 150, 200, 220, . . . Smaller, bigger 0.01, 0.03, 0.05, . . . Inside the pia mater, On the pia mater, On the dura mater (1.414, 0.1), (4.56, 10), (7.68, 25.5), . . ., (6.12, 50)

Numeric Numeric Numeric String String Numeric String Numeric or interval String (Numeric–numeric)

Response part

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

0.7

Cross-validation rates in training set Cross-validation rates in test set

0.65

2473

46.19 (test set) ω = 0.8 μ = 0.55

0.6 0.55 μ values

55.14 (test set) ω = 0.85 μ = 0.5

88.88 (training set) ω = 0.8 μ = 0.55

0.5 0.45

86.68 (training set) ω = 0.85 μ = 0.5

0.4 0.35 0.3 0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

ω values Fig. 6. Twelve (ω, ) pairs with highest cross-validation rates, among them, (0.8, 0.55) and (0.85, 0.5) have higher cross-validation rates both in training set and test set. Table 2 Feature values of target case. Electrical stimulation values Current intensity F1 Electrical frequency F3 Current polarity F5 F7 Electrode type F9 Stimulus site

320 ␮A 0.8 Hz Cathodi Smaller Inside the pia mater

F2 F4 F6 F8

Duration (ms) Stimulus object Charge density Electrode area

1.2 ms Rabbit 150 ␮C/cm2 0.05 mm2

EEP elicited values from 0.1 ms to 50 ms

10 9

EEPs (μV)

8 7 6 5 4 3 2 0.1

5

10

15

20

25

30

35

40

45

50

Time points (ms)

in the following formula.

 PPAi = 1 −

 +

 

1 + ϕi 2

 

1 + ϕi

×

2

defined as follows:



|avi−1 − pvi−1 | |avi − pvi | × + avi−1 avi

|avi+1 − pvi+1 | avi+1

ϕi =



1 < i < 500 i ∈ N + (12)

ϕi =

1

(ai − ai−1 )(pi − pi−1 ) < 0

0

others

1

(ai+1 − ai )(pi+1 − pi ) < 0

0

others

 =0 1 < i < 500 i ∈ N + and ϕ0 = ϕ500

(14)

And the PPA calculation is initialized as follows:

 PPA1 = 1 −

|av1 − pv1 | + av1

 PPA500 = 1−

 1 + ϕ500

2





×

1 + ϕ1 2

 ×

|av2 − pv2 | av2



After the definition of PPA has been determined, the formula for SPA criterion can be expressed as follows:



|av499 − pv499 | |av500 − pv500 | + av499 av500 (13)

where avi and pvi is the actual EEP value and predictive EEP value at ith time point, PPAi is the corresponding PPA value at ith time point. ϕi and ϕi are lower error direction coefficient and upper error direction coefficient for predictive value pvi , and their values are

j SPAi→j =

i

PPAp

j−i+1

1 ≤ i ≤ j ≤ 500 i, j ∈ N+

(15)

where SPAi→j is the predictive accuracy of time series segment i → j, which demonstrates the average PPA value from ith time point to jth time point, and j − i + 1 is the length of segment i → j. Moreover, to avoid the prejudices of one time hold-out method, 30 times hold-out method [11,12,14,71] is employed in test set to create 30 hold-out testing data to implement the comparison.

2474

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

Considering the computational burden, we only pick out 50 time points from 1 ms to 50 ms with 1 ms interval, i.e., ti = {1 ms, 2 ms, . . ., 50 ms}. For each hold-out time, one case from test set is randomly selected as target case and other cases compose into the source cases. Thus, 30 × 50 EEP predictive results will be achieved for each predictor. Furthermore, the Theil inequality coefficient (TIC) [11,72,73], a statistical method, is employed to compare the results of two comparative predictors and to validate the predictors. Unlike the simple sum of squared errors, the principal advantage of the TIC is that it varies between 0 and 1, where values closer to zero indicate a better model validity [74]. The definition of TIC is displayed as follows:



TIC =



50 (avi i=1

50 av2 i=1 i

+

− pvi )2



(16)

50 pv2i i=1

5. Empirical comparison and discussion 5.1. Comparison of predictive accuracy on PPA In this section, the predictive abilities of the distance-based CTSP predictors are compared. The actual EEP values at 0.1 ms, 0.2 ms, . . ., 50 ms time points of target case and the corresponding predictive EEP values generated by BTWD, ED, MD, DTW and TWED are depicted in Figs. 7 and 8, so there are total 500 × 5 predictive values in one time hold-out validation. Then, in terms of the predictive results, we will be able to calculate PPA to examine the superiority of BTWD compare with other comparative approaches. Table 3 shows PPA values of each comparative method at 0.1 ms, 1 ms, 5 ms, 10 ms, 20 ms, 30 ms, 50 ms time points, along with their mean and standard deviation (SD) statistical indices. From Table 3, we can find BTWD achieves higher predictive accuracy than ED, MD, DTW, and TWED by 0.1086, 0.1217, 0.0642 and 0.043 respectively on mean index, and the PPA behavior of BTWD is relatively steadier than others with lower SD value by −0.0597, −0.0972, −0.054 and −0.0794.

Table 4 SPA values of comparative methods in six time series segments. Segment

Comparative prediction methods BTWD

ED

MD

DTW

TWED

[0.1 ms, 1 ms] [1 ms, 5 ms] [5 ms, 10 ms] [10 ms, 20 ms] [20 ms, 30 ms] [30 ms, 50 ms]

0.8461 0.8261 0.8367 0.8313 0.8979 0.8478

0.8567 0.8248 0.8341 0.7168 0.7032 0.6716

0.8365 0.8255 0.8018 0.6877 0.7044 0.6807

0.7958 0.7414 0.7457 0.8203 0.8478 0.8311

0.8006 0.7651 0.7503 0.8504 0.8897 0.8404

Mean SD

0.8477 0.0260

0.7679 0.0795

0.7561 0.0727

0.7970 0.0447

0.8161 0.0536

[0.1 ms, 500 ms] into six segments, namely, [0.1 ms, 1 ms], [1 ms, 5 ms], [5 ms, 10 ms], [10 ms, 20 ms], [20 ms, 30 ms] and [30 ms, 50 ms], and SPA value can be figured out in each temporal segment. Table 4 lists the SPA values of comparative methods in six segments. In former 0.1 ms to 10 ms time series, BTWD outperforms DTW and TWED performances by (0.0503, 0.0455), (0.0847, 0.061) and (0.091, 0.0864), but marginally outperforms ED and MD performances by (−0.0106, 0.0096), (0.0013, 0.0006) and (0.0026, 0.0349) respectively in [0.1 ms, 1 ms], [1 ms, 5 ms] and [5 ms, 10 ms] segments. While in later 10–50 ms time series, it seems to be the opposite, because BTWD marginally outperforms DTW and TWED performances by (0.011, −0.0191), (0.0501, 0.0082) and (0.0167, 0.0074), and outperforms ED and MD performances by (0.1145, 0.1436), (0.1947, 0.1935) and (0.1762, 0.1671) in [10 ms, 20 ms], [20 ms, 30 ms] and [30 ms, 50 ms] segments. This phenomenon provides some evidences on the assumption that the effectiveness of various distance metrics for predictive results in different temporal periods is different. Overall, the BTWD metric yields to outperform other distance metrics in the process of EEP prediction throughout [0.1 ms, 50 ms] period, which outperforms other four predictive methods by 0.0798, 0.0916, 0.0507 and 0.0316 on SPA mean index, as shown in Table 4. 5.3. Comparison of predictive accuracy on TIC

5.2. Comparison of predictive accuracy on SPA As shown in Figs. 7 and 8, the path of EEP elicited curve produced by BTWD is approximate to the actual EEP curve of target case. On the other hand, there are big differences between actual results and predictive results from ED and MD predictors in latter time segments. Conversely, the differences between actual results and predictive results from DTW and TWED are relatively big in early time segments. We can also come to the same conclusion from the observation of comparison on SPA index among five predictors. In this empirical comparison, we divided the time series

Table 3 PPA values of comparative methods at 0.1 ms, 1 ms, 5 ms, 10 ms, 20 ms, 30 ms, 50 ms time point. Time point

Comparative prediction methods BTWD

ED

MD

DTW

TWED

0.1 ms 1 ms 5 ms 10 ms 20 ms 30 ms 50 ms

0.8834 0.8681 0.9561 0.7931 0.9419 0.8764 0.8735

0.7008 0.8897 0.8248 0.5734 0.7341 0.8341 0.8753

0.7042 0.8707 0.8699 0.4477 0.8026 0.7926 0.8528

0.7008 0.8897 0.7458 0.6807 0.9440 0.8960 0.8861

0.9375 1.0000 0.7603 0.5973 0.8916 0.8151 0.8897

Mean SD

0.8846 0.0536

0.7760 0.1133

0.7629 0.1508

0.8204 0.1076

0.8416 0.133

The TIC movements produced by BTWD, ED, MD, DTW and TWED are displayed in Fig. 9, from which BTWD achieves the best performance among other predictors, because the mean index of TIC on 30 hold-out data created by BTWD, ED, MD, DTW and TWED are 0.1645, 0.2096, 0.2101, 0.1804 and 0.1898, respectively. It indicates that BTWD outperforms other four comparative predictors on TIC by 0.0451, 0.0456, 0.0159 and 0.0253. Furthermore, only one TIC value of BTWD is higher than 0.3, as TIC value below 0.3 means a good agreement with actual data [73,74], while some TIC values of ED and MD, DTW and TWED are over 0.3. It confirms that the EEP prediction using BTWD predictor adequately predicts the activity of the eliciting EEPs under these experimental conditions. On the other hand, it needs to notice that the SDs of TIC for five predictors are 0.0745, 0.0802, 0.0961, 0.0660 and 0.0618, so BTWD is outperformed by time series distance metrics, which demonstrates that we should adopt more advanced optimization algorithms for p, ω, ,  and k to improve the stability of BTWD performance. 5.4. Comparison of computational complexity between BTWD and time series distances The computational complexity is another important criterion for evaluating time series distance method [59,61], especially when the dataset is large with more features and more cases [19]. It has been established in [50,52,53,58–60] that the complexities of DTW, TWED, ERP and SEA are O(n ∗ m), O(n ∗ m),

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

2475

11

Actual EEPs

BTWD based prediction

ED based prediction

MD based prediction

10 9

EEPs (μV)

8 7 6 5 4 3 2 1

0

5

10

15

20

25

30

35

40

45

50

Time points (ms) Fig. 7. Actual EEP values and EEP predictive values of BTWD, ED and MD.

11

Actual EEPs

BTWD based prediction

DTW based prediction

TWED for prediction

10 9

EEPs (μV)

8 7 6 5 4 3 2 1

0

5

10

15

20

25

30

35

40

45

50

Time points (ms) Fig. 8. Actual EEP values and EEP predictive values of BTWD, DTW and TWED.

O(n ∗ m + nlog2 (n) + mlog2 (m)) and O(2(n2 + m2 ) + 2(n + m) respectively, where n and m are the lengths of two time series being matched. In basis of the mechanisms of ED and TWED, the computational complexity of our proposed BTWD is O(n ∗ m + n + m) for (˜aj → b˜ j ) match operation, and the costs of (˜aj → ) and

BTWD

ED

(b˜ j → ) operations can be tabulated to speed up the calculation leading to O(2n + 2m) complexity. For EEP prediction, the length of each EEP time series sequence is equal, i.e., n = m, then the complexities of BTWD, DTW, TWED, ERP and SEA are O(n2 + 2n), O(4n), O(n2 ), O(n2 ), O(n2 + 2nlog2 (n)) and O(4n2 + 4n). The relevant mathematic

MD

DTW

TWED

Theil inequality coefficient (TIC)

0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0

1

2

3

4

5

6

7

8

9

10

11 12

13

14 15 16 17 Hold-out data

18 19

20

21 22

Fig. 9. TICs of five comparative predictors on 30 hold-out data.

23

24

25

26

27

28 29

30

2476

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477

Computational complexities

12000 10000 8000

BTWD for match operation BTWD for delete & insert operation DTW & TWED ERP SEA

6000 4000 2000 0 10

15

20

25 30 35 Time series length

40

45

50

Fig. 10. Computational complexities comparison between BTWD and other time series distances.

graphs for n = 10, 11, . . ., 50 are depicted in Fig. 10, from which it shows that BTWD is marginally outperformed by DTW and TWED, and significantly outperforms ERP and SEA for from the aspect of computational complexity.

authors are grateful to the editors and the anonymous reviewers for their insightful comments and suggestions.

References 6. Conclusion This study constructed a case based time series prediction method (CTSP) by using new distance metric, which demonstrated good predictive performance in EEP prediction of optic nerve visual prostheses research. As EEP experimental case contains the stationary electrical stimulation values and time-varying EEP elicited values, we proposed a biased time wrapping distance (BTWD) metric in similarity measure of case retrieval process, which was composed of stimulation set difference (Diff I) and EEP sequence difference (Diff II). Meanwhile, the time-independent bias configuration function was added into BTWD function to illustrate the different influences of Diff I and Diff II at various time points for predictive results. After that, the adaptation coefficient (AC) of each similar case was calculated, and similarity-related adaptation coefficient summation was employed to yield the predictive EEP values in principle of k nearest neighbors. Finally, we gave predictive performance comparison to investigate the superiority of EEP predictor using BTWD metric. From the results of empirical comparison, we can find that our proposed method produces superior performance in EEP prediction from the perspective of predictive accuracy and computational complexity. This research also has some limitations. Conclusions drawn out are on the foundation of the experiment. Further discussions should be considered under other validation techniques, e.g., the resubstitution method, McNemar test and t-statistics test. In addition, more EEP experimental data should be collected, and the global parameter optimization and data weighting methods should be researched to improve the efficiency of BTWD metric. Finally, besides in biomedical field, we believe that the proposed method is suitable for TSP in other practical applications areas, e.g., stock price or other real-world time-series problems, where BTWD metric could be obtained by figuring out the input parameter set difference (Diff I) and output time sequence difference (Diff II). So the applicability of BTWD to data from other time series domains should be investigated in future.

Acknowledgements This research is supported by the National Basic Research Program of China (2011CB707503 and 2011CB013305). China Postdoctoral Science Foundation funded project (2012M510112). The

[1] K. Vladimir, I. Mirjana, B. Zoran, Case-based curve behaviour prediction, Software: Practice and Experience 39 (2009) 81–103. [2] P.C. Chang, C.H. Liu, J.L. Lin, C.Y. Fan, S.P.N. Celeste, A neural network with a case based dynamic window for stock trading prediction, Expert Systems with Applications 36 (3) (2009) 6889–6898. [3] P.C. Chang, J.C. Hsieh, T.W. Liao, Evolving fuzzy rules for due-date assignment problem in semiconductor manufacturing factory, Journal of Intelligent Manufacturing 16 (5) (2005) 549–557. [4] P.C. Chang, T.W. Liao, Combing SOM and fuzzy rule base for flow time prediction in semiconductor manufacturing factory, Applied Soft Computing 6 (2) (2006) 198–206. [5] P.C. Chang, Y.W. Wang, Fuzzy Delphi and back-propagation model for sales forecasting in PCB industry, Expert Systems with Applications 30 (4) (2006) 715–726. [6] S. Cost, S. Salzberg, A weighted nearest neighbor algorithm for learning with symbolic features, Machine Learning 10 (1993) 57–78. [7] P.J. Cao, K.J. Wu, M.J. Sun, X.Y. Chai, Q.S. Ren, Evoked cortical potential and optic nerve response after direct electrical stimulation of the optic nerve in rabbits, in: Proceedings of IEEE/ICME International Conference on Complex Medical Engineering, Beijing, 2007, pp. 1310–1313. [8] A.Y. Chow, M.T. Pardue, V.Y. Chow, G.A. Peyman, Implantation of silicon chip microphotodiode arrays into the cat subretinal space, Neural Systems and Rehabilitation Engineering 9 (1) (2007) 86–95. [9] Q.S. Ren, X.Y. Chai, K.J. Wu, C.Q. Zhou, Development of C-Sight visual prosthesis based on optical nerve stimulation with penetrating electrode array, Investigative Ophthalmology & Visual Science 48 (5) (2007) 661–670. [10] J.H. Niu, Y.F. Liu, Q.S. Ren, Y. Zhou, Y. Zhou, S. Niu, Vision implants: an electrical device will bring light to the blind, Science in China Series F: Information Sciences 51 (1) (2008) 101–110. [11] J. Qi, J. Hu, Y.H. Peng, Q.S. Ren, W.M. Wang, Z.F. Zhan, Electrical evoked potentials prediction model in visual prostheses based on support vector regression with multiple weights, Applied Soft Computing 11 (8) (2011) 5230–5242. [12] J. Qi, J. Hu, Y.H. Peng, W.W. Wang, Z.F. Zhan, Integration of similarity measurement and dynamic SVM for electrically evoked potentials prediction in visual prostheses research, Expert Systems with Applications 38 (5) (2011) 5044–5060. [13] M. Stefania, P. Luigi, Accounting for the temporal dimension in case-based retrieval: a framework for medical application, Computational Intelligence 22 (3) (2006) 208–222. [14] H. Li, J. Sun, Gaussian case-based reasoning for business failure prediction with empirical data in China, Information Sciences 179 (2009) 89–108. [15] W. Beaver, Financial ratios as predictors of failure, Journal of Accounting Research 4 (1966) 71–111. [16] E.I. Altman, Financial ratios discriminant analysis and the prediction of corporate bankruptcy, Journal of Finance 23 (4) (1968) 589–609. [17] D. Martin, Early warning of bank failure: a Logit regression approach, Journal of Banking & Finance 1 (1977) 249–276. [18] P.C. Chang, C.Y. Fan, J.J. Lin, Monthly electricity demand forecasting based on a weighted evolving fuzzy neural network approach, International Journal of Electrical Power & Energy Systems 33 (1) (2011) 17–27. [19] C.J.C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery 2 (2) (1998) 121–167. [20] S.S. Panda, D. Chakraborty, S.K. Pal, Flank wear prediction in drilling using back propagation neural network and radial basis function network, Applied Soft Computing 8 (2) (2008) 858–871.

J. Qi et al. / Applied Soft Computing 13 (2013) 2467–2477 [21] T. Kimoto, K. Asakawa, Stock market prediction system with modular neural network, in: IEEE International Joint Conference on Neural Network, 1990, pp. 1–6. [22] J. Faraway, C. Chatfield, Time series forecasting with neural networks: a case study, University of Bath, Research Report 1995. [23] R. Sharda, R.B. Patil, A connectionist approach to time series prediction: an empirical test, in: R.R. Trippi, E. Turban (Eds.), Neural Networks in Finance and Investing, Probus Publishing Co., Chicago, 1994, pp. 64–451. [24] W. Cheng, L. Wanger, C.H. Lin, Forecasting the 30-year US Treasury bond with a system of neural networks, Journal of Computational Intelligence in Finance 4 (1996) 6–10. [25] E. Van, J. Robert, The Application of Neural Networks in the Forecasting of Share Prices, Finance & Technology Publishing, Haymarket, VA, USA, 1997. [26] P.C. Chang, Y.W. Wang, C.H. Liu, The development of a weighted evolving fuzzy neural network for PCB sales forecasting, Expert Systems with Applications 32 (1) (2007) 86–96. [27] P.C. Chang, C.Y. Fan, J.C. Hsieh, A weighted evolving fuzzy neural network for electricity demand forecasting, in: IEEE International Joint Conference on Intelligent Information and Database Systems, 2009, pp. 330–335. [28] H. Li, J. Sun, On performance of case-based reasoning in Chinese business failure prediction from sensitivity, specificity, positive and negative values, Applied Soft Computing 11 (1) (2011) 460–467. [29] C.H. Wu, Y. Ken, T. Huang, Patent classification system using a new hybrid genetic algorithm support vector machine, Applied Soft Computing 10 (4) (2010) 1164–1177. [30] H.H. Tsai, D.W. Sun, Color image watermark extraction based on support vector machines, Information Sciences 177 (2) (2007) 550–567. [31] S.H. Huang, T.K. Wu, Integrating GA-based time-scale feature extractions with SVMs for stock index forecasting, Expert Systems with Applications 35 (4) (2008) 2080–2088. [32] M. Espinoza, J.A.K. Suykens, B. De Moor, Short term chaotic time series prediction using symmetric LS-SVM regression, in: 2005 International Symposium on Nonlinear Theory and Applications (NOLTA), Bruges, Belgium, 2005, pp. 606–609. [33] M. Awad, Y. Motai, Dynamic classification for video stream using support vector machine, Applied Soft Computing 8 (4) (2008) 1314–1325. [34] I. Aydin, M. Karakose, E. Akin, A multi-objective artificial immune algorithm for parameter optimization in support vector machine, Applied Soft Computing 11 (1) (2011) 120–129. [35] B. Jin, Y.C. Tang, Y.Q. Zhang, Support vector machines with genetic fuzzy feature transformation for biomedical data classification, Information Sciences 177 (2) (2007) 476–489. [36] T.W. Quan, X.M. Liu, Q. Liu, Weighted least squares support vector machine local region method for nonlinear time series prediction, Applied Soft Computing 10 (2) (2010) 562–566. [37] B.W. Huang, M.L. Shin, N.S. Chiu, W.Y. Hu, C. Chiu, Price information evaluation and prediction for broiler using adapted case-based reasoning approach, Expert Systems with Applications 36 (2) (2009) 1014–1019. [38] K.J. Oh, T.Y. Kim, Financial market monitoring by case-based reasoning, Expert Systems with Applications 32 (2007) 789–800. [39] J. Qi, J. Hu, Y.H. Peng, W.M. Wang, Z.F. Zhan, AGFSM: a new FSM based on adapted Gaussian membership in case retrieval model for customer-driven design, Expert Systems with Applications 38 (1) (2011) 894–905. [40] H. Li, J. Sun, Majority voting combination of multiple case-based reasoning for financial distress prediction, Expert Systems with Applications 36 (2009) 4363–4373. [41] H. Li, J. Sun, Financial distress prediction based on OR-CBR in the principle of knearest neighbors, Expert Systems with Applications 36 (3) (2009) 4363–4373. [42] J. Qi, J. Hu, Y.H. Peng, A new adaptation method based on adaptability under k-nearest neighbors for case adaptation in case-based design, Expert Systems with Applications 39 (7) (2012) 6485–6502. [43] T.W. Liao, Clustering of time series data – a survey, Pattern Recognition 38 (11) (2005) 1857–1874. [44] R. Agrawal, G. Psaila, E.L. Wimmers, M. Zait, Querying shapes of histories, in: Proceedings of the 21st International Conference on Very Large Databases, Zurich, Switzerland, 1995, pp. 502–514. [45] C. Faloutsos, M. Ranganathan, Y. Manolopoulos, Fast subsequence matching in time-series databases, in: Proceedings of the ACM SIGMOD International Conference on Management of Data, Minneapolis, MN, 1994, pp. 419–429. [46] K. Chan, W. Fu, Efficient time series matching by wavelets, in: Proceedings of the 15th IEEE International Conference on Data Engineering, 1999, pp. 117–126. [47] F. Korn, H.V. Jagadish, C. Faloutsos, Efficiently supporting ad hoc queries in large dataset of time sequences, in: Special Interest Group on Management of Data (SIGMOD’97), 1997, pp. 289–300. [48] A.I. Manjula, M.H. Morgan, T.W. Layne, A performance comparison of piecewise linear estimation methods, in: Proceedings of the 2008 Spring Simulation Multi-Conference, 2008, pp. 273–278.

2477

[49] E.J. Keogh, Exact indexing of dynamic time warping, in: Proceedings of the International Conference on Very Large Data Bases (VLDB), Hong Kong, China, 2002, pp. 406–417. [50] E.J. Keogh, S. Kasetty, On the need for time series data mining benchmarks: a survey and empirical demonstration, in: Proceedings of the International Conference on Knowledge Discovery and Data Mining, ACM SIGKDD, Edmonton, Canada, 2002, pp. 102–111. [51] E.J. Keogh, L. Wei, X. Xi, S.H. Lee, M. Vlachos, LB Keogh supports exact indexing of shapes under rotation invariance with arbitrary representations and distance measures, in: Proceedings of the International Conference on Very Large Data Bases (VLDB), Seoul, Korea, 2006, pp. 882–893. [52] S.W. Kim, S. Park, W.W. Chu, Efficient processing of similarity search under time warping in sequence databases: an index-based approach, Information Systems 29 (6) (2004) 405–420. [53] C. Ratanamahatana, E.J. Keogh, Three myths about dynamic time warping data mining, in: Proceedings of the SIAM International Data Mining Conference (SDM), Newport Beach, CA, 2005, pp. 506–510. [54] L. Wei, E.J. Keogh, H.V. Herle, A. Mafra-Neto, Atomic wedgie: efficient query filtering for streaming times series, in: Proceedings of the IEEE International Conference on Data Mining (ICDM), Houston, TX, 2005, pp. 490–497. [55] B.K. Yi, H.V. Jagadish, C. Faloutsos, Efficient retrieval of similar time sequences under time warping, in: Proceedings of the International Conference on Data Engineering (ICDE), Orlando, FL, 1998, pp. 201–208. [56] W.S. Han, J. Lee, Y.S. Moon, H. Jiang, Ranked subsequence matching in timeseries databases, in: Proceedings of the 33rd International Conference on Very Large Data Bases, Vienna, Austria, 2007 Sept, pp. 423–434. [57] W. Jiang, M. Murugesan, C. Clifton, L. Si, Similar document detection with limited information disclosure, in: Proceedings of the 24th International Conference on Data Engineering, Cancun, Mexico, 2008, pp. 735–743. [58] Y. Zhu, D. Shasha, Warping indexes with envelope transforms for query by humming, in: Proceedings of International Conference on Management of Data, June, San Diego, California, 2003, pp. 181–192. [59] B.K. Yi, C. Faloutsos, Fast time sequence indexing for arbitrary Lp norms, in: Proceedings of the International Conference on Very Large Data Bases (VLDB), Cairo, Egypt, 2000, pp. 385–394. [60] P.F. Marteau, Time warp edit distance with stiffness adjustment for time series matching, IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (2) (2009) 306–318. [61] W.K. Loh, Y.S. Moon, J. Srivastava, Distortion-free predictive streaming timeseries matching, Information Sciences 180 (2010) 1458–1476. [62] B. Boucheham, Reduced data similarity-based matching for time series patterns, Pattern Recognition Letters 31 (2010) 629–638. [63] V.I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals, Doklady Akademii Nauk SSSR 163 (4) (1966) 707–710 (in Russian, English translation in Soviet Physics Doklady 10 (8) (1965) 845–848). [64] D. Gusfield, Algorithms on Strings, Trees and Sequences, Computer Science and Computational Biology, Cambridge University, 1997. [65] J.M. Juareza, F. Guil, J. Palma, R. Marin, Temporal similarity by measuring possibilistic uncertainty in CBR, Fuzzy Sets and Systems 160 (2009) 214–230. [66] M. Vlachos, M. Hadjieleftheriou, D. Gunopulos, E.J. Keogh, Indexing multidimensional time-series with support for multiple distance measures, in: Proceedings of the International Conference on Knowledge Discovery and Data Mining, ACM SIGKDD, Washington, DC, 2003, pp. 216–225. [67] L. Chen, R. Ng, On the marriage of Lp-norm and edit distance, in: Proceedings of Int’l Conf. Very Large Data Bases, 2004, pp. 792–801. [68] V. Mäkinen, Using edit distance in point-pattern matching, in: Proceedings of 8th Workshop on String Processing and Information Retrieval (SPIRE’01), 2001, pp. 153–161. [69] J. Qi, J. Hu, Y.H. Peng, W.M. Wang, Z.F. Zhang, A case retrieval method combined with similarity measurement and multi-criteria decision making for concurrent design, Expert Systems with Applications 36 (7) (2009) 10357–10366. [70] P.F. Marteau, S. Gibet, Adaptive sampling of motion trajectories for discrete task-based analysis and synthesis of gesture, in: Proceedings of International Gesture Workshop, 2005, pp. 224–235. [71] L.L. Liu, W.M. Zuo, D.Y. Zhang, N.M. Li, H.Z. Zhang, Classification of wrist pulse blood flow signal using time warp edit distance, in: Proceedings of ICMB 2010, LNCS 6165, 2010, pp. 137–144. [72] H. Theil, Economic Forecasts and Policy, North-Holland, Amsterdam, 1970. [73] N. Hvala, S. Strmcnik, D. Sel, S. Milanic, B. Banko, Influence of model validation on proper selection of process models – an industrial case study, Computers & Chemical Engineering 29 (7) (2005) 1507–1522. [74] C. Huilinir, R. Romeroc, C. Munozc, et al., Dynamic modeling of partial nitrification in a rotating disk biofilm reactor: calibration, validation and simulation, Biochemical Engineering Journal 52 (1) (2010) 7–18.