Decision Support in Intelligent Maintenance-planning

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 103 (2017) 316 – 323

XIIth International Symposium «Intelligent Systems», INTELS’16, 5-7 October 2016, Moscow, Russia

Decision support in intelligent maintenance-planning systems based on contextual multi-armed bandit algorithm A.V. Savchenkoa,b*, V.R. Milovb a

National Research University Higher School of Economics, Nizhny Novgorod, Russia N. Novgorod State Technical University n.a. R.E. Alekseev, Nizhny Novgorod, Russia

b

Abstract In this paper we focus on two essential problems of maintenance decision support systems, namely, 1) detection of potential dangerous situation, and 2) classification of this situation in order to recommend an appropriate repair action. The former task is usually solved with the known statistical process control techniques. The latter problem can be reduced to the contextual multiarmed bandit problem. We propose a novel algorithm with Bayesian classification of abnormal situation and the softmax rule to explore the decision space. The dangerous situations are detected with the Shewhart control charts for the distances between the current and the normal situations. It is experimentally shown, that our algorithm is more accurate than the known contextual multi-armed methods with stochastic search strategies. © 2017 2017The TheAuthors. Authors. Published Elsevier © Published by by Elsevier B.V.B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the XIIth International Symposium «Intelligent Systems». Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems” Keywords: maintenance decision support system; pattern classification; statistical process control; contextual multi-armed bandit.

1. Introduction Improving the efficiency of maintenance decision support systems is an acute problem in monitoring of mobile network equipment, public transport, pipeline control, etc.1,2. Such operations support system typically sequentially solves two tasks: 1) detection (or prediction) of potential emergencies, and 2) classification of the discovered dangerous situations in order to provide a schedule for future repair. Our experience in development of the gas pipeline monitoring3 and NetBoss XT operations support system has demonstrated the following observation.

* Corresponding author. E-mail address: [email protected]

1877-0509 © 2017 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems” doi:10.1016/j.procs.2017.01.114

317

A.V. Savchenko and V.R. Milov / Procedia Computer Science 103 (2017) 316 – 323

Though the rules to discover emergency and recommend appropriate repair action can be provided by the specialists based on the data gathered by the monitoring subsystem, the decision-maker can miss the most effective action, especially under time pressure and when a problem has many possible solutions. That is why much attention in recent developments has been paid to automatic detection and classification of potential emergencies. Historic data about earlier faults is used for learning2. Unlike conventional classification task4,5, available data often do not include the correct action; rather the result of recommended action (positive or negative) is only available. Moreover, the failures are quite rare events, so the training data is usually very small and online learning should be applied6. Hence, an effective decision-making algorithm should use not only available historic data, but also explores the decision space. In fact, it is the special case of reinforcement learning scheme with exploration exploitation dilemma7. In this paper we propose a decision-making algorithm in intelligent maintenance-planning systems, which can be successfully used even for small-sample-size problem6. At first, the potential emergencies are detected with the known statistical process control techniques, namely, Shewhart control charts for the distances between situations 8,9. When the dangerous situation is detected, the repair action is recommended by using the empirical Bayesian classifier4. If the size of the training sample is rather small, the softmax (Boltzman) stochastic strategy is used to increase the search diversity10. The rest of the paper is organized as follows. In Section 2 we remind several methods for statistical process control and contextual multi-armed bandit problem11. In the last part of this section the complete maintenanceplanning algorithm is presented. In Section 3 experimental study is shown for synthetic data generated by the Bayesian network. Finally, concluding comments are given in Section 4. 2. Materials and Methods 2.1. Statistical process control in maintenance-planning systems Let the intelligent maintenance decisions support system periodically observes the states of Lt1 objects, e.g., segments of gas pipeline3,12. The state of the l-th segment at time t=1,2,… is described by a feature vector xl(t) of dimensionality M. As it was stated in introduction, the first task is to detect as soon as possible the segment l* and time t*, at which the state of this segment becomes potentially dangerous. In this paper we do not deal with fast destructions and predictions of robustness after natural catastrophes12. We will primarily focus on maintenance planning for rather slow degradation of the observed object, e.g., corrosion or geometrical distortions of the gas tube. In such case, the most obvious way to solve the task is to apply well-known statistical process control techniques9. If only M=1 feature is observed, then the most widely used approach is the Shewhart individual control chart 8. According ' x l (t )

to 1

this t 't

¦

t 't 1 t ' 2

method,

an

average

state

x l (t )

1

t 't

¦ xl (t ')

t 't t ' 1

and

moving

range

xl (t ') xl (t ' 1) are estimated. Here Δt>0 is a time delay, which guarantees that only non-

dangerous situations are aggregated. Finally, the statistics xl (t ) xl (t ) / 'xl (t ) for the current state xl(t) is computed. If it exceeds a certain threshold (usually, 2.66), the situation at the segment l*=l in the moment t*=t is defined as potentially dangerous. There are certain variations of the described procedure. For instance, if M>1 features are analyzed, the multivariate statistical process control is implemented13 with, e.g., the Hotelling’s T2 statistic8,14 is compared with a threshold. However, this method assumes the multivariate normal distribution of the observed feature vectors. This assumption is not usually valid. Hence, in this paper we propose a slight modification of the Shewhart chart, which does not require the data to be normally distributed. Similarly, we specify a dissimilarity measure ρ(…) between feature vectors. For example, the Eucliden metric is usually an appropriate choice. Next, we compute an average distance to the normal situation

318

A.V. Savchenko and V.R. Milov / Procedia Computer Science 103 (2017) 316 – 323 t 't

¦ U xl (t '), xl (t )

1

U l (t )

(1)

t 't t ' 1

and the moving range ' U l (t )

1

t 't

¦

t 't 1 t ' 2

U ( xl (t '), xl (t )) U ( xl (t ' 1), xl (t )) .

(2)

The current state is assumed to be dangerous if statistic

U xl (t ), xl (t ) U l (t ) / 'U l (t )

(3)

exceeds a specially chosen threshold δ0. 2.2. Decision-making in classification of dangerous situations After detection of potentially dangerous object state, it should be classified2. It is required to assign an abnormal state xl*(t) at moment t=t* to one of A (At2) possible repair actions. For simplicity let us assume that these actions are statistically independent. Let there be a training set of N tuples X N ^(xn , an , rn )` , n 1, N , where xn is the feature vector of the n-th abnormal state, an{1, …, A} is the action taken to correct the n-th abnormal state, rn{0, 1} is the reward for an action. The reward can be either success (rn 1) or failure (rn 0). The initial training sample of size N0t0 is formed before the decision support system is deployed. After decision is made and a corresponding action is taken, the efficiency of the action is evaluated, and the (N+1)-th tuple is added to the training sample. It should be noted that this task differs from a conventional classification problem3,4 where the class label of each reference instance from the training sample is given (i.e. rn 1, n 1, N ). The presence of the negative decisions whose class is unknown (rn = 0) restricts the possible solutions. In such case the multi-armed bandit algorithms with a context (description of situation) can be used11. Within the framework of exploitation-exploration dilemma7 the solution usually comes down to a successive iterative two-stage procedure. At the first, exploitation, stage an optimal decision is chosen based on available training set. For instance, the maximal utility principle can be used in order to choose an action a*4 a*

argmax r ( a , xl * (t )) ,

(4)

a{1,..., A}

where r ( a, xl * (t )) E ( r a, xl * (t )) P (r 1 a , xl * (t )) is the probability of success of action a. To estimate this probability, the nonparametric techniques from the probabilistic neural network classifier 5,15 can be used:

Pˆ ( r

1 a , x l * (t ))

¦

K ( x l * (t ), x n )

¦

K ( x l * (t ), x n )

xnX N (a)

.

(5)

xnX N (a )

Here X N (a )

K(xl*(t),xn)

is

the

^(x n , rn ) (x n , a, rn ) X N `

Rosenblatt-Parzen

X N ( a )

function,

e.g.

is the training sample for action a, X N ( a )

examples with positive reward, and X N ( a ) X N ( a )

kernel

^x n

the

^x n

Gaussian

kernel16,

( x n , a ,1) X ` is a set of

( x n , a , 0) X ` is the negative sub-sample (obviously,

X N ( a ) ). In fact, expressions (4), (5) implements the empirical Bayesian classifier 4.

319


At the second stage of any multi-armed algorithm the state space is explored to refine the strategy. For example, the known UCB (upper confidence bound) method optimistically chooses the action with the highest upper bound of an expected reward16. The confidence bound can be estimated by using the known distribution of the nonparametric Nadaraya-Watson kernel regression, which is equivalent to the probabilistic neural network (4), (5) for the binary reward. Our experiments2 demonstrated that the use of the UCB method is effective only when size N0 of the initial training sample is large enough. Hence, we will use another popular strategy, namely, the softmax (Boltzmann) rule10, which usually gives satisfactory accuracy even without prior information about a right reaction. In this case an action a* is chosen by using the distribution function S(a, xl*(t)): S ( a , xl * (t ))

exp( Pˆ ( r A

¦

exp( Pˆ ( r

1 a , x l * (t )) / T )

.

(6)

1 a ', x l * (t )) / T )

a' 1

Here T is the ‘temperature” that falls with time following the rule chosen experimentally according to the precise context. Making sufficiently accurate evaluations (5) requires large representative initial training sample XN(a), which contains examples of using each action in different conditions. However, the failures are rather rare events. As a result, the efficiency of decision (4) - (6) may be low in the initial stages. In this paper we propose a modification of estimate (4). We use a united sample that combines all possible classes2. It should be noted that examples from the negative training sample X N (a ) might be positive examples for any action a ' z a with probability 1/(A-1). Moreover, references from the positive sample X N (a ) will be negative examples for any other action a ' z a . Hence, we propose to modify expression (4) in the following way:

¦

Pˆ ( r

xnX N (a)

1 a , x l * (t ))

K ( x l * (t ), x n )

A 1 ¦ A 1 a ' 1

a 'z a

¦

¦

xnX N ( a ')

K ( x l * (t ), x n )

K ( x l * (t ), x n )

.

(7)

xnX N

It is possible to use the training samples from another segments l ' z l to further improve the estimate (7), if they are identical to the l-th segment. In so doing we expect that due to adding abnormal states to the training set the evaluation of the probability density will prove far more accurate than with conventional approach (5). It is especially the case with initial training samples of small size5. At the same time the heuristic assumption that using examples from sample X N (a ) as positive examples for action a makes, according to (7), an identical contribution to estimated conditional probabilities for all other actions. Finally, we have to make a decision about the segments, which need for repair. All segments are sorted in descendent order of statistic (3). If the cost of each repair action {1, …, A} is known, and the total repair cost is fixed, the classical knapsack algorithms17 can be used to recommend the most necessary maintenance activities. The complete maintenance-planning algorithm is shown in Table 1. 2.3. Experimental study In this section we examine two essential parts of the proposed algorithm (Table 1), namely, the detection of dangerous situations (Step 2 of the algorithm) and their classification (step 3). At first, the emergency detection (1)(3) is compared with the known statistical process control techniques, namely, the Hotelling control chart applied to the features itself (traditional approach) and the deltas (increments) of corresponding features xl(t) – xl(t-1). In the paper14 it was claimed, that the latter method is more robust to the trend, e.g., the deterioration of the monitored

320


object. Finally, we examined the method implemented in the gas tube monitoring system 3, in which the future values of individual features are predicted with the linear function, and the alarm is raised, if the predicted value for any feature exceeds a threshold. Table 1. Proposed maintenance-planning algorithm Input data: an observations {xl(t)} for each segment l, an initial training sample XN , N = N0 Output data: list of potential dangerous segments L* and recommended actions 1

Initialize L* as an empty set

2

For each segment l {1, …, L} repeat

2.1

Refine an average normal situation for the l-th segment

2.2

Compute an average distance to the normal situation (1) and the moving range (2)

2.3

If statistic (3) exceeds a threshold δ0, then add l to the set L

3

For each segment l* in the set L* repeat

3.1

Repeating the evaluation of conditional probability (7)

3.2

Repeating the calculation of probability S(a, xl*(t)) of making decision a (6) for each action a {1, …, A}

3.3

Generating a random number p [0, 1], determining decision a*(l*) according to randomized criterion and adding this action to the list A*

4

By using information about the maininenace cost, solve the knapsack problem and refine the list L*

5

For each segment l* in the set L* repeat

5.1

Asking the user for correctness r of decisions a*(l*)

5.2

Adding new tuple (xl*(t), a*(l*), r) to the historic data XN

5.3

Assign N = N+1

6

Lowering “temperature” T

7

Recommending actions A* for eliminating the abnormal situation in the segments L*

In the first simulation experiment we 1000-times generated M=2 lognormal features. The normal (non-dangerous) § ª 0.7 º ª 0.10.7 º · »,« » ¸ . To introduce deterioration, we add s small deviation [0.0001 © ¬ 0.7 ¼ ¬ 0.7 0.1¼ ¹

situations are distributed as ln N ¨ «

0.0001]T to the mean values at each time step. We simulated A=3 different equiprobable destructions with the § ª0 § ª 0.1 º ª 0.01 0.7 º · § ª 0.5 º ª 0.05 0.2 º · º ª0.005 0.8º · » ¸¸ , respectively. », « »,« » ¸ , ln N ¨ « 0.5 » , « 0.7 0.01» ¸ , and ln N ¨¨ « 0.1 0.2 0.05 0 . 2 ¼ ¬ ¼¹ ¼ ¬ ¼¹ ¼ ¬0.8 0.005 ¼ ¹ ©¬ ©¬ ©¬

distributions: ln N ¨ «

The probability of destruction at each moment is equal to 5%. The duration of the invalid state is randomly chosen from the uniform distribution [5; 11). In order to reflect the real corruption, the features of the normal state are linearly weighted with the generated destructions during the invalid state, so that the distributions of the invalid states are solely used only in the last time moment. This experiment was repeated 100 times. We show the following results in Table 2: FNR (False Negative Rate), FPR (False Positive Rate), average time (in %) to detect emergency in relation to the total time of the invalid state (100% in the case of false negative), and AUC (Area Under Curve). The latter is estimated based on the ROC (receiver operating characteristic) curve (Fig. 1). Table 2. Quality metric. Method

FNR, %

FPR, %

Detection time, %

AUC

Hotelling

4.8±4.6

5.5±1.5

25.8±6.5

0.81

Hotelling (delta)14

16.8±7.4

5.3±1.5

26.6±7.6

0.68

Prediction3

3.1±3.6

7.2±1.8

15.5±5.6

0.74

Proposed approach (1)-(3)

1.1±2.1

6.7±1.6

13.6±3.7

0.82


Here the Hotelling statistics for the increments of the features (deltas) does not work better, than the original Hotelling’s T2 statistic even for small trend in the input data. Second, the linear interpolation (prediction) of features from the real operating support system3 works quite good: FNR is 1.6%-less, than FNR for the traditional Hotelling chart, and the incorrect states are detected much faster. However, FPR of prediction is 1.7% higher, because its AUC is 0.07 less than AUC of the Hotelling T2 statistic. Finally, the proposed approach (1)-(3) is the best choice in this particular case: it is characterized by 1% FNR, it discovers the incorrect state right after it is appeared (13.6% relative detection time in average). AUC of the proposed algorithm is only 0.01 higher when compared with AUC of the Hotelling chart. However, one can note by analyzing of ROC curve (Fig. 1), that in the most important case of high true positive rate (0.95-0.99) our ROC curve is much higher than other curves, meaning that the corresponding FPR is 10-20% lower. Основной Основной Основной True positive rate

Основной Основной Основной Основной Основной Основной Основной Основной

False positive rate Hotelling

Prediction

Hotelling (delta)

Proposed (1)-(3)

Fig. 1. ROC-curve for emergency detection methods.

In the second experiment classifier (4), (6), (7) is experimentally compared with the conventional statistical classification (4), (5) and contextual multi-armed methods, namely, UCB and softmax exploration strategies for the estimation (5) of unknown density. The total size of the testing set N is equal to 100 instances. Every observation from the test set is added to the training set right after classification. It is expected that each situation is rather rare event. Hence, we varied the size of the initial set N0 from 0 to 20, i.e., it is a very small training sample. The average error rates are shown in Table 3. Table 3. Error rates (%) of the classification of dangerous situations Size of initial training set N0 Method 0

3

6

9

Statistical classification (4), (5)

66.7±0.9

61.0±5.5

54.1±5.0

42.4±6.1

UCB

64.5±1.0

61.1±5.6

52.9±5.1

41.7±6.0

Softmax

62.5±5.9

60.7±6.4

53.1±5.0

39.2±5.7

321

322

A.V. Savchenko and V.R. Milov / Procedia Computer Science 103 (2017) 316 – 323 Proposed approach (4), (6), (7)

56.6±4.4

54.5±3.2

47.1±1.4

36.4±3.2

Here the proposed algorithm is 3-10% more accurate, than the known methods for the small initial training set. However, the error rate is still large (36-56%). It is caused by the simulation procedure, in which each incorrect state is generated by weighted aggregation with the same, normal, state. Thus, to demonstrate the potential of the proposed approach, we repeated the experiment for another synthetic dataset. We simulated the dangerous situations discovered at the gas tube cross sections 2,18. The cross section in each state is represented as the ellipse with the nominal size 30 of both axes. A uniformly generated noise from the range [-2;2] is added independently to the size of each ax. In the image of the first defect the cross-section of a pipe is represented as a segment of an ellipse whose start and end angles vary in equal increments over the ranges [105°, 125°] and [40°, 60°], respectively. The second defect is created similarly, but the dent is 180° rotated. The images of the last defect are showed as a segment of an ellipse whose start and end angles vary uniformly over the ranges [170°, 190°] and [-10°, 10°], respectively. The breaking is represented as a straight line segment whose length vary uniformly over the ranges [25, 35] and inclination to a randomly chosen end of the arc of the main cross-section makes 45°. A noise-resistant algorithm used for video detection of a fork-lift truck19 was taken to extract features. This algorithm includes the sequence of the following image processing operations20: adaptive thresholding followed by contour detection, combination of the closed contours, flood filling of all contours, erosion with 5x5 circle, and Canny edge detection. Additionally, morphological opening using a disk element 56 pixels in diameter was performed, and the area of the resulting shape was computed. The following features are extracted in the resulted shape: its area, perimeter, area of morphological opening result and two coordinates of the gravity center. OpenCV library was used to implement this feature extraction method. The testing set contains 1000 images. The procedure of the testing dataset generation was repeated 100 times. The results are presented in Table 4. Table 4. Error rates (%) of the classification of situations discovered in images Method

Size of initial training set N0 0

3

6

9

Statistical classification (4), (5)

66.8±1.6

40.6±4.8

23.1±5.1

15.2±3.8

UCB

65.7±4.7

36.6±5.4

19.9±4.3

12.3±4.4

Softmax

5.8±1.2

5.4±1.2

5.3±1.3

4.8±1.3

Proposed approach (4), (6), (7)

3.4±0.5

3.1±0.6

2.9±0.5

2.7±0.5

Here we see that given a small initial training sample, the use of the softmax rule (6) to explore the decision space gives a fairly high mean effectiveness. Nevertheless, the proposed algorithm is the best choice in all the cases. The gain in accuracy is rather high: the error rate of our algorithm is 2% lower, than the error rate of conventional softmax, and 10-60% lower when compared with the known classifier (4), (5). 3. Conclusion and future work In this paper we focused on intelligent decision support systems for maintenance planning with assumption about rather slow degradation of the monitored object. We proposed an algorithm (Table 1), which solves two essential problems of such systems: detection and classification of abnormal situations. Using a simple simulation experiment, we demonstrated, that the Shewhart charts applied to the distances between an observed state and the average normal state causes rather accurate decision with small FNR (Table 2, Fig. 1). In the proposed classification method (5), (7) a binary classifier is built for each action and the training sample expands through negative examples of other actions, which lowers the rate of wrong decisions. We experimentally showed that the approach allows higher accuracy even with a small size N0 of the initial training sample (Tables 3, 4). The main direction for further research of our algorithm (Table 1) is its application to the real data gathered by the recently deployed gas tube monitoring system3. Here the primary attention should be paid to the complexity of available data and the reliability problem of decision-making in the absence of information about the correct action in historical data. Another important research direction is the modification of proposed algorithm in order to process


more complex data. For example, if the observed object is described by a sample of features in each moment (rather than a simple feature vector 13), more complex techniques to detect abnormal state can be applied21, e.g., the Kullback-Leibler minimum information discrimination principle22, or the usage of autoregression-moving-average model8,9. References 1. Huynh K.T., Barros A., and Berenguer C., Maintenance decision-making for systems operating under indirect condition monitoring: value of online information and impact of measurement uncertainty. IEEE Transactions on Reliability 2012; 61, 2: 410-25. 2. Savchenko A.V., Milov V.R., The adaptive approach to abnormal situations recognition using images from condition monitoring systems. Optical Memory and Neural Networks (Information Optics) 2016; 25, 2: 79-87 3. Milov V.R., Suslov B.A., Kryukov O.V., Intellectual management decision support in gas industry. Automation and Remote Control 2011; 72, 5: 1095-1101. 4. Theodoridis S., and Koutroumbas K., Pattern Recognition. Boston: Academic Press; 2008. 5. Savchenko A.V. Search Techniques in Intelligent Classification Systems, Switzerland: Springer International Publishing: 2016. 6. Milov A.V., Savchenko A.V., Classification of dangerous situations for small sample size problem in maintenance decision support systems, In: Proc. of AIST 2016. CCIS, in press. 7. Audibert J.Y., Munos R., Szepesvári C., Exploration–exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science 2009; 410 19:1876-1902. 8. Montgomery D.C. Introduction to Statistical Quality Control, 7th Edition. New York: John Wiley & Sons; 2012. 9. Qiu P. Introduction to Statistical Process Control, CRC Press; 2013. 10. Vermorel J., Mohri M., Multi-armed bandit algorithms and empirical evaluation. In: J. Gama et al (Eds.): ECML LNCS/LNAI Springer Berlin Heidelberg 2005; 3720: 437-448. 11. Lu T., Pál D., Pál M. Contextual multi-armed bandits. In: Proc. of the International Conference on Artificial Intelligence and Statistics, 2010. p. 485-492. 12. Bakhtizin R., Kutukov S., Nabiev R., Pavlov S., Vasiliev A., Simulation method of pipeline sections ranking by environmental hazard due to oil damage spill. In: Proc. of Intellectual Service for Oil and Gas Industry: Analysis, Solutions, Perspectives. 2000, p. 163-171. 13. Bersimis S., Psarakis S., Panaretos J., Multivariate statistical process control charts: an  overview. Qual. Reliab. Engng. Int., 2007; 23: 517– 43. 14. Borisenko B.B. Modified Hotellings’s chart excluding trendinfluence and its application for digital watermarks detection. Applied Discrete Mathematics 2010; 2, 8: 42-58. (in Russian). 15. Specht D.F. Probabilistic neural networks. Neural Networks 1990 3, p.109–118. 16. MayB.C., Korda N., Lee A., Leslie D.S.’ Optimistic Bayesian sampling in contextual-bandit problems. The Journal of Machine Learning Research 2012; 13, 1: 2069-2106. 17. Kellerer H., Pferschy U., Pisinger D., Knapsack Problems. Springer; 2004. 18. Diagnostics of gas tubes, http://gazdiagnoz.narod.ru/gaz013.html (last access June 30, 2016). 19. Chernousov V.O., Savchenko A.V. A fast mathematical morphological algorithm of video-based moving forklift truck detection in noisy environment, In: D.I. Ignatov et al (Eds.): AIST 2014. CCIS. Springer International Publishing Switzerland 2014 436, p. 57–65. 20. Sonka M., Hlavac V., Boyle R., Image processing, Analysis, and machine vision, 4th edn., Cengage Learning 2014. 21. Zhitlukhin M.V., Muravlev A.A., Shiryaev A.N. On confidence intervals for Brownian motion changepoint times. Russian Mathematical Surveys 2016; 71, 1: 159–160. 22. Savchenko V.V., Ponomarev D.A. Automatic segmentation of stochastic time series using a whitening filter. Optoelectronics, Instrumentation and Data Processing 2009; 45, 1: 37-42.

323