A Selection of Useful Patterns Based on Multi-Criteria ...

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach Fatima Zahra El Mazouri

Mohammed Chaouki Abounaima

K. Zenkouar

Faculty of Sciences and Technology P.O. Box 30000 Morocco [email protected]

Faculty of Sciences and Technology P.O. Box 30000 Morocco [email protected]

Faculty of Sciences and Technology P.O. Box 30000 Morocco [email protected]

ABSTARCT Pattern discovery is one of the most important tasks in data mining, many works are developed in this context where we can notice the problem of ‘pattern explosion’ which make taking a decision about useful pattern more difficult. the goal of our study is to make an improvement in the process of extracting useful patterns from data, called also ‘pattern mining’. The aim of this article is to propose an approach to select useful patterns from a set of patterns by using multicriteria approach. To do this, we will use the famous multicriteria analysis method called ELECTRE, particularly ELECTRE I, in order to have a selection of the most relevant patterns according to proposed criteria.

appropriate patterns in the database according to some measurements (frequency, area, mean, bond). In this work, we focus on a multicriteria processes to have a selection of useful patterns (alternatives/ actions) based on a set of measures: frequency, area, mean, bond (criteria), for this purpose, we suggest to choose ELECTRE I as a multicriteria method to solve choice problematic and have a

KEYWORDS Data mining, multicriteria decision aid, patterns mining, ELECTRE I

1 INTRODUCTION The problem of extracting useful patterns from data is wellknown and important in data mining, and has been used in various fields and in a very large range of application such as bioinformatics [2] and chemo-informatics [3], biomedical [11], image [12]. Since the first article of Agrawal [1] on association rules and itemset mining, a big number of works as [1–3]. to identify the problem of pattern extraction [28] [29], or for the extraction of patterns mining like in [30,31] and we can find also other works interest to frequent pattern mining extraction [32,5], more specifically the extraction of useful patterns as in [9] which present Multimedia Suffix Tree Document model (MSTD) to discover useful patterns embedded in multimedia documents and reduce the search time assisting multimedia mining method. Other work mix user preferences in the mining tasks to limit the number of extracted patterns like the top-k patterns [33,34]. Our work come not only to extract useful patterns based just on extracting frequent patterns, but also, to select useful patterns by using some measurement like: frequency, area, mean, bond. This work has as objective to limit the number of the extracted patterns in order to keep just the most

Figure 1: Multicriteria analysis process in the extraction of useful patterns selection of the most relevant patterns, in order to avoid the very large amount number of patterns which make the recommendation of useful patterns very difficult, to more understand the approach of the selection of mining patterns according to some criteria by adopting a multicriteria method, we propose the process in Fig. 1.

2 KNOWLEDGE DISCOVERY IN DATAMINING 2.1 Preliminary

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach Data mining is the process of discovering interesting knowledge in databases, like associations, patterns, changes, significant structures and anomalies, from data stored in databases, data warehouses, or other repositories of information. It has been also popularly treated as a synonym of knowledge discovery in database, although some researchers view data mining as an essential step of knowledge discovery. Data mining presents an application specific of algorithms and analysis techniques from artificial intelligence and calculates statistics for extracting patterns from data these extraction techniques are intended to present the results as valid elements for decision making. As known Data mining is widely used in various fields and areas such as education [13], healthcare [15], spacial [14], financial [16], this list is not exhaustive there is many others applications of data mining in various domains. Data mining is "Knowledge Discovery in Databases (KDD)" includes also a combination of advancements in the fields of machine learning [17], pattern recognition [18], database [19], statistics [20], artificial intelligence [21], knowledge acquisition and data visualization. This process of extraction is conducted following six steps as shown in the diagram below Fig. 2.

ICCWCS’17, November 2017, Larache, Morocco

Table 1: A transaction database D Tid 1 2 3 4 5 6 7

Itemset ABCD ABEF ABC ACDF G D DG

The Table 1 present the transaction database which contains the different itemset, T= (1, ABCD) called a transaction with id =1 and i=ABCD is an itemset, we can consider for example a pattern P = AB. Definition 1: frequency of a pattern. The frequency of a given pattern p is the number of transactions t in D, that contain the pattern p. Definition 2: disjunctive support The disjunctive support is freq∨(X) = |{t ∈ D|∃i ∈ X : i ∈ t}|.

3 MULTICRITERIA DECISION ANALYSIS APPROACH 3.1 Multicriteria Decision Analysis (MCDA) Process

Figure 2: Knowledge discovery process

2.2 Extraction of Patterns Extraction of useful patterns from data is s a topic of data mining concerned with finding relevant patterns and is an important tool for data analysis and has been used in a large range of applications and domains. Let I be a set of distinct literals called items. A transaction is defined as a couple (tid, I) where tid is the transaction identifier and I is an itemset. A pattern (itemset) correspond to a non-null subset of I. For example, the itemset {a, b, c, d} can also be written as abcd (‘a’ is a literal or an item) The number of items in an itemset is called its length. That is, len(X) = ||X|| (X an itemset). A transaction database D is a finite set of transactions over I. A transaction t = (tid,Y) is said contains an itemset X if X ⊆ Y [6]. Example: Consider the example of a library. Each transaction represents the books bought by a customer during one year [26].

Multicriteria decision Analysis/Making (MCDA/M) has been used to evaluate a set of finite actions or alternatives (trip, candidate, course, project, car …) having multiple criteria, this effective framework is widely used to various fields [22,23]. When the analysis of the actions has led to the construction of a single criterion, we can realize an optimization on this criterion, which can be simple when the number of actions is low, otherwise we need to use more or less complicated. In the frequent case, where the analysis of the consequences of the potential actions has led to the construction of several criteria, it is the multicriteria analysis which makes it possible to give answers to the problem posed. The process of MCDA/M has different steps: 3.1.1 he definition of the problem and the purpose of the decision: The definition of the problem requires an understanding of the situation studied, the context and the actors involved in decision-making. The interaction with the various actors makes it possible to understand the decision-making process, the stakes, the purpose of the decision and the nature of the decision to be made. It is thus a question of defining the nature of the problem posed by formulating it either in a problematic of choice, sorting or storage. The purpose of the decision is to identify the set of actions or alternatives that will be the subject of the decision. 3.1.2 Consequences analysis and determination of criteria It is indeed a question of identifying and measuring the consequences of the actions on which the decision will be

ICCWCS’17, November 2017, Larache, Morocco

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach taken. The criteria derive from the consequences of the actions. Often, an action has several consequences, so the consequence of an action according to a given criterion is evaluated by a function g (with real values) defined on set A potential actions in such a way that it is possible to reason or describe the result of the comparison of two actions a and b relatively from the numbers g (a) and g (b). The evaluation of the action will therefore be performed on a set of criteria. We distinguish the true-criterion and the pseudo-criterion. For the true criterion, considering two actions a and b to compare, two situations are possible: g(b) = g(a) ⇔b Ig a (indifference) and g(b) > g(a) ⇔ b Pg a (strict preference)

(1) (2)

For the pseudo-criterion we associate with the function criterion g two threshold functions qg(g(a)) expressing an indifference threshold and pg(g(a)) expressing a threshold of preference. g(b) ≥ g(a) ⇒b Sb a

(3)

Sb: "as good as" or, S is an over-ranking relationship, that is, b is at least as good as a on a majority of the criteria without being really worse relative to the other criteria. in this case, we can say that ‘b outclasses a’, we note b Sb a. Two thresholds are introduced (constant or function of g) such as: g(b) - g(a) ≤ qg(g(a)) ⇔ b Ig a

(4)

pg(g(a)) < g(b) - g(a) ⇔ b Pg a

(5)

Where qg is a threshold of indifference and pg a threshold of preference. The situation not covered by these two eventualities, namely: qg(g(a)) < g(b) - g(a) ≤ qg(g(a))

(6)

corresponds to a situation of hesitation (indeterminacy) between indifference and preference strict called weak preference and noted Qg. we can translate all that in Fig. 3:

Table 2: Choice of the multicriteria method (Roger et al., 2000) Nature of the problem criterion Truecriterion Pseudocriterion

Nature problem Choosing Sorting I IS

ranking II

TRI

III, V

3.1.4 Performance of actions When the analysis of the actions led to the construction of a single criterion, one can realize an optimization on this criterion, which can be simple when the number of action is weak, if not it is necessary to resort to tools more or less complicated. In the frequent case, where the analysis of the consequences of the potential actions has led to the construction of several criteria, it is the multicriteria analysis that makes it possible to give answers to the problem posed. For each action considered, and for each criterion a preference threshold p, indifference q and a veto threshold v are estimated. Each criterion is given a weight k reflecting its contribution in the final decision. The result of the consequence analysis is presented in a performance table. See Table 3. Table 3: Performances table Criteria Weights Thresholds Actions a1 a2 . . . ai . . . am

g1 k1 p1 q1 v1

g2 k2 p2 q2 v2

…

gj kj pj qj vj

…

g1(a1) . . . . g1(ai)

g2(a1) . . . . g2(ai)

...

..

…

gj(a1) . . . . gj(ai)

…

gn(a1) . . . . gn(ai)

g1(am)

g2(am)

…

gj(am)

…

gn(am)

… … …

… … …

gn kn pn qn vn

3.2 ELECTRE methods

Figure 3: the difference and preference situation 3.1.3 Choosing a multicriteria decision aid method This step depends on the nature of the problem. Several methods have been developed, Table 2 identifies some methods depending on the nature of the problem studied.

Many methods have been developed in the context of MCDA like the famous family of methods named ELECTRE (Elimination and choice translating the reality) with 6 version, (ELECTRE I, ELECTRE II, ELECTRE III, ELECTRE IV, ELECTRE IS, ELECTRE TRI). Each method is used according to the problematic (choosing, sorting, outranking) and according to the type of criteria (true criteria, pseudo-criteria) [8]. Table 2 show the case of using each version of ELCTRE family methods depending on the type of criterion (true or pseudo) and the nature of the problematic posed.

ICCWCS’17, November 2017, Larache, Morocco

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach 3.3 Description of ELECTRE I Method

Thus, “a outrank b”, if (9) is verified: 𝑎𝑆𝑏 ⇔ 𝐶(𝑎, 𝑏) ≥ 𝑐 𝑎𝑛𝑑 𝐷(𝑎, 𝑏) ≤ 𝑑.

ELECTRE I was proposed by B. Roy (1968) allowing to solve multicriteria problems of choice. This method identifies the subset of actions offering the best possible compromise. It is often used to select competing projects, in order to identify the most successful subset of projects on the basis of the criteria considered. In the case of the ELECTRE I method, we define true criteria coded in numerical scales with identical ranges. (see [10] for more details). ELECTRE methods contain two main procedures: 1. Construction of an outranking relation(s): aims at comparing in a comprehensive way each pair of actions 2. Exploitation procedure: used to elaborate recommendations from the results obtained in the first phase. The nature of the recommendations depends on the type of problematic (choosing, ranking or sorting). Hence, each method is characterized by its construction and its exploitation procedures.

3.4 ELECTRE I Algorithm ELECTRE I was developed to be successful when applied to a large range of fields [18], This method is known by its simplicity, in addition ELECTRE I method is applied only when all the criteria have been coded in numerical scales with identical ranges. To more understand this method, we propose the following algorithm of ELECTRE I method: Consider a set A of m actions, which represent the purpose of the decision, the goal is to identify a subset of actions offering a better compromise among the initial set proposed by B. Roy [24]. For each criterion, an evaluation function gj (where j = 1 to n is the number of criteria) is defined, for each criterion, a weight kj is evaluated that increases with the importance of the criterion. The result of decision analysis of this method is based on two important notions concordance and discordance defined as follows: The concordance index for two alternatives a and b is denoted by C(a, b), between 0 and 1, measures the relevance of the assertion "a S b" ( a outrank b) as follows [25]: C(a,b) =

∑∀𝑗:𝑔𝑗(𝑎)≥𝑔𝑗 (𝑏) 𝑘𝑗 𝐾

(7)

with K= ∑𝑛𝑗=1 𝑘𝑗 The discordance index D(a,b), this level measures the power of the discordant coalition, meaning that if its value surpasses a given level, l, the assertion is no longer valid. Discordant coalition exerts no power whenever d(aSb) ≤ l. The concordance index is expressed as follows: 𝑖𝑓 𝑗, 𝑔𝑗 (𝑎) ≥ 𝑔𝑗 (𝑏) 𝐷(𝑎, 𝑏) = 0 D(a,b) { (8) 1 𝐷(𝑎, 𝑏) = 𝑚𝑎𝑥[ 𝑔𝑗 (𝑏) − 𝑔𝑗 (𝑎)] 𝛿

With δ is the maximum difference between two alternatives for the same criterion. The outranking relationship for ELECTRE I is constructed by comparing the concordance and discordance indices with concordance thresholds 𝑐 and discordance d.

4 APPLICATION OF METHOD WITH CRITERIA

ELECTRE I DATAMINING

In the context of choosing the best patterns, there is many works which have an objective of extracting and running a domination test with respect to the user preferences and finally output the skyline patterns. This approach is not feasible in practice because the collection of patterns is often too big to be manageable, some constraints might be introduced in order to limit the size of the collection but the consistency of the result may be lost. Another work come with the idea to take benefit of theoretical relationships between pattern condensed representations and skypatterns. These results improve skypattern extraction with, an efficient approach which only takes as an input the data set and the measures expressing the user preferences and returns skypatterns [27]. A key idea of our work is to mine mining patterns by considering some measurements expressing the user preferences by using multicriteria approach, for this purpose we use ELECTRE I as the most appropriate method for choosing problematic having as objective to isolate a subset of relevant solutions. We consider the example adopted from [27], we use the table of transaction Table 4 containing itemset patterns. As mentioned previously, a transaction dataset is a multiset of patterns, which each pattern named a transaction, Table 4 (a) present a transactional dataset D, that 6 transactions denoted by t1, t2, t3, …, t6 described by 6 items denoted by A, …, F. the Table 4 (b) present the value of each item. Table 4: Example of a toy data set and measures (a) A toy data set Tid t1 t2 t3

Items ABCDEF ABCDEF AB

t4 t5 t6

A C

D E

(b) Value of each item Items A

Val 10

B C

55 70

D E

30 15

F

25

As presented in [7] general definitions of a very large set of interesting measures M. We will present some specific of these measurement: we choose 4 measures: 1) m1: frequency of a pattern 2) m2: area: 𝑿 → 𝒇𝒓𝒆𝒒(𝑿) ∗ 𝒍𝒆𝒏𝒈𝒉𝒕(𝑿) 𝒎𝒊𝒏(𝑿.𝒗𝒂𝒍)+𝒎𝒂𝒙(𝑿.𝒗𝒂𝒍) 3) m3: the mean 𝑿 → (Given a 𝟐

function val: I → R+, we extend it to a pattern X and note X.val the multiset {val(i)|i ∈ X}. This kind of function is used with the usual SQL-like primitive sum, min and max.

ICCWCS’17, November 2017, Larache, Morocco

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach For instance, sum(X.val) is the sum of val for each item of X.) 𝒇𝒓𝒆𝒒(𝑿) 4) m4: bond 𝑿 →

P1

𝒇𝒓𝒆𝒒 𝒗(𝑿)

P2

By using those measures m1, m2, m3, m4 as criteria noted respectively Cr1, Cr2, Cr3, Cr4, we consider the patterns {ABCDEF, AB, AC, A} as alternatives noted respectively P1, P2, P3, P4.

P4

Table 5: Performance Table P1 P2 P3 P4 Weight kj

Cr1 2 3 3 4 4

Cr2 6 2 2 1 3

Cr3 40 32.5 40 10 2

Cr4 0.33 0.75 0.75 1 1

This table named the matrix of performances or decision matrix, which has n rows and m columns, or n*m elements, as shown in the Table 5. Each element, such as gj(ai), is either a single numerical value or a single grade, representing the performance of alternative ai on criterion j. To solve the problem of choosing the useful patterns, we calculate the concordance (7) and discordance indices (8) as first step: The matrix of concordance indices is given below: Table 6: Concordance indices table P1 P2 P3 P4

P1 1 0.5 0.7 0.5

P2 0.5 1 1 0.5

P3 0.5 0.8 1 0.5

P4 0.5 0.5 0.5 1

The Table 6 show the result of the concordance indices C(a,b) that measures the strength of support, given the available evidence, that a is at least as good as b considering all criteria. Discordance index D(a,b) measures the strength of the evidence against this hypothesis. Expressed in this example in the Table 7. The discordance matrix is obtained as follows: Table 7: Discordance table P1 P2 P3 P4

P1 0.25 0.8 1

P2 0.5 0 0.75

P3 0.5 0.25 1

P4 1 0.5 0.5 -

The last step of the ELECTRE I algorithm is to construct the outrank graph for c=1 and d=0, we obtain the following graph:

P3

Figure 4: Outrank graph According to this graph in Fig. 4 P1 and P2 are incomparable, we notice also that P3 outrank P2. In conclusion, the P3 is considered as the most useful pattern (pattern mining) to the decision maker’s preferences.

5 CONCLUSION In this paper, we presented a new method to select useful patterns mining by using multi-criteria approach. Our approach consists to use of ELECTRE I method, the objective of our method is to find the more useful patterns selected according to the preference of decision makers. We studied extraction of patterns according to four measurements commonly used in literature for comparing and searching for selected patterns, we used these measurement as criteria with ELECTRE I multicriteria method for selecting patterns mining.

REFERENCES [1]

Agrawal, R., Imielinski, T and Swami. N. 1993. Mining association rules between sets of items. In large databases. In ACM SIGMOD International Conference on Management of Data, 207-216. Baltimore: ACM Press. [2] M. J. Zaki and K. Sequeira, “Data mining in computational biology” in Handbook of Computational Molecular Biology. Chapman & Hall/CRC Press, 2006, ch. 38, p. 1–26. [3] J. Auer and J. Bajorath, Emerging chemical patterns: A new methodology for molecular classification and compound selection, J. Chem. Inf. Mod, vol. 46, no. 6, p. 2502–2514, 2006. [4] ] H. Mannila and H. Toivonen, “Levelwise search and borders of theories in knowledge discovery,” DMKD, vol. 1, no. 3, p. 241– 258, 1997. [5] ] R. Agrawal, T. Imielinski, A. Swami, Mining association rules between sets of items in large databases, ACM SIGMOD Rec. 22 (2) (1993) 207–216. [6] Moonjung Cho, Jian Pei, Haixun Wang, Wei Wang, Preferencebased Frequent Pattern Mining.International Journal of Data Warehous and Mining (IJDWM), 1(4) 2005 pages:22. [7] A. Soulet and B. Crémilleux, “Mining constraint-based patterns using automatic relaxation,” Intell. Data Anal., vol. 13, no. 1, p. 109–133, 2009. [8] B. Roy, D. Bouyssou, Aide multicritère méthodes et cas, Economica, Paris, 1993. [9] K. Pushpalatha, V.S. Ananthanarayana, A tree based representation for effective pattern discovery from multimedia documents, Pattern Recognition Letters. 000 (2016) 1–11. [10] L. Maystre, J. Pictet, J. Simos, Méthode multicritère ELECTRE, Presses Polytechniques et Universitaires Romandes (PPUR), 1994, 340 pages. [11] J. Neesha, A.R. Nur’Aini, H. Wahidah, Data Mining in Healthcare – A Review, Procedia Computer Science. 72 (2015) 306 – 313.

A Selection of Useful Patterns Based on Multi-Criteria Analysis Approach [12] Hedieh Sajedi, Steganalysis based on steganography pattern Discovery, journal of information security and applications. 30 (2 0 1 6) 3–14. [13] Asif R., Merceron A., Ali S.A. & Haider N.G., Analyzing undergraduate students' performance using educational data mining, Computers & Education (2017), doi:10.1016/ j. compedu.2017.05.007. [14] Shashi Shekhar, Yan Li, Reem Y. Ali, Emre Eftelioglu, Zhe Jiang, Spatial and Spatiotemporal Data Mining, Comprehensive Geographic Information Systems, 2018, Pages 264-286. [15] J. Neesha, A.R. Nur’Aini, H. Wahidah, Data Mining in Healthcare – A Review, Procedia Computer Science. 72 (2015) 306 – 313. [16] Ila Dutta, Shantanu Dutta, Bijan Raahemi, Detecting Financial Restatements Using Data Mining Techniques, Expert Systems With Applications (2017), doi: 10.1016/j.eswa.2017.08.030. [17] David E. Jones, Hamidreza Ghandehari, Julio C. Facelli, A review of the applications of data mining and machine learning for the prediction of biomedical properties of nanoparticles, Computer Methods and Programs in Biomedicine (2016), http://dx.doi.org/doi: 10.1016/j.cmpb.2016.04.025. [18] Eva Armengol, Dion«õs Boixader, Francisco Grimaldo, Special Issue on Pattern Recognition Techniques in Data Mining, Pattern Recognition Letters (2017), doi: 10.1016/j.patrec.2017.02.014. [19] Ashis Kumar Chanda, Chowdhury Farhan Ahmed, Md. Samiullah, Carson K. Leung, A new framework for mining weighted periodic patterns in time series databases, Expert Systems with Applications (2017), doi: 10.1016/j.eswa.2017.02.028. [20] Giovanni Montana, Stefan Van Aelst, Ruben H. Zamar, Special Issue on Advances in Data Mining and Robust Statistics, Computational Statistics and Data Analysis 93 (2016) 388–389. [21] Shahab Dean Mohaghegh, Reservoir simulation and modeling based on artificial intelligence and data mining (AI&DM), Journal of Natural Gas Science and Engineering. 3 (2011) 697-705. [22] M. Mousavi, H. Gitinavard, S.M. Mousavi, A soft computing basedmodified ELECTRE model for renewable energy policy selection with unknown information Renewable and Sustainable Energy Reviews, Volume 68, Part 1, February 2017, Pages 774-78. [23] Getnet D. Betrie, Rehan Sadiq, Kevin A. Morin, Solomon Tesfamariam, Selection of remedial alternatives for mine sites: A multicriteria decision analysis approach, Journal of Environmental Management, Volume 119, 15 April 2013, Pages 36-46. [24] Roy, Bernard (1968). “Classement et choix en présence de points de vue multiples (la méthode ELECTRE)”. La Revue d'Informatique et de Recherche Opérationelle (RIRO) (8): 57–75. [25] A.-M. Addi, A. Tarik; G. Fatima, "Comparative survey of association rule mining algorithms based on multiple criteria decision analysis approach”. In Control, Engineering & Information Technology (CEIT), 2015 3rd International Conference on, vol., no., pp.1-6, 25-27 May 2015. [26] S.Jabbour, S. Kaci, L. Sais, Y. Salhi, Penalty based Preferences in Itemset Mining, Association for the Advancement of Artificial Intelligence (www.aaai.org). [27] Arnaud Soulet, Chedy Raissi, Marc Plantevit, Bruno Crémilleux, Mining Dominant Patterns in the Sky, 11th IEEE International Conference on Data Mining 2011. [28] A. Histace, M. Ménard, C. Cavaro-Ménard, Selective diffusion for oriented pattern extraction: Application to tagged cardiac MRI enhancement, Journal of Pattern Recognition Letters 30 (2009) 1356–1365. [29] V. Pouget, S. Jonathas, R. Job, J.R. Vaillé, F.Wrobel, F. Saigné, Structural pattern extraction fromasynchronous two-photon laser fault injection using spectral analysis, Microelectronics Reliability, In press, No of pages 5. [30] Bernardete Ribeiro, Ning Chen, Alexander Kovacec, Shaping Graph Pattern Mining for Financial Risk, Neurocomputing (2017), doi: 10.1016/j.neucom.2017.01.119. [31] Peter Liang, Yingzhen Yang, Yang Cai, Pattern mining from saccadic motion data, Procedia Computer Science 1 (2010) 2529– 2538. [32] Shamila Nasreen, Muhammad Awais Azam, Khurram Shehzad, Usman Naeem, Mustansar Ali Ghazanfar, Frequent Pattern Mining Algorithms for Finding Associated Frequent Patterns for Data Streams: A Survey, Procedia Computer Science 37 (2014) 109 – 116. [33] Y. Ke, J. Cheng, and J. X. Yu, “Top-k correlative graph Mining”. In SIAM DM, 2009, p. 1038–1049. [34] J. Wang, J. Han, Y. Lu, and P. Tzvetkov, “TFP: An efficient algorithm for mining top-k frequent closed itemsets,” TKDE, vol. 17, p. 652–664, 2005.

ICCWCS’17, November 2017, Larache, Morocco