Two-level Evolutionary Algorithm for Discovering Relations Between Nodes’ Features in a Complex Network ˇ David Jesenko∗, Marjan Mernik, Borut Zalik, Domen Mongus University of Maribor, Faculty of Electrical Engineering and Computer Science, Smetanova 17, SI-2000 Maribor, Slovenia
Abstract Complex network theory offers an efficient mathematical framework for modelling natural phenomena. However, these studies focus mainly on the topological characteristics of networks, while the actual reasons behind the networks’ formation remain overlooked. This paper proposes a new approach to complex network analysis. By searching for the optimal functional definition of the network’s edge set, it allows an examination of the influences of the physical properties of the nodes on the network’s structure and behaviour (i.e. changes of the network’s structure when the physical properties of nodes change). A twolevel evolutionary algorithm is proposed for this purpose, whereby the search for a suitable function form is achieved at the first level, while the second level is used for optimal function fitting. In this way, not only the features with the largest influences are identified, but also the intensities of their influences are estimated. Synthetic networks are examined in order to show the superiority of the proposed approach over traditional machine learning algorithms, while the applicability of the proposed method is demonstrated on a real-world study of the behaviour of biological cells. Keywords: complex networks, topology, function fitting, machine learning, evolutionary algorithms
1. Introduction Complex networks has become a popular research topic in many different fields of science, such as coupled biological and chemical systems, evolutionary dynamics [1], social interacting species and human formations like cities [2]. ∗ Corresponding
author. Tel.: +386-2-220-7476; fax: +386-2-220-7272. Email address:
[email protected] (David Jesenko) URL: http://gemma.feri.um.si (David Jesenko)
These are only a few examples of systems composed of a large number of highly interconnected individual elements [3]. Graphs offer the most natural approach to their modelling, where individual elements are considered as nodes, while the edges describe interactions between them. However, graphs used for modelling natural phenomena are characterised by their irregularities and non-trivial topological features [4], and are often dynamic. Due to these specific properties, they are referred to as complex networks [5]. Initiated by [6] and [7], research of this subject began with the purpose of developing new mathematical concepts and tools for the characterisation of real-world networks. To date, a great effort has been invested in understanding the evolution of the networks [8, 9], their structure and dynamics [5, 10], the relations between structures and functions [11, 12], and other topological characteristics (e.g., clustering coefficient, cyclic coefficient, degree distribution, entropy, centrality, sub-graphs and fractal dimensionality [13, 14]). These studies have shown that many real-world networks share certain fundamental properties. For example, the so-called small-word phenomenon describes the fact that the topological distances between nodes are typically small [15] (e.g., as shown by the famous experiments carried out by Stanley Milgram [16], one can reach any other person in the world through approximately six intermediate persons). Another example is the so-called scale-free networks, in which the number of connections of nodes is distributed by the power law [17]. This may have an immense impact on the network’s behaviour (e.g., a typical scale-free network, such as the internet, collapses completely when approximately 3% of the highest degree routers are removed deterministically) [18]. In addition to the topological distance and the degree distribution of the nodes, transitivity, diameter and clustering factors are commonly observed properties of the real-world networks [5]. Although these may provide significant insights into the behaviour of the network, they are still unable to explain the reasons behind the network’s formation and the role played by the nodes’ non-topological features, such as the location of the nodes or their physical properties (e.g., their heights, sizes or weights). This paper proposes a new approach to the analysis of the complex networks. By estimating the functional definition of a network’s topology, the proposed method allows influence examination of the non-topological node features on the network’s properties and behaviour. Within this context, the following contributions of the paper can be exposed: • a two-level evolutionary algorithm (EA) that is capable of identifying those properties (features) of nodes that allow for successful prediction of edges between them and may, consequently, reveal new insights into to the reasons behind the network’s formation, • the search for an optimal function type at the first level of EA that allows for examination of the non-linear codependency between the nodes’ features and leads to the improved accuracy of the algorithm in comparison to the traditional machine learning approaches (see Section 4.3),
2
• the search for optimal function’s coefficients at the second level of EA that quantifies the importance of each particular feature within the definition of an edge set. While a two-level EA is a methodological novelty, the search for an optimal function type, together with the optimization of its coefficients, may be considered as a new application domain of EA. The rest of the paper is organised as follows: an overview of the related work is given in Section 2. Section 3 provides details of the proposed method. Validation of the proposed method is presented in Section 4. Section 5 concludes the paper. 2. Related work The main objective of this work is to provide a new approach to complex networks analysis, capable of revealing those physical properties (features) of the nodes that contribute to the network’s formation. More precisely, by searching for an accurate functional definition of the edge-set, the algorithm is designed to extract those determinative features (and the corresponding weights) that specify, whether an edge between two nodes is established or not. In this respect, the proposed approach is similar to the feature extraction (selection) approaches used in the machine learning, e.g. in the cases where a large number of irrelevant features deteriorate the learning performance and it is, therefore, essential to reduce their number. The latter can be achieved either by selecting only the most relevant features or by deriving new features from the original ones. In the continuation, we provide a short overview of the related approaches. Although none of these methods have yet been applied for the analysis of complex networks, they all use evolutionary computation to perform feature extraction. In the machine learning (ML), feature construction is often achieved by a functional expression using the original features [19]. Genetic Programming (GP) has frequently been applied for feature construction, as evolutionary computation explores and exploits the searching space successfully [20]. A general framework for GP-based feature construction was described in [21]. This work pointed out the limited ability of many classifiers to construct an appropriate representation of an individual, which were overcome by the flexibility of GP. Krawiec also suggested that the individual should be split into an evolving and a hidden parts [21]. However, this method has only been used for simple datasets, where linear functional expressions have been sufficient. GP and Genetic Algorithm (GA) were combined for a novel feature construction in [22]. GP was applied for constructing new features from those available, while GA was used to determine, which features are the most predictive to construct a decision tree. The approach has not been used in the complex networks, yet. A feature extraction with GA was also proposed in [23], where GA was responsible for discovering interactions between features. This approach was applied to the poker hand problem. The search for mathematical or functional relations between features was also presented in [24], where the feature 3
construction process consists of a random selection of mathematical functions, constants and previously generated features. Finally, evolutionary computation has been used for feature selection, too. This can be seen as a special case of the feature construction process, where only simple functional expressions are used (the feature may or may not be included) without discovering complicated relationships between features. For example, GP-based feature extraction was used in [25], while grammar guided genetic programming (G3P) was proposed in [26]. 3. Method This section proposes a new approach to the analysis of complex networks that discovers interrelationships between the properties of nodes, resulting in the network topology and its behaviour. We define a complex network C = (N, E) by a set of nodes N = {ni } and a set of edges E = {ei,j }, where any given edge is defined by a pair of adjacent nodes ei,j = (ni , nj ). Each node is, furthermore, related with a feature vector describing its properties. For this purpose, let V : N → RD be a mapping function that returns a D−dimensional feature vector corresponding to a given ni , namely V[ni ]. When considering adjacency relations between the nodes in a real-world study, we often have reason to believe that the adjacencies were established as a consequence of the nodes’ properties. Instead of using an adjacency matrix or some other common representation of an edge set, we model a threshold function by which the network topology can be defined. For this purpose, let a mapping function T : (V, V) → R acting on pairs of properties of nodes, and a threshold th. A definition of the edge set E T can then be given as E T = {(ni , nj )|T (V[ni ], V[nj ]) ≥ th},
(1)
where ni , nj ∈ N . In addition to a more compact representation of the edge set, such a function may reveal significant insights into the network’s formation by revealing the functional relations existing between the physical features of the nodes. As an illustrative example, we may consider a network of tourists (represented by nodes), where two are connected by an edge if they have enough money cumulatively to buy themselves a sightseeing tour with a carriage. In this case, the function T is defined by the sum of the money of two tourists and the threshold value th corresponds to the price of the ride. While this trivial example is used only to demonstrate the rational behind the edge-set definition, a practical example from two real-world studies are given in Section 4. In the continuation, a two-level evolutionary algorithm is proposed for obtaining the optimal definition of T and estimating its coefficients together with the value of threshold th. The first level is intended to search for the optimal function type, while the second level is used to achieve its optimal fitting for the establishment of a ”learning topology”. As shown in Fig. 1, the proposed approach starts with the definition of the input parameters, the random generation of the initial population, and its evaluation. After the initialisation step, 4
the algorithm enters a traditional loop of crossovers, mutations and selections, where the fitness function is implemented in the form of a separate EA. The algorithm finishes when the maximal number of evaluations is achieved or when a sufficient solution is found. In the continuation, both EAs are explained in detail.
Optimization of function form Function fitting
Start
Start Initialization Crossover
Fitness
Initialization
Mutation
Crossover
Selection
Mutation no
Selection
End condition
yes
End
no
End condition
yes
End
Figure 1: Flowchart of the proposed algorithm, where the function fitting (i.e. Level 2) is used for estimating individuals’ fitnesses during the optimisation of the function type (i.e. Level 1).
5
3.1. Definition of the individual In order to search for the optimal definition of function T , a template for its representation is first considered. With the intention of providing a flexible definition accounting for most of the elementary types (e.g., polynomial, rational, exponential and N −th root, as well as common geometric expressions), we define T as T (ni , nj ) =
L a K a H a
k
h
cl,k,h (V(ni )[l] ∗l,k,h V(nj )[l] )pl,k,h ,
(2)
l=0 k=0 h=0
where: ` P Q • is a generic symbol or product over the ` used P for Q denoting sum ∈ { , }, series of terms, i.e., • L is the number of features (properties) associated with each node, • K and H define a degree of the function T , • cl,k,h is a coefficient associated with the term defined by l, k, h, • ∗l,k,h is a generic symbol used for denoting one of the possible operators, i.e., ∗l,k,h ∈ {+, −, ×, /}, and • pl,k,h is a potency associated with the term defined by l, k, h. Eq. 2 allows the encoding of T in the form of an individual, where each of the genes defines one term of the function, with the exception and ` of `the first` the last genes. The first gene is used to define operators L , K , and H , while the last gene encodes threshold value th. Genes representing the terms of Eq. 2 are composed of coefficients c, potencies p and operators ∗. A graphic representation of an individual and its genes is shown in Fig. 2.
π
L
K
H
p0
c 0,0,0
π
π
*
c ,0,
0
p0
,1
0 ,0
c
,0, 1
*
,H
L,K
*
0,0,1
pL
,K, H
L,K,H
th
0,0,0
Figure 2: Graphic representation of an individual, where the hatched parts represent parameters searched on level one, while the optimal values of other parts are estimated at the level two EA.
According to the definition given by Eq. 2, T is defined by (L + 1) × (K + 1) × (H + 1) terms. Consequently, a large number of coefficients and potencies need to be estimated when the number of properties of nodes or the degree of T is high. One way to reduce the searching space is to randomly exclude some of
6
the terms. This can be achieved by presetting their coefficients in the form of identity elements. In this case, cl,k,h should be set to 0 in the case ` coefficients P of summation (i.e., when = ), while cl,k,h 1 and pl,k,h = 0 should be ` =Q used in the case of multiplication (i.e., when = ). The selection of terms included in (or excluded from) the definition of T , as well as the search for suitable operators, is done during the optimization of the function type. 3.2. Search for the optimal function type The search for the optimal function type is achieved`at the level one EA (see Algorithm 1), where the optimization of parameters and ∗ is performed. The initialization of the input EA parameters is done, including the definition of probabilities of mutation and crossover, size of population, maximal number of evaluations allowed, and the range of the coefficients and the threshold. The initial population consists of individuals, each representing one possible solution for the optimal definition of function T . The individuals contain a random number (from the range [0, (L + 1) × (K + 1) × (H + 1)) of identity elements, at random positions. After the initial population is generated, its evaluation is achieved by running function fitting at the second level of the EA before entering the loop of crossovers, mutations and selections. These operators are defined as follows: • Crossover at the first level is achieved by exchanging the genes between the two parents. By gene, we mean a set of three ` parameters (coefficient ` ` c, potency p and operator ∗), a combination of L , K , and H , or threshold th (see Fig. 3). For this purpose, a set of parents is generated by assigning a pair randomly to each of the individuals. In this way, we ensure that any given individual can potentially be involved in the crossover, while the actual number of crossovers is defined by the crossover probability. Two-point crossover is implemented here. As shown in Fig. 3, the genes between the first and the second crossover point of the first parent are replaced by the genes from the second parent. Both crossover points are selected at random. Every child is evaluated and its fitness is compared to the highest of the fitnesses between the parents. If the child’s fitness is higher than those of the parent, the child is used to replace it. • Mutation is implemented as a random change of an operator (see hatched parts of genes in Fig. 2). Note that the values of c, p and th are optimised at level two of the EA, thus their mutation is not needed at this level. The number of mutated individuals equals the probability of mutation, while their selection is made at random. Similarly to the crossover, two-point mutation is used, where all of the terms between the first and the second mutation point are changed randomly. Each newborn is again compared to its parent, and only the one with the highest fitness is preserved. • Selection is achieved by comparing each individual sequentially from the population with a randomly selected opponent, where only the fittest one
7
π
L
K
c
H
0,0
p0
,0
p0
c 0,0,1 ,0,
0
π
π
*
,0,
1
*
c
0,0,1
π K
c
H
p0
0
, 0,0
p0
c 0,0,1 ,0,
0
,0,
1
*
c
0,0,1
π H
c
p0
0
, 0 ,0
π
π
π
π
*
*
c
0,1,0
0,0
p0
,2
p0
,0,
0
*
0,0,1
,0,
1
p0
c 0,1,0 ,0, 2
c
p0
2
, 0,0
*
,1,
0
*
c
0,1,0
,1,
1
p0
,1,
2
*
c L,K,
H
pL
,K,
H
*
0,1,2
,1
0,1
p0
c 0,1,2 ,1, 1
*
th
L,K,H
,1, 2
*
H
c L,K,
pL
,K,
H
*
0,1,2
th
L,K,H
0,1,1
p0
c 0,1,0 ,0, 2
*
0,1,0
,1,
0
c
p0
,1 0,1
*
0,0,2
0,0,0
,1
0,1
p0
c 0,1,2
*
0,0,2
c 0,0,1
p0 0,1,1
*
0,0,0
K
,0, 2
,1, 0
0,0,2
* L
,2
p0
c 0,1,0
*
0,0,0
L
0,0
p0
p0
c 0,1,2 ,1,
1
,1, 2
*
0,1,2
H
c L,K,
*
pL
,K,
L,K,H
H
th
0,1,1
Figure 3: Two-point crossover of individuals.
advances to the next generation. Thus, the selection is of a tournament type, with greedy selection. Finally, the main loop ends when the condition of maximal function evaluations is achieved, or when the desired level of accuracy is reached [27]. Note that the fitness of an individual at this level is defined by the level of accuracy that can be achieved by the function type it encodes. Thus, the optimal fitting of a given function type is required during the fitness estimation. For this purpose, the second EA is proposed in the following section (see Algorithm 2). 3.3. Function fitting Function fitting is responsible for finding optimal values of coefficients c, potencies p and threshold th, according to the given function type. Note that, when evaluating an individual that was born by crossover or mutation at the previous level, candidates for optimal values of c, p and th already exist. In this case, a portion of the starting population (80% in our case) is generated by mutating a given candidate, while the rest of the generation is generated at random. Obviously, when evaluating individuals from the initial population at the previous level, function fitting starts from zero and all of the individuals need to be created at random. Each individual is evaluated entering the main loop of crossovers, mutations and selections. Fitness function and the following EA operators are defined as: • Fitness of individuals is realised with the construction of a complex network that arises from the provided definition of T . The similarity of the constructed network and the target network is estimated by comparing
8
Data: Set the control parameters of 2-level genetic algorithm SP : size of population M F E: maximal number of fitness evaluations P C: probability of crossover P M : probability of mutation RC: range of coefficients RT : range of threshold P RI: percentage of random individuals on level 2 N E: number of fitness evaluations on level 2 P CV : percentage of the change of values on level 2 Result: Equation of connectivity in complex network Algorithm Optimization of function form total eval ← 0; //Initialization; for i ← 1 to SP do P (i).terms ← Generate terms; P (i).F 1 score ← F unctionF itting(P (i).terms, N U LL, total eval, SP, ...); end //Find optimal equation for target complex network; while total eval < M F E do for i ← 1 to SP do if crossover(PC) then child ← Crossover of operators; if F unctionF itting(child.terms, child.coef f, total eval, SP, ...) > P (i).F 1 score then P (i) ← child; end if mutation(PM) then mutant ← Mutation of operators; if F unctionF itting(mutant.terms, mutant.coef f, total eval, SP, ...) > P (i).F 1 score then P (i) ← mutant; end end Greedy selection with maximization of F1 score; end return equation with highest F 1 score; end Algorithm 1: First level of proposed EA.
9
the edge sets of both networks, as the node sets are identical for both networks. Note that an edge set of a target network is given by E, while an edge set defined by T is, according to Eq. 1, given by E T . A quantitative measurement of similarity between the two networks can thus be achieved by the following measures: – true positive (TP) is a correctly identified edge, meaning that a given eij exists in both networks, i.e., eij ∈ E and eij ∈ E T , – false positive (FP) is an incorrectly identified edge eij that does not exist in the target network but exists in the generated network, i.e., eij ∈ / E and eij ∈ E T , – true negative (TN) is a correctly rejected edge eij that does not exist in any of the networks, i.e., eij ∈ / E and eij ∈ / ET , – false negative (FN) is an incorrectly rejected edge eij that exists in the target network but cannot be found in the generated network, i.e., eij ∈ E and eij ∈ / ET . By assuming that errors defined by FP and FN are equally important, we can use the harmonic mean of precision and sensitivity for fitness function (see Eq. 3), known as F1 score measure [28], defined by: 2T P . (3) 2T P + F P + F N Note that the definition of the F1 score does not require true negatives to be measured. This is particularly useful in our case, as node-to-node comparison is performed in order to estimate F1 score and, thus, a significantly larger number of negatives is usually expected in comparison to positives. This imposes a thread of overemphasising true negatives and, consequently, estimating unrealistically high fitness even in those cases, where actual accuracy in terms of other indicators (i.e. TP, FP, and FN) is, in fact, low. F1 =
• Crossover at this level is achieved by a similar principle to that on level one. However, only coefficients, potencies`and`the threshold ` are exchanged between the parents, while operators ∗, L , K , and H do not need to be exchanged, as both parents encode the same function type. A graphic representation of crossover is shown in Fig. 4. • Mutation is` done`only on coefficients, potencies and the threshold, while ` operators ∗, L , K , and H are not mutated in order to preserve the function type. Furthermore, in order to ensure an acceptable probability of an individual survivor, the range of mutation is limited. In the case of coefficients and the threshold, mutation can change the value up to 40% of the maximal value, while potencies can be changed by 0.5. Apart from this, the process of selecting individuals and the individuals’ genes is identical to the mutation implemented at level one. 10
π
L
K
c
H
0,0
p0
,0
p0
c 0,0,1 ,0,
0
π
π
*
,0,
1
*
c
0,0,1
π K
c
H
p0
0
, 0,0
p0
c 0,0,1 ,0,
0
,0,
1
*
c
0,0,1
π H
c
p0
0
, 0 ,0
π
π
π
π
*
0,0,0
*
c
0,1,0
0,0
p0
,2
p0
,0,
0
*
0,0,1
,0,
1
p0
c 0,1,0 ,0, 2
c
p0
2
, 0,0
*
,1
0,1
p0
c 0,1,2 ,1,
1
*
,1,
0
*
c
0,1,0
p0
,1,
2
*
c L,K,
H
pL
,K,
H
*
0,1,2
,1
0,1
p0
c 0,1,2 ,1, 1
*
0,0,2
c 0,0,1
p0
th
L,K,H
0,1,1
*
0,0,0
K
,0, 2
,1, 0
0,0,2
* L
,2
p0
c 0,1,0
*
0,0,0
L
0,0
p0
,1, 2
*
H
c L,K,
pL
,K,
H
*
0,1,2
th
L,K,H
0,1,1
p0
c 0,1,0 ,0, 2
*
0,1,0
,1,
0
c
p0
,1 0,1
*
0,0,2
p0
c 0,1,2 ,1,
1
*
0,1,2
,1, 2
H
c L,K,
*
pL
,K,
L,K,H
H
th
0,1,1
Figure 4: Two-point crossover of individuals on the second level, where the darkest parts are the same in all individuals.
• Selection does not differ from the selection at the previous level. It is of a tournament type with greedy selection. Function fitting ends when the optimal parameters for the function type are found (the same condition as for level one), or when the highest F1 score does not make any progress in the five consecutive generations. Finally, the best individual is passed back to level one of the method and the algorithm continues there. 4. Results 4.1. Parameter sensitivity The proposed method relies on eight input parameters, whereby three of them are used for both levels (size of population (SP), probability of mutation (PM), and probability of crossover (PC)), while the other parameters (range of coefficients (RC), range of threshold (RT), percentage of the change of values (PCV), percentage of random individuals (PRI), and number of evaluations (NE)) are used only on the second level. In order to examine their influences on method performance and estimate their optimal values, a systematic sensitivity analysis was performed, as proposed by Chang and Delleur [29]. For this purpose, a spatially embedded network was used, initially constructed for a study of relationships between buildings within urban environments. Its nodes thus corresponded to the individual buildings, while the edges between them where defined heuristically in such a way that buildings from the same municipality formed a single connected component within the network. The height, perimeter, area and volume of the buildings were measured using airborne Light 11
Function FunctionFitting(terms t, coeff c, total eval, SP, RC, ST, PRI, PC, PM, PCV) num eval ← 0; //Initialization; for i ← 1 to SP do P (i).terms ← t; if c == N U LL then P (i).coef f ← Random coefficients, potencies, and threshold (RC and RT ); else P (i).coef f ← Some random (P RI, RC, and RT ), other from c (1 − P RI); end P (i).F 1 score ← F itness(P (i), num eval); end //Find optimal coefficients for terms t; while num eval < N E do for i ← 1 to SP do if crossover(PC) then child ← Crossover of coefficients, potencies, or threshold; if F itness(child, num eval) > P (i).F 1 score then P (i) ← child; end if mutation(PM) then mutant ← Mutation of coefficients, potencies, or threshold (P CV ); if F itness(mutant, num eval) > P (i).F 1 score then P (i) ← mutant; end end Greedy selection with maximization of F1 score; end total eval ← total eval + num eval; return coef f with highest F 1 score; end Function Fitness(equation e, num eval) num eval ← num eval + 1; Construct complex network with equation e; Compare with target network and count T P , F N and F P ; F 1 score ← solution by Eq. 3; return F 1 score; end Algorithm 2: Second level of the presented method including fitness function.
12
ID 1 2 3 4
Equation 0.6(volumei +volumej ) ≥ 18.2 0.1D(ni ,nj )2 −14.8(perimeteri +perimeterj ) ≥ 0.2 −5.6D(ni ,nj )2 −13.2(heighti ×heightj ) ≥ 0.4 −1.2D(ni ,nj )2 9.7(heighti ×heightj ) ≥ 10.5 0.1D(ni ,nj )2
F1 score 0.756 0.811 0.938 1
Table 1: Examples of equations obtained by the proposed method and the achieved F1 scores, where D represents the distance between nodes ni and nj .
Detection and Ranging (LiDAR) data [30] and assigned as features of individual nodes. Examples of estimated equations can be seen in Table 1. Obviously, the optimal definition of the edge set was obtained by Eq. ID4, where it can be seen clearly that the building heights and distances between them are the main reasons for forming an edge. From application perspective, such definition may allow urban planners to define restrictions on the positions and heights of the new buildings in order to preserve visibility of the border between municipalities, as this can be seen in Fig. 5. This example served us as a test case on which sensitivity analysis was performed.
Figure 5: Network of connected buildings within the same municipality.
13
Parameter Number of independent runs Number of function evaluations Size of population Probability of crossover Probability of mutation Coefficients range Threshold range Percentage of change of coefficients Percentage of change of threshold Percentage of random individuals on level 2 Number of function evaluations on level 2
Value 50 50000 50 60% 50% [−15, 15] [0, 120] 40% 40% 20% 150
Table 2: Algorithm and experiments settings.
The ranges of the input parameter values were first defined during the sensitivity analysis. Values from the range [0.1, 1.0] were used when considering the percentage of change of values, the percentage of randomly generated individuals, the probability of mutation and the probability of crossover. The range of coefficients was [5, 50], the threshold range was [0, 200] and the size of the population was from the range [20, 200]. The number of function evaluations on the second level was considered to be from the range [30, 300]. Taking these restrictions into account, 2, 000 sets of parameter values were generated randomly and the proposed EA was evaluated accordingly. The obtained results were classified as acceptable if the run finished with the score F 1 ≥ 0.98; otherwise the results were regarded as unacceptable. In Fig. 6, the cumulative frequencies of acceptable and unacceptable results are shown with regard to each of the input parameters. Insensitive parameters are characterised by similar cumulative frequencies of acceptable and unacceptable results throughout the whole range of parameter values, as indicated by the plots in Figs. 6a, 6b, 6d, 6f, and 6h. On the other hand, the size of the population (see Fig. 6c), the threshold value (see Fig. 6e) and the percentage of value change at level 2 (see Fig. 6g) were recognised as those parameters to which the method is sensitive. Of these parameters, the highest influence on the results was observed in the case of population size, where a significantly larger number (approximately 26.2%) of acceptable results was achieved when its value was equal to 50 than in any other case. Considering the random distribution of the other parameters, we could therefore claim that this is its an optimal value. The optimal values of the other sensitive parameters were also defined by a similar principle. While influences of the sensitive parameters on the method’s performance could be recognised straightforwardly, the optimal values of the insensitive parameters have to be tuned accordingly. In our case, a model-based approach with the linear regression, as proposed in [31], was applied. The estimated optimal values of the parameters are given in Table 2. Since similar results were obtained during several additional sensitivity analy-
14
acceptable
100
100
50
50
0
0 0
Cumulative frequencies [%]
unacceptable
0.5 1 Probability of mutation (a)
0
100
100
50
50
0
0.5 1 Probability of crossover (b)
0 0
50 Size of population (c)
100
0
100
100
50
50
0
20 40 Range of coefficients (d)
0 0
100 Threshold range (e)
200
0 0.5 1 Random individuals on level 2 (f)
100
100
50
50
0
0
0 0.5 1 Proportional change of coefficients (g)
0 100 200 300 Number of evaluations on level 2 (h)
Figure 6: Cumulative frequencies associated with acceptable and unacceptable runs, with regard to (a) probability of mutation, (b) probability of crossover, (c) population size, (d) range of coefficients, (e) range of threshold, (f) percentage of random individuals on level 2, (g) proportional change of the coefficients and the threshold value on level 2, and (h) number of function evaluations on level 2.
15
increasing L
increasing K and H
increasing L, K, and H
F1 score
1
0.8
0.6 0
5
10 15 Number of terms
20
25
Figure 7: Average F1 scores achieved with the increasing complexity of the equation.
ses, we may assume that the described guidelines can be generalized and, thus, all results reported in the continuation were achieved using this setting. 4.2. The influence of equation complexity on convergence The influence of the equation complexity on the convergence of the proposed method was studied during the first test. For this purpose, a set of 100 randomly distributed nodes with random feature values were generated, while a synthetic network was constructed over them. Thus, each of the tested networks had identical node sets, while the edge sets were defined by a set of randomly defined equations with an increasing number of terms. The latter was achieved by increasing the number of considered features (i.e., L), increasing the degree of the equations (i.e., K and H), and by increasing them both. For each of the settings, 50 independent runs of the proposed EA were performed with stopping criteria set by F 1 = 1 or 50, 000 evaluations. The achieved average F1 scores with regard to equation complexity are shown in Fig. 7. As expected, by increasing the number of terms used in the definition of the edge set, the achieved average F1 scores decreased gradually (see Fig. 7) in all cases. However, the largest decrease in the method’s accuracy was observed when increasing only the number of considered features (L) and keeping the degree of the equation at the fixed value K = H = 1. This indicates that the search for the optimal equation was more difficult if all of its terms were of the first degree, as they all needed to be tuned appropriately. On the other hand, when increasing K and H, while keeping the number of considered features at the fixed value L = 1, the tuning of the high-degree terms became more important and, consequently, optimal values of the coefficients of the lowdegree terms did not need to be found. This became even more obvious when L was increased together with K and H, as increasingly more low-degree terms were introduced within the equation in comparison to those terms of the higher 16
ID 1 2 3
Equation q 4(li − lj ) − 7 (li2 − lj2 ) ≥ 120 q 2.9(li − lj ) − 20.5 (li2 + lj2 ) ≥ 86.9 q 2.7(li − lj ) + 4.9 (li2 /lj2 ) ≥ 84.5
F1 score 0.987 0.987 0.987
Table 3: Different equations that produced equivalent results.
degrees. In addition, a closer inspection of the results showed that, when considering equations of the higher degrees, the given edge set might not always be defined uniquely and several different equations may lead to the same results (see Table 3). Thus, despite the fact that the solution might be found faster by increasing the degree of the equation, the risk of finding a false positive result is also increased in this way. It is therefore important not to overestimate the degree of the equation, and to ensure the representativeness of the network under study. From a practical perspective, the representativeness of the network, and related appropriateness of the selected equation degree, can be tuned by comparing the resulting equations from multiple runs of the EA. In addition, multiple equations can be estimated and the domain specific knowledge can be applied for filtering false positives and interpreting the results. 4.3. Comparison of algorithms In order to demonstrate the value of the proposed method, a comparison was performed with several well-known machine learning (ML) algorithms (decision tree (DT), random forest (RF), support vector machine (SVM) and neural network (NN)). For this purpose, synthetic networks as described in the previous sections were considered, while OpenCV [32] implementations were used of common classification algorithms. In order to apply them in this specific case, two feature vectors were concatenated corresponding to two different nodes. A binary value was then associated with each concatenated vector, defining whether or not the corresponding nodes were linked by an edge (see Fig. 8). Although a completely fair comparison between different ML algorithms is hard to achieve due to the specifics of their purposes and designs, we ensured in this way that they all operate on the same set of values while predicting a single binary number. In other words, the performed tests indicate the capability of each of the algorithms to find the optimal ”threshold function”, acting on feature vectors of two different nodes. Moreover, sensitivity analysis as described in Section 4.1 node i
V(ni)[0] V(ni)[1]
edge
node j
...
V(ni)[L] V(nj)[0] V(nj)[1]
...
V(nj)[L]
yes/no
Figure 8: Representation of concatenated feature vectors and additional field defining the existence of an edge between two corresponding nodes.
17
EA
DT
RF
SVM
NN
F1 score
1
0.5
0 1
2
3
4
5 6 7 Number of terms
8
9
10
Figure 9: Comparison of the proposed method with other prediction methods.
has been performed for all tested ML algorithms in order to tune their parameters. For the purposes of the experiments, the obtained data was separated into learning and testing sets by the ratio 9:1, and the average results were measured of 50 independent runs. The comparison is shown in Fig. 9. As expected, the accuracy of all of the methods decreased gradually with the increasing number of terms used for the edge set definition. However, the decrease in F1 scores was considerably lower in the case of the proposed method in comparison to all of the other classical machine learning algorithms. Especially when considering non-linear equations with more than 5 terms, all of the compared algorithms failed to achieve F1 scores higher than 0.5. It is therefore obvious that these methods are not appropriate for the given task. In addition, the proposed method (2LG) was tested against 2-level nongreedy (2LN), 1-level greedy (1LG) and 1-level non-greedy (1LN) versions of the EA using the same testing procedure. The main difference between these methods is the number of levels (1 or 2) and the type of selection (greedy or non-greedy). The results are shown in Fig. 10, where it can be seen clearly that the 2-level EAs are dominant over the 1-level EAs. We can therefore confirm that separating the search for the optimal function type from the function fitting is beneficial in general, as the variation operators at the first level are done over an already tuned coefficient. In this way, we prevent a possibly beneficial change in the function type achieved at the first level being spoiled by a faulty simultaneous change in the values of coefficients. Moreover, the greedy EAs were more successful than the non-greedy EAs, as they prevented the possible spreading of bad solutions directly. Note also that all the tested EAs used the same evaluation procedure and, thus, the execution times of all were approximately the same.
18
2LG
2LN
1LG
1LN
F1 score
1 0.8 0.6 0.4 1
2
3
4
5 6 7 Number of terms
8
9
10
Figure 10: Comparison of the proposed method with other genetic algorithms.
4.4. Time complexity R WinThe proposed method was implemented in C++ under a Microsoft dows 7 operating system and the CPU execution times were measured on an R Intel CoreTM i5 CPU with 32 GB of main memory. The analysis of time complexity was done by examining the influences of different edge- and node-sets sizes on the execution times in cases of four different types of synthetic complex networks (i.e. small-world, scale-free, random, and regular [33]). Small-world networks were generated with the Barab´asi-Albert model [8], scale-free by the copy model described in [34], random networks were generated by the Erd¨osR´enyi model [35], and a lattice was used in the case of networks with regular properties [33]. Examples of the used networks can be seen in Fig. (11). The results presented in the continuation were all obtained as an average of 50 independent runs of the proposed EA with the stopping criteria given by F 1 ≥ 0.9. The first analysis was designed to identify the effectiveness of the proposed method, when applied to networks with equal numbers of nodes (namely |N | = 100) but with a different number of edges (namely |E| ∈ [100, 1000]) and different topologies. The measured execution times and the number of evaluations needed to reach the stopping criteria are shown in Fig. 12. Two conclusions can be drawn from the results. It is apparent that the number of edges on fixed number of nodes, does not influence the method’s performances (see Fig. 12a). This is an expected result, as all node-to-node comparisons need to be performed in each evaluation (i.e. construction of network), regardless of the actual number of edges. For the same reason, the execution times (see Fig. 12b) (and the number of evaluations as seen in Fig. 12c) were also not affected by the actual networks’ topologies and similar performances were measured for all four types of networks. Here, the slightly shorter execution times measured on small-world networks can be attributed to the differences in the functional definition of an edge-set that essentially defines the search space of the EA (see
19
Figure 11: Examples of used networks, where (a) small-world, (b) scale-free, (c) random, and (d) regular properties can be seen.
Section 4.2). By demonstrating that the size of the edge-set does not influence the method’s performances, subsequent analysis of the time complexity were focused on the influences of node-sets. For this purpose, the node-sets of sizes |N | ∈ [100, 1000] were used, while the obtained results are shown in Fig. 13. Here, Fig. 13a demonstrates the insensitivity of the method to the size of the note-set in terms of evaluations needed for reaching the stopping criterion. While this once more confirms that the searching space of the EA is defined only by the equation complexity, Figs. 13b and 13c show clearly that the execution time as well as the time needed to execute single evaluation of the fitness function are heavily influenced by the size of the node-set. In fact, as all pairs of nodes need to
20
Number of function evaluations
small-world
2
·10
scale-free
random
regular
4
1.5 1 0.5 0
100
200
300
400
500 600 (a)
700
800
900 1,000
100
200
300
400
500
600
700
800
900 1,000
600
700
800
900 1,000
12
Time [s]
9 6 3
Time per function evaluation [ms]
0
(b) 1 0.75 0.5 0.25 0
100
200
300
400
500 (c)
Number of edges Figure 12: The (a) number of required evaluations, (b) execution times, and (c) times per function evaluation for small-world, scale-free, random, and regular networks with fixed number of nodes and various number of edges.
21
Number of function evaluations
small-world
2
·10
scale-free
random
regular
4
1.5 1 0.5 0
100
200
300
400
500 600 (a)
700
800
900 1,000
100
200
300
400
500
600
700
800
900 1,000
600
700
800
900 1,000
240
Time [s]
180 120 60
Time per function evaluation [ms]
0
(b) 16 12 8 4 0
100
200
300
400
500 (c)
Number of nodes Figure 13: Results for (a) number of function evaluations to reach requested F1 score, (b) execution time, and (c) time needed for single function evaluation with various number of nodes.
22
be considered during the construction of the network, the time complexity is O(|N |2 ), where |N | is the number of nodes. 4.5. Real case study As a real case example, the proposed method was applied within a study on the functional heterogeneity of beta cells present in islets of Langerhans [36]. Here, a complex network was considered for examining interactions between individual beta cells extracted from the pancreas tissue of mice. The experiment was done by first incubating the slices of pancreas tissue within a calcium dye. The cells were then stimulated by raising the level of glucose (from low to high), to which they reacted with a rapid increase in calcium levels. As a result of the calcium reacting with the dye, fluorescence light was emitted and microscopic images of this phenomenon were taken with a sampling frequency of approximately 0.5 Hz at 512 × 512 pixels. By monitoring the changes in the emitted light (i.e., pixel intensities) within the predefined regions, it was possible to observe the excitement levels of multiple cells at once. The obtained time series were used to construct two complex networks (one for low and one for high glucose levels). In this regard, the observed nuclei defined a node set of the complex network, where any pair of nodes was linked by an edge if the value of the Pearson correlation coefficient between the associated responses was above a certain threshold (i.e., 0.88 in both of our examples). More details on the construction of the complex network are given in [36] and [37], while its spatial analysis based on the proposed EA is described in the continuation. During this analysis, two complex networks were considered: CL = (NL , EL ) and CH = (NH , EH ), obtained at a low and a high level of glucose, respectively. Although both were defined by the same set (i.e., NL = NH ) of 136 nodes, their edge sets differed significantly. In order to examine the influence of the cell spatial distribution on the observed correlation in their behaviour, only spatial coordinates xi and yi were assigned to each of the nodes. The proposed EA was first applied on CL in order to search for the optimal definition of EL , i.e., functional connectivity patterns at a low level of glucose. However, during the 50 runs of the EA, the highest F1 score achieved was below 0.25, clearly indicating that there is no apparent relationship between the spatial distribution of the cells and their behaviour at low levels of glucose. On the other hand, when searching for the optimal definition of EH (obtained at a high glucose level), an average F1 score of over 0.8 was achieved in all 50 independent runs of the proposed EA. Moreover, similar equation forms were obtained in all cases. Examples of the obtained equations can be seen in Table 4. We therefore demonstrated that the functional connectivity in islets of Langerhans is largely independent on the Euclidean distance when exposed to low levels of glucose, but their behaviour becomes synchronised via calcium waves spreading throughout the islets when they are exposed to the high level of glucose. This is consistent with the results of previous studies reported in [38, 39]. A network of beta cells constructed at high glucose levels is shown in Fig. 14a. In order to improve visual examination, an islet of Langerhans was separated from the network (see Fig. 14b) and reconstructed separately in accordance 23
ID 1 2 3
Equation 4.8D(n , n ) + 11.7(x i j i − xj ) − 4.1(yi + yj ) ≥ 32.7 p 41.6 D(ni , nj ) − 36(xi − xj ) + 18.8(yi − yj ) ≥ 102.6 14.8D(ni , nj ) − 13.2(xi − xj ) + 6.3(yi − yj ) ≥ 32.2
F1 score 0.757 0.8 0.8
Table 4: Examples of estimated equations and corresponding F1 scores, where D represents the distance between nodes ni and nj .
with Eq. 2 and Eq. 3 from Table 4. The red lines in Fig. 14c indicate false positive detections, while the black lines are true positives. In this case, no false negatives were detected.
Figure 14: Beta cells and (a) a complex network constructed at a high level of glucose, where (b) represents an islet of Langerhans, and (c) is a reconstructed islet with the proposed method.
4.6. Discussion According to the results, the proposed two level EA has proved to be an efficient solution for searching the optimal functional definition of the given network’s edge-set. In comparison to other EA designs, separating the problem 24
into the search for the optimal function type and function fitting (i.e. optimization of function coefficients) has resulted in a faster convergence of the solution. From the implementation perspective, this means that improved accuracy of the solution can be achieved in a limited amount of time (i.e. when the number of evaluations is used as a stopping criterion), or a desired accuracy of the solution can be achieved faster (i.e. when stopping criterion is defined by F1 score). In all of the cases, the bottleneck in terms of time complexity has proved to be an evaluation of the fitness function, which is quadratic in regards to the size of the network’s edge-set. However, as the evaluation of a given individual is done independently from the others, multiple evaluations can be done trivially in parallel in order to mitigate the problem (i.e. the problem may be considered as embarrassingly parallel [40]). Although some traditional ML approaches might still clearly be faster, the comparison with them demonstrated their general inability to deal with composite functional definitions of edge-sets that might contain more than five terms. With the increasing popularity of the analysis based on complex networks, the proposed method is applicable in various fields additional to those already discussed in this paper. With the primary motivation to uncover (deterministic) relations between physical properties of the nodes and a given state of the network, the proposed approach is particularly adapted to studies of biological processes (e.g. networks of competition in plant communities [41] and studies of animal behaviours [42]), social networks [43], and geographical (spatially embedded) networks [44]). Although such well-defined relations might be applied for predicting the structural changes within the network when being subjected to contentiously changing properties of nodes (e.g. foreseeing changes in competition within the plant community during the growth of the plants), the proposed approach in its current form is not able to deal with the systems’ dynamic, e.g. network-based environment with the time-delays, packet dropouts, and/or quantization [45, 46, 47], and complex networks based on fuzzy dynamic models [48, 49, 50]. 5. Conclusion A two-level evolutionary algorithm for the analysis of complex networks is presented in this paper. The proposed algorithm searches for non-topological features of nodes, by which the connectivity of the complex network can be defined. In this way, not only those features that have the largest influences on the complex network are revealed, but also the intensities of their effects are estimated by the equation coefficients. While searching for the optimal solution, separating the search for the optimal function type from the actual function fitting (i.e., searching for optimal coefficients) proved beneficial. Consequently, 2-level EAs yield better results than 1-level EAs. Furthermore, the proposed method confirms that a greedy approach performs better than a non-greedy approach. According to the results, the proposed method is also superior to other machine learning algorithms, especially when the network’s topology is
25
determined by non-linear relationships between the nodes’ features. Future directions of this work are aimed at applying the proposed method within different domains, where studies of biological/ecological networks shall be in particular focus (some concrete applications are also exposed in the discussion section). In addition, adaptation of the approach to the dynamic properties of the networks will be researched from the methodological perspective. Acknowledgements This work was supported by the Slovenian Research Agency under Grants J2-6764 and P2-0041. References [1] I. Zelinka, D. Davendra, S. Roman, J. Roman, Do evolutionary algorithms dynamics create complex network structures?, Complex Systems 20 (2) (2011) 127–140. [2] F. Derudder, B. Witlox, G. Catalano, Hierarchical tendencies and regional patterns in the world city network: A global urban analysis of 234 cities, Regional Studies 37 (9) (2003) 875–886. [3] B. J. O’Roak, L. Vives, S. Girirajan, E. Karakoc, N. Krumm, B. P. Coe, R. Levy, A. Ko, C. Lee, J. D. Smith, et al., Sporadic autism exomes reveal a highly interconnected protein network of de novo mutations, Nature 485 (7397) (2012) 246–250. [4] D. Davendra, I. Zelinka, M. Metlicka, R. Senkerik, M. Pluhacek, Complex network analysis of differential evolution algorithm applied to flowshop with no-wait problem, in: Differential Evolution (SDE), 2014 IEEE Symposium on, IEEE, 2014, pp. 1–8. [5] S. Boccaletti, V. Latora, Y. Moreno, M. Chavez, D.-U. Hwang, Complex networks: Structure and dynamics, Physics reports 424 (4) (2006) 175–308. [6] D. J. Watts, S. H. Strogatz, Collective dynamics of ’small-world’ networks, Nature 393 (6684) (1998) 440–442. [7] A.-L. Barab´ asi, R. Albert, Emergence of scaling in random networks, Science 286 (5439) (1999) 509–512. [8] R. Albert, A.-L. Barab´ asi, Statistical mechanics of complex networks, Reviews of modern physics 74 (1) (2002) 47. [9] S. N. Dorogovtsev, J. F. Mendes, Evolution of networks, Advances in physics 51 (4) (2002) 1079–1187.
26
ˇ ˇ [10] K. Rizman Zalik, B. Zalik, A local multiresolution algorithm for detecting communities of unbalanced structures, Physica A: Statistical Mechanics and its Applications 407 (2014) 380–393. [11] M. E. Newman, The structure and function of complex networks, SIAM review 45 (2) (2003) 167–256. ˇ [12] K. R. Zalik, Maximal Neighbor Similarity Reveals Real Communities in Networks., Scientific reports 5 (November) (2015) 18374. ˇ [13] D. Jesenko, D. Mongus, B. Zalik, Spatially embedded complex network estimation using fractal dimension, in: Computer Science Research Conference, 2014, pp. 5–8. [14] L. d. F. Costa, F. A. Rodrigues, G. Travieso, P. R. Villas Boas, Characterization of complex networks: A survey of measurements, Advances in Physics 56 (1) (2007) 167–242. [15] H. Gibson, P. Vickers, Using adjacency matrices to lay out larger smallworld networks, Applied Soft Computing 42 (2016) 80–92. [16] S. Milgram, The small world problem, Psychology today 2 (1) (1967) 60–67. [17] M. Newman, Networks: an introduction, Oxford University Press, 2010. [18] R. Albert, H. Jeong, A.-L. Barab´asi, Error and attack tolerance of complex networks, Nature 406 (6794) (2000) 378–382. [19] P. Sondhi, Feature Construction Methods: A Survey, http://sifaka. cs. uiuc. edu, 2009. ˇ [20] M. Crepinˇ sek, S.-H. Liu, M. Mernik, Exploration and exploitation in evolutionary algorithms: a survey, ACM Computing Surveys (CSUR) 45 (3) (2013) 35:1–35:33. [21] K. Krawiec, Genetic programming-based construction of features for machine learning and knowledge discovery tasks, Genetic Programming and Evolvable Machines 3 (4) (2002) 329–343. [22] M. G. Smith, L. Bull, Genetic programming with a genetic algorithm for feature construction and selection, Genetic Programming and Evolvable Machines 6 (3) (2005) 265–281. [23] L. S. Shafti, E. P´erez, Evolutionary multi-feature construction for data reduction: A case study, Applied Soft Computing 9 (4) (2009) 1296–1303. [24] O. Dor, Y. Reich, Enhancing learning algorithms to support data with short sequence features by automated feature discovery, Knowledge-Based Systems 52 (2013) 114–132.
27
[25] J. G. Moreno-Torres, X. Llor`a, D. E. Goldberg, R. Bhargava, Repairing fractures between data using genetic programming-based feature extraction: A case study in cancer diagnosis, Information Sciences 222 (2013) 805–823. [26] J. M. Luna, J. R. Romero, S. Ventura, Grammar-based multi-objective algorithms for mining association rules, Data & Knowledge Engineering 86 (2013) 19–37. ˇ [27] M. Crepinˇ sek, S.-H. Liu, M. Mernik, Replication and comparison of computational experiments in applied evolutionary computing: Common pitfalls and guidelines to avoid them, Applied Soft Computing 19 (2014) 161–170. [28] D. M. W. Powers, Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation, Bioinfo Publications 2 (1) (2011) 37–63. [29] F.-J. Chang, J. Delleur, Systematic parameter estimation of watershed acidification model, Hydrological Processes 6 (1) (1992) 29–44. ˇ [30] D. Mongus, N. Lukaˇc, B. Zalik, Ground and building extraction from LiDAR data based on differential morphological profiles and locally fitted surfaces, ISPRS Journal of Photogrammetry and Remote Sensing 93 (2014) 145–156. [31] A. Czarn, C. MacNish, K. Vijayan, B. Turlach, R. Gupta, Statistical exploratory analysis of genetic algorithms, Evolutionary Computation, IEEE Transactions on 8 (4) (2004) 405–421. [32] G. Bradski, The OpenCV library, Dr. Dobb’s Journal of Software Tools 25 (11) (2000) 120–126. [33] M. T. Thai, P. M. Pardalos, Handbook of optimization in complex networks: theory and applications, Vol. 57, Springer Science & Business Media, 2011. [34] R. Kumar, P. Raghavan, S. Rajagopalan, D. Sivakumar, A. Tomkins, E. Upfal, Stochastic models for the web graph, in: Foundations of Computer Science, 2000. Proceedings. 41st Annual Symposium on, IEEE, 2000, pp. 57–65. [35] P. Erd¨ os, A. R´enyi, On random graphs, Publicationes Mathematicae (Debrecen) 6 (1959) 290–297. [36] R. Markoviˇc, A. Stoˇzer, M. Gosak, J. Dolenˇsek, M. Marhl, M. S. Rupnik, Progressive glucose stimulation of islet beta cells reveals a transition from segregated to integrated modular functional connectivity patterns, Scientific reports 5 (2015) 7845.
28
ˇ ˇ [37] D. Speliˇ c, J. Dolenˇsek, A. Stoˇzer, M. S. Rupnik, B. Zalik, D. Mongus, Improved analysis of membrane potential oscillations in the network of cells from islet of langerhans, in GraphiCon2013 - 23rd International Conference on Computer Graphics and Vision 23 (2013) 39–42. [38] A. Stoˇzer, J. Dolenˇsek, M. S. Rupnik, Glucose-stimulated calcium dynamics in islets of langerhans in acute mouse pancreas tissue slices, PloS one 8 (2013) e54638. ˇ ˇ [39] J. Dolenˇsek, D. Speliˇ c, M. S. Klemen, B. Zalik, M. Gosak, M. S. Rupnik, A. Stoˇzer, Membrane Potential and Calcium Dynamics in Beta Cells from Mouse Pancreas Tissue Slices: Theory, Experimentation, and Analysis, Sensors 15 (11) (2015) 27393–27419. [40] E. Cant´ u-Paz, A survey of parallel genetic algorithms, R´eseaux et syst`ems r´epartis, Calculateurs parall`eles 10 (2) (1998) 141–171. [41] S. R. Proulx, D. E. L. Promislow, P. C. Phillips, Network thinking in ecology and evolution, Trends in Ecology and Evolution 20 (6 SPEC. ISS.) (2005) 345–353. [42] A. D. M. Wilson, J. W. Brownscombe, J. Krause, S. Krause, L. F. G. Gutowsky, E. J. Brooks, S. J. Cooke, Integrating network analysis, sensor tags, and observation to understand shark ecology and behavior, Behavioral Ecology 26 (6) (2015) 1577–1586. [43] F. Villanustre, B. Furht, Social network analytics: Hidden and complex fraud schemes, in: Big Data Technologies and Applications, Springer, 2016, pp. 341–346. [44] M. Zanin, F. Lillo, Modelling the air transport with complex networks: A short review, The European Physical Journal Special Topics 215 (1) (2013) 5–21. [45] T. Wang, H. Gao, J. Qiu, A Combined Adaptive Neural Network and Nonlinear Model Predictive Control for Multirate Networked Industrial Process Control, IEEE Transactions on Neural Networks and Learning Systems 27 (2) (2015) 416–425. [46] J. Qiu, Y. Wei, H. R. Karimi, New approach to delay-dependent H∞ control for continuous-time Markovian jump systems with time-varying delay and deficient transition descriptions, Journal of the Franklin Institute 352 (1) (2015) 189–215. [47] J. Qiu, H. Gao, S. X. Ding, Recent Advances on Fuzzy-Model-Based Nonlinear Networked Control Systems: A Survey, IEEE Transactions on Industrial Electronics 63 (2) (2016) 1207–1217.
29
[48] J. Qiu, G. Feng, H. Gao, S. Member, Static-Output-Feedback H∞ Control of Continuous-Time T-S Fuzzy Affine Systems Via Piecewise Lyapunov Functions, IEEE Transactions on Fuzzy Systems 21 (2) (2013) 245–261. [49] J. Qiu, H. Tian, Q. Lu, H. Gao, Nonsynchronized robust filtering design for continuous-time T-S fuzzy affine dynamic systems based on piecewise lyapunov functions, IEEE Transactions on Cybernetics 43 (6) (2013) 1755– 1766. [50] J. Qiu, S. Member, S. X. Ding, H. Gao, Fuzzy-Model-Based Reliable Static Output Feedback H∞ Control of Nonlinear Hyperbolic PDE Systems, IEEE Transactions on Fuzzy Systems 24 (2) (2016) 388–400.
30
