Component Deployment Optimisation with Bayesian Learning Aldeida Aleti
Indika Meedeniya
Faculty of Information and Communication Technologies Swinburne University of Technology, Hawthorn, VIC 3122, Australia
Faculty of Information and Communication Technologies Swinburne University of Technology, Hawthorn, VIC 3122, Australia
[email protected]
[email protected]
ABSTRACT
1. INTRODUCTION
Implementing embedded software systems involves many important design decisions, such as finding (near) optimal component deployment architectures, that have a strong influence on the quality of the final system perceived by its users. These decisions are difficult not only because of the complexity of current systems, but also due to the large number of possible design options. An automation of the design space exploration will help to make better decisions and to reduce the time of this process. In this paper, a new method called Bayesian Heuristic for Component Deployment Optimisation (BHCDO) is proposed. BHCDO constructs solutions based on a Bayesian learning mechanism which guides the search for assignments that result in new deployment architectures with an improved quality. This is achieved by calculating the posterior probability that a particular componenthost assignment is a good decision, resulting in a high quality deployment architecture, given some observed evidence during the search. Experiments on a series of randomly generated problems show that BHCDO efficiently automates the search for component deployment design alternatives and outperforms state of the art optimisation methods.
When designing deployment architectures for embedded systems, software engineers often face problems associated with making good decisions regarding a large number of quality attributes, such as safety, availability, reliability, etc. As embedded systems get larger and more complex, e.g. integrated architectures for automotive and aerospace control applications, the task of finding even a near-optimal deployment architecture becomes unmanageable for a human. An automation of the design space exploration would help not only to make better decisions, but also to reduce the time of this process. The need for automated design space exploration has been recognised by previous research in industry [11, 48] and academia [8, 14, 26]. Some approaches employ exact methods such as Binary Integer Programming (BIP) [5], Graph Cutting [30], and Linear Programming [7]. However, in [30], the authors show that finding solutions for an unconstrained design problem is NP-hard. Even when applying constraints that restrict the possible search space, exact algorithms work only on small problem instances [33]. Indeed, the majority of the approaches in the optimisation of component deployment problem employ approximate methods. Search-Based Software Engineering (SBSE) applies metaheuristics, such as Evolutionary Algorithms [23, 45, 47, 25, 49, 2], Simulated Annealing [32], Tabu Search [34], Ant Colony Optimisation [15, 2], which find solutions from the wide range of possibilities that are sufficiently good according to a specific quality function. Allowing an optimisation algorithm to find a solution from such a wide search space enables partial or full automation of previously manual software engineering tasks, often leading to solutions that a human software engineer may not have been able to think of. The review of the related approaches shows that metaheuristics are widely used to solve the component deployment problem. All commonly used heuristics include parameters which have a significant effect on the performance of the algorithm. Recommendations for common values of these parameters usually exist in the literature. However, different parameter settings will be ideal for different problems or even different instances of the same problem. In fact, it has been shown that the parameters of metaheuristics are problem dependent [22]. There is no optimal parameter configuration that will work with every problem [22] It is common practice in the AI community to parameterise an algorithm with values found during preliminary tri-
Categories and Subject Descriptors D.2.11 [Software Architectures]; D.2.8 [Software Engineering]: Metrics; G.1.6 [Optimization]
General Terms Algorithms, Design, Performance
Keywords Component Deployment, Architecture Optimisation, Bayesian Learning
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CBSE’11, June 20–24, 2011, Boulder, Colorado, USA. Copyright 2011 ACM 978-1-4503-0723-9/11/06 ...$10.00.
11
Moreover, a considerable number of tools have been developed over the past decade to tackle the problem of finding optimal and near optimal deployment architectures with respect to different quality attributes, such as DeSi [40], ArchE [19], DAnCE [18], Palladio [6] and the RACE framework [50]. The DeSi tool, developed by Mikic-Rakic et al. [40] suggests an optimisation model for the best allocation of the components on the hardware hosts while maximising the availability of the system under certain constraints, such as the allocation of two components on the same host. The ArchE tool [4] is a design assistant that helps software architects to make decisions with respect to relevant quality attributes. ArchE contains a rule-based expert systems that search the design space with so called reasoning frameworks. Each reasoning framework supports one quality domain and currently a reasoning framework is implemented for modifiability [4]. To allow the support of other quality domains ArchE provides a well defined interface to also generate reasoning frameworks for other quality attributes [19]. The RACE framework presented by Shankaran et al. [50] and the DAnCE implementation by Deng et al. [18] addresses the needs of deployment decisions at real-time. The review of the related approaches shows that approximate optimisation methods, and in particular evolutionary algorithms, are widely and successfully used to solve complex problems in the embedded systems domain. However, these achievements are the result of careful algorithm design which involves substantial efforts in defining a problem’s representation and the selection of the algorithm parameters. These efforts are a reflection of the fact that approximate methods behave differently for different problems and even problem instances. Instead, BHCDO as proposed in this paper contributes to the solution of the component deployment optimisation problem by employing a heuristic which does not have any parameters that need to be tuned to the problem it is solving. None of the existing approaches uses a similar optimisation technique. As a result, our research complements the existing approaches and as the experimental results indicate, could help to achieve faster and better results in the component deployment optimisation problem.
als. This tuning practice, however, can be time-consuming and error-prone. It presents an even bigger challenge to interdisciplinary users of AI techniques, since successful applications are often the result of a lengthy trial-and-error procedure, whose purpose is to tune the parameters of a metaheuristic for the particular problem. This makes parameter setting a time consuming procedure and the selected parameters at the end of this process are not necessarily optimal [22]. In this paper, a new optimisation strategy called Bayesian Heuristic for Component Deployment Optimisation (BHCDO) is proposed. BHCDO is tailored to the component deployment problem and constructs solutions based on a learning strategy. The learning is accomplished by employing Bayes’ theorem which calculates the posterior probability of a new component-host assignment being a good decision given some observed evidence during the search. In other words, the belief about what is known to be ”good” (the prior probability) is updated with the information contained in the likelihood of this belief being true (the data from the new constructed solutions), calculated as conditional probabilities. Accordingly, BHCDO efficiently solves the component optimisation problem, without any prior effort in parameter tuning. A case study from the automotive industry is used to illustrate the approach. Results from experiments carried on randomly generated examples show that BHCDO performs better than two widely used optimisation methods.
2.
RELATED WORK
The influence of the deployment architecture to system quality has been observed by many authors, in both software and embedded systems domain [51, 36, 42, 44, 43]. To evaluate the quality of a system with respect to a selected deployment alternative, new models propagating hardware quality attributes to the software level and/or user interface have been introduced [5, 10, 31, 28, 29, 3, 12]. Besides the quantification of system quality taking system deployment into account, methods have been designed to automatically search the space of possible deployment options and reporting promising candidates. The majority of the approaches in the optimisation of the component deployment problem employ approximate methods [39, 31, 5, 30, 23, 16, 25, 33, 52, 37, 51, 45, 43]. Broadly speaking, these approaches can be classified according to their optimisation model and procedure. Some of the methods are aiming at any deployment satisfying predefined constraints or user requirements [33, 12, 38], others are targeting methods to find an optimal deployment or at least candidates being close to it [39, 51, 25, 8, 9], often in combination with given constraints [40, 39, 25, 9]. In [26] an optimisation model is formulated to allow the best architectural decisions, while maximising the satisfaction of the quality attributes of the system. Fredriksson et al. [25] present a framework for allocating components to real time tasks, focusing on minimising the resource usage such as CPU time and memory. They have formalised a model for components and tasks, and proposed a scheduling and optimisation algorithm with real-time analysis to ensure feasible allocation. In general, approximate algorithms used for the optimisation of embedded systems fall into two categories: heuristic [3] and meta-heuristics like genetic algorithms [13, 42], Tabu search [34] and Ant Colony Optimization (ACO) [35, 16].
3.
COMPONENT DEPLOYMENT OPTIMISATION
The notion of a deployment architecture for an embedded system refers to the allocation of software components to hardware resources (hosts) and the assignment of intercomponent communications to network links. In the following subsections, the formal annotations of the elements of the component deployment optimisation will be given in details. These will be used later on to help the description of the optimisation heuristic and the case study.
3.1
Software architecture
Let C = {c1 , c2 , ..., cn }, where n ∈ N, denote the set of software components, and let the parameters of the software architecture be given as follows: • • • •
12
Communication frequency, cf : C × C → R. Event size, es : C × C → N. Workload, wl : C → N Component memory, cmem : C → N
3.2
Hardware architecture
3.5 Constraints Not all deployment candidates d ∈ D represent feasible alternatives. For instance, placing all components to a single host is syntactically possible, but not feasible due to memory constraints. The set of constraints Ω is defined independently of the quality functions, and is together with the quality functions used by employed optimisation algorithm. Currently, we consider only two constraints Ω = {Ωmem , Ωloc }, defined below. The optimisation algorithm is however independent of the constraint definition (it checks all constraints Ω), so other constraints can be added.
Let H = {h1 , h2 , ..., hm }, where m ∈ N, denote the set of hardware resources and let the parameters of the hardware architecture be given as follows: • • • • •
Processing speed, ps : H → N Network bandwidth, nb : H × H → N. Network reliability r : H × H → R. Network delay nd : H × H → N. Host memory hmem : H → N.
• Let d−1 : H → P(C) denote the inverse relation to d ∈ D, i.e. d−1 (a) = {c ∈ C | d(c) = h}. Then the memory constraint Ωmem : D → {true, f alse} is defined as follows: cmem(c) ≤ hmem(h) Ωmem (d) = ∀h ∈ H :
3.3 Deployment architecture The deployment problem can be defined as D = {d | d : C → H}, where D is the set of all functions assigning components to hosts. Note that, since C and H are finite, D is also finite, and represents the set of all mapping candidates. A single deployment alternative is di = {(c1 , hi1 ), (c2 , hi2 ), ..., (cn , hin )} ∈ D, but for the sake of readability, we write it as di = [hi1 , hi2 , ..., hin ]. The decisions regarding the deployment architecture influence not only the functional attributes, but also the quality of the resulting system. The quality of a system is at least as important as its functionality, a fact that has been highlighted by the research community, e.g. Papadopoulos and Grante [45]. The metrics that are used to evaluate the quality of the system are commonly referred to as non-functional attributes or quality attributes, where safety, reliability, performance, energy consumption and maintainability are just a few examples. To find a good deployment for the given attributes, a large design space needs to be searched, even for a very small case. For example, in the case of 45 software components to be allocated to 20 hosts, there are 4520 different solutions in the design space. Despite the complexity of the problem, the goal is to find the best possible alternatives as quickly as possible.
c∈d−1 (h)
• The localisation constraints Ωloc : D → {true, f alse} is defined as follows: Ωloc (d) = ∀c ∈ C : (h ∈ lr(c) ⇒ d(c) = h) where lr : C → P(H), is the localisation restriction.
4.
BHCDO builds new deployment architectures by performing learnt ”good” moves through the design space. Assignments are made stochastically biased by the probability of a component-host assignment being good with respect to the quality attributes Q. A ”good” move is a componenthost assignment that produces non-dominated deployment architectures. In a maximisation problem a solution d∗ is non-dominated if there exists no other d such that Q(d) ≤ Q(d∗ ) for all objectives, and Q(d) < Q(d∗ ) for at least one objective. In other words, if there exists no other feasible deployment architecture d which would be superior in at least one quality attribute while being no worse in all other objectives of d∗ , then d∗ is said to be non-dominated. The set of all nondominated solutions D∗ is known as the non-dominated set. In each step of BHCDO, a selected component will be assigned to a host chosen by using the posterior probabilities of different component-host assignments producing a nondominated deployment architecture. The information about the probabilities of assignments being good is updated during the search. Every newly constructed solution, changes the belief of what is ”good”. In a high level, the main steps of BHCDO are as follows:
3.4 Quality attributes The quality attributes to be optimised by BHCDO are denoted as Q : D → R. To illustrate our approach we use three orthogonal quality attributes: data transmission reliability (dtr), defined as in Equation 1, communication overhead (co) following Medvidovic and Malek [39], defined as in Equation 2, and scheduling length (sl), defined as in Equation 3.
dtr =
n n
cf (ci , cj ) · r(d(ci ), d(cj ))
(1)
i=1 j=1
co =
n n
+
i=1 j=1
cf (ci , cj ) · es(ci , cj ) nb(d(ci ), d(cj )) · r(d(ci ), d(cj ))
sl =
• k solutions are created uniformly randomly to construct deployment architectures, which form the initial population. • The quality of each feasible deployment architecture is measured using the quality attributes defined in 3.4. • The non-dominated solutions are obtained. • The effect of each component-host assignment is calculated as the conditional probability of that assignment producing a non-dominated deployment, shown in Algorithm 1.
cf (ci , cj ) · nd(d(ci ), d(cj ))
i=1 j=1 n n
m
j=0 c∈d−1 (hj )
wl(c) ps(hj )
BHCDO - BAYESIAN HEURISTIC FOR COMPONENT DEPLOYMENT OPTIMISATION
(2)
(3)
13
• Every next step a solution is constructed by assigning components to hosts.
Algorithm 1: Conditional probabilities for componenthost assignments
– The assignment decision is based on the posterior probabilities calculated by Algorithm 3. – After assignments are made, conditional probabilities are updated with the new data.
1 2 3
• Among the created deployment solutions, the feasible individuals are evaluated. • If the evaluation shows that a solution dominates solutions already in the population, it is added and the dominated solutions are removed from the population.
4 5 6 7 8 9
In the next subsections, the steps of the algorithm will be explained in more details and a case study from the automotive industry will be used to illustrate the approach.
10
4.1
13
11 12
Algorithm description
14
An important aspect to component deployment optimisation is a deeper understanding of how a component-host assignment affects the quality of the overall deployment architecture. Without that understanding, it is not easy to ascertain how to construct new deployment architectures. The method used here aims at establishing the posterior probability of achieving a non-dominated deployment architecture given the usage of particular component-host assignments, denoted as a(ci , hj ). For simplicity, P (a(ci , hj )) denotes the probability of assigning component ci to host hj . P (a(ci , hj )) is the prior knowledge that a component-host assignment contributes to a non-dominated deployment. This is also called the prior probability and stands for the belief we have before any evidence becomes available. The practical application of Bayesian reasoning does not appear to be affected by the initial setting of prior probabilities [41]. Similarly, P (a∗ ) denotes the probability of assignments to produce non-dominated deployments.
15 16 17 18 19 20 21 22 23
P (a(ci , hj )|a∗ ) = end end
∗ kij k∗
;
in the quality of a deployment architecture. The next stage of BHCDO involves the decision process for the next deployment architectures, which combines conditional probabilities calculated in Algorithm 1 to predict the effect that a certain combination of component-host assignments will have on the quality of the next deployment architectures. Components that need to be assigned are selected randomly until the last one. To assign the first selected component, the conditional probabilities are used as an input to the roulette wheel selection strategy, as described in Algorithm 2. In the roulette wheel selection strategy, for the first assignment, the conditional probability is used to associate a probability of selection with each component-host assignment, i.e. a proportion of the wheel is assigned to each of the possible component-host assignment based on their conditional probability. This is achieved by dividing the conditional probability of a component-host assignment by the total conditional probabilities of all the possible assignments for that component, thereby normalising them to 1. Then a random selection is made roulette wheel fashion. While candidate component-host assignments with a higher quality will be less likely to be eliminated, there is still a chance that they may be. With this kind of fitness proportionate selection there is a chance that some weaker componenthost assignments may survive the selection process; this is an advantage, as though a solution may be not as good, it may include some component which could prove useful in some new combinations of assignments. For the assignment of the next components posterior probabilities are used, which incorporate conditional probabilities of possible combinations of different component-host as-
4.1.1 The calculation of conditional probabilities Initially, t deployment architectures are constructed by k total random component-host assignments. The quality of each of the t deployment architectures is measured in terms of the quality attributes that are being optimised. The set of d∗ non-dominated solutions is then obtained. The k∗ assignments that are part of the d∗ non-dominated deployments are deemed successful. The prior probability of an ∗ assignment being successful P (a∗ ) is therefore kk . The assignment of a component ci to a host hj has a prior probability of P (a(ci , hj )), calculated as the number of times the assignment was used in the t deployments kij as a proportion of all assignments k. The conditional probability of assigning component ci to host hj given the probability of an assignment succeeding P (a(ci , hj )|a∗ ) can be calculated as the ratio of the success∗ ful assignments (kij ) which use a certain component-host assignment a(ci , hj ) and the total number of successful assignments, i.e. assignments which lead to a non-dominated k∗ deployment architecture ( kij∗ ). Algorithm 1 describes this process.
4.1.2
calculate quality Q(d) of t deployment architectures; find the d∗ non-dominated deployment architectures; foreach deployment architecture d ∈ D do foreach component ci , i ∈ n do foreach host hj , j ∈ m do if component ci is assigned to host hj then increment k; increment kij ; if d ∈ D∗ then ∗ increment kij ; increment k∗ ; end end end end end ∗ P (a∗ ) = kk ; calculate conditional probabilities for each component-host assignment; foreach component ci , i ∈ n do foreach host hj , j ∈ m do k P (a(ci , hj )) = kij ;
Deploying components using posterior probabilities
In the previous subsections, the first stage of BHCDO was described, where conditional probabilities are calculated to measure the effect that each component-host assignment has
14
The posterior probabilities are then sent to the roulette wheel selection described in Algorithm 2, which in turn uses them to associate a probability of selection with each component-host assignment. The host chosen from this fitness proportionate selection is used to assign the component at hand. This procedure is repeated until all components are deployed.
Algorithm 2: Roulette wheel selection 1 2 3 4 5
6 7 8 9 10 11 12 13 14
foreach component ci , i ∈ n do sumCP = 0; foreach host hj , j ∈ m do sumCP + = P (a(ci , hj )|a∗ ); P (a(ci , hj )|a∗ ) is the conditional probability of allocation a(ci , hj ) given a solution is in non-dominated set. Initially, this the prior probability of the allocation obtained from random generations.; cumulativeij = sumCP ; end number = Random[0, sumCP ]; foreach host hj , j ∈ m do if number < cumulativeij then return a(ci , hj ) end end end
4.2
Case study
The illustration of our approach is carried out on a case study of an embedded automotive control system designed according to already published models [24, 27] and with inputs from the Australian Automotive Industry via the cooperate research centre (AutoCRC). The case study focuses on an Anti-lock Brake System (ABS). The ABS is currently used in most modern cars to minimise hazards associated with skidding and loss of control due to locked wheels during braking. Proper rotation during brake operations allows better manoeuvrability and enhances the performance of braking. The software architecture of an ABS is depicted in Figure 1 (components 0 to 8). The ABS Main Unit is the major decision making unit regarding the braking levels for individual wheels, while the Load Compensator unit assists with computing adjustment factors from the wheel load sensor inputs. Components 4 to 7 represent transceiver software components associated with each wheel, and communicate with sensors and brake actuators. Brake Pedal is the software component that reads from the paddle sensor and sends the data to the Emergency Stop Detection software module.
signments to predict the effect that a certain combination of component-host assignments will have on the quality of the next deployment architectures. In principle, a posterior probability is the probability that the hypothesis is true, i.e. the assignment is producing non-dominated solutions, given the evidence, which in our case is the conditional probability of a specific assignment producing a non-dominated deployment architecture. These steps are given in Algorithm 3. Algorithm 3: Component deployment using posterior probabilities 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
3
while not built all deployment architectures do randomly select a component to assign; use conditional probabilities as an input to the roulette wheel selection; assign the component to the selected host from the roulette wheel; while not assigned all components do randomly select the next component cnext to assign; foreach host hs , s ∈ m do ,hs )|a∗ )·P (a∗ ) P (a∗ |a(cnext , hs )) = P (a(cPnext (a(cnext ,hs )) posterior = P (a∗ |a(cnext , hs )) foreach component ci , i ∈ n do foreach host hj , j ∈ m do if component ci is assigned to host hj then post = post · P (a(ci , hj )|a∗ ) end end end end use posterior probabilities as an input to the roulette wheel selection; assign the component to the selected host from the roulette wheel; end end
Load Compensator
1 Emergency Stop Detector
7 WAC-F
0
6 WAC-R 5 WSR-F 4 WSR-R
ABS Main Unit
8 Cruise
Control 2
Brake Pedal
WAC : Wheel Actuator Controllers (Front and Rear) WSR : Wheel Sensor Readers (Front and Rear)
Figure 1: Software components and their interactions in ABS system
The Hardware Architecture comprises of a distributed set of Electronic Control Units (ECUs) having different capacities of memory, processing power, access to sensors, etc. Communication among ECUs is achieved by buses. Many types of buses can be present, having different characteristics of data rates and reliability. For example in an automotive system, a highly reliable bus may be used for important
15
Sol. 1 2 3 4 5 6
control functions, while a less reliable bus may suffice for multi-media streaming. The hardware architecture used for this case study is shown in Figure 2
Bus0 (CAN) ECU0
ECU4
ECU2
[ [ [ [ [ [
Deployment 4,5,5,5,4,0,4,0,4 4,5,5,4,4,0,5,0,4 4,2,2,4,4,0,5,0,4 4,5,5,0,4,0,4,0,4 4,2,2,0,4,0,4,0,4 4,2,2,2,4,0,4,0,4
] ] ] ] ] ]
dtr 0.936834 0.935343 0.949654 0.935417 0.949729 0.959743
co 14.12 14.06 13.35 13.75 13.04 12.98
sl 1.09 1.10 1.77 1.36 2.04 2.45
Table 2: Deployment solutions obtained from BHCDO ECU1
ECU5
ECU3
Bus1 (Front LIN)
the optimisation process by BHCDO. The second column identifies the deployment decisions. For example, in the first deployment architecture, components 0,4,6 and 9 are allocated into the fourth ECU, the second, third and fourth components are allocated into the fifth ECU, and the remaining components are allocated into ECU 0. Note that the fifth and the sixth solutions are almost the same apart from the allocation of the third component. Both solutions are considered non-dominated, which means that neither is better. Solution 5 has a better (shorter) scheduling length, whereas solution 6 is better in two quality attributes with a higher data transmission reliability and a lower communication overhead. The system architect needs to choose between these solutions which make a trade off between the quality attributes according to its preferences.
Bus2 (Rear LIN)
Figure 2: Hardware architecture of the ABS system
In the architecture specification, parameters are annotated to each software component and hardware resource as introduced in Section 3. These parameters include component event size (es), communication frequency (cf ) network bandwidth (bw) and network delay (nd), given in Table 1. Comp. cmem ID (KB) 0 512 1 128 2 128 3 256 4 128 5 128 6 128 7 128 8 128
wl (MI) 12 6 4 10 4 4 4 4 10
(a) Software Components ECU ps hmem ID (MIPS) (KB) 0 4 512 1 2 512 2 2 512 3 2 1024 4 11 1024 5 11 512 (c) ECUs
Trans cf es ci → cj (KB) 0→6 0.5 2 0→7 0.5 2 1→3 1 2 2→1 1 2 3→0 1 2 4→0 0.7 1 4→3 0.3 2 5→0 0.7 1 5→3 0.3 2 8→0 1 0 (b) Component Interactions
BUS nb nd ID (KBPS) (ms) 0 128 2 1 64 10 2 64 10
5.
EXPERIMENTS
In this section, the experiments used to validate our approach are given. In the subsection of the experimental settings, the description of the two algorithms used to compare our approach with are described. Finally, the experimental conditions are introduced together with the performance metric which measures the performance of the optimisation algorithms.
5.1
Experimental settings
BHCDO was compared to two state of the art optimisation algorithms: NSGA-II (Non-dominated Sorting Genetic Algorithm) [17] and Pareto-Ant Colony Optimisation (PACO) [20]. For both optimisation methods, identical parameter settings, as suggested in the original studies [17, 20] were used.
r 0.9 0.8 0.8
5.1.1
NSGA-II
NSGA-II is an Evolutionary Algorithm which has been demonstrated as one of the most efficient algorithms for multi-objective optimization on a number of benchmark problems [17]. A brief description of the basic components of NSGA-II will be given in this section. The NSGA-II algorithm and its detailed implementation procedure can be found in [17]. NSGA-II uses non-dominated sorting for fitness assignments. All individuals that are not dominated by any other individuals, are assigned to front number 1. All individuals only dominated by individuals in front number 1 are assigned front number 2, and so on. A mating pool is then created with solutions selected from the population according to the ranks that has been assigned to them during the ranking process. The solutions with a lower rank have a higher chance of being selected to be part of the mating pool than the ones with a higher rank value. This helps the quick convergence of the algorithm towards
(d) Buses
Table 1: Parameters of software and hardware elements Even if the number of components in the system is not very large, the size of the search space of all deployment candidates before constraint checking is 96 (≈ 5.3 · 105 ), which is already too large to be traversed with exact algorithms. In this case study, we employ BHCDO as described in Algorithm 3, to complete the optimisation task. The execution of the algorithm was limited to 20 000 candidate evaluations, and performed on a duo-core 2.26 GHz processor computer. The algorithm started with an initial population of 50 random individuals, and finished reporting 6 non-dominated solutions in less than 1 minutes (45 seconds). Table 2 depicts six the non-dominated solutions (nondominated deployment candidates), reported at the end of
16
0.982
0.984
0.987
0.983
0.986
BHCDO NSGA-II P-ACO
0.976
0.974
0.982
0.985 0.984
BHCDO NSGA-II P-ACO
0.983 0.982 0.981
Average Hypervolume
0.978
Average Hypervolume
Average Hypervolume
0.98
0.988
0
2000
4000
6000
8000
10000
0.978
BHCDO NSGA-II P-ACO
0.978
0.976
0.979 0.97
0.98 0.979
0.977
0.98
0.972
0.981
0
2000
4000
Solution Evaluations
6000
8000
10000
0.975
0
2000
4000
Solution Evaluations
(a) 25 Components, 10 Hosts
6000
8000
10000
Solution Evaluations
(b) 50 Components, 20 Hosts
(c) 70 Components, 30 Hosts
0.990
0.983
0.989
0.982 0.981 0.980 0.979 0.978 0.977
0.986
Hypervolume indicator
0.984
Hypervolume indicator
Hypervolume indicator
Figure 3: Average hypervolume growth of the three randomly generated deployment problems
0.988 0.987 0.986 0.985 0.984 0.983 0.982
0.976
BHCDO
NSGA-II
P-ACO
Optimisation scheme
(a) 25 Components, 10 Hosts
0.985
0.984
0.983
0.982
0.981
BHCDO
NSGA-II
P-ACO
BHCDO
Optimisation scheme
NSGA-II
P-ACO
Optimisation scheme
(b) 50 Components, 20 Hosts
(c) 70 Components, 30 Hosts
Figure 4: Boxplots of the 30 trials of the three problem instances using the three different optimisation schemes.
Hence, results concerning the performance of approximate algorithms such as Evolutionary Algorithms, are usually reported as mean values over repeated trials. For the current comparison, all algorithms were granted 10 000 function evaluations per trial, repeating the trials 30 times for each optimisation scheme and problem instance. The allocation of 10 000 is based on initial experiments. Although there are indications that all algorithms still make small but steady improvements after this number of evaluations. Each trial is selected from randomly generated problem instances from component deployment problems with different sizes and characteristics. The configurations of the three problem instances are as follows; 25 components to 10 hosts (h10 c25), 50 components to 20 hosts (c20 h50) and 70 components to 30 hosts (c70 h30).
the optimal solutions. The mating pool will then serve for the random selection of the individuals to reproduce using the genetic operators, i.e. crossover and mutation operators.
5.1.2
P-ACO
Among the stochastic approaches, ACO is a commonly used constructive algorithm which builds every solution component by component. Devised over 10 years ago by Dorigo [21], it has been applied to a wide variety of problems. Due to its constructive nature, it can be expected to perform well on highly constrained problems. For the application to the deployment problem, the Pareto Ant Colony Optimisation (P-ACO) [20] based on a successful variation of ACO, Ant Colony System (ACS) was chosen. Doerner [20] devised the P-ACO algorithm for the constrained portfolio optimisation problem. The approach keeps a population of solutions which updates the pheromone maps for every objective. Pheromones for different objectives may be - and most often are - updated according to different solutions which perform well by the relevant objective. For a more detailed description of the algorithm see [20].
5.2
5.3
Performance metric
Different metrics for measuring the performance of multiobjective optimisation methods exist. Zitzler et al. [54] provide a comprehensive review of quality indicators for multiobjective optimisation, finding that many commonly used metrics do not reliably reflect the performance of an optimisation algorithm. One of the few recommended metrics is the hypervolume indicator, which measures the hypervolume between the approximate solutions and a fixed reference point in the result space. In the case of maximisation problems, the most intuitive reference point is zero in each
Experimental conditions
Approximate algorithms are not expected to deliver exact and repeatable results, but to provide good approximate solutions where exact approaches cannot be devised.
17
Table 3: The means, standard deviations and the KS-test values of hypervolume indicators for the 30 runs for each optimisation scheme and problem instance (for example in h10 c25, h stands for hardware hosts and c stands for software components).
Problem h10 c25 h20 c50 h30 c70
BHCDO 0.981 0.987 0.984
Mean NSGA 0.980 0.986 0.982
PACO 0.979 0.984 0.982
Standard Deviation BHCDO NSGA PACO 9.03E-04 6.58E-04 1.07E-03 5.36E-04 4.95E-04 8.03E-04 3.82E-04 3.54E-04 4.78E-04
ks-test vs. PACO d p 0.900 0.000 1.000 0.000 0.950 0.000
Deployment Optimisation (BHCDO) was introduced, which automatically finds solutions for the component deployment problem. BHCDO is a constructive heuristic, which build deployment architectures by using a Bayesian learning mechanism. The learnt information was used to guide the search for assignments that result in non-dominated deployment architectures. This was achieved by determining the likelihood of a particular component-host assignment being a good decision given some observed evidence during the search. The main benefit of this approach is the ability to handle the large search space of component deployment optimisation problem, by automating the identification of good architecture design alternatives. Additionally, in contrast to other approximate optimisation methods, BHCDO does not require any parameter tuning to have a good performance. With respect to the performance of BHCDO, results from experiments on a case study from the automotive industry and randomly generated examples indicated that this heuristic exhibited a better performance than state of the art optimisation algorithms. For the comparison with other optimisation algorithms and the use of additional evaluation models our tool Archeopterix [1] was used as the experimental platform, where BHCDO was implemented. The tool and experimental results can be found at http://mercury.it. swin.edu.au/g_archeopterix/experiments/CBSE2011. It is a further plan to generalise this method for any type of assignment problem.
dimension, which is the value used for our results. For a detailed description of hypervolume indicator see [53].
6.
ks-test vs. NSGA d p 0.750 0.000 0.850 0.000 0.950 0.000
RESULTS
All 30 trial outcomes for each optimisation scheme and component deployment problem instance are used to establish the fitness function using hypervolume indicator. For each problem instance and optimisation scheme, the average hypervolume growth of the 30 runs during 10 000 function evaluations is plotted in Figure 3. Averages of 30 runs are reported every 500 function evaluation. During the whole time BHCDO reports better values of the hypervolume indicator. Empirical distributions of the results are presented as boxplots in Figure 4. This is for a visual representation of how the three optimisation schemes perform with respect to each other. As it can be seen from the boxplots, the fitness function of our method is always better than the other optimisation schemes. We can see that the distribution of the data is not always normal. In some cases we get a wider peak, such as the results from PACO in Figure 4a, whereas in other cases the kurtosis (peakedness) is smaller, such as results form NSGA II in Figure 4b. However, this representation is clearly insufficient, and means and standard deviations of the solutions are presented, grouped in Table 3. Results show that for every problem instance BHCDO produces better results than NSGA II and PACO. As BHCDO clearly outperforms the two other optimisation schemes, to check for a statistical difference, the optimised results are validated using the Kolmogorov-Smirnov nonparametric test (ks) [46]. The 30 hypervolume indicators of the repeated trials for each of the three problem instances were submitted to the ks analysis. BHCDO was compared to both optimisation algorithms, i.e. NSGA-II and PACO, with a null hypothesis of a significant difference between the performances (BHCDO vs. NSGA-II and BHCDO vs. PACO). The results of the ks are shown in Table 3. All ks tests, used for establishing differences between independent datasets under the assumption that they are not normally distributed, result in a confirmation of the null hypothesis with a d-value of at least 0.750 at 100% confidence level. Hence we conclude that the superior performance of BHCDO is statistically significant. The advantage of BHCDO over the other algorithms grows with the increasing complexity of the problem. When solving problems with more components and hosts, BHCDO finds results with better fitness.
8. ACKNOWLEDGEMENT This original research was proudly supported by the Commonwealth of Australia, through the Cooperative Research Centre for Advanced Automotive Technology (projects: C4509 Dependability optimization at an architectural level of system design, and C4-501 Safe and Reliable Integration and Deployment Architectures for Automotive Software Systems).
9.
REFERENCES
[1] A. Aleti, S. Bj¨ ornander, L. Grunske, and I. Meedeniya. ArcheOpterix: An extendable tool for architecture optimization ofAADL models. In Model-based Methodologies for Pervasive and Embedded Software (MOMPES), pages 61–71. ACM and IEEE Digital Libraries, 2009. [2] A. Aleti, L. Grunske, I. Meedeniya, and I. Moser. Let the ants deploy your software - an ACO based deployment optimisation strategy. In Automated Software Engineering, ASE’09, pages 505–509. IEEE Computer Society, 2009. [3] I. Assayad, A. Girault, and H. Kalla. A bi-criteria scheduling heuristic for distributed embedded systems
7. CONCLUSION In this paper, a new Bayesian Heuristic for Component
18
[4]
[5]
[6]
[7]
[8]
[9] [10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
under reliability and real-time constraints. In Dependable Systems and Networks, pages 347–356. IEEE Computer Society, 2004. F. Bachmann, L. J. Bass, M. Klein, and C. P. Shelton. Experience using an expert system to assist an architect in designing for modifiability. In 4th Working IEEE/IFIP Conference on Software Architecture (WICSA 2004), pages 281–284. IEEE Computer Society, 2004. M. C. Bastarrica, A. A. Shvartsman, and S. A. Demurjian. A binary integer programming model for optimal object distribution. In OPODIS, pages 211–226. Hermes, 1998. S. Becker, H. Koziolek, and R. Reussner. The palladio component model for model-driven performance prediction. Journal of Systems and Software, 82(1):3–22, 2009. L. Benini, G. D. Micheli, E. Macii, M. Poncino, and S. Quer. Power optimization of core-based systems by address bus encoding. IEEE Trans. VLSI Syst, 6(4):554–562, 1998. T. Blickle, J. Teich, and L. Thiele. System-level synthesis using evolutionary algorithms. pages 23–58, 1998. T. Blickle, J. Teich, and L. Thiele. System-level synthesis using evolutionary algorithms. 2002. E. Bondarev, P. de With, M. Chaudron, and J. Musken. Modelling of input-parameter dependency for performance predictions of component-based embedded systems. In 31st EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO-SEAA), pages 36–43. IEEE Computer Society, 2005. M. Broy. Challenges in automotive software engineering. In International Conference on Software Engineering (ICSE’06), pages 33–42. ACM, 2006. R. Calinescu and M. Kwiatkowska. Using quantitative analysis to implement autonomic it systems. In International Conference on Software Engineering, ICSE, pages 100–110. IEEE, 2009. D. W. Coit and A. E. Smith. Reliability optimization of series-parallel systems using a genetic algorithm. IEEE Transactions on Reliability, 45(2):254–260, 1996. V. Cortellessa, F. Marinelli, and P. Potena. Automated selection of software components based on cost/reliability tradeoff. In Software Architecture,(EWSA’06), volume 4344, pages 66–81. Springer, 2006. M. J. Csorba and P. E. Heegaard. Swarm intelligence heuristics for component deployment. In F. A. Aagesen and S. J. Knapskog, editors, Networked Services and Applications - Engineering, Control and Management, 16th EUNICE/IFIP WG 6.6 Workshop, EUNICE 2010, Trondheim, Norway, June 28-30, 2010. Proceedings, volume 6164 of Lecture Notes in Computer Science, pages 51–64. Springer, 2010. M. J. Csorba, P. E. Heegaard, and P. Herrmann. Cost-efficient deployment of collaborating components. In Distributed Applications and Interoperable Systems, volume 5053 of LNCS, pages 253–268, 2008. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
19
algorithm:NSGA-II. IEEE-Evolutionary Computation, 6:182–197, 2002. G. Deng, J. Balasubramanian, W. Otte, D. C. Schmidt, and A. S. Gokhale. Dance: A qos-enabled component deployment and configuration engine. In Component Deployment, volume 3798 of LNCS, pages 67–82. Springer, 2005. A. D´ıaz Pace, H. Kim, L. Bass, P. Bianco, and F. Bachmann. Integrating quality-attribute reasoning frameworks in theArchE design assistant. In Quality of Software-Architectures, QoSA’08, volume 5281 of LNCS, pages 171–188. Springer, 2008. K. Doerner, W. J. Gutjahr, R. F. Hartl, C. Strauss, and C. Stummer. ParetoAntColonyOptimization:AMetaheuristicApproach toMultiobjectivePortfolioSelection. Annals of Operations Research, 131(1–4):79–99, 2004. M. Dorigo and GianniDi Caro. The ant colony optimization meta-heuristic. In New Ideas in Optimization, pages 11–32. McGraw-Hill, 1999. A. E. Eiben, R. Hinterding, and Z. Michalewicz. Parameter control in evolutionary algorithms. IEEE Transations on Evolutionary Computation, 3(2):124–141, 2007. M. Eisenring, L. Thiele, and E. Zitzler. Conflicting Criteria in Embedded System Design. IEEE Design and Test, 17(2):51–59, 2000. J. Fredriksson, T. Nolte, M. Nolin, and H. Schmidt. Contract-based reusable worst-case execution time estimate. In The International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), pages 39–46, 2007. J. Fredriksson, K. Sandstr¨ om, and Mikael˚ Akerholm. Optimizing resource usage in component-based real-time systems. In Component-Based Software Engineering, volume 3489 of LNCS, pages 49–65. Springer, 2005. L. Grunske. Identifying ”good” architectural design alternatives with multi-objective optimization strategies. In International Conference on Software Engineering, ICSE, pages 849–852. ACM, 2006. L. Grunske. Towards an Integration of Standard Component-Based Safety Evaluation Techniques with SaveCCM. In QoSA 2006,, volume 4214 of LNCS, pages 199–213. Springer, 2006. L. Grunske, B. Kaiser, and R. H. Reussner. Specification and evaluation of safety properties in a component-based software engineering process. In Embedded Software Development with Components -An Overview on Current Research Trends, pages 737–738. Springer-Verlag, 2005. L. Grunske, P. A. Lindsay, E. Bondarev, Y. Papadopoulos, and D. Parker. An outline of an architecture-based method for optimizing dependability attributes of software-intensive systems. In Workshops on Software Architectures for Dependable Systems, WADS IV, volume 4615 of Lecture Notes in Computer Science, pages 188–209. Springer, 2006. G. C. Hunt and M. L. Scott. The coign automatic distributed partitioning system. In Proceedings of the
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41] [42]
[43]
3rd symposium on Operating Systems Design and Implementation, pages 187–200. USENIX Assoc, 1999. S. Islam, R. Lindstrom, and N. Suri. Dependability driven integration of mixed criticalitySW components. In International Symposium on object/component/service-oriented Real-time distributed Computing, ISORC, pages 485–495. IEEE Computer Society, 2006. S. Islam and N. Suri. A multi variable optimization approach for the design of integrated dependable real-time embedded systems. In Embedded and Ubiquitous Computing, International Conference, Proceedings, volume 4808 of LNCS, pages 517–530. Springer, 2007. T. Kichkaylo and V. Karamcheti. Optimal resource-aware deployment planning for component-based distributed applications. In HPDC:High Performance Distributed Computing, pages 150–159. IEEE Computer Society, 2004. S. Kulturel-Konak, D. W. Coit, and F. Baheranwala. Pruned pareto-optimal sets for the system redundancy allocation problem based on multiple prioritized objectives. Journal of Heuristics, 14(4):335–357, 2008. Y.-C. Liang and A. E. Smith. An ant colony optimization algorithm for the redundancy allocation problem (RAP). IEEE Transactions on Reliability, 53(3):417–423, 2004. S. Malek. A User-Centric Approach for Improving a Distributed Software System’s Deployment Architecture. PhD thesis, Faculty of the graduate schools, University of Southern California, 2007. S. Malek, M. Mikic-Rakic, and N. Medvidovic. A decentralized redeployment algorithm for improving the availability of distributed systems. In Comp. Dep., volume 3798 of LNCS, pages 99–114. Springer, 2005. A. Martens and H. Koziolek. Automatic, model-based software performance improvement for component-based software designs. In Proceedings of the Sixth International Workshop on Formal Engineering approches to Software Components and Architectures (FESCA 2009), volume 253 of Electronic Notes in Theoretical Computer Science, pages 77 – 93. Elsevier, 2009. N. Medvidovic and S. Malek. Software deployment architecture and quality-of-service in pervasive environments. In Workshop on the Engineering of Software Services for Pervasive Environements, ESSPE, pages 47–51. ACM, 2007. M. Mikic-Rakic, S. Malek, N. Beckman, and N. Medvidovic. A tailorable environment for assessing the quality of deployment architectures in highly distributed settings. In Component Deployment,CD’04, volume 3083 of LNCS, pages 1–17. Springer, 2004. R. E. Neapolitan. Learning Bayesian Networks. Pearson Prentice Hall, 2004. M. Nicholson. Selecting a Topology for Safety-Critical Real-Time Control Systems. PhD thesis, Department of Computer Science, University of York, 1998. A. D. Pace, H. Kim, L. Bass, P. Bianco, and F. Bachmann. Integrating quality-attribute reasoning frameworks in the archE design assistant. In Quality
[44]
[45]
[46]
[47]
[48]
[49]
[50]
[51]
[52]
[53]
[54]
20
of Software Architectures, QoSA’08, volume 5281, pages 171–188. Springer, 2008. Y. Papadopoulos and C. Grante. Techniques and tools for automated safety analysis& decision support for redundancy allocation in automotive systems. In COMPSAC 2003, pages 105–110. IEEE Computer Society, 2003. Y. Papadopoulos and C. Grante. Evolving car designs using model-based automated safety analysis and optimisation techniques. The Journal of Systems and Software, 76(1):77–89, 2005. A. N. Pettitt and M. A. Stephens. The kolmogorov-smirnov goodness-of-fit statistic with discrete and grouped data. Technometrics, 19(2):205–210, 1977. A. D. Pimentel, C. Erbas, and S. Polstra. A systematic approach to exploring embedded system architectures at multiple abstraction levels. IEEE Transaction Computers, 55(2):99–112, 2006. A. Pretschner, M. Broy, I. H. Kr¨ uger, and T. Stauner. Software engineering for automotive systems:Aroadmap. In International Conference on Software Engineering,ISCE’07, pages 55–71, 2007. D. Salazar, C. M. Rocco, and B. J. Galvan. Optimization of constrained multiple-objective reliability problems using evolutionary algorithms. Reliability Engineering & System Safety, 91(9):1057–1070, 2006. N. Shankaran, J. Balasubramanian, D. C. Schmidt, G. Biswas, P. J. Lardieri, E. Mulholland, and T. Damiano. A framework for (re)deploying components in distributed real-time and embedded systems. In Proceedings of the 2006ACM Symposium on Applied Computing (SAC), pages 737–738, 2006. V. S. Sharma, P. Jalote, and K. S. Trivedi. Evaluating performance attributes of layered software architecture. In Component-Based Software Engineering, 8th International Symposium, CBSE, volume 3489 of LNCS, pages 66–81. Springer, 2005. R. Zhao and B. Liu. Redundancy optimization problems with uncertainty of combining randomness and fuzziness. European Journal of Operational Research, 157(3):716–735, 2004. E. Zitzler, D. Brockhoff, and L. Thiele. The Hypervolume Indicator Revisited:On the Design of Pareto-compliant Indicator Via Weighted Integration. In Evolutionary Multi-Criterion Optimization, EMO’07, volume 4403 of LNCS, pages 862–876. Springer, 2007. E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and da V. G. Fonseca. Performance assessment of multiobjective optimizers: an analysis and review. IEEE Tran. on Evolutionary Comp., 7:117–132, 2003.
