Hierarchical Criterion Approach for Architecture ... - ScienceDirect

Available online at www.sciencedirect.com

ScienceDirect Procedia Engineering 178 (2017) 233 – 242

16thConference on Reliability and Statistics in Transportation and Communication, RelStat’2016, 19-22 October, 2016, Riga, Latvia

Hierarchical Criterion Approach for Architecture Structure Selection of Transportation Software Sergey Orlov, Andrei Vishnyakov* Transport and Telecommunication Institute, Lomonosova str.1, Riga, LV-1019, Latvia

Abstract Architectural structural design of modern transportation software has a huge impact on a quality and cost of the development process, hence it’s required to pay reasonable attention while making architecture design decisions. The technique proposed in this paper allows selecting the optimal software architecture among several alternatives. This selection technique is reduced to the criteria importance theory for decision-making problems with a hierarchical criterion structure. For applying it, we need to pick up a set of metrics that assess the characteristics of the software architecture. Next, we need to determine metrics scale and create the hierarchical criterion structure with all the relations between software metric groups. © Published by by Elsevier Ltd.Ltd. This is an open access article under the CC BY-NC-ND license © 2017 2017The TheAuthors. Authors.Published Elsevier (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility ofthe scientific committee of the International Conference on Reliability and Statistics in Peer-review underand responsibility of the scientific committee of the International Conference on Reliability and Statistics in Transportation Communication. Transportation and Communication Keywords: multicriteria decision analysis, hierarchical criterion, criteria importance theory, software architecture, architectural patterns

1. Introduction and related work The formation of architecture is the first and fundamental step in the software design process and provides the framework of a software system that can perform the full range of detailed requirements (Orlov, 2016; Bass et al., 2013).

* Corresponding author. E-mail address: [email protected]

1877-7058 © 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the International Conference on Reliability and Statistics in Transportation and Communication

doi:10.1016/j.proeng.2017.01.104

234

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

Most of the existing techniques for constructing software architecture are not well formalized and are usually not based on any mathematical theory (Bass et al., 2013). Therefore, the problem of software architecture selection and analysis based on quantitative evaluation is very important. The analysis of architecture enables early prediction of a system’s qualities. In other words, it would be desirable to have a formalized technique that is based on mathematical theory, and which allows the user to analyse and make decisions when choosing software architecture or its components. Several techniques have been proposed to assist software architects in making architecture decisions (Aleti et al., 2013; Falessi et al., 2011). There are several groups of such techniques, where some of them focused on architecture trade-off analysis, quality evaluation model analysis, performance optimization and some others well-known techniques (Aleti et al., 2013; Falessi et al., 2011). Some other studies propose the usage of the criteria of efficiency and the architecture efficiency metrics for quantitative evaluation of a software architecture structure (Orlov and Vishnyakov, 2010; Orlov and Vishnyakov, 2014). The disadvantage of this method is that the components of the architecture efficiency metrics are explicitly defined, and we cannot easily extend them to reflect the required software architecture features. In this paper, we propose a technique that allows us to make architectural decisions when creating structure of a software architecture using set of architectural patterns. In other words, this technique allows us to choose the best structural organization of the software architecture that is build using the architectural patterns. The proposed technique is based on the so-called criteria importance theory for decision making problems with a hierarchical criterion structure (Podinovski, 1994; Podinovski and Podinovskaya, 2014). It allows decisions to be made when choosing a software architecture system from among several alternatives and lacks the disadvantages that exist with other methods. 2. Model definition for software architecture selection To create the technique for selectingthe optimal software architecture, we build a model based on criteria importance theory for decision-making problems with a hierarchical criterion structure (Podinovski and Podinovskaya, 2014). To applying it, we need to pick a set of metrics that assess the characteristics of the software architecture. These metrics are then used to build a hierarchical criterion structure. Next, we need to determine the metrics scale and information about their importance. The mathematical model of individual decision making for multiple criteria includes the following components: • set of alternative software architectures X; • vector criterionf; • preference and indifference relations of the decision maker (DM), which are denoted as Р (preference) and I (indifference). Each alternative x from the set of alternatives X is characterized by a number of criteria fi,i = 1, ...,m, which are called particular criteria. The ordered set of such criteria forms a vector criterion f = (f1, …,fт).The criterion fi is the function defined on X and taking its values from zi which is called a common scale (or number of assessments of such criterion). According to the standardsoftware engineering terminology, we callparticular criteria metrics. Assume that we have a number different of software architectureoptions. For example, the architecture of some transport system can be implemented based on Service-oriented pattern; alternative architecture could be based on a Multitier pattern; another with the use of a Microservices architectural pattern and so on. We define a set of such alternative software architectures as X = {Xi ⎜i = 1, …,n}. Let us assume that all alternative metrics are homogeneous, i.e. they are all measured using the same scale and have the same range defined as Z0. Suppose that the number of scale gradationsis finite, then: Z0 = {1, …,q}, where q> 1. In other words, each metric fi from the set of alternative software architectures X can take the values from the set of scale gradations Z0. Assume that all estimates are expressed in numerical form and higher values are preferableto smaller ones. Thus, each software architecture alternative Xi characterized by values fi (Xi) of every metric and forms its vector estimate y = f(Xi)= (f1(Xi), ..., fт(Xi)). Alternative software architectures are compared by comparing their vector estimates. The set of all possible vector estimates is defined asZ = Z0m.

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

The obtained individual metrics are used as part of a hierarchical structure, an example of which is shown on Fig. 1. We build our hierarchical structure using a “bottom-up” approach, where on the bottom level (level 3) we have eleven individual metrics, which are denoted as f1,…,f11. In the next step, we need to combine the individual metrics into groups so that their logical grouping allows aggregated metrics to be obtained. Therefore, at level two the individual metrics are grouped into disjoint sets and    ,  ,    . form new vector criteria. In our example, the metrics related to this level are denoted as ,    Similar operations are performed at level 1, so we get two vector criteria:  and    . Finally, at the top level (level 0) we get the single vector criterion   . It is worth noting that each intermediate vector criterion should have a clear meaning. For example, if criterion   determines the domain-orientation, then metrics ,  ,  and  must represent some components of domainorientation.

Fig. 1. Five-level hierarchy and the corresponding coefficients of importance α.

In addition, we need to determine the coefficients of importance for each vector criterion (with the exception of the top-level vector criterion). To calculate the coefficients of importance of the lower lever vector criterion, we  should consider the coefficient of importance of corresponded higher-level vector criterion. For example, value   depends on  . To obtain the coefficients of importance, we first need to get qualitative and quantitative information about the importance of the metrics. The fact that the metric fi is equally important to metric fj is denoted as fi ≈ fj. A vector estimate that includes such metrics has indifferent preference:    ,where yij — vector estimate, which is obtained from vector y by replacing its components yi and yj. The fact that metric fi from one group is more important than metric fj from another group is denoted as fi‫ظ‬fj. In that case, for the pair of vector estimates y and yij, the DM prefers the first one to the second one, and it is denoted as yP0yij. In addition to this, it is necessary to be able to quantitatively indicate how many times the metrics from one group are more important than the metrics from the other group. To do so, we define the matrix of degrees of importance superiority H = ||hij||, where i, j = 1, ...,m. When using it the preference relation is written as follows: ೔ೕ fi‫ ظ‬fj, which means that metric fi is hij times as important as metric fj. The set of relations fi ≈ fj and fi‫ظ‬fj for the metrics used forms the qualitative information about the metrics ೔ೕ importance Ω. On the contrary, the set of relations fi‫ ظ‬fj forms the quantitative information about the metrics importance Θ.

235

236

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

For our problems with the hierarchical structure, we need to use the degree of importance superiority applicable to groups of criteria. One of the approaches is to create a relevant NT-model (Podinovski, 1994) based on our quantitative information. To create the NT-model, we should get the coefficients of group importance, and based on these we get the coefficients of importance of the individual scalar criteria and the final importance coefficients. Since all the criteria in such model are of equal importance, we can reduce the problem to a general criteria importance theory problem. We can also consider vector criteria of different levels as groups of component scalar criteria (Podinovski and Podinovskaya, 2014).Thus, we can consider the level 0 criterion as a metric for selection of the optimal software architecture, i.e. getting the value of this metric for all the alternatives to determine the optimal solution. To apply that approach, we should (using quantitative information on the criteria importance) getthe final coefficients of importance and then use a suitable decision rule. 3. Model The software architecture selection technique consists of the following steps: 1. Selection of software architecture metrics; 2. Organization of the selected metrics into the software architecture metrics hierarchy; 3. Unification and normalization of the selected metrics; 4. Creation of the model based on criteria importance theory for decision-making problems with a hierarchical criterion structure. This step also includes the definition of all relations between software metric groups and definition of all the coefficients of importance; 5. Selection of the alternative software architectures for the required software system; 6. Obtaining the values of each metric for every software architecture, i.e. formation of vector criteria; 7. Solving the problem of criteria importance theory for decision-making problems with a hierarchical criterion structure. 3.1. Metric selection For the formation of the vector criteria, it is necessary to define a set of metrics that will be used. For a comprehensive evaluation of the architecture, we define the following individual metrics: Inverted Architecture Functional Points; Architecture Stability; Architecture Relational Cohesion; Architecture Suitability; Architecture Simplicity; Architecture Responsibility; Architecture Maturity; Architecture Extensibility; Architecture Scalability; Architecture Interoperability; Architecture Availability. These individual metrics are grouped in the hierarchical structure and define new aggregated metrics, including: Structural Quality (Architecture Size; Architecture Links); System Characteristics (Domain Characteristics; Agility). All these metrics form Architecture Quality metric. The complete hierarchy of the software metrics is represented in Fig. 2. 3.2. Architecture size metrics For architecture size metrics, we consider the metrics that directly or indirectly measure the size of a software system.One of metrics which indirectly measure the software complexity is the Functional Points metric (Orlov, 2016; International Function Point Users Group, 1999; Orlov and Vishnyakov, 2014). The input data for this metric are obtained using system requirements. Since the application of each architectural pattern has its own features, impacts and requires a different amount of effort, it is advisable to consider the software architecture impact for each separate component. Thus, the metric is defined as follows:

⎛ AFP = UFP × ⎜⎜ 0.65 + 0.01× ⎝

⎞

14

∑CF ⎟⎟⎠ , i

i =1

(1)

237

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

where UFP – Un adjusted Function Point count; CFi – defined as  ሺ   ൈ  ǡ ͷሻ, where: Fi – degree of influence coefficient from FP metric; and ci– influence of architectural pattern on the i-th system's characteristic.

Fig. 2. Software architecture metrics hierarchy.

To get the civalues, first, we need to evaluate a characteristic using the scale represented in Table 1 and then get the corresponding ci value. Table 1. Characteristics evaluation scale correspondence to ci value. Score As

Description

ci

1

use of a pattern reduces the significance of a system characteristic

½

2

use of a pattern slightly reduces the significance of a system characteristic

⅔

3

no influence

1

4

use of a pattern slightly actualizes a system characteristic

1½

5

use of a pattern actualizes a system characteristic

2

The Architecture Functional Points metric must be normalized, and its value must be proportional to the preference scale. Therefore, we define replace it by the Inverted Architecture Functional Points: AFP'i = 1 −

AFPi , AFPmax

(2)

where: AFPi – Architecture Functional Points for the i-th architecture; AFPmax – maximum value of AFP for compared architectures. 3.3. Architecture links metrics For software architecture links, we consider the strength of the relationship between the architectural components and inside each individual component. Roughly, this is similar to the relationship between classes in Object Oriented Programming, although herewe are evaluating software architecture components. For such evaluation, we define two new metrics, which are based on principles of package design (Martin, 2003). One of the metrics is used to measure the coupling, whilethe other is used to measure the cohesion.

238

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

3.3.1. Architecture Stability (AS) Adaptation of the original instability metric(Martin, 2003; Fenton and Bieman, 2014) for architectural functional components allows us to define the following stability of the architecture’s component metric: N

AS = 1 − AI = 1 −

Cei i + Cei , N

∑ Ca i =1

(3)

where: AI – architecture instability; Cei– efferent coupling (outgoing dependencies) of the i-th architecture’s component; Cai– afferent coupling (incoming dependencies) of the i-th architecture’s component; N– number of components of the architecture. 3.3.2. Architecture Relational Cohesion(ARC) Adaptation of the Package Relational Cohesion metric (Martin, 2003; Fenton and Bieman, 2014) for architectural functional components allows us to define the following relational cohesion of the architecture’s component metric: N

ARC =

Ri + 1

∑ NPR i =1

i

N

(4)

,

where: Ri– the number of relations between classes and interfaces in the i-th architecture’s component; NPRi– the number of possible relations between classes and interfaces in the i-th architecture’s component; and N– number of components of the architecture. 3.4. System characteristics For the third group of metrics let us choose the metrics,that evaluate the architecture’s systemcharacteristics. Metrics from this group are based on international standard “Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of system and software product quality” (ISO/IEC DIS 25023, 2014; Fenton and Bieman, 2014).We split the metrics from this group into two subgroups, the architecture’s domain characteristics and architecture’s agility. We define the scale for evaluatingthe quality characteristics in the range from 0 to 1, where a low value means that this characteristic is poorly supported by the considered architecture and a high value of the characteristic indicates the opposite. 3.4.1. Domain characteristics Metrics from this subgroup evaluate the architecture’s domain-orientation. The group includes the metrics listed in Table 2. Table 2. Domain characteristics. Name

Description

Measurement function

Suitability

Proportion of components in the software architecture that is well fitted for the required domain

X = A / B, where: A – Number of components in the software architecture that is well fitted for the required domain; B – Total number of components in the software architecture

Simplicity

Proportion of components in the software architecture that have simple structure

X = A / B, where: A– Number of simple components in the software architecture; B– Total number of components in the software architecture

Responsibility

Proportion of components in the software architecture have a single responsibility

X = A / B, where: A – Number of components in the software architecture with single responsibility; B – Total number of components

Maturity

Proportion of components in the software architecture that is implemented with the reuse of known patterns

X = A / B, where: A – Number of components in the software architecture that reuse any software patterns; B – Total number of components

239

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

3.4.2. Agility Metrics from this subgroup evaluate the architecture’s flexibility. The group includes the metrics listed in Table 3. Table 3. Agility. Name

Description

Measurement function

Extensibility

Proportion of components in the software architecture that could be potentially be extended

X = A / B, where: A – Number of components in the software architecture that could be extended; B – Total number of components in the software architecture

Scalability

Proportion of components in the software architecture that could be easily scaled

X = A / B, where: A – Number of components in the software architecture that could be scaled; B – Total number of components in the software architecture

Interoperability

Proportion of components in the software architecture that is easy to integrate with other systems

X = A / B, where: A – Number of components in the software architecture that is easy to integrate with other systems; B – Total number of components in the software architecture

Availability

Proportion of components in the software architecture that is well suitable to handle large amount of data

X = A / B, where: A – Number of components in the software architecture that is well suitable to handle large amount of data; B – Total number of components in the software architecture

3.5. Preference relations for hierarchical model Let us determine the preference relations for all metrics in the hierarchical structure for each level. For level 1, we define the following relation: Structural Quality‫ظ‬Ȁ System Characteristics. Level 2 contains the relations for the following groups: Structural quality (Architecture Links ‫ظ‬Ȁ Architecture Size); System Characteristics (Domain characteristics ‫ظ‬Ȁ Agility). The bottom level is composed of the following relations: Architecture Links (Architecture Stability ‫ظ‬Ȁ Architecture Relational Cohesion);Domain Characteristics (Suitability ‫ظ‬Ȁ Simplicity; Simplicity ≈ Responsibility; Responsibility ‫ظ‬Ȁ Maturity);Agility (Extensibility ‫ظ‬Ȁ Scalability; Scalability ≈ Interoperability; Interoperability ‫ظ‬Ȁ Availability). These relations form the quantitative information (Θ) about the criteria importance; and they are used to calculate the importance coefficients of the groups for our NT-model. In our case, there are two groups at the 1st level, and four subgroups on the 2nd level.At the first level, we have determined the following relationship: 1 1 f123 f3/2 f 4567891011 .

(5)

As long as sum of the importance coefficients of one group must be equal to one, we obtain the group importance coefficients of the first level: 3 2 1 1 α {123} = , α {45678901011} = . 5 5

(6)

The second level is defined by the following relationships: 2 2 f 232 f3/2 f12 ; f 4567 f3/2 f891011 .

(7)

This allows us to get the group’s importance coefficients for the second level: 2 3 3 2 2 2 2 2 α {1} = , α {23} = , α {4567} = , α {891011} = . 5 5 5 5

(8)

240

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

Next, we need to consider the impact of the first-level importance coefficients on the coefficients of the second level. To do that, the group’s importance coefficient of the second level should be multiplied by the group’s coefficient from level 1, which includes that second-level subgroup:

6 1− 2 9 1 2 ⋅ α {23} = ; ;α{23} = α {123} 25 25 6 1− 2 4 1 2 1 2 = α {4567891011} ⋅ α {4567} = ;α {891011} = α{4567891011} ⋅ α{891011} = . 25 25

1− 2 1 2 = α{123} ⋅ α {1} = α {1}

α

1− 2 {4567}

(9)

The influence coefficients of the higher levels in the definitions above are marked by the superscript “1–2”. The third level is defined by the following relationships:

f23 f3/2 f33 ; f43 f3/2 f53 ; f53 ≈ f63 ; f63 f3/2 f73 ; f83 f3/2 f93 ; f93 ≈ f103 ; f103 f3/2 f113.

(10)

Finally, there are the following initial importance coefficients of partial criteria for the third level: 3 2 9 6 6 4 9 6 6 4 α 13 = 1, α 23 = , α 33 = , α 43 = , α 53 = , α 63 = , α 73 = , α 83 = , α 93 = ,α 103 = ,α 113 = . 5 5 25 25 25 25 25 25 25 25

(11)

Having this information allows us to obtain the final coefficients of importance. To do this, the original importance coefficients of the particular criteria are multiplied by the influence coefficients of the first- and second-level groups that include these particular criteria: 6 27 18 54 1− 2 1− 2 1− 2 ; a2 = α{23} ; a3 = α {23} ; a4 = α{4567} ; ⋅ α 23 = ⋅ α 33 = ⋅ α 43 = 25 125 125 625 36 36 24 36 1− 2 1− 2 1− 2 1− 2 a5 = α{4567} ; a6 = α{4567} ; a7 = α {4567} ; a8 = α{891011} ; ⋅ α 53 = ⋅ α 63 = ⋅ α 73 = ⋅ α 83 = 625 625 625 625 24 24 16 1− 2 1− 2 1− 2 a9 = α {891011} ⋅ α 93 = ⋅ α103 = ⋅ α113 = ; a10 = α{891011} ; a11 = α {891011} . 625 625 625 1− 2 a1 = α{1} ⋅ α13 =

(12)

After reducing to a common denominator, it looks like:

150 135 90 54 36 36 ; a2 = ; a3 = ; a4 = ; a5 = ; a6 = ; 625 625 625 625 625 625 24 36 24 24 16 ; a8 = ; a9 = ; a10 = ; a11 = . a7 = 625 625 625 625 625 a1 =

(13)

Therefore, N for the NT-model is the vector (150, 135, 90, 54, 36, 36, 24, 36, 24, 24, 16), and the dimension of the NT-estimates is equal to 625. By applying the final coefficients of importance for the NT-model we get NT-estimates for each alternative. Since all criteria in the NT-model are of equal importance, for the comparison of two vector estimates we can use any method of Criteria Importance Theory.

4. Case study To validate the proposed model, we conducted a case study for which we built several software architectures using different architectural patterns.From all possible solutions, we haveconsidered three different approaches for the creation of the architecture for such system. All these options are based on different architectural patterns that provide various capabilities and introduce specific requirements. The considered solutions include:

241

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

• A1– Architecture that is based on a Service Oriented Architecture (SOA) architectural pattern; • A2– Architecture that is based on a Multitier architectural pattern; • A3– Architecture that is based on a Microservices architectural pattern. It is necessary to obtain values for the selected metrics for the given software architectural solutions. Metrics f1, f2and f3 are evaluated based on the created software architectural models; and metrics f4 – f11 are obtained using expert evaluation. The obtained metric values are given in Table 4. Table 4. Obtained values of metrics. Notation

Metric

A1

A2

f1

AFP

1200

1050

A3 800

f2

AI

0.43

0.45

0.56

f3

ARC

0.67

0.59

0.49

f4

Suitability

0.76

0.64

0.52

f5

Simplicity

0.66

0.48

0.38 0.63

f6

Responsibility

0.76

0.52

f7

Maturity

0.62

0.76

0.42

f8

Extensibility

0.78

0.38

0.72

f9

Scalability

0.70

0.58

0.68

f10

Interoperability

0.76

0.64

0.74

f11

Availability

0.64

0.52

0.48

It is advisable to convert the obtained values to a common ordinal scale. After conversion the obtained values to a common ordinal scale, we have obtained the three following vector estimates: y1 = (1, 6, 7, 8, 7, 8, 7, 8, 7, 8, 7);y2 = (2, 6, 6, 7, 5, 6, 8, 4, 6, 7, 6);y3 = (4, 5, 5, 6, 4, 7, 5, 7, 7, 8, 5).

(14)

By applying the final coefficients of importance for the NT-model we get the following NT-estimates represented in compact form: y1NT = (1150, 6135, 7190, 8150);y2 NT = (2150, 436, 536, 6301, 778, 824);y3 NT = (4186, 5265, 654, 796, 824),

(15)

where the superscripts indicate how many times each value is included in the NT-estimate. By substituting the above-listed data into the proposed decision-making model, we get the result thatA1 is nondominated; A2 and A3 are dominated by A1; and A2is dominated by A3.

5. Conclusions The paper proposes a technique that allows selection of the optimal software architecture among several alternatives for a required domain. This selection technique is reduced to the criteria importance theory for decision making problems with a hierarchical criterion structure. We defined a model based on criteria importance theory with a hierarchical criterion structure that could be used to choose the optimal software architecture. To apply the model, we picked a set of metrics that assess the characteristics of software architecture; determined the scale for the metrics; created the hierarchical criterion structure that is required to determine the coefficients of importance. For the formation of the hierarchical criterion structure, we defined a set of metrics that includes metrics from different areas, some of which assess the size, links, domain and agility characteristics of the software architecture.

242

Sergey Orlov and Andrei Vishnyakov / Procedia Engineering 178 (2017) 233 – 242

To validate the proposed model, we conducted a case study for which we built three software architectures using different architectural patterns. For every software architecture alternative, the metrics values were evaluated and the corresponding vector estimate and NT-estimate obtained. The result of the case study proved the correctness of the proposed model, which is based on the criteria importance theory for decision making problems with a hierarchical criterion structure. During the case study, we identified the optimal software architecture for the transport system with the specified requirements. The results obtained indicate that the proposed technique is applicable for solving problems of the selection of optimal software architecture. As the next step of the research, we are planning to undertake extension of the proposed technique under uncertainty.

References Aleti, A., Buhnova, B., Grunske, L., Koziolek, A. and Meedeniya, I. (2013) Software Architecture Optimization Methods: A Systematic Literature Review, IEEE Transactions on Software Engineering, 39(5), 658-683. Bass, L., Clements, P. and Kazman, R. (2013) Software Architecture in Practice, 3rd ed. New Jersey: Addison Wesley. 608 p. Falessi, D., Cantone, G., Kazman, R.and Kruchten, P. (2011) Decision-making techniques for software architecture design: A comparative survey, ACM Computing Surveys (CSUR), 43(4),1–28. Fenton, N.and Bieman, J. (2014) Software Metrics: A Rigorous and Practical Approach, Third Edition. CRC Press.617 p. International Function Point Users Group (1999) Function Point Counting Practices Manual, International Function Point Users Group, Release 4.1, Westerville, Ohio, 335 p. ISO/IEC (2014), ISO/IEC DIS 25023:2014 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) — Measurement of system and software product quality.International Organization for Standardization, 47 p. Martin, R.C. (2003) Agile Software Development: Principles, Patterns and Practices. Prentice Hall.552 p. Orlov, S. (2016) Software Engineering. Technologies of Software Development: A Textbook for Universities, 5th ed. St. Petersburg: Piter. 2016. 641 p. Orlov, S. and Vishnyakov, A. (2010) Pattern-oriented decisions for logistics and transport software. Transport and Telecommunication, 11(4), 46–58. Orlov, S.and Vishnyakov, A. (2014) Pattern-oriented architecture design of software for logistics and transport applications. Transport and Telecommunication, 15(1), 27–41. Podinovski, V. (1994) Criteria importance theory.Mathematical Social Sciences,27, 237–252. Podinovski, V.and Podinovskaya, O. (2014) Criteria importance theory for decision making problems with a hierarchical criterion structure: Working paper WP7/2014/04. Moscow: Publishing House of the Higher School of Economics, 28 p.