Defining Project Approach using Decision Tree and ...

Available online at www.sciencedirect.com

ScienceDirect Procedia Engineering 172 (2017) 791 – 799

Modern Building Materials, Structures and Techniques, MBMST 2016

Defining project approach using decision tree and quasi-hierarchical multiple criteria method Maciej Nowak* University of Economics in Katowice, ul. 1 Maja 50, 40-287 Katowice, Poland

Abstract The success of any project strongly depends on the decisions made at the initial stage of it’s life cycle. They define the general formula for project realization, specifying how the main goal of the project is going to be reached. The information available in this phase is usually limited. Moreover, the decisions that must be taken are interconnected – choosing a particular solution concerning one of the considered issues determines the set of options that can be analyzed when consecutive problems are solved. Thus, when project approach is defined we face a dynamic decision making problem under risk. A decision tree is a widely used technique for modelling and solving such problems. The paper presents the way in which a multiple criteria decision tree and quasi-hierarchical approach can be used for determining the project approach. © 2017 Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier Ltd. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of MBMST 2016. Peer-review under responsibility of the organizing committee of MBMST 2016 Keywords: project approach; decision making under risk; multiple criteria decision tree; quasi-hierarchical approach.

1. Introduction The success of any project strongly depends on the decisions made at the initial stage of it’s life cycle. Although various terms are used to refer to this phase, there is a consensus on the set of activities that should be completed during it. According to PRINCE2 methodology once the project mandate document is ready, a Pre-Project phase and the first process – Starting up a Project – is ready to begin [8]. One of it’s goals is to define a project approach, which articulates how the project will be carried out (e.g., whether products will be built from scratch or bought off

* Corresponding author. Tel.: +48-32-2577973; fax: +48-32-2577471. E-mail address: [email protected]

1877-7058 © 2017 Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of MBMST 2016

doi:10.1016/j.proeng.2017.02.125

792

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

the shelf, etc.). This means that it is necessary to evaluate various delivery solutions considering any organizational strategies and standards, security constraints, training needs, as well as solutions currently applied in the sector where the organization operates. Project approach is usually a result of a multi-phase decision process. The choices made at a particular stage determine options considered at consecutive phases. Thus, the definition of project approach requires interdependent decisions, leading step by step to the final solution. Two main factors make this process difficult: multiple criteria used for evaluating various options and uncertainty. It is generally accepted that the project goal should be defined clearly and unambiguously. However, when the project approach is defined, a lot of detailed issues still need to be clarified. They affect all the project attributes: scope, cost, schedule and quality, which means that trade-offs among them must be taken into account. Additionally, as the choices made at this step refer to the future, we are faced with uncertainty. This lead us to the conclusion that defining the project approach can be formulated as a dynamic multiple criteria decision making problem under risk. Decision tree is a widely used technique for modelling and solving such problems. This method assumes that the decision process consists of a finite number of periods, at which various decisions are made. For each decision, a finite, and usually relatively small number of options is defined. The result of the decision process is determined on one hand by decisions made, on the another, however, on the events that occurred at successive periods. Applications of decision trees in project selection and resource allocation were presented in [1,3,4,6,12,13]. In classical version the decision tree is used to identify a strategy maximizing expected profit or minimizing expected loss. However, real-world decision problems, including determination of the project approach, involve multiple criteria. Haimes et al. [5] considered a dynamic multiple criteria decision problem and proposed a method for generating the set of efficient solutions. Lootsma [7] combined decision tree with two cardinal methods: multiplicative AHP and SMART in order to aggregate multidimensional consequences. More recently Frini et al. [2] solved the multi-criteria decision tree problem without generating the set of all efficient solutions. Their approach combined advantages of decomposition with the application of multi-criteria decision aid (MCDA) methods at each decision node. In previous work [11], an new procedure for solving decision problems that arise at the initial stage of the project life cycle was proposed. It combines decision tree and interactive approach. The problem is solved in two phases. First, the set of efficient strategies is identified. Next, an interactive procedure is used to select the final solution of the problem. This procedure can be used effectively only for problems that are relatively small. In this paper a new method based on a quasi-hierarchical approach is proposed. The paper is structured as follows. In the next section the multiple criteria decision tree problem is discussed. Then, the procedure for identification of near-optimal solutions in decision tree is presented. In section 4 an interactive quasi-hierarchical procedure for multiple criteria decision tree is proposed. Section 5 presents how the procedure proposed in the paper can be used for defining the project approach. The last section groups conclusions. 2. Multiple criteria decision tree We consider a decision process consisting of T periods. In decision tree we use three types of nodes: decision nodes (represented by squares), chance nodes (represented by circles) and terminal nodes (represented by triangles). The branches extending from decision nodes are decision branches, representing possible alternatives available at that point, while branches extending from chance nodes represent the possible events that may occur at that point. Each event is assigned a subjective probability; the sum of probabilities for the events in a set must equal one. Each path in the tree ended in a particular terminal node, represents a particular process realization, defined by a series of decisions made by the decision maker and events that occurred in consecutive chance nodes. Let us assume the following notation: T – number of periods, Yt – the set of decision nodes of period t, nt – the number of decision nodes of period t, YT + 1 – the set of terminal nodes, nT + 1 – the number of terminal nodes, Xt yit – the set of decisions that can be made at node yit ,



793

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

mit – the number of decisions that can be made at node yit , Ξt xit j – the set of states of nature emerging from alternative xit j , Mijt – the number of states of nature emerging from alternative xit j , P([ it j k ) – probability, that the state of nature [it j k will occur, where:



t1,T , i1, n  j1, m t

Mi j t

it

¦[

t i jk

1

(1)

k 1

: yit , xit j ,[it j k – transition function defining the decision or terminal node which is achieved from the decision node yit assuming the decision xit j is made and the state of nature [it j k occurs. Strategy xˆ p defines decisions that should be made is decision nodes:

xˆ p

[ xˆ1p1 , xˆ 2p1 ,..., xˆ 2p n2 ,!, xˆ Tp nT ]

(2)

For the decision maker not all components of this vector are essential. Making a particular decision at period t means that the process cannot reach some of decision nodes at period t + 1. It would seem then that the information on the decisions assigned to nodes that cannot be reached as a result of decisions made at previous periods is unnecessary. We wile use it, however, to modify the strategy, when we will look for near-optimal solutions. By u p we will denote vector specifying which of the decision nodes are achievable when strategy xˆ p is applied:

up

[u1p1 , u 2p1 ,..., u 2p n2 ,!, uTp nT ]

(3)

We assume that u1p1 1 (the first decision node is always achievable). Other components of u p can be determined using the following formula:

u tp l

­°1 when W 1,t 1 W  W yt yi YW [ i j k ΞxˆWp i l ® °¯0 otherwise

: yiW xˆWp i , [iW j k

(4)

The solution of the problem should define decisions for decision nodes achievable for the considered strategy. Thus, it is determined by the components xˆ p for which the corresponding component of u p is equal to 1. This means that two strategies xˆ p and xˆ q may generate the same solution of the problem. This takes place when two conditions are fulfilled:

t1,T i1, n u tp i

uqt i

t1,T i1, n u tp i

1 Ÿ xˆ tp i

t

t

(5)

xˆqt i

(6)

In this paper a multiple criteria problem is considered. We assume that the decision maker specified K objectives. They can be defined for example as follows: “Maximize the profit”, “Maximize the probability of success”, “Maximize the evaluation with respect to a qualitative criterion”, etc. In order to analyze how good is a particular strategy in relation to the objective we must define a criterion. We assume that one criterion is used to express how good are strategies in relation to a particular objective, and two measures can be used to define them: expected value and random event probability. Let us first consider expected value criterion. By f k yiT 1 we denote the value of k-th criterion obtained when the process terminates in node yiT 1 . In order to calculate the expected value for strategy xˆ p it is necessary to calculate expected values for all decision nodes, starting from the last period. Let Gk p yit be the expected value of k-th criterion obtained in node yit when strategy xˆ p is implemented. For terminal nodes we assume:





794

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799



Gk p yiT 1



f k yiT 1

(7)



The following formula can be used for calculating Gk p yit for decision nodes:



Gk p yit

¦ P[ G yW 

t i jk

kp

(8)

l

[ it j k Ξt xˆ tp i







where: ylW : yit , xˆtp i ,[it j k . As we assume that we have a single decision node in period 1, Gk p y11 represents the value of k-th criterion for strategy xˆ p . To solve the problem, we must specify the decisions that should be made in each decision node. A folding-backand-averaging-out procedure can be used to find the solution maximizing the expected value. We start from the last period. For each node in this period we select the decision optimizing the expected value. Next we go one period back, and for each decision node in this period we select the best decision taking into account optimal values determined for decision nodes of the last period. The procedure ends when the optimal value for the first decision node is determined. Let us now assume that the decision maker is interested in the probability that a particular random event S will occur, and 'S be the set of terminal nodes under which event S occurs. In order to find the strategy maximizing the probability of S, we define the criterion value for terminal nodes in the following way:



f k yiT 1

­1 if yiT 1  'S ® ¯0 otherwise

(9)

This makes possible to use the folding-back-and-averaging-out procedure described above to determine the strategy under which the probability of S is maximum. When a single strategy is the best with respect to all criteria, the problem is trivial. However in most situations we are faced with conflicting criteria. A commonly used approach solves the problem in two phases, starting from identification of efficient (or non-dominated) solutions – the ones for which it is not possible to improve the value of any criteria without decreasing the value of at least one of the other criteria. A procedure for generating efficient strategies when various types of criteria (expected value, probability of random event, conditional expected value) are considered is presented in [10]. When the number of efficient strategies is small, it is not difficult for the decision maker to choose the final solution. However, even for problems of moderate size, it can be quite difficult to decide which efficient strategy should be finally implemented. Moreover, for large problems even identification of the set of all efficient strategies can be a real problem. Instead we can apply a quasi-hierarchical approach. It assumes that the decision maker is able to define a hierarchy of criteria, and to specify by how much a more important criterion can be decreased in order to improve the value of a less important one. Such an approach is applied in this paper. 3. Identifying near-optimal strategies To apply a quasi-hierarchical approach, it is necessary to identify not only the optimal and near-optimal strategies. A near-optimal strategy is the one for which the value of the criterion is not less than the threshold specified by the decision maker. While the procedure for identifying optimal strategy in decision tree is well known and widely used, the identification of near-optimal in such problem has not been considered in detail up to now. When the number of decision nodes is relatively small we just can analyze all strategies and select the ones for which the value of the criterion is satisfactory for the decision maker. However in larger problems this method may be time-consuming. Below a procedure for identifying near-optimal strategies in decision tree is presented. Let us assume that the decision maker specified the value Zk, which is the minimal acceptable value of criterion k. In order to identify all strategies satisfying this requirement, we must identify the optimal strategy xˆ p first. This can be done using the procedure described below.

795

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

Algorithm 1: 1. For each terminal node yiT 1  YT 1 assume:



Gk p yiT 1 2. 3.



f k yiT 1

(10)

Assume t = T. t t For each decision node yi  Y complete the following steps: a) for each decision xit j  Xt yit compute:





Fk p xit j

¦ P[ G yW 

t i jk

kp

(11)

l

[ it j k Ξt xit j

where: ylW b) assume:



Gk p yit

c)

: yit , xit j ,[it j k ,



max Fk p xit j



xit j X t y it

(12)



assume that xˆ tp i is the decision for which Fk p xit j is maximum.

If t > 1, assume t := t – 1 and go to (3) The end of the procedure. Having the optimal strategy, we can identify near-optimal strategies. The algorithm presented below is based on simple observation. Note that despite the case of alternate optimal solutions, modification of the optimal strategy by changing the decision in any decision node, results in decreasing the value of the criterion under consideration. Thus, in order to identify the strategy which is the second in the ranking it is enough to analyze only such strategies that differ from the optimal one by decision in exactly one decision node achievable when the optimal strategy is implemented. The procedure uses the following notation: LS – the set of near-optimal strategies, LSB – the set of strategies to be analyzed – the ones that still are worth to be modified. The following procedure can be used to identify near-optimal strategies. Algorithm 2: 1. Identify optimal strategy xˆ p using Algorithm 1. 2. Assume LS {xˆ p } , LS B {xˆ p } . 3. If LS B ‡ , go to (6). 4. Select any strategy xˆ q from the set LSB and remove it from that set: 4. 5.

^ `

LSB : LSB \ xˆ q 5.

Identify all strategies that differ from xˆ q by the decision in one decision node which is achievable when xˆ q is implemented. For each new strategy xˆ r analyze, whether it generates a solution which is different from solutions generated by strategies from the set LS. If this requirement is satisfied, calculate the criterion value for xˆ r and if it is not less than Zk, add strategy to sets LS and LSB:

LS

6.

(13)

{xˆ r } , LSB {xˆ r }

Go to (4). The end of the procedure.

(14)

796

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

The algorithm modifies strategies from the set LSB by changing the decision in exactly one decision node, which can be achieved when the strategy is implemented. Note, that the set of decision nodes achievable when new strategy is implemented can be different than for strategy which is modified. This is not a problem, however, as each strategy specifies decision for all decision nodes, including the ones that cannot be achieved when the strategy is implemented. For each new strategy we analyze, whether it generates a solution different that differs from the solutions generated by strategies generated previously. If it is true, we calculate the value of the criterion, and check whether it satisfies the constraint defined by the decision maker. If not, the strategy is not worth to be analyzed, as its modification will not result in improving criterion value. The procedure ends, when the set LSB is empty. An numerical example describing the way in which the procedure works is presented in [9]. 4. Quasi-hierarchical method for solving multiple criteria decision tree problem Let us assume that the decision maker ordered the criteria starting from the most important one. Thus, he/she is first of all interested in maximizing criterion no. 1, then criterion no. 2, and so on. The solution of the multiplecriteria problem using a quasi-hierarchical approach can be generated using the following algorithm. Algorithm 3: 1. Generate optimal strategies with respect to each criterion using algorithm 1. 2. Present the solutions identified in step (1) to the decision maker. 3. Ask the decision maker to specify aspiration levels Zk for all criteria. 4. For each criterion identify the set of strategies LSk satisfying the constraint specified in step (3). 5. Assume J = K. 6. Identify the set LS being the intersection of LSk:

LS : 7. 8.

9.

 LS k

k 1, J

(15)

If LS z ‡ , go to (9). Inform the decision maker, a strategy satisfying all the requirements specified in step (3) does not exist. Ask the decision maker whether he/she accepts ignoring the requirement defined for the least important criterion. If the answer is yes, assume J = J – 1 and go to 6, otherwise go to 3. From the set LS select the final solution taking into account hierarchy of the criteria.

5. Application Defining project approach usually requires a series of interdependent decisions. As these decisions are made during an initial phase of the project life cycle, the information about their consequences is usually incomplete and fragmentary. Moreover, partial choices are mutually related, since earlier decisions influence which decisions can be considered in the consecutive stages of the process. Thus, while defining the project approach we face with a dynamic decision making problem, which can be modelled by a decision tree. In order to show how the procedure presented in this paper can be applied for defining project approach, let us consider a real problem that was analyzed by a Polish company providing solutions for the railway industry. It decided to enter a new market. The problem concerns decisions made when the company considered entering a new market. It was possible to operate as a general contractor or cooperate with a local company. Three objectives were considered: (1) to maximize the probability of success, (2) to maximize profit margin generated when the offer is accepted, (3) to maximize the evaluation describing the strategic fit. The process is successful if the company decides to submit an offer and it is accepted. Expected value is used to evaluate strategies with respect to the second criterion. Finally, each final state is evaluated with respect to the last criterion using 4 point scale, where 0 means that the company abandons bid or the company’s offer is not accepted, 1 – the company implements the project with a local partner providing part of the equipment, 2 – the company executes the project with a local partner employed for completing

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

a part of installation work only, 3 – the company implements the project as a general contractor. To evaluate strategies with the respect of the criterion strategic fit, we analyze the probability that the final node with a rant not less than 2 is reached. 6 2A

c

2B

d

2

a

6A

h

6B

i

6C 7

7A

j

8

8A

k

9A

l

9

9B 3A

1A

10A

3

10 3B

e

4A

f

1

1B 4

4B b

11

11A

12

12A

13

13A

o

14

14A

p

15A

q

g 15

5

m

10B n

15B

5A

r

Fig. 1. Decision tree of the problem.

The decision tree describing the decision process is presented on fig. 1. The details of the decision-making process under consideration are given below. Period 1: x Decision node 1: 1A – implementation of the project in collaboration with a local company (next node: a); 1B – implementation of the project as a general contractor (next node: b). x State-of-nature node a: a1 – the company finds a local representative for cooperation (next node: 2); a2 – the company is not able to find a cooperator (next node: 3). x State-of-nature node b: b1 – the company is facing technical and organizational problems during the tender preparation (next node: 4); b2 – the company is able to prepare the tender without too much trouble (next node: 5). Period 2: x Decision node 2: 2A – the collaborating company is employed as the supplier of some part of equipment (next node: c); 2B – the collaborating company is employed for completing a part of installation work only (next node: d). x State-of-nature node c: c1 – problems with adaptation of devices supplied by the local cooperator occurred (next node: 6); c2 – no problems with adaptation are identified (next node: 7). x State-of-nature node d: d1 – an agreement concerning the distribution of responsibilities has been reached, no problems arise from the implementation of the assigned tasks (next node: 8); d2 – an agreement concerning the distribution of responsibilities has been reached, there are problems arising from implementation of the assigned tasks (next node: 9). x Decision node 3: 3A – giving up tender submission (next node: terminal node); 3B – turning back to the original concept – the completion of the project as a general contractor (next node: e). x State-of-nature node e: e1 – the company is facing problems with the organisation of the project (next node: 10); e2 – the company is not facing any problems with the organisation of the project (next node: 11). x Decision node 4: 4A – hiring a consulting firm to support project implementation (next node: f); 4B – turning back to the original concept – to establish cooperation with a local company (next node: g). x State-of-nature node f: f1 – problems with implementation are not solved (next node: 12); f2 – with the help of the consulting firm problems are solved (next node: 13).

797

798

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799

x State-of-nature node g: g1 – cooperation with a local company makes it possible to solve problems (next node: 14); g2 – problems identified during tender preparation are not solved (next node: 15). x Decision node 5: 5A – submitting the tender (next node: r). Period 3: x Decision node 6: 6A – completing the contract by using only devices produced by the company itself and submitting the tender (next node: h); 6B – adaptation works and submitting the tender (next node: i); 6C – giving up tender submission (next node: terminal node). x Decision node 7: 7A – submitting the tender (next node: j). x Decision node 8: 8A – submitting the tender (next node: k). x Decision node 9: 9A – organizing additional training for the employees of the cooperator and submit the tender (next node: l); 9B – giving up tender submission (next node: terminal node). x Decision node 10: 10A – hiring a consulting company and submitting the tender (next node: m); 10B – giving up tender submission (next node: terminal node). x Decision node 11: 11A – submitting the tender (next node: n). x Decision node 12: 12A – giving up tender submission (next node: terminal node). x Decision node 13: 13A – submitting the tender (next node: o). x Decision node 14: 14A – submitting the tender (next node: p). x Decision node 15: 15A – organizing additional training for the employees of the cooperator and submit the tender (next node: q); 15B – giving up tender submission (next node: terminal node). The decision to submit the tender in each case leads to a state-of-nature node in which two states are considered: the company’s offer is accepted or rejected. Table 1 presents probabilities of states of nature. In table 2, profits margins and the evaluations with respect to strategic fit criterion are presented. Table 1. Probabilities of states of nature. State of nature

Probability

State of nature

Probability

State of nature

Probability

State of nature

Probability

State of nature

Probability

State of nature

Probability

a1

0.70

d1

0.60

g1

0.30

j1

0,35

m1

0,25

p1

0,30

a2

0.30

d2

0.40

g2

0.70

j2

0,65

m2

0,75

p2

0,70

b1

0.60

e1

0.40

h1

0,30

k1

0,30

n1

0,25

q1

0,30

b2

0.40

e2

0.60

h2

0,70

k2

0,70

n2

0,75

q2

0,70

c1

0.60

f1

0.60

i1

0,35

l1

0,30

o1

0,25

r1

0,25

c2

0.40

f2

0.40

i2

0,65

l2

0,70

o2

0,75

r2

0,75

Each terminal node represents a particular scenario of the process. Part of them represents the success of: these are the nodes which are reached as a result of states of nature h1 – r1. On the other hand terminal nodes h2 – r2 mean that the tender is not accepted. The terminal nodes that are reached as a result of decisions to give up the submission (3A, 6C, 9B, 10B, 12A, 15B) correspond to the process failure as well. First the optimal solutions with respect all criteria are identified. The strategy for which maximizes probability of success is: 1A – 2A – 3B – 6B – 7A – 10A – 11A. The maximal probability of success is 0.320. The maximal expected profit 19,372 is reached for strategy: 1A – 2A – 3B– 6A – 7A – 10A – 11A. Finally the strategy that maximizes the probability of strategic fit not less than 2 is: 1A – 2B – 3B – 8A – 9A – 10A – 11A. The maximal value of this probability is 0.285. The decision maker defined the following aspiration levels: Z1 = 0.25, Z2 = 12,000, Z3 = 0.25. We use algorithm 2 to identify such strategies. The only strategy satisfying the requirements formulated by the decision maker is 1B – 4B – 5A – 14A – 15A, and as a result it is proposed to the decision maker as a final solutions. The probability of success for this strategy is 0.28, expected profit: 12,893, the probability of strategic fit not less than 2: 0.28.

799

Maciej Nowak / Procedia Engineering 172 (2017) 791 – 799 Table 2. Values of profit margin and strategic fit criterion. Final decision / state of nature

Profit margin

Strategic fit

Final decision / state of nature

Profit margin

Strategic fit

6A / h1

148933

2

10A / m2

-46400

0

6A / h2

-36233

0

10B

-6867

0

6B / i1

126947

1

11A / n1

160173

3

6B / i2

-54500

0

11A / n2

-26750

0

6C

-5573

0

12A

-7844

0

7A / j1

138833

1

13A / o1

163893

3

7A / j2

-34783

0

13A / o2

-46700

0

8A / k1

151200

2

14A / p1

142167

2

8A / k2

-54730

0

14A / p2

-46733

0

9A / l1

150093

2

15A / q1

138868

2

9A / l2

-46467

0

15A / q2

-46033

0

9B

-7867

0

15B

-7833

0

3B

-5440

0

5A / r1

174067

3

10A / m1

152113

3

5A / r2

-34221

0

6. Conclusions Defining project approach includes a series of interdependent decisions. As these decisions are made under risk, the decision tree seems to be an efficient tool. In this paper a multiple criteria technique based on the decision tree is proposed for such problems. The procedure uses various types of criteria for comparing alternate strategies. In future work the decision maker’s attitude to risk will be taken into account by applying stochastic dominance rules. References [1] L. Chiu, T.E. Gear, An application and case history of a dynamic R&D portfolio selection model, IEEE Transactions in Engineering Management 26 (1979) 2-7. [2] A. Frini, A. Guitouni, J.M. Martel, A general decomposition approach for multi-criteria decision trees, European Journal of Operational Research 220 (2012) 452-460. [3] T.E. Gear, A.G. Lockett, A dynamic model of some multistage aspects of research and development portfolios, IEEE Transactions in Engineering Management 20 (1973) 22-29. [4] D. Granot, D. Zuckerman, Optimal sequencing and resource allocation in research and development projects, Management Science 37 (1991) 140-156. [5] Y. Haimes, D. Li, V. Tulsiani, Multiobjective decision tree method, Risk Analysis 10 (1990) 111-129. [6] S.W. Hess, Swinging on the branch of a tree: project selection applications, Interfaces 23 (1993) 5-12. [7] F.A. Lootsma, Multicriteria Decision Analysis in a Decision Tree, European Journal of Operational Research 101 (1997) 442-451. [8] Managing Successful Projects with PRINCE2®, 5th ed., The Stationary Office, 2009. [9] M. Nowak, Applying a quasi-hierarchical approach in a multiple criteria decision tree, Studia Ekonomiczne 208 (2014) 59-73 (in Polish). [10] M. Nowak, Solving a Multicriteria Decision Tree Problem using Interactive Approach, in: A.M.J. Skulimowski, J. Kacprzyk (Eds.) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions, Advances of Intelligent Systems and Computing 364 (2016) 301-314. [11] M. Nowak, B. Nowak, An application of the multiple criteria decision tree in project planning, Procedia Technology 9 (2013) 826-835. [12] J.S. Stonebraker, C.W. Kirkwood, Formulating and solving sequential decision analysis models with continuous variables, IEEE Transactions in Engineering Management 44 (1997) 43-53. [13] H. Thomas, Decision analysis and strategic management of research and development: a comparison between applications in electronics and ethical pharmaceuticals, R&D Management 15 (1985) 3-22.