Usefulness and perceived usefulness of Decision ... - SISEF.ORG

iForest

Research Article doi: 10.3832/ifor1356-008 (Early View)

Biogeosciences and Forestry

Usefulness and perceived usefulness of Decision Support Systems (DSSs) in participatory forest planning: the final users’ point of view Fabio Pastorella (1), José G Borges (2), Isabella De Meo (3)

In recent decades, the focus of forestry Decision Support Systems (DSSs) has expanded to consider the social dimension of forestry and to support participatory decision-making. A large number of models and tools have become available to solve forest man agement planning problems. The Usefulness of a DSS depends on the range of tools that it incorporates, and many researches have been developed to evaluate DSSs using Usefulness as parameter. The assessment of Usefulness concerns the effectiveness of a DSS. Furthermore, most assessments take into account the degree of Perceived Usefulness, which is considered an indicator of the impact a system has on job performance. The present study focuses on the analysis of final users’ point of view on the Useful ness and Perceived Usefulness of DSSs in participatory forest planning. The research investigates how forest users’ characteristics and context influence their views on the potentialities of DSSs to enhance both the various phases of the participatory planning process (Usefulness) and job performance (Perceived Usefulness). The study is based on quantitative data collected through two questionnaires e-mailed to a sample of 150 DSSs end users. The questionnaires focused on Usefulness and on Perceived Usefulness topics, respectively. Results indicate that special attention must be given to motivating respondents with a clear explanation of the survey objectives when e-mailing questionnaires. Moreover, results show that, in general, respondents consider DSSs useful at each step of the participatory process, despite differences emerge among steps. The research also shows that respondents’ Perceived Usefulness of DSSs was higher before actually engaging with DSSs. Furthermore, the results highlight differences in Perceived Usefulness to improve job performance, suggesting that the use of DSSs may actually improve job performance more than expected. Specifically, results indicate that improving the technical descriptions of methodologies incorporated in a DSS may contribute to increasing the Perceived Usefulness. The information provided within this research contributes to the advancement of knowledge regarding the Usefulness of DSSs as perceived by forest stakeholders, which in turn supports the improvement of DSS architectures and the development of participatory processes in forest management planning. Keywords: Forest Management, Decision Support Systems, Participatory Planning, Usefulness, Perceived Usefulness

Introduction

Decision support systems (DSSs) are computer systems designed to address complex decision-making processes (Mallach 1994, Newman et al. 2000, Turban & Aronson 2004, Inman et al. 2011). Specifically,

DSSs are a subset of computer-based Information Systems (IS), and may support the recording, processing and dissemination of information. Typically, the users are groups of individuals who act together in a field with a common purpose (Díez & McIntosh

(1) Consiglio per la Ricerca in Agricoltura e l’analisi dell’economia agraria, Forest Monitoring and Planning Research Unit (CRA-MPF), p.zza Nicolini 6, I-38123 Villazzano di Trento (Italy); (2) Forest Research Centre, School of Agriculture, University of Lisbon, Apada da Ajuda, 1349-017 Lisboa (Portugal); (3) Consiglio per la Ricerca in Agricoltura e l’analisi dell’economia agraria, Agrobiology and Pedology Research Center, p.zza M. D’Azeglio 30, I50121 Firenze (Italy)

@ Isabella De Meo ([email protected]) Received: May 20, 2014 - Accepted: Aug 20, 2015 Citation: Pastorella F, Borges JG, De Meo I (2015). Usefulness and perceived usefulness of Decision Support Systems (DSSs) in participatory forest planning: the final users’ point of view. iForest (early view). – doi: 10.3832/ifor1356-008 [online 2016-01-02] Communicated by: Luca Salvati

© SISEF http://www.sisef.it/iforest/

e1

2009). The forestry literature reports several approaches to characterize DSSs used to address forest management planning problems (Schuster et al. 1993, Nabuurs & Paivinen 1996, Mower 1997, Borges et al. 2003, Gordon et al. 2004, Rauscher et al. 2005, Reynolds et al. 2005, Reynolds et al. 2008, Vainikainen et al. 2008, Menzel et al. 2012). According to Borges et al. (2014), a forestry management planning DSS consists of a computerized tool with a graphicbased user interface that includes a data management module and a solution module – for example, a module that provides guidance and support so as to define the timing and location of forest management options. In recent decades, the focus of forestry DSSs has expanded to consider the new resource base and a broad range of forest functions and benefits. The social dimension of forestry has become more prominent, representing a key element of Sustainable Forest Management (SFM). The importance of involving stakeholders in the decision-making process – in particuiForest (early view): e1-e8

iForest – Biogeosciences and Forestry

Pastorella F et al. - iForest (early view) lar in the definition of forest management strategies – has been underlined by several authors (Hickey 2004, Sheppard & Meitner 2005, Martins & Borges 2007, De Meo et al. 2011, Marques et al. 2011, Cantiani 2012, Bruña-García & Marey-Pérez 2014). As a consequence, participatory approaches to improve the planning processes using the views and experiences of stakeholders and experiential knowledge are often a highly valued component of forestry DSSs (Marques et al. 2013). Nevertheless, the decision-making processes may engage with scientific complexity and political and social uncertainty, becoming gradually more complicated (Kangas et al. 2008, Lawrence & Stewart 2011, Nordström et al. 2011). In order to address these substantial changes in the decision-making context, a large number of models and tools have become available to support individual decision-making and to help address current and emergent forest management planning problems. DSSs in the forest sector typically include models and methodologies that simulate various forest variables to support the forecast of the outcomes of current decisions. Some DSSs also include the potentiality of inserting information about the preferences of decision-makers and participants (Menzel et al. 2012). The possibility of developing DSSs incorporating techniques to help further decisionmakers build their preferences, e.g., by providing them with information about tradeoffs between criteria (Borges et al. 2014) is promising, as it may contribute to the effectiveness of participatory approaches. The Usefulness of a DSS thus depends on the range of tools that it incorporates. Usefulness is defined as the total value a user perceives from using a new technology (Rogers 2003) and is a critical success factor for the effectiveness of a DSS application. As reported by Díez & McIntosh (2009) the evaluation of an IS and the assessment of its usefulness are difficult tasks for researchers and technicians, as the basis and outcomes are strictly related to the context in which it is used. Typically, these tasks encompass both technical and implementation assessments. The former corresponds to testing the technical aspects of the IS, while the second focuses on the overall Usefulness of the system (Sauter 2010), that is, whether the implementation of the system satisfies users’ needs and expectations (Baroni et al. 2010). Many theories and practical experiments have been established to evaluate IS through a post-implementation process using Usefulness as parameter. Examples include the Technology Acceptance Model (TAM – Davis 1989) and its variants (Lucas & Spitler 1999), the Theory of Planned Behaviour (TPB – Gordon 2006), the analysis of acceptability of a computer system (Nielsen 1993), and the analysis of model quality (Rittgen 2010). The assessment of Usefulness concerns the effectiveness of a DSS and focuses e2

both on its functionality and usability. In practice, the assessment of Usefulness generally includes meetings devoted to monitoring the participants’ interaction with the DSS. Most assessments also consider the degree of Perceived Usefulness (PU – Arciniegas et al. 2013) and/or the Perceived Ease of Use (PEU). PU is an indicator of the impact a system has on work effectiveness and is defined as the degree to which a person believes that using a particular system may enhance his/her job performance (Davis 1989). A DSS PU thus increases with the perception of a positive relationship between using the DSS and job performance. The PU is a key indicator of the TAM, widely used in the evaluation of an IS – e.g., King & He (2006) reported references to TAM in 140 papers. The PU indicator is further considered by the Theory of Planned Behaviour (TPB – Taylor & Todd 1995) and other DSS Usefulness assessment models (Kripanont 2007). DSS assessment studies suggest that PU and PEU are the two main critical success factors for the adoption and use of information technology (Davis 1989). Before deciding to use a system, the user must perceive the system as being easy to use (Perceived Ease of Use) and as being useful, i.e., contributing to making his/her work easier (Perceived Usefulness – Venkatesh et al. 2003). PU has been used to assess DSSs in many fields. Examples for specific subjects include computer science (Adams et al. 1992); acceptance of a website (Kamis et al. 2008, Yahya et al. 2012); Executive Information Systems (Averweg 2008); Enterprise Resource Planning systems (Amoako-Gyampah 2007); 3G mobile services (Suki & Suki 2011); landscape and river basin management (Volk et al. 2010); agriculture and farming systems research (Newman et al. 2000); river rehabilitation (Hostmann 2005); and information systems and managers’ performance (Franz & Robey 1986). The case of forestry has been considered when assessing forest farmers’ intention of engaging in online trading (Li et al. 2012), forest education tools (Li et al. 2011) and DSSs for forest management (Garg et al. 2006). The final users’ PU and the way they see the Usefulness of forest DSSs are affected by several elements, such as the local socio-cultural and territorial context, the availability and culture of using DSSs, the final users’ background, and their profession (Amoako-Gyampah 2007, Inman et al. 2011, De Meo et al. 2013). Italian and Finnish researchers’ and practitioners’ perception of DSS Usefulness in participatory forest planning has been analyzed by De Meo et al. (2013). The present study has a dual purpose. Firstly, it will extend the research of De Meo et al. (2013) to a wider geographical area. Secondly, it will focus not only on analyzing final users’ point of view on the Usefulness of DSSs in participatory forest planning, but also on the Perceived Usefulness. The emphasis is

on investigating how forest users’ characteristics and context (e.g., country, age and background) influence their views on the potentialities of DSSs to enhance both the various phases of the participatory planning process (the Usefulness) and job performance (the Perceived Usefulness).

Materials and methods Sampling and survey

The study is based on quantitative questionnaire data collected from February through May 2013. Two questionnaires were submitted by e-mail to a sample of 150 DSS end users. The first focused on Usefulness issues (U questionnaire), while the second addressed Perceived Usefulness topics (PU questionnaire). The sample was selected using an iterative process based on the principles of snowball sampling (Harrison & Qureshi 2000, Hislop et al. 2004): during the first iteration, DSS end users involved in the COST Action FP0804 (FORSYS) were selected, while during the second iteration, further DSS end users were identified. The sample of 150 DSS end users was composed of 31 National Coordinators involved in the FORSYS COST Action, 56 Portuguese stakeholders (from Research Institutes, Agencies, Companies) involved in forest management and 63 DSS users contacted in the second stage of sample selection. The sample included: (i) professors and researchers at universities or research institutes; (ii) representatives of national or regional forestry centers; (iii) professionals in private associations; and (iv) representatives of state enterprises. They were divided into two main categories: professionals and researchers. It is not possible to know the origins of DSS users contacted by peers in the second iteration, but from the evidence in the results it is supposed that most of them are from Europe. The geographic distribution of the 87 DSS users involved in the first iteration was: 79 from other European countries, 2 from Africa, 2 from North America and South America, and 1 from Asia and Oceania. In terms of gender, 81% of the sample was represented by males and 19% by females. Respondents were required to return answers to the questionnaires by e-mail within six weeks. Practices reviewed by Dennis (2003) were taken into account to avoid some problems related to auto-compiled questionnaire techniques, such as a large number of missing responses, delays in collection, and sample distortion due to auto-selection. In particular, follow-up reminder e-mails were sent after two weeks, and in some cases respondents were also contacted by phone about a week after the questionnaires had been sent. The response rates to the U and PU questionnaires were 14% and 17%, respectively. Some authors (Yahya et al. 2012, Becker et al. 2013, McMichael & Shipworth 2013) clasiForest (early view): e1-e8

sify these response rates as low. However, for questionnaires sent by e-mail, the absolute number of responses is reported to be more significant than the rate (Deutz et al. 2013). Hochman & Carberry (2011) reported as significant findings from a survey with 23 responses. In the case of this research, a total of 46 responses were received: 25 and 21 responses to the PU and U questionnaires, respectively.

Questionnaires

The two questionnaires comprised seven (the U questionnaire) and six (the PU questionnaire) close-ended questions, chosen with the aim of keeping the structure simple. In some cases, the respondents had to choose from a list of predefined responses, while other questions were formulated to offer an n-options ranking scale for selection by respondents. The full content of both the U and PU questionnaires are reported in Appendix 1 and Appendix 2, respectively. The U questionnaire comprises a general section focusing on the characteristics of the management planning problems faced by the respondents (Appendix 1). Specifically, respondents were asked to define the temporal and spatial scales of their forest management planning problems. A third question focuses on the users’ experiences using DSSs. In the questionnaire focusing on U, two additional questions from De Meo et al. (2013) were included. These questions investigate the potential usefulness of DSS in improving each stage of the participatory planning process. They divided this process into 6 steps and 13 phases (Appendix 1). Respondents were asked to indicate the usefulness of the DSS for each step and phase, indicating a value from “highly useful” to “not useful at all”. The optional response “I do not know” was also provided. The last part of the questionnaire deals with the socio-professional situation of the respondents, namely, their year and place of birth, their professional position and their role in participatory planning. The PU questionnaire also comprises a general section concerning the management planning problems faced by the DSS end users and their experience using DSSs (Appendix 2). In addition, the PU questionnaire includes two questions to gauge the degree of Usefulness as perceived currently by the respondent (question 5) and perceived earlier before the actual engagement of the respondent with DSS (question 6). Respondents were also provided with the option “Unable to form an opinion”. Users were asked to respond to questions 5 and 6 by providing a score on a five-point Likert’s scale, according to their agreement or disagreement with six statements adapted from Davis (1989) to measure user acceptance of computer tools. A low score indicates negative Perceived Usefulness, while a high score indicates positive perceptions. iForest (early view): e1-e8

Data Analysis

Voluntary responses received by e-mail were collected and stored in a spreadsheet. In order to analyze the agreement to the statements concerning the PU of DSSs (questions 5 and 6 reported in Appendix 1), respondents were divided into four groups (Tab. 1). This grouping took into account the users’ experience with DSS and the statement reference period (current period or period before engaging with the DSS). Group A included all respondents and referred to the current PU (all respondents answering question 5). Group B comprised respondents experienced with DSS, reporting their PU before engaging with DSS (question 6). In addition, among respondents to question 5, we separated respondents who had already engaged with DSS (group C) from those who had not (group D) to get their current PU. To avoid random errors, the reliability of each group’s responses was analyzed by applying Cronbach’s alpha coefficient (α). This statistic was introduced by Cronbach (1951) and is widely used in psychometric studies to test the internal consistency of multiple-item scales. Internal consistency describes the extent to which all the items in a test measure the same concept. The technique is based on the fact that the variance of the sum of a group of independent variables is the sum of their variances (Bland & Altman 1997). Therefore, the variance of the sum will increase if the variables are positively correlated. Values of the coefficient range from 0 to 1, with higher values indicating greater reliability. When items in the scale are identical (thus perfectly correlated) then α = 1, while when items are independent, then α = 0. It is commonly accepted that a value of 0.8 provides a reasonable rate of reliability (Gliem & Gliem 2003, Tavakol & Dennick 2011). Differences among groups were analyzed with the aim of detecting whether experience with DSSs influenced PU. In particular, PU was analyzed by comparing data from the following pairs of groups: BC (comparison of current PU and PU before engaging with DSSs from users with experience with DSS); C-D (comparison between current PU of users experienced with DSSs and users that have not yet engaged with DSSs); B-D (PU of users before engaging with DSSs). In order to ascertain whether experience with DSSs influences PU and to analyze pairwise differences among groups of respondents, a comparison by means, standard deviations and medians was carried out. In fact, since normal distribution in the various groups could not be guaranteed, a more in-depth statistical analysis was conducted using a non-parametric test. Data concerning PU, grouped by experience with DSSs and by statement reference period, were compared by means of the Wilcoxon’s test at p < 0.05 for significance. The test was carried out using the “wilcox.test” function in R statistical software system.

Tab. 1 - Characteristics of respondents classified into homogeneous groups (see text for details).

A B

Experience with DSSs yes + no yes

C D

yes no

Groups

Reference period current before using the DSS current current

Results and discussion Profile of the respondents

The profile characteristics of the respondents are shown in Tab. 2. Respondents were mainly from European countries, and in particular from Italy, Spain, Germany and Portugal (28, 16, 12 and 12%, respectively). Such distribution by nationality reflected the fact that most of the contacted DSS users were from European countries; furthermore, the Authors’ nationality is Italian and Portuguese and this may partly explain why the majority of responses are from Italy and Portugal. Concerning their role in the participatory planning process, the respondents are mainly experts (32%), researchers (16%) or facilitators (16%). However, in some cases it was difficult to identify their main role because some respondents (36%) declared that they had mixed expertise (e.g., expertfacilitator, facilitator-researcher). Respondents between the ages of 31 and 50 years represented 76% of the whole sample. A total of 13 respondents reported experience with DSSs, while 12 reported no engagement with DSSs. There were approximately an equal number of the former in each of the three levels of experience (1-5, 6-10, >10 years). Concerning the temporal scale of forest management, most respondents (44%) were involved in long-term planning (more than 10 years). Only 4% declared that they were involved in short-term planning. Concerning the spatial scale, 40% of the respondents declared that they were involved in forest-level planning, where the focus is the joint management of a set of stands in a forested landscape. Some respondents indicated experience with DSSs in more than one combination of temporal and spatial scales.

Usefulness

The U questionnaire was completed by 21 respondents, a response rate lower than that of the PU questionnaire. Respondents having experience with DSSs in participatory planning indicated the use of one to four DSSs. Seven interviewees declared that they had used only one DSS. Respondents with little or no experience with DSSs in participatory planning were asked to specify the reason why they did not use DSSs (Tab. 3). Results seem to indie3


DSS usefulness in participatory forest planning


Pastorella F et al. - iForest (early view) Tab. 2 - Main characteristics of the respondents. (n/a): data not available. Country Germany

Italy

Lebanon Netherlands Portugal

Slovenia Spain

Sweden Switzerland Ukraine USA

Age class 20-30 31-40 41-50 41-50 31-40 41-50 31-40 51-60 41-50 31-40 31-40 41-50 31-40 41-50 41-50 41-50 31-40 31-40 20-30 51-60 31-40 31-40 51-60 41-50 51-60

Temporal scale long long all long long medium medium medium-long medium-long long medium medium-long long long long medium-long medium-long long all medium short long medium-long long all

cate that the options listed could not help to explain the motivation for not using DSSs. In fact, the option “Other: I do not know DDSs” was reported by seven respondents as the main reason. The answers to the questionnaires pro-

Spatial scale stand regional all forest stand forest stand-forest forest-regional stand-forest forest regional forest-regional stand stand forest forest all stand forest forest forest forest forest-regional forest all

Role in participatory planning researcher facilitator-researcher researcher researcher expert researcher researcher expert expert expert facilitator expert-developer-facilitator facilitator facilitator stakeholder expert expert forest manager facilitator stakeholder expert facilitator-expert expert researcher facilitator-expert

vided information used to assess the usefulness of DSS at each step and each phase of the participatory processes (Tab. 4). In general terms, the respondents consider DSSs somewhat useful at every step. The items analyzed appear reliable for analysis

Tab. 3 - Distribution of respondents expressing why they did not use DSSs. Statement DSS available is too costly to use DSS available is too difficult to use DSS available does not include properties needed I cannot see the benefit DSS would provide for the planning task DSS is not needed for participatory planning I’m involved in Other, what?: I do not know DDSs

Number of respondents 1 1 2 1 1 7

Years of experience 6-10 > 10 6-10 n/a n/a 1-5 n/a n/a n/a > 10 n/a n/a n/a n/a n/a > 10 1-5 1-5 n/a n/a n/a 1-5 n/a n/a > 10

purposes: in fact, the overall Cronbach’s alpha reliability coefficient achieved was 0.97 at a level of p = 0.05, indicating that the test used is free from random error and that the items have a high internal consistency. According to the respondents, the usefulness of DSSs is higher during the “monitoring” (average 3.50) and “design” (3.42) steps. Furthermore, the phases in which the use of DSSs is considered more useful are problem structuring (3.77) and creating new knowledge (3.71). By contrast, the step in which DSSs are not perceived as very useful is the intelligence step, and in particular the phases of “identification of stakeholders” (2.33) and “definition of criteria, goals and constraints” (2.71). The low

Tab. 4 - Usefulness at each step and phase of the Participatory Planning Process. Steps Organization

Intelligence

Design

Choice Monitoring

e4

Usefulness Phases 3.34 a - Organizing the process (time frame, budget definition) b - Problem structuring (focussing on the problem situation, formulating a joint problem, raising awareness, assessing current status) 3.01 c - Identifying stakeholders d - Defining criteria, goals and constraints e - Eliciting stakeholders’ preferences f - Eliciting decision makers’ preferences g - Gathering/identifying information/expert knowledge regarding the decision/problem h - Creating new information and knowledge i - Aggregating stakeholders’ preferences 3.42 j - Identifying alternatives k - Exploring alternatives (investigating, discussing, modifying) l - Illustrating effects 3.07 m - Selecting the best option 3.50 n - Achieving goals set by the stakeholders

Usefulness 2.92 3.77 2.33 2.71 2.92 2.83 3.38 3.71 3.15 3.38 3.38 3.50 3.07 3.50

iForest (early view): e1-e8

value for identifying stakeholders implies that respondents do not see DSSs as useful in providing evidence to determine the key actors to be involved in the participatory process. Our results showed that the use of DSSs during the Intelligence phase is critical, as confirmed by previous reports (Raggad 1996, De Meo et al. 2013). According to respondents, Intelligence seems the step with the lesser usefulness for DSSs. This fact highlights the likely importance of providing users with good examples of how DSSs could be used to facilitate the Intelligence step of participatory planning. On the other hand, if the low scores for DSS usefulness are due to a lack of proper DSSs for the Intelligence step, the main recommended action is to create such software. Some respondents indicated that they only used DSSs in some phases of the participatory processes. In particular, they indicated that “organizing the process” (28% of responses), “identification of stakeholders” (20%) and “achievement of the goals set by the stakeholders” (12%) are the phases where the use of DSS is more difficult. In order to avoid such difficulties, some improvements are needed in the development of DSSs. In this context, Van Meensel et al. (2012) showed how the approach used in developing DSSs may result in features that positively influence critical factors, such as selecting stakeholders, setting objectives and evaluating the DSS. Answers provided by respondents might also explain their negative perception of DSS usefulness. In most cases, respondents stated that DSSs were not used either because the properties needed to support the participatory process are missing, or the respondents failed to see the benefit DSSs might provide. In fact, most answers list as prominent the reasons “DSS available does not include properties needed” and “I cannot see the benefit DSSs would provide for the planning task”. Both reasons are related to the organization step and its “identification of stakeholders” phase.

Perceived usefulness

The PU questionnaire was completed by 25 respondents. Each group of respondents obtained a significant Cronbach’s alpha reliability at 0.05 level (Tab. 5), showing that the test used is free from random error and it is reliable for analysis purposes. These results agree with those reported by Davis (1989), Adams et al. (1992), and Suki & Suki (2011). Concerning the average of the six statements, very few differences among the groups were detected, indicating that experience does not seem to influence PU (Tab. 6). Nevertheless, the highest average value is associated with the PU before engaging with DSSs given by respondents reporting experience with DSS (group B). The statements associated with highest iForest (early view): e1-e8

average PU value were “using DSSs improves/would improve my job performance” (statement number 2 in Tab. 6) and “I find/would find DSSs useful in my job” (statement number 6 in Tab. 6), given by respondents reporting experience with DSSs (groups A, B and C). As expected, the lowest average PU value was associated with statement 2 given by respondents with no experience using DSSs (group D). The paired group comparison of PU by mean values shows small differences in terms of overall average values, reaching values around ± 0.1 (Tab. 7). Negative values indicate that the second group has a higher value. Nevertheless, the differences increase to higher values when the analysis focuses on individual statements. The highest differences occur in response to the statements “Using DSSs improves/would improve my job performance” (statement 2 in Tab. 7), “I find/would find DSSs useful in my job” (statement 6 in Tab. 7) and “Using DSSs in my job increases/would increase my productivity” (statement 6 in Tab. 7). The comparison between the current PU reported by the whole sample (Group A) and the PU before engagement with DSSs reported by respondents with experience using DSSs (Group B) highlighted that respondents perceived DSSs as being more useful before engaging with them. Moreover, respondents reported a decrease in PU in response to statements related to the impact of DSSs on performance, productivity, and job efficacy. Regarding current PU, the comparison between respondents with experience using DSSs (group C) and respondents who have not engaged with DSSs (group D) showed a high PU difference in response to “Using DSSs improves/would improve my job performance” (statement 2 in Tab. 7). By con-

Tab. 5 - Cronbach’s alpha reliability values for each respondent group. Groups A B C D

Cronbach’s alpha reliability 0.92 0.87 0.93 0.86

trast, the value in response to “using DSSs in my job increases/would increase my productivity” (statement 3 in Tab. 7) was relatively high among respondents with no experience using DSSs. Concerning the other statements, values tended to be higher among respondents engaged with DSSs or showed differences very close to zero. The Mann-Whitney-Wilcoxon test showed that the data were not statistically different when comparing pairwise groups in BC (p-value = 0.5967), C-D (p-value = 0.6338), or B-D (p-value = 0.2536). Results indicate that no significant differences in perception were due to experience with DSSs.

Conclusions

The response rate achieved in this study confirms the evidence reported by Dennis (2003), who suggested that when e-mailing questionnaires, attention must be paid to motivating additional potential respondents by clearly explaining the objectives of the research. Nonetheless, both questionnaires achieved a high Cronbach’s alpha reliability value, showing that they are free from random error and that the methodology is robust. In general, respondents consider DSSs useful. Nevertheless, PU was higher before

Tab. 6 - Average (Av), standard deviation (SD) and median values (Med) of Perceived Usefulness per statement for each group of respondents. Statement 1 2 3 4 5 6 Grand mean

A Av 1.17 1.33 0.92 1.25 1.08 1.42 1.19

B SD Med 1.19 2 0.98 2 1.16 1 0.87 2 1.16 2 0.67 2 1.00 2

Av 1.00 1.58 1.08 1.25 1.25 1.67 1.31

C SD Med 1.21 1 0.67 2 1.00 1 0.87 2 0.97 2 0.65 2 0.91 2

Av 1.08 1.46 1.00 1.25 1.17 1.54 1.25

D SD Med 1.20 2 0.83 1 1.08 1 0.87 2 1.06 2 0.66 2 0.96 1

Av 1.11 0.89 1.22 1.22 1.11 1.33 1.15

SD Med 0.87 1 0.70 2 0.63 1 0.79 1 0.87 1 0.82 2 0.79 1

Tab. 7 - Pairwise comparisons by means of the Wilcoxon’s test of average Perceived Usefulness values per statement. Statement 1 2 3 4 5 6 Grand mean

B-C -0.08 0.12 0.08 0.00 0.08 0.13 0.06

C-D -0.03 0.46 -0.22 0.03 0.06 0.21 0.10

B-D -0.11 0.69 -0.14 0.03 0.14 0.33 0.16

e5




Pastorella F et al. - iForest (early view) actually engaging with DSSs. This suggests that either respondents’ expectations were unrealistic or the DSS development project failed to meet the users’ needs. Results highlighted further differences in PU in terms of improving job performance. In this case, the PU reported by groups A, B and C is higher than the PU reported by group D, suggesting that DSS use may actually improve job performance more than was expected. This outcome confirms the results by Kamis et al. (2008) who reported that, in general, DSS use may increase perceived usefulness directly or through perceived ease of use. The main reasons indicated to explain why DSSs were not used in some phases during a participatory planning process were “available DSS does not include properties needed” and “DSSs are not believed to provide benefits for the planning task”. Cox (1996) and Newman et al. (2000) examined and reviewed the main problems in adopting DSSs in agriculture. According to our results, they suggested that DSS process models may be inadequate in terms of distinguishing practitioners’ matters. The latter hypothesized that the benefits of many DSSs are unclear in the development stage. Therefore, the system does not meet the need of any group of users. Nevertheless, as a general rule, DSSs appeared useful at each step of the participatory process, although differences did emerge among steps. Problem structuring, monitoring the achievement of goals set by the stakeholders and creating new information and knowledge were the steps at which the use of DSS seems to be more helpful. By contrast, DSSs were considered less useful in identifying stakeholders, defining criteria, goals and constraints, and eliciting decision-maker’s preferences. According to Sheppard & Meitner (2005), enhancing the technical descriptions of models and methodologies incorporated in a DSS might help increase the PU of a DSS. The main limit of this research is the small number of respondents. This limitation is firstly due to the fact that very few DSS final users are actively involved in participatory planning at present. However, the qualitative analysis reported here may contribute to the advancement of knowledge on the usefulness of DSSs as perceived by forest stakeholders. Our results suggest that DSS developers should pay close attention to users’ perception in order to increase both the usefulness and PU of DSSs. In particular, the development of software that can overcome constraints in the intelligence phases is needed. Given the rapid development in participatory forest planning, further investigation is required to improve methodologies aimed at effectively collecting up-to-date information about DSS final users’ opinions and perspectives. This is influential for the definition of approaches to enhance DSS e6

architectures/structures and development processes, thus fostering the successful use of DSSs in forest management planning participatory processes.

Acknowledgements

This paper is a result of a short-term scientific mission (STSM) developed for the Working Group on participatory processes of the COST-action FP0804-Forest Management Decision Support Systems (FORSYS). The authors have contributed to this paper in equal parts.

References

Adams DA, Ryan Nelson R, Todd PA (1992). Perceived usefulness, easy of use, and usage of information technology: a replication. MIS Quarterly 16: 227-247. - doi: 10.2307/249577 Amoako-Gyampah K (2007). Perceived usefulness, user involvement and behavioral intention: an empirical study of ERP implementation. Computers in Human Behavior 23: 1232-1248. doi: 10.1016/j.chb.2004.12.002 Arciniegas G, Janssen R, Rietveld P (2013). Effectiveness of collaborative map-based decision support tools: results of an experiment. Environmental Modelling and Software 39: 159-175. - doi: 10.1016/j.envsoft.2012.02.021 Averweg U (2008). Information technology acceptance in South Africa: an investigation of perceived usefulness, perceived ease of use, and actual system use constructs. The African Journal of Information Systems 1: 44-66. [online] URL: http://digitalcommons.kennesaw. edu/ajis/vol1/iss1/4/ Baroni P, Fogli D, Giacomin M, Guida G, Parasiliti Provenza L, Rossi M, Bohanec M, Znidaršič M (2010). Supporting DSS acceptability through a user-centered design methodology: experiences in emergency management. In: Proceedings of the 2010 conference on “Bridging the Socio-technical Gap in Decision Support Systems: Challenges for the Next Decade” (Respìcio A, Adam F, Philips-Wren G, Teixeira C, Telhada J eds). Lisbon (Portugal) 8-10 July 2010. IOS Press, Amsterdam, The Netherlands, pp. 87-98. [online] URL: http://www.researchgate. net/profile/Massimiliano_Giacomin/publication/ 220762778 Becker J, Reist M, Friedli K, Strabel D, Wüthrich M, Steiner A (2013). Current attitudes of bovine practitioners, claw-trimmers and farmers in Switzerland to pain and painful interventions in the feet in dairy cattle. Veterinary journal 196: 467-476. - doi: 10.1016/j.tvjl.2012.12.021 Bland JM, Altman DG (1997). Statistics notes: Cronbach’s alpha. British Medical Journal 314: 572. - doi: 10.1136/bmj.314.7080.572 Borges JG, Falcão A, Miragaia C, Marques P, Marques M (2003). A decision support system for forest resources management in Portugal. In: “System Analysis in Forest Resources” (Arthaud GJ, Barrett TM eds). Kluwer Academic Publishers, Managing Forest Ecosystems Vol. 7, Dordrecht, The Netherlands, pp. 155-164. - doi: 10.1007/978-94-017-0307-9_17 Borges JG, Nordström EM, Garcia-Gonzalo J, Hujala T, Trasobares A (2014). Computer-based tools for supporting forest management. The experience and the expertise world-wide. De-

partment of Forest Resource Management, Swedish University of Agricultural Sciences, Umea, Sweden, pp. 503. [online] URL: http:// hdl.handle.net/10400.5/7205 Bruña-García X, Marey-Pérez M (2014). Public participation: a need of forest planning. iForest 7: 215-225. - doi: 10.3832/ifor0979-007 Cantiani MG (2012). Forest planning and public participation: a possible methodological approach. iForest 5: 72-82. - doi: 10.3832/ifor06 02-009 Cox PG (1996). Some issues in the design of agricultural decision support systems. Agricultural Systems 52: 355-381. - doi: 10.1016/0308-521X (96)00063-7 Cronbach LJ (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16: 297-334. - doi: 10.1007/BF02310555 Davis FD (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13: 319-340. doi: 10.2307/249008 De Meo I, Cantiani MG, Ferretti F Paletto A (2011). Stakeholders’ perception as support for forest landscape planning. International Journal of Ecology 2011, article ID 685708, pp. 8. [online] URL: http://downloads.hindawi.com/ journals/ijeco/2011/685708.pdf De Meo I, Ferretti F, Hujala T, Kangas A (2013). The usefulness of Decision Support Systems in participatory forest planning: a comparison between Finland and Italy. Forest Systems 22 (2): 304-319. [online] URL: http://revistas.inia. es/index.php/fs/article/view/2953 Dennis WJ (2003). Raising response rates in mail surveys of small business owners: results of an experiment. Journal of Small Business Management 41: 278-295. - doi: 10.1111/1540-627X.00082 Deutz P, McGuire M, Neighbour G (2013). Ecodesign practice in the context of a structured design process: an interdisciplinary empirical study of UK manufacturers. Journal of Cleaner Production 39: 117-128. - doi: 10.1016/j.jclepro.20 12.08.035 Díez E, McIntosh BS (2009). A review of the factors which influence the use and usefulness of information systems. Environmental Modelling and Software 24: 588-602. - doi: 10.1016/j.env soft.2008.10.009 Franz CR, Robey D (1986). Organizational context, user involvement, and the usefulness of information systems. Decision Sciences 17 (3): 329-356. - doi: 10.1111/j.1540-5915.1986.tb0023 0.x Garg RK, Gera M, Das JK (2006). A variable-based approach to the design, development, implementation and institutionalization of information systems in the forest sector. Forestry 79: 515-533. - doi: 10.1093/forestry/cpl032 Gliem JA, Gliem RR (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. Midwest Research to Practice Conference in Adult Continuing and Community Education 2008: 82-88. [online] URL: http://hdl.handle.net/1805/344 Gordon SN, Johnson KN, Reynolds KM, Crist P, Brown N (2004). Decision support systems for forest biodiversity: evaluation of current systems and future needs. Final report, project A10, National Commission on Science and Sustainable Forestry, Washington, DC, USA, pp. 37.


Gordon SN (2006). Decision support systems for forest biodiversity management: a review of tools and an analytical-deliberative framework for understanding their successful application. PhD thesis, Department of Forest Resources, Oregon State University, Corvallis, OR, USA, pp. 272. [online] URL: http://ir.library.oregonstate. edu.rajatorrent.com/xmlui/handle/1957/2592 Harrison SR, Qureshi ME (2000). Choice of stakeholder groups in multicriteria decision models. Natural Resources Forum 24: 1-19. - doi: 10.1111/j. 1477-8947.2000.tb00925.x Hickey GM (2004). Regulatory approaches to monitoring sustainable forest management. International Forestry Review 6: 89-98. - doi: 10.1505/ifor.6.2.89.38394 Hislop M, Twery M, Vihemäki H (2004). Involving people in forestry: a toolbox for public involvement in forest and woodland planning. Forestry Commission, Edinburgh, UK, pp. 32. Hochman Z, Carberry PS (2011). Emerging consensus on desirable characteristics of tools to support farmers’ management of climate risk in Australia. Agricultural Systems 104: 441-450. doi: 10.1016/j.agsy.2011.03.001 Hostmann M (2005). Decision support for river rehabilitation. PhD thesis, Swiss Federal Institute of Technology, Zurich, Switzerland, pp. 159. - doi: 10.3929/ethz-a-005083385 Inman D, Blind M, Ribarova I, Krause A, Roosenschoon O, Kassahun A, Scholten H, Arampatzis G, Abrami G, McIntosh B, Jeffrey P (2011). Perceived effectiveness of environmental decision support systems in participatory planning: evidence from small groups of end-users. Environmental Modelling and Software 26: 302-309. doi: 10.1016/j.envsoft.2010.08.005 Kamis A, Koufaris M, Stern T (2008). Using an attribute-based decision support system for user-customized products online: an experimental investigation. MIS Quarterly 32: 159-177. [online] URL: http://www.jstor.org/stable/2514 8832 Kangas A, Kangas J, Kurttila M (2008). Decision support for forest management. In: “Managing forest ecosystems, vol. 16”. Springer Science and Business Media, USA, pp. 222. [online] URL: http://books.google.com/books?id=h7mR kLHlH8C King WR, He J (2006). A meta-analysis of the technology acceptance model. Information and Management 43: 740-755. - doi: 10.1016/j.im.20 06.05.003 Kripanont N (2007). Examining a technology acceptance model of internet usage by academics within Thai business schools. PhD thesis, School of Information Systems. Faculty of Business and Law Victoria University, Melbourne, Australia, pp. 432. [online] URL: http:// vuir.vu.edu.au/id/eprint/1512 Lawrence A, Stewart A (2011). Sustainable forestry decisions: on the interface between technology and participation. Mathematical and Computational Forestry and Natural-Resource Science 3: 42-52. Li MC, Hou HT, Kuo YE, Yu KH, Yang CH (2011). Save the forests: a pilot study of a role-playing game for environmental education. In: Proceedings of the “19th International Conference on Computers in Education, ICCE 2011”. Chiang Mai (Thailand) 28 Nov-2 Dec 2011. National Elec-


tronics and Computer Technology Center, Bangkok, Thailand, pp. 634. Li Y, Fan K, Wen J, Ma N (2012). The research on influencing factors of forest farmers’ intention of adopting online trading based on logistic regression model. Advances in Information Sciences and Service Sciences 22: 658-665. Lucas HC, Spitler VK (1999). Technology use and performance: a field study of broker workstations. Decision Sciences 30: 291-311. - doi: 10.11 11/j.1540-5915.1999.tb01611.x Mallach E (1994). Understanding decision support systems and expert systems. Irwin Inc., Chicago, USA, pp. 695. Marques A, Borges JG, Sousa P, Pinho AM (2011). An enterprise architecture approach to forest management decision support design. An application to pulpwood supply management in Portugal. European Journal of Forest Research 30: 935-948. - doi: 10.1007/s10342-011-0482-8 Marques AF, Borges JG, Garcia-Gonzalo J, Lucas B, Melo I (2013). A participatory approach to design a toolbox to support forest management planning at regional level. Forest Systems 22: 340-358. - doi: 10.5424/fs/2013222-03120 Martins H, Borges JG (2007). Addressing collaborative planning methods and tools in forest management. Forest Ecology and Management 248: 107-118. - doi: 10.1016/j.foreco.2007.02.039 McMichael M, Shipworth D (2013). The value of social networks in the diffusion of energy-efficiency innovations in UK households. Energy Policy 53: 159-168. - doi: 10.1016/j.enpol.2012.10. 039 Menzel S, Nordström EM, Buchecker M, Marques A, Saarikoski H, Kangas A (2012). Decision support systems in forest management: requirements from a participatory planning perspective. European Journal of Forest Research 131: 1367-1379. - doi: 10.1007/s10342-012-0604-y Mower HT (1997). Decision support systems for ecosystem management: an evaluation of existing systems. Gen. Tech. Rep. RM-GTR-296, Fort Rocky Mountain Forest and Range Experiment Station, USDA Forest Service, Fort Collins, CO, USA, pp. 154. Nabuurs G, Paivinen R (1996). Large scale forestry scenario models - a compilation and review. EFI Working Paper 10, Joensuu, Finland, pp. 174. Newman S, Lynch T, Plummer A (2000). Success and failure of decision support systems: learning as we go. Journal of Animal Science 77: 112. Nielsen J (1993). Usability engineering. Academic Press, Boston, USA, pp. 358. Nordström EM, Eriksson LO, Ohman K (2011). Multiple criteria decision analysis with consideration to place-specific values in participatory forest planning. Silva Fennica 45: 253-265. doi: 10.14214/sf.116 Raggad BG (1996). Effects of information structure and problem solving on decision-support system choice. Industrial Management and Data Systems 96: 21-27. - doi: 10.1108/0263557961 0107675 Rauscher HM, Reynolds KM, Vacik H (2005). Decision-support systems for forest management. Computers and Electronics in Agriculture 49: 1-5. - doi: 10.1016/j.compag.2005.02.001 Reynolds KM, Borges JG, Vacik H, Lexer MJ

(2005). Information and communication technology in forest management and conservation. In: Report by the IUFRO Task Force on “Information Technology and the Forest Sector” (Hetemaki L, Nilsson S eds). IUFRO World Series vol. 18, Vienna, Austria, pp. 150-171. Reynolds KM, Twery M, Lexer MJ, Vacik H, Ray D, Shao G, Borges JG (2008). Decision support systems in natural resource management. In: “Handbook on Decision Support Systems” (Burstein F, Holsapple C eds). Springer, International Handbooks on Information Systems Series, Handbook on Decision Support System 2, Springer-Verlag, Berlin-Heidelberg, Germany, pp. 499-534. Rittgen P (2010). Quality and perceived usefulness of process models. In: Proceedings of the “2010 ACM Symposium on Applied Computing” (Shin SY, Ossowski S, Schumacher M, Palakal MJ, Hung CC eds). Sierre (Switzerland) 22-26 Mar 2010. ACM New York, USA, pp. 65-72. [online] URL: http://dl.acm.org/citation.cfm?id= 1774105 Rogers EM (2003). Diffusion of innovations (5 th edn). The Free Press, New York, USA, pp. 550. Sauter VL (2010). Decision support systems for business intelligence (2nd edn). Wiley, Hoboken, NJ, USA, pp. 471. Sheppard SRJ, Meitner M (2005). Using multi-criteria analysis and visualization for sustainable forest management planning with stakeholder groups. Forest Ecology and Management 207: 171-187. - doi: 10.1016/j.foreco.2004.10.032 Schuster E, Leefers L, Thompson J (1993). A guide to computer-based analytical tools for implementing National Forest plans. Gen. Tech. Rep. INT-296, Intermountain Forest and Range Experiment Station, USDA Forest Service, Moscow, ID, USA, pp. 269. Suki NM, Suki NM (2011). Exploring the relationship between perceived usefulness, perceived ease of use, perceived enjoyment, attitude and subscribers’ intention towards using 3G mobile services. Journal of Information Technology Management 22: 1-7. [online] URL: http://jitm. ubalt.edu/XXII-1/article1.pdf Taylor S, Todd PA (1995). Understanding information technology usage: a test of competing models. Information Systems Research 6: 144176. - doi: 10.1287/isre.6.2.144 Tavakol M, Dennick R (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education 2: 53-55. - doi: 10.5116/ijme.4d fb.8dfd Turban E, Aronson J (2004). Decision support systems and intelligent systems (7th edn). Prentice-Hall, Inc., Upper Saddle River, New Jersey, USA, pp. 890. Vainikainen N, Kangas A, Kangas J (2008). Empirical study on voting power in participatory forest planning. Journal of Environmental Management 88: 173-180. - doi: 10.1016/j.jenvman.20 07.02.004 Van Meensel J, Lauwers L, Kempen I, Desseina J, Van Huylenbroeck G (2012). Effect of a participatory approach on the successful development of agricultural decision support systems: the case of Pigs2win. Decision Support Systems 54: 164-172. - doi: 10.1016/j.dss.2012.05.002 Venkatesh V, Morris MG, Davis GB, Davis FD (2003). User acceptance of information tech-

e7




Pastorella F et al. - iForest (early view) nology: toward a unified view. MIS Quarterly 27: 425-478. [online] URL: http://www.jstor. org/stable/30036540 Yahya M, Nadzar F, Rahman BA (2012). Examining user acceptance of E-Syariah portal among Syariah users in Malaysia. Procedia Social and Behavioral Sciences 67: 349-359. doi: 10.1016/j. sbspro.2012.11.338

Volk M, Lautenbach S, van Delden H, Newham LTH, Seppelt R (2010). How can we make progress with decision support systems in landscape and river basin management? Lessons learned from a comparative analysis of four different decision support systems. Environmental management 46: 834-849. - doi: 10.1007/ s00267-009-9417-2

Supplementary Material Appendix 1 - Questionnaire about the usefulness of DSSs in participatory planning. Link: [email protected] Appendix 2 - Questionnaire about perceived usefulness of DSSs in participatory planning on forest management. Link: [email protected]

e8
