business intelligence (bi) success and the role of ... - Semantic Scholar

INTELLIGENT SYSTEMS IN ACCOUNTING, FINANCE AND MANAGEMENT Intell. Sys. Acc. Fin. Mgmt. 18, 161–176 (2011) Published online 31 January 2012 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/isaf.329

BUSINESS INTELLIGENCE (BI) SUCCESS AND THE ROLE OF BI CAPABILITIES OYKU ISIKa*, MARY C. JONESb AND ANNA SIDOROVAb a

Operations and Technology Management Center, Vlerick Leuven Gent Management School, Leuven, Belgium b

Information Technology & Decision Sciences Department, University of North Texas, Denton, TX, USA

SUMMARY Business intelligence (BI) has become the top priority for many organizations who have implemented BI solutions to improve their decision-making process. Yet, not all BI initiatives have fulfilled the expectations. We suggest that one of the reasons for failure is the lack of an understanding of the critical factors that define the success of BI applications, and that BI capabilities are among those critical factors. We present findings from a survey of 116 BI professionals that provides a snapshot of user satisfaction with various BI capabilities and the relationship between these capabilities and user satisfaction with BI. Our findings suggest that users are generally satisfied with BI overall and with BI capabilities. However, the BI capabilities with which they are most satisfied are not necessarily the ones that are the most strongly related to BI success. Of the five capabilities that were the most highly correlated with overall satisfaction with BI, only one was specifically related to data. Another interesting finding implies that, although users are not highly satisfied with the level of interaction of BI with other systems, this capability is highly correlated with BI success. Implications of these findings for the successful use and management of BI are discussed. Copyright © 2012 John Wiley & Sons, Ltd. Keywords: business intelligence; BI; BI capabilities; BI success; BI satisfaction

1.

INTRODUCTION

In response to an ever increasing amount of data to analyse and growing pressure to provide better and quicker responses to customers, many organizations have turned to business intelligence (BI) applications as a means to improve organizational decision making. Coined by the Gartner Group in 1990s, the term BI came to embrace a variety of information technology (IT)-based tools and approaches for helping organizations to make better use of the increasingly vast amounts of data accumulated from both internal and external sources. Thus, BI can be defined as a system comprised of both technical and organizational elements that presents historical information to its users for analysis and enables effective decision making and management support, for the overall purpose of increasing organizational performance (Eckerson, 2003; Watson et al., 2004). Organizations today collect enormous amounts of data from numerous sources, and using BI to collect, organize and analyse this data can add great value to a business (Gile et al., 2006). BI can also provide executives with right time data and allow them to make informed decisions to put them ahead of their competitors (Viaene et al., 2009). In 2010, BI topped the list of the most important application and technology developments in an annual survey of IT executives (Luftman and Ben-Zvi, 2010). Research also shows that most companies may focus their IT investment plans on BI (Evelson, 2011). * Correspondence to: Oyku Isik, Operations and Technology Management Center, Vlerick Leuven Gent Management School, Leuven, Belgium. E-mail: [email protected]

Copyright © 2012 John Wiley & Sons, Ltd.

162

O. ISIK ET AL.

Despite all the interest and investments, not all BI initiatives live up to management’s expectations. We posit that failures occur when organizations make BI adoption decisions without a clear understanding of the critical BI capabilities that define the success of BI applications. BI capabilities are critical functionalities that help an organization improve its performance and adapt to environmental change (Watson and Wixom, 2007). They range from data reliability to the flexibility of the BI in decisionmaking support. As organizations take advantage of these capabilities, their BI use increases, as does the value derived from BI applications (Watson and Wixom, 2007). In this study, we examine the relationship between various BI capabilities and user satisfaction with BI to provide a snapshot of the role of BI capabilities in BI success. BI satisfaction is used in this profile as a surrogate for BI success (Lonnqvist and Pirttimaki, 2006). We use survey data collected from BI professionals (business managers who use BI for strategic, tactical and operational decision making) to understand the role of BI capabilities. Specifically, we focus on 10 BI capabilities that are identified as important to BI success in the literature (Hostmann et al., 2007). They are quantitative and qualitative data quality, internal and external data source quality, internal and external data reliability, user access, flexibility, interaction with other systems, and risk management support capabilities. Our key findings suggest that users are most satisfied with internal and quantitative data capabilities of BI and least satisfied with external data capabilities. Overall, we observed that level of satisfaction for advanced BI capabilities (e.g. interaction with other systems and external data reliability) is lower. Another interesting finding is that although less than half of our respondents were satisfied with the level of interaction of BI with other systems, this capability was highly correlated with overall BI satisfaction. Section 2 reviews the literature for this study. We then describe the data collection and analysis in Section3. After reporting on the findings in Section 4, we conclude with a discussion on our findings and their implications in Section 5.

2.

LITERATURE BACKGROUND

BI has become an interest for information systems (IS), which implies systems composed of processes, people and information that improves the effectiveness and efficiency of organizations, researchers relatively recently and is a research stream that is still in development (Ponelis and Britz, 2011). Ever since the term BI has been coined, various definitions of BI have emerged in the academic and practitioner literature. While some broadly define BI as a holistic and sophisticated approach to crossorganizational decision support (Moss and Atre, 2003; Alter, 2004), others approach BI from a more technical point of view (Burton and Hostmann, 2005; White, 2005). BI, however, is comprised of both technical and organizational elements (Watson et al., 2006). In the most general sense, BI presents historical information to its users for analysis to enable effective decision making and for management support (Eckerson, 2003). For the purpose of this research, BI is defined as a system comprised of both technical and organizational elements that presents historical information to its users for analysis, to enable effective decision making and management support, for the overall purpose of increasing organizational performance. Companies realize benefit from their BI initiatives when they manage to link it with their business strategy (Viaene, 2008). Yet, most organizations struggle to measure how BI impacts the organization’s performance and thus have difficulty realizing how successful their BI initiative is. In the most general sense, BI success can be defined as the positive value an organization obtains from its BI investment. Yet, how an organization defines BI success depends on what benefits that organization needs from its Copyright © 2012 John Wiley & Sons, Ltd.

Intell. Sys. Acc. Fin. Mgmt., 18: 161–176 (2011) DOI: 10.1002/isaf

BUSINESS INTELLIGENCE (BI) SUCCESS AND THE ROLE OF BI CAPABILITIES

163

BI initiative (Miller, 2007). BI success may represent attainment of benefits such as improved profitability, increased efficiency (Eckerson, 2003) and reduced costs (Pirttimaki et al., 2006). For the purpose of this research, BI success is defined as the positive benefits organizations achieve through use of their BI. Another approach to measuring BI success is subjective measurement (Lonnqvist and Pirttimaki, 2006). This involves measuring the satisfaction of the decision maker with BI by asking questions regarding the effectiveness of the BI (Davison, 2001). This way, it is possible to learn what users think of various aspects of the system, such as ease of use, timeliness and usefulness. With this method, it is also possible to understand the perceptions of the extent to which the users realized their expected benefits with BI. Most of the BI users today meet their information-processing and decision-making needs through BI as it facilitates organizational information-processing capacity (Gallegos, 1999; Nelson et al., 2005). BI does so by combining data collection, data storage and knowledge management with analytical tools so that decision makers can convert complex information into effective decisions (Negash, 2004). The wide selection of BI platforms and applications available in the market today shows that there is a demand for a variety of BI functionalities. Different organizational characteristics and strategic goals may also require using different BI capabilities. BI capabilities are critical functionalities of BI that help an organization improve its adaptation to change as well as improve its performance (Watson and Wixom, 2007). With the right capabilities, BI can help an organization predict changes in product demand or detect an increase in a competitor’s new product market share and respond quickly by introducing a competing product (Watson and Wixom, 2007). BI capabilities are critical functionalities that help an organization improve its performance and its adaptation to change (Watson and Wixom, 2007). BI capabilities have remained largely unexamined in academic IS research, although they have been widely discussed in practitioner oriented literature (Eckerson, 2004; Watson and Wixom, 2007). For example, a recent Gartner Group research report about the evolution of BI relates BI capabilities related to information access and analysis to decision-making style within an organization (Hostmann et al., 2007), where information access and analysis includes methods and technologies used to collect and analyse the information. In our paper we delineate such information access and analysis capabilities and relate them to the overall BI success. Specifically, we examine such capabilities as the sources from which the data are obtained, data types, data reliability, user access in terms of authorization and/or authentication, flexibility of the system, interaction with other systems, and the level of risk supported by the system (see Table I for a summary of definitions). We discuss each of the capabilities below. 2.1. Data Quality Data quality refers to the consistency and comprehensiveness of the data (Giovinazzo, 2009). BI systems today have the capacity to work with different types of data such as numerical and non-numerical data, for which the quality is equally critical. The difference in the level of data quality is one of the factors that may explain why some organizations are successful with their BI initiative while some are not. Research implies that clean and relevant data is one of the most important BI success factors (Eckerson, 2003; Isik et al., 2010). The Data Warehousing Institute estimates that data quality issues alone cost US businesses over $600 billion dollars a year (Graham, 2008). 2.2. Data Source Quality A data source can be defined as the place where the data that are used for analysis reside and are retrieved (Hostmann et al., 2007). BI requires the collection of data from both internal and external Copyright © 2012 John Wiley & Sons, Ltd.


164

O. ISIK ET AL.

Table I. BI capability definitions

BI capability

Definition

Example reference

Quantitative data quality

The capability of BI to manage data that can be measured or Sukumaran and Sureka (2006) identified on a numerical scale, and analysed with statistical methods in an accurate, comprehensive and consistent way. Sukumaran and Sureka (2006) Qualitative data quality The capability of BI to manage data that is non-numerical and in text, image or sound format that needs to be interpreted for analysis purposes, in an accurate, comprehensive and consistent way. Internal data reliability The capability of BI to provide and manage internal data without Parikh and Haddad (2008) any conflicts, inconsistencies, in a reliable and up-to-date manner. External data reliability The capability of BI to provide and manage external data without Hostmann et al. (2007) any conflicts, inconsistencies, in a reliable and up-to-date manner. Harding (2003) Internal data source quality The capability of BI to store, retrieve and disseminate data from internal data sources such as a data warehouse, a data mart, or an online analytical processing (OLAP) cube in a concise, available and readily usable manner. External data source qualityThe capability of BI to store, retrieve and disseminate data from Harding (2003) external data sources such as websites, spreadsheets, suppliers and vendors in a concise, available and readily usable manner. White (2005) Interaction with other The capability of BI to provide enterprise business integration systems through a unified view of business data, business processes and business applications by managing the flow of events as well as a single personalized interface to the user. User access The capability of BI to manage different information access Hostmann et al. (2007) mechanisms to provide the right users the right accessibilities. Gebauer and Schober (2006) Flexibility The capability of BI to provide decision support when variations exist in the business processes, technology or the business environment in general. Risk management support The capability of BI to support decision making under conditions Harding (2003) of uncertainty when all the facts are not known.

sources (Harding, 2003). Internal data are generally integrated and managed within a traditional BI application information management infrastructure, such as a data warehouse, a data mart or an online analytical processing (OLAP) cube (Hostmann et al., 2007). External data includes the data that organizations exchange with customers, suppliers and vendors. This is rarely inserted into a data warehouse. Often, external data is retrieved from websites, spreadsheets, audio files and video files. The quality of these sources may have a direct impact on the effectiveness of the BI system and satisfaction of its users. It is evident in research that organizations such as Allstate insurance company and 1-800-Contacts retailer, who are renowned for their successful BI solutions, pay critical attention to the sources from which they obtain their data (Howson, 2006).

2.3. Data Reliability Data reliability refers to the dependability and accuracy of the data (Hannigan and Palendrano, 2002). Organizations make critical decisions based on the data they collect every day, so it is vital for them to have accurate and reliable data. Yet, there is evidence that organizations of all sizes are negatively impacted by imperfection, duplication and inaccuracy of the data they use (Damianakis, 2008). The Gartner Group, for example, estimates that more than 50 % of BI projects through 2007 failed because of data reliability issues (Graham, 2008). Research also shows that organizations that have won awards with their successful BI initiatives pay critical attention to the type of data they use, and the reliability of their data by acting early during their BI initiative and dedicating a working group to data-related issues (Howson, 2006). Copyright © 2012 John Wiley & Sons, Ltd.



165

2.4. Interaction with Other Systems Many organizations prefer having IS applications interacting at multiple levels so that enterprise business integration can occur (White, 2005). This integration can be at the data level, application level, business process level or user level, yet these four levels are not isolated from each other (White, 2005). There are different technologies available for these integration types. For example, enterprise information integration enables applications to see dispersed data as though it resided in a single database and enterprise application integration enables applications to communicate with each other using standard interfaces (Swaminathan, 2006). For organizations that use data from multiple sources and feed the data to multiple information systems, the quality of communication between these systems directly affects the overall performance (Swaminathan, 2006). 2.5. User Access One size does not fit all with BI. Organizations may need to employ different BI tools from different vendors because different groups of users have different reporting and analysis needs (Howson, 2006). Whether the organization prefers to use best-of-breed applications or a single BI suite, matching the tool capabilities with user types is always a good strategy (Howson, 2006). This capability also relates to the extent which the user can access the information they need for decision making with BI. While some organizations limit user access through practising authorization and access control, others prefer to allow full access to all types of users through a web-centric approach (Hostmann et al., 2007). For example, the BI tool provided by Lyzasoft Inc. is an all-in-one tool that includes integrated reporting, ad hoc querying and connectivity to data sources as a client-side desktop application (Swoyer, 2008). On the other hand, QlikTech International developed QlikView, a web-centric BI application that provides analytical and reporting capabilities for all types of users, especially easier to use for nontechnical users (Havenstein, 2006). While web-centric systems are generally shared by large numbers of users, desktop applications are mostly dedicated to specific users (Hostmann et al., 2007). 2.6. Flexibility Flexibility refers to the capability of BI to provide decision support when variations exist in the business processes, technology or the business environment in general (Gebauer and Schober, 2006). The amount of flexibility directly impacts the performance of a system: while insufficient flexibility may prevent using the system for certain situations, too much flexibility may increase complexity and reduce usability (Silver, 1991; Gebauer and Schober, 2006). It is critical that BI provides the necessary flexibility in the decision-making process, especially for applications or processes where innovation and dynamism are required (Dreyer, 2006). Technology does not always support exceptional situations, although organizations need the flexibility and robust functionality to obtain the optimum potential from BI (Antebi, 2007). Where the problems about which decisions are made require flexibility in their assessment, this capability is key to BI success (Clark et al., 2007). 2.7. Risk Management Support Risk management support refers to the capability to support decisions under conditions of uncertainty (Harding, 2003). People, processes, technology and even external events can cause risks for an Copyright © 2012 John Wiley & Sons, Ltd.


166

O. ISIK ET AL.

organization (Imhoff, 2005), and how successfully the organization manages that risk is a capability that directly impacts BI performance. Risk and uncertainty exist in every business decision, and organizations may use BI to minimize uncertainty and make better decisions. In particular, organizations which have specific and well-defined problems to solve have a low tolerance for risk and thus may be using BI to manage that risk (Hostmann et al., 2007). As organizations take advantage of these capabilities, their BI use increases, and so does the maturity level of BI (Watson and Wixom, 2007). Mature BI increases organizational responsiveness, which positively affects organizational performance. Thus, it is important to recognize BI capabilities to better apply it to strategic needs (Ross et al., 1996).

3.

DATA COLLECTION

The target population for this research consisted of business managers who use BI for strategic, tactical and operational decision making across a range of organizations and industries. Data were collected using an online survey between July and September of 2008. The firms were randomly selected and the contact information of decision makers was obtained from a publicly available mailing list of a market research company; 116 responses from a variety of industries, company sizes, organizational levels, functional areas and BI experience were received (Table II). In total, the survey link was emailed to over 5000 recipients. This corresponds to a response rate lower than 1 %, which is not necessarily surprising for web-based surveys (Basi, 1999). The survey included questions measuring user satisfaction with BI as well as user assessments of specific BI capabilities. Items measuring user satisfaction were selected from Hartono et al.’s (2007) management support system success dimensions and Doll and Torkzadeh’s (1988) end-user satisfaction measure. BI capabilities were operationalized with items developed based on the Gartner Group reports as well as other practitioner-oriented publications from the Data Warehousing Institute (Harding, 2003; Sukumaran and Sureka, 2006; Damianakis, 2008). Responses for all items in the survey were captured on a five-point Likert scale.

4.

FINDINGS

4.1. Satisfaction with BI Respondents were asked how satisfied they were with several aspects of their BI systems and how satisfied they were with BI overall (Figure 1). We found no significant differences in the level of satisfaction among users from different industries, functional areas or organizational level. Results do indicate that users with more BI experience were more satisfied than less-experienced BI users, in terms of the user friendliness of and the decision-making support provided by their BI. One reason for this may be that more advanced BI users are more aware of or make use of a greater variety of BI capabilities and have been using BI for a longer time. We assessed BI satisfaction across all respondents, regardless of experience or other characteristics. Almost 70 % of all respondents indicated that they were satisfied or strongly satisfied with their BI overall (Table III). Only 12 % of the respondents reported some level of dissatisfaction (strongly dissatisfied or dissatisfied). With respect to specific aspects of BI, the lowest level of satisfaction reported was for user friendliness of BI, with only 58.6 % of users indicating some level of satisfaction. The highest Copyright © 2012 John Wiley & Sons, Ltd.


167


Table II. Profile of respondents Category Functional area Information technology Management Finance Marketing Sales Supply chain Operations research Other Level in organization Executive Middle Operational BI experience New BI user Intermediate BI user Advance BI user Industry Manufacturing Insurance/real estate/legal Medical/health Transportation/utilities Wholesale/retail/distribution Banking Data processing services Education Business service/consultant Number of company employees $100 million $100 million to $499 million $500 million to $1 billion ≥$1 billion

Respondents (%) 46.6 9.5 7.8 7.8 5.2 2.6 0.9 19.8 18.1 40.5 25 12.1 37.1 50.9 10.3 9.5 8.6 7.8 7.8 5.2 4.3 11.2 14.7 23.3 9.5 8.6 23.3 9.5 25.9 32.8 12.9 9.5 34.5

level of satisfaction reported was for the ability of BI to provide precise information, with 78.5 % of respondents satisfied or strongly satisfied. Overall, the results suggest that the majority of respondents were satisfied with BI. Yet, fewer than 25 % reported being strongly satisfied with any aspect of their BI. This is an indication that BI users and managers are not yet getting full leverage from BI in terms of precision, timeliness, decision-making support or even ease of use (user friendliness) that they seem to expect. Table IV presents the means and standard deviations of responses to the BI satisfaction items. 4.2. Assessment of BI Capabilities We next examined user assessment of the 10 specific BI capabilities. Respondents were most satisfied with quantitative data quality and internal data reliability (Table V). This is not surprising given the attention that is paid to these two capabilities by organizations, as well as vendors and consultants. We also found high levels of satisfaction of the risk management support capabilities of BI and the level of user access to BI applications. The capabilities with which respondents were least satisfied were Copyright © 2012 John Wiley & Sons, Ltd.


168

O. ISIK ET AL.

Figure 1. Satisfaction with BI.

the quality of external data sources, external data reliability and the extent of BI interaction with other systems. For each of the capabilities we used several questions to capture satisfaction with various dimensions of each in order to provide a richer picture of BI satisfaction. These are discussed in the following sections. Quantitative versus Qualitative Data Quality Both quantitative and qualitative data quality were assessed based on four dimensions: accuracy, comprehensiveness, consistency and overall quality of the data (Table VI). Users were generally satisfied with all aspects of quantitative data quality. Over 70 % of respondents were either satisfied or strongly satisfied that their BI provided overall high-quality quantitative data that are both accurate and consistent.1 In addition, 68 % of respondents indicated that their BI system provided comprehensive quantitative data. Users were notably less satisfied with the quality of qualitative data. Fewer than 50 % of respondents indicated that they were satisfied with the extent to which their BI system provided accurate, consistent, comprehensive qualitative data. Approximately 20 % of respondents were dissatisfied with the overall quality of qualitative data provided by their BI. Internal versus External Data Reliability Internal and external data reliability were assessed based on four dimensions: inconsistencies and conflicts associated with the data, data accuracy, regular updates of the data and overall data reliability. Survey respondents were generally satisfied with the consistency, reliability and accuracy of internal data. Over 75 % of respondents were satisfied with the reliability of the internal data collected by their BI system, while fewer than 30 % reported dissatisfaction with the extent of conflicts or inconsistencies in their internal data. Over 75 % indicated that they are satisfied that the data for their BI system is regularly updated. Users were, however, less satisfied with the overall reliability, consistency and accuracy of external data. Fewer than 40 % expressed satisfaction with the reliability or accuracy of the external data used by BI systems (Table VII). For ease of reading, from this point forward we use the term ‘satisfied’ to indicate respondents who were either satisfied or strongly satisfied, and the term ‘dissatisfied’ to indicate respondents who were either dissatisfied or strongly dissatisfied.

1



169


Table III. BI satisfaction response percentages BI satisfaction items

Strongly Dissatisfied Neither satisfied nor Satisfied Strongly dissatisfied dissatisfied satisfied

The BI that I am using overall How well the BI that I am using provides precise information I need How well the BI that I am using supports my decision making How well the BI that I am using provides information I need in time How user friendly the BI that I am using is

3.4 0.9

8.6 10.3

18.1 10.3

56.0 56.9

13.8 21.6

1.7

10.3

18.1

50.9

19.0

2.6

13.8

17.2

47.4

19.0

4.3

11.2

25.9

36.2

22.4

Table IV. BI satisfaction response descriptive statistics BI satisfaction items

Mean

Standard deviation

The BI that I am using overall How well the BI that I am using provides precise information I need How well the BI that I am using supports my decision making How well the BI that I am using provides information I need in time How user friendly the BI that I am using is

3.68 3.88 3.75 3.66 3.61

0.938 0.896 0.941 1.021 1.086

Table V. BI capabilities descriptive statistics BI capability Quantitative data quality Internal data reliability Risk management support User access Internal data source quality Flexibility Qualitative data quality Interaction with other systems External data reliability External data source quality 1

Mean1

Standard deviation

3.89 3.86 3.85 3.77 3.68 3.51 3.34 3.24 3.21 2.89

0.842 0.779 0.907 0.917 0.929 0.890 1.022 0.947 0.870 1.045

Measured on a five-point scale, where 1 = strongly dissatisfied and 5 = strongly satisfied.

Table VI. Quantitative versus qualitative data quality response percentages Data quality from BI

Strongly dissatisfied

Dissatisfied

Neither satisfied nor dissatisfied

Satisfied

Strongly satisfied

0.9 2.6 2.6 2.6

5.2 3.4 6.0 7.8

18.1 26.7 19.8 19.8

51.7 49.1 47.4 49.1

24.1 18.1 24.1 20.7

6.9 6.0 6.0 6.9

15.5 11.2 15.5 12.1

31.9 34.5 33.6 34.5

35.3 37.9 34.5 37.1

10.3 10.3 10.3 9.5

Quantitative Accuracy of quantitative data Comprehensiveness of quantitative data Consistency of quantitative data Quality of quantitative data Qualitative Quality of qualitative data Accuracy of qualitative data Comprehensiveness of qualitative data Consistency of qualitative data Copyright © 2012 John Wiley & Sons, Ltd.


170

O. ISIK ET AL.

Table VII. Internal versus external data reliability response percentages Data reliability

Internal Reliability of internal data collected for BI Resolution of inconsistencies and conflicts in the internal data collected for BI Accuracy of internal data collected for BI Recency of internal data collected for BI External Reliability of external data collected for BI Resolution of inconsistencies and conflicts in the external data collected for BI Accuracy of external data collected for BI Recency of external data collected for BI


Dissatisfied


Satisfied

Strongly satisfied

0.9 6.9

6.9 20.7

15.5 25.0

57.8 40.5

19.0 6.9

0.9 0.9

9.5 5.2

25.0 19.0

44.8 43.1

19.8 31.9

8.6 6.0

10.3 16.4

39.7 36.2

35.3 34.5

6.0 6.9

4.3 6.9

10.3 12.9

44.8 42.2

32.8 30.2

7.8 7.8

The Quality of Data Sources With regard to data sources for their BI systems, the users were generally satisfied with the quality of internal data sources and less satisfied with the quality of external data sources. Quality of data sources was assessed on four dimensions: availability, usability, conciseness and ease of understanding. Availability and usability of internal data sources were assessed as adequate by 73 % and 66 % of respondents respectively. Fewer users, however, were satisfied with the conciseness and ease of understanding of internal data sources (55 % and 40 % respectively). An even lower level of satisfaction was associated with the quality of external data sources. Less than 30 % of respondents were satisfied with any of the aspects of external data quality (Table VIII). User Access In general, the majority of users were satisfied with various aspects of their access to BI system (Table IX). User access was assessed on two dimensions: extent of access to BI data and the vehicle through which access is provided. Approximately 70 % were satisfied with the extent of access to BI data and capabilities and over 60 % were satisfied with the way they access BI applications. Risk Management Support We found a good bit of variation in the level of satisfaction with different risk management support capabilities (Table X). Only 45 % of respondents were satisfied with the way their BI system supports Table VIII. The quality of data sources response percentages Data source quality

Internal Availability of internal data sources used for BI Usability of internal data sources used for BI Ease of understanding of internal data sources used for BI Conciseness of internal data sources used for BI External Availability of external data sources used for BI Usability of external data sources used for BI Ease of understanding of external data sources used for BI Copyright © 2012 John Wiley & Sons, Ltd.


Dissatisfied


Satisfied

Strongly satisfied

2.6 2.6 7.8

7.8 11.2 13.8

17.2 20.7 24.1

46.6 44.8 41.4

25.9 20.7 12.9

6.0

17.2

37.1

30.2

9.5

11.2 13.8 12.9

18.1 21.6 24.1

41.4 37.9 37.9

19.0 23.3 20.7

10.3 3.4 4.3


171


Table IX. User access response percentages User access quality

Quality of the way the user accesses BI Access to the information user needs in BI How well BI fits the types of decisions user makes using BI


Dissatisfied


Satisfied

Strongly satisfied

3.4 5.2 2.6

14.7 8.6 6.0

20.7 13.8 22.4

42.2 40.5 49.1

19.0 31.9 19.8

Table X. Risk management support response percentages Risk

Strongly Dissatisfied Neither satisfied Satisfied Strongly dissatisfied nor dissatisfied satisfied

BI support of decisions associated with high level of risk (e.g. entering a new market, hiring a new manager) BI support of decisions motivated by exploration and discovery of new opportunities (e.g. starting a new business line, creating a new product design) BI support in minimizing uncertainties in decision making process BI support in managing risk by monitoring and regulating the operations (e.g. monitoring key performance indicators (KPIs), customizing alerts or creating dashboards)

2.6

17.2

35.3

37.9

6.9

4.3

12.1

27.6

45.7

10.3

2.6 2.6

6.0 11.2

19.0 17.2

56.9 47.4

15.5 21.6

decisions associated with high levels of risk, and 20 % disagreed that their BI system even offers such capabilities. Over 55 % of respondents indicated that they were satisfied with how well their BI systems supported exploration of new opportunities (versus 20 % who indicated the lack of such capabilities). The majority of respondents (over 70 %) were satisfied that their BI system helps them reduce uncertainty in their decision making and that BI helps them manage risk through operational controls. Flexibility Users were not very satisfied with the flexibility of their BI system (Table XI). Fewer than 25 % were satisfied with their BI system’s ability to easily accommodate changes to their business environment, allow for rapid changes or make it easy to deal with exceptional situations. Thus, a picture emerges of BI solutions that are fairly static in their ability to respond to rapidly changing environments. Interaction with Other Systems Users were split in their satisfaction with the level of interaction of BI system with other systems within the enterprise (Table XII). Just over half (53 %) of respondents were satisfied with their BI system’s ability to offer a unified view of business data and processes. Fewer than 45 % were satisfied with their Table XI. Flexibility response percentages Flexibility

How quickly BI accommodates changes in business requirements How well BI makes it easier to deal with exceptions How well the BI components are organized and integrated to allow for rapid changes Copyright © 2012 John Wiley & Sons, Ltd.


Dissatisfied


Satisfied

Strongly satisfied

6.0

19.0

28.4

33.6

12.9

3.4 4.3

20.7 18.1

30.2 26.7

36.2 37.9

9.5 12.9


172

O. ISIK ET AL.

Table XII. Interaction with other systems response percentage Interaction with other systems

How well BI provides a unified view of business data and processes How well BI provides links among multiple business applications How well BI provides a comprehensive electronic catalog of the various enterprise information resources in the organization How well BI provides easy and seamless access to data from other applications and systems

Strongly Dissatisfied Neither satisfied Satisfied Strongly dissatisfied nor dissatisfied satisfied 4.3 4.3 8.6

15.5 6.9 16.4

26.7 20.7 30.2

40.5 48.3 32.8

12.9 19.8 12.1

10.3

21.6

29.3

30.2

8.6

BI system’s ability to offer a comprehensive catalogue of enterprise information resources and fewer than 40 % were satisfied with their BI offering easy and seamless access to data from other systems.

4.3. Overall Assessment of BI Capabilities In sum, our findings regarding user satisfaction with BI capabilities are as follows: • users are generally satisfied with BI overall; • users are most satisfied with internal and quantitative data capabilities of BI; • users are next most satisfied with BI capabilities to support risk in decision making and user access to BI; • users are least satisfied with external data capabilities. In spite of the overall relatively high level of satisfaction with BI capabilities, we observe a pattern where the users are more satisfied with basic BI capabilities such as quantitative data quality, user access and flexibility. Their level of satisfaction is lower for more advanced capabilities, such as the quality of qualitative data, quality of external data sources and external data reliability. Because satisfaction with BI capabilities can be considered as an indicator of how solid these capabilities are, we conclude that quantitative and internal data capabilities are quite well done. Also well done are the way in which users access their BI systems, the BI responsiveness to change (flexibility), the capability to smoothly interact with other systems and the capability to support risk in decision making. Most decisions require good user accessibility to sound quantitative data that can be pulled from a variety of systems and require some degree of flexibility in the decision-making process (Hostmann et al., 2007). These basic capabilities form the foundation of a useful, or successful, BI. Other capabilities, such as those relating to external data sources and qualitative data, are more advanced and are required for a different type of decision making (Hostmann et al., 2007). Our findings indicate that these more advanced capabilities are less well done. Thus, a picture of BI solutions that are solidly grounded in basic capabilities emerges. These solutions are most appropriate for supporting decisions that largely require looking backward to answer ‘Where have we been?’ or ‘Where are we now?’ questions. Organizations have not widely made use of more advanced BI solutions that adequately support forward-looking strategic decisions which require a broader set of data and sources to answer questions such as ‘Why are we where we are?’ and ‘Where can or should we go?’ BI Capabilities and BI Success We next examined, in more detail, the relationship between satisfaction with specific BI capabilities and overall satisfaction with BI. Findings showed that all but one of the 10 capabilities examined were significantly and positively correlated with overall BI satisfaction at a = 0.05 (Table XIII). Specifically, Copyright © 2012 John Wiley & Sons, Ltd.



173

Table XIII. Correlations between BI success and satisfaction with BI capabilities BI capabilities

BI success Pearson correlation

Sig. (2-tailed)

N

0.687* 0.564* 0.519* 0.508* 0.507* 0.368* 0.366* 0.293* 0.194** 0.172

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.037 0.065

116 116 116 116 116 116 116 116 116 116

User access Flexibility Quantitative data quality Interaction with other systems Risk management support Internal data reliability Internal data source quality Qualitative data quality External data source quality External data reliability *Correlation is significant at the 0.01 level (2-tailed). **Correlation is significant at the 0.05 level (2-tailed).

higher data quality, higher quality of data sources, better user access quality, higher flexibility, and better risk management support, and stronger interaction with other systems are associated with higher levels of overall BI satisfaction. Although external data quality was not significantly correlated with overall BI satisfaction, at a = 0.05, it was at a = 0.10. Of the five capabilities that were the most highly correlated with overall satisfaction with BI, only one was specifically related to data (quantitative data quality). Furthermore, only three of the five capabilities most closely correlated with BI success are characterized by a high level of user satisfaction (quantitative data quality, risk management support and user access). Although less than 50 % of users were satisfied with the level of interaction of BI with other systems and with the flexibility of their BI system, these two capabilities were among the top five most correlated with overall BI satisfaction. This suggests that while BI systems are largely good at providing some of the capabilities that are critical to BI success, there is room for improvement in others. In summary, our findings suggest that: • adequate user access is a cornerstone of the overall user satisfaction with BI; • users value the most BI capabilities that allow them to deal with uncertainty and change in the environment; • users see the value of BI in providing capabilities for interaction among various enterprise applications; • BI capabilities that most strongly relate to BI success are not necessarily the ones with which users are most satisfied.

5.

CONCLUSIONS

The purpose of this study was to create a snapshot of user satisfaction with BI as well as their assessment of specific BI capabilities, and to examine the relationship between such capabilities and BI satisfaction. Our findings indicate that the majority of respondents are relatively satisfied with BI overall and have a positive assessment of specific BI capabilities. Yet, our results point out several implications for developing better BI solutions. For example, improving the level of interaction between BI and other enterprise systems may be a key to BI success. Although users are not highly satisfied with this Copyright © 2012 John Wiley & Sons, Ltd.


174

O. ISIK ET AL.

capability, it is strongly correlated with the level of satisfaction with BI overall. In addition, many respondents were not satisfied with BI capabilities associated with providing access to external data. Although such capabilities were not as strongly related to overall BI satisfaction as others, this may suggest that the BI usage among our respondents is more internally focused, or it may be an indication of lower expectations that the users have of such external data capabilities. It is also possible that as the BI practice matures, users are likely to pay more attention and expect more of such external data capabilities. Therefore, improving external data capabilities presents another important direction in managing BI. Our findings also highlight the strong relationship between user assessment of specific BI capabilities and their overall satisfaction with BI. Users particularly value adequate user access, BI capabilities that help them deal with uncertain and changing environment and the level of interaction of BI systems with other systems. As BI continues to mature, it becomes increasingly critical to the support of decisions that keep organizations competitive. Our findings indicate that, largely, BI solutions provide a solid set of basic capabilities to support decisions grounded in basic questions like ‘What has happened?’ or ‘What is happening in our organization?’ Developing stronger, more advanced BI capabilities in organizations to support decisions based on the answers to forward-looking questions may be key to providing BI that successfully facilitates decisions in today’s dynamic, information-based organizational environments.

REFERENCES

Alter A. 2004. A work system view of DSS in its fourth decade. Decision Support Systems 38(3): 319–327. Antebi O. 2007. Managing by exception. Panorama Business Intelligence, http://www.panorama.com/blog/?p=13 (accessed 21 January 2012). Basi RK. 1999. WWW response rates to socio-demographic items. Journal of the Market Research Society 41(4): 397–401. Burton B, Hostmann B. 2005. Findings from Sydney Symposium: Perceptions of Business Intelligence. Gartner Research, Stamford, CT, http://www.gartner.com (accessed 14 May 2011). Clark TD, Jones MC, Armstrong C. 2007. The dynamic structure of management support systems:theory development, research focus and direction. MIS Quarterly 31(3): 579–615. Damianakis S. 2008. The ins and outs of imperfect data. Information Management Direct, http://www.dmreview. com/dmdirect/2008_77/10001491-1.html?portal=data_quality (accessed 14 March 2011). Davison L. 2001. Measuring competitive intelligence effectiveness: insights from the advertising industry. Competitive Intelligence Review 12(4): 25–38. Doll WJ, Torkzadeh G. 1988. The measurement of end-user computing satisfaction. MIS Quarterly 12(2): 259–274. Dreyer L. 2006. The “Right Time” for Operational Business Intelligence?TDWI What Works, http://tdwi.org/ articles/2006/05/09/the-right-time-for-operational-business-intelligence.aspx (accessed 21 January 2012). Eckerson WW. 2004. Gauge your data warehouse maturity. Information Management Magazine, http://www. information-management.com/issues/20041101/1012391-1.html (accessed 21 January 2012). Eckerson WW. 2003. Smart companies in the 21st century: the secrets of creating successful business intelligence solutions. TDWI The Data Warehousing Institute Report Series, http://tdwi.org/research/2003/07/exective-summarysmart-companies-in-the-21st-century-the-secrets-to-creating-successful-business-int.aspx (accessed 14 March 2011). Evelson B. 2011. Trends 2011 and beyond: business intelligence. Forrester Research, Cambridge, MA, 31 March, http://www.forrester.com (accessed 14 May 2011). Gallegos F. 1999. Decision support systems: an overview. Information Strategy 15(2): 42–47. Gebauer J, Schober F. 2006. Information system flexibility and the cost efficiency of business processes. Journal of the Association for Information Systems 7(3): 122–145. Gile K, Kirby JP, Karel R, Teubner C, Driver E, Murphy B. 2006. Topic overview: business intelligence. Forrester Research, Cambridge, MA, 5 June, http://www.forrester.com (accessed 14 May 2011). Copyright © 2012 John Wiley & Sons, Ltd.



175

Giovinazzo W. 2009. BI: Only as Good as its Data Quality. Information Management Special Reports, http://www. information-management.com/specialreports/2009_157/business_intelligence_bi_data_quality_governance_decision_making-10015888-1.html (accessed 21 January 2012). Graham P. 2008. Data Quality: You Don’t Just Need a Dashboard! Strategy Execution. Information Management Magazine, http://www.information-management.com/issues/2007_50/10001727-1.html (accessed 21 January 2012). Hannigan TJ, Palendrano C. 2002. Personalization Can Be Quite Dynamic. Information Management Magazine, http://www.information-anagement.com/issues/20021001/5798-1.html (accessed 21 January 2012). Harding W. 2003. BI crucial to making the right decision. Financial Executive 19(2): 49–50. Hartono E, Santhanam R, Holsapple CW. 2007. Factors that contribute to management support system success: an analysis of field studies. Decision Support Systems 43(1): 256–268. Havenstein H. 2006. QlikTech looks to broaden access to BI data. ComputerWorld, http://www.computerworld. com/action/article.do?command=viewArticleBasic&articleId=9004369 (accessed 21 January 2012). Hostmann B, Herschel G, Rayner N. 2007. The evolution of business intelligence: the four worlds. Gartner Research Report, http://www.gartner.com/DisplayDocument?id=509002 (accessed 14 March 2011). Howson C. 2006. The Seven Pillars of BI Success, Information Week, http://www.informationweek.com/news/ software/bi/191902420 (accessed 21 January 2012). Imhoff C. 2005. Risky business! Using business intelligence to mitigate operational risk. Information Management Magazine, http://www.information-management.com/issues/20050801/1033577-1.html (accessed 21 January 2012). Isik O, Jones M, Sidorova A. 2010. Business intelligence success: an empirical evaluation of the role of BI capabilities and the decision environment. Proceedings of the BI Congress II: Pre-ICIS Conference, 11–12 December, St. Louis, MO. Lonnqvist A, Pirttimaki V. 2006. The measurement of business intelligence. Business Intelligence 23(1): 32–40. Luftman J, Ben-Zvi T. 2010. Key issues for IT executives 2009: difficult economy’s impact on IT. MIS Quarterly Executive 9(1): 203–213. Miller D. 2007. Measuring BI success: business goals and business requirements. Information Management Online (October), http://www.dmreview.com/news/10000100-1.html (accessed 14 May 2011). Moss LT, Atre S. 2003. Business Intelligence Roadmap: The Complete Project Lifecycle for Decision-Support Applications. Addison-Wesley: Boston, MA. Negash S. 2004. Business intelligence. Communications of the Association for Information Systems 13: 177–195. Nelson RR, Todd PA, Wixom BH. 2005. Antecedents of information and system quality: within the context of data warehousing. Journal of Management Information Systems 21(4): 199–235. Parikh AA, Haddad J. 2008. Right-time information for the real-time enterprise timely information drives business. Information Management Direct, http://www.information-management.com/infodirect/2008_92/10002003-1. html?portal=data_quality (accessed 14 March 2011). Pirttimaki V, Lonnqvist A, Karjaluoto A. 2006. Measurement of business intelligence in a Finnish telecommunications company. Electronic Journal of Knowledge Management 4(1): 83–90. Ponelis S, Britz JJ. 2011. The role of business intelligence in information-intensive small businesses: initial results from an interpretive study. MWAIS 2011 Proceedings, Paper 23. Ross JW, Beath CM, Goodhue DL. 1996. Develop Long-Term Competitiveness through IT Assets. Sloan Management Review 38(1): 31–44. Silver MS. 1991. Systems that Support Decision Makers: Description and Analysis. Wiley & Sons: Chichester, United Kingdom. Sukumaran S, Sureka A. 2006. Integrating structured and unstructured data using text tagging and annotation. Business Intelligence Journal 11(2): 8–17. Swaminathan A. 2006. Enterprise Information Integration: A Technology for Providing Integrated Views. What Works in BI and DW,Volume 22, http://tdwi.org/articles/2006/10/23/enterprise-information-integration-atechnology-for-providing-integrated-views.aspx (accessed 21 January 2012). Swoyer S. 2008. Lyza Empowers New Class of BI Consumers. Enterprise Systems, http://esj.com/articles/2008/09/ 24/lyza-empowers-new-class-of-bi-consumers.aspx (accessed 21 January 2012). Viaene S. 2008. Linking business intelligence into your business. IT Professional 10(6): 28–34. Viaene S, De Hertogh S, Lutin L, Maandag A, Den Hengst S, Doeleman R. 2009. Intelligence-led policing at the Amsterdam-Amstelland Police Department: operationalized business intelligence with an enterprise ambition. Intelligent Systems in Accounting, Finance and Management 16(4): 279–292. Copyright © 2012 John Wiley & Sons, Ltd.


176

O. ISIK ET AL.

Watson HJ, Wixom B. 2007. Enterprise agility and mature BI capabilities. Business Intelligence Journal 12(3): 4–6. Watson HJ, Fuller C, Ariyachandra T. 2004. Data warehouse governance: best practices at Blue Cross and Blue Shield of North Carolina. Decision Support Systems 38(3): 435–450. Watson HJ, Wixom BH, Hoffer JA, Anderson-Lehman R, Reynolds AM. 2006. Real-time business intelligence: best practices in Continental Airlines. Business Intelligence 23(1): 7–18. White C. 2005. The next generation of business intelligence: operational BI. Information Management Magazine, http://www.information-management.com/issues/20050501/1026064-1.html (accessed 14 March 2011).

