Measuring user satisfaction with B2E portals ... - Semantic Scholar

Measuring user satisfaction with B2E portals Dr. Ly-Fie Sugianto Clayton School of IT, Monash University [email protected] Dewi Rooslani Tojib Clayton School of IT, Monash University [email protected]

An increasing number of organisations have implemented business-to-employee (b2e) portals as a support system for their workforce. B2e portals assist employees to perform their tasks more effectively and efficiently. Organisations use b2e portals to disseminate information, facilitate collaborations and communications in the hope of increasing work productivity. This paper discusses the factors driving the adoption of b2e portals and examines some issues that could inhibit its proliferation. Moreover, it presents user satisfaction as a means of measuring the success of portal usage. This measure of user satisfaction with the b2e portal is based on an empirical study of several universities in Australia.

Introduction A distinctive characteristic of this area of globalization is the voluminous flow of information, as seen in today’s workplaces. With the increasing importance of information as a valuable asset, the mainstream acceptance of the internet and its enabling technology comes as no surprise at all. Many organisations today deploy internet and mobile devices as they believe that their employees need to be empowered with all the tools they need to be efficient and effective. Today, these tools comprise both information and applications. Everyone in an organisation must have access to these tools, in context, in order to achieve his/her own objectives and accomplish the goals of the organisation. With today's enterprises offering a growing portfolio of applications and rich information sources, it has become increasingly difficult to navigate around the rich information environment to find the right information. A solution that has emerged in the past decade is to consolidate and personalise the information based on the role of the user of the computerised system. This system is known as a portal. When considering the taxonomy of portals, there are different classifications of portals based on: the target audience, the deployment purpose, the scope, the technology, and their functionality (refer to [1] for more detail discussion). Specifically in relation to corporate portals, Eckerson in [2] characterized four generations of portals: the first generation is the Referential portal featuring a search engine and a hierarchical function; the second generation is the Personalized portal with the ability to personalize views based on the user’s preference and specification; the third generation is the Interactive type where the portal is embedded with applications to support work productivity; and, the fourth generation is the Specialized portal that hosts applications whose accessibility depends upon the role of the employee within the organisation. B2e portals, generally known as staff portals, are a type of corporate portal, whose purpose is to support the access and availability of customized and personalized information for employees. Through internet connection, employees are able to access news, resources and applications in the password-protected portals. The portals link employees to a range of internal and external information and connect them with other employees using desktops,

© Monash Business Review Volume 3 Issue 1 – April 2007

1

notebooks and Personal Digital Assistants (PDAs). With the increasing proliferation of internet use and mobile technologies, it stands to reason that resources accessible through portals will be ubiquitous. The use of employee portals has been growing steadily. Our recent internet survey found that b2e portals have been implemented worldwide in various industries (see Table 1) including health care services, aerospace, automobile, education, energy utilities, financial services, high technology, legal, manufacturer, media telecommunications, professional services, public sector, and retailers. Merrill Lynch Capital Markets envisage that portals will become a US$14 billion business outnumbering the growth rates of other hot enterprise applications markets [3]. The growth of such portals is believed to remain strong [4]. The same view is also reported by Forrester Research, Inc. [5]. According to Banks [6], the number of organisations implementing b2e portals is also increasing in Australia. B2e portals have the potential to deliver many benefits to organisations and their employees if they are implemented in a rigorous manner [7]. Research on factors influencing portal adoption [1], its use in various industries [7], and modeling user satisfaction [8] have been conducted at Monash University. Table 1: B2E Portals Implementation in Various Industrial Sectors

INDUSTRY COMPANY Aerospace: America West Airlines American Airlines Boeing Co Pratt & Whitney British Airways Air France Automotive: Daimler Chrysler Ford Motor Company General Motor Banking & Finance: ABN Amro Bank of America USB AG Visa Barclays Bank Union Bank of California Energy utilities: Cinergy Shell Enbridge Inc. Pioneer Natural Resources Syncrude Canada ENI Government: Dept. of Edu. and Skills (UK) Health care/pharmaceutical: Clarion Healthcare Network GlaxoSmithKline VHA Hospital Modelo Insurance: Aetna Life Insurance Media telecommunications: BBC Comcast

PORTAL SOLUTION

VENDOR

COMPASS JetNet Boeing Employee Portal

Computer Assoc. Int. Inc. Plumtree Software Inc. Plumtree Software Inc. Plumtree Software Inc. In-house development Sun Microsystems Inc.

ESS Portal

my.ford.com Socrates

Visa Employee Portal

MyEni

DC e-Life Plumtree Software Inc. Workscape Inc. In-house development Broad Vision, Inc. Viador, Inc. Abilizer Solutions, Inc. Autonomy Inc. MineShare Inc. Plumtree Software Inc. SAP Enterprise Portal Plumtree Software Inc. Plumtree Software Inc. Plumtree Software Inc. SAP Enterprise Portal Plumtree Software Inc.

myGSK @Work

ATG Portal Plumtree Software Inc. Plumtree Software Inc.

Viador Inc

MyComcast

SAP Enterprise Portal Plumtree Software Inc.

© Monash Business Review Volume 3 Issue 1 – April 2007

2

Ericsson MTV Networks Spanish Telefonica Public relations: Ketchum Public Relations Retail industry: Best Buy Guess Hy-Vee Technology: Hewlett Packard Sun Microsystem Cisco Systems Hitachi Software Consumer goods & services: Pulmuone Maple Leaf Foods BAT Mexico Education: State University of New York Monash University Hotel: Radisson Edwardian Hotels Electronics: Philips Electronics

Zoops Global Network Rufus

Plumtree Software Inc.

myKGN

Plumtree Software Inc.

TagZone GUESS Express The Hy-Vee Net

Plumtree Software Inc. Plumtree Software Inc. Plumtree Software Inc.

@HP

Hewlett Packard Abilizer Solutions, Inc. Viador Inc IBM

myMapleLeaf BAT Mexico Interact

My.monash

Plumtree Software Inc. Plumtree Software Inc. In-house development with the assistance of Accenture Brio Technology In-house development CITRIX Systems Corechange

Self-service model The self-service model is the key attraction of b2e portal deployment in many organisations. The self-service model involves modifying internal processes so that paper-based documents (manual) are transferred to online documents (automation). An efficient online self-service facility allows employees to initiate these services at their own convenience. This means that responsibility for the administration of all paper work rests with the employees themselves. As a result, the amount of time that HR staff spends on administrative tasks dramatically reduces. The self-service application also reduces calls to HR staff by 75% and eliminates 80% of the paperwork [9]. Consequently, HR staff can focus their efforts on achieving more strategic goals for the organisations, such as reducing turnover and developing skill inventories. The self-service model has been implemented across different functional units, including HR Self-service (HRSS) system, the Benefit Self-service (BSS) system, and the Finance and Accounting self-service (FASS) system. HR Self-service provides information about company calendar listing holidays and paydays, employee policies, job listings, and a contact list for the entire HR department. Using the HRSS system, employees can maintain their contact information, check leave balances, process new-hire applications, manage performance appraisals, and plan compensation. The BSS system provides employees with general information about benefits and allows them to enrol in and manage health care and retirement plans, recreation, career planning, and online training. Other benefits typically offered by organisations include discounts for travel, hotel, car rental and entertainment. Access to such benefits through a self-booking system is highly desirable from the employees’ perspective. Likewise, by providing such benefits, employers aim to attract and retain a committed workforce and encourage positive citizenship behaviour [10]. It is also common for companies to offer the FASS which enables employees to submit expense reports, order office supplies, book travel arrangements, and undertake payroll functions. The self-service model can also be customised according to the user’s role. While the Employee Self-service (ESS) system is aimed at facilitating the roles of employees in general, the Manager Self-service (MSS) system is mainly developed to facilitate the roles of managers, for example, in accessing the data of employees for operational reviews, reviewing

© Monash Business Review Volume 3 Issue 1 – April 2007

3

performance appraisals and expenses, calculating compensations, and requesting approvals. Through ESS and MSS, staff can access the required information electronically and make onthe-spot decisions about employment, personal and professional development, and payment.

Distributed workplace Organisations today seem to be moving towards more distributed workplaces with telecommuting, teleworking and other novel office arrangements. Such flexible work arrangements can lead to a lack of effective communication within the organisation and may prevent employees from working collaboratively with their colleagues. One possible approach that is increasingly adopted by many organisations is for them to maintain communication with their employees and to build a sense of community through the development and implementation of b2e portal technology. B2e portals are perceived as a solution that enables employees to perform their work-related and personal-related tasks any time anywhere, as well as to remain connected with the organisation and their colleagues. As can be seen in Figure 1, a major driver of b2e portal adoption (at 22%) is the fact that employees are now distributed geographically. This shows how important it is for organisations to build a more committed workforce by maintaining efficient and effective intercommunication.

Employee Resistance to Change

Cross Organisational Boundaries 3%

3%

Increased Specialization 2% Others 5%

Increased 5% Competition

Aggressive Growth 37%

Product 7% Complexity

Leverage Investment 8%

Information Overload 8%

Distributed Personnel 22%

Figure 1: Driving Factors for B2e Portal Application* *Source: Plumtree Software (2002a) Corporate Portals, A Simple View of a Complex World. [Online].

Some challenges Although b2e portal implementation can bring benefits to organisations and their employees, there are some challenges associated with it. In the organisational context, this includes management’s commitment of financial resources for the cost of set-up, maintenance, system integration and content management. In particular, the high cost of b2e portal installation that can reach as high as US$ 5 million [11] is the main inhibitor preventing organisations with financial constraints from adopting these portals. In the technological context, the main obstacle of b2e portal acceptance is systems compatibility – at the lowest level, with client browsers. In the socio-work context, employees may refuse to use an automated system because they prefer human to human interaction as part of their work culture. Also, as in other computer- based transactions, portal users may question data confidentiality and be concerned about privacy issues.

© Monash Business Review Volume 3 Issue 1 – April 2007

4

User satisfaction as a success measure Because the cost of installing portals is generally high, organisations that intend to deploy b2e portals need to be aware of the measure of success of such an investment. A common approach used in business literature is to measure the Return on Investment (ROI) [12]. However, as with other success measures including cost benefit analysis, system usage estimation, and utility analysis [13], measuring ROI is sometimes not practical. In our research, we have developed an alternative way to measure the success of b2e portal implementation. Our approach is through the measurement of user satisfaction with b2e portals. The concept of user satisfaction has been widely adopted, if not the most common and accepted surrogate measure of system success [14-16]. Furthermore, we believe that the investments are intended to make their employees satisfied with the portal so that they are actively using the portal to perform their tasks. Our preliminary study includes a literature review of the studies in user satisfaction. We found that studies specifically examining user satisfaction with b2e portals are very scarce. The existing user satisfaction scales in the Information Systems (IS) field are not entirely appropriate for measuring user satisfaction with b2e portals. Scales measuring general user satisfaction developed by Bailey & Pearson [17] and Ives, et al [18] tend to overlook environments in which end users interact directly with the system. Another common user satisfaction scale developed by Doll & Torkzadeh [19] is also considered inappropriate as previous researchers such as Xiao & Dasgupta [20] and Abdinnour-Helm, Chaparro, & Farmer [21] found that significant modifications are necessary in order for the scale to suit the web-based environment. On a more general note, the existing user satisfaction scales do not suit the b2e portal environment as they do not take into account the b2e portal technology where functionalities are distinctive from the other systems (e.g., search and retrieval processes, work flow systems, online self-service applications, and collaboration tools). These observations led us to conclude that there was the need to develop a new scale for determining user satisfaction with b2e portals. The existence of this scale would be useful for practitioners and would further enrich the growing body of knowledge about user satisfaction with IS/IT applications. In developing a reliable and valid scale to measure user satisfaction with b2e portals, we followed the systematic and sequential procedures of scale development: conceptual development, item generation, content validation, exploratory and confirmatory study [22, 23]. A summary of the b2e Portal User Satisfaction (B2EPUS) scale development process is described in the following paragraph. For a more detailed and complete discussion, please refer to [8, 24]. A combination of information gleaned from the literature review on user satisfaction and b2e portal studies led us to determine nine dimensions of the B2EPUS construct: Ease of Use, Layout, Convenience of Access, Information Content, Communication, Timeliness, Efficiency, Confidentiality, and Security. These dimensions formed our initial conceptual model of B2EPUS and a pool of 47 items was generated to measure these dimensions. A panel of international experts was invited to give comments on these dimensions and their items. Based on their suggestions, the conceptual model was revised and only 28 items remained in the scale. The field study was conducted in the Australian higher education sector. Ten of the thirtyeight Australian universities contacted did indicate that they have built and implemented b2e portals. Emails inviting them to participate in the research study were then sent to these ten universities, to which five responded and agreed to participate in the study. Since it was not possible to have direct access to the employees’ contact details, a non-probability convenience sampling method was adopted. A university global email was sent to the university employees, inviting them to participate in the research study which was in the form of an online questionnaire. It included information on: the purpose of the research, how to participate in the study, and the website address for the online survey. Two follow-up emails were sent each fortnight during the survey period which lasted for one month. Three hundred and two responses were collected at this stage. These responses were then randomly split into two; 145 cases were used for our exploratory study and the remaining 157 cases were used for our confirmatory study.

© Monash Business Review Volume 3 Issue 1 – April 2007

5

In the exploratory study, the data was subjected to Exploratory Factor Analysis (EFA). The iterative procedures of factor analysis resulted in a final scale of 22 items belonging to 5 distinct dimensions associated with the B2EPUS construct. Although the results of the factor analysis revealed a different pattern from that of the revised conceptual model, nearly all dimensions proposed in the conceptual model appeared in the EFA. These five dimensions were then named Confidentiality, Ease of Use, Portal Design, Usefulness, and Convenience of Access. Furthermore, to test the factor structure more rigorously, the confirmatory study was performed. This resulted in a final 18 items in the B2EPUS scale. The five dimensions: Confidentiality, Ease of Use, Portal Design, Usefulness, and Convenience of Access identified in the exploratory study sufficiently captured the B2EPUS construct. Assessment of reliability and validity (i.e. convergent, discriminant, criterion-related, nomological validity) revealed that the B2EPUS scale is a reliable and valid measure of the B2EPUS construct. Figure 2 shows the evolution of the B2EPUS dimensions.

INITIAL CONCEPTUAL MODEL

REVISED CONCEPTUAL MODEL

EMPIRICAL MODEL

Security Security Confidentiality Confidentiality Information Content Information Content Timeliness Ease of Use Layout Ease of Use Ease of Use

Portal Design

Efficiency

Usefulness

Communication

Communication

Convenience of Access

Convenience of Access

Usefulness

Convenience of Access

Figure 2: Evolution of the B2EPUS Dimensions

To conclude, our empirical data shows that the B2EPUS construct can be measured by five dimensions, namely Convenience of Access, Usefulness, Ease of Use, Portal Design, and Confidentiality. The 18-item B2EPUS scale is an accessible and easily administered measure which can be used in an organisational setting for various purposes. The scale can be adapted or supplemented to fit the specific research or practical needs of a particular environment. Furthermore, organisations may use the scale as a feedback mechanism to uncover user perceptions of the strengths and weaknesses of the portal. In other words, the scale can assist organisations to measure the extent to which their b2e portals deliver the intended benefits, offer insights on how to improve the portals, and help create overall employee satisfaction and productivity.

Conclusions One possible approach that is increasingly adopted by many organisations in order to maintain communication with their employees and to build sense of community, is through the development and implementation of b2e portal technology. This paper is an attempt to further the understanding of the use of b2e portal by identifying the benefits it can bring to

© Monash Business Review Volume 3 Issue 1 – April 2007

6

organisations. The paper also presents an alternative way to measure the success of a b2e portal by proposing the application of the B2EPUS scale. The B2EPUS scale developed in this study is a significant contribution to the IS field. To the best of our knowledge, this was the first study that attempted to operationalize the B2EPUS construct into conceptually distinct indicators which can be observed and assessed. The findings provided a better understanding of the multidimensionality of the B2EPUS construct, which comprises five empirically distinguishable dimensions, namely: Ease of Use, Convenience of Access, Confidentiality, Portal Design, and Usefulness. More importantly, this study provides the groundwork for expanding research on user satisfaction studies within a web-based environment. The widely used internet wireless technologies promote the predominant use of web-based systems within organizations. Hence, studies of user satisfaction with portals are worth investigating so as to make better use of the available technology. The 18-item B2EPUS scale is an accessible and easily administered measure. An organization may administer this scale to investigate its own portal performance. The scale is able to provide an indication of employees’ overall satisfaction with the portal. It can be used to compare user satisfaction with specific components (that is, Convenience of Access, Usefulness, Ease of Use, Portal Design, and Confidentiality) of the b2e portal. Thus, it enables a portal project manager to recognise potential weaknesses of the portal. The B2EPUS scale can further be useful when applied longitudinally. An organization can use the scale to evaluate the performance of the portal at consecutive times. By administering the scale over successive periods, the portal project leader can clearly assess the extent to which employees are satisfied with the portal in both the early and later implementation stages. Scores from past periods can then be compared with scores from the current period in order to build a performance profile. Without such an assessment tool, it would be difficult for a portal project leader to identify areas of concern and take appropriate measures to address these. Being able to accurately assess employee satisfaction with the portal enables portal project team to better plan, coordinate and control their portal implementation project. Future research could utilize the B2EPUS scale with new data sets in other contexts. Researchers could perform cross-cultural validation using different large samples from different industries for greater generalisation of the scale. A test-retest reliability investigation should also be considered in order to examine the stability of the construct over time. Future studies may also investigate the perceptual differences among employees with various roles in the workplace in relation to their individual satisfaction with B2E portals. With different types of working arrangements operating in today’s workplace, each employee may perceive the portal differently. For instance, telecommuting home-based employees may view the portal as a collaboration tool. The ability of the portal to be accessed through the internet allows them to stay connected with colleagues and retain their sense of community. On the other hand, office-based employees may perceive the portal as their desktop destination since they can perform their allocated responsibilities, administrative tasks, and individual-related tasks through the portal. Consequently, different types of employees may utilise the portal in different ways and thus it would be interesting to examine whether their satisfaction levels differ. Such examination is crucial to ensure that the portal can satisfy the needs and wants of all employees.

References [1] Sugianto, L.F., Rahim, M., & Alahakoon, D. (2005). B2E Portal Adoption: A Conceptual Model. Proceedings of the International Conference on Information and Automation, Colombo, Sri Lanka. [2] Eckerson, W. (1999). Plumtree blossoms: New version fulfils enterprise portal requirements, Retrieved March 25, 2002 from http://www.e-global.es/017/017_eckerson_plumtree.pdf. [3] Brooks, M. (2001). Communications - General Productivity Gains with Employee Portals. Communication & Benefits Management, Spring 35-39. [4] Cedar Crestone (2004) Cedar 2004 Workforce Technologies Survey, Retrieved Sep 30, 2005, http://www.cedarcrestone.com/whitepapers/Cedar_2004_Workforce_Technologies_Survey.pdf. [5] Hamerman, P. (2004). Creating The Online Employee Experience, Best Practices, Forrester Research, Inc., 4 October.

© Monash Business Review Volume 3 Issue 1 – April 2007

7

nd

[6] Banks, J. (2004). Integrating people and content with business processes, 2 Annual Conference on Corporate Portal World Australia, May 4-6. [7] Sugianto, L. F. & Tojib, D.R. (2006). Information at Your Fingertip: B2E Portal as a Strategic Tool for Today’s Workforce, in A. Cox. (Ed.), Portals: people, processes, and technology (pp.125-137), London: Facet Publishing. [8] Tojib, D.R. & Sugianto, L. F. (2007). The Development and Empirical Validation of B2E Portal User Satisfaction (B2EPUS) Scale. Journal of End User Computing. In Press. [9] Gale, S.F (2003) Self-service Causes Call Volume to Drop 75%, Retrieved November 12, 2004, from http://www.workforce.com/archive/feature/23/39/04/233907. [10] Huang, J. H., Yang, C., Jin, B. H., & Chiu, H. (2004). Measuring satisfaction with business-to-employee systems. Computers in Human Behaviour, 20, 17-35. [11] Bannan, K.J. (2002) If You Build It (Right) They Will Come. Retrieved 2 May 2003,from http://www.econtentmag.com/Articles/ArticleReader.aspx?ArticleID=774&IssueId=121. [12] Davis, I. (2002). ROI from a New Wave of B2E Portal: Closing the Loop Between Plan and Proof for Your B2E Portal Initiative. White Paper, Massachusetts: ATG Group, Inc. [13] Thong, J.Y.L. & Yap, C.S. (1996). Information Systems Effectiveness: A User Satisfaction Approach. Information Processing and Management. 32(5), 601-610. [14] Melone, N.P. (1990). A theoretical assessment of the user satisfaction construct in information research. Management Science. 36(1), 76-91.

systems

[15] Palvia, P.C. (1996). A model and instrument for measuring small business user satisfaction with information technology. Information & Management, 31, 151-163. [16] Khalifa, M. & Liu, V. (2004). The state of research on information system satisfaction. Journal of Information Technology Theory and Application, 5(4), 37-49. [17] Bailey, J. E. & Pearson, S. W. (1983). Development of a tool for measurement and analysing computer user satisfaction. Management Science, 29(5), 530-545. [18] Ives, B., Olson, M. H., & Baroudi, J. J. (1983). The measurement of user information satisfaction. Communications of the ACM, 26 (10), 785-793. [19] Doll, W. J. & Torkzadeh, G. (1988). The measurement of end-user computing satisfaction. MIS Quarterly, 12(2), 259-275. [20] Xiao, L. & Dasgupta, S. (2002). Measurement of user satisfaction with web based IS: an empirical study. Proceedings of the Eight Americas Conference on Information Systems (pp.1149-1156). [21] Abdinnour-Helm, S.F., Chaparro, B.S., & Farmer, S.M. (2005). Using the end user computing satisfaction instrument to measure satisfaction with a website. Decision Sciences, 36(2), 341-363. [22] Yoo, B. & Donthu, N. (2001). Developing a scale to measure the perceived quality of an internet shopping site (SiteQual). Quarterly Journal of Electronic Commerce, 2(1), 31-45. [23] Muylle, S., Moenert, R., & Despontin, M.( 2004). The conceptualization and empirical validation satisfaction. Information & Management, 41, 543-560.

of website user

[24] Sugianto, L. F., & Tojib, D., (2007). Modelling User Satisfaction with Employee Portal. International Journal of Business and Information. In Press.

© Monash Business Review Volume 3 Issue 1 – April 2007

8