Network Authentication using Single Sign-On: The Challenge of Aligning Mental Models Rosa Heckle, Wayne G. Lutters, David Gurzick University of Maryland, Baltimore County 100 Hilltop Circle Baltimore, MD 21250, USA
heckler1, lutters,
[email protected] ABSTRACT
1. INTRODUCTION
Healthcare organizations are struggling to meet industry best practices for information security as well as complying with regulatory requirements. Single sign-on technology is emerging as a leading technology for password authentication management and promises to improve security while curbing system maintenance costs. While the technology seems to be a simple viable solution for authentication, when placed in context, many socio-technical complexities emerge. One of these complexities is that of the mismatch between the users’ mental models and the system model.
In hospital systems nationwide, calls for more efficient operations are issued in tandem with demands for stricter privacy and security controls as well as demands for individual accountability. In response to these calls, hospital administrators are challenged to reach an optimum level of security while negotiating the tradeoffs associated with the acceptance, costs, and usability of potential solutions. Authentication mechanisms for data access controls and audit capabilities are one priority of any comprehensive security program. Currently single sign-on (SSO) technologies have emerged as an effective means of addressing these authentication challenges.
This study was a 15-month ethnographic field study that followed the implementation of a single sign-on system in a hospital environment. It resulted in the finding that the misaligned mental models caused difficulties not only for the user but for the system administrators. The findings also indicate that not only was the user’s mental model of the technology inaccurate, but the presentation of the technology by the information technology group contributed to this misaligned understanding. The end result was dissatisfaction with the new technology for both end users and the system administrators. In order to address the critical issue of mental model misalignment in the implementation of SSO technology, practitioners must first gain an understanding of the preexisting mental models had by the target users regarding authentication and then use this information to guide implementation of the new technology.
Categories and Subject Descriptors H.5.3 Group and Organization Interfaces
General Terms Management, Design, Security, Human Factors
Keywords Management, Security, Human Factors, Mental Models, SSO Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHIMIT'08, November 14-15, 2008, San Diego, CA, U.S.A. Copyright 2008 ACM 978-1-60558-355-6/08/0011...$5.00
The decision to use an SSO system is underpinned by a number of realities regarding security practices in general and security practices as they apply to a hospital setting. While strong security measures are a must to ensure patient privacy, unencumbered access to data is also of critical importance to patient care. While in this environment, total security is an impractical goal, total insecurity is just not an option, either. Rather, the optimal system exists in a middle ground, where the tradeoffs in security and usability are equally weighed against the objectives of the system and the needs of its users. This can be difficult however, when considering the many design decisions and tradeoffs that accompany a security implementation. For instance, those mechanisms that are easy to use frequently relinquish some security strength, just as those mechanisms that offer stronger security often prove more cumbersome to use. Mechanisms that promise both usability and strength come with a greater financial cost. The authentication method of choice then, for most industries, has been the traditional id/password. This system has the advantage of being both simple and economical. Problems arise, however, when users have to manage a large number of username/PIN/password combinations as they navigate all of the applications needed within a system. A number of studies have documented the issues with password authentication citing human cognitive limitations at the core of the problem [3, 19]. Users have difficulty remembering cryptic passwords (made more prevalent by policy requirements that necessitate “strong” passwords), associating which password is for which system, and remembering passwords for those systems that are used infrequently [17]. This problem can be placed in context by considering the number and diversity of authentication systems in a hospital setting that clinicians interact with to perform their daily activities. For example, it is not inconceivable to assume that a typical nurse on an intensive care floor may interact with nearly
half a dozen distinct authentication systems as he or she performs a daily routine, including, for example, those for corporate email, drug management, documentation and patient tracking, order entry and lookup, and for certain equipment (e.g. blood glucose monitors). As the number of applications used varies with the role of the clinician, for some this number may be considerably higher. Users, such as hospital clinicians, have developed coping strategies for dealing with both the requirements for password creation and the limits of human cognition by simply writing down their passwords [3], using familiar names, reusing the same username/password combination across systems [9], or repeatedly requesting their passwords to be reset. While these coping mechanisms help the users remember their passwords, they ultimately subvert the security mechanism and create security vulnerabilities. In addition, the numerous password resets are costly to the organizations and users. These costs can take several forms, including inconvenience to users, loss of productivity due to employees not being able to access information, and helpdesk costs for passwords resets. Because of these vulnerabilities, the id/password authentication techniques are often referred to as “weak authentication.” Still, even with these concerns in mind, the id/password remains the most prevalent form of authentication.
2. LITERATURE REVIEW 2.1 Single sign-on technology Single sign-on technology serves to mitigate the shortcomings of id/password authentication. As the name implies, SSO systems are designed to allow a user to log in to a network once and then navigate through the range of his/her allowed applications seamlessly, without the need to re-enter authentication credentials for each application. Research studies on SSO suggest that while it greatly improves user experience by relieving them of the burden of remembering multiple user ids and passwords, it also noticeably reduces help desk calls, and improves security [4]. However, it also cautions that an SSO product is not a cure-all. Without very careful planning, implementation and verification, SSO products can introduce new security holes [4]. Technically SSO appears like a simple solution; however, its implementation reveals hidden complexities. For example, a study by Audun Josang, et al , analyzed some of the trust requirements resulting from various identity management models. The authors found that trust requirements for a particular authentication technology are directly correlated to the user’s perceived risk exposure and that this trust is necessary for user acceptance of the technology. With respect to SSO technologies, it suggests that trust relationships between federated parties are harder to establish particularly if one party has a significantly higher risk exposure than the other [11]. Logically all SSOs provide the same functionality; however, how this is technically carried out depends on the vendor product. Among the different vendors of the technology, there are differences as to how the actual sign-on process is accomplished. The technology vendor in this case was Novell and the product was Novell SecureLogin. To use SecureLogin the user must first complete an initial enrollment process. Enrollment is a one-time process where the user is first identified, then a new set of credentials for logging into the SSO system is created, and finally the existing logins of the user are related to this new account.
From this point on, the user signs in through the SSO gateway with a single set of credentials and each time he/she needs to access a secured application, the SSO system proxies the stored credentials to that particular application to verify. By making authentication for the user of multiple systems much more convenient, SSO has the potential to increase both the usability and security of the systems it covers. Though administrators of any industry would likely welcome such benefits, they are of critical importance to those in a hospital setting.
2.2 Healthcare environment’s unique challenges In a hospital setting, specific implementation challenges for authentication mechanisms exist. These challenges are expressed in the research of Bardram and Pellissier, who address the usability issues surrounding authentication mechanisms placed in a hospital environment. Bardram, in a field study of authentication in a hospital environment, investigated the usability of authentication mechanisms as they were applied to the login for access to electronic patient data. The study reported that from a usability point of view, the conventional login procedures caused considerable problems because conventional login procedures do not consider the ramifications of using the technology in context; for example in a hospital environment, medical work is nomadic, interrupted, and cooperative around sharing common material [5]. In concluding that many problems arise because authentication technologies designed and developed for the office environment are often times simply implemented in a healthcare environment (without due consideration for the particular dynamics of a hospital setting), he posits that there needs to be an authentication technology that can recognize the movement from individual work to collaborative work. Like Bardam, Pellissier [15] suggested that contextual factors introduce specific requirements that must be considered when deciding on an effective authentication mechanism. From his observations he concluded that certain additional technology might be required to compliment simple authentication mechanisms to ensure usability and compliance. Among several other options, single sign-on was suggested as a technology that would be useful where users need access to multiple applications to perform their work. He suggested that single sign-on may be a helpful technology to simplify the user’s authentication process.
2.3 Importance of mental models on usability Valid mental models can make a difference in the successful use of new systems. Most of the studies for mental models have been in the cognitive sciences [20]. However, several research studies have addressed mental models and their implications on information systems design. Mental models are defined as representations of the world and the environment; perceptions and understandings that individuals carry in their minds. They are formed by all individuals in an effort to simplify the complexities of their environment and to be able to understand, explain, and predict occurrences. Norman [13] theorized that when building an IS system, there are three models that must be considered. These include the conceptual model (or design model), the user’s mental model, and the system model or image. The design model is from the system builder’s perspective, and is a representation of the application to the user; the user’s mental models are determined from what the user understands of the system; and the system
model or image is the actual way that a system works from the system developer’s perspective. The point is that the systems designers need to match the user’s mental model with the conceptual model for users to be able to use the system as intended. The most problematic of the three is the user’s model. It is important for the users to have an accurate model of the system as it will help them complete the task as required [6]. In fact, users presented with a model of a new software system perform better than users who are given procedural instructions only [11]. Therefore, providing training and education can influence or adjust a user’s mental model to match the system model [8].
study was largely descriptive, aiming to discern both the process and factors impacting both usability and security. The primary data collection techniques used were ethnographically informed – participant observation, contextual interviews, and document review. The decision to triangulate among these three is based on a desire to capture the interplay of formal and informal systems [14]. The strength of observation is the ability to discover discrepancies between what participants say (and often believe) should happen and what actually does happen.
3. THE STUDY 3.1 Study site: General Hospital
Contextual interviews complemented the observation and allowed the first author to get into the user’s heads and probe into their thinking. Doing this while they were actually performing a task (in the moment), provided a deeper insight and understanding of why they did what they did, the way they did it. In fact, the majority of the interviews conducted for this study were openended, unscheduled, and opportunistic. Usually these were brief (5-10 minutes) and done right at the person’s work area as not to interrupt their routine work. The field notes taken during the observations and contextual interviews were transcribed immediately after the particular event while they were still fresh.
This single sign-on pilot took place in an organization, which will be referred to as General Hospital from this point forward. General is a suburban hospital with 292 acute-care beds, handling nearly 22,000 inpatient admissions annually. It employs approximately 2,600 full time individuals, with 1,200 of them being physicians. The hospital handles 60,000 emergency room visits and performs 40,000 surgeries annually.
Participants for this research were a mix of both clinical and administrative staff. Clinical staff included physicians, nurses, technicians, radiologists, or those individuals who were directly involved in patient care. Administrative staff included all other employees; such as those in the billing offices, administrative assistants in the physicians’ offices, volunteer offices, etc. A summary of the collected data is given in Table 1.
Technology use for GH is rated as “moderate’ in accordance with the metrics used by the American Hospital Association. However, GH is progressively moving to a fully functional electronic health record. Both the clinical and administrative staffs use computers as a normal part of their work, and they authenticate to at least 4 to 6 applications on a daily basis.
Table 1. Data Collection
Mental models are foundational to human-computer interaction research [7,16,18], while this paper does not seek to extend the theory, we found the original notion to be a valuable sensitizing concept in the analysis of the SSO roll-out
Every medical organization is a unique configuration of fairly routine roles, routines, and requirements. Therefore, the lessons learned about the SSO implementation at General should have a high degree of transferability to other organizations with similar configurations. Figure 1 depicts a nurses’ station with its shared workstations.
Figure 1 Nurses’ Station
3.2 Study design The study was a 15-month ethnographic field study with the specific intent of gaining a deeper understanding of the people, the routine work practices, and the environmental factors that impacted the implementation of a single sign-on mechanism. The
Data Collection Methods Observation Observation
Data
Quantity
Four Units 48 hours Radiology Department, 14 hours Lab Observation Patient Billing 8 hours Observation Vendor Selection 7 meetings Contextual Interviews Informal 50 Interviews Hands-on Tutorials, Training 4 Observation modules Observation MIS/Design Meetings 180 hours Participant Unit 48 implementation 30 hours Observation pilot Participant Physicians Offices/Clinics 8 hours Observation Participant Patient Billing 8 hours Observation Contextual Interviews Informal 60 Interviews Data Review Generated reports from the Help Desk call report logs 57 Document Review Documents and Policies from the Organization’s Intranet and Internet The following sections describe the reactions of various stakeholders of the SSO project, focusing on the challenges they encountered during its implementation. The content analysis of the data collected during this process pointed out a difficulty with the enrollment process for the users. It was determined that there was a mismatch in the users’ understanding of the operation of the
SSO system and way that the SSO system actually worked. Explored through the mental model theory as described by Norman [13], indicates that a main culprit to this misunderstanding was an inaccurate mental model held by the users of the system. Following a discussion of these findings, a set of recommendations for minimizing related challenges in other SSO implementations is offered.
4. WHAT TOOK PLACE 4.1 Taking a simplistic view At first MIS (the name given to the management of information systems division of General Hospital) held a very simplistic view of the project, which was fueled by the technological determinist discourse surrounding SSO. Much of the vendor and consultant rhetoric surrounding SSO through the brochures and media documents extolled the virtues of SSO and simplified the implementation effort. MIS viewed the project simply as an exchange in the authentication mechanism used and largely a backend concern. In their view, as far as the users would be concerned, it was a simple change in the authentication screen. Before the users logged into various applications; now they log into Novell and SSO takes care of the authentication to each application. It would bring about some changes, yes, but nothing much different from what they were already doing. Authentication was a normal part of the work day and a familiar element of their organizational culture, as the following comments illustrate: “They are not doing anything different than before [Compliance Officer].” “They’re not doing anything different. They still have to log in just like they did before. It’s just that they are now logging onto the network rather than Meditech [Project Lead].” “So I really have to have them understand that they’re not logging into anything more than they did before. One time you’re logging in. That’s it [Project Manager].” Some of the initial design choices were based on technological considerations alone. SSO was viewed as a technology that was part of the network infrastructure and would ultimately be maintained by someone in the networking group, whose main concern would be to create scripts when new applications were added to SSO. One of the important criteria used for choosing the Novell product was the fact that the organization already ran on a Novell platform. The ease of integrating this product into their existing architecture and facilitating its maintenance was their main concern. From the user perspective, the implementation was seen as a quick transfer from one authentication mechanisms to another, with most changes being transparent to the user. The implementation schedule reflected this ideal as the implementation cycle from start to completion was to take place over six months for full implementation throughout the organization. However, this schedule began to change as the realities of the implementation complexities began to unfold.
4.2 Communication and training In the beginning there was a comprehensive training and communication strategy. Brochures were created to communicate what SSO was and what the users could expect. However, as time passed hints of technical and political difficulties emerged and soon materialized as delays in the project. In an effort to stay on track scheduled training was diluted to voluntary drop-in sessions, brochure distribution became burdensome and scattered. The need for a ‘full training session’ was soon dispelled as MIS felt that there was really not much to train. The users were already logging in to the system; “it’s the same procedure just a different screen”. MIS did recognize the fact that the users would have to set up their passwords for SSO initially. (This process is formally termed ‘enrollment’, though this term was not used.) However, the project lead felt that the instructions for enrollment were clear. The users could simply follow the instructions given to them on the screens when they tried to authenticate to an application, and that is all that was necessary. However, the enrollment process was not as simple to the everyday user as MIS felt it should be, and issues to this began to emerge during the drop-in training sessions. Without training and without MIS guidance through the enrollment process, users were left to their own ideas of how the technology was supposed to work, and issues emerged. From a review of the Help Desk call log and the questions asked by users, it was evident that the users did not have a clear understanding of SSO’s functionality. Their perception of how it worked was not matching the reality – a matter that became clearly evident during the training sessions.
4.3 Training reveals mismatch in mental models Users had the perception that single sign-on meant they only needed one password to get into all their applications. This is exactly what SSO allowed, but this is not exactly how it worked. With Novell’s SecureLogin each application still maintained an individual password. This password still expired when the application prescribed, and the user still had to create a new password when this happened. When using SecureLogin, however, the user did not have to remember or manage these passwords. Once they completed the one-time enrollment through SSO, whenever the application password was needed, SSO would provide it, even when the password for an application needed to be renewed. All the user would have to do at password renewal is provide a new password. However, in practice the process was not as clear. All the users focused on was the ‘one password’ verbiage, as the following comments suggest: “You know, people are thinking that they are only going to have one password [MIS Trainer]”. “I thought this thing said I would only have one password [User1].” “It says (pointing to the brochure), one password. What do you mean; we still have all our passwords [User2]?” Some users specifically pointed to the verbiage in the brochure as to where they got their mental perception of how SSO was going to work. “It says right here: One user name and password.” Figure 2 depicts the SSO brochure that was sent to all users before the implementation.
Step 1 – enter your Novell login ID to log onto the network
Step 2 – create a secret user identity to be used for automatic password resets Figure 2 Excerpt from the SSO Communication Brochure Users were cultivating this perception not only from the verbiage on the brochure, but also from their experiences with the current environment, in which users log into each application with a different unique password. In their thinking, they would now only use the one password for all their applications, signing into each application when necessary with one password. This perception of having only one password caused confusion and frustration during training. This confusion during training stemmed from the fact that the enrollment process involved multiple steps. In addition to the actual SSO enrollment, the user had to create a secret identity for the password self-service capability, which included answering six challenge questions. While the challenge questions and secret identity were to have been completed several weeks before the SSO implementation, many people had not gone through that process. In addition, many of the clinical employees were not familiar with the network login because they never used it. For those individuals the trainers needed to call the Help Desk to get the user’s network id before they could even begin the enrollment process. During the training sessions each individual was guided through the multi-step enrollment process for all the applications. Figure 3 represents the steps that were to be followed. However, the fact that they were going through an enrollment process was not explained to the users, and it was the enrollment screens that caused a problem. The directions on the screen, (shown in Figure 3, Step 5), says “Please enter your credentials to log onto Meditech.” The users were confused with this direction, and they wondered if this was asking for the Novell network id that they had just created and used, or what password exactly it was asking for. At this point the user would ask ‘which one, the new one I just created [User3]?” “I’m not sure which ID and password they want. The ones I just used [User4]?” Some users continued on and entered a password, and when the application would not authenticate them, they would complain that the application wasn’t working. “It doesn’t like the new password I created [User 5],” they would say. The MIS analyst would have to explain to them that they needed to enter the password they had been using for the application prior to coming to training. The user then would voice their perception that it was going to be all one password; “Why, I thought it was going to be all the same password [User 5]”.
Employee ID: xxxxxxxxxx (You must remember this exactly as you enter it) Step 3 – answer the six challenge questions Challenge Questions: 1. What is the name of the first company you worked for? 2. What is your best friend's first name? 3. What is the last name of your favorite teacher? 4. What is your favorite music type? 5. What is your place of birth (city only)? 6. What street did you grow up on? Step 4 – choose an application to enroll in that application Step 5 - the following screen would appear, with direction.
“Please enter your credentials to logon to Meditech” Steps 6 - Repeat Steps 4 and 5 for each application accessed Figure 3 Steps for the enrollment process After the first few training sessions the mismatch between the users’ mental models and reality became clear. In an effort to get users’ mental model to match the system model, the remainder of the training sessions continued with a preemptive explanation of how SSO worked, and then proceeded to the actual enrollment process. This was done in an effort to sensitize users to the correct model, making it easier for them. This was consistent with the Mental Model theory, which states that users’ mental models can be influenced and conditioned during the implementation stage through training, with the assumption that you can help users develop more appropriate mental models of system functionality. In this case, once a user had formed a correct mental model of the system, they were able to use the system more effectively.
4.4 Problems with enrollment during roll-out For those users that did not go to training, they had similar issues with their understanding of SSO during the actual roll-out. The following vignettes describe the observations made during rollout.
4.4.1 First User Roll-out For this first group of users, the SSO application was automatically pushed out early in the morning. This meant that when an individual turned on their computer that morning, the SSO application would install itself and the user would have to enroll in any of the SSO enabled applications before beginning work. The users were aware of the SSO application becoming active that morning. Many of them had the brochures visible on their desks, and an email was also sent out the day before. However, they still seemed confused about SSO. It was quite evident after helping a few users get enrolled, that their vision of SSO did not match the functionality of the application, and they did not understand the initial enrollment process. Their confusion became evident during enrollment when the users became confused as to which password they were supposed to enter; their Novell password or the application’s original password. “I thought it was just one password [user6]. “ “Oh, so my passwords are still there [user 7]?” “I thought once I changed it, like I just did, I would use that one. No [user8]?” Once MIS took the time to explain how SSO worked, and the initial enrollment process that they were going through, it seemed to be much easier with enrollment in other subsequent applications, and the users concerns were calmed. However, for those that did not have MIS to clarify the process, they experienced difficulties. This became evident when reviewing the Help Desk call logs for that particular day. The Help desk experienced an overload of calls with users trying to get onto their systems at the start of the early work shift.. Entries from the Help Desk call log show the same problems. For example: “I thought that my user id was the same as my Novell and I thought my password was Robert. But I am unable to get in [Patient Accounts 1].” “SSO question. She gets in under her OLD Meditech password instead of her current [Patient Accounts 2].” “She is having problems with SSO. Accessed PC and deleted her User ID and PW in MT, and deleted her PW in Kronos, Then had her open MT and use her previous MT and Kronos PWs [Patient Accounts 3].”
had to reset and walk through the process with the group [Help Desk Response 1].”
4.4.2 Second User Roll-out The second group of administrative offices was done just a week later and worked the same way and had the same issues; those who started early before MIS’s arrival had some difficulties and contacted the Help Desk for guidance. Once MIS arrived and guided the users through enrollment, things went much smoother. Again, once the way in which SSO worked was clarified, the users had an easier time with enrollment. At the end of this second implementation session, the MIS analyst had run out of patience and expressed his opinion that if the users would only read the question presented during enrollment and simply answered it, they would not have any problems. He was referring to the enrollment process and the directions to enter their credentials. He strongly felt that the users simply did not read the directions given to them. He felt that the process was simple, and only required the user to simply read the screen and follow the directions. However, while the Project Lead was frustrated with the users, the Help Desk was frustrated with the process. They, again, had experienced a high-call volume concerning SSO early that morning prior to the MIS analyst’s arrival on site.
4.4.3 Third User Roll-out. The ‘go live’ for the third group of administrative offices was split and set for two consecutive days. For this group a different implementation strategy was used. The Project Lead, being unaware of the call volume with regard to SSO during the previous go-lives, felt that there was no need to have a MIS representative on site during enrollment. He felt that the process was self-explanatory. “all they have to do is read the question and answer it”. He felt that they would be fine; and if they did have a problem, then they would call the Help Desk. Then, if the Help Desk was unable to resolve it, they would contact him. With that said, he sat at his desk and worked. This implementation strategy seemed very viable to him as the enrollment process was very straightforward in his mind. However, while he was comfortable with this, the Help Desk was not. Throughout the morning the first author would stop by his desk and ask him how things were going with the implementation. “Seems to be going great” he replied. “I haven’t heard a complaint yet.” However, when the first author decided to take a quick step around the corner to the Help Desk area to see if they had gotten any calls, they were busy! They were fielding many calls from users who were trying to enroll with SSO. The first author did this several times that day; asking the project lead how things were going and then walking over to the Help Desk and seeing how things were going. The scenario played out the same way each time; the help desk analysts were very busy with SSO calls, yet the project lead had no inkling of it. After the second day of the implementation, the Help Desk was fed up and wrote a formal e-mail of complaint to the project manager.
4.4.4 Change in Roll-out Strategy “Could you please reset my Lawson password? I'm having trouble getting in [Patient Accounts]. All users in Gail's area are now able to get into Lawson self service. I
This implementation strategy did not play out well with management as both the users and the Help Desk were frustrated, and the next scheduled implementation was laid out differently.
The pattern was recognized from these first ‘go lives’ that the users were having difficulty getting enrolled with SSO. The project manager realized that those users who had attended the drop in sessions (who actually had already enrolled and now understood how SSO worked), had an easier transition than those who did not. The project manager also realized that those who went to the training sessions were able to provide help and guidance to others. So for the next set of implementations, MIS was to personally train one or two super-users in specific offices. Once those users were trained and had used SSO for a while, MIS would go live with SSO for the entire office, relying on those super-users to help the others. In addition, an MIS analyst was required to be on-site during the implementation to provide handon assistance.
4.5 Users did not have a complete mental model of the new security environment The use of single sign-on was not just a change in authentication screens, but a paradigm shift. Single sign-on controls authentication on the network level; once a user signed onto the network, all the SSO-enabled applications would be readily available without further authentication. This was a paradigm shift and the users were not cognizant of that. Before SSO the users in this case were used to authenticating on the application level. In the collaborative areas in the hospital, the computers were never shut down; they were on 24/7. When necessary, a generic user id was used to log onto the network, and users would then authenticate to each application they needed to use. The shift from application level authentication to network level was important for the users to understand. Before SSO users would exit the application and that meant they were logged off. However, with SSO, exiting the application did not log the user off; it just exited the application, leaving the user still logged onto the network. The behavior of logging off was critical to the success of SSO. However, the importance of a consistent log-off behavior was not presented to the users during implementation nor during training. To illustrate: during the drop-in training session, it was observed that at least 60 percent of the individuals got up to leave without logging off of the network. They had logged onto the network with their individual user IDs, but when they were done, they simply exited the application and then got up to walk away. At one point one of the individuals offered her workstation to the next person waiting: “You can use this one, it’s already logged on.” The users were then reminded to log off, but still they were never told why it was important to do so, particularly under SSO. Organizational policy reiterates the importance of logging off; however, practice did not follow policy. Therefore, since they were not told of its importance, they did not do it. For instance, on the afternoon of an implementation, the project lead and the first author took a walk through one of the offices that had been converted to SSO. As we walked through, one of the individuals was having a problem with her computer. The project lead asked her to get up so that he could work on the computer. The individual got up and simply moved to the desk next to her and continued working; the workstation was logged in and no one was using it. The interesting part was that the movement to the other computer was seamless; neither she nor the project leader thought twice about it. However, this simple move caused a breach of
security. Without adjusting the users’ mental models to the total security environment under SSO, these incidents would recur.
5. DISCUSSION 5.1 Mismatch in users’ mental models and system models The indications from these first few implementations show a mismatch between the user’s understanding and mental models with the actual system model of SSO. Mental Models Theory states that mental models are like mental maps that help individuals find their way and give meaning to things. The theory is clear in its assertion that valid mental models can make a difference in the successful use of new systems. However, “if the mental model is wrong, the user will be wrong too” [12], which is what was happening in this case. Users form mental models through a mixture of prior knowledge, experience, and incoming new information through some form of media or training, and use these models to carry out tasks which involve unconscious and conscious processes, where images and analogies are activated. The users in this case formed their mental models of the SSO application from their past experiences with having to manage several passwords on a daily basis, from word-of-mouth about the SSO project, and from the SSO brochures and presentations created specifically for the introduction of SSO to the user community. During the training sessions and in the roll-outs, it was discovered that many users had a misconception of how single sign-on worked. The mental model that many had was that they would have one password that they could use with all their different applications.
5.2 Training vs. enrollment terminology The theory of mental models also warns that the terminology used when talking about a system helps users formulate a mental model, and therefore is very important. From the project initiation, MIS used the term ‘training’ when actually referring to SSO enrollment. The term “enrollment” was intermixed with the term ‘training’. Enrollment is necessary in authentication to capture the user’s credentials, and is a one-time event. By intermixing these terms, they were sending the wrong message to users. The term “training” has a certain connotation. Colloquially the word “train” is defined as, “to give the discipline and instruction, drill, practice, etc., designed to impart proficiency or efficiency [1].” “User training” seemed to signify that the users needed to remember what they were doing because they would need to do it again later. Referring back to Figure 3, the training/enrollment process was 5 steps, and included answering six challenge questions. As users were going through the enrollment process, they thought they had to remember the entire process – including for some the setting of the challenge questions. “I’m never going to remember all of this [Trainee1].” “This is more than I have to remember now [Trainee2].” “This is too much [Trainee3].” They seemed to get frustrated with the entire process, and this incited a very negative attitude toward SSO. “I don’t see how this is going to help me [Trainee4].”
On the other hand, the term “enroll” infers a one-time process. “Enroll” is defined as “to write the name of (a person) in a roll or register; place upon a list; register [2].” When it was explained to the user that this was a process they only had to go through once, they seemed to calm down and proceed without anxiety, creating a much better attitude and perception of SSO. With the knowledge that they only had to go through the process one time, the comments with regard to SSO changed, adopting a more positive tone (“I’m going to like this [User 5]”, “This is great [User 6]”).
5.3 Training to adjust user’s mental models Often times when project timelines and budgets get tight the first thing to go is the education and training. However, in systems that users may be sensitive to, training is critical. It not only allows the user to understand the system, but also provides a level of comfort begetting of a more favorable perception of the system. This is very important when implementing a system that from the onset is recognized as a very political or a mandated system. This is particularly important for authentication systems. Authentication systems are different from other IS systems, and their implementation brings different connotations to its users. As previously mentioned, security systems do not assist the user in the completion of a particular task they set out to do. Rather, it is a hurdle which they must clear before they can even get started. Therefore, it is imperative that the system be easy to use, and work properly. In addition, authentication systems have privacy implications for the user, if the technology is not working well, the users become concerned for their vulnerability to the system. Aligning the mental models of the users, so they understand how the system is to work, can help quell their concerns. Having matching mental models will allow the system implementation to go smoother, the smoother the implementation, the better acceptance.
5.4 Aligning mental models beyond enrollment Correcting the user’s mental models of SSO to match the actual functionality of the application was effective in getting users to successfully enroll in SSO and to better perceive what SSO was actually doing for them. In addition, by having matching models, the users can more easily maneuver the process when new applications are added into the single sign-on, or when they are asked to perform routine maintenance on their passwords. Thus limiting calls to the Help Desk each time a password needs to be changed, or a new application is added and the user needs to enroll. In addition, while the revised mental models allowed users to understand how SSO technology managed their passwords, it did not give them a full understanding of the SSO environment and the importance of logging off, nor did it educate them as to the reasons for implementing SSO. When moving to SSO authentication there is a paradigm shift that must be addressed. The users are used to sharing computer, while properly logging into an application with their individual authentication. However, with SSO, you cannot have two users using the same computer without one logging off first. Without emphasizing the required behavior and getting the users to understand the new security environment, vulnerability is created. While the importance of logging off is stressed in the policies, when using the SSO technology it is even more critical. Having a mental model of the
complete security process surrounding SSO usage may be more effective in getting using SSO appropriately after this initial enrollment, and for SSO acceptance.
5.5 Not all users are alike The power of mental models is that they let users figure out what might happen when using a system. Research indicates that an inaccurate mapping between the user’s model and the actual functioning of the system can increase task complexity and result in performance errors [7]. Users not only interpret the visible parts of the system but also make assumptions as to what is happening technically on the back end. In general terms, if the model is wrong, the way the system is used is likely to be incorrect [12]. Applied to human computer interaction, this implies that designers need to determine what knowledge the user has and what cues can be added to a system that would allow the user to build on their existing mental model towards the direction of the actual system implementation. A mental model is a personal internal representation that is formed and used when people interact with a system. They are built in the moment at the time of the interaction by associating previous knowledge with the incoming information from the context and tasks’ demands [19]. Therefore, since mental models are an internal representation by individuals, they will differ. This difference can occur not only from one individual to another, but also across the lifespan of a single individual’s use of a system. These differences are caused by several factors. They emerge from expectations for how the system should operate. Some of which are established ahead of time, built from the experiences they have had with similar systems and from general perceptions of how a type of system works. Others are established upon arriving, created from their initial encounters with the system and refined over time through the observation of the system in use [7, 12]. In a hospital environment, the diversity of its staff is not only seen in the individual themselves, but also in their roles. Some roles, such as billing clerks, clinic administrators, and other office staff were very technical. With few exceptions, individuals in these roles were assigned their own computer and were looking forward to the implementation of SSO because it would minimize the number of passwords they would have to manage. In contrast, the clinical staff that more often shared available computers was resistant to the implementation. The expectations and experiences of these two groups with authentication greatly affected the interaction with the SSO system. While the administrative staff could tolerate a delay in accessing a patient record, the clinical staff needed unencumbered and timely access to patient data. However, even amongst the clinical staff differences could be seen. For example, there were two types of physicians; staff physicians and non-staff (physicians with privileges). Each group had a particular type of password and applications that they would use. The staff physicians understood the importance of the system and were very familiar with the hospital authentication mechanisms; however, the non-staff physicians had a difficult time with enrollment because they usually logged in from home and were not as familiar with the systems. Each of these groups came with very different mental models of how SSO should work. In the case of the administrative office worker, they were used to logging in and having their own computer. This was in contrast to the clinical users in the collaborative areas, who were used to sharing network logins and
logging into many applications simultaneously. HCI practitioners must consider this diversity when designing effective and usable SSO systems. To the user the interface is the system, and they must be able to know “what to do” and “how to do it” intuitively. The goal for the interface designer is to design an interface that users find predictable and intuitive as each user is approaching the interface from a different perspective [18]. They must also allow users to build on prior knowledge gained from prior experience. For example, in the environment of General Hospital, when a user exits an application a dialog box can be displayed reminding the user that they are still logged on, and they must log off the network to be completely signed off. As another example, the clinical users need to be reminded to use their individual personal login and that they can no longer use another’s network login. Such notifications can be incorporated into the interface of the system, further aligning the users’ mental model to that of the system’s.
5.6 Aligning the technology with the work practices As previously stated in Section 2.1, the choice to implement the Novell SecureLogin product was strongly influenced by the fact that it fit well with the current technology infrastructure, and met the financial constraints. Consideration was given to the user front end, but the consensus was that the product delivered what they were looking for, and all SSO products worked the same on the front end, anyway. While discussions about work practices and the fit of SSO within these work practices were discussed, they were not explored in detail. The findings in this case show, while the technology on the backend may have fit well, it caused considerable problems within the work practices. These problems caused considerable implementation delay, increased the financial outlay, and caused increased hostility toward the project. To alleviate these consequences, it is important for designers to consider the work practices and match the system to the work practices.
6. LIMITATIONS AND FUTURE WORK
In addition, while all SSO technologies have the same basic premise of logging in once and having access to multiple applications, how the technology accomplishes that goal is different with each vendor product. Therefore, users may not have the same issues with other SSO products. Future work, could first assess user’s mental models of SSO in general, and then match those mental models with the manner in which the available products work to determine if one product has a better match. Then, it could be taken further to actually see what problems arise during enrollment with some of these other products. This research explores how this one implementation of SSO played out in this one hospital, but it raises questions about its effectiveness in other hospitals or other collaborative environments. A cross-case analysis could clarify and further pinpoint where the issues lie. To further explore what may or may not be generalizable in the findings, an additional study can be done of a similar implementation at another hospital or other collaborative environment.
7. CONCLUSION Single sign-on is a simple means of managing passwords and authenticating to varied applications for users. However, its implementation has many complexities that require a shift in thinking from an individual application level to the network level. In this case the mental model that the users at GH had of the single sign-on system was incorrect. This inaccuracy made it difficult for the user to interact with the system, and in turn stressed the Help Desk with calls for assistance. With a better alignment of these models, the implementation went smoother, frustration was minimized, and the technology was perceived in a more favorable light. However, to fully realize the intended usage of SSO, the user’s mental models must also be adjusted to reflect the SSO environment, not just the SSO technology.
8. REFERENCES [1] The New Oxford American Dictionary. McKean, E. ed., Oxford University Press, 2005.
Objectivity on the part of the researcher is problematic as their background and experience does influence the findings and their presentation. In this case the authors come with an information systems analysis and development background from private industry and not a health services organization. As such, the first authors particularly identified with the MIS group. While the author tried to be as comprehensive as possible when reporting the findings, their biases may lean more toward the MIS group than with the clinical user group.
[2] Webster's New Millennium Dictionary of English, Preview Edition. Kipfer, B.A. ed., Lexico Publishing Group, Long Beach, 2007.
The second issue that arises with qualitative research work is with the inability to generalize the findings; however, the purpose of this study is not for generalizability, but on providing a rich description and deeper understanding of the studied phenomena. While this study may not portray an account of how it is in all hospitals, it does provide a detailed illustration of what has happened in one hospital. From this, others can judge whether there is knowledge and examples that can be applied to another situation. Additionally, lessons learned from this research can be applied to future research into information security-related fieldwork, and to the future designs and evaluations of authentication mechanisms.
[5] Bardram, E. The trouble with login: on usability and computer security in ubiquitous computing. Personal Ubiquitous Computing., 9 (6). 357-367.
[3] Adams, A. and Sasse, M.A. Users are not the enemy. Communications of the ACM, 42 (12). 40-46. [4] Anchan, D. and Pegah, M. Regaining single sign-on taming the beast. Proceedings of the 31st annual ACM SIGUCCS conference on User services. 166-171.
[6] Borgman, C. The users mental model of an information retrieval system: an experiment on a prototype online catalog. International Journal of Human-Computer Studies, 51 (2). 435-452. [7] Carroll, J and Olson, J. “Mental Models in Human-Computer Interaction” in Handbook of Human-Computer Interaction, M Helander (ed), Elsevier, 1988
[8] Fein, R.M., Olson, G.M. and Olson, J.S. A mental model can help with learning to operate a complex device. Conference on Human Factors in Computing Systems. 157-158. [9] Ives, B., Walsh, K.R. and Schneider, H. The domino effect of password reuse. Communications of the ACM, 47 (4). 75-78. [10] Jøsang, A., J. Fabre, et al.. Trust Requirements in Identity Management. Australasian Information Security Workshop, Newcastle, Australia, 2005. [11] McDaniel, S. What’s Your Idea of a Mental Model. Boxes and Arrows. [12] Norman, D. The Design of Everyday Things. Doubleday/Currency, New York, 1988. [13] Norman, D.A. and Collyer, B. The design of everyday things. Basic Books New York, 2002. [14] Patton, M.Q. Qualitative evaluation and research methods. [15] Pellissier, S.V. Effective Authentication in a Medical Environment - Business Case Analysis. ATI IPT Technical
Report 01-01. DAMD17-99-C-9001, Frederick, MD, USA, 2001. [16] Preece, Rogers, & Sharp. Interaction design: Beyond humancomputer interaction. John Wiley & Sons, Inc, 2002 [17] Sasse, M.A. Eliciting and Describing Users’ Models of Computer Systems Computer Science, University of Birmingham, Birmingham, UK, 1997. [18] Shneiderman, B. Designing the user interface: strategies for effective human-computer interaction. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA, 1992. [19] Van Der Veer, Gerrit and Melguize, Maria,(2003) “Mental Models’, In J.A. Jacko and A. Sears (EDS.), The Human Computer Interaction Handbook, p 52-80, Mahwah, NJ: Lawrence Associates [20] Yan, J., Blackwell, A., Anderson, R. and Grant, A. Password memorability and security: empirical results. Security & Privacy Magazine, IEEE, 2 (5). 25-31.
