Relative Importance of Usability and Functionality ...

Relative Importance of Usability and Functionality Factors for Computer-assisted Navigation System for Cryoablation of Kidney Tumors Fethi Calisir Management Faculty Industrial Engineering Department Istanbul Technical University Macka, Istanbul 34367, Turkey Ecem Basak Management Faculty Industrial Engineering Department Istanbul Technical University Macka, Istanbul 34367, Turkey Duygun Erol Barkana Faculty of Engineering and Architecture Department of Electrical and Electronics Engineering Yeditepe University Atasehir, Istanbul 34755, Turkey Abstract Computer-assisted navigation (CAN) is a surgical system used for performing surgeries and helps a surgeon to navigate his way through the 3D presentation of the patient that are captured as computed tomography images. Any surgical intervention using this tool is safe and less invasive. Usability and functionality, which are interrelated, are the important factors for the acceptability of a system, and both should be considered notable in the process of design. The present study aims to examine the surgeons' evaluations of the relative importance of the usability and the functionality factors in the computer-assisted navigated system designed for cryoablation of kidney tumors. A survey methodology was used in this study to gather data. 12 questionnaires have been collected from surgeons. The results of the analysis show that response time, efficiency and ease of use are found to be the most important factors among usability factors and archivability and accessibility, auto-run and data security are found to be the most important factors among functionality factors. The findings of this study make a contribution to the early design stages of the CAN system for cryoablation of kidney tumors and provide a better understanding of the expectations and perceptions of surgeons.

Keywords

Usability, functionality, computer-assisted navigation system

1. Introduction

Surgeons are more likely to interpret volumetric, functional, and navigation-assisting data through the development of interfaces and visual tools. In order to avoid risks and handle real-time situations, navigation systems have been commonly used in clinical practice (Wegner, 1998). Hence, computer-assisted navigation (CAN) is a useful surgical system to perform an operation and helps a surgeon to navigate his way through the 3D presentations of the patient

that are captured by computed tomography (CT) images (Adams et al., 1990). It also helps surgeons for surgical interventions safely with less invasive procedures (Mezger, Jendrewski, & Bartels, 2013) and is helpful and comforting way finding aid because it can indicate the to-be-adopted direction from the current position and perspective of the user (Münzer, Zimmer, Schwalm, Baus, & Aslan, 2006). However, the complexity of the system, problems related with the repeatability of surgeon’s actions, accuracy of the information, and visual representation of images prevent surgeons to use this surgical system to the maximum advantage (Martelli, Zaffagnini, Bignozzi, Lopomo, & Marcacci, 2007). To avoid this, a surgical navigation device is designed regarding its technical features. Besides the technical factors of a system, such as technical characteristics or compatibilities of the technical system components, human factors are also taken into consideration in designing a surgical device to increase the accuracy of surgical outcome and decrease invasiveness. A user-centered surgical device is designed by applying the ergonomic system design approach to meet functionality and usability requirements (Brandt et al., 1999). Therefore, it is suggested that the functionality and usability are important for the acceptability of any given system (Lu & Yeung, 1998), because users prefer usable and functional systems more frequently. Usability is defined as the extent to which a system facilitates users to utilize its functions easily and appropriately, whereas functionality is defined as the extent to which the system operates in the way it is structured and is expected to perform as users’ desire it to do (Bertot, Snead, Jaeger, & McClure, 2006; Nielsen, 2003). Systems focused on only functionality are not able to meet usability needs (Kline & Seffah, 2005). It is also possible that a functional system is not usable or vice versa (Lu & Yeung, 1998; Nielsen, 2003). Therefore, both usability and functionality are interrelated and should be considered notable in the process of designing (Nielsen, 2003). It is crucial to give equal importance to both functionality and usability in the design of a system to meet users’ expectations. Both aspects—the interaction between the user and the system as well as the operations performed by the system—should be taken into account with a user-centered design (Kline & Seffah, 2005). Since meeting users’ needs are an important consideration, understanding the user’s expectation of balance between functionality and usability is critical for a user-centered design. Many studies have examined the usability and functionality factors of different systems, such as digital library (Bertot et al., 2006), website (Calisir, Bayraktaroglu, Gumussoy, Topcu, & Mutlu, 2010; Topaloglu, Gumussoy, Bayraktaroglu, & Calisir, 2013), online catalogue (Craven, Johnson, & Butters, 2010), or vehicle navigation system (Burnett, 2000). However, the present study aims to examine the surgeons’ evaluations of the relative importance of the usability and the functionality factors in the computer-assisted navigated system designed for cryoablation of kidney tumors.

2. Usability and Functionality Factors Used in this Study

A literature review was performed to determine the usability and functionality factors that are important for the CAN system. The literature reviewed for usability and functionality included studies on the CAN system designed for cryoablation of kidney tumors. Usability and functionality factors are presented in Table 1. Table 1: Factors related to usability and functionality Usability factors Functionality factors K1. Navigation F1. User guidance or support K2. Interaction F2. Data security K3. Learnability F3. Autorun K4. Ease of use F4. Customisability K5. Response time F5. Archivebility and Accessibility K6. Memorability F6. Ability to interact with external systems K7. Efficiency F7. Validation The usability factors examined in the questionnaire were navigation, interaction, learnability, ease-of-use, response time, memorability and efficiency. • Navigation refers to finding the way to the desired information through menus, graphical components, controls, displays, and layout (Palmer, 2002), as well as, even while doing this, knowing where one is at the instant (Roy, Dewit, & Aubert, 2001).

• • •

• • •

Interaction refers to the responses produced by the system to the user’s actions (Palmer, 2002) and the interaction between the system and the user (Chakraborty, Srivastava, & Warren, 2005). Learnability refers to the level of effort needed to learn how to operate the system (Calero, Ruiz, & Piattini, 2005). Ease of use refers to being able to operate the system without experiencing any difficulty or trouble (Chakraborty et al., 2005). Ease of use is one of the most important components along with navigation for usability and plays a significant role in the adoption of a system by users (Pearson, Pearson, & Green, 2007). Response time refers to the time needed by the system to respond to your activities or requests(Palmer, 2002). Memorability refers to the ease of recall of the main functions and their presentation on the system when reusing the system. An inconsistent interface structure raises the memory load on users (Seffah, Mohamed, Habieb-Mammar, & Abran, 2008). Efficiency refers to the ability of the system to allow working quickly to attain desired goals with the minimum number of steps/actions (Nielsen, 2003; Reilly, Leibrandt, Zonno, Simpson, & Morris, 2004).

Functionality factors examined in the questionnaire were user guidance or support, data security, autorun, customizability, archivebility and accessibility, ability to interact with external systems, and validation. • User guidance and support refers to the system that offers help and describes the necessary information about the steps that can be followed when the users encounter a problem or have a request (Waite & Harrison, 2007). • Data security refers to ensuring that the data are kept safe from corruption and that access to it is suitably controlled (Calero et al., 2005; Seffah et al., 2008). • Autorun refers to the system to run some operations/processes automatically. • Customizability refers to the flexibility to change system characteristics to a level that meets your needs or preferences. Customization increases user satisfaction by limiting information overload on users with respect to their preferences (Pearson et al., 2007). • Archivebility and accessibility refers to the ability to keep the successfully or unsuccessfully ended operations in the system, to archive data dated back to a specific date and needed very rarely, and access to these data. • The ability to interact with external systems refers to the ability to send and get the required information to/from external systems through different ways and formats on a controlled basis. • Validation refers to the ability to control the validity of data inserted into the system by other systems or users.

3. Methodology

The questionnaire was formed by two main parts to gather data. The first part consisted of demographic questions designed to solicit information about gender, age, country, working experience, computer-assisted navigated system experience, and kidney surgery experience. A total of 12 questionnaires were collected from surgeons. Only one respondent is female, the average age of the respondents was 45. Seventy-five percent of the respondents were from Turkey. Forty-two percent of them had work experience more than 15 years. Sixty-nine percent of them performed more than 15 surgeries over the last three years. Only 17% of them use a CAN system in their operations. The summary of demographic profiles of surgeons is given in Table 2. The second section of the questionnaire asked respondents to indicate their opinion about the importance of the factors related to usability and functionality. A seven-point Likert-scale type was used to measure the items. In a seven-point Likert-scale type, “one represents not important at all” and seven represents “extremely important”.

4. Results

Table 3 show the relative importance degrees of all the factors. The most important factor in terms of usability is the response time with a mean of 6.50. This points out that the required time to respond to the activity of a user is considered as an important factor by the surgeons. An optimal navigation system should give an answer quickly to surgeon's action while operating. The findings of Konradt, Lückel, and Ellwart (2012) are consistent with our findings and they have found that response time is an antecedent of usability and negatively influences it. Palmer (2002) has also showed that the responsiveness is related to more successful and high quality system. In addition,

the other most important factors are efficiency, ease of use and learnability with a mean of 6.25, 6.25, and 6.17 for the surgical navigation system. The surgeons expect the CAN system to facilitate surgical operations and achieve the tasks more quickly. Moreover, an easy to use navigation system is more likely to enhance the surgeon’s operational performance. Similar to our findings, Buchanan and Salako (2009) have showed that efficiency and learnability are the key attributes of usability of a digital library. However, in an earlier study of Calisir et al. (2010) efficiency and learnability are found to be the least important factors related to usability and ease of use is perceived as one of the most important factors. The reason behind this difference between the two studies may be related to the examined systems, because this study focused on CAN whereas Calisir et al. (2010) studied online auction and shopping web sites. Table 2: Demographic profiles of the respondents Gender (%) Female: 8.33 Male: 91.67 Age (year) Average: 45 Country (%) Turkey: 75 Denmark: 12.5 Italy: 12.5 Working experience (%) 1-5 years: 17 5-10 years: 8 10-15 years: 25 >15 years: 42 N/A: 8 Kidney surgery per year over the last three years (%) 0-1 surgery: 8 1-5 surgeries: 15 5-10 surgeries: 0 10-15 surgeries:8 >15 surgeries 69 Computer-assisted navigation system experience (%) Yes: 17 No:75 N/A:8 Table 3: The average and standard deviation of usability and functionality factors Usability

Avg.

Std. Dev.

Response time Efficiency Ease of use Learnability Interaction Navigation Memorability

6.50 6.25 6.25 6.17 5.83 5.58 5.42

0.67 0.62 0.87 1.19 1.27 1.24 0.90

Functionality Archivebility and Accessibility Autorun Data security User guidance and support Ability to interact with external systems Customizability Validation

Avg. 5.83 5.58 5.58 5.50 5.33 5.25 5.17

Std. Dev. 1.34 1.08 1.73 1.09 1.23 1.42 1.47

Among functionality factors, archivebility and accessibility are found to be the most critical factors with a mean of 5.83. This result points out that the data storage is an important technical aspect for the respondents, because it may be suggested that surgeons wants to track the data on the operations. Moreover, the ability to gain access provides interaction with all content of the system. Bertot at al. (2006) have also emphasized on the accessibility as a functionality factor, but later it is found that limiting accessibility to functionality cause to weaken the interpretation of the results and then scaled accessibility as third dimension in the user-centered design. From this point, accessibility may be examined apart from functionality in the future. Moreover, auto-run and data security with a mean of 5.58 are the other crucial factors in the CAN system design. This points out that surgeons are eager to use some operations automatically, not manually, and the results also imply that the data storage is critical in the system, so the data itself is also crucial. A system which keeps the data safe and uncorrupted is more likely to be preferred by surgeons. Similar to our findings, Calero et al. (2005) and Seffah et al. (2008) have shown that data security is an important consideration in accepting the system as trustworthy. As Tarafdar and Zhang (2005) suggest, security influences usability negatively because of complicated functionality. Therefore, since respondents gave higher importance on the factors related to usability, our findings are consistent with the findings of Tarafdar and Zhang (2005). On the other hand, memorability is the least crucial factor with a mean of 5.42 and is followed by navigation (5.58) and interaction (5.83) in terms of usability. Among these factors, the importance of degree of navigation is surprising. Although navigation is known as an important usability factor in the literature (Calisir et al., 2010;

Topaloglu et al., 2013), its effect on usability is found very low compared to other usability factors, however it has still relatively high importance compared to functionality factors. Hence, validation is the least critical functionality factor with a mean of 5.17 and is followed by customizability (5.25) and ability to interact with external systems (5.33). Additionally, these are the least important factors in both usability and functionality aspects according to the results so a little emphasis should be given to these factors as compared to the others. As users give higher emphasis on the usability related factors in the user-centered design, this result is acceptable. This finding is also consistent with the findings of Calisir et al. (2010) and Topaloğlu et al. (2013).

5. Conclusions

This study analyzes the relative importance of usability and functionality factors for a CAN system for cryoablation of kidney tumors. For this, 12 questionnaires have been collected from surgeons. The results of the analysis imply that response time, efficiency and ease of use are found to be the most important factors among usability factors, whereas interaction, navigation and memorability are the least important factors. Archivability and accessibility, auto-run and data security are found to be the most important factors among functionality factors, whereas ability to interact with external systems, customizability, and validation are found to be the least important factors. In terms of both aspects, response time, ease of use and efficiency are the most important factors among usability and functionality factors. This implies that respondents give higher emphasis on the usability related factors. In conclusion, the findings of this study may make a contribution to the early design stages of CAN system for cryoablation of kidney tumors and provide a better understanding of the expectations and perceptions of surgeons.

6. Acknowledgment

This work is supported in part by European Union Seventh Framework Program FP7/2007-2013under grant agreement number 270396 (I-SUR).

References

Adams, L., Krybus, W., Meyer-Ebrecht, D., Rueger, R., Gilsbach, J. M., Moesges, R., & Schloendorff, G. (1990). Computer-assisted surgery. IEEE Computer Graphics and Applications, 10(3), 43–51. Bertot, J. C., Snead, J. T., Jaeger, P. T., & McClure, C. R. (2006). Functionality, usability, and accessibility: Iterative user-centered evaluation strategies for digital libraries. Performance Measurement and Metrics, 7(1), 17–28. Brandt, G., Zimolong, A., Carrat, L., Merloz, P., Staudte, H. W., Lavallée, S., Radermacher, K., et al. (1999). Crigos: a compact robot for image-guided orthopedic surgery. IEEE Tansactions on Iformation Technology in Biomedicine, 3(4), 252–60. Buchanan, S., & Salako, A. (2009). Evaluating the usability and usefulness of a digital library. Library Review, 58(9), 638–651. Burnett, G. (2000). “Turn right at the traffic lights”: The requirement for landmarks in vehicle navigation systems. Journal of Navigation, 53(3), 499–510. Calero, C., Ruiz, J., & Piattini, M. (2005). Classifying web metrics using the web quality model. Online Information Review, 29(3), 227–248. Calisir, F., Bayraktaroglu, a. E., Gumussoy, C. A., Topcu, Y. I., & Mutlu, T. (2010). The relative importance of usability and functionality factors for online auction and shopping web sites. Online Information Review, 34(3), 420–439. Chakraborty, G., Srivastava, P., & Warren, D. L. (2005). Understanding corporate B2B web sites’ effectiveness from North American and European perspective. Industrial Marketing Management, 34(5), 420–429. Craven, J., Johnson, F., & Butters, G. (2010). The usability and functionality of an online catalogue. Aslib Proceedings (Vol. 62, pp. 70–84). Kline, R. B., & Seffah, A. (2005). Evaluation of integrated software development environments: Challenges and results from three empirical studies. International Journal of Human-Computer Studies, 63(6), 607–627. Konradt, U., Lückel, L., & Ellwart, T. (2012). The role of usability in business-to-business e-commerce systems: Predictors and its impact on user’s strain and commercial transactions. Advances in Human-Computer Interaction, 2012(20), 1–11.

Lu, M., & Yeung, W. (1998). A framework for effective commercial Web application development. Internet Research, 8(2), 166–173. Martelli, S., Zaffagnini, S., Bignozzi, S., Lopomo, N., & Marcacci, M. (2007). Description and validation of a navigation system for intra-operative evaluation of knee laxity. Computer Aided Surgery, 12(3), 181–188. Mezger, U., Jendrewski, C., & Bartels, M. (2013). Navigation in surgery. Langenbeck’s Archives of Surgery, 398(4), 501–14. Münzer, S., Zimmer, H. D., Schwalm, M., Baus, J., & Aslan, I. (2006). Computer-assisted navigation and the acquisition of route and survey knowledge. Journal of Environmental Psychology, 26(4), 300–308. Nielsen, J. (2003). Usability 101 : Introduction to usability why usability is important. Jakob Nielse’s Alertbox. Palmer, J. W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151–167. Pearson, J. M., Pearson, A., & Green, D. (2007). Determining the importance of key criteria in web usability. Management Research News, 30(11), 816–828. Reilly, E. F., Leibrandt, T. J., Zonno, A. J., Simpson, M. C., & Morris, J. B. (2004). General surgery residency program websites: Usefulness and usability for resident applicants. Current surgery, 61(2), 236–40. Roy, M. C., Dewit, O., & Aubert, B. A. (2001). The impact of interface usability on trust in web retailers. Internet, 11(5), 388–398. Seffah, A., Mohamed, T., Habieb-Mammar, H., & Abran, A. (2008). Reconciling usability and interactive system architecture using patterns. Journal of Systems and Software, 81(11), 1845–1852. Tarafdar, M., & Zhang, J. (2005). Analyzing the influence of web site design parameters on web site usability. Information Resourch Management Journal, 18(4), 62–80. Topaloglu, H., Gumussoy, C. A., Bayraktaroglu, A. E., & Calisir, F. (2013). The relative importance of usability and functionality factors for e-health web sites. Human Factors and Ergonomics in Manufacturing & Service Industries, 23(4), 336–345. Waite, K., & Harrison, T. (2007). Internet archaeology: Uncovering pension sector web site evolution. Internet Research, 17(2), 180–195. Wegner, K. (1998). Surgical navigation system and method using audio feedback. ICAD Proceedings (pp. 31–40).

Biography

Fethi Calisir is a Professor of Industrial Engineering at Istanbul Technical University. He graduated with a BS from Istanbul Technical University in 1989, a MS from the University of Miami in 1993, and a PhD from Purdue University in Industrial Engineering in 1996. His current research interests include IT Project Management, Software Usability, and Human Computer Interaction. His research papers have appeared in Computers in Human Behavior, Computers & Industrial Engineering, Total Quality Management, Accident Analysis & Prevention, Technovation, Managing Service Quality, International Journal of Information Management, Management Research News, and Internet Research. Ecem Başak is a Research Assistant of Industrial Engineering at Istanbul Technical University. She graduated with a BS in 2011. She is expected to get her MS from Istanbul Technical University in Industrial Engineering in 2014. Her current research interests include Technology Acceptance, Usability and Functionality, Human-computer Interaction. Duygu Erol Barkana is an Associate Professor of Electrical and Electronics Engineering at Yeditepe University. She got her BS degree from Eskişehir Osmangazi University in Electrical and Electronic Engineering in 2001, MS and PhD degrees from Vanderbilt University in Electrical Engineering and Computer Science in 2003 and 2007, respectively. She is the director of Robotics Research Laboratory in Yeditepe University. Her research interests are in the areas of Robotics, Control systems, Human-robot Interaction.