Reusable Knowledge for Satisficing Usability Requirements 1
Luiz Marcio Cysneiros1, Vera Werneck2 and Andre Kushniruk3
Department of Mathematics and Statistics - Information Technology Program York University
[email protected] 2 Informatic and Computer Science Department Rio de Janeiro State University (UERJ) – Brazil
[email protected] 3 School of Health Information Science University of Victoria, Canada
[email protected]
Abstract. Usability is becoming increasingly recognized as being an important factor in the acceptance of systems by end users. Usability requirements can be considered to be requirements that capture the usability goals and associated measures for a system under development. In order to ensure usable systems we must ensure identification of appropriate requirements regarding these critical aspects of systems. However, a number of difficulties exist, for example it may be difficult to quantify and precisely specify these qualities in software systems. There is a basic need for systematic approaches to reason, model and analyze usability from the early stages of the software development. Furthermore, it is necessary to develop a usable ontology or classification of measurable aspects of usability that can be used to aid in the specification of usability requirements. These ontologies should be represented in a way that facilitates their use as guidelines for the requirements elicitation process. This work builds on review of literature in the area of humancomputer interaction and the emerging field of usability engineering in developing a catalog of aspects of usability that can be considered during requirements gathering. This catalogue is used to guide the requirements engineer through alternatives for achieving usability. The approach is based on the use of the i* framework, having usability modeled as a special type of goal. We show how usability can be modelled through different viewpoints with different alternatives for operationalizing it. An example in the health care domain is used to illustrate. 1. Introduction Despite the fact that neglecting or not properly dealing with non-functional requirements (NFR) can jeopardize the success of your software [3], [10] [4] [6], [11], functionalities have been the main focus of software development process. NFRs such as performance, safety, accuracy and usability have been systematically put aside. Among them, usability has been pointed out as
being one of the most important factors in the acceptance of systems by end users. As a classic example, usability played an important role in the deactivation of the London Ambulance Service software [7]. Usability requirements can be considered to be requirements that capture the usability goals and associated measures for a system under development [15] [17]. Usability goals may include a range of system aspects related to effectiveness, efficiency, learnability and memorability [15]. Usability aspects may strongly influence future design decisions. For example, if one is developing a system to help physicians to retrieve possible treatments for a disease from many existing databases, usability aspects may play a crucial role. For example, the use of a handheld PDA (Personal Digital Assistant) for deploying a decision support system to end users could prove to be an imperative, which might demand the use of alternative system architectures. On the other hand, if images have to be retrieved to support some of the treatment, using a PDA may not be the best option. From this perspective we see that usability, as well as other Non-Functional requirements, should be dealt with from the early stages of software development [2][4][6]. Achieving usability goals might even help to disclose new functional requirements. For example, when developing a system for a clinical analysis laboratory we once faced a situation where the stakeholders were concerned with the usability of a routine to allow physicians to review patients’ reports on-line. This situation arose because most reports were longer than one or even two computer screens in length. When dealing with printed reports, physicians do not typically experience many problems locating test results that may be located several pages back. However, when doing the same task on-line it may be difficult to find the specific tests they want to locate. To solve this problem we created mechanisms to search for specific tests. However, the stakeholders considered this insufficient when a prototype was shown to them. They informed us that because there were many fewer tests on a screen than in the paper version it would be easy for a physician to forget about tests he had seen 4 or 5 screens before. To
cope with this, we introduced a new functionality to the system, i.e., the capability to cross reference results looking for inconsistencies. As with other NFRs, usability can be rarely said to be satisfied by software. It may be said that we frequently satisfy usability within acceptable limits. Simon [18] coined the term “satisfice” referring to these situations. Currently, there are a diverse range of definitions and connotations to the term usability. There are also a considerable number of works that tackle some individual aspects of usability. However, they are currently scattered among many different literatures making it difficult for a software engineer to easily reason about different ways to achieve usability. Moreover even largely cited works from the usability domain such as [12], [13], [16] and [17] do not stress different alternatives to satisfice each usability requirement. Most of the solutions shown are in the style of guidelines instead of showing possible operationalizations to satisfice usability requirements. Finally, these works fail to show software engineers looking for guidance on achieving usability, how many of the alternatives can impact other NFRs. For example, using graphical user interfaces (GUI) may indeed contribute (help) to both attractiveness and effectiveness of a system [15], however this may also be prejudicial (hurt) performance and raise cost concerns. It may also hurt portability concerns (in the sense of having the system available in as many places and time as possible) since the use of a PDA helps portability, to make the system available in most places and time. This would be even clearer if we consider that many PDAs currently have phone capability providing web access. Therefore, there is the need for establishing reusable knowledge on possible alternatives to operationalize usability requirements. This knowledge could be used as a guide to requirements engineering when addressing usability requirements and as a consequence facilitate usability being efficiently dealt with from the early stages of software development. This work presents a reusable knowledge base to guide requirements engineering on possible alternatives to operationalize usability requirements and indicate how many of these alternatives would impact other NFR such as security performance and availability. This knowledge base is expressed using the i* framework [21] to create a catalogue to store the knowledge. It builds on review of literature in the area of human-computer interaction and the emerging field of usability engineering as well as on some personal experiences aimed at developing a catalog of aspects of usability that can be considered during requirements gathering. We use this catalogue to produce a systematic approach to guide the requirements engineer through alternatives for achieving usability. Each alternative is modeled in order to allow the requirements
engineer to evaluate these alternatives in contrast with other NFRs. The paper starts by presenting the catalogue for usability. In Section 3 we will show some examples drawn from the case study we carried out to evaluate the use of the reusable knowledge base. Section 4 concludes the work 2.
Usability Catalogue
There are a number of works presenting guidelines for good usability [12] [13] [8] [9]. Most of them focus either on ergonomics aspects related to the actual user interface design and layout, or on usefulness aspects related to how well a system allows a user to carry out a required task [8][15][16]. In this work we will be addressing both of these aspects of usability, and in addition ease of learning [15]. A catalogue is used to capture knowledge about achieving usability in many different situations. The knowledge may come from various sources ranging from the many existing works in the area to specific knowledge accumulated by individuals. Having this catalogue available, we can reuse its knowledge or add new knowledge to it. In this catalogue, usability is interpreted by refining it into subgoals and subsubgoals, eventually linking them to implementable mechanisms. Various subgoals and mechanisms may contribute to usability to varying degrees. Each stakeholders' interpretation of usability may lead to different goal refinements and mechanisms. The various interpretations of usability can be collected and organized into a catalogue for reference during requirements elicitation, analysis and design. Hence, the knowledge in this catalogue will be represented using a primarily hierarchical structure to allow the representation of the organization’s knowledge starting with the higher-level goals to achieve usability. The catalogue also allows for representing different ways of achieving the same goal so one can choose the way that is best suited to the domain being analyzed. Along with the operationalizations for usability, we also show possible correlations to other, possibly conflicting requirements. By doing so we are able to show that one specific solution might achieve usability and contribute positively or negatively to other requirements such as performance and security. The catalogue was built using i* [21] constructs including: softgoals, goals, tasks, and beliefs. The softgoal concept is used in i* to express non-functional requirements. NFRs frequently interact with each other in complex ways. Qualitative reasoning can be carried out using contribution links among softgoals. The semantics of the links are based on the satisficing concept [5] introduced in section 1. The most common contribution types are Help/Hurt (positive/negative but not sufficient
to meet the parental goal), Some+/Some(positive/negative of unknown degree), whereas Make/Brake indicates positive/negative of sufficient degree. Although these distinctions are coarse grained, they are enough to help us decide whether we need further refinement and search for more specific softgoals and operationalizations or not. Contribution links allow one to decompose NFRs to the point that one can say that the operationalizations to this NFR have been reached (i.e., the goals are no longer “soft”). In fact, operationalizations can be viewed as functional requirements that have arisen from the need to meet NFRs. This can explain why people frequently face doubts about if a requirement is functional or nonfunctional. Take for example the development of an information system for a clinical analysis laboratory. We may have stated a requirement like: “Samples should be traceable so one can know, at different times, where this sample is”. This may appear to be a functional requirement while, in fact, it is a non-functional requirement: “The software must handle samples” constrained by the NFR Traceability. The fact that an NFR when operationalized may result in new functional requirements points to the virtual impossibility of eliciting all the functional requirements, and only then
Figure 1 – Initial Approach to Usability Catalogue eliciting Non-Functional requirements. Operationalizations are typically specified as tasks, each indicating a particular way of doing something. All the subcomponents of a task (refined using the task decomposition link ( ) must be carried out. If there is more than one way to accomplish something, then the state of affairs to be achieved is represented as a goal with means-end links ( ) linking to the alternatives. Contribution links are the core of design decisions. By reasoning about how different operationalizations would contribute to satisfice a softgoal, one may decide which the best alternative to purse is. Based on the semantics of the contribution links [5], decision values are propagated from an offspring to its parents allowing one to visualize Softgoal Resource Goal Task
Figure 2 – Satisficing Usefulness Softgoal
Correlation Link Contribution Link Dependency Link Decomposition Link Means-End Link
what impact would come from adopting one alternative instead of another. A prototype tool has been developed to support propagating the contributions automatically but allowing interventions of the designer in case of conflicts or undecided situations arises. Figure 1 gives an idea of how this catalogue was built. We can see that we decompose de general softgoal usability into three main concepts used to address usability requirements, Ergonomics, Usefulness and Ease of learning. We further decompose each of these subgoals into more detailed softgoals. Ergonomics for example is refined into Use perceptual aspects and Use cognitive aspects. In turn Ease of learning is refined into only one subgoal Ease of new/occasional users accomplishing tasks. We kept refining each softgoal moving towards concrete mechanisms that would operationalize each of the as explained above. Unfortunately, the resulting graph is too large to fit in here. We will be presenting here selected parts of the catalogue. The entire catalogue can be seen at [1]. Note that this catalogue should be considered as being partial. Although it is a comprehensive set of existing knowledge on Usability Requirements we understand that it is not complete. In fact, we encourage others to enrich it with personal experiences or further knowledge that had been left aside. Figure 2 shows operationalizations for the softgoal Usefulness that appears in Figure 1.We can see for example that one way to achieve Usefulness is to Improve cognition We then established that the goal Assure Timeliness of Information would help operationalizing this softgoal. In turn, this goal has two Softgoal
Resource Goal Task
alternatives to be satisfied, either by the task Conduct usability testing or by the task Guarantee constant access. Since we determined that Conduct usability testing was sufficient to allow concrete measures we stopped there and considered this task to operationalize the Assure Timeliness of Information goal as well as other goals that can be seen in Figure 2. On the other hand, Guarantee constant access could still be further refined into the goal Provide Continuous access which in its turn could be operationalized in three different ways Use a desktop, Use PDA or Use TABLETs. Use a desktop, besides helping to operationalize usability will also at some extent help on performance aspects since a desktop can usually be more powerful then the other two alternatives. It will also help availability but only if used on line. However, it would bring about portability concerns. Again, here we consider portability in the sense of being able to have data and/or software available as much as possible the whole time. In their turn, Use PDA and Use TABLETs would make portability since we can carry those most of the time and places. We can also see the task Use GUI [Use Cognitive Aspects] representing one operationalization that can be used for satisficing the Use Cognitive Aspects softgoal seen in Figure 1. While Use desktop and Use TABLETs would help this operationalization, while Use PDA would compromise it. Figure 3 shows operationalizations for another softgoal from Figure 1 i.e. Ease of learning including how the operationalization for this softgoal would impact other NFRs. It also shows some of the operationalizations
Correlation Link Contribution Link Dependency Link Decomposition Link Means-End Link
Figure 3 – Satisficing Ease of Learning Softgoal
for the Use Cognitive Aspects that would have some impact on ease of learning or vice versa. We can see there for example that one operationalization for Ease of Learning would be to provide On line Help. Although it would also help achieving Effectiveness it may also hurt concerns regarding Cost and Delivery time. While the Use of known layout may also operationalize Ease of learning without hurting Cost and Delivery time NFRs Thus having access to the catalogue, a requirements engineer can reuse the existing knowledge for satisficing usability requirements. Typically, he/she would evaluate what is the environment to where usability is being evaluated and choose one or more approaches and proceed top-down. For example, if dealing with an environment with a lack of expertise on using computers as for example the health care domain, the requirements engineer may opt to approach usability using usefulness instead of ease of learning as primary concern. Then, the requirements engineer may select the way of tackling usefulness from the catalogue which will best suit the user needs and proceed until operationalizations are found. In many cases more than one possible operationalization would be found and the requirements engineer would have to evaluate which would have less negative impacts with other NFRs to guide the choice. Another way to use the catalogue is to check if some specific operationalization would hurt any other NFR. Suppose the requirements engineering arrives at the conclusion that it is necessary to use a PDA, he/she could use the catalogue to check how using a PDA would help or hurt other NFRs. Although sometimes these correlation may be clear, at other times they are not. Having a collection of knowledge on this matter available could be of a great help to requirements engineering. In the next section we will show some examples from our case study to better illustrate the usefulness of the catalogue. 3.
Case Study
We conducted two case studies to evaluate the use of the proposed catalogue. The first case study uses a team with two undergraduate students in the last year of studies working on the Guardian Angel Project description [19]. The other was carried out by one of the authors together with one undergraduate student using a real life case to create a system to control surgeries. In this paper we will focus on the later. The aim of the Surgery Control System is to control the planning and scheduling of surgeries in the Pedro Hernesto University Hospital (HUPES) in Rio de Janeiro, Brazil. The hospital needs to optimise and administrate the surgeries to give a better treatment in a Public Hospital.
The patient is sent to the surgery area after a physician’s examination from HUPES or other Public Hospitals. Then the patient fills out a form with personal data and some generic symptoms and diseases history data. Pre-surgery procedures consist of the patient’s general exams like blood and urine analysis, as well as more detailed exams when needed such as CAT scan or MRI for evaluation of the patient’s surgical condition risk. An anesthetist physician analyses the patient’s exams and gives the grade of patient surgical risk. The surgery is planned for the patients with low or average risk. However patients with high risk can be operated on if death is imminent if the surgery is not performed. At this point they inform the patient of the time and the day of the procedures to be performed. The patient goes to the hospital in the appointed day and the procedures involved in pre-surgery begin, for example some exams that may have to be done or some alimentary restrictions. The day of the surgery may or may not be the same day that the patient arrives at the hospital. After the surgery the surgeon records the information about the surgery’s effectiveness, the prescriptions and the nutrition administered to the patient. Sometimes the patient cannot be operated in the day planned because an emergency case arrives and surgery is immediately performed on the new patient after some initial exams. There are some cases in which the surgery is preplanned and the patient leaves the hospital without the surgery being performed. After the surgery is complete, the surgeon discharges the patient and plans one or more appointments before he leaves the hospital, for example to remove the sutures. The current procedures have a number of problems such as potentially lengthy wait times of the patients in the hospital waiting for the surgery. In many cases the patients enter more than once to the hospital for the same operation. At other times the surgeon remains inactive because he does not have any operation planned although his waiting list is very long. The head surgeon wants a system offering special features to help surgeons to better use their time, (for example by allowing them to schedule their own surgeries, except for the high risk operations) or by allowing them to stay home/office when they do not have a surgery to perform, while at other times be on call for special occasions like emergency surgery. All the surgeons have a mobile phone and some have a computer home or even a PDA. Others surgeons do not frequently use the computer, so the system has to deal with different kind of users. The system should also allow the patient to provide his preference about the day of week and time and to access the system at home to confirm the surgery date. The patient should be able to see his data, prescription and nutrition restriction.
In this work we show how the use of the catalogue helped us to address usability requirements from the earlier stages of software development. We here illustrated the modeling of the above scenario using the i* framework [21]. Although we applied a systematic process to model usability together with other NFRs using catalogues to guide us, due to space limitation we will not be detailing the process for doing so. This will be subject for future work. Here we will only be showing how the catalogue guided us in getting to usability solutions and to evaluate different alternatives for that. In a nutshell, to elicit usability concerns we visit every agent asking ourselves and the stakeholders where usability should be an issue to be tackled. Specifically, we inspect tasks and goals inside this agent evaluating if this task or goal would require specific usability concerns. For that purpose we use the catalogue for guiding which usability alternative should be used. Figure 4 shows part of the i* model to address the above scenario before usability was tackled. Once we have defined the basic needs (both functional and non-functional) for the system to be
developed, we can start reasoning about usability. Usability itself is a softgoal that will contribute for the Usable Health care system. Thus here we would have to use the catalogue to guide us on the possible ways to satisfice usability here. We could for example consider if the Surgery Control System for the Surgeon would have any specific concerns for Usefulness. This software is expected to be heavily used inside the HUPES and also outside for the surgeons to consult the surgeries and scheduling their surgeries. It is also expected that the system would send messages to surgeons’ mobiles phones so the surgeons can connect to the system to inform their availability to perform an emergency surgery. Therefore, Usefulness is a major concern. To assess that, we determined that among the possible ways of satisfying Usefulness, Improve Cognition would be required leading to a further refinement for Assure Timeliness of Information. To guarantee that, we should Provide Continuous Access. We can do that by Using a Desktop, PDA or Tablets. However, from the
Figure 4 – Partial Model Before Usability
catalogue we see that the use of a desktop would hurt Availability of Information because for many applications as in health care or in the retail industry, the need to reach a desktop in order to enter or consult data can be highly inconvenient. On the other hand, Using a PDA or Using tablets contributes positively (help correlation link) to Availability of Information. Since all surgeons carry a mobile phone all the time, Use a Tablet will allow the surgeons to be constantly informed about surgeries information and will increase portability on the Notify Surgeon task. However this could hurt Cost concerns since not all surgeons would be able to afford having one Tablet. Since most the mobile phones the surgeons carry are also PDA this alternative may be more feasible than using a Tablet. Thus by either carrying a mobile phone with PDA capabilities or a Tablet, a surgeon can retrieve his patients, surgeries or agenda records and have them available constantly. Figure 5 shows the model representing this reasoning Aside from that, we should check other tasks and goals to evaluate whether they need special concerns for usability. The task To Schedule the Surgery was such a case. It is decomposed into the tasks of Show Physicians Available and Search a Date Available and the goal Set a Surgery Date. This goal is performed for one of three others tasks: Set an Emergency Surgery, Set
Surgery with High Risk and Set a Surgery with Low or Average Risk. The two first tasks are decomposed in Notify the Surgeon and Confirm the Surgery. They also decomposed in the softgoal Usability since it plays a major role in allowing the Surgeon to achieve the tasks mentioned before. Again, using the catalogue we decided to decompose Usability into Provide Improved Presentation because it is important to provide ease routines for the Surgeon. This is further decomposed into the Use Graphics goal. However, analyzing the catalogue we see that the Use of a Tablets or Use PDA might not improve the Use of Graphics. After representing all the possible correlations we could find, we must reason about which operationalizations will be kept (Satisfied) or not (Denied). We can see that after tradeoffs between Availability of Information, Accessibility, and Use of Graphics, it was decided that even at the expense of compromising the others, Availability was considered to be most critical to the project, since if the Surgeon can not be notified it is very likely that he will fail to either enter and receive the information during the day. So to enter the confirmation he can use the PDA to confirm the date and his availability. This way, as represented in Figure X, we represent that Use a Tablets and Use of PDA will be satisfied while Use a Desktop will be denied.
Figure 5 – First Reasoning on Usability
Propagating these decisions we end up, for example, having the Usability for Set an Emergency Surgery or Set a Surgery with Average or Low Risk considered Weakly Denied since the Use of Graphics will be compromised. On the other hand, Availability of information is evaluated as satisfied. Accessibility and portability are also evaluated satisfy since the use of the Tablets and Use PDA will impact positively. Aside from the general usability tackled above, we also search all the tasks and goals searching for possible special concerns regarding usability. Figure 6 shows a model for representing Usability for the Surgery Control System software agent. The task To Schedule the Surgery is decomposed into the tasks of Show
Emergency Surgery, Set Surgery with High Risk and Set a Surgery with Low or Average Risk. The two first tasks are decomposed into Notify the Surgeon and Confirm the Surgery. They are also decomposed into the softgoal Usability since it plays a major role in allowing the Surgeon to achieve the tasks mentioned before. Again, using the catalogue we decided to decompose Usability into Provide Improved Presentation because it is important to provide ease routines for the Surgeon. This is further decomposed into the Use Graphics goal. However, analyzing the catalogue we see that Use of Tablets might improve the Use of Graphics. After representing all the possible correlations we could find, we must reason about which
Figure 6 – Further Reasoning on Usability Physicians Available and Search a Date Available and the goal Set a Surgery Date. This goal is performed for either one of three others tasks: Set an
operationalizations should be kept (Satisfied) or not (Denied). We can see that after tradeoffs between Availability of Information, Accessibility, Cost and
Use of Graphics, it was decided that even at the expense of compromising the others, Availability was considered to be most critical to the project, since if the Surgeon can not be notified, it is very likely that he will fail to enter and receive information during the day. If he does not have a PDA or a Desktop to enter the information he can phone HUPES. This way, as represented in Figure 6, we represent that both Use a Tablets and Use of PDA will be satisficed while Use a Desktop will be denied. Propagating these decisions we end up, for example, having the Usability for Set an Emergency Surgery or Set a Surgery with Average or Low Risk considered Weakly Denied since the use of Graphics could be compromised. On the other hand, Availability of information is evaluated as satisficed. Accessibility and portability are also evaluated as satisficed since the use of the Tablets will impact positively. Both the teams in this work reported they could more easily address usability than previous projects they have participated before. Some of the students have even had experience in real life projects. Although we can not quantify at the moment how much guidance the catalogue offered, we believe from their impressions it was of great help. The team working in the Guardian Angel project was particularly impressed with the way the catalogue facilitate them to avoid pitfalls regarding conflicts with other NFRs. One problem this team saw in the catalogue was not being able to address it using different levels of granularity regarding the stored knowledge. This problem posed some challenges on the way they had to search the catalogue of knowledge. 4.
Conclusion
In this paper we present a reusable catalogue with strategies to satisfice usability requirements. This catalogue collects a large amount of knowledge on achieving usability goals from the literature and enriches it with personal experiences also. The aim of this catalogue is to help the requirements engineer to address usability requirements from the early stages of software development. It also aims to alert requirements engineering for possible conflicts that might arise from choosing among many ways of satisficing usability. Being a very large catalogue it was only possible to show part of it here (around 30% of it). However the full catalogue is available at [1]. As said before, this catalogue does not intend to be complete. It is designed so requirements engineers can add their own experience to the catalogue. Although we present examples based on the i* framework we believe that, being goal oriented, the catalogue can help requirements engineers using most of the existing approaches such as KAOS [20] and GBRAM [14].
We recognize that the structure used to store and retrieve information from this catalogue can pose some challenges. However, at the present moment we are not aware of other forms of representation that would allow easy understanding and use. Furthermore, from the results of our case studies we believe that the present representation allows the requirements engineer to benefit from the knowledge there represented. Future work includes replicating the Guardian Angel project without using the catalogue by a third team and measuring the time spent by each team and the quality of the resulting models. We also intend to have each team implementing one part (the same) of the project. This would allow us to measure quality not only from the models perspective but also from the perspective of existing software. We are also investigating different alternatives for storing and retrieving information from catalogues such as the one presented here, in order to facilitate retrieving information at different levels of granularity. 5.
References
[1] http://www.math.yorku.ca/~cysneiro/usability.htm [2] Boehm, Barry e In, Hoh. “Identifying Quality-Requirement Conflicts”. IEEE Software, March 1996, pp. 25-35. [3] Breitman, K. K, Leite J.C.S.P. e Finkelstein Anthony. The World's Stage: A Survey on Requirements Engineering Using a Real-Life Case Study. Journal of the Brazilian Computer Society No 1 Vol. 6 Jul. 1999 pp:13:37. [4] Chung, L., Nixon, B. “Dealing with Non-Functional Requirements: Three Experimental Studies of a ProcessOriented Approach” Proc. 17th Int. Con. on Software Eng. Seattle, Washington, April pp: 24-28, 1995. [5] Chung, L., Nixon, B., Yu, E. and Mylopoulos,J. “NonFunctional Requirements in Software Engineering” Kluwer Academic Publishers 2000. [6] Cysneiros,L.M. and Leite, J.C.S.P. “Non-Functional Requirements: From Elicitation to Conceptual Model” IEEE Transactions on Software Engineering – May, 2004 (Vol. 30, No. 5). [7] Finkelstein, A. and Dowell J. “A comedy of Errors: The London Ambulance Service Case Study” Proceedings of the Eighth International Workshop on Software Specification and Design, IEEE Computer Society Press pp 2-5 1996. [8] Hix, D. and Hartson, H.R. “Developing User Interfaces: Ensuring Usability Through Product & Process”, John Wiley & Sons, 1993. [9] Kushniruk, A. “Evaluation in the design of health information systems: Application of approaches emerging from usability engineering”, Computers in Biology and Medicine, 32, 141-149. [10] Lindstrom, D.R. “Five Ways to Destroy a Development Project” IEEE Software, September 1993, pp. 55-58.
[11] Mylopoulos,J. Chung, L., Yu, E. and Nixon, B., “Representing and Using Non-functional Requirements: A Process-Oriented Approach”, IEEE Trans. on Software Eng, 18(6), pp:483-497, June 1992. [12] Nielsen, J. Mack R.L. “Usability Inspection Methods”, John Wiley & Sons, 1994. [13] Nielsen, J. “Designing Web Usability: The Practice of Simplicity”, New Riders Publishing, 1999. [14] Potts, C., K. Takahashi and A. I. Antón., Inquiry-Based Requirements Analysis. IEEE Software, 1994. march: p. 21-32. [15] Preece, J. Rogers, Y. and Sharp H. “Interaction Design” John Wiley & Sons, 2002. [16] Roson, M., Carroll, J. Cerra, D. “Usability Engineering: Scenario-based Development of Human-Computer Interaction”, Morgan Kaufmann Publishers, 2001. [17] Shneiderman, B. “Designing the User Interface”, AddisonWesley, 1997. [18] Simon,H.A. "The Sciences of the Artificial", 3rd edition MIT Press, 1981 [19] Szolovits, P., Doyle, J., Long, W.J. “Guardian Angel: Patient-Centered Health Information Systems” Technical Report MIT/LCS/TR-604, http://www.ga.org/ga/manifesto/GAtr.html [20] VanLamsweerde, A. Goal-Oriented Requirements Engineering: A Guided Tour. in Proc. of 5th IEEE Int. Symp. on Requirements Engineering. 2001: IEEE. [21] Yu, E. “Towards Modelling and Reasoning Support for Early-Phase Requirements Engineering” in Proc. Of the 3rd IEEE Int. Symp. on Requirements Engineering, pp:226-235, 1997.
