Correspondence Validation Method for GUI Operations and Scenarios by Operation History Analysis Junko Shirogane Tokyo Woman’s Christian University
[email protected]
Yoshiaki Fukazawa Waseda University
[email protected]
ABSTRACT
Interactions between users and software are usually described as scenarios so that it is easy to reflect users’ viewpoints in software development. In many cases, scenarios are written in a natural language so that users can communicate with software developers smoothly. However, it is difficult to validate correspondences between the flows of operations written in scenarios and those in software. We assume that the flows of software can be expressed by the flows of Graphical User Interface (GUI) operations. In this paper, we propose a method for validating the correspondences between the flows of operations in scenarios and those of actual GUIs. This validation can be performed by analyzing the historic data of GUI operations heuristically. The label names on widgets are extracted from the histories, terms corresponding to label names on widgets are extracted from scenarios, and then the orders of appearance of label names and f terms are validated.
many cases, scenarios are written in a natural language so that users who do not have knowledge of software development can easily understand scenarios. Therefore, in the development of usable software [3], scenarios are paid attention to and are often used as requirement specifications. The flows of operations of Graphical User Interfaces (GUIs) are required to be realized along with scenarios. However, it is difficult to validate the correspondences between scenarios and operations of GUIs in actual software, because scenarios are often written in a natural language. In our research, we propose a method for automatically validating the correspondences between the flows of the operations in the actual software and those described in scenarios. In our method, the histories of users’ GUI operations are analyzed, and the correspondences between the result of the analysis and the flows of operations described in scenarios are validated. In our research, we develop our system using the Java Swing package in Java programming language, which is also used for the examples in this paper. However, our method is applicable to any other programming language and GUI package.
Author Keywords
GUI, scenario, history analysis, correspondence validation ACM Classification Keywords
H.5.2 [Information Interfaces and Presentation]: User Interfaces - Graphical user interfaces (GUI) INTRODUCTION
The usability [1] of software is important so that users can use computers easily. To develop usable software, it is necessary for software developers and users to communicate effectively. Scenarios [2] can help such effective communication, so they are widely used in the software development phase. In scenarios, interactions between users and software are described in order according to users’ viewpoints. That is, it can be said that the flows of operations of user interfaces in software are described in scenarios. In many cases, scenarios are described focusing on users; thus, users’ viewpoints can be easily reflected in software development. Also, in Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IUI’08, January 13-16, 2008, Maspalomas, Gran Canaria, Spain. Copyright 2008 ACM 978-1-59593-987-6/ 08/ 0001 $5.00
257
SCENARIO DESCRIPTION AND GUI
Scenarios are used for describing interactions between actors and target software in use case diagrams of Unified Modeling Language (UML) [4]. Actors of diagrams represent users and other related systems. Use case diagrams represent functions in the target software. Therefore, scenarios represent the steps for running each function in accordance with users’ viewpoints. In our method, scenarios are described in the itemized natural language based on the reference [2]. Each step written in the itemized natural language is called an “event”. Also, in scenarios, it is possible for software developers to describe various detailed levels of steps as events. In our method, the correspondences between the steps of scenarios and the operations of the software are analyzed. For this purpose, it is necessary to obtain each individual item for input, selection and display on GUIs from scenarios. Therefore, we assume that these items on GUIs are described in the target scenarios in our method. There is not always just one flow of interaction between actors and software in a use case [2]. There are two types of flows of interactions, successful flows and unsuccessful flows. In successful flows, the function described in a use case is executed successfully. The most typical scenario of successful flows is called a “basic scenario”, and the other successful flows are called “alternative scenarios”. In the un-
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Scenario “Rent videos” A clerk selects “Rent video”. The clerk inputs member’s ID. The system searches for the member’s entry. The clerk selects “Add rent videos”. The clerk inputs video ID for renting. The clerk selects the term of renting. The system displays the confirmation of renting videos. The clerk checks the confirmation and select “Confirm”. The system calculates the charge and display it. The clerk receives money from the member and selects “Confirm”.
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
Scenario “Register members” A clerk selects “Register new members”. The clerk inputs member’s name. The clerk inputs member’s address. The clerk inputs member’s telephone number. The clerk selects member’s sex. The system displays member’s ID. The system displays member’s name. The system displays member’s address. The system displays member’s telephone number. The system displays member’s sex. The clerk selects “Confirm”.
Figure 1. Example of video rental management system
Figure 2. Example of a flow of window
successful flows, the function described in a use case is executed unsuccessfully as a result of errors and mistakes. Scenarios of unsuccessful flows are called “exception scenarios”. In our method, these alternative and exception scenarios are handled as independent scenarios that have slightly different flows of events from the basic scenario. Figure 1 shows examples of scenarios in our method. The basic scenarios of the use cases “Rent videos” and “Register members” in a video rental management system are shown in these examples. Events are the itemized steps of these examples, such as ‘A clerk selects “Rent video”.’ and ‘A clerk selects “Register new members”.’ Also, an example of GUIs realizing the scenario “Rent videos” in Figure 1 is shown in Figure 2.
Obtaining histories of GUI operations
First, our system adds a function of recording the histories of GUI operations to the target software. This function is generated by parsing source programs of the software and is automatically added. It is necessary to record the histories of GUI operations because they are used for extracting the flows of user operations for the validation. Therefore, the following are recorded as the histories of user operations. • An active window in each step of operation • All displayed windows in each step of operation • Types of widgets that users operate • Variable names of widgets that users operate • Label names of widgets that users operate
OVERVIEW OF VALIDATION
In our method, the correspondences between scenarios and the actual flows of operations in software are validated. To realize this purpose, our system obtains the histories of user GUI operations, analyzes them, and validates the correspondences using the results of analysis and scenarios. The details are as follows.
258
• Types of user operations of widgets, such as mouse clicks and keystrokes Widgets often have label names. For example, there is a button named “OK” in Figure 2 (2). This “OK” is the label name, and this button is used for notifying the software
that the users have finished inputting text. That is, this label name “OK” indicates this notification. Thus, label names show the instructions of widgets, that is, what users do when using the widgets. For example, button widgets and menu item widgets often have label names, such as “OK” in Figure 2 (2). However, in many cases, some types of widgets, such as text field widgets for text input and list box widgets for item selections, do not have label names. Their label names are often located near them. For example, the box to the right of “Member’s ID” in Figure 2 (2) is a text field widget. Its label name is “Member’s ID” on the left. Thus, the widgets of label names are often different from widgets indicated by the label names. In this paper, we call this widget of label names indicating the instruction of a certain widget an “instruction label”. Also, we call a widget for which the instruction is indicated by a certain instruction label a “target widget”. Target widgets include text field widgets, list box widgets and combo box widgets. For validation, it is necessary to identify label names for all widgets. Therefore, our system assesses label widgets located near target widgets and identifies the associations between label widgets and target widgets. When the label widget that is associated with a target widget is identified, it is considered to be the instruction label for the target widget. Therefore, the label name of the instruction label is used as the label name of the target widget. For identifying the associations between instruction labels and target widgets, the following are recorded as histories in displaying windows. • Types of widgets arranged in an active window • Variable names of widgets arranged in an active window • Label names of widgets arranged in an active window • Coordinates of widgets arranged in an active window
Figure 3. Example of history of GUI operations
• Sizes of widgets arranged in an active window An example of a history of GUI operations in operating the GUI of Figure 2 is shown in Figure 3. (1) is the entry of the type, variable names, and label name of a widget, (2) is the entry of the coordinates and size of a widget. (3) is the entry of a window becoming active, and (4) is the entries of the variable names, label names, and the types of operations, such as mouse clicks and keystrokes, for the widget operated by users. Analyzing histories of GUI operations
Our system analyzes user histories of GUI operations. There are two types of analysis, i.e., the analysis of the widget coordinates and the analysis of flows of operations. Analysis of widget coordinates
Our system assesses associations between instruction labels and target widgets using the histories of the coordinates of widgets. The label name of the identified instruction label is used for the label name of the target widget. In Figure 2, a text field widget is the box to the right of the string “Member’s ID” in (2), a list box widget is the box under the string “Selected videos” in (3), and a combo box widget is the box to the right of the string “Term of renting” in (4).
259
The instruction labels are often located immediately to the right or above the target widgets. We surveyed the GUIs of 9 widely used software packages, such as mail clients including Outlook Express [5] and Thunderbird [6], web browsers including Internet Explorer [7] and Firefox [8] and the text editors including Notepad in Microsoft Windows [9]. The results are shown in Table . In this table, “Total” means the total number of each target widget in the GUIs of the 9 software packages. “Upper” indicates the number of target widgets for which instruction labels are located above the widget, “Left” indicates the number of target widgets for which instruction labels are located to the left, “Lower” indicates the number of target widgets for which instruction labels are located below, and “Right” indicates the number of target widgets for which instruction labels are located at the right. “Other” indicates the number of target widgets for which instruction labels are not located close to the widget. Text field widgets Combo box widgets List box widgets
Total 277 270 90
Upper 40 26 42
Left 185 189 0
Lower 0 0 0
Table 1. Results of positional relationship
Right 0 0 0
Other 52 55 48
In “Other” of Table , in the case of text field widgets, there were many widgets for which label widgets were located to the left and right to form a complete, an example being “Confirm new messages every minutes.” In the case of combo box widgets, there were widgets for which label widgets were located to the left and right to form a sentence, similar to text field widgets. Also, there were widgets for which radio button widgets and check box widgets were located to the left. These indicated that users first selected these radio button widgets and check box widgets, then they specified the details of the selections using combo box widgets. In this case, radio button widgets and check box widgets were considered to be instruction labels. For example, in the selection of a folder for saving download files, there were options of “Ask the saving folder for each file” and “Save all files to this folder” using radio button widgets. When the latter was selected, a folder was selected using a combo box. In the case of list box widgets, there were many widgets for which text field widgets and combo box widgets were located above, and these widgets were linked to the list box widgets. In this case, items selected in list box widgets were entered into text field widgets, and the details for the selections of combo box widgets were selected using list box widgets. For example, in selecting fonts, there was a list box widget including several font names and there was a text field widget on the list box widget. When a font name was selected from the list box widget, it was entered on the text field widget. Thus, there were many list box widgets for which instruction labels were not located nearby. When instruction widgets were located nearby, they were above list box widgets. According to the results of this survey, our present system assesses the associations of instruction labels with target widgets by considering that instruction labels are located to the left or above target widgets. To associate instruction labels with target widgets, the coordinates and sizes of widgets are used. The shape of widgets is rectangular, and the coordinates recorded in the histories of GUI operations are those of the upper left corner of the widgets. The widths and heights of widgets are also recorded in the histories. Using these coordinates and sizes, our system calculates the coordinates of the four corners of the widgets. In this section, the coordinates of the upper left and lower right corners of a label widget are given as (lab x1, lab y1) and (lab x2, lab y2), respectively. The coordinates of the upper left and lower right corners of a target widget are given as (wid x1, wid y1) and (wid x2, wid y2), respectively. Our system calculates the length (dlef t ) between the upper left corner of a target widget and the upper right corner of a label widget using expression (1). Also, our system calculates the length (dupper ) between the upper left corner of a target widget and the lower left corner of a label widget using expression (2).
dlef t = dupper =
√ √
(lab x2 − wid x1)2 + (lab y1 − wid y1)2 (1) (lab x1 − wid x1)2 + (lab y2 − wid y1)2 (2)
For a certain target widget, the label widget for which dlef t
260
is smallest, and (lab x2 − wid x1) is greater than or equal to zero is considered to be located the closest left of the target widget. Similarly, the label widget for which dupper is smallest, and (lab y2 − wid y1) is greater than or equal to zero is considered to be located the closest above the target widget. From the above calculations, our system assesses the label widgets that are located at the closest left of and the closest above each target widget. Then, our system specifies instruction labels as the label widgets that are closest left of target widgets in text field widgets and combo box widgets and the label widgets that are closest above target widgets in list box widgets. For example, “Member’s ID” in Figure 2 (2) is the label widget closest to the left of the text field widget. Therefore, the instruction label for the text field is identified as the label widget “Member’s ID” Analysis of flows of operations
Widgets are arranged in windows, and a window consists of several widgets. Considering the flows of GUI operations, users first input text and select items using widgets in a window. After these operations, the window is switched to the next window, then users operate the next window. In the histories of GUI operations, entries for a certain widget operated many times are often recorded. This is because each keystroke on a keyboard in a text field widget and each mouse click on a button widget is recorded. Therefore, unnecessary detailed operations are recorded, including the progress of text input, the correction of mistakes during input, and the modification of selections. However, these detailed operations are not described in scenarios. As described above, we assume that individual items for input, selection, and display on GUIs are described in the target scenarios in our method. Many of these items for input, selection, and display indicate each individual widget in GUIs. That is, it can be considered that the operation of a widget is described once in scenarios. Thus, if these detailed operations are counted individually, appropriate results cannot be obtained for validating the correspondences between the scenarios and the GUIs of the target software. In our method, our system first extracts the flows of window switching by analyzing the histories of GUI operations. Entries of windows becoming active are recorded. Histories from one window becoming active to the next window becoming active are the histories for the active window which includes entries for the coordinates, sizes and operations of widgets on the window. In our method, we consider these entries of windows becoming active as being separate from the histories for windows, and our system extracts the flows of window switching. The extracted flows of window switching is the order in which windows become active in the histories of GUI operations. Then, our system analyzes the histories of operations for widgets arranged in each extracted window. In this analysis of histories for widgets, even if there are several entries of operations for a widget, our system stores only one occurrence of an operation for the wid-
get. Our system does not store the number of times the operations is performed. For example, when a window becomes active, entries such as that shown in Figure 3 (3) are recorded. This entry indicates that the window in Figure 2 (2) has become active. The history from this entry to the entry indicating that the next window has become active is the the history for window (2). Also, in Figure 3 (4), the entries of text input to the text field widget “castIdText” are recorded as each character input. For these entries, our system only stores the fact that characters were entered into the text field widget. Also, in many cases, a main window is displayed after the software is started. The main window is considered to be the starting point from which users use the various functions of the software. In the main window, first, a function is selected from a menu, then the associated function is invoked. At the same time, window switching starts for this function. When the usage of the function finishes, the window switching also finishes, and the displayed windows become the same as those when the software was started. In our method, we consider that a flow of window switching is from the start of the usage of a function to when the displayed windows are the same as those when the software was started. Thus, our system extracts many different flows of window switching. We call each of these flows a “function window flow (FWF)”. That is, the windows that are displayed in each operation step are recorded in the histories of GUI operations. Therefore, our system stores the windows that are displayed when the software was started. Then, our system compares the displayed windows in each operation step with those when software was started. If the displayed windows in certain operation step and those when the software was started are the same, our system extracts the entry of the operation step as the end of the usage of the function, i.e., the end of window switching. The entry for the end of the usage of a function is also considered as the start of the usage of another function. Thus, our system constructs a flow of window switching from the start to the end of the usage of a function. Also, it is considered that the same FWFs appear a number of times in the histories of GUI operations, that is, there are cases where several FWFs involve the same window switching. In such case, these FWFs are stored as one type of FWF. That is, several types of FWFs are extracted, and these FWFs are not duplicated. For example, in Figure 2, the main window is (1). In this figure, the function of renting videos is invoked, windows are switched, and the display finally returns to window (1). The flow of window switching is in Figure 2 from (2) to (6). In this example system, window (1) is displayed upon starting the software. When windows (2)-(6) are operated, window (1) and one of windows (2)-(6) are displayed. The states in these operation steps are not the same as when the software was started. After the operation of window (6) finishes, only window (1) is displayed. This state is the same as when the software was started, that is, the usage of the function of renting videos has finished. Therefore, our system constructs a FWF as the flow of window switching (1)-(6).
261
Also, there are sometimes loops of some events in scenarios. For example, events 4-6 in Figure 1 may repeatedly occur. These events correspond to window switching (3)-(4) in Figure 2; thus, window (4) is repeatedly displayed. However, the number of loops is not fixed. Thus, it is considered that the occurrence of repeated events in a scenario is described only once. Therefore, even if a window is displayed several times in a FWF, the window displayed is counted only once in the FWF. Analyzing scenarios
Scenarios are used for describing interactions between software and users in an itemized natural language. For validating the correspondences between scenarios written in a natural language and the results of analysis in the section “Analyzing histories of GUI operations”, terms used in the events in scenarios are extracted. These extracted terms are processed as candidate label names of widgets in GUIs. Validating scenarios and GUIs
Interactions between users and software are described in scenarios. In other words, it can be considered that scenarios are similar to manuals for using software. This means that the label names of widgets in GUIs are often included in events in scenarios. In our method, our system compares the results of analysis in the section “Analyzing histories of GUI operations” with the order of appearance of label names of widgets analyzed in the section “Analyzing scenarios”. Then, our system analyzes the associations of each event in scenarios with each switched window. Identification of target scenarios
As described previously, scenarios can be classified into the basic scenario, the alternative scenario, and the exception scenario, that is, there often are several scenarios that can be used for describing interactions between users and software. Also, there are many use cases in software. That is, there are many scenarios. Also, a function is described using a use case. An interaction between users and the software is described in a scenario. A FWF is a flow of window switching during the usage of a function. Therefore, when a basic scenario, alternative scenarios, and exception scenarios are independent scenarios, a FWF is considered to be associated with a scenario. In this step, candidate scenarios associated with each FWF are identified. In our method, our system calculates the frequencies with which label names of operated widgets in FWF extracted in the section “Analysis of flows of operations” appear in each scenario using the terms extracted in scenario analysis, as described in the section “Analyzing scenarios”. This calculation is performed for all combinations of scenarios and FWFs. Scenarios that have high frequencies for each FWF are considered to be more likely to be associated with the FWF than other scenarios that have low frequencies. From this calculation, ranks for the frequencies of scenarios for each FWF are obtained.
In this calculation, the frequencies of appearance of label names in scenarios are calculated using a vector space model [10] that is widely used for information retrieval. In the model, the weight dij (i = 1, 2, ..., m, j = 1, 2, ..., n) of indexing term i in document j is defined. n is the number of documents and m is the number of indexing terms. Also, a matrix D is defined as expression (3). In addition, the weight qi of retrieving keywords in m-indexing terms is defined.
d11 d12 d21 d22 D= ... ... dm1 dm2
... d1n ... d2n ... ... ... dmn
(3)
Upon evaluating expression (4) using dij and qi , the frequencies of appearances of label names in scenarios for each FWF can be obtained. If the value of this expression is high, the frequency of appearance can be considered to be high. m ∑
√
dij qi √ m m ∑ ∑ d2ij qi2 i=1
i=1
(4)
i=1
In our method, n-documents are defined as scenarios, and retrieving keywords are defined as the label names of widgets appearing in FWFs extracted in the section “Analysis of flows of operations” . The weight qi of label name i is defined as 1. Also, dij is defined as the number of times that the label name i appears in scenario j. Then, the frequencies are calculated. The scenario with the highest frequency can be considered to be the most likely scenario to be associated with the FWF.
can be associated with windows in FWFs. In these associations, when an event represents the switching of windows in a FWF, the event is associated with these windows. Then, our system sorts events associated with these windows in the order of window switching. After this sorting, it is necessary for the order of the sorted events to be the same as that of events described in the scenario. Therefore, events that do not become the same as the order described in the scenario after sorting are identified as having no association with windows. Also, if there are windows with which no events in a scenario are associated, the scenario and the FWF are identified as having no association. In all scenarios for which the frequencies calculated in the section “Identification of target scenarios” are not 0, our system identifies the association with the FWF. Then, our system shows the results of associating scenarios with FWFs. In these results, the scenario that the number of events associated with windows in the FWF is many is considered as being more likely to be associated with the FWF. Therefore, our system shows the results in decreasing order of the number of associated events for each FWF. SYSTEM ARCHITECTURE
In our research, we propose a method for validating the correspondences between scenarios and GUIs by analyzing the histories of GUI operations of the software. The architecture of our system is shown in Figure 4. Our system validates scenarios and GUIs by the following four steps. 1. Generate operation history-recording function 2. Analyze operation histories 3. Analyze scenarios 4. Validate correspondences Generating operation history-recording function
Validation of interactions in scenarios
Scenarios that have high frequencies calculated in the previous section are those in which the frequencies of label names appearing in a FWF are high. However, it is not necessarily the case that the order of appearance of label names in events in scenarios is the same as the order of appearance of label names in the histories of GUI operations. Also, events are described in order of appearance in a scenario, however, events do not always occur exactly in the described order. In addition, the order of operations of widgets in a window can often be random. However, the order of windows displayed can be considered to be almost fixed in scenarios. In this step, our system compares the order of the occurrence of events with the order of windows displayed. In our method, our system first identifies the associations between each event in a scenario and each window in a FWF extracted from the section “Analyzing histories of GUI operations”. For this purpose, terms in scenarios extracted in the section “Analyzing scenarios” are used. Our system searches for an event in which the label name of a widget extracted from the histories appears as terms. Thus, events in scenarios
262
First, our system generates a function for recording the histories by parsing software source programs and automatically adds this function to the target software. This function records the histories of operating windows and widgets in the software. JavaCC [11] is used for parsing the source programs. The programs generated for the function of recording the histories of GUI operations are written in AspectJ [12] and are automatically added to the target software. JavaCC is a tool for parsing source programs for Java. This is a widely used tool for parsing. AspectJ is a widely used tool for realizing Aspect Oriented Programming (AOP) in Java. In AOP, programs required to be written in many classes are modularized as “aspects” separate from classes. Then, these programs modularized as aspects are added to developed classes. In AOP, source programs for classes are not modified. It is necessary to add the function for recording the histories of GUI operations to programs for the definition and generation of windows and widgets throughout the whole of the
Figure 4. System architecture
target software. The programs for defining and generating windows and widgets are written in many classes in the target software. It is difficult to develop this function as some classes. Therefore, it is appropriate to develop this function using AspectJ. Also, using AspectJ, the source programs of the target software are not modified; thus it is easy to remove this function.
In this step, to identify the associations between extracted FWFs and scenarios, our system first extracts terms from events in each scenario. Using these terms, it is possible to identify terms in each event. Then, matrix (3) in the section “Identification of target scenario”, which represents the number of times label names appear in each scenario, can be obtained. Then, the frequencies of label names in scenarios are calculated using expression (4).
Analyzing operation histories
In this step, our system analyzes the histories of GUI operations and extracts FWFs and operations for each widget. Words of a natural language often have inflected forms, and the label names of widgets include some symbols such as “:”. However, terms with the same inflected forms as label names and symbols are not necessarily used in scenarios. Our system obtains uninflected forms of terms and removes unnecessary terms such as symbols using CaboCha [13], which is a natural language parser. The series of terms resulting from this parsing are used for validating scenarios. From the analysis of the histories of GUI operations in Figure 2, the relationships between windows and widgets arranged in windows are extracted as shown in Table . The label names in this table are the parsed terms. Also, one of the FWFs is extracted as the window switching of windows in the order (1), (2), (3), (4), (5), (6). Window (1) (2) (3) (4) (5) (6)
Label name (Type of widget) rent video (menu item) member id (text field), ok (button) add rent video (button) video id for rent (text field) term of rent (combo box), ok (button) confirm (button) confirm (button)
To extract terms from events, events are parsed using CaboCha [13]. CaboCha deconstructs each sentence into chunks and tokens. A chunk is a segment and a token is a result of morphological analysis. In the morphological analysis, a term is deconstructed into words that are as detailed as possible. However, terms in events extracted by CaboCha are used for associating the label names of widgets in GUIs; thus, the terms at segment level are considered to be appropriate. Also, terms with the same inflected forms and symbols in events are not necessarily used as the label names of widgets. Thus, words in tokens are modified to uninflected forms, and unnecessary words, such as symbols, are removed from tokens in a chunk. In our system, these parsed words of tokens in a chunk are connected, and they are used as terms. The terms of the scenario “Rent videos” in Figure 1 after this extraction are shown in Table . clerk member id confirm term of rent calculate
rent video input video confirmation receive
select system add display money
member member entry id charge
Table 3. Terms of the scenario “Rent videos”
Next, our system searches for scenarios to be associated with each extracted FWF . For example, label names extracted in the section “Analyzing operation histories” in the video rental management system are shown in Table . Matrix of (3) in the section “Identification of target scenarios” is ob-
Table 2. Relationships between windows and widgets
Analyzing scenarios
263
tained by considering these label names as retrieving keywords. Using this matrix, our system calculates expression (4). In this result, the frequency is 0.75 in the scenario “Rent videos” in Figure 1 and 0.33 in the scenario “Register members”. Therefore, the scenario “Rent videos” can be considered to be more likely than “Register members” for the FWF of Figure 2. Validating correspondences
In this step, our system identifies events associated with each window. Then, the system compares the order of display of windows with the order of appearance of events. From this step, the results of validating the correspondences between scenarios and GUIs are obtained. From the result of identifying target scenarios, the scenario “Rent videos” in Figure 1 can be considered to be the most likely for the FWF in Figure 2. Thus, the scenario “Rent videos” is used as an example for validating the FWF in this section. In our method, as described above, terms extracted from the scenario are considered to be associated with the label names of each widget. That is, in our example, the system compares the label names in Table with those in Table . First, according to Table , the label name that appears in window (1) in Figure 2 is “rent video”. An event that includes this term in the scenario “Rent videos” in Figure 1 is event 1. Thus, event 1 is associated with window (1). Next, the label name that appears in window (2) is “member id”. An event that includes this term is event 2. Thus, event 2 is associated with window (2). Similarly, our system identifies associations between events (3)-(10) with windows (3)-(6). Figure 5 shows the associations of each event in the scenario “Rent videos” in Figure 1 with each window in Figure 2 obtained by the above processes.
Figure 5. Example of associations of events and windows
described in the section “Analysis of flows of operations”. “Exists” indicates the numbers of FWFs for which the corresponding scenarios are identified in the validation. “Mistakes” indicates the numbers of FWFs in “Exists” for which the most likely corresponding scenarios identified in the validation are not appropriate. “Not exists” indicates the numbers of FWFs for which no corresponding scenarios are identified in the validation, even if there are actually corresponding scenarios. Video Order
EVALUATION Evaluation of identifying scenarios
For evaluation, we used two systems, the video rental management system used as discussed in this paper and the order processing system referred to in reference [2]. We described scenarios of these systems, developed their GUIs, and obtained their histories of GUI operations during use. Then, our system validated the correspondences between the scenarios and the GUIs. We described the scenarios of the order processing system and developed their GUIs in accordance with reference [2]. However, in the GUIs of reference [2], a login window is displayed when the software starts, and the main window is displayed after logging in. In our present system, the main window is considered as the window that appears when the software starts; thus, we developed GUIs of the order processing system excluding the login window. Table shows the results. “Video” indicates the video rental management system, and “Order” indicates the order processing system. “Scenarios” indicates the numbers of all scenarios including basic, alternative and exception scenarios. “Flows” indicates the numbers of extracted FWFs as
264
Scenarios 29 59
Flows 19 9
Exists 12 8
Mistakes 1 (5) (5)
Not exists 0 0
Table 4. The results of validation
According to Table , there were “Mistakes” in the video rental management system. In one FWF, the most possibly corresponding scenario identified by our system was not appropriate. This FWF was for paying late fees in returning videos. However, the corresponding scenario to this FWF identified by our system was the scenario that no late fees were required in returning videos. The end of this FWF was the messages for late fees. In this case, the final window of this FWF includes only some label widgets for the messages and a button widget for conformation of the late fees. The operation for this window was only pushing the button widget. Also, the end of the FWF of the scenario for no late fee was the messages for successfully returning videos. The final window of this includes only some label widgets for the messages and a button widget for conformation of the late fees. The operation for this window was only pushing the button widget. The difference of these two types of FWFs was the message in the final window. However, the label
names of operate widgets in these two types of FWF are the same. Therefore, the corresponding scenario to the FWF for paying late fees was not appropriately identified. Also, in five FWFs in brackets of “Mistakes” in Table , several of the most likely corresponding scenarios were identified for each FWF. In these several scenarios, one was an appropriate scenario, but the other was not appropriate. So, the number is put in brackets. The reason is considered that the FWFs corresponding to different scenarios have the quite same window switching. For example, in searching for the entry of a member, there were several types of searching, such as searching by the name of the member and searching by the member’s ID. In this case, the types of searching was selected in the first window, keywords were entered in the second window, and then the results were displayed in the final window. The widgets on each of these windows were the same, and only the label names of the widgets were different. In our system, all types of these searching were handled as one FWF, because all types of these searching have the same window switching. Therefore, several of the most likely corresponding scenarios were identified for one FWF. However, in these five FWFs, identified several scenarios for each FWF were appropriate, if FWFs that include different label names of widgets were handled as different FWFs in having the same window switching. There were the “Mistakes” in the order processing system. Several of the most likely corresponding scenarios were identified for each FWF. Among these several scenarios, one was an appropriate scenario, but the others were not appropriate. Thus, the number is put in brackets. The reason for the existence of “Mistakes” is considered to be that the terms used in scenarios and label names of the widgets of GUIs in reference [2] were often not exactly the same. For example, the term “credit card payment information” was used in the scenario, however, the terms “credit card number” and “expiration date” were used as the label names of widgets of GUIs. However, according to Table , the most appropriate scenarios for each FWF is identified in the video rental management system. Also, in the FWFs of “Mistakes”, the appropriate scenarios are included in the most likely corresponding scenarios. That is, all appropriate scenarios are identified as the most likely corresponding scenarios. According to this result, the correspondences between the flows of operations realized in software and scenarios written in a natural language can be validated by our method. Evaluation of identifying instruction labels
We also evaluated whether the appropriate instruction labels were identified. In this evaluation, the sample systems are also the video rental management system and the order processing system. The result is shown in Table . “Widget” indicates the total number of target widgets in each system. “Valid” indicates the number of target widgets that our system appropriately identified instruction labels. “Not valid” indicates the number of target widgets that our system incorrectly identified instruction labels. “Not identified” indicates
265
the number of target widgets that our system could not identify instruction labels.
Video Order
Widget 29 34
Valid 17 28
Not valid 12 0
Not identified 0 6
Table 5. The results of idenfifying instruction labels
There were “Not valid” in the video rental management system. 8 target widgets of “Not valid” were text field widgets, and 4 target widgets are list box widgets. In all of these text field widgets, radio button widgets were located to the left. When one of the radio button widgets are selected, the corresponding text field widget could become editable. That is, the radio button widgets could be considered as instruction labels. However, in our present system, radio button widgets are not identified as instruction labels, so our system identified the closest label widgets of text field widgets as instruction labels. In all of those list box widgets, identified instruction labels were for indicating the purposes of windows. For example, one of these list box widgets was for showing results of searching for videos. Only one label widget was arranged the window in which the list box widget was arranged, and its label name was “This is the result for searching for videos”. We considered that these label widgets were for indicating the purposes of the window and were not the instruction label for the list box widget, because we assume that instruction labels indicate only the name of items for input, selection, and display. Therefore, the instruction labels for these list box widgets should not have been identified. Also, there were “Not identified” in the order processing system. These 6 target widgets were all table widgets. Our system identifies instruction labels as label widgets the closest above table widgets. However, there were no label widgets above these table widgets. In our present system, this result was considered to be appropriate. According to Table , 59% of instruction labels in video rental management system were identified appropriately. Also, 100% of instruction labels in order processing system were identified appropriately, because all of target widgets that instruction labels should be identified were associated with appropriate instruction labels, and all of target widgets that did not have instruction labels did not associated with instruction labels. Therefore, it is necessary to improve so that our system can identify widgets except for label widgets as instruction labels, however, the method itself is effective to identify instruction labels. RELATED WORKS
Several methods for identifying the correspondences between models and software has been proposed. Zhao et al. proposed a method for identifying the requirements and functions realized in software [14]. In their method, they assume that meaningful names are used as the identi-
fiers of functions and variables defined in the source programs. Indexing terms are extracted from the target of requirements documentation which is written in a natural language, and are associated with identifiers in source programs using information retrieval technology. In this method, correspondences between the functions realized in the software and the requirements are found. However, correspondences between the flows of operations described in scenarios and those realized in the software are not validated. Liang and Xu proposed a method for validating software developed along with scenarios [15]. In this method, scenarios are described using extended sequence diagrams, and a system that monitors the execution of the software is generated using the sequence diagrams. Correspondences between the scenarios and software are validated using the generated function. In this method, the logic of the software can be validated; however, the validation of correspondences between the flows of operations described in scenarios and those realized in the software is not carried out. Dwyer et al. proposed a method for validating the behaviors of software using models of GUIs [16]. In this method, the states of GUI components and models of event handling are described, and the source programs of the software are analyzed. Thus, correspondences between the models and software are validated. However, the models used for this method require specific descriptions; thus, so scenarios written in a natural language cannot be used. G. Little et al. proposed a method for recording histories of interactions on the web as pseudo-natural language text [17]. These histories are interpretable and executable. This method are used for capturing, sharing, automating and personalizing business processes on the web. However, in a sense, this methods are for generating scenarios from histories, and the validation of correspondences between the flows of operations described in scenarios and those realized in the software is not carried out. CONCLUSION
In this paper, we proposed a method for validating the correspondences between the scenarios and flows of operations of GUIs. In our method, scenarios are parsed and deconstructed into terms. Also, the histories of GUI operations are analyzed and the label names of widgets are extracted. Then, the validation is performed by associating the label names of widgets with the terms in scenarios. Our future planned works are as follows. • Improvement of the extraction of FWFs from the histories of GUI operations when the main window is not displayed when software is started. • Validation and improvement of the calculation of the associations of instruction labels with target widgets. • Improvement of the accuracy of validation results using a thesaurus and ontology.
266
REFERENCES
1. X. Ferre, N. Juristo, H. Windl and L. Constantine: “Usability Basics for Software Developers”, IEEE Software Vol. 18, No. 19 (2001). 2. G. Schneider and J. P. Winters: “Applying Use Cases: A Practical Guide”, Addison-Wesley Pub. Co. (1998). 3. J. M. Carroll: “Making Use: Scenario-Based Design of Human-Computer Interactions”,MIT Press (2000). 4. T. Weilkiens and B. Oestereich: “UML 2 Certification Guide: Fundamental & Intermediate Exams”, Morgan Kaufmann Pub (2006). 5. Microsoft Outlook Express: http://www.microsoft.com/windows/ie/ie6/using/ oe/default.mspx 6. Thunderbird: http://www.mozilla.com/thunderbird/ 7. Microsoft Internet Explorer: http://www.microsoft.com/windows/products/ winfamily/ie/default.mspx 8. Firefox: http://www.mozilla.com/firefox/ 9. Microsoft Windows: http://www.microsoft.com/windows/default.mspx 10. G. Salton, A. Wong and C. S. Yang: “A Vector Space Model for Automatic Indexing”, Communications of the ACM Vol. 18, No. 11 (1975). 11. JavaCC: https://javacc.dev.java.net/ 12. J. D. Gradecki and N. Lesiecki: “Mastering Aspectj: Aspect-Oriented Programming in Java”, John Wiley & Sons Inc. (2003). 13. “CaboCha: Yet Another Japanese Dependency Structure Analyzer”: http://chasen.org/ taku/software/cabocha/ 14. W. Zhao, L. Zhang, Y. Liu, J. Luo and J. Sun: “Understanding how the requirements are implemented in source code”, Proc. of the 10th Asia-Pacific Software Engineering Conference (APSEC’03) (2003). 15. D. Liang and K. Xu: “Testing Scenario Implementation with Behavior Contracts”, Proc. of the 30th Annual International Computer Software and Applications Conference (COMPSAC’06) (2006). 16. M. B. Dwyer, Robby, O. Tkachuk and W. Visser: “Analyzing Interaction Orderings with Model Checking”, Proc. of the 19th International Conference on Automated Software Engineering (ASE’04) (2004). 17. G. Little, T. A. Lau, A. Cypher, J. Lin, E. M. Haber and E. Kandogan: “Koala: Capture, Share, Automate, Personalize Business Processes on the Web”, Proc. of the Computer/Human Interaction 2007 Conference (CHI2007) (2007)
