CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
EdgeWrite with Integrated Corner Sequence Help Benoît Martin LITA, Universite Paul Verlaine - Metz Ile du Saulcy, 57045 METZ CEDEX 1, France
[email protected]
Poika Isokoski Department of Computer Sciences / TAUCHI 33014, University of Tampere, Finland
[email protected] However, there are aspects of EdgeWrite use that have not been investigated. Methods of introducing the EdgeWrite characters to the users are an area that has received little attention. In all experiments that we are aware of, the characters have been presented to the users in a character chart that lists the Latin alphabet together with their EdgeWrite representations.
ABSTRACT
We describe a system that informs the users of the shape of the EdgeWrite characters within the visual feedback area of EdgeWrite. We compared two versions (static and dynamic) of this design to a printed character chart in a five-session text entry experiment with three 8-participant groups. The participants were able to use EdgeWrite with the integrated help systems. There were no statistically significant differences in text entry rate between the group using the character chart and the two groups using the integrated help. However, the group with the dynamic help was faster than the group with the static help while maintaining a low corrected error rate.
The character chart is an effective way to introduce the characters. Line sequences that form shapes are easy to remember. An alphabetical order in the character charts makes finding characters fast. Consequently, character charts are difficult to beat in efficiency. Unfortunately, character charts are not a part of the text entry user interface. If they are introduced into the user interface, they can occupy a large part of the display area. Alternative means of presentation are problematic as well. Hand-held charts can get lost and a chart printed on the back of the device necessitates turning the device to see it.
Author Keywords
Text entry, EdgeWrite, character chart, visualization ACM Classification Keywords
H5.2. [Information interfaces and presentation]: User Interfaces --- Input devices and strategies, Graphical User Interfaces. General Terms: Human Factors, Experimentation, Performance.
Other text entry methods such as soft keyboards and Quikwriting [3] display the characters directly on the user interface. Our goal was to do the same in EdgeWrite without hurting user performance. We implemented two versions of an integrated help system and evaluated them in a five-session text entry experiment.
INTRODUCTION
Many recent text entry method proposals have been motivated by mobile text messaging. Others have alleviated the difficulties that some people with disabilities have in text entry. EdgeWrite, proposed by Wobbrock et al. [5], has been studied in both contexts.
In the following we will describe EdgeWrite, the design and implementation of the integrated help, the experiment, and its results followed by our conclusions.
Originally the EdgeWrite system used a rectangular input area where the user moved a stylus through a sequence of corners to enter a character. The edges of the input area do not generate input in EdgeWrite, but they can be leaned on when traveling from one corner to another. EdgeWrite has shown promise for being useful for people with disabilities that affect their motor capabilities [6, 10].
JOYSTICK EDGEWRITE
EdgeWrite characters consist of sequences of five different tokens: four corners and a segmentation token. Because of this they can be entered with many different input devices. Here we will describe a version of EdgeWrite intended for joystick use. The source code for the EdgeWrite recognizer and the default feedback display were received from Jacob Wobbrock. We modified only the visual feedback; no changes to the recognizer were made. Our joystick interface, replicated the published characteristics of Wobbrock’s implementation [7].
The properties of EdgeWrite are well known thanks to a large body of work on it [4, 5, 6, 7, 8, 9, 10, 11, 12]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2008, April 5–10, 2008, Florence, Italy. Copyright 2008 ACM 978-1-60558-011-1/08/04…$5.00.
In gamepad use the corners of EdgeWrite area are mapped to the corners of the stick movement area. When the stick stays in the center for a 100 milliseconds, the system generates a segmentation token and tries to recognize the current corner sequence. In our experiment the required center dwell was about 120ms in practice due to interaction
583
CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
of processing delays and the 35ms polling interval used for reading the joystick data.
The dashed curve in Figure 3 shows the path that the stick might take to enter an “a”. In comparison to Figure 1 we can see that in joystick use the characters have extra segments in the beginning and in the end to connect the starting and ending points to the center. The square in the center of Figure 3 is the center area within which the stick had to stay for 120 ms to end a character. The triangular areas in the corners are the areas within which the stick had to visit to add a corner into the corner sequence. When corners 1 and 4 were hit, the neighboring corner triangles (2 and 8) shrank to make hitting them less likely when crossing the input area diagonally. This technique has been found beneficial for right handed users by Wobbrock et al. [7]. In left-handed use the shrinking corners should presumably be shifted to the other diagonal.
The character set that we used was the one included in the EdgeWrite recognizer. The first four alphabets are shown in Figure 1. The relation of the character shape and the EdgeWrite input area together with the corner coding is shown around “A”. For a more complete character chart, please refer to Appendix A. 1 8
2
A 824
4
B 1848
C 2184
D 2484
Figure 1. Four EdgeWrite characters and their corner sequences. The dots mark the starting points. In addition to the characters shown in Appendix A, the EdgeWrite recognizer recognizes alternative shapes for many characters. Upon non-recognition the recognizer recursively drops the first corner from the sequence and recognizes the remaining sequence. These tricks remain hidden from the users, but improve the recognition rate. The Logitech RumblePad 2 gamepad that was used in our experiment is shown in Figure 2. The rectangular mounting holes for the sticks are important for EdgeWrite use. With round holes the user would not feel the corners. Figure 3. The feedback display of joystick EdgeWrite and a curve showing the stick movement for an “a”. THE HELP SYSTEM
Ideally EdgeWrite users would know the corner sequences and would not need visual feedback on their stick movements. New users, however, do not know the corner sequences. Conventionally help on the corner sequences has been offered as character charts such as Appendix A. The EdgeWrite system that is available for download [12] includes character charts that can be displayed using a “Help” menu. Our goal was to integrate the information on the character chart into the feedback display of EdgeWrite. This could be done in many ways: a miniature character chart could be shown superimposed on the feedback window, there could be a query mechanism that would allow the user to browse a larger character chart through the small window, instead of the character chart the corner sequences could be displayed in textual form, or the corner sequences could be revealed one corner at a time. We chose the last alternative.
Figure 2. Logitech RumblePad 2 gamepad. The state of the EdgeWrite system is shown on the display. Figure 3 shows the feedback window of our joystick EdgeWrite implementation. The square area mapped directly to the joystick position. After hitting the first corner, the stick movement was shown as an orange line (not shown in Figure 3) that connected the recorded stick positions. The stick track was erased when the character was completed.
The Static Help
In its initial state our system displayed each character in the first corner of its sequence as shown in Figure 4. The
584
CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
characters were presented in rows of similar characters. When possible, the rows had an internal order such as alphabetical or numerical order. The background color separated character types to make visual search easier.
display the same information in the integrated help and character chart and avoid complications due to character chart design. Therefore, we included only one corner sequence per character in both forms of help. All sequences remained functional in the recognizer. The issues with multiple shapes per character were left for further work. An Example
As an example, on the operation of the integrated help we will walk through the entry of an “i”. In Figure 4 we can see that “i” is shown in the upper left corner. Therefore, to enter “i”, the user needs to move the stick to the upper left corner. As seen in the leftmost screenshot in Figure 5, the display has changed when the stick arrived in the upper left corner. The upper left corner is empty and the characters previously displayed there are now divided among the other corners. Each character has moved to the next corner in its sequence. Because only characters whose sequence starts from the upper left corner are shown, the number of shown characters has decreased significantly. We can see that “i” is now in the lower left corner. Therefore, the user will next move the stick there.
Figure 4. The initial state of the integrated character help. As seen in Figure 4, there were two kinds of character displays. The alphabets and numerals were shown in the main feedback area, and sets of icons for commands and special characters, such as those shown in Table 1, were shown on the left and right edges of the display. The reason for two kinds of visualizations was that we wanted to separate normal characters from commands. In addition space, the most frequent character, is learned so fast that including it among the alphabet was not considered necessary. The reason for using only the left and right edges for the icons was that an optional word completion system [9] needs the top and bottom for its operation. The word completion system was not used in our experiment, but did not want our design to be incompatible with it.
As seen in the middle screenshot of Figure 5, the characters previously in the lower left corner have now moved to the other corners except for “i” that is now above the central rectangle. The position of “i” now signals that the corner sequence for “i” is complete and that “i” can be entered by moving the stick to the center. In addition to the lower case “i” in the center, we can see that an upper case “I” has appeared in the upper left corner. This happens with all lower case alphabets. The upper left corner at the end of a corner sequence is reserved for upper case characters. As seen in the rightmost screenshot in Figure 5, centering the stick erases the orange stick track from the display. The “i” remains to be displayed above the central rectangle as a reminder of the last entered character. The character help display in the corners has returned to the initial state except that accented versions of “i” have appeared in some corners. The background color for the accented characters is different to make noticing their appearance easier. If the user wishes to replace the newly entered “i” with one of the accented versions, it can be done just like entering any other character by following the desired character from corner to corner.
Backspace Home Enter End Left arrow Alphanumeric mode Right arrow Punctuation mode Space Extended mode Tabulator Table 1. The most frequent sidebar icons.
The EdgeWrite mechanism of entering the character first and then replacing it with the accented version afterwards is a happy coincidence for the help design. In many desktop keyboard character layouts accented characters are composed by first entering the accent and then the character. This order would have been more challenging since it would have required the display of all accents in the initial state of the help making it more difficult to read.
It is possible to show in the integrated help only one shape for each character or all shapes that the recognizer knows. For example the help can display only the sequence 824 for “a” although the recognizer recognized sequences 814, 8248, 8148, and 218424 as well (see Figure 1 for corner numbering). Adding all alternative shapes would have increased the number of characters in the initial integrated help display from 37 to 58 characters (57% increase). Adding all alternative shapes would have increased the character chart for the alphabet and numerals from 37 to 138 EdgeWrite characters (273% increase). We wanted to
The help displays in Figures 4 and 5 show only Alphabetical character mode including accents, alphabets and numerals. Other characters were entered through two
585
CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
Figure 5. Entering “i” with the static integrated help. additional modes called Punctuation and Extended character mode. The mode changes were represented by the two bottom icons in the icon strip in the lower right corner (see Table 1). For our prototype we amended the mode changing mechanism of EdgeWrite to include direct links between all three modes. The initial displays in punctuation and extended modes are shown in Figure 6.
following corner, the characters of the selected corner traveled from their old location to the new. The speed at which the characters travel from corner to corner must, of course, be decided when implementing the animation. We tried many alternatives and found that it is important that the characters do not travel too fast. The speed of the animation does not limit the speed of text entry because the users do not need to wait for the animation to complete before moving the stick. It is enough to see the direction to which the character starts its movement. It was important to minimize the likelihood that the characters occlude each other during the start of the animation. Because of this it was better to have the characters travel at a constant speed. The alternative approach of making the characters travel a constant portion of the distance (1% for example) per frame leads to a higher probability of occlusions.
Figure 6. Punctuation and Extended modes.
We ended up with the speed of three pixels per frame. Frames were rendered at 8 ms intervals. The display hardware updated the display every 17 ms. Therefore, the participants saw characters moving on average in 6 pixel jumps. The time needed for a character to travel one edge of the EdgeWrite square was about 725 ms.
The Dynamic Help
One of our main concerns regarding the character help system was whether the visual search involved in using the help system would slow the users down significantly. With a character chart the user can acquire the corner sequence with one consultation of the chart whereas the help system forces a visual search of all corners potentially many times during the entry of a character. We found no way to remove the piece-wise revealing of the characters in the system, but we thought that it might be possible to make the visual search less demanding by giving direction hints as the user arrives in a corner. We did this by animating the movement of the characters to the next corners.
Anticipated Effects of the Integrated Character Help
We expected that with the integrated character help users would be able to use EdgeWrite without a character chart or other outside source of for the corner sequences. We also expected that the time for completing a character would be distributed differently when using the character chart and the integrated help. With the character chart there should be a pause in joystick activity before starting an unknown character because of the chart consultation. With the integrated help the pause between characters might be shorter because the users would not need to shift their attention to a separate character chart. However, there might be a pause in each corner when the user is searching the other corners for the desired character.
The animation is illustrated in Figure 7. The initial state of the display was the same as with the static help. When the stick moved to the first corner, the characters in the three other corners disappeared. The characters in the selected corner appeared in the next corner in their sequence just like in the static version. In addition to appearing in the
586
CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
Figure 7. Illustration of the dynamic help. The new positions appear as in the static version, but in addition the characters travel from the old position to the new. The trajectory of “i” is shown with the black dashed arrow. Regarding the dynamic help we expected that experienced users would be looking at the character that they want to enter and the direction to which the character starts to travel would tell them which corner to aim for next. In order to see the same hint in the next corner the user has to find the character there. Finding it in one corner should take less time than searching all three possible corners. Thus, when using the dynamic help the pause preceding an unknown character might be shorter than it is with the character chart, and the pauses in the corners might be shorter than they are with the static help.
beginners, we did not know which of the mentioned character sets is included in the EdgeWrite recognizer. We suspected a combination of the two with some other improvements as well. Therefore, to quantify the guessability of the character set to be used in our experiment we collected our own guessability data. The data were collected with a simple pen and paper method where the picture of the EdgeWrite input area was printed for each character. The participant was given textual instructions that explained the overall constraints of EdgeWrite input (i.e. only corners count) and asked to draw the path that they thought would be used for each character. The participants were told to try to emulate the shape of the Latin alphabet as well as they could under the EdgeWrite constraints. The participants were not required to consider the whole character set, so drawing the same sequence for two different characters was permitted.
What we did not know before the experiment was how these factors balance out in a writing situation. We were hoping that the help would not slow the users down too much, but we were not sure. Overall we expected user performance with the different help systems to converge with training. An experienced EdgeWrite user does not need help to remember the corner sequences. In fact we would expect that the whole visual feedback window would be unnecessary for sufficiently skilled users. When the visual feedback is not used, it should not have an effect on user performance. Therefore, experts should have the same performance regardless of the method used to learn the alphabet. There might, however, be help-related differences in how long it takes to develop this kind of skill.
The results are shown in Figure 8. The “replies” bars show the number of corner sequences that we were able to decipher from the participants’ drawings. In total we had 56 participants, but as seen in Figure 8 some of them failed to provide drawings for some characters, or provided drawings that despite the instructions broke some of the EdgeWrite rules (e.g. failed to visit corners). The “whole charset” bars show the number of sequences that the default EdgeWrite recognizer would have recognized correctly. The “reduced charset” bars show the number of drawings that match the sequences in Appendix A.
IS HELP NEEDED?
Before running costly experiments we needed to verify that there is a need for character help for new EdgeWrite users. In other words we needed to know if there are characters with corner sequences that are difficult to guess without a character chart or some other form of help.
Our data includes the characters that Wobbrock et al [8] used in their guessability experiment and some other characters. The guessability score in our data (i.e. percentage of correct sequences) was 41.36%. This is not directly comparable with Wobbrock et al. because of the additional characters. However, a comparable figure including only the alphabet and numerals can be computed. It was 46.31%. This is reasonably close to the 51% figure reported by Wobbrock et al. A possible reason for the lower score is that we did not require the participants to resolve conflicts between characters. For example many
Wobbrock et al. [8] have published data on the guessability of EdgeWrite characters - guessability meaning the ability of people to produce the corner sequences without being shown what they are. Wobbrock et al. give two guessability numbers for EdgeWrite: 51% for the “original” and 80% for the improved “user designed” character set. Although both numbers suggest that some help is needed for
587
CHI 2008 Proceedings · Post-QWERTY QWERTY Replies
60
April 5-10, 2008 · Florence, Italy
Whole charset
Reduced charset
50 40 30 20 10 0
a b c d e f g h i
j
k l m n o p q r s
t u v w x y z ç œ sp bk ,
.
;
: 1 2 3 4 5 6 7 8 9 0 + -
*
/ % & = €
Figure 8. The correct guesses per character (sp = space, bk = backspace). of Metz. Due to the sex distribution of the student population only one of the participants was female. One participant was left handed. None of the participants had previous experience with EdgeWrite.
participants entered the same sequence for “S” and “5”. The guessability of the “reduced charset” was 31%. Overall, we concluded that beginners need help with EdgeWrite corner sequences and that the help is especially important for non-alphabet characters.
Procedure
Each participant participated in five sessions. In the beginning of the first session they were given a leaflet with a 4.5 page description of the EdgeWrite system. The description of the static integrated help was one page shorter than the description of the dynamic integrated help. The leaflet also included the instructions for completing the transcription task. In addition to the written instructions the participants’ questions were answered.
EXPERIMENT: PAPER HELP VS. INTEGRATED HELP
To investigate the effects of the integrated help system we collected user performance data on EdgeWrite use with three different ways of presenting the corner sequences: the character chart in Appendix A shown on an A4 paper (paper help), with the static integrated help (static help), and with the animated integrated help (dynamic help). In this section we describe this experiment in detail.
The only complete corner sequence included in the instructions was that of “i” which was used as an example. In principle it was possible to memorize the first corner of all characters and the two first corners of the characters starting form the upper left corner based on the screenshots that illustrated the integrated help for “i”. However, we suspect that the participants were not motivated or capable of memorizing these sequences in the limited time that they spent with the EdgeWrite leaflet.
Design
We needed to record data on the first use of EdgeWrite. The first exposure contaminated the users so that it was impossible to subsequently record and compare their first use with another kind of character help. Therefore, a between groups design was chosen. To see how EdgeWrite writing skill develops over the first hour of usage, we needed to record the user’s performance in several sessions making the inclusion of large groups of participants difficult. Considering our resources, we ended up with eight participants per group each completing five 15-minute sessions of transcription.
After the participants had finished reading the instructions, they began the first 15-minute transcription task. The phrases to transcribe were chosen randomly among 500 phrases. These phrases were a French translation phrase set by Soukoreff and MacKenzie [4]. Unlike the original set, the translation included upper case characters and punctuation where grammatically appropriate. As usual with sets of this size, the character frequency correlation to “normal” usage of the language was high. The correlation between the character frequencies in the translated phrase set and the frequencies reported in Wikipedia for the French language was 0.98. We obtained similar figures with other sources of character frequency data.
The independent variables in the experiment were the type of help (paper, static, dynamic) and the amount of training (session number). The dependent variables were measures of user performance. These were text entry rate (words per minute), user effort (keystrokes per character), error rate (minimum string distance between the presented and transcribed phrases), and the time spent per character divided to preparation time and entry time. Participants
Participants were instructed to transcribe the phrases as fast as possible while correcting those errors that they noticed. Correcting errors was allowed only using the backspace,
24 participants between 20 and 26 years of age (M=23) were recruited from the students and staff of the University
588
CHI 2008 Proceedings · Post-QWERTY QWERTY
April 5-10, 2008 · Florence, Italy
but some participants used the cursor movement commands anyway – sometimes by accident. These accidents may have necessitated more cursor movement to get back to the end of the phrase. Unfortunately cursor movements are not handled by the input stream analysis algorithm of Wobbrock and Myers [11]. Consequently, we report the error rates results using the Minimum String Distance (MSD) and Keystrokes Per Character (KSPC) metrics described by MacKenzie and Soukoreff [2].
presentation mode: paper, static, or dynamic) to test for differences in sessions 1-4 with bonferroni-corrected posthoc tests. Because of the different phrase set in session 5, comparisons between sessions 1-4 and 5 were not interesting. Therefore, separate bonferroni-corrected t-tests were used to test for differences between help presentation modes in session 5. Text entry rate
The results for text entry rate are summarized in Figure 9. The text entry rate was computed using the transcribed phases except for the first character of each phrase and the “enter” at the end of the phrase which were excluded. Time spent on corrections was included. One word per minute equals 5 characters including spaces.
After completing a phrase the participants had to enter the character corresponding to the “enter” key to see the next phrase to transcribe. The phrase presentation software did not present the next phrase until the length of the transcribed phrase was long enough. This prevented accidental entry of the “enter” from causing a large number of errors due to missing characters in the transcribed phrase.
7 Words per minute (WPM)
For the fifth session we changed the phrase set to one with 62 phrases that required extensive use of the punctuation and extended modes. This set included phrases like “æ = ligature de a et e” and “"C:\\Temp\\Edge Write"”. The purpose of the fifth session was to force the participants back to intensive use of the help after they had learned the basic usage of EdgeWrite. Apparatus
The modified EdgeWrite described above was used in the experiment. The size of the Edgewrite box was 242 pixels plus the 30 pixel sidebars. A 14 pt. font was used for the help characters. The system had right and left stick usage modes for right and left handed users. The characters were not mirrored for left handed users. A detailed log of the stick movements, corner activations, and the resulting characters was saved for later analysis.
6 5 4 3 2
paper static dynamic
1 0 1
2
3 Session
4
5
Figure 9. Average text entry rate over the 5 sessions. Besides the obvious main effect for session (F3,63=326, p
