Whether Moving or Not: Modeling and Predicting Error Rates in Pointing Regardless of Target Motion Eunju Park, Hyunju Kim, Injung Lee, Byungjoo Lee* Graduate School of Culture Technology, KAIST
[email protected]
arXiv:1806.02973v1 [cs.HC] 8 Jun 2018
ABSTRACT
Understanding the mechanism by which a user’s error rate changes in point-and-click tasks is essential to enhance the usability of various interactions such as typing, gaming, and playing digital instruments. The most well-known explanation for this problem is the inverse relationship between the movement time and the pointing error rate, called the speed– accuracy trade–off. However, recent studies have shown that movement time is not an optimal indicator to explain the change in user’s pointing error rate. They also have limitations that can only predict error rates for fixed targets. Based on perceptual control theory and the recent model of moving target selection, this study presents a novel model that accurately predicts pointing error rates regardless of target motion. In two user studies on fixed and moving targets, the model explains the observed error rate with a high coefficient of determination. This allows various interactions that were not previously possible.
INTRODUCTION
Figure 1. Overview of the idea: This study proposes a novel model that accurately predicts pointing error rates regardless of target motion. The model analyzes the pointing trajectory and assumes the last submovement as the twin of the already known moving target selection. More specifically, the error rate is predicted from four measurements: the size of the target (W ), the velocity of the cursor (vc ) and target (vt ) during the last submovement, and the time spent in the last submovement (tc ). In both user studies performed on each of the fixed and moving targets, the error rate predicted by our model was highly correlated with the experimentally measured error rate (R2 = 0.98 and R2 = 0.93).
Point-and-click is still an important and fundamental task in human-computer interaction (HCI) today. In the task, users are given a target of size W that is a distance D away from their end effector. The user’s goal is to move the cursor into the target (i.e., pointing), and, when the cursor is positioned within the target, to activate a button to generate a discrete event (i.e., clicking). In this process, user performance has been measured mainly as two variables: movement time (MT ) and error rate (ER). Movement time is the time from when the user perceives the target to when the click event occurs, and the error rate is the rate at which the user failed to click inside the target. Roughly speaking, the movement time can be regarded as the performance of the pointing process and the error rate as the performance of the clicking process.
In point-and-click tasks, it has been known that movement time and error rate are in fact correlated. More specifically, the shorter the movement time, the greater the error rate. This phenomenon is called speed–accuracy trade–off and attracted many researchers’ interest in cognitive psychology and HCI. If we can fully explain the mechanism underlying the correlation of those two variables, we can estimate the error rate only by observing the movement time of individual clicks without having to observe the statistics of multiple clicks. This can be widely used, for example, by creating smart buttons that deny activation at clicks above a certain potential error rate (i.e., prevention of error), or developing optimal difficulty in games that need to shoot moving enemies (i.e., design of error).
ACM ISBN .
However, no one has yet clearly identified the mechanism by which the user’s error rate changes in point-and-click tasks. In particular, Fitts’ law, the most widely known model for the task, describes how the size of the target and the distance to the target are related to the movement time, but is silent about the error rate change. Instead, it assumes a limited case where a pointing error rate of about 4% occurs. However, in real-world situations, not in the laboratory, the time pressure given to the
Author Keywords
Pointing error rate; temporal pointing; moving target selection; perceptual control theory; smart buttons.
DOI:
user can vary greatly, and error rates greater or less than what Fitts’ law assumes can occur frequently. While the concept of effective width [50] compensates the limit of Fitts’ law to some extent, it is still criticized that the law cannot deal with speed–accuracy trade–off as a whole. Also, in today’s emerging tangible interfaces, error rates often increase in the process of replacing familiar physical interfaces with virtual interfaces. For example, touch screen or virtual reality (VR) buttons have a much higher pointing error rate than conventional physical buttons [38, 35, 33, 43], but the lack of a model that can explain the mechanism makes it difficult to improve their usability. Previous studies have been approaching the point-and-click issue from an information theoretical point of view and the mechanism of varying error rate could not be clarified. From the viewpoint of information theory, human cognition and behavior are regarded as an information processing black box, and researchers try to explain the relationship between given stimulus (target W and D) and user ’s behavior (MT and ER) through a descriptive model. However, the assumption that the user’s point-and-click behavior is entirely predetermined from a given stimulus seems to be an oversimplification of the phenomenon. In fact, the user actively controls the movement of the cursor through multiple submovements in the middle of pointing. Furthermore, even if the same stimulus is given, users can control how much error rate is generated in the click process through their own will [55]. This study contributes to both theory and practice by presenting a novel model that accurately predicts the user’s error rate in point-and-click tasks. Through perceptual control theory [45] and a recently published model of moving target selection [36], this study considers a user performing point-and-click tasks as an intermittent controller rather than an information processor. Specifically, we verify that the error rate of the user is strongly determined from the features of the last submovement of the pointing trajectory, and that the process from the last submovement to the click can be regarded as the twin of the moving target selection task. The parameters of the models presented in this study have distinct cognitive meanings, unlike those of Fitts’ law, and thus can provide more implications to designers. In the following chapters, previous studies on speed–accuracy trade–off and the efforts to obtain a model of pointing error rates are introduced. Then it presents the perceptual control theory and the moving target selection model, which are the building blocks of the model proposed in this study, and explain the process in which our model is derived from them. Finally, our model is verified by two user studies. The result showed that our model is possible to account for the pointing error rate with high coefficient of determination for both fixed (R2 = 0.98) and moving targets (R2 = 0.93). Prediction of the pointing error rate, especially for the moving target, was previously impossible and is one of the main contributions of this study.
RELATED WORK
This section reviews the theories and models related to user performance in point-and-click tasks. Previous studies conducted in the information theoretical perspective are introduced first and recent studies are introduced to complement their limitations. This summary shows how our abstract understanding of the point-and-click task extends gradually to lower-level mechanisms. Speed–Accuracy Trade–Off
When a user performs a point-and-click task, the shorter the given time, the more click errors are generated. This phenomenon is called speed–accuracy trade–off. This occurs not only in pointing but also in the choice reaction time test at which decisions must be made within time constraints [51]. Conventional theories including Fitts’ law suggest that this phenomenon is caused by the limitation of channel capacity in human information processing. That is, a smaller or more distant target would require the user to process more information with the same channel capacity [40], resulting in a longer movement time or a higher error rate. However, studies in HCI have not taken a balanced view of the trade-off between movement time and error rate [26]. Usually, they were interested in the effect of a change in task difficulty on movement time, assuming that a low pointing error rate of about 4% is maintained. The movement time, in this case, is well explained by Fitts’ law [20], which is also introduced as the main model for analyzing point-and-click tasks in the ISO standard [31]. However, it should be noted that Fitts’ law is only dealing with a subset of the speed–accuracy trade–off [26]. This contrasts with the fact that many researchers [46, 44] in psychology have actively studied ballistic pointing with a high error rate with short movement time. The Schmidt’s law [46] predicts the standard deviation of the click point distribution in such cases. In this context, some recent studies attempted to build models that collectively describe the various facets of the speed– accuracy trade–off in point-and-click tasks [23, 26, 24, 25]. Although still adhering to the information theoretical and descriptive view of the phenomenon, these studies can explain Fitts and Schmidt’s paradigm as a single continuous model. However, their research has several limitations, such as requiring statistics from multiple trials or analyzing only the extreme subsets of all data points. Also, their model cannot account for the user’s pointing performance on moving targets. Point-and-Click Error Rates for Fixed Targets
The absence of a comprehensive model of speed–accuracy trade–off makes it difficult to build models that predict error rates in point-and-click tasks. However, by establishing effective assumptions, researchers were able to obtain models that could account for changes in error rates. In general, a pointing trajectory can be divided into several submovements [52] through analysis of its speed profile. Schmidt and his colleagues tried to estimate the error rate from the probability that the end point of each submovement is located within the target, assuming that the user’s motion is divided
into two ballistic submovements [41]. The model is innovative in that it uses the concept of submovement but it is overlooked that the number of actual submovements is often more than two [37] and that the frequency secondary submovement may be changed when the given time pressure is changed. The frequency of submovement is a statistical value derived from multiple clicks, making it difficult to apply to real-time applications. Our model borrows Schmidt’s basic idea of submovement, but, it allows error rate estimation for each individual click. Based on Fitts’ law, Wobbrock and his colleagues [53, 54] derived an effective model that explains the error rate of users in point-and-click tasks when time pressure varies. Their model is derived from the limited facet of the speed–accuracy trade–off, but it has the advantage that the error rate estimation is simple compared to other existing models: ( ) i MTe −a 1 h √ 2.066 ·W (2 b − 1) ER = 1 − er f (1) D 2 Where a and b are empirical parameters inherited from Fitts’ law, D is the target distance, and W is the width or size of the target. MTe is the movement time actually taken in pointing and is an independent variable of the model. The model describes the user’s error rate successfully for both onedimensional [53] and two-dimensional [54] point-and-click tasks, but essentially cannot hold in situations where Fitts’ law is violated [14, 1, 13, 22, 48, 55]. In particular, the disproportionate effect of the target width [44] shows that there is a problem with the versatility of the model. Furthermore, movement time MTe , an important independent parameter of the model, cannot be measured without knowing the beginning of the point-and-click movement [12]. This limits the practical application of the model. There are other significant studies on the pointing error rate [44, 11], but no effective model for prediction has been proposed except from the above-mentioned Wobbrock’s study. Therefore, the predictive performance of Wobbrock’s model will be considered as the baseline of this study. However, Wobbrock’s model is silent about the pointing error rate for a moving target, unlike our proposed model.
[47]. Those models are mostly simple control laws that simulate user behavior but do not predict error rates as explicit expressions. The fact that spatial and temporal requirements are given to the user at the same time makes it difficult to construct a pointing model for a moving target. A recent study, on the other hand, presented an error rate model for pointing tasks given only temporal requirements [38, 36]. In that task, called a moving target selection, users must generate a click event when a moving target reaches a selection region of a certain size. There, the user can generate a click event with negligible movement but must perceive (1) when the target will reach the selection region and (2) how long it will stay in the selection region. Each of these perceptions is called temporal distance (Dt ) and temporal width (Wt ), and consequently the error rate (ER) of the user can be expressed as: (1 − cµ ) Wt cµ 1 W √ · ) + er f ( √ · t ) ER = 1 − er f ( 2 cσ 2 Dt cσ 2 Dt (2) 1 +δ) where, Dt = ( νt e c −1 As shown in the above equation, the error rate of the user increases as the temporal distance Dt of the target becomes longer or the temporal width Wt becomes shorter. This is similar to the Fitts’ task, where the movement time increases as the distance to the target increases or the size of the target decreases. tc is the cue viewing time at which the user was able to observe the movement of the target towards the selection region. The model also has four empirical parameters: cµ , cσ , ν, and δ . Unlike Fitts’ law, which is a descriptive model, these parameters have definite cognitive meaning because it was derived from cue integration theory [18]. The meaning of each parameter will be explained later in the model part. In their user studies, the model described the error rate as a high coefficient of determination (R2 =0.81 to 0.96). Our model extends Lee’s moving target selection model to a common point-and-click task.
Point-and-Click Error Rates for Moving Targets
Moving targets often appear when users are intentionally challenged, such as in games or learning. This is called moving target interception [4] or anticipation-coincidence task [19] in psychology, and a few studies have been conducted recently in HCI field [28, 36, 38, 8, 9]. To date, there are no predictive models to explain the user’s error rate in point-and-click tasks for moving targets. Even if the target has the same size and speed, the relative velocity to the user can be different, leading to various behaviors, which makes the modeling of the moving target more difficult [38]. For example, four or more different pointing strategies (pursuit, head-on, receding, and perpendicular) can be characterized [47]. Due to this complexity, most existing models assume online-corrected movement rather than preprogrammed movement of the user
Perceptual Control Theory
Previous studies on point-and-click tasks have been influenced by behaviorist perspectives in cognitive psychology. Behaviorism is the view that human behavior is determined entirely by given stimuli [49]. This resulted in efforts to explain movement time and error rate from static characteristics of stimulus such as the size or distance of a given target. Behaviorism has established itself as the mainstream of psychology by providing ease of experimentation of human behavior, but has been criticized as insufficient to explain the dynamic nature of human behavior [45]. Studies that regard human beings as a controller have complemented the limits of behaviorism. In this regard, a user is
Figure 2. The model proposed in this study can predict the user’s error rate regardless of target motion in point-and-click task. This is possible because it analyzes only the relative motion between the target and the cursor, rather than each absolute position. During the point-and-click process, the movement of the cursor can be divided into multiple submovements. As an intermittent controller, humans control each submovement independently of the preceding submovement. The last submovement can be assumed to be a moving target selection task where the user is still and the target is moving to the cursor at the relative speed. The Wt and tc values obtained by analyzing the last submovement are assigned to a known moving target selection model to predict the error rate.
an active entity that attempts to reduce the deviation between the target state and the current state (i.e., negative feedback). There are various views on what they actually control, but most models assume that users directly control their behavior [16, 17, 3, 6, 7, 5, 15] or the state of their surroundings [37, 42]. However, humans do not directly know their own state or the state of the external environment, but only perceive it. Based on this, William T. Powers published the perceptual control theory (PCT) in 1973 [45]. He wrote in the fourth chapter of Behavior: The Control of Perception, "Only the difference (if any) between that quantity and its reference condition calls for a "response". Furthermore, it is not the actual environmental situation that leads to responses, but that situation as perceived by the organism" [45, p. 48]. According to the theory, humans control the perceived error itself. In other words, in the point-and-click process, users do not control the position of the cursor or the movement of the arm, but simply control the perception of the relative positional relationship between the cursor and the target. Perceptual control theory (PCT) is an unfamiliar theory in HCI, but one recent study has applied it to simulate button presses [43]. Summary
Various studies on point-and-click tasks have been published in HCI and psychology, but there is no model for predicting user error rates regardless of target motion. This study builds a model of point-and-click task from the PCT point of view. In particular, our model assumes that the situation where the user moves toward the target and the situation where the target approaches the user is the same in the perception of the relative positional relationship. This allows the existing moving target selection model to be applied to the user’s last submovement, and makes it possible to predict the pointing error rate regardless of the target motion. MODEL OVERVIEW
In essence, the model proposed in this study predicts the error rate of users by considering the point-and-click task as another known task, moving target selection. In the moving target selection task, users must generate a click event when a moving target passes through a given selection region. The error rate, which indicates the rate at which the user failed the task, is precisely predictable from the model proposed by Lee and his
colleagues [36] (see Equation 2). More specifically, for each click, the last submovement of user movement is considered as a moving target selection task. To do this, we make three theoretical assumptions based on human motor control and perceptual control theory (see Figure 2): Submovement decomposition: A point-and-click movement can be divided into several submovements [52, 16, 41] using local accelerations and decelerations in the cursor trajectory. Each submovement is a ballistic movement based on the user’s openloop control. Intermittent control: Each submovement is independently programmed and executed from the preceding submovement [3, 17, 37, 16, 41]. In particular, the model assumes that only the characteristics of the last submovement leading to a click event determine the error rate. Perception of relative motion: The user perceives and controls only the relative motion between the target and the cursor. From the perspective of relative motion, the last submovement can be assumed to be a moving target selection task [36] where the user is still and the target is moving to the cursor at the relative velocity. Here, since the cursor can be regarded as a point, the size of the selection region can be regarded as equal to the size of the target. The intermittent control of human behavior, and the presence of submovement, are widely known from studies of human motor control in the past. However, the final assumption about the perception of relative motion can be unfamiliar to readers. William T. Powers gives an example in his book to help understand [45]. Suppose a person is trying to put a long stick upright on his or her palm. In this process, a person moves his / her hand to the left if the stick is to the left of the reference point and to the right if it is to the right of the reference point. In other words, he or she pursues the relative purpose of "reducing the error between the stick and the reference point" rather than the absolute goal of "making the stick go to a certain position". The absolute position of the stick is not important to the person controlling the stick. From this perspective, it is not the position of the cursor that users control in point-and-click tasks. Rather, users try to control and reduce the relative positional difference between the cursor and the target. Because of this relativity, it can be assumed that in each submovement, the user’s cursor is
virtually fixed, and that the target is approaching the cursor. This is the same situation given in the moving target selection task! [36] In this study, we will validate the above assumptions through two user studies. Independent Variables
Our model predicts the user’s error rate for point-and-click tasks from two measurements: (1) cue viewing time tc and (2) temporal width Wt . These two variables are measured per each click and substituted into Equation 3 (written again) to calculate the error rate ER: (1 − cµ ) Wt cµ W 1 √ · ) + er f ( √ · t ) ER = 1 − er f ( 2 cσ 2 Dt cσ 2 Dt (3) 1 where, Dt = ( νt +δ) e c −1
First, tc , in a normal moving target selection task, is the time at which a user can observe the moving target until it reaches the selection region. The longer the tc , the better the user can anticipate the input timing from the visual information given by the motion of the target. In this study, we consider the last submovement of the point-and-click task as the moving target selection, so users can observe relative movement between the cursor and the target from the beginning of the submovement to the click. Therefore, the time from the beginning of the last submovement (tsub ) to the moment of the click (tclick ) can be regarded as tc : tc = (tclick − tsub )
(4)
Second, Wt is the duration that the target passes through the selection region. Only click events that occur during this time are considered successful. In the last submovement, the target moves relative to the user’s cursor. And the time from the moment the target first contacts the user’s cursor to the moment the cursor leaves the cursor is Wt . This is determined from the velocity ~vt of the target, the velocity ~vc of the cursor, the positional relationship of the target and the cursor, and the shape of the target. However, for brevity, it can be assumed that the relative velocity between the target and the cursor in the last submovement is pointing toward the center of the target. This makes it possible to estimate Wt only from the velocity of the target, the velocity of the cursor, and the size of the target as follows: Wt =
W k~vt −~vc k
(5)
Empirical Parameters
To calculate the error rate by substituting the above Wt and tc into the Equation 3, the four parameters (cµ , cσ , ν, δ ) of the equation must be determined empirically in advance. cσ is the amount of noise present in the user’s temporal perception. People with high cσ have difficulty in performing timing
Algorithm 1 Obtain error rate per point-and-click 1: tc ,Wt , ER ← 0 2: while true do LogTra jectory() 3: if ClickEvent == True then 4: FilterTra jectory() 5: SegmentLastSubmovement() 6: tc ← (tclick − tsub ) 7: Wt ← k~vt W −~vc k 8: ER ← getErrorRate(tc ,Wt ) 9: ClearLog() 10: end if 11: end while
tasks by perceiving the temporal structure inherent in a given stimulus. cµ is a value between 0 and 1 representing the aim point of the user in the selection region. If cµ is 0 or 1, the user wants to generate a click event at the moment the target first contacts or leaves the selection region, respectively. If the timing input is assumed to be a Gaussian distribution, the error rate is lowest when cµ is 0.5. ν is the rate at which the user obtains information from the motion of the target to perceive the Dt . This value is higher when the user acquires a moving target. The larger ν gets, the lower the error rate becomes for the same cue viewing time tc . δ represents the minimum value of Dt that the user can perceive when given a sufficiently long tc . The inverse of δ is simply the maximum reliability of the information given by the target motion. As δ decreases, the error rate of the user also decreases. Refer to their paper [36] for a detailed derivation of the model. The previous study [36] reported these four parameter values for the actual moving target selection task (see Table 1). This can vary slightly for different users and applications. However, the study also has shown that parameter values obtained from completely different applications can linearly explain user error rates in other applications. Contributions
The model we propose is particularly novel in the following aspects: (1) it does not need to estimate the starting point of motion unlike the existing models because it considers only the last submovement in the point-and-click process; (2) the error rate can be predicted regardless of the target motion; (3) empirical parameters have a clear physical meaning because they are based on the previously published model derived from Table 1. The experimental results for the existing model (CHI’18) and the results from the two user studies for this paper, summarized together. Studies cµ cσ ν δ R2 Moving target selection [36] 0.295 0.083 20.2 0.366 0.81 Moving target selection 0.118 0.0316 N/A 0.484 0.96 (Flappy Bird) [36] Moving target selection 0.496 0.186 28.7 0.191 0.89 (Cake Tower [36]) Point-and-Click Study 1 0.217 0.158 6.591 0.344 0.98 (fixed target) Point-and-Click Study 2 0.24 0.172 38.51 0.29 0.93 (moving target w/o outlier) Point-and-Click Study 2 0.17 0.126 37.01 0.45 0.89 (moving target with outlier)
cue integration theory [18]; (4) the error rate can be predicted from only two independent variables (tc and Wt ). Note) Our model was derive to predict the error rate, not the individual error. In other words, our model does not determine whether a user’s click is intended or an error [39], and if so, what kind of error it is [32]. In order to derive the user’s intention, it is necessary to consider not only the pointing error rate but also the cost and the risk of the error in the interaction [2, 21]. This requires further study in the future. SYSTEM IMPLEMENTATION
The model proposed in this study is implemented as an algorithm with one thread, which enables to predict the pointing error rate for each click in real time (see Algorithm 1). This section describes the detailed process of implementing the algorithm as a real system. The implemented system predicts the user’s error rate for each click through three steps. First, the trajectory of the cursor and the trajectory of the target are logged until the occurrence of one click event (trajectory logging). Second, the system analyzes the speed of the cursor and parses the last submovement (submovement segmentation). The duration of the last submovement is tc . Third, the system divides the size of the target by the relative speed between the cursor and the target in the last submovement to obtain Wt (target analysis). Step 1: Real-Time Trajectory Logging
The system first logs the target and cursor trajectories in real time until one click event occurs. When a click occurs, the system analyzes the logged trajectory until that time, obtains the error rate, and initializes the corresponding logging variable. Therefore, the system does not require much memory. In the case of indirect pointing input devices such as a mouse or trackpad, it is easy to log the trajectory of the cursor. However, logging of cursor trajectories indirect input devices such as touch screens requires special devices such as proximity sensors [27]. This study implemented the system for a computer mouse which is the most widely used indirect input device. An indirect input device needs a mapping function that creates a cursor motion from the raw signal of the device, which is called a gain function [10]. The system logs the pixel coordinates of the cursor (xi , yi ) and their timestamps (t i ) after the gain function is applied. Since the sampling rate of the input device is usually higher than the refresh rate of the display, the target trajectory (xˆi ,yˆi ) is also logged in synchronization with the trajectory of the input device. If the input device’s sampling rate is f Hz, then the system will log the following data for each click: i i i i i x y xˆ yˆ t , for i = 0 to N where t i = 1/ f × i t 0 is the moment the preceding click occurred and t N is the moment when the current click event occurred. The target’s trajectory is logged to calculate the relative velocity between the cursor and the target in Step 3. Therefore if the velocity of the target is explicitly known, the system does not need to log it.
Figure 3. To predict the user error rate in a point-and-click task, the system should analyze the speed profile of the cursor to segment the submovements. The local extrema can be found by applying the Persistence1D algorithm to the cursor speed profile with noise removed through the Gaussian kernel filter (σ =3). Each neighboring minimum-maximum-minimum triplet is considered to be a possible submovement. Among them, the user error rate can be predicted by analyzing the last submovement occurred just before the click event. The drawn data is an actual trial of one participant in Study 2.
Step 2: Submovement Segmentation to Obtain tc When a click event occurs, the speed profile is calculated from the cursor’s logged trajectory. Since the trajectory of the cursor is logged at relatively constant time intervals, the speed of the cursor (si ) can be expressed as: [si ] = kvxi , vyi k = kxi − xi−1 , yi − yi−1 k × f , for i = 1 to N
The sensor noise in the calculated speed profile has a significant effect on the performance of the submovement segmentation, so it must pass through a low pass filter. The study smoothed the speed profile through a Gaussian kernel filter (σ = 3). The system then identifies the local minima and maxima in the speed profile and each neighboring minimummaximum-minimum triplet is considered to be a possible submovement. We use Persistence1D [34] as an algorithm to find local extrema in the speed profile. This algorithm returns all pairs of minima and maxima that exceed the pre-defined persistence value (0.2). To prevent jitter from click motion from being missegmented into a submovement, only triplets with a minimum submovement length of 2 pixels are considered as submovement. These processes are summarized in Figure 3. Among the local minima obtained earlier, the minimum point immediately before the click event can be regarded as the starting point of the last submovement (tsub ). In this case, the first independent variable tc of the model can be calculated as the duration from tsub to the moment the click event occurs (t N ): tc = (tclick − tsub ) = (t N − tsub )
(6)
Step 3: Target Analysis to Obtain Wt Wt is the time it takes the cursor to pass through the target when the last submovement is considered as a moving target selection task. In other words, it is the time limit given to the user for successful target selection. Wt is determined by the relative velocity of the target and the cursor, and the size of the target (W ). Since the system implemented in this study assumes a circular target, W is the diameter of the target.
Suppose that at the beginning of submovement, the cursor and target positions are (xsub , ysub ) and (xˆsub , yˆsub ), respectively. If so, the velocity vector of the cursor (~vc ) and target (~vt ) until the moment of click can be written as: ~vc = (xN − xsub ) (yN − ysub ) /tc (7) ~vt = (xˆN − xˆsub ) (yˆN − yˆsub ) /tc At this time, the cursor and the target approach each other as the relative speed k~vt −~vc k. As a result, Wt can be calculated from Equation 5. The tc and Wt calculated through these three steps are assigned to Equation 3 to predict the error rate (ER). After that, all trajectories that have been logged are removed, and the system repeats Steps 1 to 3 until the next click event occurs (see Algorithm 1). USER STUDIES
Based on the implemented system, two user studies were performed to verify the error rates predicted from the model. The first study measured the error rate of two-dimensional pointand-click tasks on a fixed circular target. Unlike a typical pointing experiment, users were given a time limit, which resulted in a wide range of error rates. We use Wobbrock’s error rate model [53] as the baseline for performance comparison. The second study was a two-dimensional point-and-click task for a circular moving target. The target had linear motion at a constant velocity and various error rates are observed by providing a wide range of target speeds. Since there is no existing model for explicitly predicting the error rate for this task, the experimental results are analyzed without the baseline. STUDY 1: POINT-AND-CLICK ON A FIXED TARGET
How to perform point-and-click tasks for fixed targets is documented in the ISO 9241-9 standard [29]. However, the standard assumes a situation in which the experimental results are interpreted by Fitts’ law, so it is not suitable for observing a wide range of error rates. We followed the standard as far as possible, but we observed a broad range of point-and-click behaviors by adding an independent variable called a time limit to the experiment. Method Participants
Twelve paid participants from the local university (7 males, 5 females) were recruited. The average age of participants was 24.42 years (σ =3.26). All the participants were right-handed, and they all have experience using a mouse. Their average mouse usage time per day was 5.63 hours (σ =3.59). The participants played games 6.86 hours (σ =2.61) per a week with a computer mouse. Seven participants wore glasses and their average visual acuity was 1.1 (σ =0.231). Design
The experiment followed a 2×3×6 within-subject design with three independent variables: target width, target distance and time limit. The levels were the following: • Target width: 4.8 and 8.4 mm
Figure 4. Task screen in Study 1: The blue target and the red target are placed on the screen at the same time and the participant is instructed to click on the blue target first and then click on the red target as quickly and accurately as possible. This follows the ISO 9241-9 standard, but gives the user a specific time pressure by making the red target disappear after a period of time after clicking on the blue target. This allows a sufficiently wide range of error rates to be generated from the participants, from easy tasks to difficult tasks. The angle of approach was given to the participant in a clockwise sequence of 360 degrees divided into 20 steps.
• Target distance: 48, 144, and 240 mm • Time limit: 300, 400, 500, 600, 700, and 800 ms Twenty angle of approaches were tested for each target widthtarget distance condition. A time limit condition did not change to the next condition until all corresponding widthdistance conditions had been completed and each time limit condition is repeated twice. As a result, participants performed 240 pointing trials for each time limit condition. The time limit conditions are given in a random order, and within a time limit condition, the target width and target distance are given in random order. In the end, 17,280 (=20 × 2 × 3 × 2 × 12) input events from 12 participants were logged. Task
Participants had to select two circular targets on the screen. After clicking on the blue target, clicking on the red target ended the trial (see Figure 4). If the participant did not click on the red target within the given time limit after clicking on the blue target, the red target disappeared. Even if the red target was disappeared due to time limit violation, participants had to click to go to the next trial. If the participant clicked inside the red target (or the disappeared red target), the trial was considered successful. Participants were asked to make pointing as quickly and accurately as possible. They were also asked to complete each trial within the time limit. Apparatus
The application was implemented on a 3 GHz desktop computer (Mac mini, 10.13.1 High Sierra). A 27-inch LED monitor (LG 27UD69P) was used and the resolution of the task screen was 2560×1440 pixels. Pointing device was two-button wired optical mouse (Samsung SNJ-B138) with a resolution of 400 DPI and the polling rate of the device was 125Hz. The cursor was a standard arrowhead pointing to the upper left.
The mouse gain function maintained the default setting of the OS. Procedure
Participants performed the task with the same posture as they were using the computer. They sat on a regular office chair and the monitor was installed at the participant’s eye level. Before the experiment, experimenter briefly introduced the task to the participants. Subsequently, the participants filled in a pre-questionnaire. A practice session was given until participants were accustomed to the task. Result and Discussion Analysis of Data
For all trials, the movement time that the participants actually performed the task was about 124% of the given time limit. However, as the last submovement already started at 72% (σ =51%) of the time limit, participants did not intentionally wait for the target to disappear. Fifty-two trials with trajectory lengths of 2 pixels or less were considered accidental clicks and were removed (0.3% of the total data). No other data was removed. The overall average error rate for all participants’ trials was 63%. This is 2 to 3 times higher than the error rate in other studies [53, 54]. This is important in that we can assess whether our model predicts error rates well for a wide range of error rates. The duration of the last submovement, or tc , was measured to be 296 ms (σ =119 ms) on average, which is similar to the known values in previous studies [30, 37]. Wt was measured to be 300 ms on average (σ =472). These results are summarized in the Table 2.
Figure 5. Results from Study 1: Both the Wobbrock model and the model proposed in this study well explained the observed error rate. From the two-fold cross validation, the Wobbrock model and our model showed a mean absolute error of 4.04% and 3.64%, respectively.
Overall Model Fit
All model fittings were made using the Global Optimization Toolbox provided by MATLAB. In order to increase the accuracy of the fitting of the Wobbrock model, the actual movement time, instead of the given time limit condition, was used as the MTe variable (see Equation 1). However, the binned average was based on the time limit condition given for each trial. As a result, the Wobbrock model was fitted with a total of 36 data points (2 target widths × 3 target distances × 6 time limits). As shown in the previous studies [53, 54], the model successfully explains the error rate of the user (R2 = 0.94, see Figure 5). The a and b values of the model were 100.1 ms and 166.5 ms/bit, respectively. Because these values are based on Fitts’ law, throughput can be calculated as the reciprocal of b. As a result, we obtained 6 bit/s similar to the previously measured value for the mouse [40]. Table 2. Independent variables and error rates of our model measured in Study 1 and Study 2: Participants showed a similar level of pointing error rate whether the target was moving or not. Moving targets, especially Wt, were shorter and more difficult to click properly. However, the efficiency of obtaining timing information from the movement of the target has increased (higher ν measured in Study 2, see Table 1) and so shorter Wt has not affected the error rate. Studies ER (%) tc (ms) Wt (ms) Point-and-Click Study 1 63 296 (σ =119) 300 (σ =472) (fixed targets) Point-and-Click Study 2 63 280 (σ =230) 188 (σ =294) (moving targets)
Unlike Wobbrock’s model, the independent variables tc and Wt of the model proposed in this study are measured as continuous values. So in order to calculate the error rates, we have to consider how to bin the data. In our model, the variable that determines the error rate of the user is Wt /Dt (see Equation 3). Therefore, we performed a binning average of 479 trials from the left after sorting the whole data in order of decreasing Wt /Dt (or increasing error rate). This allows us to get 36 final data points as we fit the Wobbrock model. But in our model, Dt depends on the empirical parameters (ν and δ ) that we have to obtain through fitting. This means that during the fitting process, all data must be sorted and binned for each step. The following values from a previous study [36] were used as the initial condition of the empirical parameters for the first sorting: cµ =0.25, cσ =0.08, ν =20.2, δ =0.366. As a result, our model fitted with the observed error rate with a high coefficient of determination (R2 = 0.98, see Figure 5). The empirical parameters obtained as a result of fitting are summarized in the Table 1. We also performed two-fold cross validation with random sampling for each model. The mean absolute error (MAE) values obtained were 4.04% for the Wobbrock model and 3.64% for our model. STUDY 2: POINT-AND-CLICK ON A MOVING TARGET
The point-and-click task for a moving target is much more complex than for a fixed target. If we can predict the pointing
Figure 7. Results from Study 2: The model proposed in this study well explained the observed error rate (R2 = 0.89). From the two-fold cross validation, the model showed a mean absolute error of 6.35%. Removing the outliers raises the R2 value to 0.93. Figure 6. Task screen in Study 2: A single target was created at random size, position, and velocity and moved linearly, and participants were required to click on it. When the target hit the wall, the direction of movement was reversed like an elastic collision while maintaining the speed. The target trail is added to aid understanding and does not exist in the actual experiment.
error rate as a single model, regardless of target motion, it will be useful for interaction designers. In the second study, participants performed a point-and-click task on a two-dimensional target with a linear motion. In order to satisfy ecological validity, we reproduced the speed range of the target in commercial games such as Fruit Ninja 1 (107 mm/s) and Piano Tiles 2 (314 mm/s). Above all, we wanted to see if our model could predict the error rate for totally different tasks, keeping the system setting used in the first study. Method
Participants: Sixteen paid participants (15 males, 1 female) were recruited from the local university. The average age of participants was 23.69 years (σ =2.18). All the participants were right-handed, and they all have experience using a mouse. Their average mouse usage time per a week was 4.72 hours (σ =3.0). The participants played games 8.13 (σ =11.21) hours on average per a week with a mouse. Thirteen participants wore glasses and the average visual acuity was 0.93 (σ =0.39). Design: The experiment followed a within-subject design with two independent variables: target speed and target width. The target speed and target width were given randomly within a following range: • Target velocity: from 0 mm/s to 510 mm/s • Target width: from 9.6 mm to 24 mm The location where the target is created and the direction the target moves are randomly determined for each trial. Participants performed total 9 blocks of trials and each block consisted of 200 trials. As a result, 28,800 input events from 16 participants were logged (=16 × 9 × 200). Task: Participants were instructed to click on a blue circular target with a linear motion at a constant speed (see Figure 6). If the target collides with the wall (edge of the screen), 1 https://fruitninja.com/ 2 http://tanksw.com/piano-tiles/
the target is bounced at the same angle as the incident angle. The trial was considered successful only when the participant clicked inside the target. Regardless of success, if a click event occurs, the current target disappears and a new target is created. Participants were asked to make pointing as quickly and accurately as possible. Apparatus: The apparatus and other system settings were the same as in Study 1. Procedure: Participants sat on a regular chair. The display was installed at the participant’s eye level. After filling in a pre-questionnaire asking about previous gaming experience, the experimenter briefly explained about the task. 50 trials were given to participants as a practice session before starting the main study. In the experiment, one minute of a break was provided at the end of each block. It took about an hour per person to finish the study. Result and Discussion Analysis of Data
826 trials with trajectory lengths of 2 pixels or less were considered accidental clicks and were removed (2.9% of the total data). Due to the dynamic task situation, the ratio of accidental clicks was higher than Study 1. No other data has been removed. The overall average error rate for all participants’ trials was 63%. This value is almost the same as Study 1, showing that the participants performed well in spite of the higher level of task difficulty. The duration of the last submovement, or tc , was measured to be 280 ms (σ =230 ms) on average, which is Similar to the submovement duration reported in previous studies [30, 37]. Wt was measured to be 188 ms on average (σ =294 ms). Overall Model Fit
All model fittings were made using the Global Optimization Toolbox provided by MATLAB. In the same manner as in Study 1, the data of all trials were binned by 500, and finally 36 averaged data points were obtained. Our model fitted from the same initial condition as Study 1 describes the observed error rate as a high coefficient of determination (R2 = 0.89, see Figure 7). The empirical parameters obtained as a result of fitting are summarized in the Table 1. The ν value was higher than Study 1, indicating
that participants were more dependent on the given visual information than when pointing to a fixed target. We also performed two-fold cross validation with random sampling. The mean absolute error (MAE) was 6.35% for our model. To analyze the reason why the R2 value was lower than Study 1, the distance between the target and the cursor was observed at the moment when the submovement started and at the moment when the click was made. For brevity, our model assumed that the submovement was always approaching the target, but in the real data, the 4,184 trial (14.5% of total) was farther away from the target at the moment of the click event than at the start of the submovement. Removing these trials and fitting again increases R2 to 0.93 (see Table 1). This implies that further study is needed on how to calculate the Wt from the relative movement of the target and cursor. Still, our model showed that we could predict the pointing error rate for a moving target, which was not possible before, with very high accuracy. CONCLUSION
The model proposed in this study accurately predicted users’ pointing error rate with a simple algorithm regardless of the target movement (R2 = 0.89 to 0.98 and MAE=3.64% to 6.35%). Our model explains the complex situation in which the target and the user are simultaneously moving into a much simpler task: the moving target selection. Here, the perceptual control theory provided a theoretical background to consider only the relative motion between the user and the target. The empirical parameters of our model tell us more about the cognitive characteristics of users than Fitts’ law. Our model can contribute to future research on games, music, crowdsourcing, and input techniques. REFERENCES
1. Jos J Adam, Robin Mol, Jay Pratt, and Martin H Fischer. 2006. Moving farther but faster: an exception to Fitts’s law. Psychological science 17, 9 (2006), 794–798. 2. Nikola Banovic, Tovi Grossman, and George Fitzmaurice. 2013. The effect of time-based cost of error in target-directed pointing tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1373–1382. 3. WDA Beggs and CI Howarth. 1972. The accuracy of aiming at a target: Some further evidence for a theory of intermittent control. Acta psychologica 36, 3 (1972), 171–177. 4. James J Belisle. 1963. Accuracy, reliability, and refractoriness in a coincidence-anticipation task. Table 3. Results of cross validation in Study 1 and 2: Our model showed better performance for the fixed target than the Wobbrock model. Our model also showed a low mean absolute error (MAE) for moving targets. Studies MAE (%) Wobbrock’s model (fixed targets) 4.04 Our model (fixed targets) 3.64 Our model (moving targets) 6.35
Research Quarterly. American Association for Health, Physical Education and Recreation 34, 3 (1963), 271–281. 5. Daniel Bullock and Stephen Grossberg. 1989. VITE and FLETE: Neural modules for trajectory formation and postural control. In Advances in Psychology. Vol. 62. Elsevier, 253–297. 6. Robin T Bye and Peter D Neilson. 2008. The BUMP model of response planning: Variable horizon predictive control accounts for the speed–accuracy tradeoffs and velocity profiles of aimed movement. Human movement science 27, 5 (2008), 771–798. 7. Robin T Bye and Peter D Neilson. 2010. The BUMP model of response planning: intermittent predictive control accounts for 10 Hz physiological tremor. Human movement science 29, 5 (2010), 713–736. 8. Juan Sebastián Casallas. 2015. Prediction of user action in moving-target selection tasks. Ph.D. Dissertation. Iowa State University. 9. Juan Sebastián Casallas, James H Oliver, Jonathan W Kelly, Frédéric Merienne, and Samir Garbaya. 2013. Towards a model for predicting intention in 3D moving-target selection tasks. In International Conference on Engineering Psychology and Cognitive Ergonomics. Springer, 13–22. 10. Géry Casiez and Nicolas Roussel. 2011. No more bricolage!: methods and tools to characterize, replicate and compare pointing transfer functions. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 603–614. 11. Richmond B Chan and Dudley S Childress. 1990. On a unifying noise-velocity relationship and information transmission in human-machine systems. IEEE Transactions on systems, Man, and Cybernetics 20, 5 (1990), 1125–1135. 12. Olivier Chapuis, Renaud Blanch, and Michel Beaudouin-Lafon. 2007. Fitts’ law in the wild: A field study of aimed movements. (2007). 13. Olivier Chapuis and Pierre Dragicevic. 2008. Small targets: why are they so difficult to acquire. Laboratoire de Recherche en Informatique, Tech. Rep (2008). 14. Olivier Chapuis and Pierre Dragicevic. 2011. Effects of motor scale, visual scale, and quantization on small target acquisition difficulty. ACM Transactions on Computer-Human Interaction (TOCHI) 18, 3 (2011), 13. 15. Edward M Connelly. 1984. A control model: An alternative interpretation of Fitts’ Law. In Proceedings of the Human Factors Society Annual Meeting, Vol. 28. Sage Publications Sage CA: Los Angeles, CA, 625–628. 16. ERFW Crossman and PJ Goodeve. 1983. Feedback control of hand-movement and Fitts’ law. The Quarterly Journal of Experimental Psychology Section A 35, 2 (1983), 251–278.
17. COLIN G DRURY. 1994. A model for movements under intermittent illumination. Ergonomics 37, 7 (1994), 1245–1251. 18. Marc O Ernst and Martin S Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870 (2002), 429. 19. Brett R Fajen and William H Warren. 2007. Behavioral dynamics of intercepting a moving target. Experimental Brain Research 180, 2 (2007), 303–319. 20. Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology 47, 6 (1954), 381. 21. Julia Galliers, Alistair Sutcliffe, and Shailey Minocha. 1999. An impact analysis method for safety-critical user interface design. ACM Transactions on Computer-Human Interaction (TOCHI) 6, 4 (1999), 341–369. 22. Cheryl M Glazebrook, Dovin Kiernan, Timothy N Welsh, ˘ Zs ´ Law and Luc Tremblay. 2015. How one breaks FittsâA and gets away with it: Moving further and faster involves more efficient online control. Human movement science 39 (2015), 163–176. 23. Julien Gori, Olivier Rioul, and Yves Guiard. 2017. To Miss is Human: Information-Theoretic Rationale for Target Misses in Fitts’ Law. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 260–264. 24. Yves Guiard and Halla B Olafsdottir. 2011. On the measurement of movement difficulty in the standard approach to Fitts’ law. PLoS one 6, 10 (2011), e24389. 25. Yves Guiard, Halla B Olafsdottir, and Simon T Perrault. 2011. Fitt’s law as an explicit time/error trade-off. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1619–1628. 26. Yves Guiard and Olivier Rioul. 2015. A mathematical description of the speed/accuracy trade-off of aimed movement. In Proceedings of the 2015 British HCI Conference. ACM, 91–100. 27. Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O’Hara, Gavin Smyth, and William Buxton. 2016. Pre-touch sensing for mobile interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2869–2881. 28. Jin Huang, Feng Tian, Xiangmin Fan, Xiaolong (Luke) Zhang, and Shumin Zhai. 2018. Understanding the Uncertainty in 1D Unidirectional Moving Target Selection. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, to be determined. 29. ISO ISO. 9241-9 Ergonomic requirements for office work with visual display terminals (VDTs)-Part 9:
Requirements for non-keyboard input devices (FDIS-Final Draft International Standard), 2000. International Organization for Standardization (????). 30. Richard J Jagacinski, Daniel W Repperger, Martin S Moran, Sharon L Ward, and Betty Glass. 1980. Fitts’ law and the microstructure of rapid discrete movements. Journal of Experimental Psychology: Human Perception and Performance 6, 2 (1980), 309. 31. Timo Jokela, Netta Iivari, Juha Matero, and Minna Karukka. 2003. The standard of user-centered design and the standard definition of usability: analyzing ISO 13407 against ISO 9241-11. In Proceedings of the Latin American conference on Human-computer interaction. ACM, 53–60. 32. Akiyo Kano, Janet C Read, Alan Dix, and I Scott MacKenzie. 2007. ExpECT: an expanded error categorisation method for text input. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1. British Computer Society, 147–156. 33. Sunjun Kim, Byungjoo Lee, and Antti Oulasvirta. 2018. Impact Activation Improves Rapid Button Pressing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, to appear. 34. Y Kozlov and T Weinkauf. 2015. Persistence1D: Extracting and filtering minima and maxima of 1d functions. h ttp://people. mpi-inf. mpg. de/ weinkauf/notes/persistence1d. html, accessed (2015), 11–01. 35. Byungjoo Lee, Qiao Deng, Eve Hoggan, and Antti Oulasvirta. 2017. Boxer: a multimodal collision technique for virtual objects. In Proceedings of the 19th ACM International Conference on Multimodal Interaction. ACM, 252–260. 36. Byungjoo Lee, Sunjun Kim, and Antti Oulasvirta. 2018. Moving Target Selection: A Cue Integration Model. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, to appear. 37. Byungjoo Lee, Mathieu Nancel, and Antti Oulasvirta. 2016. AutoGain: Adapting Gain Functions by Optimizing Submovement Efficiency. arXiv preprint arXiv:1611.08154 (2016). 38. Byungjoo Lee and Antti Oulasvirta. 2016. Modelling error rates in temporal pointing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1857–1868. 39. Clayton Lewis and Donald A Norman. 1995. Designing for error. In Readings in Human–Computer Interaction. Elsevier, 686–697. 40. I Scott MacKenzie and Poika Isokoski. 2008. Fitts’ throughput and the speed-accuracy tradeoff. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1633–1636.
41. David E Meyer, Richard A Abrams, Sylvan Kornblum, Charles E Wright, and JE Keith Smith. 1988. Optimality in human motor performance: ideal control of rapid aimed movements. Psychological review 95, 3 (1988), 340. 42. Jörg Müller, Antti Oulasvirta, and Roderick Murray-Smith. 2017. Control theoretic models of pointing. ACM Transactions on Computer-Human Interaction (TOCHI) 24, 4 (2017), 27. 43. Antti Oulasvirta, Sunjun Kim, and Byungjoo Lee. 2018. Neuromechanics of a Button Press. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, to appear. 44. Réjean Plamondon and Adel M Alimi. 1997. Speed/accuracy trade-offs in target-directed movements. Behavioral and brain sciences 20, 2 (1997), 279–303. 45. William T Powers and William T Powers. 1973. Behavior: The control of perception. Aldine Chicago. 46. Richard A Schmidt, Howard Zelaznik, Brian Hawkins, James S Frank, and John T Quinn Jr. 1979. Motor-output variability: A theory for the accuracy of rapid motor acts. Psychological review 86, 5 (1979), 415. 47. James R Tresilian. 2005. Hitting a moving target: perception and action in the timing of rapid interceptions. Perception & Psychophysics 67, 1 (2005), 129–149. 48. Stephen A Wallace and Karl M Newell. 1983. Visual control of discrete aiming movements. The Quarterly
Journal of Experimental Psychology 35, 2 (1983), 311–321. 49. John B Watson. 1913. Psychology as the behaviorist views it. Psychological review 20, 2 (1913), 158. 50. Alan Traviss Welford. 1968. Fundamentals of skill. (1968). 51. Wayne A Wickelgren. 1977. Speed-accuracy tradeoff and information processing dynamics. Acta psychologica 41, 1 (1977), 67–85. 52. Deric Wisleder and Natalia Dounskaia. 2007. The role of different submovement types during pointing to a target. Experimental Brain Research 176, 1 (2007), 132–149. 53. Jacob O Wobbrock, Edward Cutrell, Susumu Harada, and I Scott MacKenzie. 2008. An error model for pointing based on Fitts’ law. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1613–1622. 54. Jacob O Wobbrock, Alex Jansen, and Kristen Shinohara. 2011. Modeling and predicting pointing errors in two dimensions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1653–1656. 55. Shumin Zhai, Jing Kong, and Xiangshi Ren. 2004. ˘ Z´ law tasksâA ˘ Ton ˇ the Speed–accuracy tradeoff in FittsâA equivalency of actual and nominal pointing precision. International journal of human-computer studies 61, 6 (2004), 823–856.
