Structure of Hand/Mouse Movements - IEEE Xplore

790

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

Structure of Hand/Mouse Movements Yu Chen, Errol R. Hoffmann, and Ravindra S. Goonetilleke

Abstract—This study investigated the distribution of movement time (MT) and submovement time as a function of Fitts’ index of difficulty (ID) to help understand the underlying mechanisms of Fitts’ law. Many previous studies have shown that movements in such a task can be broken into a series of submovements, but have not been able to account for the variation of MT with ID under different conditions and for different tasks. The trajectory of computer mouse movements was recorded and analyzed to identify the distribution of submovements and MTs. The movement trajectory consists of a series of submovements with predictable probabilities. The first submovement is ballistic in nature and has an initiation time proportional to the movement amplitude. The verification time is dependent on target size. The probabilities of occurrence of each submovement can account for the variations in MT. The developed method has application to the evaluation of computer input devices, as the structure of the movement shows the ability of a person to control a device to minimize the time for movement. Index Terms—Ballistic movement, Fitts’ law, index of difficulty, movement time (MT), motor control, submovements.

I. INTRODUCTION Fitts’ law [1], [2] expresses a speed–accuracy relationship between speed of movement, the amplitude of the movement, and allowable tolerance with which the movement is to be completed. The law relates movement time (MT) to the index of difficulty (ID), in various settings [3]–[6] as follows:   2A M T = a + b log2 = a + b ID (1) W where A is the amplitude of movement, W is the target width, and a and b are empirically derived constants. Fitts’ law is applicable when the value of ID is sufficiently large so that it is necessary to use ongoing visual control to accurately make the movement. At low values of ID or when the MT is less than about 200 ms, the MT may not be sufficient for visually controlled corrections to be made continuously during the movement [7]–[10], [19], [20]. These rapid voluntary movements without ongoing visual corrections to the path are called ballistic movements, and the corresponding MT is solely dependent on the square root of the amplitude of movement [11], [12]. The following equation is physically based and has been experimentally verified [13]: √ (2) MT = a + b A. Even though MT is predicted well using (1) and (2), the underlying structure of movements is still not fully explained. Manuscript received April 27, 2014; revised December 22, 2014 and September 7, 2014; accepted February 27, 2015. Date of publication June 4, 2015; date of current version November 12, 2015. This paper was recommended by Associate Editor G. Thomas. The authors are with the Department of Industrial Engineering and Logistics Management, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/THMS.2015.2430872

The concept of submovements in a visually controlled movement possibly arises from the “deterministic iterative corrections model” of Crossman and Goodeve [8]. In that model, it is assumed that the human moves a certain distance, which is a proportion of the total distance to the target, then makes a discrete correction of location, before making the next submovement and so on until the target boundary is reached. The model consists of a set of sequential discrete corrections to accurately move to the target. Consequently, there are discrete jumps in the occurrence of the number of submovements as the ID of the movement is increased. Although Woodworth [14] was able to distinguish two phases of a visually controlled movement, the first formal proposal of the concept of submovements was made by Carlton [15], [16] who demonstrated their existence for movements with varying ID values. Jagacinski et al. [17] confirmed the presence of submovements. Submovements play an important role in the explanation of Fitts’ law since they are related to the closure of the control loop through which movement control is regulated. This control loop may be simply described as taking in visual information about the limb location relative to the target, making rapid decisions about necessary control action, and modifying muscle activity to change speed and location. Such actions take place within the context of the minimum time required for loop closure, a time that is commonly termed the “corrective reaction time.” This time has been determined by a number of experimenters (see review of Carlton [18]) and has been found to be task dependent to a certain extent and to be variable within participants in a given experiment [19], [20]. Thus, submovement analysis is important for understanding the underlying mechanisms leading to Fitts’ law. In the widely accepted explanation of Fitts’ law, the “stochastic optimized submovement model” of Meyer et al. [21] assumed that there is an initial submovement and a corrective submovement when the first submovement is off-target. The model arrives at an MT relationship that is dependent on the ratio of (A/W), as in Fitts’ law, but to a power that is the inverse of the number of submovements used in making the movement. The model assumes only two submovements, whereas data show that more may be required to accurately complete a movement [8], [9], [15]–[17], [22]. A problem with the derivation of Meyer et al. [21] is that it assumes the model of Schmidt et al. [24] for the spatial variability of the first submovement. This is a necessary part of the derivation in that it sets up variability that is then corrected by the second submovement. However, the Schmidt et al.’s equation was derived and experimentally validated for movements of fixed duration, which is not the case for the first submovement of a movement that requires ongoing visual corrections. As the first submovement has been shown to appear as a ballistic movement (see later), it is likely that the endpoint variability will be different to that assumed by Meyer et al., which was for a movement

2168-2291 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

Fig. 1.

791

product of the probability Pi (ID) on the intersection points with the curves and their corresponding ti . The model mathematically combines the probability of occurrence of a given number submovement with its particular time component in order to give the total time for the movement. This way, a model for the structure of the total movement is to be formulated.

Illustration of submovement probability model.

II. METHODS made in a fixed time (see the Appendix). Thus, although widely accepted, the Meyer et al.’s [21] model needs revision in terms of actual submovements made in a movement, since the assumptions made do not necessarily accord with reality. Aim of the Study: The aim of this study is to examine the structure of movements that have ongoing visual control, from the viewpoint of the submovements that make-up the total movement. As in [23], we divide the gross movement into stages comprising initiation, submovement(s), pause(s), and verification. Humans are not deterministic in the movements they make, as assumed in the early models of movement [9]. Rather, they exhibit both inter- and intravariability in their movements, both in the time taken for the movement and the structure of the movement, in terms of the number of submovements or path corrections made during the movement. The reasons for this variability in performance are not fully understood [37] and may arise from “system noise” in various forms, such as force variability. Thus, each of these stages has a stochastic nature, and hence, the total MT will vary. This stochastic nature of the MT is exploited to further the understanding of Fitts’ law. We initially assume that the discrete submovements are of uniform duration, and based on this assumption, a probabilistic model is proposed for MT. This assumption is experimentally tested, and the overall MT is predicted from the probability distribution and averaged durations of submovements. A. Movement Time Based on Submovement Probability As indicated in [15]–[17], a movement consists of a number of discrete submovements. At any ID, MT can be accounted for as the summation of the product of submovement time and its probability of occurrence MT(ID) =

n 

Pi (ID) × (ti + Si )

(3)

i=l

where ti is the MT for the ith submovement, Pi (ID) is the probability of having i submovements at a certain ID, where n  Pi (ID) = 1, n is the total submovement number, and si is i=1

the pause time before each submovement with s1 = 0. The proposed probability model is illustrated in Fig. 1. The Pi curve is the probability curve of having i submovements. When i = 1, the function is a growth curve with upper limit converging to 1; when i >1, the curves are assumed to be close to normal distributions, i.e., one-peak symmetrical functions. For each curve, the mean MT is certain and is equal to ti . At a certain ID, e.g., at one of the dotted lines, the MT is the

A. Independent Measures The target widths (W) were 10.5, 21, and 42 pixels with amplitudes (A) of 84, 168, 238, 336, 672, and 1344 pixels, i.e., 3 “W”s × 6 “A”s = 18 conditions corresponding to ten IDs of 2, 3, 3.5, 4, 4.5, 5, 5.5, 6, 7, and 8. B. Participants Nine students, with normal vision, from the Hong Kong University of Science and Technology participated in the experiment. Their mean age was 24.3 years with a standard deviation of 2.1 years. All were right-handed without any reported body injuries. They took part under the ethical guidelines of the University of Science and Technology. C. Apparatus A high-precision gaming laser mouse (Logitech G3 Laser Mouse) and mouse mat (Razer Mantis Speed) were used along with a notebook computer (Standard 14.1-in LCD display with resolution of 1400 × 1050 pixels, Model Thinkpad T60, CPU Intel Centrino Duo [email protected] GHz, Memory 2 GB, Graphic Card ATI X1400, USB2.0) and recording program (written in Microsoft Visual Basic). On the screen, 1 pixel was approximately 0.28 mm. The position of the cursor on the screen was recorded every 2 ms, corresponding to the USB laser mouse frequency of 500 Hz. The workstation settings included a work table with a fixed height of 73 cm and a height-adjustable (in the range of 40–52 cm) swivel chair. One-centimeter mouse movement was equivalent to approximately 6.3 cm cursor movement on the LCD screen. D. Procedure The participants were asked to adjust the chair height, position, and the angle of the laptop and the mouse to maintain a comfortable posture. The target was displayed on the screen prior to commencement of the movement and the participant made his movement when ready. Participants were asked to click the start button and move the mouse cursor to the target area with a discrete rightward movement as fast as possible while maintaining accuracy. The target was clicked when the cursor was within the target area. Target height was constant and was equivalent to screen height. The mouse gain was kept constant, in all conditions, at the default setting of MS Windows XP with the “Enhance the mouse precision” option unticked to obtain a linear mouse cursor gain. The resolution of the mouse was set at its default of 800 dpi (dots per inch). Each condition was randomly generated 100 times by the program. Subjects

792

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

TABLE I PARSING REQUIREMENTS FOR SEPARATING SUBMOVEMENTS

were asked to complete 1800 trials balanced in five time blocks. The order of the 360 trials within each block was randomized. The start button was 10 × 10 pixels (around 2×3 mm due to differing twip sizes in the Visual Basic 6.0 environment) to make sure that subjects stop and click in a stable manner with zero velocity prior to starting a movement. Subjects were allowed us to practice for 90 trials and had a 2-min rest after 90 trials. An error was either a missed click on the target or an overtime event (set as 15 s to prevent computer memory overflow). The accuracy required was a minimum of 96% of the trials hitting the target to ensure that the effective target width would not be greater than W [4], [6]. If the accuracy was lower than 96%, i.e., there were more than four errors, the 90 trials were repeated in a different random order. When the target was clicked, the program paused and the window showed the completion of a trial. A cumulative score was calculated, which was positively related to the difficulty (ID) and negatively related to the trial time. An error resulted in a deduction of 50 points. A ranking table of the final scores of all subjects was maintained to motivate the subjects to do the task as fast as possible while maintaining the speed/accuracy assumption in relation to Fitts’ law [1], [2].

Fig. 2. Criteria used to parse the velocity profiles into submovements (see Table I). “Sub” refers to a submovement; the first submovement noted as “rapid” covers a large part of the total movement distance.

E. Dependent Measures Because the movement investigated is related to a mouse cursor, supplemental criteria were used to define the beginning and end of a submovement. The criteria used are summarized in Table I and illustrated in Fig. 2. When comparing with the Fitts and Peterson’s [2] discrete movement task using a stylus, their

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

793

TABLE II EXPERIMENTAL RESULTS ID

A pixels

W pixels

Sub #

IT ms

VT ms

PMT ms

MTF i t t s ms

PMTm o d e l ms

MTm o d e l ms

2 3 3.5 4 5 6 3 4 4.5 5 6 7 4 5 5.5 6 7 8

84 168 238 336 672 1344 84 168 238 336 672 1344 84 168 238 336 672 1344

42 42 42 42 42 42 21 21 21 21 21 21 10.5 10.5 10.5 10.5 10.5 10.5

1.54 1.86 2.07 2.25 2.82 3.07 1.86 2.21 2.40 2.59 3.18 3.43 2.01 2.37 2.61 2.89 3.37 3.66

23.96 22.22 24.27 26.10 28.48 35.11 25.36 24.21 25.17 25.54 27.64 34.11 22.20 23.89 25.34 24.00 28.17 34.50

93.93 98.57 96.89 91.89 95.78 106.03 136.22 136.71 136.64 142.54 141.87 159.31 215.32 212.41 217.25 217.78 211.79 216.58

271.31 360.98 407.88 462.79 590.48 669.35 351.32 443.39 484.39 526.50 671.49 739.97 382.14 472.12 516.42 581.64 705.74 793.21

362.80 455.49 500.84 545.77 678.72 765.77 485.19 574.60 617.68 665.96 808.03 898.22 599.02 685.27 738.97 801.89 917.48 1011.92

296.23 362.39 401.03 443.50 542.21 661.00 346.34 420.80 463.48 510.11 615.60 727.48 404.75 487.42 534.10 583.51 682.09 755.66

387.75 453.03 491.17 533.22 631.93 753.08 496.05 569.62 612.09 658.96 766.87 882.61 612.64 695.35 742.91 793.91 896.40 968.59

Note: Sub # is the experimental average number of submovements under a given condition.

MT should be equal to the pure MT defined here plus verification time (VT), but excluding the initiation time (IT), i.e., the time period from when the cursor gains a certain velocity leaving the start location to the instant when the mouse button is pressed inside the target. Details of the criteria for each of the above time intervals are given in Table I. F. Data Processing: Obtaining Submovements From Movement Data The model is based on the kinematic microstructure of the movement, and hence, partitioning algorithms for submovements are essential for processing the data and for model validation. The Jagacinski et al.’s [17] definition for the end of a submovement is used in this research, when either of the following criteria is true. 1) The velocity changes sign, indicating a shift in movement direction. 2) When the velocity does not change sign but the movement trajectory decelerates and reaccelerates toward the target. The cursor position data were transformed into displacement and filtered with a fourth-order Butterworth low-pass digital filter using the signal-processing toolbox in MATLAB. A cutoff frequency (fc ) of 10 Hz, corresponding to hand tremor [25]– [27] was used. High-accuracy differentiation formulas [28] were used on the smoothed displacement data to obtain velocity. The same filter was used to smooth velocity and then differentiated to obtain acceleration, following a filtering with doubled cutoff frequency due to the doubled order of the acceleration. These velocity and acceleration profiles were then used to identify the submovements in each trial according to the criteria in Table I, using a program written in MATLAB. The program gave the IT, MT, VT, the number of submovements, the duration of each submovement, and pause time before each submovement. Some submovements were combined to achieve known characteristics associated with movement. The first submovement is normally a rapid submovement [17], [23] known as the distance-covering

phase of the movement, where a large part of the total movement distance (maybe 90% or more of total distance) is covered in a short time [29]. If it was not rapid, the first rapid submovement was merged with the preceding slow submovements, and the resulting submovement was considered as the first submovement. If there was no rapid submovement, the original submovement pattern was kept. For the first submovement, the pause time before it is assumed to be zero. The data were used to calculate the probabilities of each submovement. The “slow” submovements are those following the distance-covering submovement, where the person is making path corrections in order to home-in on the target. The proportion of the movement distance is generally small in this phase of the movement and yet the time taken for each submovement may be similar to that for the distancecovering phase of the movement. The criteria for measuring the beginning and end of the movement, along with those for the individual submovements, are given in Table I. G. Data Analysis The validity of Fitts’ law was first checked with linear regression. To investigate the effect of differences among all IDs for the dependent measures of MT, a repeated-measure ANOVA was conducted. The effect of A and W for the dependent measures of VT and IT was also tested with a repeated-measure ANOVA. The mean probabilities of occurrence of the various submovements were then calculated to determine the MT from the probability model [see (3)]. III. RESULTS AND ANALYSIS This section presents results in which a priori significance levels were α = 0.05. Trends are reported for α = 0.01. A summary of all data for IT, VT, MTFitts , and average number of submovements (Sub#) used by participants for each experimental condition is given in Table II.

794

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

Fig. 3. MTm o d e l (pure MT plus VT) calculated by (8) compared with the corresponding experiment data MTF itts .

A. General Validity of Fitts’ Law The average MT of the nine subjects for the 18 experimental conditions is plotted in Fig. 3. All linear regressions exhibited r2 values close to unity (ranged from 0.958 to 0.997; p < 0.05) again showing the validity of Fitts’ law for mouse movement. The regression for MT versus ID is MT = 134 + 110 (ID) ,

2

r = 0.992.

(4)

B. Effect of Index of Difficulty To investigate the effect of differences among all IDs, a repeated-measure ANOVA was conducted, with factors of Block, ID, and Participants (random). There were significant main effects of ID [F(9,72) = 469; p < 0.0001] and Block [F(4,32) = 3.36; p = 0.021] and an interaction of ID x Block [F(36,288) = 2.06; p < 0.001]. A Tukey post-hoc analysis on the interaction showed no significant differences, but there is a trend for the gradient of MT with ID to decrease with increase of practice on the task. C. Analysis of Initiation and Verification Time The ANOVA for IT showed a significant effect of Amplitude only [F(5,40) = 5.33; p < 0.001]. The IT is highly correlated (r > 0.95) with A, but not highly correlated with ID (r = 0.73) or with W (r = 0.025). This result is in agreement with the results of Munro et al. [30] for the reaction time in movements, who found that it was only the amplitude of a movement that effects the time for initiation of a movement. This would indicate that only the first submovement is planned during the IT, as there is no effect of the target width, which would be associated with the homing-in phase of the movement. The ANOVA for VT showed a significant main effect of Amplitude [F(5,40) = 2.91; p = 0.025] and Width [F(2,16) = 287; p < 0.001] and an Amplitude × Width interaction [F(10,80) = 2.45; p = 0.013]. However, the post-hoc Tukey test on the Amplitude × Width interaction showed that it was only the one condition of A = 1344 pixels and W = 21 pixels that contributed to the significant interaction among all the conditions containing W = 21 pixels.

Fig. 4. Average probability plot of all submovement numbers over all subjects. Each series indicates certain number of submovememts (SubM) with its corresponding regression curve.

VT is negatively correlated with target width, W (r = 0.94); as target width increases; therefore, VT decreases. More specifically, VT is strongly correlated with log2 (W) (r >0.95), but is not correlated with ID, A or log2 (A) (r = 0.58, 0.08, and 0.06, respectively). D. Submovement-Probability Model The mean submovement time, except the first, are all very similar to each other: (t1 = 232.65 ms, t2 = 136.9 ms, t3 = 136.30 ms, t4 = 127.25 ms, t5 = 125.00 ms, t6 = 124.37 ms, and t7 = 117.28 ms). An ANOVA of submovement times showed a significant effect of submovement number [F(6,96) = 74; p < 0.001] on the mean submovement time. A Tukey post-hoc test showed that this effect was only between the first submovement time and all subsequent submovement times. Because the first submovement is longer than the others, and as it may not take place under closed-loop visual control [9], [17], a further test was run on the first submovement time. The amplitude “A” had a significant effect on the first submovement (p < 0.001), but the effect of the target width was nonsignificant (p = 0.351). This result suggests that the first submovement is most likely performed ballistically and thus has a movement time that is linearly related (r2 = 0.988) to the square root of the movement amplitude [11] rather than to ID (r2 = 0.722) or W (r2 = 0.0002). The implication of this is that the first submovement (Sub1) has an amplitude that is an approximately constant proportion of the total movement amplitude. Thus, the duration of the first submove (t1 ) is not a constant, but is to be calculated using the linear regression √ (5) MTsub1 = 150 + 4.21 A, r2 = 0.988. The mean probabilities of the various numbers of submovements across all subjects are shown in Fig. 4. The submovement probability curves after the first submovement are fitted using Gaussian functions with zero offset, i.e., the base lines of the probability curves are at zero. The probability of having one submovement should converge to 1 theoretically when ID approaches 0. The probability of having one submovement is fitted with the growth/cumulative function using a Boltzmann curve with upper limit of 1 and lower bound of 0, using Origin-

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

Pro 8.0 (OriginLab, Northampton, MA, USA) (see Fig. 4). The probability curves fit well for one submovement (r2 = 0.934, p < 0.001), two (r2 = 0.915, p < 0.001), three (r2 = 0.820, p < 0.001), four (r2 = 0.959, p < 0.001), five (r2 = 0.971, p < 0.001), six (r2 = 0.962, p < 0.001), and seven (r2 = 0.798, p < 0.001) submovements. At each value of Index of Difficulty used in the experiment, the probability values of each number of submovements were calculated from the probability curves illustrated in Fig. 4. Using these probabilities and the corresponding times for each submovement, (3) was used to calculate MTs. These MTs (MTm o del ) are shown in Fig. 3, compared with the actual measured MTs (MTFitts ). IV. DISCUSSION A. Learning Even though the ID × Block interaction was significant, there was only a small effect of Block shown by the decrease in gradient of the MT versus ID relationship with increased Block number. This difference was small and post-hoc tests were not able to show where significant differences occurred, indicating that there was sufficient practice and the effect was not due to learning, fatigue or monotony with the task. Within each block, Fitts’ law applied, with MT linearly related to ID. B. Initiation Time The amplitude of movement, A, was the only main factor having a significant effect on IT, which is different from the results of [2], [22], and [23], who reported a correlation with ID. The more recent research of Munro et al. [30] shows clearly that IT is dependent only on the amplitude of the movement. This is possibly because the participant determines the movement strategy based on the distance required for the first submovement or “distance-covering” phase of the movement; he is unable to preplan the “homing-in” phase at this stage, and it must be performed online during the movement. It is noted that, in the case of [2], the task was different in that a two-choice decision of target direction was required prior to commencement of the movement; thus, the IT in that case includes time for selection of different sets of muscle groups. C. Verification Time The VT appears to be primarily dependent on target width, W, and the interaction, A × W. The effect of “A” only exists at one level of the interaction (A = 1344 and W = 21). It was shown above that the VT is strongly related to log2 (W). Welford (1968), where he discussed the time spent on a target prior to reversal of a reciprocal movement, called this time the time for clearance of the decision mechanisms, or the time to confirm that a movement has been successfully completed. In the present case, the cursor has a width of 1 pixel, and hence, the task of the subject is to determine if the 1-pixel location of the cursor is within the target width W. In terms of an information model, the subject is selecting 1 pixel out of a total of W pixels of the target width, and hence, the reaction time is related to log2 (1/W)

Fig. 5.

795

Mean submovement numbers for various IDs.

or –log2 (W). The smaller the target width, the higher the degree of uncertainty of being within the target location and hence the greater amount of information (H) to be processed to determine that the movement has been successfully completed V T ∼ H = − log2 (W/1) = − log2 (W ).

(6)

D. Submovement-Probability Model The probability of having only one submovement at ID = 2 is far from being close to 1, as might have been expected from the ability of participants to complete ballistic hand/arm movements with a single submovement at such low ID values, as found by Gan and Hoffmann [11]) for arm movements. By simply extending the Boltzmann curve fitting, the probability of having only one submovement, the probability = 1 at ID = 0 cannot be achieved, and this is possibly a limitation of the fitted model. Data from other authors [10], [15], [31] show an acceleration versus time profile that is essentially sinusoidal, without any perturbations, indicating that a single submovement can be achieved at low ID values. Another finding is that the averaged pure MT and the number of submovements used by participants are also linearly correlated with ID [r2 = 0.997 and r2 = 0.948 (see Fig. 5 and Table II) respectively, which is consistent with the previous studies using a mouse cursor task [22], [32], where a correlation of r = 0.92 between the submovement numbers and the ratio of A/W was present. Both authors have pure MT being highly correlated with ID (0.93 < r < 0.994 and r2 = 0.994). The number of submovements and pure MTs are highly positively correlated (r = 0.999, p < 0.001). With a higher number of submovements, there is a higher probability of more terms in the formula, resulting in a longer MT. Studies of number of submovements used in movements with ongoing visual control [22], [23], [39] generally show that the average number of submovements is less than two, with a linear relationship between number of submovements and Index of Difficulty. At low ID, the number of submovements is close to unity. This is consistent with a model in which ballistic movements are made at low ID and do not use ongoing visual control—the movements are completed without path correction. Wu et al. [38] also found that, with no vision available during the movement, at ID = 1, the target can be hit on 100% of trials,

796

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

reducing to 93% at ID = 3, and 32% at ID = 6. This result is consistent with the data of the present experiment. The approximate constancy of the later submovement times suggests an approximately constant time for control-loop closure or corrective reaction time during the homing-in phase of the movement. The values are consistent with those found by Carlton [10] using similar, but less sensitive, measurement techniques. Therefore, from the above analysis related to submovements [see (5)], the submovement-probability model can be modified to be √ PMTm o del (ID) = a + b A + (k + s) × P2 (ID) + 2(k + s) × P3 (ID) + 3(k + s) × P4 (ID) + · · · (7) √ where t1 = a + b A [see (5)] is the time required for the first submovement, k is the average submovement time for higher level submovements, and s is an average pause time. t1 follows the ballistic movement relationship. The modeled MT (PMTm o del in Table II) is not different from the experimental data (PMT in Table II; t = 1.99, p > 0.05). The two are significantly correlated (Pearson correlation = 0.966; p = 0.001). The use of the ballistic equation for the first submovement is consistent with the data of Meyer et al. [21], which, when reanalyzed, shows that variability proportional to the amplitude of the submovement. This linearity is predicted from the ballistic MT equation, as shown in the Appendix. Fitts’ original MT includes not only the pure MT, but also VT; therefore, the VT is added into (7) to test the result for the original Fitts’ scenario. The VT should be negatively and linearly proportional to log2 W; therefore, the probability model can be modified to be √ MTm o del (ID) = a + b1 A + (k + s)P2 (ID) +

n 

(i − 1)(k + s)Pi (ID) − b2 log2 W

(8)

3

where n is the maximum number of submovement, which is 7 in the current experiment, and W is the width of the target. The model results calculated from (8) (MTm o del in Table II) versus the experimental data (MTFitts in Table II) are plotted in Fig. 3, which shows that the MT predicted by the model is very close to the actual time. A t-test indicates (see Table II) that the regressions of predicted and actual MT data are not significantly different (t = 2.02, p = 0.06). The two are strongly correlated (Pearson correlation = 0.996; p < 0.001) as well. It, therefore, appears that the probability model may give very good prediction of MT, based purely on the time for the various components of the movement. The regression equation is MT(ID) = 165 + 101.5 (ID) ,

r2 = 0.977.

(9)

The question then arises of how (8) is related to the simple relationship we know as Fitts’ law. Although this cannot be shown analytically, the following explanation is at least a partial answer. If the MT beyond the first submovement is MT2 (ID),

this is given by MT2 (ID) = (k + s)P2 (ID) +

n 

(i − 1)(k + s)Pi (ID)

i=3

(10) and this varies as MT2 (ID) ∼ log2 (ID). In√addition, numerically (typically with r2 about 0.96), log2 A ∼ A. Then MT(ID) = a + b1 log2 A + b3 log2 (2A/W ) − b2 log2 W. (11) It is possible to rewrite this expression in the simple form of Fitts’ law where there is an “effective target width” [4], [6] MT(ID) = a + b log2 (2A/We )

(12)

c

where the effective target width We = W is less than the set target width (W). The empirical constant c = (b2 +b3 )/(b1 +b3 ), with b1 , b2 , b3 > 0, requires that b2 be greater than b1 . There is much research demonstrating that subjects generally do not use the full target width available and that higher correlations between MT and ID are obtained when using this effective target width [21], [34]. V. CONCLUSION This study has investigated the distribution of MT and the probabilities of submovements as a function of ID, in order to explain the underlying mechanism of Fitts’ law. A computerbased mouse movement was adopted. The trajectory of the movement was recorded and analyzed to identify the distributions of the submovement durations and the overall MTs. Given the assumptions of the Fitts’ speed/accuracy tradeoff for conducting the experiment and averaging data across all subjects for analyses, results have shown the probabilistic characteristics of movements. Analyses showed that information about the probability structure of submovements can be obtained from the two complimentary models developed in this paper. The number of submovements in this mouse movement task is significantly larger than those reported by Walker et al. [23], which showed the different effect of the resolution of the equipment in a mouse tracking task. The IT showed a strong linear relationship with movement amplitude, A, rather than ID [2], [22], [23]. The difference is likely to be due to the way in which the targets were presented: in [2], there was a choice reaction of movement direction involved; in [22], the task was essentially reciprocal with the next target being presented when the prior target was captured. The method used in [23] was essentially the same as in the present experiment, but no significant variation with movement amplitude was found. The probability model can be compared to, and merged with, the feedback control model of Crossman and Goodeve [8]; it is in fact a probabilistic version of that model. The microstructure of the movement is clearly decomposed into parts that are described by the feedback model as “pulses,” which means the submovement model can be validated by the current parsing method and data. A more specific form of the feedback formula with precise gains can be derived as well. In the microstructure, besides the first submovement following the ballistic movement paradigm, the second submovement

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

duration has extended time compared with later submovements and hence may be performed ballistically. This relationship is not clear and needs further investigation. In addition, the pause time before the second submovement does not have a very strong linear relationship with log2 (A) with one outlier at the lowest amplitude level. Later submovement times and pause times also have similar patterns, but these patterns degrade with increasing number of submovements. These relationships may give more insight into the microstructure. Further work is suggested for investigating these as well as later submovement times and the patterns of pause time. The experiments show that the model of Meyer et al. [21] is not correct due to the high correlation of the first submovement time with the square root of movement amplitude. This relationship shows the strong ballistic nature of the first submovement [11] with the MT of that component having very high correlation (0.99) with the square-root amplitude of movement. This is contrary to the Meyer et al. model in which it is assumed that the first submovement is made in a constrained time, which is the condition for which Schmidt’s law [24] is applicable. There are other effects that have not been studied in this research, for example, the effects of gain between the mouse input and screen output. It has been shown that control gain can have a significant effect of performance in the Fitts paradigm [40]. The importance of studying structure of a movement with ongoing visual control has been demonstrated in this research. To a large extent, the knowledge obtained is of a theoretical nature, but it is suggested that there are practical applications arising in evaluation of computer devices. An example comes from the recent research of Lin and Tsai [36], who used a twosubmovement model to evaluate input devices. They were able to show that the variability error at the end of the first submovement was a critical factor in the time taken for capture of a target. Devices with a higher variability of endpoint at the end of the first submovement took longer to complete a movement. They were also able to show that, with reasonable accuracy, the twosubmovement model could give a good simulation of the times for target capture in a Fitts’ task. The current method would allow extension of the method to higher submovement numbers. APPENDIX END-POINT VARIABILITY OF BALLISTIC MOVEMENTS We start with the equation for ballistic movements [11], [12], [35], derived on the assumption of a sinusoidal variation of muscle torque with time. This equation is, apart from the multiplying constant (2π), identical for all assumptions of muscle torque variation; the only requirement is that the limb come to a stop at the target. This implies that the impulse at the accelerating phase of the motion is the same as that for the decelerating phase of the motion. It is shown by direct application of Newton’s equations of motion  2πIA or Tm (MT)2 = const I A. (13) MT = LTm Here, “I” is the mass moment of inertia of the moved limb, L is the length of the limb, A is the amplitude of the movement,

797

and Tm is the maximum torque (assumed to be independent of A) applied to the limb. For small variations in the variables Tm and MT (fixed “I”), with σ (Tm ) = k1 Tm

and

σ (M T ) = k2 M T

(14)

the effect of these variations on the variability of the final amplitude can be shown to be  We = A 4k12 + k22 = const × A (15) where We is the standard deviation of hits at the target. REFERENCES [1] P. M Fitts, “The information capacity of the human motor system in controlling the amplitude of movement,” J. Exp. Psychol., vol. 47, pp. 381–391, 1954. [2] P. M. Fitts and J. R. Peterson, “Information capacity of discrete motor responses,” J. Exp. Psychol., vol. 67, pp. 103–112, 1964. [3] A. K. Mithal and S. A. Douglas, “Differences in movement microstructure of the mouse and the finger-controlled isometric joystick,” in Proc. SIGCHI Conf. Human Factors Comput. Syst., Vancouver, BC, Canada, Apr 13–18 1996, pp. 300–307. [4] International Standardization Organization, ISO9241-9: Ergonomic Design for Office Work with Visual Display Terminals (VDTs)—Part 9: Requirements for Non-keyboard Input Devices, 2000. [5] J. G. Philips and T. J. Triggs, “Characteristics of cursor trajectories controlled by the computer mouse,” Ergonomics, vol. 44, no. 5, pp. 527–536, 2001. [6] I. S. MacKenzie, “Fitts’ law as a performance model in human-computer interaction,” Ph.D. dissertation, Univ. Toronto, Toronto, ON, Canada, 2001. [7] E. R. F. W. Crossman, “The measurement of perceptual load in manual operations,” Unpublished Doctoral Dissertation, Univ. Birmingham, Birmingham, U.K., 1956. [8] E. R. F. W. Crossman and P. J. Goodeve, “Feedback control of handmovement and Fitts’ law,” Commun. Exp. Psychol. Soc. Reprinted Quart. J. Exp. Psychol., vol. 35A, pp. 251–278, 1963. [9] S.W. Keele and M. I. Posner, “Processing of visual feedback in rapid movements,” J. Exp. Psychol., vol. 77, pp. 155–158, 1968. [10] L. G. Carlton, “Processing visual feedback information for movement control,” J. Exp. Psychol.: Human Perception Perform., vol. 7, pp. 1019–1030, 1981. [11] K.-C. Gan and E. R. Hoffmann, “Geometrical conditions for ballistic and visually controlled movements,” Ergonomics, vol. 31, pp. 829–839, 1988. [12] E. R. Hoffmann and K.-C. Gan, “Directional ballistic movement with transported mass,” Ergonomics, vol. 31, no. 5, pp. 841–856, 1988. [13] R. F. Lin and C. G. Drury, “Verification of models for ballistic movement time and endpoint variability,” Ergonomics, vol. 6, no. 4, pp. 623–636, 2013. [14] R. S. Woodworth, “The accuracy of voluntary movement,” Psychol. Rev., vol. 3, pp. 1–119. 1899. [15] L. G. Carlton, “Control processes in the production of discrete aiming responses,” J. Human Movement Stud., vol. 5, pp. 115–124, 1979. [16] L. G. Carlton, “Movement control characteristics of aiming responses,” Ergonomics, vol. 23, pp. 1019–1032, 1980. [17] R. J. Jagacinski, D. W. Repperger, M. S. Moran, S. L. Ward, and B. Glass, “Fitts’ law and the microstructure of rapid discrete movements,” J. Exp. Psychol.: Human Perception Perform., vol. 6, pp. 309–320, 1980. [18] L. G. Carlton, “Visual processing time and the control of movement,” in Vision and Motor Control, L. Proteau and D. Elliott, Eds. New York, NY, USA: Elsevier, 1992, pp. 3–31. [19] J.-F. Lin, C. G. Drury, C.-M. Chou, Y.-D. Lin, and Y.-Q. Lin, “Measuring corrective reaction time with the intermittent illumination model,” Lecture Notes Comput. Sci., vol. 6761, pp. 397–405, 2011. [20] R. F. Lin and C.-H. Hsu, “Measuring individual corrective reaction time using the intermittent illumination model,” Ergonomics, vol. 57, pp. 1337–1352, 2014. [21] D. E. Meyer, R. A. Abrams, S. Kornblum, C. E. Wright, and J. E. K. Smith, “Optimality in human motor performance: Ideal control of rapid aimed movements,” Psychol. Rev., vol. 95, pp. 340–370, 1988.

798

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 6, DECEMBER 2015

[22] M. Liao, R. J. Jagacinski, and N. Greenberg, “Quantifying the performance limitations of older and younger adults in target acquisition task,” J. Exp. Psychol.: Human Perception Perform., vol. 23, pp. 1644–1664, 1997. [23] N. Walker, D. E. Meyer, and J. B. Smelcer, “Spatial and temporal characteristics of rapid cursor-positioning movements with electromechanical mice in human-computer interaction,” Human Factors, vol. 35, no. 3, pp. 431–458, 1993. [24] R. A. Schmidt, H. Zelaznik, B. Hawkins, J. S. Frank, and J. T. Quinn, “Motor-output variability: A theory for the accuracy of rapid motor acts,” Psychol. Rev., vol. 86, no. 5, pp. 415–451, 1979. [25] E. A. Andreeva and V. I. Chernov, “Control of plane motions of a joint angle,” Autom. Remote Control, vol. 33, pp. 1162–1175, 1972. [26] R. N. Stiles, “Mechanical and neural feedback factors in postural hand tremor of normal subjects,” J. Neurophysiol., vol. 44, no. 1, pp. 40–59, 1980. [27] R. B. Stein and R. G. Lee, “Tremor and clonus,” in Handbook of Physiology: Section 1: The Nervous System Volume II, Parts 1 & 2: Motor Control. V. B. Brooks, Ed. Bethesda, MD, USA: Amer. Physiol. Soc., 1988, pp. 325–343. [28] S. C. Chapra and R. P. Canale, Numerical Methods for Engineers. New York, NY, USA: McGraw-Hill, 2006. [29] A. T. Welford, Fundamentals of Skill. London, U.K.: Methuen Press, 1968. [30] H. Munro, M. S. Plumb, A. D. Wilson, J. H. G. Williams, and M. MonWilliams, “The effect of distance on reaction time in aiming movements,” Exp. Brain Res., vol. 183, pp. 249–257, 2007.

[31] S. Bouisset and F. Lestienne, “The organization of a simple voluntary movement as analysed from its kinematic properties,” Brain Res., vol. 71, pp. 451–457, 1974. [32] S. G. Thompson, D. S. McConnell, J. S. Slocum, and M. Bohan, “Kinematic analysis of multiple constraints on a pointing task,” Human Movement Sci., vol. 26, pp. 11–26, 2007. [33] P. J. Cordo, “Mechanisms controlling accurate changes in elbow torque in humans,” J. Neurosci., vol. 7, no. 2, pp. 432–442, 1987. [34] C. G. Drury and E. R. Hoffmann, “A model for movement time on dataentry keyboards,” Ergonomics, vol. 35, no. 2, pp. 129–147, 1992. [35] E. R. Hoffmann, “An ergonomics approach to predetermined motion time systems,” in Proc. 9th Ann. Conf. Australian Inst. Ind. Eng., Singapore, 1981, pp. 33–47. [36] R. F. Lin and Y.-C. Tsai, “The use of ballistic movement as an additional method to assess performance of computer mice,” Int. J. Ind. Ergonom., vol. 45, pp. 71–81, 2015. [37] A. B. Slifkin and K. M. Newell, “Is variability in human performance a reflection of system noise,” Current Directions Psychol. Sci., vol. 7, pp. 170–177, 1998. [38] J. Wu, J. Yang, and T. Honda, “Fitts’ law holds for pointing movements under conditions of restricted visual feedback,” Human Movement Sci., vol. 29, pp. 882–892, 2010. [39] J.-F. Lin and C. G. Drury, “Modeling Fitts’ law,” presented at the 9th Pan-Pacific Conf. Ergonom., Kaohsiung, Taiwan, Nov. 7–10, 2010. [40] M. L. Lin, R. G. Radwin, and G. C. Vanderheiden, “Gain effects using a head-controlled computer input device,” Ergonomics, vol. 35, pp. 159–175, 1992.