Adjunct Proceedings of the 9th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’17), September 24–27, 2017, Oldenburg, Germany.
Multi-Level Force Touch Discrimination on Central Information Displays in Cars Jochen Huber
Abstract
Synaptics
Multi-level force input on touchscreens in the car has the potential to increase input richness and to address scalability issues due to the increasing number of features and. However, the number of levels and the corresponding force threshold is not known. We built a center console prototype that features a 10” forcesensitive touchscreen with haptic feedback to study the user perceptual sensitivity in discriminating force differences. We contribute results from an extensive controlled experiment with 20 participants, investigating the number of practical levels of force and their corresponding thresholds. We also contribute design implications for future force-enabled touch HMIs in the center console.
Zug, Switzerland
[email protected] Mohamed Sheik-Nainar Synaptics Inc. San Jose, CA 95131, USA
[email protected] Nada Matic Synaptics Inc. San Jose, CA 95131, USA
[email protected]
Author Keywords
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. AutomotiveUI '17 Adjunct, September 24–27, 2017, Oldenburg, Germany © 2017 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-5151-5/17/09…$15.00 https://doi.org/10.1145/3131726.3132043
Automotive; Touch; Force touch; Force; Experiment; User sensory threshold; Pressure-based input.
CCS Concepts H.5.2. Information interfaces and presentation: User Interfaces
Introduction Over the past years, touch input technology has become pervasive in automobiles, enabling interactions with displays, virtual switches and capacitive buttons. Today, touchscreen technology is an integral part of the
Interactive Demos and Industrial Showcases
AutomotiveUI Adjunct Proceedings ’17, Oldenburg, Germany
central information display (CID), with a forecast shipment volume beyond 50 million units in 2017 [8]. However, the usability of in-car touch input is limited by (a) the increasing number of features in the car and (b) road conditions that impact the user performance [2,5]. Researchers and manufacturers have started to address these challenges by investigating ways to scaffold touchscreen interaction. One avenue of research focuses on predicting the interface target by analyzing the target acquisition process (i.e. hand movement towards the display) as so-called intent-aware user interfaces [1,3]. Another line of work seeks to extend touchscreen input by adding a single level force input for discrete actions such as direct finger selection of items or button activation [11] and distant interactions [10]. Beyond that, multi-level force input in the car has the potential to increase input richness [6,7], e.g. for virtual multi-stage push button switches. Nonetheless, there many questions that need to be answered: How many levels of force can users discriminate reliably on force-sensitive CID touchscreens? What are appropriate force thresholds for interaction and do they hold for driving and non-driving situations equally?
Figure 1. Simulator setup overview (top) and center console prototype (bottom)
We built a center console prototype that features a 10” force-sensitive touchscreen with haptic feedback (see Figure 1) to understand the user experience of forceenabled touch interaction with the CID. We studied the user perceptual sensitivity in discriminating force differences for CID interaction in an extensive controlled experiment with 20 participants. We investigated the number of practical levels of force and their corresponding thresholds in two conditions: while
223
driving and being parked. Based on the discussion of the results, we contribute design implications for future force-enabled touch HMIs in the center console.
Controlled Experiment We conducted a controlled experiment using a driving simulator setup, running Mattes’ Lane Change Task software [9]. The setup was comprised of the center console prototype with a force-sensitive touchscreen and haptic feedback through solenoids. The force report range was up to 12N. The setup also included a driving seat, a Logitech G27 steering wheel and a 27” display. Experiment Design and Method Depending on the condition, participants were either asked to drive in the driving simulator or remain parked. The driving condition followed the lane change task methodology. Participants drove on a three-lane course of which two lanes were closed. Virtual signs prompted participants to drive into the open lane, resulting in frequent lane changes. In addition, a force discrimination task was presented on the CID using a two-alternative forced choice method (inspired by [6]). Participants were presented pairs of stimuli and had to rate whether the stimuli could be activated using similar or different force. This similarity rating for the stimuli allowed us to estimate the “just-noticeable-difference” (JND) [4] in terms of force activation. The JND is considered at a similarity rating of 50%. A similarity rating of 25% is used as a threshold to consider two stimuli different, i.e. at least 75% of the participants reported the stimuli pair to be different. These estimations are used to derive practical levels of force and their corresponding thresholds.
Interactive Demos and Industrial Showcases
AutomotiveUI Adjunct Proceedings ’17, Oldenburg, Germany
The force discrimination task was implemented as follows: A series of two consecutive buttons were displayed on the CID touchscreen. Each button had to be activated by applying force on the touchscreen. Targets presented were initially gray. Upon applying force equal to or above the preset threshold, the color changed to blue and both haptic and auditory feedback were provided (see Figure 2.1). First a circular button was displayed, then a rectangular button (Figure 2.1 and 2.2 respectively). Eventually, the participant was prompted to rate whether the applied force felt similar or different per button pair (Figure 2.3), by verbally reporting to the experimenter. Participants also had the option to redo the task as often they liked before providing a response.
1
2
3
Figure 2. (1) Shows the first prompt of a button pair after the threshold was successfully hit, turning the circle blue and providing both haptic and auditory feedback. (2) Shows the second prompt of a button pair (rectangle) and (3) illustrates the non-interactive prompt, asking the participant to respond verbally whether the stimuli were similar, different or the option to redo.
224
The force thresholds for each button pair were computed based on a nominal force, ranging from low forces of 100g, 200g and 300g to high forces of 500g and 700g, which were adjusted by a difference factor (0%, 10%, 20%, 30%, 50%, 70%) as illustrated in Table 1. In sum, the independent variables were (i) nominal forces, (ii) percent differences and (iii) driving vs. parked. The dependent variable was the similarity rating, task completion time and touchpoint data. Pairs were presented in random order and each button pair was repeated twice with order balancing. The study was conducted with 20 participants in singleuser sessions (10 female, 3 left handed). The participants had an average driving experience of 18 years (σ = 8y), an average car usage of 6 days per week (σ = 1.5d), 25% had a touchscreen in their car and 70% had prior exposure to force touch input (e.g. through their smartphone). The age was distributed as follows: 3 participants under 30, 9 between 30 and 40, 5 between 40 and 50 and 3 above 50 years of age. Each session lasted about 45 minutes. Participants were given ample time to test drive the simulator and tryout the tasks. The study commenced once the participants felt comfortable. In total, 2400 trials (5 nominal forces * 6 percent differences * 2 conditions * 2 repetitions * 20 participants) were recorded and a total of 272790 touch points logged. Pauses between button pairs were randomized between 2 and 5 seconds. The force sensors were calibrated to an accuracy of >98% before the beginning of each participant.
Interactive Demos and Industrial Showcases
AutomotiveUI Adjunct Proceedings ’17, Oldenburg, Germany
Table 1. Overview of nominal forces and percent differences used. Each cell shows the force stimuli pair.
Table 2. Average similarity ratings across all participants as well as driving and parked conditions.
Nominal Forces [g] 200
300
500
700
0% 100 / 100 200 / 200 300 / 300 500 / 500 700 / 700 10% 95 / 105
190 / 210 285 / 315 475 / 525 665 / 735
20% 90 / 110
180 / 220 270 / 330 450 / 550 630 / 770
30% 85 / 115
170 / 230 255 / 345 425 / 575 595 / 805
50% 75 / 125
150 / 250 225 / 375 375 / 625 525 / 875
70% 65 / 135
130 / 270 195 / 405 325 / 675 455 / 945
Results and Discussion Table 2 shows the average similarity ratings across all participants, i.e. the percentage of participants who rated stimuli as “similar”. The colors in table 2 are mapped as follows: Dark green indicates discriminable cases (similarity 75%). The analysis yielded no significant main effect for the driving vs. parked condition. A significant interaction between nominal forces and percent differences was observed, F(20,380) = 3.432 (p < .001). Also, the analysis revealed significant main effects for both nominal forces (p < .05) and percent differences (p < .001). Similarity Ratings The pair-wise analysis of the average similarity ratings revealed two main themes: (1) the higher the nominal
225
Percent Difference
Percent Difference
100
Nominal Forces [g] 100
200
300
500
700
75.00%
71.25%
66.25%
56.25%
55.00%
10% 76.25%
63.75%
65.00%
56.25%
45.00%
20% 72.50%
67.50%
65.00%
57.50%
52.50%
30% 58.75%
61.25%
45.00%
40.00%
30.00%
50% 66.25%
53.75%
41.25%
25.00%
20.00%
70% 65.00%
41.25%
28.75%
18.75%
3.75%
0%
force, the less likely a user perceives two identical stimuli as similar, and (2) the larger the absolute force difference (i.e. 500g nominal force with 70% difference yields a stimuli pair of 325g/675g), the more likely a user perceives two stimuli as not similar. Based on the latter observation, we plotted the average similarity ratings to the absolute force differences as illustrated in Figure 3. A linear model fit (R² = .86, p < .001) estimates the JND at an absolute difference of 118g, whereas at least a 292g difference is estimated for two stimuli to be considered different (25% rating). Furthermore, the model predicts a 10% rating at an absolute difference of at least 397g and a rating of 0% at a difference of 460g. Thus, we hypothesize that a maximum of 4 force levels across a force spectrum of 0N-12N might be possible with 292g between levels. In addition, taps could be disambiguated based on the temporal signature of the touch data.
Similarity Rating
Interactive Demos and Industrial Showcases
AutomotiveUI Adjunct Proceedings ’17, Oldenburg, Germany
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 0
100
200
300
400
500
Absolute Force Difference [g]
292g (25%)
118g (JND)
397g (10%)
460g (0%)
Figure 3. Average similarity ratings plotted against absolute force differences in Grams. Line indicates linear fit (R² = .86, p < .001)
Touchpoint Data Analysis The analysis of the touchpoint data showed that the actual applied forces by the participants have a high correlation with target forces (r = .8, p < .001). No extreme over- or under-shooting was detected. Furthermore, we investigated the efficiency of the user performance per absolute force difference. Figure 4 illustrates the recorded task time per absolute target force. Our analysis revealed a low impact of the absolute target force on the task time (r = .17 for first target, r = .34 for second target, p < .001). The same observation holds for the actual applied forces by the participants. We therefore conclude that multi-level force interactions on touchscreens do not impact the efficiency of the user performance considerably.
Based on our results, we identified force thresholds for multi-level force input. Differences found between the conditions of driving and being parked were not significant. Our findings suggest no considerable impact of multi-level force input on the efficiency of the user performance. We hypothesize that the maximum number of force levels for discrete actions depends on the available force report range (e.g. 4 force levels in addition to ordinary touch input for a range of 12N). Our results provide guidelines for implementing these based on absolute force differences, which we aim to investigate for the implementation of virtual multistage button switches. We also hypothesize that performing multi-level force input will not have considerable impact on input efficiency.
Summary
The results highlight the potential of multi-level force input on the CID that goes beyond prior work which focused on emulating single button pushes [11]. We hope that our work is helpful for researchers and practitioners aiming to increase input richness of touchsensitive surface in the automobile such as touchscreens.
In this paper, we explored multi-level force input on a force-sensing touchscreen in the center console. We contributed results from a controlled experiment with 20 participants that explored the user sensory threshold in discriminating force differences while driving and being parked.
226
600
Interactive Demos and Industrial Showcases
AutomotiveUI Adjunct Proceedings ’17, Oldenburg, Germany
Figure 4. Illustration of recorded task time per absolute target force (bold lines indicate median values).
References 1. Bashar I. Ahmad, Simon J. Godsill, Lee Skrypchuk, Patrick M. Langdon, and Robert Hardy. 2015. Intelligent in-vehicle touchscreen aware of the user intent for reducing distractions: a pilot study. Adjunct Proceedings of AutomotiveUI '15, ACM, 2–7. 2. Bashar I. Ahmad, Patrick M. Langdon, Simon J. Godsill, Robert Hardy, Lee Skrypchuk, and Richard Donkor. 2015. Touchscreen usability and input performance in vehicles under different road conditions: an evaluative study. Proceedings of AutomotiveUI '15, ACM, 47–54. 3. Ilhan Aslan, Alina Krischkowsky, Alexander Meschtscherjakov, Martin Wuchse, and Manfred Tscheligi. 2015. A leap for touch: proximity sensitive touch targets in cars. Proceedings of AutomotiveUI '15, ACM, 39–46. 4. Gustav Fechner. 1966. Elements of psychophysics. 5. Natassia Goode et al. 2012. The impact of on-road motion on BMS touch screen device operation. Ergonomics 55, 9: 986–996. 6. Jochen Huber, Benjamin Petry, Mohamed SheikNainar, and Nada Matic. 2016. Enhancing Touch Control on the Steering Wheel with Force Input.
227
Proceedings of the 23rd SID Vehicle Display Symposium, SID, 15-20. 7. Jochen Huber, Mohamed Sheik-Nainar, and Nada Matic. 2016. Towards an Interaction Language for Force-enabled Touchpads in Cars. Proceedings of AutomotiveUI '16, ACM, 197-202. 8. IHS Markit. 2017. Automotive Touch Screen Shipments to Top 50 Million Units in 2017. Retrieved May 30, 2017. http://news.ihsmarkit.com/pressrelease/technology/automotive-touch-screenshipments-top-50-million-units-2017-ihs-markitsays. 9. Stefan Mattes. 2003. The lane-change-task as a tool for driver distraction evaluation. Quality of Work and Products in Enterprises of the Future: 57–60. 10.Alexander Ng, Stephen A. Brewster, Frank Beruscha, and Wolfgang Krautter. 2017. An Evaluation of Input Controls for In-Car Interactions. Proceedings CHI '17, ACM, 2845–2852. 11.Hendrik Richter, Ronald Ecker, Christopher Deisler, and Andreas Butz. 2010. HapTouch and the 2+1 state model: potentials of haptic feedback on touch based in-vehicle information systems. Proceedings Automotive'10, ACM, 72–79.
