INTERNATIONAL JOURNAL OF HUMAN–COMPUTER INTERACTION, 18(1), 19–38 Copyright © 2005, Lawrence Erlbaum Associates, Inc.
Development of a Tongue-Operated Switch Array as an Alternative Input Device Dongshin Kim Mitchell E. Tyler David J. Beebe Department of Biomedical Engineering University of Wisconsin–Madison
This article presents a tongue-operated switch array (TOSA) that provides not only an alternate input for a computer or operative system, but also an approach for silent and hands-free communication among humans or between human and machine. A TOSA has been designed and fabricated using printed circuit board technology and a membrane-switching mechanism and is integrated with a dental palate mold made from a silicone impression material. The TOSA has 5 switches (4 switches are laid out in cardinal directions and a fifth switch is located in the center). Human participant experiments have been conducted to evaluate and improve device performance. The characteristics of tactile sensation and mobility of the tongue are used to quantify the performance and optimize the geometric design of the TOSA. The results from controlled studies using repeated measures with 4 participants revealed a maximum average accuracy of 91% with SD = 5 in a switch depression task and a maximum repetition rate of 2.47 depressions/sec (SD = .21). These results indicate that operation on all switches is highly accurate and fast enough for use as an alternate input device.
1. INTRODUCTION If we cannot use our hands in operating devices such as a computer or motorized wheel chair, what other organ can be substituted for hands? This question may be very significant to quadriplegics or those who need an alternative to manual or voice-actuated inputs for tasks such as steering a wheelchair or navigating a virtual environment (Nutt, Arlanch, Nigg, & Staaufert, 1997). Scuba divers or soldiers performing military action might need a method for hands-free or silent communica-
This work is supported by Office of Naval Research Grant N0001495RF55555. The authors extend their gratitude to Abhishek K. Agarwal and Matthew Delisle for their technical assistance in fabrication and integration of the TOSA prototypes. Requests for reprints should be sent to David J. Beebe, Room 2142 Engineering Centers Bldg, 1550 Engineering Drive, Madison, WI 53706. E-mail:
[email protected]
20
Kim, Tyler, Beebe
tion. According to the somatosensory map of the brain, a tactile-sensitive area similar in size to that of the hands is located in the oral cavity (Sherwoood, 2001). The tongue is especially capable of very precise, complicated, and elaborate movements and also possesses high tactile sensitivity. These attributes suggest that the oral cavity is well-suited for communication interfaces (Tang & Beebe, 2001). If a switching device can be designed to interact efficiently with the tongue, it can provide an alternate method as a solution to the aforementioned question. Figure 1 shows the concept of a tongue-operated device conveying on-off signals that can be expressed as a directional or clicking command similar to functions that computer mouse buttons and ball tracks generate. The device can be used by depressing corresponding switches with the tongue when the user is unable to use the hands or needs additional input methods during operations such as driving a motorized wheelchair or operating a computer. By providing an alternative input method, this device has a strong potential to enhance human–computer interaction by lowering the limitations of human input methods. Dagenais (1995) demonstrated that devices could be built into the oral cavity to detect contacts between tongue and palate by using an electropalatography (EPG). Using this device, during speech the tongue makes contact with the palatometer, a type of sensor plate, instead of the real palate. This study demonstrated that such devices may be used inside the mouth without interfering with normal oral activity. Despite the conceptual merits of such a device, only recently have a few types of tongue-operated input devices been developed. One example is an isometric pointing device called Tonguepoint, which is a miniaturized joystick mounted on a mouthpiece, adapted from the IBM Trackpoint™ used on some laptop personal computers (PCs). Preliminary research indicated that it could be used for simple pointing tasks and speaking with the device inside the mouth is feasible (Salem & Zhai, 1997). Another tongue-operated input device is the Tongue Touch Keypad from NewAbilities (Mountain View, CA). This is a retainer-like device that fits against the palate and has nine membrane keypads in the bottom side. It is designed to allow disabled persons to control wheelchairs, electric beds, computers, and so forth. The device communicates with a remote base unit through wireless RF transmission.
FIGURE 1
Concept diagram of a tongue-operated device.
Tongue-Operated Switch Array
21
Another prototype of a tongue-operating device is the tongue mouse made by the Institute of Microsystem Technology in Switzerland (Nutt et al., 1997). The prototype is made of 16 × 16 piezoelectric ceramic sensor strips with a flexible printed foil. The sensor module is mounted on a dental plate customized for each user. The sensor array measures the strength and position of the tongue touch, allowing a user to operate the mouse pointer on a computer by moving the tongue across the sensor area. Bach-y-Rita, Kaczmarek, Tyler, and Garcia-Lara (1998) developed a different kind of tongue device that transfers information from a computer to a human via a 49-point electrotactile stimulus array. They demonstrated the feasibility of the development of a tactile vision substitution system using an electrotactile display. Their research showed that electrical stimulation of the tongue results in a better perception when compared to electrotactile stimulation of the fingertip. Bach-y-Rita et al.’s device displays images from a TV camera onto the tongue. Although prior tongue-operating devices show promise, they tend to be expensive and require significant training to operate effectively. This study presents development of a tongue-operated switch array (TOSA) that is simple to use and cheap to manufacture because the TOSA is fabricated on a printed circuit board (PCB) and is comprised of inexpensive materials including Mylar film and adhesive tape. The fabrication process itself is also fairly simple and inexpensive. In addition, the TOSA does not require any complicated electrical circuitry that would increase fabrication cost. The cost would be comparable to that of a generic PC mouse. After fabrication of the device, three human participant tests and two modifications were completed step-by-step to achieve an optimal design and performance.
2. TOSA The TOSA is a type of pointing device that may function similar to a mouse, trackball, or the arrow keys found on a standard computer keyboard. The shape of the TOSA is similar to a 5-key membrane switch panel. By depressing membrane switches with the tongue, the TOSA can provide alternative computer input methods for those who are unable to use their hands or need an additional input mechanism besides the hands during a specific operation, such as quadriplegics, scuba divers, and drivers. The tongue operates in three functional modes during the performance of a task on the TOSA. The first mode is perception (scanning and locating the geometric position of a desired switch, using sensitive tactile sensors on the tongue). The second mode is manipulation (the movement of the tongue generated by the intrinsic and extrinsic muscles during operation). The last mode is force application (this is primarily contributed by one of the major extrinsic muscles, the Genioglossus). The aforementioned modes of the tongue are involved in the human experiments to quantify and characterize performance of the TOSA with respect to correctness and repeatability.
22
Kim, Tyler, Beebe
2.1. Preliminary Design Design considerations. The tongue is suitable for operating a device due to its strength and mobility (Mortimore & Douglas, 1996). However, due to the limited volume of the oral cavity, it is a difficult place to install and operate a human–machine interface. To overcome this disadvantage, the TOSA needed to have a simple mechanism that could be integrated into a small space. The layout of the switches needed to be simple (a cardinal orientation) to operate because all operations depend only on tactile sensing (no visual aid). The material of the TOSA needed to be biocompatible and waterproof in the salivary environment of the oral cavity. These requirements can be satisfied if a proper way to use the sensitivity and versatility of the tongue can be found. Mechanism. The mechanism of the TOSA is based on the concept of a membrane switch that does not require a long stroke and is easy to fabricate in small areas (see Figure 2). A typical membrane switch is a normally open, momentary contact device made of two layers of conductor separated by a buffer layer. The bottom or membrane layer in this design is a thin and pliable metallized Mylar membrane layer. This can be depressed by the tongue into a well in the insulation layer to make contact with the static layer and it returns to its original shape (open switch) after release. On the static layer are the electrical circuit patterns. The static layer is bonded to a rigid plate that provides the stable foundation for the device. Switch design. The switch of the TOSA is designed to make contact with the membrane layer. The switch has a circular boundary frame and interdigitated electrodes. The specifications of the switch are shown in Figure 3. The switch is electrically open until the metallized Mylar membrane layer makes contact with at least two neighboring electrodes, thereby closing the switch and completing the circuit. Air fills the well providing insulation between these two conductive layers. The membrane switch provides a multiplex and small layout and requires only logic
FIGURE 2
Diagram of the membrane switch showing layered construction.
Tongue-Operated Switch Array
FIGURE 3 (units: mm).
Switch
23
layout
level signals instead of high-level voltages. The membrane switch is designed to be moisture resistant and requires a low actuation force. The last two advantages are very important factors for the design of the TOSA because the TOSA should be waterproof and the tongue is only capable of producing a low actuation force. To specify the force threshold for switch actuation, it is necessary to determine the maximal pressure the tongue can produce. However, this pressure value is difficult to obtain. Sha, England, Parisi, and Strobel (2000) attempted to determine this value, but there was no description of the contact surface of the tongue with the measuring device in their experiments. To determine this value, if one assumes that the contacting area of the tongue is 19 mm in diameter when it makes the maximal force with a lingual force transducer (Sha et al., 2000), the maximal pressure (Pm) can be calculated, where F0 and A0 represent the maximal force and contacting area, respectively: Pm =
F0 F0 = = A0 πr0 2
29 (N ) æ 0.019 ö÷2 π ´çç ÷ è 0 ÷ø
= 102282.42 (Pa)
(1)
If the maximal pressure (Pm) is evenly distributed over all contacting areas, the local reacting force (F) on each switch, which is 5 mm in diameter, can be computed as follows: F = PA = Pπr 2 = 102282.42´ π ´0.0052 = 8.03 (N )
(2)
From the aforementioned result, a 5-mm diameter switch must be designed to operate at an applied force of approximately 8.0 N or less. The actual force requirements to operate the switches on the TOSA are reported in the latter part of this article.
Layout design. The preliminary version of the TOSA is shown in Figure 4. The width of the layout is 25 mm and the length is 32 mm, making it small enough to fit in the oral cavity. There are five switches, with five leads, which are .22 mm wide, and one common power lead, which is .4 mm wide. The common power lead
24
Kim, Tyler, Beebe
FIGURE 4 mm).
Layout of the tongue-operated switch array: Preliminary version (units:
is bigger than others so it is not easily broken. The contact terminals have a 1.27-mm pitch so that the TOSA can be connected with the 1.27-mm pitch Parlex™ laminated flat cables. All four peripheral switches are spaced 15 mm from the center one to help the user distinguish them when exploring the device with the tongue. The switches are arranged in a cardinal direction configuration (i.e., left, right, top, and bottom). The corners on the leads are smoothed to alleviate high concentrations of current density. The TOSA is fabricated on a PCB that is subsequently machine-cut into the shape of a “paddle.” It can be attached to a rigid frame or simply held by hand so that the user can easily place the device in the oral cavity in the best corresponding and comfortable position by adjusting the orientation of the frame or hand.
2.2. Fabrication The TOSA plate is fabricated using PCB kits from Kepro Circuit Systems, Inc. (St. Louis, MO). The five switches were laid out using AutoCAD in the cardinal directions, with a fifth switch in the center. The pattern was transferred to the PCB by photoresist development and the unwanted conductive material was removed by a subsequent copper etching process. Several configurations of the spacing layer thickness (3M double-sided adhesive tape, St. Paul, MN), metallized Mylar thickness (Dupont), and size of spheres (switch locater “nibs”) were tested to determine the optimal ones for use on the TOSA. The cross-section of one of the switches of the TOSA is shown in Figure 5.
2.3. Integration The device is completed by integrating the TOSA with a dental palate mold (Figure 6) made from a silicone impression material (CutterSil Putty Plus, Heraeus Kulzer,
Tongue-Operated Switch Array
FIGURE 5
25
Cross-section of one of the switches on the TOSA.
Inc., South Bend, IN). Various combinations of the mold and hardener were attempted until a pliable, yet supportive substructure was created. The TOSA is mounted on the bottom side of the dental palate mold using 3M Super Glue gel (St. Paul, MN). The TOSA is then connected to a National Instruments DAQPad–6508, an external data acquisition device that communicates with the host computer via a USB port. The initial signal level of the input port is +5 V. All the ports are initially set to the pull-up status (pull-down and floating status are, in general, less stable).
2.4. Preliminary experiment Objective. A set of simplified psychophysical tests were conducted to perform a basic device characterization. The tests were designed to determine if the user could reliably and repeatedly locate and actuate the switches. The results would help determine what changes, if any, were necessary to the spatial configuration and to the switch size and construction.
Method. The tests were performed on three adult human participants over two trial sessions. To simplify the task, only four (one in each of the cardinal directions) of five switches were used during the operational testing of the TOSA. The
FIGURE 6
The tongue-operated switch array. (a) Top view. (b) Side view.
26
Kim, Tyler, Beebe
participants were instructed to use the TOSA to control the motion of a ball to traverse a straight path across a computer screen between two predefined target areas (at the end of each path) as shown in Figure 7. The step-wise motion of the ball was fixed at two pixels/activation of the TOSA. The time it takes the participant to reach each target area can then be used as an indirect indicator of the effectiveness of the TOSA switches (Agarwal et al., 2001).
Results. Participant performance data and analysis are shown in Figure 8. Data for movement in the “right” direction are not available due to device failure as participants attested that it was too difficult to activate the switch and could not complete the path. Subjective feedback on comfort, functionality, and feasibility of accessing and depressing all switches was recorded.
FIGURE 7 TOSA simulation environment. (a) Depressing the “left” switch, the ball is directed from Target 0 to Target 1. (b) Depressing the “up” and “right” switch, the ball continues to navigate around the square path.
FIGURE 8 The mean lap time taken by each participant to move the ball in each intended direction.
Tongue-Operated Switch Array
27
Preliminary tests of the operation and functionality of the TOSA showed that participants were able to manipulate the device and direct the ball around the square path. Participants could activate the “up” and “down” switches with greater ease compared to other switches, as demonstrated by the traversal times, but they reported difficulty in activating the “left” and “right” switches, which is exhibited in the incomplete performance times under this condition (see Figure 8). To determine the cause of these disparities in switch performance, a load cell was used to measure the minimum amount of force needed to activate each switch. The absolute values and the ratio of the forces (normalized to the “up” switch) are shown in Table 1. It is reasonable to assert that some of the disparity in the actuation forces between the “up/down” and “left/right” switches may be attributed to differences in participant performance in the tests. However, the overall trend in the data points to a larger issue in that it is not just the actuation force that affects performance, but also switch location relative to the tongue. Specifically, the results show that although the “up” switch was the easiest to actuate, overall participant performance was best on the “down” switch, which required nearly five times greater actuation force. Nonetheless, participants were able to adapt to the different force requirements and operate the device fairly successfully, although the “left/right” switches, on average, required far more force and participants had great difficulty spanning this range of force. The test results suggest the direction of the improvement in performance of the TOSA. The theoretical maximal force on a 5-mm diameter switch is 8.03 N (2). However, participants experienced difficulties at a much smaller force (i.e., 1.11 N). The reason seems to be that the participants cannot concentrate the necessary force with their tongue on the small switch area. From the experimental results, however, a smaller force is sufficient to operate the TOSA if the actuation threshold is limited to 0.78 N or less. This magnitude is possible if proper improvements are made on the TOSA.
2.5. Switch Redesign There are several factors that affect the performance of the TOSA. Having a low activating force on the TOSA is essential because the participant will otherwise experience difficulties during operation if the required activating force is too high. However, having the actuation force too low increases the possibility of inadvertent activation of more that one switch or the wrong switch. Additionally, the uniformity Table 1.
Force (Ratio)
Force Requirement to Activate Switches on the Early Tongue-Operated Switch Array Left
Up
Right
Down
1.11 N (6.7)
.17 N (1)
3.34 N (20)
.78 N (4.7)
Note. The number in parentheses indicates the relative ratio normalized to the “up” value, which is the lowest.
28
Kim, Tyler, Beebe
of the activating force across all switches is also important to allow the participant to perform the operations well. Finally, an ample linear distance between switches, as well as the overall location of the TOSAin the oral cavity, is essential for the user to be able to distinguish and accurately activate each switch consistently. Subsequent device design and fabrication modifications were made, including ensuring a uniform (Mylar) membrane tension across the TOSA and the inclusion of air channels between the switch well to alleviate differences in air pressure and required forces to activate the switches. The additional benefit of the air channels between the wells in the closed system is that as the air is displaced from the well of a depressed switch it is then distributed to the other switch wells, thereby increasing the local pressure at those switches and reducing the potential for unintentional activation. The diagram in Figure 9 shows the spacing layer with 1-mm wide air channels connecting with switch wells on the redesigned TOSA. The geometric layout is also modified to increase the distances between the switches to avoid perceptual confusion and allow greater focus of tongue force on a single switch. Finally, there was more attention to the accuracy of the amount of epoxy deposited on top of the switches. All of these modifications allow for greater device uniformity, thereby providing both lower and more uniform absolute magnitude of the forces required to activate the switches.
3. HUMAN SUBJECT EXPERIMENT Once modifications were completed, it was necessary to assess not only specific device characteristics, but also the performance of the TOSA in a simulated real-world task. Consequently, a set of psychophysical experiments needed to be developed to help quantify and characterize system performance as a whole. Although the studies were conducted to characterize the tongue operation on the TOSA, some results may have implications about the general characteristics of tactile perception and mobility of the tongue.
FIGURE 9 The air channels during the switches on the new TOSA.
Tongue-Operated Switch Array
29
3.1. Evaluation Software The TOSA evaluation program is designed to allow an experimenter to specifically create and order a sequence of TOSA tasks and record participant performance data. It is composed of two main categories that include both “correctness” and “repeatability” experiments. The front panel of the evaluation program has three sections (Monitoring, Operating, and Displaying zones). The Monitoring zone shows the status of each switch of the TOSA with five circular lamps that correspond to the switches on the TOSA. During an experimental trial, a participant is to depress the switch “targeted” by a square frame that flashes. Figure 10 shows the front panel in which the left switch is indicated and is being successfully depressed. The corresponding circular lamp becomes highlighted and the square frame immediately moves to the next switch in a predetermined random order. The Operating zone shows six buttons. The “TEST#1” and “TEST#2” buttons are for the Correctness experiment, which includes both the movement and repeatability tests. The “WARMING UP” button allows participants to practice the experimental environment by depressing switches several times. All test processes are stopped and canceled when the “STOP” button is depressed during an experiment. The “SAVE#1” and “SAVE#2” buttons will save the data of Correctness and Repeatability tests, respectively. Participant personal information is stored in the edit box below the operating zone and saved with the experimental data after each session. The Display zone is used to present Correctness and Repeatability data during each experiment and session. Figure 11 shows an example of an experimental procedure.
FIGURE 10 The front panel of the TOSA evaluation program.
30
Kim, Tyler, Beebe
FIGURE 11
Example of an experimental procedure.
3.2. Correctness Experiment The correctness experiment was designed to determine how often a participant can accurately depress the correct (designated) switch on the TOSA. This tests both the participant’s perception ability of the geometric location of the switch on the TOSA and the psychomotor ability to concentrate the tongue’s required force on that spot. For the experimental environment, five circular buttons were presented on the computer screen and a frame was drawn around the designated button to be depressed. The frame moved to another button immediately after the participant depressed the appropriate switch on the TOSA associated with that framed button. The movement order of the frame was randomized such that participants could not predict the next button to be depressed. Each participant was instructed to visually focus on the experimental panel on the computer screen, observe the framed button, and depress the corresponding switch on the TOSA as fast and accurately as possible. In cases where a participant depressed the wrong button, the frame would remain on the button until the right button was depressed. There were 2 sessions for this experimental section and 20 trials for each switch in each session. The results of this experiment will potentially reveal any performance limitations due to the location or configuration of the TOSA.
3.3. Repeatability Experiment The repeatability experiment was designed to measure the time it takes a participant to repeatedly depress the same switch on the TOSA. It quantified the participants’ ability to create and maintain a concentrated force on the switch as well as the tongue’s mobility across the TOSA. The experimental environment was similar to the correctness experimental setup in which five circular buttons were displayed on the computer screen and a frame was drawn around the button to be depressed. The frame moved to another button immediately after a participant correctly de-
Tongue-Operated Switch Array
31
pressed the marked button five times. Again, the frame moving order was programmed to move randomly so that participants could not predict the next button to be depressed. Each participant was instructed to visually focus on the experimental display panel on the computer screen, observe the framed button, and repeatedly depress the corresponding switch on the TOSA as fast and accurately as possible. In cases where a participant depressed the wrong button, the frame would remain on the button until the correct button was depressed five times successfully. There were two sessions for the repeatability test and three trials for each switch in each session.
3.4. Preparation and Sequences Before starting the experiments, the forces to activate the switches on the TOSA were individually measured by a load cell and the results are described in Table 2. The average activating force was .25 N (SD = .04). This is small enough to be manipulated by the tongue and less than the operable reaction force, .78 N. This is suggested from the results of the preliminary experiment, which showed that participants could operate the TOSA when the required force was below .78 N (Table 1 and Figure 8). The experimental application used the internal timer of the host computer with a sampling period of 50 ms (20 Hz), which is fast enough to accurately measure the maximal tongue reaction time—154 ms for normal humans (Dworkin, Aronson, & Mulder, 1980). Each participant used the same TOSA, which was cleaned both before and after each experiment with ethyl alcohol followed by a de-ionized water rinse. Before each session, the experimental system was inspected to ensure proper connection. Before the first session, each participant was informed about the purpose and the procedure of the experiments. Subsequently, each participant was instructed to practice by performing the “WARMING UP” procedure, which helps participants to understand and recognize the function of the TOSA. This preliminary practice took about 3 to 4 min. The formal experiments described in the previous sections are repeated for each human participant in two sessions. Each session (one of two) takes about 5 to 10 min to finish. Four normal male participants (mean age = 25.8 years [SD = 3.1], mean weight = 151.7 lb [SD = 21.4], and mean height = 69.7 in. [SD = 3.5]) participated in the experiment. Only adult men were used to eliminate the effects of tongue strength differences between men and women (Dworkin et al., 1980).
Table 2.
Force
Force Requirement to Activate Switches on the Intermediate Tongue-Operated Switch Array
Left
Up
Center
Right
Down
.22 N
.28 N
.28 N
.28 N
.17 N
32
Kim, Tyler, Beebe
3.5. Intermediate Results and Design Modification Evaluation of correctness. The correctness data of each switch are shown in Figure 12. For all four participants, the correctness ratio of the center switch, at .5, is the worst one, whereas the other switches are all greater than .8. The center switch data also show greater variability compared to the other switches. The probable reason for this poor performance is attributed to the difficulty in triggering the center switch because it is hard to distinguish it from the other switches, and it is difficult to concentrate the force exclusively in the center switch area. For the outer switches, the participants simply need to bend the tongue to the rightmost, leftmost, innermost, or outermost positions to access them. Consequently, performance on these switches showed fewer errors and less variability. A confusion matrix was constructed from the correctness data to evaluate the magnitude and type of errors. In Table 3, the first left column represents the switch designated to be depressed and each row represents the percentage of correct depressions. The confusion matrix shows that the center switch causes the most confusion across all switches. This indicates that the switches surrounding the center switch were often mistakenly triggered when the center one was supposed to be pressed. This is probably due to their proximity to the center switch. This conclusion is confirmed in the results shown in Figure 12, which indicated that the center switch is
FIGURE 12 Intermediate mean correctness of depressing each switch for all participants and trials. Error bars indicate standard deviation. Table 3.
Down Up Center Right Left
Confusion Matrix for the Intermediate Layout
Down
Up
Center
Right
Left
85 10 17 6 3
2 79 9 2 5
6 9 49 7 5
3 1 18 82 4
4 2 7 2 84
Tongue-Operated Switch Array
33
not easy to distinguish. This suggests that there should be a unique mark on the center switch so that it can be more easily localized and distinguished from the other switches (e.g., by attaching additional or larger “nibs”). The participants also reported it was hard to discriminate between the switches because there was no cue providing a reference tongue position to the other switches in the oral cavity. A unique mark on top of the center switch could also play a role as the reference cue.
Repeatability performance. To analyze switch-dependent factors, the average repeat time of each switch was calculated and shown in Figure 13. As before, the data indicate that performance on the “center” switch was the worst and the “up” switch was the second poorest. This is consistent with the correctness results. The average repeat time was .85 sec for all switches and .56 sec for the right, left, and up switches. The repeat time and error bars for the right and left switches were smaller compared to the other switches, indicating that the “right” and “left” switches were more easily found and depressed.
Design modification based on the intermediate results. When the correctness ratios were calculated for each switch, the worst was 50%, for the “center” switch, whereas the average of the other switches was approximately 82% (SD = 3). Two factors are thought to be the cause for this: (a) the difficulty of discrimination of each switch, especially “center,” and (b) the lack of a concentrated force on each switch. Most participants reported having a difficult time in concentrating their tongue on a switch. When the contact area between the tongue and TOSA becomes larger, the applying force on a switch gets smaller. Thus, the correctness ratio is decreased, which happens when participants try to depress the center switch. The discrimination difficulties are thought to be caused by confusion of the location of the center switch, and because neighboring switches are easily depressed. Consequently, if switches were spaced further apart, and if there was a unique mark (i.e.,
FIGURE 13 Intermediate mean depression repeat time of each switch for all participants and trials. Error bars indicate standard deviation.
34
Kim, Tyler, Beebe
the reference cue) on the center switch, the center would be distinguished more readily from the other switches by the tongue. In addition, the reference cue might improve the accuracy of performance on the other switches. Although these changes may improve performance on the “center” switch, a different problem arises in addressing the “up” switch, the second worst in terms of repeatability. The “up” switch is more difficult to depress than the others because of the mechanics of the tongue. When the tongue protrudes to operate the “up” switch, the flattened tongue shape reduces the force concentration, thereby making it harder to depress it compared to the others. To compensate, the diameter of the “up” switch was enlarged to 8.25 mm in the final layout so that users may more easily locate and depress it. The distance between both the “left” and “right” switches from center was widened by 2 mm and the diameter of “center” and “down” switches was also enlarged to 7 mm. The final geometric layout for the TOSA is shown in Figure 14.
3.6. Final Results Evaluation experiments were conducted on the final TOSA design using the same procedure as in the intermediate studies. Four normal male participants (mean age = 25 years [SD = 3.4], mean weight = 151.7 lb [SD = 21.4], and mean height = 69.7 in [SD =3.5]) took part in the experiment. Three of the four participants had also taken part in the intermediate studies. Because the fourth participant was not available for the final experiments, a fifth participant, who shared similar physical attributes with the fourth participant, was chosen. Before starting the experiments, the forces to activate the switches on the TOSA were individually measured by a load cell and the results are described in Table 4. The forces were not much different from that of the intermediate layout. The final results, presented in Figures 15 and 16, showed that participants were able to manipulate the device and direct the system to perform a task successfully. They also indicated that participants could easily and consistently activate all switches, which means the performance of the final design was
FIGURE 14
The final geometric layout for the tongue-operated switch array.
Tongue-Operated Switch Array Table 4.
Force
35
Force Requirement to Activate Switches on the Final Tongue-Operated Switch Array
Left
Up
Center
Right
Down
.28 N
.28 N
.28 N
.28 N
.39 N
FIGURE 15 Final mean correctness of depressing each switch for all participants and trials. Error bars indicate standard deviation.
FIGURE 16 Final mean depression repeat time of each switch for all participants and trials. Error bars indicate standard deviation.
greatly enhanced from the intermediate design. In particular, the correctness ratio was substantially improved, from 76% (SD = 13) to 91% (SD = 5), resulting in reduced participant confusion (see Table 5). Additionally, the repeatability results show that the repeat time was consistent between each switch and greatly reduced from .85 sec (SD =.4) to .41 sec (SD =.03). This is three times greater than the average minimum repeat time (.12 sec) reported by Dworkin et al. (1980) when participants spoke the syllable /t/. However, the probable reason for the difference in these repeat times is attributed to the different experimental methods. Here the repeat
36
Kim, Tyler, Beebe Table 5.
Down Up Center Right Left
Confusion Matrix for the Final Layout
Down
Up
Center
Right
Left
83 5 6 1 2
4 90 7 1 2
9 3 83 6 5
4 2 2 90 1
0 1 2 1 90
time was measured from the depression of the correct switch and involved both perceptive and motor processes to detect and depress the switch with the required force. Dworkin et al. measured the repeat time of the tongue when participants simply spoke one syllable quickly without involving a significant perceptive and actuating process. Table 6 summarizes all the experiments by indicating several experimental factors.
4. CONCLUSIONS AND FUTURE WORK The design, prototyping, optimization, and evaluation of an oral tactile interface have been described. Sending of commands by the tongue was made possible by the fabrication of a TOSA created by using PCB technology and a membrane-switching mechanism. Human participant experiments revealed the mobility of the tongue in quantifying the performance of the TOSA in a simulated task.
4.1. Overall Performance The results indicate that practice improves the versatility and skills of the tongue and therefore performance, although it is unknown how much practice is needed Table 6.
Summary of All Human Participant Experiments
Preliminary Participants
3 men
Tongue-Operated Switch Array Design Software and method Modifications
5-switch design (center switch was not used) Navigation Original design.
Intermediate
Final
4 men (two of them participated in the preliminary test) 5-switch design (all switches are used)
4 men (three of them participated in the intermediate test) 5-switch design (all switches are used)
Correctness and repeatability Air channels were added and the distance between the switches was increased.
Correctness and repeatability Diameter of the “up,” “down,” and “center” switch was enlarged and the distance between both the “left” and “right” switches from center was increased more.
Tongue-Operated Switch Array
37
and to what extent the tongue can be trained for specific tasks. The operating ability of the TOSA was also dependent on individual characteristics, wherein one participant showed good performance in both the correctness and repeatability tests, another participant showed a significant learning effect in the repeatability experiment. Although the overall performance of the TOSA was good, there was still some confusion caused by switch proximity and inability to discriminate them. In particular, if the reference cue on the center switch is marked more clearly, the total performance of the TOSA may be further increased and be more suitable for the general application. It should be noted that the resulting data came from male participants and the same trends may not directly transfer to female participants due to the weakness of the female tongue (Dworkin et al., 1980).
4.2. Future Work Integration with a two-way oral tactile interface. A potential application of the TOSA is to use the device as part of a two-way oral tactile communication system that does not burden the hands, vision, or hearing. This interface can also be integrated into currently developed rehabilitation devices such as the controller for a motorized wheelchair or a computer cursor for highly handicapped persons (Tang & Beebe, 1999, 2001).
Device for tongue-related research. More switches could be added so that a modified TOSA could be used in research related to tongue functions such as speech disorders, swallowing, and pathologic diagnoses of the tongue (Dagenais, 1995; Dworkin et al., 1980; Scardella et al., 1993). The fabrication method is easy and convenient to realize as a tool for those applications.
REFERENCES Agarwal, A. K., Kim, D., Delisle, M., Tang, H., Tyler, M., & Beebe, D. J. (2001). Two-way communication through an oral-based tactile interface: Preliminary results. Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Istanbul, Turkey, October 25–28. Bach-y-Rita, P., Kaczmarek, K. A., Tyler, M. E., & Garcia-Lara, J. (1998). Form perception with a 49-point electrotactile stimulus array on the tongue: A technical note. Journal of Rehabilitation Research and Development, 35, 427–430. Dagenais, P. A. (1995). Electropalatography in the treatment of articulation/phonological disorders. Journal of Communication Disorders, 28, 303–329. Dworkin, J. P., Aronson, A. E., & Mulder, D. W. (1980). Tongue force in normals and in dysarthric patients with amyotrophic lateral sclerosis. Journal of Speech and Hearing Research, 23, 828–837. Mortimore, I., & Douglas, N. J. (1996). Genioglossus strength and fatiguability: Relationship to apnea/hypopnea index. American Journal of Respiratory and Critical Care Medicine, 153, A532.
38
Kim, Tyler, Beebe
Nutt, W., Arlanch, C., Nigg, S., & Staaufert, G. (1997). Tongue-mouse for quadriplegics. Journal of Micromechanics and Microengineering, 8, 155–157. Salem, C., & Zhai, S. (1997, March). An isometric tongue pointing device. Paper presented at CHI 97—Conference on Human Factors in Computer Systems, Atlanta, GA. Scardella, A. T., Krawciw, N., Petrozzino, J., Co, M. A., Santiago, T. V., & Edelman, N. H. (1993). Strength and endurance characteristics of the normal human genioglossus. American Review of Respiratory Disease, 148, 179–184. Sha, B. F. B., England, S. J., Parisi, R. A., & Strobel, R. J. (2000). Force production of the genioglossus as a function of muscle length in normal humans. Journal of Applied Physiology, 88, 1678–1684. Sherwoood, L. (2001). Human physiology: From cells to systems. CA: Brooks Cole. Tang, H., & Beebe, D. J. (1999). An ultra-flexible electrotactile display for the roof of the mouth. Paper presented at the 21st International Conference of the IEEE Engineering Medicine Biology Society, Atlanta, GA. Tang, H., & Beebe, D. J. (2001, October). Two-way tactile communication through the oral tactile sense. Paper presented at the IEEE Transactions on Neural Systems and Rehabilitation Engineering, Atlanta, GA.
