Interfaces and Human Computer Interaction 2008

IADIS International Conference Interfaces and Human Computer Interaction 2008

A COMPARATIVE EVALUATION OF COLOR BLINKING AND ELECTRO-TACTILE IMAGING OF HIDDEN GRAPHS Tatiana G. Evreinova, Grigori Evreinov, Roope Raisamo Department of Computer Sciences University of Tampere FIN-33014 Tampere, Finland

ABSTRACT Developing special educational tool could facilitate accessing graphical information on PDA and desktop platforms for visually challenged users. We measured and compared the performance of ten blindfolded subjects with touch tablet where the stylus movements were augmented with color-blinks or electro-tactile signals stimulating transcutaneously the visual analyzer of participants when inspecting invisible graphic images. The performance was evaluated in terms of the both feedbacks used and the time spent to complete non-visual inspection of hidden graphs taking into account exploration behavior and the capture radius. Although the use of color blinking signals appeared to be more beneficial in this task, the electro-tactile signals might positively further grasping the primitive and composite attributes of the simulated graphic entities once the perceptible quality of the phosphene sensations elicited during electro-tactile stimulation of the retinal area had been bettered.

KEYWORDS Graph grasping creativity, visually challenged people, directional-predictive signals.

1. INTRODUCTION Visually challenged people experience significant problems when trying to access the graphic information presented on desktop and ultra-mobile devices. The graphic information cannot easily be transformed or decomposed into discrete components, which could have a particular sound denotation without losing their integrity and semantic meaning. Compared to tactile and sonification display techniques, electro-tactile imaging is significantly advantageous in accessing and manipulating information regarding specific relationships between values of the graphical data arrays, and their general features. In our earlier papers [Evreinova et al., 2007; Evreinova et al., 2008] we have explored the use of stylus movements augmented with sound, vibro-tactile and electro-tactile feedback signals as the possible alternatives to cross-modal coordination in the absence of visual information based on dynamical visualization of the user activity. Given the comparison of the results obtained, we found out that the assistive electro-tactile signals with well-adjusted perceptual parameters are the most beneficial for that purpose. Provided through stylus with attached double-strip ring-type electrodes, the electro-tactile signals gave a rise to a clear, painless, perceptually controlled tactile imaging of the graphical features and did not require great cognitive efforts to encode the meaning of the graphical information being presented to the user. On the other hand, several research studies on applying electrical stimulation to the retinal area of the eye, have proven that the evoked visual sensations (phosphene phenomenon) are being easily perceivable and distinguishable from each other (Brindley and Lewin, 1968; Carpenter, 1972). The results reported in studies done by Humayun et al. (1999) on the invasive medical attempts undertaken to artificially stimulate the occipital cortex of the blind patients confirm partial recovering of the visual sensation such as the possibility of light perception, motion detection, and even discrimination of simple shapes. However, Bindley and Lewin (1968) stated that the levels of safe electrical stimulation and the underlying mechanisms of cellular damage are still unknown. Being a painless alternative in noninvasive treatment, with the low rate of side effects reported in Humayun et al. (1999), microcurrent therapy still is very costly to be used over prolonged period of time [Hambrecht, 1992].

109

ISBN: 978-972-8924-59-1 © 2008 IADIS

Yet, in series of our previous studies [Evreinova et al., 2003], we have introduced the peripheral monitor providing the visually challenged persons with real-time imaging of textual information by means of the color blinking patterns consisting of four light units having three gradations of brightness. This assistive aid does not require recognizing a precise form of the characters. With further practice, the use of the proposed aid may result in achieving pre-attentive processing color-blinking signals likewise monochrome Morse code. An adequate light stimulation of the visual analyzer also promotes deceleration of the visual nerve degeneration. The use of stylus movements accompanied with color-blinking feedback signals is one of possible techniques to develop cross-modal coordination in the absence of visual information based on dynamical visualization of the user activity. On the other hand, the visual phosphene sensations evoked in a visual cortex through electrical transcutaneous stimulation applied to the retinal area of the eye could support nonvisual interaction with hidden graphic entities in less complex manner than color-blinking signals and, thus, to facilitate grasping graphic attributes of encoded images. Thus, the key problem to be investigated is to find out which kind of these alternative feedback cues would require less cognitive efforts in interpreting the meaning encoded. Taking all of the above into consideration, we see a need to continue an exploration on finding the convenient and efficient alternative of making manipulating with hidden graphic entities easily manageable for visually challenged user by means of employing directional-predictive electro-tactile and color-blinking feedback signals with properly adjusted perceptual parameters. The goal of this study was to evaluate and compare the user performance in discovering the features of the invisible graphic images when two kinds of the feedback cues guiding the user actions through grasping the feature of those images were employed to augment the stylus movements regarding the graph: 1) directional-predictive electro-tactile patterns (DPeTP) provided through electrodes positioned on the frame of spectacles and stimulating transcutaneously the retinal area of the visual analyzer; 2) directional-predictive color-blinking patterns (DPBlk) provided through a rectangular bi-color light emitting diode coupled with spectacles and stimulating the peripheral area of the eye.

2. METHOD DESIGN 2.1 Participants Ten unpaid right-handed volunteers (six males and four females) participated in the study. The age of the subjects ranged from 20 to 35 years. All the participants had an experience of 6 hours of usage of the version of the game application “Hidden Graphs” where the stylus movements during the inspection of hidden graphs were augmented with directional-predictive sounds, vibrations and electro-tactile signals. The subjects also had a prior experience on using color blinking signals. The working area of a touch tablet was invisible for the participants during the testing to avoid visual prediction or/and approximation of the detected locations and scanpaths concerning touch tablet features. Herewith, graphs were always hidden from the subjects.

2.2 Apparatus 2.2.1 “The Hidden Graphs” Software and Interaction Style Vesterinen and Evreinov (2007) implemented game application called “The Hidden Graphs”. The goal of the application was to explore non-visually the hidden graphs and to capture as many features of the virtual graphic image as possible. The game consists of training and testing phases. The training phase allows preliminary scanning of the gameplay field by using a stylus-type input device. The subject has to choose a right behavioral strategy which could allow integrating the feedback signals and rebuilding the mental model of the image based on discrete feedback cues and haptic feedback regarding locations, which were inspected. In the present experimental setup, the Wacom Graphire4 USB graphics tablet having an active manipulandum surface of 280 mm × 260 mm and including a cordless stylus was used. In the testing phase, the subject has to confirm the detected positions and the sequence suggested.

110


The directional-predictive electro-tactile and color-blinking signals were used in relation to the participant gestures and taking into account the concept of a capture radius. In the software application, the capture radius (Rc) used during the graph inspection was defined as a range of pixels that the software application considers that the user has reached the target point located on or near the graph curve. The size of the Rc varied from 5 to 20 pixels. Directional-predictive signals were used in relation to subject gestures with the Rc. The center of the Rc was associated with the nearest point of the graph regarding the stylus location.

2.2.2 Hidden Graphs To avoid side effects similar to visual completion or cognitive transfer, two-dimensional ambiguous graphs, which were not having distinct geometrical shapes, have been used. The subjects were asked to discover the graphs, which were never presented through graphical images. Two-dimensional arrays were plotted and stored as the pictures after testing. Visualization of the graph arrays is shown in Figure 1.

Figure 1. The hidden graphs used and the number of pixels to be inspected in each array.

To eliminate a dependence on the input device resolution, the size of the gameplay field was specified in relative units and constituted 250 by 250 pixels. All the graphs were created with the discreteness index 8 to decrease the number of the array elements (pixels). That is, when the Rc was equal to 20 pixels in the first level of the game, crossing electro-tactile or color-blinking pattern would be activated sequentially in two locations of the stylus movements within Rc at inspection of the invisible graph in any direction. Otherwise we ought to use a special delay to split the feedback signals. Still, we had doubts that any temporal delay can distort direct pointing and may aggravate cognitive integration of the discovered spatial features of the hidden images.

2.2.3 BlkGlasses The BlkGlasses used to augment stylus movements with color-blinking signals during the inspection of hidden graphic images consisted of a rectangular bi-color light emitting diode (LED) having three leads, which was coupled with frame of spectacles having non-transparent glasses (Figure 2). The LED was located close to an eye in a paracentral unfocused position and diffused luminescence was provided [Evreinova et al., 2003]. The BlkGlasses were connected with the experimental software through PC parallel LPT port.

Figure 2. The BlkGlasses used in experiment.

2.2.4 eGlasses To augment stylus movements during the inspection of hidden graphic images with electro-tactile feedbacks, the eGlasses were used. The eGlasses had non-transparent glasses and the electrodes positioned on the frame of spectacles so that they could have good contact with temples (Figure 3).

111

ISBN: 978-972-8924-59-1 © 2008 IADIS

Figure 3. The eGlasses and electro-tactile unit used in the experiment.

2.2.5 Electro-Tactile Unit The eGlasses were connected with the experimental software through the implemented electro-tactile unit which was used to produce predefined parameters of DPeTP. The parameters of the current pulses were stabilized and controlled through the parallel port of the PC. The basic concept of the method is explained in Figure 4. An external load RZ which is presented by skin impedance measured between electrodes D1-D2 and usually is varied within a range 7.5-200 kOhm was connected with the output of the current sources J1 and J2. Each of these current sources provides initial current of 0.26 mA through resistors R1 and R2, herewith R1 >> RZ and R2 >> RZ. At that, potentials U1 and U2 are equal. The resulting current J applied in external load (RZ) has a zero value. The current direction (polarity) depends of the state of the switches T1 and T2.

Figure 4. Shaping the current pulses of the direct and reverse polarities.

To shape the direct polarity of the current pulses switch T1 shortcuts resistor R1 (Figure 4, on the left). The current pulses of the reverse polarity are being shaped when switch T2 shortcuts resistor R2 (Figure 4, on the right). In a case of shaping the current pulses of direct polarity, the current through external load (RZ) is provided by the output of the source J2. The current pulses of reverse polarity are being shaped with the source J1. The control of the switches T1 and T2 is programmatically carried out using the parallel port and optocouplers (SFH615A).

2.2.6 The “ePattern Constructor” Software The software was implemented under Microsoft Visual Basic 6.0 and used to produce and edit bipolar or monopolar basic and composite electro-tactile patterns. The experimenter could manipulate the software controls in order to change the parameters of electro-tactile patterns such as number, intensity, polarity, and delay between the pulses. Once the parameters were changed, they were automatically stored in a log file. Later, this file could be loaded and executed within the application.

2.2.7 Designing Directional-Predictive Electro-Tactile Signals The DPeTP had approximately the same duration of 200 ms and consisted of biphasic pulses reported in research studies (Hambrecht et al., 1992; Humayun et al., 1999) as having a great perceiving intensity. Crossing electro-tactile pattern (CeTP), indicating the stylus location within Rc-distance from the graph, consisted of 10 negative and 10 positive monopolar pulses with duration of 5 ms each and with a delay of 5 ms between pulses. The resulting duration of the CeTP was 203 ms. The electro-tactile pattern elicited the sensation of the small white light spots. Backward electro-tactile pattern (BeTP), indicating when the stylus moved backward regarding the graph and the distance to the graph was increased up to four Rc-distances, was composed of 15 positive and 15

112


negative monopolar pulses with duration of 2 ms each, and a delay of 5 ms between the pulses. The resulting duration of BeTP was 216 ms. The electro-tactile pattern evoked the sensation of small white light clouds. Toward electro-tactile pattern (TeTP), indicating the movements of a stylus moving toward the graph within four Rc-distances, consisted of 3 positive and 3 negative pulses with duration of 25 ms each, and delay of 5 ms between the pulses. The resulting duration of the electro-tactile pattern was 183 ms. This electrotactile pattern produced the sensation of vibrating white light spot.

2.2.8 Designing Directional-Predictive Color-Blinking Signals The basic DPBlks had twice shorter duration than DPeTPs that is of 100 ms because of their perceived intensity was greater than those of DPeTPs. Crossing color blinking pattern (CBlk) consisted of 3 pulses with duration of 35 ms each and two delays of 3 ms between the pulses. The resulting duration of the CBlk was 111 ms. This blinking signal elicited the green color blinking sensation of full intensity. Backward color blinking pattern (BBlk) was composed of 4 with duration of 25 ms each, and two delays of 5 ms between the pulses. The resulting duration of the BBlk was 110 ms. The signal evoked the red color blinking sensation of full intensity. Toward color blinking pattern (TBlk) consisted of one pulse with duration of 35 ms for the 1st and 2nd bursts with one delay of 25 ms between them. The resulting duration of the TBlk was 95 ms. The signal produced the yellow color blinking sensation of full intensity.

2.3 Procedure The subjects were tested individually in the usability laboratory at the local university. Prior to data collection, they were told that the stylus movements toward, backward or across the hidden graph will be accompanied with corresponding directional-predictive electro-tactile or color-blinking feedback signal. The subjects were advised to anchor the hand on the desk where the touch tablet was located. They were asked to leave the hand relaxed while grasping the stylus and to explore the working field relying on electro-tactile sensations or color-blinking patterns. The subjects, who wore glasses, were asked to remove them before the test. When the eGlasses were used, the electrolyte gel was applied to the temples of the participants to provide a better contact between the eyeglasses electrodes and the skin surface. The entire experiment took six days. The subjects completed three sessions inspecting the hidden graphs augmented with directional-predictive color-blinking feedback signals and other three sessions inspecting the graphs augmented with directional-predictive electro-tactile feedback signals, with no more than one session per day. Each session lasted for one hour in average. The subjects could rest as desired between the trials. Each subject made an inspection of 40 graphic images in each session. Thus, each of 5 graphic images was inspected 8 times being presented randomly during each session. Each trial supposed successful completion of the training and confirmation phases of the test. In a total, each subject tested 120 times each kind of feedback type. All the data such as the capture radius, the time spent to complete the inspection and the confirmation phases of the test, and the numbers of directional-predictive electro-tactile and color-blinking signals used during the inspection, were automatically collected and stored in a log file for further analysis.

3. RESULTS AND DISCUSSION The subjects were free to choose from which location of the touch tablet to start an inspection of the invisible graphic image. Most of the players had a tendency to start the inspection from the lower left corner of the touch tablet by doing the spiral and straight-line gestures along the game field. Some of the players explored the working field by doing the stylus movements mostly in the diagonal directions, as these movements could easily be coordinated to the carpus (hand) position which was almost static. The average numbers of the directional-predictive color-blinking and electro-tactile feedback signals used to perform non-visual inspection of the hidden graphic images during inspection phase at 4 values of a capture radius Rc through 6 test sessions were statistically assessed and compared. The results are presented in Figures 5, 6 and 7. The data revealed that when the inspection of the invisible graphic images was assisted with directional-predictive electro-tactile feedback signals, most of the players kept the same behavioral pattern within four capture radiuses. As we can see from Figure 5, the subjects tended to employ CeTP

113

ISBN: 978-972-8924-59-1 © 2008 IADIS

Number of DPS used

signals more frequently than CBlk signals [F = 10.75, 2.7, 2.4 and 1.27 at Rc = 5, 10, 15 and 20 pixels, p < 0.001]. They followed the directional-predictive electro-tactile signals until they would lose the track outside the radius of capture and BeTP signal evoking the sensation of small white light cloud would stop the participant movement and the subject would need to back track stylus in the inverse direction. After sensing the vibrating white light spot elicited by TeTP signal, a stylus would cross the graph and CeTP signal perceived as a small white light spot followed the crossing stylus movement immediately. CBlk CeTP

600 400 200 0 5

10 15 Capture radius, pxls

20

Figure 5. The average number of CBlk and CeTP used during inspection phase at different capture radius.

Number of DPS used

Figures 6 and 7 demonstrate that within four capture radiuses the players were prone to employ the smaller number of TeTP signals than that of the TBlk signals [F = 11.2, 3.8, 3.5 and 0.75 at Rc = 5, 10, 15 and 20 pixels, p < 0.001] and the greater number of the BeTP signals than that of the BBlk signals [F = 10.8, 2.74, 3.2 and 1.4 at Rc = 5, 10, 15 and 20 pixels, p < 0.001]. BBlk

600

BeTP 400 200 0 5


20

Number of DPS used

Figure 6. The average number of BBlk and BeTP used during inspection phase at different capture radius.

TBlk TeTP

600 400 200 0 5


20

Figure 7. The average number of TBlk and TeTP used during inspection phase at different capture radius.

The reason behind this asymmetric behavior is the difference in the perceptive strength and quality of visual sensations the designed feedback signals, and their following impact on the stylus movements. The TeTP signal eliciting the strong vibrating white light spots evoked almost an immediate stop of the stylus movements while tiresomely perceptible red-green color blinking sensation produced when TBlk signal was employed may require an additional confirmation. On the contrary, the strength of the phosphenes shaped in

114


a form of BeTP signal was not efficient enough to provide well perceptible visual sensation in comparison to those elicited by BBlk signal. During the confirmation phase of the test, similar symmetry in the behavioral pattern of the subjects has been detected. The players were inclined to employ greater number of the CBlk signals than CeTP signals [t(3) = 1.76, p < 0.05], a smaller number of the TeTP signals than TBlk signals [t(3) = -1.25, p < 0.05], and a greater number of the BeTP signals than BBlk signals [t(3) = 2.35, p < 0.05] to complete the trials with the use of any of four capture radiuses. Figure 8 illustrates the grand mean time spent by the players to accomplish the confirmation phase of the test when the non-visual inspection of the graphs was assisted with DPBlk or DPeTP signals. The participants consistently required a greater time to confirm the inspected graphs assisted with DPeTP signals than these assisted with DPBlk signals when the capture radius was equal to 5 or 10 pixels (86 ms vs. 64 ms at Rc = 5 pixels and 74 ms vs. 54 ms at Rc = 10 pixels) than in a case when a larger capture radius was used (65 ms vs. 43 ms at Rc = 15 pixels and 42 ms vs. 33 ms at Rc = 20 pixels). Such a tendency was observed because the phosphenes elicited by directional-predictive electro-tactile signals have a weaker perceptible force and therefore resulted in weaker motor reaction than directional-predictive color-blinking signals. Shorter duration of the directional-predictive color-blinking signals also appeared to be beneficial when it was needed to complete the trial faster. DPBlk DPeTP

Time, s

100 75 50 25 0 5

10 15 20 Capture radius, pxls

Figure 8. Mean time spent by the participants to complete the confirmation phase assisted with DPBlk and DPeTP signals.

4. CONCLUSION When performing non-visual interaction with the graphical entities by means of a stylus-type input device, there are many possible alternatives for delivering feedback cues coordinated with exploratory stylus movements. The signals drawing attention and guiding the user behavior can describe the particular graph features or the features concerning the exploratory movements in relation to the graph (Evreinova et al., 2008). A key question is what kind of feedback signals would require less cognitive efforts in interpreting their meaning by visually challenged people. In our previous research, a game software application “Hidden Graphs” was implemented to study the features of non-visual inspection of the virtual invisible graphic images. Stylus movements regarding the invisible images were augmented with sound, vibro-tactile and electro-tactile feedback signals taking into account the concept of the capture radius and the features of an exploratory behavior. The results indicated that the assistive electro-tactile signals with well-adjusted perceptual parameters were the most beneficial for that purpose. In the present study, the touch tablet user performance was assessed and compared in relation to discovering the features of invisible graphic images when the stylus movements were augmented with electro-tactile and color blinking feedback signals stimulating transcutaneously the retinal area of the visual analyzer. The perceptual parameters of the directional-predictive signals were adjusted in such a way that the visual sensations elicited with assistive cues in participants would be well distinguishable while employed for imaging to a minimum extent.

115

ISBN: 978-972-8924-59-1 © 2008 IADIS

The performance of ten subjects was investigated in terms of the number of feedbacks used and time spent to complete inspection of hidden graphs. The average numbers of the directional-predictive signals used by the participants to perform non-visual inspection of the invisible graphic images through 6 test sessions at four game levels were assessed statistically and compared. The data analysis revealed that the players were inclined to employ a greater number of the crossing electro-tactile feedback signals than crossing color-blinking signals [F = 10.75, 2.7, 2.4 and 1.27 at Rc = 5, 10, 15 and 20 pixels, p < 0.001]. The players used lesser number of toward electro-tactile feedback signals than that of toward colorblinking signals [F = 11.2, 3.8, 3.5 and 0.75 at Rc = 5, 10, 15 and 20 pixels, p < 0.001] but greater number of backward electro-tactile feedback signals than that of backward color-blinking feedback signals [F = 10.8, 2.74, 3.2 and 1.4 at Rc = 5, 10, 15 and 20 pixels, p < 0.001] to complete inspection phase of the test at any level. The reason behind this asymmetric gameplay behavior could be the difference in the perceptive strength and quality of visual sensations the designed feedback signals elicited in participants, and their following impact on the stylus movements. The results revealed similar symmetry in the behavioral pattern of the participants at the confirmation phase of the test. The mean completion time required by the players to complete the confirmation phase of the test with the use of directional-predictive color-blinking signals was consistently shorter than that with the use of directional-predictive electro-tactile signals when the smaller capture radius was used. This is because the phosphenes elicited by directional-predictive electro-tactile signals have a weaker perceptible force and therefore resulted in weaker motor reaction than directional-predictive color-blinking signals. The experimental findings confirmed the beneficial use of color blinking signals in this task. However, we suppose that upon improving sensory quality of the phosphenes elicited during transcutaneous electrotactile stimulation, electro-tactile signals might facilitate grasping the primitive and composite attributes of the simulated graphic entities.

ACKNOWLEDGEMENT This work was financially supported by the Academy of Finland (grants 107278 and 115596).

REFERENCES Brindley G. and Lewin W., 1968. The sensations produced by electrical stimulation of the visual cortex. J. Physiol. 196, pp. 479 – 493. Carpenter R., (1972) Electrical stimulation of the human eye in different adaptational states. J Physiol., Feb; 221(1), pp. 137–148. Evreinova T. G., Evreinov G., Vesterinen L., and Raisamo R., 2008. Non-visual imaging of graphs through directionalpredictive sounds, vibrations, and electro-tactile patterns. Applied to International Journal of Human-Computer Studies. Evreinova T. G., Evreinov G., Raisamo R. and Vesterinen L., (2008). Non-Visual Interaction with Graphs Assisted with Directional-Predictive Sounds and Vibrations: A Comparative Study. In: Universal Access in Information Society, Vol. 7(1-2), pp. 93-102. Evreinova T.G., Vesterinen L., Evreinov G. and Raisamo R., (2007). An Exploration of Directional-Predictive Sounds for Non-Visual Interaction with Graphs. In Knowl. Inf. Syst. (KAIS) Journal, (13): pp. 221-241, Evreinova T. G., Evreinov G., and Raisamo R., (2003). Color-Blinking Code and Low Cost Peripheral Monitor for People Who Are Deaf or Have Low Vision. In: AMSE Journal, The special issues “C”. Tassin la Demi-Lune, France, pp. 129-138. Hambrecht F.T. et al., (1992). Microstimulation of the visual cortex in a blind human. In: Proceedings of the 4th Vienna International Workshop on Functional Electrostimulation. Vienna, Austria. Humayun M.S., de Juan E. Jr., Weiland J. D., et al., (1999). Pattern electrical stimulation of the human retina. Vision Res., 39, pp. 2569–2576.

116

Interfaces and Human Computer Interaction 2008

Interfaces and Human Computer Interaction 2008

Suggest Documents