assessing perceptual learning dynamics during ... - Semantic Scholar

IADIS International Conference Cognition and Exploratory Learning in Digital Age (CELDA 2004)

ASSESSING PERCEPTUAL LEARNING DYNAMICS DURING VISUAL SEARCH IN VIRTUAL IMMERSION USING EYE-TRACKING TECHNOLOGIES 1

Patrice Renaud, Ph.D.

Université du Québec en Outaouais ; 2Institut Philippe Pinel de Montréal 1 283, boul. Alexandre-Taché, Gatineau, Canada ; 210905, Henri-Bourassa Est, Montréal, Canada

Guillaume Albert, B.Sc. Université du Québec en Outaouais 283, boul. Alexandre-Taché, Gatineau, Canada

Louise Sauvé, Ph.D. TÉLUQ Québec 4750, avenue Henri-Julien, Montréal, Canada

Lise Renaud, Ph.D. Université du Québec à Montréal 320, rue Sainte-Catherine Est, Montréal, Canada

Jean Décarie, MA Université du Québec à Montréal 320, rue Sainte-Catherine Est, Montréal, Canada

Stéphane Bouchard, Ph.D. Université du Québec en Outaouais 283, boul. Alexandre-Taché, Gatineau, Canada

ABSTRACT This short paper proposes a methodology to assess perceptual learning during virtual immersion. This methodology relies upon eye-tracking technologies and fractal dynamics analyses performed on gaze behavior recorded in relation to virtual objects’ features. KEYWORDS

Assessment, perceptual learning, virtual reality, eye-tracking, fractal dynamics, perceptual invariants

1. INTRODUCTION More and more virtual reality (VR) technologies are becoming tools of predilection for training purposes, and education in general (Ota et al, 2002; Montovani et al, 2003). By immersing the learner in simulated contexts these technologies allow the possibility to mobilize motor and sensory processes in a way akin to reality. Beside immersion, the possibility to track motor behavior for assessment purposes is another asset of VR which is as interesting to training but much less exploited. Tracking eye movements during interactions with

413

ISBN: 972-98947-7-9 © 2004 IADIS

virtual objects may especially be rich in information about how a learner is progressing at the perceptual level, that is at how well he is in touch with the visual features lying at the basis of his learning (Duchowski et al. 2002; Vora et al., 2002).

2. THEORETICAL BACKGROUND 2.1 Perceptual learning Perception involves the extraction of invariant features from the surrounding (Gibson, 1979). It is through the coordination of the action-perception couple that this extraction may take place in a continuous fashion (Kelso, 1995; Renaud et al., 2001, 2002 a, b; 2003; Treffner & Kelso, 1999). Hence we may define perceptual learning as an “increase in the ability to extract information from the environment, as a result of experience and practice with stimulation coming from it” (Gibson, 1969; Kellman, 2002). From this theoretical perspective it seems possible to assess perceptual learning by tracking motor behaviors involved in visual perception in order to detect changes in the dynamics of the latter during a learning process.

2.2 Eye-movements Foveation involves eye movements initiated to align visual stimuli with the fovea, the center of gaze, and to keep this alignment stable. Foveation is accomplished about 230 000 times per day (Hoffman, 1997). This continuous and very rapid activity yields patterns whose geometry may seem completely random at first sight. The works of Yarbus (1967) and of Noton and Stark (1971) lead nevertheless to the conclusion that eye movements can be explained as scanpaths expressing interests and motives active in the beholder. The clusterings of saccades and fixations over the visual features of the scenes used in Yarbus experiments were interpreted as manifestations of the orientation and concentration of visual overt attention and perception.

2.3 Fractal dynamics Fractal geometry was developed to describe the potentially infinite similitudes that exist across differing scales of observation (Mandelbrot, 1975). Fractal dynamics have been found in various physiological (Hausdorff et al., 1995; Hausdorff et al., 1996) and behavioral phenomena (Shaw & Kinsella-Shaw, 1988; Treffner & Kelso, 1999). They have also been observed in eye-head coordination phenomena (Renaud et al., 2002b; Shelhamer, 1998). Moreover, it was demonstrated that head-tracking behavior during virtual immersions can exhibit fluctuations organized in terms of long-range correlations with scale invariance (Renaud et al., 2002a, Renaud et al., 2001).

3. METHODOLOGY Our method relies upon a technological setting including what is usually necessary to present virtual environments in immersion plus equipments dedicated to eye-movements tracking from within a head mounted display (HMD). A special mounting built from a monocular infra-red eye-tracking system combined within a binocular HMD is used to track eye-movements. Head-movements are recorded from a tracking system rendering the 6 degrees-of-freedom (DOF) of translation and rotation. Our method performs gaze analysis by the way of virtual measurement points (VMP) placed on virtual objects to analyze eye-movements in relation to specific features of these objects. Gaze radial angular deviation (GRAD) and head radial angular deviation (HRAD) from VMPs are given by the combinations of the 6 DOF developed by head-movements and the x and y coordinates rendered by the eye-tracking system (Duchowski et al. 2002; Renaud et al., 2002b; Vora et al., 2002). VMPs are locked to virtual objects and move jointly with them. While the variations in the 6 DOF developed by head-movements define changes in the global scene presented in the HMD, the 2 DOF given by the eye-tracking device allow the computation of the exact position of the line of sight relative to VMPs.

414

IADIS International Conference Cognition and Exploratory Learning in Digital Age (CELDA 2004)

GRADs and HRADs are sampled at 12Hz to give time series reflecting visual search performance. These time series are analyzed in order to find parameters expressing the extraction of perceptual invariants. Perceptual learning developing during visual search should be traceable in the evolution of these parameters’ values.

4. APPARATUS AND STIMULI Our experimentations are running on a Pentium IV computer (2.8 GHz). The HMD is a binocular V8 model from virtual research with an image resolution of 640 X 480 pixels, a contrast ratio of 200:1 and a field of view of 60° diagonal. Our head-tracking device is an Intersense IS-900. This tracking system renders the 6 DOF (position: x, y and z / angular rotation: yaw, pitch and roll) of the HMD sensor. The rendering is based on a hybrid technology of inertia and ultrasonic tracking. The accuracy of this system is of 3mm RMS in translations, and 0,15 degree RMS in rotations. Our eye-tracking device is a ASL 504 model. This system relies on the corneal reflection of an infra-red source that is measured relative to the pupil center location. These particular corneal reflections, known as the first Purkinje images (Duchowski & Vertegaal, 2000) can be located with video-based eye trackers collecting infra-red reflections. A single eye tracker returns 2 DOF, i.e. variations in a x and y plane. Accuracy is of 0,5 degree.

5. DATA ANALYSIS Two distinct analyses were performed on the time series that resulted from a visual search made in a virtual environment. These are done in order to better understand the nature of the attractor underlying the oculomotor behavior dynamics exerted in relation to visual features of virtual stimuli tagged with a VMP. By doing so, we aimed at discovering dynamical invariants that could be indicative of the perceptual invariance extraction process accomplished via eye-head coordination in immersion. The data presented below are an illustrative sample coming from a one session visual search made by one subject during a virtual immersion. Both GRAD and HRAD are analyzed according to the following techniques.

5.1 Correlation dimension (D2) The correlation dimension (D2) is the most commonly used index of fractal dimensionality and information complexity in time series (Heath, 2000). D2 expresses invariance amidst nonlinear dynamical processes by shedding light on the self-similarity of the underlying attractors’ geometry. Here, D2 is computed following a method relying on singular value decomposition. As Sprott and Rowlands (1995) state it: With each pass through the data, a new data point is taken, and a hyperdimensional sphere of embedding dimension D and radius r is centered on that point. The fraction of subsequent data points in the record within that sphere is then calculated for various values of r, and a plot is made of the log of this number versus the log of the radius. The correlation dimension is taken as the average slope of the cumulative curve over the middle one-quarter of the vertical scale, and the error is taken as half the difference of the maximum and minimum slope over the same range. D2 thus gives a measure of the fractal complexity of the underlying attractor:

with Pi being the probability to find a point of the attractor within the i-th subspace of phase space (phase space is subdivided into spheres of radius r). The number M(r) of spheres that contain attractor points, is related to the dimension of the attractor:A proper D2 requires that the plot of the correlation dimensions

415

ISBN: 972-98947-7-9 © 2004 IADIS

versus the embedding dimensions forms an asymptotical plateau. As the embedding dimension increases, D2 should also increase but eventually saturate at the correct value. If not, this means that the attractor is not well bounded in a specific region of the system’s phase space and that the underlying process is akin to noise, i.e. that there is no structures or topological invariances in the dynamical system.

5.2 Surrogate data test A surrogate data method introduced by Theiler and colleagues (1992) and adapted by Hausdorff and colleagues (1995) was used here to statistically differentiate the computed correlation dimension from dimensions coming from random processes. From our time series, 20 surrogate time series were generated by doing a Fourier transform of the original data (the phase of each Fourier component was set to a random value between 0 and 2π) while preserving their power spectrum and correlation function (Heath, 2000; Sprott & Rowlands, 1995). The mean and standard deviation of the surrogate exponents were then computed from the 20 surrogate time series and compared to the original correlation dimension to determine statistical significance. S, the number of standard deviations between the original exponents and the mean surrogate exponent, was obtained from S = (D2 - Ds)/SD, where D2 is the original correlation dimension, and Ds and SD are respectively the mean and standard deviation of the surrogate data sets. The null hypothesis was rejected if S > 3, that is to say that D2 was considered different from random if it fell more than 3 standard deviations from Ds.

6. RESULTS Figure 1 presents GRAD and HRAD as a function of time; what is seemingly random reveals itself as a deterministic and fractal process with D2’s computing. Indeed we obtained a D2 value of 2.951 + - 0.105 for GRAD and 2.565 + - 0.363 for subject HRAD. Figure 2 shows a well saturated asymptotical plateau for the values of D2 as a function of embedding dimensions, for both measures. These values of D2 tend to show that the available DOF of the eye-head coordination system compressed and coordinated themselves in order to make the free exploration behavior emerge relative to the VMP. To be sure that these results were clearly not coming from a random process, we performed the above described surrogate data test. We got a Ds of 5.470 and a SD of 0.118 for GRAD and a Ds of 4.596 and a SD of 0,082 for HRAD (see Figure 3 for the plot of D2s as a function of embedding dimensions for one representative surrogate time series). These results mean that the original D2 value falls more than 20 standard deviations away from Ds for GRAD and more than 24 standard deviations for HRAD. These results are thus significantly different from random processes. It thus seems that the eye-head coordination behavior relative to the VMP from which D2 was computed is a very labile but yet highly organized dynamical process with an underlying invariant self-similar signature. It is speculated here that this kind of dynamic pattern may reveal some of the mechanisms at work in the extraction of perceptual invariance.

Figure 1. GRAD and HRAD as a function of time.

416

IADIS International Conference Cognition and Exploratory Learning in Digital Age (CELDA 2004)

Figure 2. The plot of GRAD (left) and HRAD (right) D2s as a function of embedding dimensions.

Figure 3. The plot of D2s as a function of embedding dimensions for one representative surrogate time series.

7. CONCLUSION The basis of a methodology to assess perceptual learning from the measure of perceptual invariance extraction behavior was proposed. This methodology as applied in virtual immersion could serve as a mean to keep track of the rate at which a learner is acquiring and maintaining critical perceptual learning. Next steps in this research project will consist in testing this methodology with specific learning contents.

REFERENCES Duchowski, A. et al, 2002. 3-D eye movement analysis. In Behavior Research Methods, Instrument & Computers, Vol. 34, No. 4, pp. 573-591. Gibson, E.J., 1969. Principles of perceptual learning and development. Appleton-Century-Crofts, New York, USA. Gibson, J.J., 1979. The ecological approach to visual perception. Erlbaum, Mahwah, USA. Hausdorff, J.M. et al, 1995. Is walking a random walk? Evidence for long-range correlations in the stride interval of human gait. In Journal of Applied Physiology, Vol. 78, pp. 349-358. Hausdorff, J.M. et al, 1996. Fractal dynamics of human gait: stability of long-range correlations in stride interval fluctuations. In Journal of Applied Physiology, Vol. 80, pp. 1448-1457. Heath, R., 2000. Nonlinear dynamics. Techniques and applications in psychology. Lawrence Erlbaum, Mahwah, USA. Hoffman, J. E., 1997. Visual attention and eye movements. In H. Pashler (Ed.), Attention. London: University College London Press, pp. 120-153. Kellman, P.J., 2002. Perceptual learning. In D. Medin, and H. Pashler (Eds.) Steven’s handbook of experimental psychology, vol. II: Memory and cognitive processes. 3rd edn, John Wiley and sons, New York, USA, pp. 259-299. Kelso, J.A.S., 1995. Dynamic Patterns: The Self-organization of Brain and Behavior. MIT Press, Cambridge, USA. Mandelbrot, B., 1975. Les objets fractals. Flammarion, Paris, France. Mantovani, F. et al, 2003. Virtual reality training for health-care professionals. In Cyberpsychology & Behavior, Vol. 6, No. 4, pp. 389-395. Noton, D. and Stark, L., 1971. Eye movements and visual perception. In Scientific American, Vol. 224, pp. 34-43. Ota, D. et al, 2002. Virtual reality in surgical education. In Computers in Biology and Medicine, Vol. 25, No. 2, pp. 127137. Renaud, P. et al, 2001. Head-tracking fractal dynamics in visually pursuing virtual objects. In W. Sulis, and I. Trofimova (Eds.). Nonlinear Dynamics in Life and Social Sciences, IOS Press, NATO ASI Series, Amsterdam, Netherlands, pp. 333-346.

417

ISBN: 972-98947-7-9 © 2004 IADIS

Renaud, P. et al, 2002a. Behavioral avoidance dynamics in the presence of a virtual spider. In IEEE (Institute of Electrical and Electronics Engineers) Transactions in Information Technology and Biomedecine, Vol. 6, No. 3, pp. 235-243. Renaud, P., et al, 2002b. Extracting perceptual and motor invariants using eye-tracking technologies in virtual immersions. Proceedings of HAVE'2002-IEEE (Institute of Electrical and Electronics Engineers) International Workshop on Haptic Virtual Environments and their Applications, Ottawa, Canada, pp. 73-78. Renaud, P. et al, 2003. Eye-tracking technologies in immersive environments: a general methodology to analyze affordance-based interactions from oculomotor dynamics. Cyberpsychology and Behavior, Vol. 6, No. 5, pp. 519-526. Shelhamer, M., 1998. Nonlinear Dynamic Systems Evaluation of ‘Rhythmic’ Eye Movements (Optokinetic Nystagmus). In Journal of Neuroscience Methods, Vol. 83, pp. 45-56. Sprott, J.C. and Rowlands, G., 1995. Chaos Data Analyzer. The Professional Version. PC User’s Manual. American Institute of Physics, New York, USA. Theiler J., et al, 1992. Testing for nonlinearities in time series: the method of surrogate data. In Physica D, Vol.58, pp. 77-84. Treffner, P.J. and Kelso, J.A.S., 1999. Dynamic encounters: Long memory during functional stabilization. In Ecological psychology, Vol. 11, pp. 103-137. Vora, J. et al, 2002. Using virtual reality technology for aircraft visual inspection training: presence and comparison studies. In Applied Ergnomics, Vol. 33, pp. 559-570. Yarbus, A.F., 1967. Eye Movements and Vision. Plenum Press, New York, USA.

418