A New Technique for Improving Pupil Detection Algorithm - IEEE Xplore

0 downloads 0 Views 933KB Size Report
3 “Grigore T. Popa” University of Medicine and Pharmacy, Iaşi, Romania [email protected]. Abstract—In this paper are presented several techniques ...
A New Technique for Improving Pupil Detection Algorithm Radu Gabriel Bozomitu1, Vlad Cehan1, Robert Gabriel Lupu2, Cristian Rotariu3, Constantin Barabaşa1 1

Faculty of Electronics, Telecommunication and Information Technology, “Gheorghe Asachi” Technical University, Iaşi, Romania [email protected]; [email protected]; [email protected] 2 Faculty of Computer Engineering and Automatic Control, “Gheorghe Asachi” Technical University, Iaşi, Romania [email protected] 3 “Grigore T. Popa” University of Medicine and Pharmacy, Iaşi, Romania [email protected]

Abstract—In this paper are presented several techniques used for improving the pupil detection algorithm based on a head mounted eye tracking system employing an infrared video camera. The coordinates of the eye pupil center obtained by any eye tracker device are affected by many noise sources. In order to improve the pupil detection position, several different techniques of real time filtering and high frequency spikes canceling from the signal provided by the pupil detection algorithm are presented. The optimal parameters of the real time filter, depending on the application sampling frequency, have been determined.

I.

INTRODUCTION

In the last years, eye tracking techniques have become more common for applications in various fields, such as: people identification (biometrics), communications with neuromotor disabled people (assistive technology), human activity recognition, marketing, advertising, teaching, aviation, virtual reality [1-14]. Eye tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement [23]. There are different methods for detecting eye movement. The most widely used variant employs video images captured with a video camera - in the visible spectrum (day light) or in infrared light - from which the eye position coordinates are extracted. If the camera is placed at the base of the screen, the system is called remote eye tracking. If the camera is mounted on glasses, right underneath the eyes, the system is called head mounted eye tracking, the latter being the object of our study. Other methods use search coils (contact lens with an inductive sensor) [4] when alternating magnetic fields are generated by magnets positioned around the eye, can be implemented on the basis of electro-oculography [4] or corneal reflection [5]. This paper presents an improved eye tracker device, based on the processing of the eye video image, captured with an infrared video camera. The application of the proposed system, employing keywords technology, engages the interest of the assistive technology for the communication with neuromotor disabled people. The video camera is connected to a computer which performs real-time eye image processing. The eye image

978-1-4673-7488-0/15/$31.00 ©2015 IEEE

processing algorithm takes the pictures delivered from the video camera and detects the eye pupil position to calculate the user gaze’s direction. In order to improve eye pupil detection, the dark-pupil technique has been implemented. The result of the infrared illumination is that the pupil is clearly demarcated as a bright region due to the photoreflective nature of the back of the eye [6]. On the other hand, by using this illumination type, a consistent and uniform illumination of the eye, without any user discomfort can be obtained [7]. Since eye movements are not constant but made up of saccades (rapid movements) and fixations (short stops), the signals provided by the detection algorithm consists of a high frequency noise component (higher than 5 – 6 Hz in our application) and a low-frequency drift component [7-8]. The high frequency noise component of the detected signals is due to the noise generated by the video camera electronic circuits, non-static light conditions and imperfections in the image processing algorithm used to estimate the gaze-point. When the user fixates his/her eyes on a point on the screen, the coordinates of the measured position will resemble Gaussian noise with a slow change of the mean [8]. In this paper it is shown that this noise component can be eliminated by real time filtering, using a fourth order infinite impulse response (IIR) low-pass filter (LPF) in Chebyshev approximation. The real time signal provided by the pupil detection algorithm also contains a lower frequency drift, which is due to the difficulty of keeping the eye absolutely fixed on a point [8]. When trying to fixate a point, the eyes tend to drift off slightly, which is compensated by a corrective saccade if the original point still draws attention [8]. Other factors which affect the pupil detection algorithm are represented by involuntary head movements and pupil changes of shape and diameter during fixations, even if the light conditions are static [9]. The main sources of noise are the physiological tremor of the eye [13], the corneal reflection due to the infrared illumination of the eye, involuntary blinking, pupil shape and pupil position on the sclera, etc. In addition, there may also be occasional high frequency (HF) spikes due to involuntary blinking and corneal reflection, which could be eliminated.

In this paper a new technique, which improves the operation of the pupil detection algorithm is presented. II.

PUPIL DETECTION ALGORITHMS

In recent years, various methods of iris or eye pupil detection used for gaze detection have been proposed, based on Starburst algorithm and Hough transform, applied to man-machine interfaces [6, 10]. Eye-tracking algorithms can be classified into two approaches: feature-based and model-based approaches [6]. Feature-based approaches detect and localize image features related to the position of the eye [6]. A commonality among feature-based approaches is that a criterion (e.g. a threshold) is needed to decide when a feature is present or absent [6]. Some of the feature-based algorithms use a binarization technique which leads to good stability, but is sensitive to illumination conditions [7]. Model-based approaches do not explicitly detect features but rather find the best fitting model that is consistent with the image [6, 10]. For example, integro-differential operators can be used to find the best-fitting circle [11] or ellipse [12] for the limbus and pupil contour. The model-based approach can provide a more precise estimate of the pupil center than a feature-based approach, given that a feature-defining criterion is not applied to the image data [6]. In our research we used a head mounted eye tracking system, which consists of an infrared webcam mounted on frame glasses right underneath the eye (Fig. 1), and a PC for image acquisition and processing. The subjects who tested the system were asked to place their head in a chin rest and look at the user screen placed approximately 60cm away. Frame glasses Infrared camera

In order to significantly diminish the effects of noise and increase pupil center stability, a new technique is proposed, based on real time filtering and the cancelling of the HF spikes from the signals provided by the pupil detection algorithm. The practical system tests show that the image processing rate, that is the sampling frequency of the signal (fs), can vary from 10 to 50 frames per second, depending on the video camera performances and the processing power of the calculation unit on which the pupil detection algorithm runs. III.

FILTERING DESIGN METHODS

Due to all noise contributors, previously described, the determination of the cursor position and stability on the screen according to gaze direction is affected by errors. Spectral analysis of the signals provided by pupil detection algorithm shows that the cursor instability on the screen is due to spectral components of high frequency, which must be rejected. In order to increase the cursor stability on the user screen of the eye tracking system, various solutions have been proposed in the literature [3, 8-9, 14]. In [14] Spakov drew a brief comparison and proposed new comparison criteria. Many solutions for increasing the cursor stability of an eye tracker system are based on filtering. In order to find the best filtering solution, different signals from several users, have been analyzed. Our experiments in the field of assistive technology have shown that for the canceling of the higher frequency spectral components corresponding to noise, a low-pass filter with a band-pass of approximatively 2Hz (fpass in Fig. 2) is needed. For the real time filtering of the signals provided by the pupil detection algorithm, the IIR type filters have been considered. These filters have the combined advantage of requiring fewer coefficients, less memory space and of working faster than the FIR filters counterparts. In addition, the IIR filters are more efficient for the digital signal processing platform in this case. In order to avoid aliasing, the sampling frequency, fs, must satisfy the Nyquist condition (Fig. 2):

f s ≥ 2 Bmax , f s = (10 − 50) Hz

(1)

where Bmax is the maximum bandwidth of the signals provided by pupil detection algorithm. Thus, for Bmax = 5Hz (experimentally detected), the minimum value of fs that satisfy the condition (1) is 10Hz.

Figure 1. Structure of the eye tracking system

The tested eye tracker device employs an improved Starburst algorithm that combines feature-based and model-based approaches, which is robust to illumination conditions, but the provided pupil center coordinates are unstable on both axes. Due to the noisy eye images obtained from the video camera, the pupil center coordinates provided by the RANdom SAmple Consensus (RANSAC) algorithm varies from frame to frame. These variations of the coordinates act as noise over the signals provided by the pupil detection algorithm.

Figure 2. Signal spectrum (──) and design condition of the low-pass filter

According to the above discussion, the IIR low-pass filter has to meet the following design requirements (Fig. 2):

f pass = 2Hz , Apass = 0.5dB , f stop = 3Hz , Astop > 18.5dB , f s ≥ 10Hz (2)

The parameters of the IIR LPF, which satisfy the design requirements depend on the sampling frequeency fs of the signal provided by the pupil detection algorithm, which runs on the H eye tracker system, in the range of (10 – 50) Hz. In order to find the best filtering charracteristics, in the following, different order IIR filters in differrent approximations are analyzed comparatively. Two types of IIIR low-pass filters, in Butterworth (Bw) and Chebyshev (Ch) approximations, of t Bw ensures first, second and fourth orders have been tested. maximum flat pass-band and Ch ensures the highest attenuation in the stop-band. Other types of filters, with intermediate characteristics were not taken into consideeration. Obviously, attenuation in the stop-band depends on filtter type, order, and sampling frequency, according to the charaacteristics presented below. In Fig. 3 is shown the stop-band atttenuation (at 3Hz) provided by both filters analyzed (Bw and Ch) C of first, second and fourth orders, for different values of the sampling s frequency (fs) in the range of (10 – 50) Hz. d requirements As the sampling frequency is lower, the design can be achieved with a lower order filter. In other words, as the sampling frequency is lower, the same ordeer filter provides a higher stop-band attenuation, as illustrated in Fig. 3. On the mpling frequency is other hand, the minimum value of the sam limited by the condition of avoiding aliasing in equation (1). According to the characteristics presenteed in Fig. 3, it is obvious that the higher band-stop attenuationn is achieved by the two LPF of fourth order for the minimum value of the sampling frequency fs = 10Hz. It is worth noticing thatt the group delay at 2Hz is minimal for fs = 10Hz. (The group dellay at 2Hz is a time delay measure of the amplitude envelope of the 2Hz sinusoidal signal component.)

Figure 3. Band-stop attenuation (at 3Hz) of two typess of filters for different sampling frequency of the captured signal in the raange of (10 – 50)Hz

The sampling frequency of the signal proovided by the pupil detection algorithm used in the proposed eyye tracker device is about 30Hz. Thus, according to the characteristics illustrated in Fig. 3 and the design requirements from (2), for this application

978-1-4673-7488-0/15/$31.00 ©2015 IEEE

in the field of assistive technollogy, a fourth order Chebyshev IIR LPF with fs = 30Hz it is requuired. The filter coefficients (ak, bk) are determined with the Matlab program and the values obtaineed for a sampling frequency of 30Hz are presented in Table I. The T transfer characteristic of the filter is illustrated in Fig. 4. 4 The IIR filtering equation, expressed in a recursive form with w the finite difference equation and implemented in the proposed algorithm can be written as: 4

4

k =0

k =1

y (n) = ∑ ak x(n − k ) −∑ bk y (n − k )

(3)

Figure 4. Transfer characteristic of the t 4th order IIR LPF with fs = 30Hz in Chebyshev appproximation TABLE I.

T COEFFICIENTS OF 4TH ORDER IIR LPF WITH fs = 30Hz IN CHEBYSHEV APPROXIMATIONS P

Coefficients a0 a1 a2 a3 a4

IV.

Values 0.10037369 -0.18911234 0.25152887 -0.18911234 0.10037369

Coefficients b0 b1 b2 b3 b4

Values 1.00000000 -2.52750129 2.59652127 -1.22696482 0.23199641

SPIKES SIG GNAL CANCELING

Due to involuntary blinking and a corneal reflection, there are several situations when the puppil detection algorithm does not work properly. The consequence is that in this time interval the pupil edge is not detected and significant high frequency spikes are generated (Fig. 5). On the other hand, the low frequency changes are caused by a drift geenerated by pupil size and shape changes and involuntary movem ments of the head [8]. During the transition from looking at onee point on the user screen to another, the signal mean and magnitude m will change rapidly. Consequently, a level detector able a to detect fast changes in the signal level on both axes is reequired in order to separate the fixations from the saccadic mootions in the signal. There are several methods for detecting chhanges in a signal, used to detect faults in systems or to quickly give g an alarm [8]. A method of detecting changes in the mean is Cumulative Sum (CUSUM) i order to cancel HF spikes. In [8], but this is difficult to use in this research, signal level chaange is detected based on the envelope detector principle. By B using this technique, the maximum/minimum value of the t signal in real time can be determined precisely. The tim me constant of the envelope detector must be chosen so that the HF spikes be neglected and m/minimum value of the signal. thus not detected as maximum The detected level is used as dynamic d reference level by the

pupil detection algorithm in order to eliminnate the HF spikes from the signals. While the signal is lowerr than the dynamic reference level, its current sample takes the value of the previous sample and thus the spike is eliminated. This procedure is illustrated in Fig. 5 for two practtical applications in assistive technology: when the user gaze folllows a repetitively sinusoid on the entire screen with a period off approximately two seconds (Fig. 5.a)) and when the user gazee follows randomly nine distinct zones on the screen, reprresenting different ideograms or keywords, according to the user’s needs (Fig. 5.b)). The experimental results illustrated in Figg. 5 show that if the signal provided by the pupil detection algorithm (X) is below the threshold representing the dynamic reeference level, the spikes are eliminated from the signal by the proposed p algorithm. Next, the resulted signal (without HF spikkes) XP is low-pass filtered in order to reject the high frequencyy noise component, and the Xf signal is obtained.

Different types of IIR LP PF have been comparatively analysed. The design of thesee filters have been performed considering a trade-off betweenn the filter’s order and delay for the sampling frequency providedd by the eye tracker application. For the proposed eye tracking application (with the sampling frequency of 30Hz), the best results have been obtained with the fourth order IIR LPF in Chebyshhev approximation. The optimal parameters of thhe real-time filters which meet the design requirements dependd on the sampling frequency of the signal provided by the pupil detection algorithm. For different applications with the sampling frequency smaller than 20Hz, the signal filttering it is not always necessary. The proposed techniques haave increased significantly the accuracy and reliability of the eyye tracking system. ACKNOWLEEDGMENTS The work has been carried out within the program Joint Applied Research Projects, funnded by the Romanian National Authority for Scientific Research (MEN – UEFISCDI), contract PN-II-PT-PCCA-2013-4-0761, no. n 21/2014 (SIACT). REFER RENCES [1] [2]

[3]

[4] [5]

[6]

[7]

Figure 5. Raw signal X (──), processed signal withouut HF spikes, XP (─·─), LPF signal, Xf (---) and the dynamic reference level onn the X axis when the user gaze follows on the screen: a) a sinusoid; b) nine distinct random zones

The improved pupil detection algorithm m presented in this paper has been implemented on the proposedd eye tracker device and the experimental results obtained shoowed a significant improvement of the cursor’s stability on the user u screen. V.

[8]

[9]

[10] [11]

CONCLUSION

The goal of this work has been to improve the performance of an eye tracking system by using real time low w-pass filtering and HF spikes cancelling from the signals provvided by the pupil detection algorithm. Thus, the overshoot (which determines cursor oscillation on the user screen) proviided by HF spikes when high order filters are used, are eliminateed completely.

[12] [13]

[14]

R. Bolle and S. Pankanti, eds, e A. Jain, “Biometrics: Personal Identification in a Networked Sociiety”, Kluwer, 1999; Amer Al-Rahayfen, Miad Faezipoour, “Eye Tracking and Head Movement Detection: A State-of-Art Surveyy”, Rehabilitaion Devices and Systems, IEEE Journal of Translational Enngineering in Health and Medicine, vol. 1, 2013, 10.1109/JTEHM.2013.22289879, 2013; Juan J. Cerrolaza, Arantxa Villaanuevay, Maria Villanueva and Rafael Cabeza, “Error Characterization and Compensation in Eye Tracking Systems”, Symposium on Eye Traacking Research & Applications ETRA 2012, Santa Barbara, CA, March 28 2 – 30, 2012; Young L.R., Sheena D., “Eye movement measurement techniques”, Amer. Physiologist 30: 315-30, 19975; Jacob R.J.K., “The Use of Eye E Movements in Human-Computer Interaction Techniques: What Yoou Look At is What You Get,” ACM Transactions on Information Systeems, vol. 9, no. 3, pp. 152-169, 1991; Dongheng Li, David Winfield, Deerrick J. Parkhurst, “Starburst: A hybrid algorithm for video-based eye tracking t combining feature-based and model-based approaches”, Humann Computer Interaction Program, Iowa State University, Ames, Iowa, 500010; R. G. Lupu, F. Ungureanu, R. G. G Bozomitu, V. Cehan, “Eye Tracking Performance Improvement”, in Prroc of the 18th International Conference on System Theory, Control and Computing, ICSTCC 2014, 17 – 19 October, 2014, Sinaia, Romania; Pontus Olsson, “Real-time andd Offline Filters for Eye Tracking”, Master’s Degree Project, Stockhoolm, Sweden, April 2007, XR-EE-RT, 2007:011; Sheena D. , Borah B., “Compenssation for some second-order effects to improve eye position measuremeents”, Eye movements: Cognition and visual perception, pp. 257-268, 19981, Hillsdale, Nj: Erlbaum; N. Cherabit, F. Z. Chelali, A. Djerradi, “Circular Hough Transform for Iris localization”, Science and Technoology 2012, 2(5), pp. 114-121; J. Daugman, “High confidence viisual recognition of persons by a test of statistical independence”, IEEE Transactions on Pattern Analysis and Machine Intellegence, vol. 15, no. 11, pp. 1148–1161, 1993; K. Nishino and S. Nayar, “Eyes for fo relighting”, ACM SIGGRAPH 2004, vol. 23, no. 3, pp. 704–711, 2004; J. H. McAuley and C. D. Marrsden, “Physiological and pathological tremors and rhythmic central mootor control”, Oxford University Press 2000, http://brain.oxfordjournals.org/content/123/8/1545.full.pdf+html; Oleg Špakov, “Comparison of eye movement filters used in HCI”, Proceeding of the 2012 Sympoosium on Eye Tracking Research & Applications, ETRA’12, ACM, ppp. 281-284, 2012.