Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Procedia Engineering
ProcediaProcedia Engineering 00 (2011) Engineering 29 000–000 (2012) 3608 – 3612 www.elsevier.com/locate/procedia
2012 International Workshop on Information and Electronics Engineering (IWIEE)
Non-Invasive Eye Tracking Technology Based on Corneal Reflex Di Gaoa*, Guisheng Yina, Weijie Chenga,Xiaoning Fenga a
Harbin Engineering University, Harbin, 150001, China
Abstract In order to ameliorate deficiencies existing in eye tracking devices, a new method of eye tracking technology using the five near-infrared light as the source of corneal reflex is put forward and it overcomes the shortcoming of existing devices, such as complex, having too many restriction on the position and needing to wear special equipment, and so on. This method does not need any wearing equipment and can adapt to the natural head movement, the algorithm designed calculates the position of sight in high accuracy. A new method of pupil edge fitting is put forward to fit circularly and remove false points to get accurate position of pupil center in order to improve mapping accuracy. The grayscale of image is used to initialize eye position quickly and accurately. The theory of cross-ratio invariance is used to make coordinate mapping and calculate the attention coordinate accurately.
© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Harbin University of Science and Technology Open access under CC BY-NC-ND license. Keyword: Eye tracking; corneal reflex;Pupil edge fitting; Cross-ratio
1. Introduction In recent years, with the fast development of science and technology, eye tracking technology has been the focus research of the foreign and national professionals. However, due to the eye tracking technology’s relatively late start and restrictions of technical support, the research is still in an embryonic stage and none of the practical technology can be used in our daily lives. Eye tracking technology [1] can be based on the hardware and based on software by different medium. There are several categories of eye tracking devices, such as compulsory and non-compulsory device ;
*Di Gao. Tel.: +86+ 13946141381; E-mail address:
[email protected]
1877-7058 © 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. doi:10.1016/j.proeng.2012.01.539
2
DiD.Gao / Procedia – 3612 Gaoetetal. al./ ProcediaEngineering Engineering29 00(2012) (2011)3608 000–000
wearable and the wearable device; contact and non-contact device.In recent years, To overcome the Interferences caused by Eye tracking devices, the non-interfering Eye tracking method which is realized by software [2] is proposed. Its working principle is to use image processing technology. Video camera is used to catch eye images and to process images. Then software is used to realize the eye location and tracking eye location in the image, which can accurately detect and extract eye movement position information. The line of Watch coordinates can be concluded by using analysis and calculation. At present, some study on Eye tracking technology based on corneal reflex has been done.。But core of Pupil edge fitting，Such problems as localization of Pupil Center、calculate eye gaze technology do not have accurate methods.（In some existing research (such as LiuRuiAn [3])，people adopted a random elliptic fitting to extract the pupil edge. But there will be noise point in images, which may take noisy points as elliptical points fitting. The result from above-mentioned method is not accurate, which can’t fix the pupil centre accurately. Kalman filter method [4] is adopted to use original data to estimate the future data. Since using Kalman filter of the fast change of motion, the filter can’t track the real state of the change, which makes a large deviation. The application of research on eye tracking technology is extremely extensive. The realization of the eye tracking technology is to expand visual control. Visual control could take the place of manual operation, regards to man-machine interaction. This paper puts forward a non-invasive eye tracking technology based on corneal reflex. This method does not need any wearing equipment, and can adapt to the natural head movement. The algorithm designed by this paper calculates the position of sight in high accuracy. The technology uses five outside line light source to obtain accurate parameter and use projective geometry principle to do spatial mapping, so as to improve the accuracy of visual position. In the process of eye operation, the system needs to make change and map between three-dimensional space coordinate system and two dimensions coordinate system to obtain accurate eye-gaze coordinate position. Firstly, we adjust and set the parameters; secondly, we use image acquisition card to grab eye image, then to process the image in the PC as to calculate the pupil centre position; finally, we use single camera, display screen,eye and infrared source space coordinate mapping to calculate eye gaze screen coordinate. The set screen button will respond to the eye gaze screen coordinate.
2. Eye Tracking Scheme 2.1. ROI looking for the ROI To quickly extract eye features is the foundation for the next step of work. This study has proposed a new and fast detection method which is on the basis of experiments. Different parts of eyes have different reflectivity and refractive index to infrared light. The pupil is weakest to the Infrared light reflection. That’s why the pupil is black in Infrared light, but the other part of eye is not black. We can quickly fix the eye position using the above-mentioned point which can save the time and the expense. At the same time, the method has high precision. We can adopt classic threshold technology to process gray image. Making the use of all the points which are less than threshold value (xi, yi)(i=1…n) to rough fix pupil centre position in the binary image: 1 N (1) x pupil = ∑ xn N n =1 1 N (2) y pupil = ∑ yn N n =1
3609
3610
Di Gao al. / Procedia Engineering 29 (2012) 3608 – 3612 D. Gao et al./etProcedia Engineering 00 (2011) 000–000
3
According to the rough position pupil centre, we can set the 60*60 pixels rectangular region which center around the pupil centre to be ROI. Then we can narrow the range of image processing and only operate on the areas of interest to improve image processing efficiency. 2.2 Light Spot Coordinate Calculation: Due to the effect of image noise and head movement, the light spot intensity might be instability. The light spot could be a little dark, sometimes. It is more difficult to trace the light spot. Therefore we propose a quick method to extract light spot. Firstly, we divide image according to experienced threshold. Because the gray scale value of light spot in the image is highest, we can use proper threshold processing to get light spot information. Secondly, we use mass center method to calculate their central coordinate, at the same time, we can use the relative position of light spot to detect and eliminate false light spot information. Thirdly, if there is no light spot at all, we can use the light spot coordinate that is found to forecast the missing one as while as decreasing threshold and searching the missing light spot in the prediction region. It will not stop until we find the light spot. 2.3 accurate localization of pupil center Image preprocessing：Due to the effect of noise, light spot and the shelter from upper eyelid and eyebrow, the accurate extraction of the pupil edge becomes very difficult. Firstly, we blackening the light spot coordinate we found, so it will reduce the influence of the light spot to the pupil edge. Then, using median filter to smooth can further reduce the influence of light spot and noise. Due to the large computation of median filter，we only process ROI to reduce the computation and to meet real-time requirement. Extraction of pupil threshold：Because eye image formation is sensitive to infrared light, the entire gray value of image increases when the light is stronger and vice versa. If we set fixed threshold, it can’t adapt to the natural conditions to make pupil position produce error when the image degree of light and dark changing with the light condition. Therefore we have to use adaptive threshold to ensure the accuracy of pupil localization. Through analysis of eye histogram to get the pupil threshold can ensure that every threshold of image has appropriate adaptive threshold. About the gray image, we can return pixel point numbers of each gray scale. The Y-axis in the figure3 is pixel point numbers（y(n)），and the X-axis is grayscale value(n). Due to the noise of the image, we need to use formula (3) to average neighbour 3 points low-pass filter to histogram data. Then we can get smooth histogram to analysis. y= ( y (n − 1) + y (n) + y (n + 1)) / 3 (3) s The left wave crest corresponds to the pupil, and to right corresponding to iris and light spot. We extract adaptive threshold by looking for wave trough. Extraction of pupil boundary point：Due to being sensitive to the accurate division to the threshold and the difficulty of the influence of the light spot on the cornea, we can’t get boundary points of pupil well with simple threshold segmentation technology.
DiD.Gao / Procedia – 3612 Gaoetetal. al./ ProcediaEngineering Engineering29 00(2012) (2011)3608 000–000
4
2.4 Coordinate mapping According to the four calculated faculae on the screen and the calculated pupil’s central coordinate, projective geometry theory is used to process the space coordinates mapping. Invariance of Cross Ratio principle [11] is used to calculate.
3. Experiment and analysis This research implements a vision controlling mouse system. This system is designed with visual C++ in windows XP. People sit before the computer in a normal computer operation habit. The screen’s size is 17-inch and the screen’s resolution is 1280×1024 pixels. The viewpoints can be demarcated through 49 ticks on the screen. We extract the characteristic parameters of different testers first, characteristic parameters are checked and process the vision location experiment. The viewpoints can be demarcated through 49 ticks on the screen. We can check the accuracy of the experiment by seeing the difference between the actual gazing point and the tick’s coordinate. We implement a testing interface which has 20 areas of 200*120 pixels, and the tester gaze at the button center in order to locate the mouse point’s location. Click the mouse on site if the system calculates the eye gazing point in the button area, the button flashes once, else nothing happens or other button flashes. 5 testers’ statistical testing data is followed in Table 2 and 3: Table.1 statistics on 5 testers by traditional method Person NO.
Clicks
Responses
Corrections
Validities
1
100
43
37
37%
2
100
51
45
45%
3
100
47
40
40%
4
100
38
30
30%
5
100
61
50
50%
Table.2 statistics on the 5 testers by proposed method Person NO.
Clicks
Responses
Corrections
Validities
1
100
89
85
85%
2
100
97
93
93%
3
100
98
95
95%
4
100
92
90
90%
5
100
95
88
88%
From the button response experiment, we can see that this method improves the precision of the location a lot. The effective data improves 40% compared with the traditional method. On the one side it shows that the precision improves, on the other side it shows the stability of gazing point improves. The head can move freely in the area that CCD video camera can capture. The head’s movement has little influence on the result. It has the best result when the head faces the screen straightly and the distance between the head and screen is 50-80CM. Besides the nearer the gazing point approaches the
3611
3612
Di Gao al. / Procedia Engineering 29 (2012) 3608 – 3612 D. Gao et al./etProcedia Engineering 00 (2011) 000–000
5
screen center, the more accurate the location’s coordinate is. Every square button’s location is precise. But the left side and right side may be less precise than the center. The experiment result proves that using the cycling elliptic fitting method to extract the pupil’s edge and reject the false points to get accurate position of pupil center. It improves a lot compared with the traditional method. Using cross-radio to map is more precise than Kalman filter method.
4. Conclusion This research puts forward a new eye tracking technology. This method does not need any wearing equipment and reduces the limitation to users. A new method of pupil edge fitting is put forward to fit circularly and remove false points to get accurate position of pupil center in order to improve mapping accuracy. Compared with the current research this research is more accurate. This research can be used to every aspect of human-computer interaction and it has broad application prospect 5. Acknowledgements This work was partially supported by Fundamental Research Funds for the Central Universities of China (HEUCF100605). References [1]Xiaokun Li; Wee, W.G.. An efficient method for eye tracking and eye-gazed FOV estimation. Image Processing (ICIP), 2009 16th IEEE International Conference on Digital Object Identifier: 10.1109/ICIP.2009.5413997 Publication Year: 2009 , Page(s): 2597 – 2600 [2]Liu, Tao; Pang, Changle;Eye-Gaze Tracking Research Based on Image Processing .Image and Signal Processing, 2008. CISP '08. Congress on Volume: 4 . 2008: 176 – 180 [3]Ruian Liu; Xin Zhou; Nailin Wang; Adaptive Regulation of CCD Camera in Eye Gaze Tracking System Image and Signal Processing, 2009. CISP '09. 2nd International Congress on [4]Chenjian Ran; Zili Deng;Two correlated measurement fusion Kalman filtering algorithms based on orthogonal transformation and their functional equivalence.Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE Conference on . 2009: 2351 - 2356 [5]Yi Hong; Sam Kwong; Hanli Wang;Decision-based median filter using k-nearest noise-free pixels. Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on ,2009: 1193 – 1196 [6]Makila, P.M.; Intrinsic difficulties in stochastic control of unstable convolution operators on Z. Automatic Control, IEEE Transactions on Volume: 48 , Issue: 11 2003: 2015 – 2018 [7]Zhang Jin-Yu; Chen Yan; Huang Xian-Xiang; Edge detection of images based on improved Sobel operator and genetic algorithms. Image Analysis and Signal Processing, 2009. IASP 2009. International Conference on 2009: 31 – 35 [8]Yuan-Kai Huo; Gen Wei; Yu-Dong Zhang; Le-Nan Wu; An adaptive threshold for the Canny Operator of edge detection. Image Analysis and Signal Processing (IASP), 2010 International Conference on 2010: 371 – 374 [9]Kerr, D.; Coleman, S.; Scotney, B.; Near-Circular Corner and Edge Detection Operators. Machine Vision and Image Processing Conference, 2007.IMVIP 2007. International 2007: 7 - 14 [10]A. Flizgibbon, M. Pilu, R.B. Fishe, Direct least square fitting of ellipses, IEEE Trans. Pattern Anal.Mach. Intell. 21 (5) (1999) 476–480. [11]J ． BatHe ， E ． Mouaddib and J ． Salvi ． Recent progress in coded structured fight as a technique to solve the correspondence problem：a survey[J]．Pattern Recognition, 1998，3l(7) ：963—982
