Region of Eye Contact of Humanoid Nao Robot Is ...

Region of Eye Contact of Humanoid Nao Robot Is Similar to That of a Human Raymond H. Cuijpers and David van der Pol Eindhoven University of Technology, Den Dolech 2, 5600 MB Eindhoven, The Netherlands [email protected], [email protected]

Abstract. Eye contact is an important social cue in human-human interaction, but it is unclear how easily it carries over to humanoid robots. In this study we investigated whether the tolerance of making eye contact is similar for the Nao robot as compared to human lookers. We measured the region of eye contact (REC) in three conditions (sitting, standing and eye height). We found that the REC of the Nao robot is similar to that of human lookers. We also compared the centre of REC with the robot’s gaze direction when looking straight at the observer’s nose bridge. We found that the nose bridge lies slightly above the computed centre of the REC. This can be interpreted as an asymmetry in the downward direction of the REC. Taken together these results enable us to model eye contact and the tolerance for breaking eye contact with the Nao robot. Keywords: gaze perception, eye contact, cognitive robotics, social robotics.

1

Introduction

Robots are applied more and more in domestic environments within which they need to interact with people in increasingly sophisticated ways [14]. For example, in the KSERA project a humanoid robot is integrated with a smart home environment in order to support independent living of seniors in their own homes [15, 7]. Such applications place an increasing demand on robots with respect to their social and cognitive skills [18]. It is generally believed that robots with decent social and cognitive skill are more easily accepted [5]. Especially, for humanoid robots people expect a certain basic level of verbal and non-verbal behavioural intelligence to be present [14]. One such non-verbal social cue is gaze and eye contact: it has been shown that mutual eye contact with robots can add fluency to the engagement [1] and increase the level of engagement [9, 13]. Thus, eye contact is relevant to many interactive robotic applications such as care for elderly [16, 15]. Eye contact is especially relevant when using robots in therapy interventions for autistic spectrum disorder [2, 12], because eye contact facilitates shared attention and turn taking, which is known to be impaired in people with autism. G. Herrmann et al. (Eds.): ICSR 2013, LNAI 8239, pp. 280–289, 2013. © Springer International Publishing Switzerland 2013

Region of Eye Contact of Humanoid Nao Robot

281

However, much is still unclear about eye contact with robotic agents. As the facial features of humanoid robots vary considerably between platforms and deviate substantially from human facial features, it is necessary to investigate the perception of eye contact for robotic agents. A popular platform for studying non-verbal communication is Aldebaran Robotics’ Nao robot [2, 16]. The Nao robot is a 57cm tall humanoid robot with a moveable head and facial features that resemble eyes and a mouth. Although Nao’s physical appearance resembles that of a human, its detailed features still differ considerably. For one, its visual sensors differ from the human visual system: its main camera has a limited resolution and a limited field of view, and the ’eyes’ of the Nao robot cannot move within the head nor do they correspond to the actual camera locations. Even so, it is intuitively clear that robot can ’look at you’ or not by orienting its head, and, thus, it can establish eye contact. In human-human interaction studies eye contact influences all sorts of social behaviours and many relevant parameters associated to gazing have been identified like gaze direction, gaze duration, gaze frequency, glancing, interactions with facial expressions and more [8]. For eye contact a particularly relevant feature is the region of eye contact (REC). It is a measure for the tolerance people have for gaze directions as being perceived as eye contact. This range of gaze directions maps onto a region within the observer’s face (see Figure 1), and it is much larger than one would expect based on the accuracy with which people can distinguish gaze directions [6]. The REC is about 8.1 degrees across for a viewing distance of 1m, and about 3.9 degrees for a distance of 5m [6]. We found a REC that is 7 degrees across at a distance of 1m, both horizontally and vertically, for pictorial displays of human faces [10], which is very similar. We also found that it is slightly larger in the downward direction than in the other directions (see Figure 2). When uncertain, perceivers are more prone to judge gaze as eye contact, resulting in an increase of size of the REC with distance [6]. According to Chen [3] the vertical asymmetry of the REC occurs because people reduce their eyelid separation when looking down and not when looking up. Closing the eyelids obscures the pupil position inducing additional uncertainty resulting in a larger REC in the downward direction. In this study we therefore measure the region of perceived eye-contact for the Nao robot. We expected that the shape of the REC would be similar to humanhuman interaction, but that the size may be larger because the facial features of the Nao robot only resemble that of a human leading to more uncertainty. In addition, we varied the relative height of the Nao robot with respect to the observer. As the Nao robot is small it will frequently happen that Nao will be looking up whereas the observer looks down. We suspected that this could negatively affect the tolerance for eye contact, and, thus , increase the size and/or asymmetry of the REC. With this information we can model ’eye contact’ with the Nao robot and use this as a social cue to make human-robot interaction more natural and acceptable.

282

R.H. Cuijpers and D. van der Pol

Fig. 1. Region of Eye-Contact (REC) is the range of gaze locations of looker on the face of the observer for which eye contact is perceived

Fig. 2. The size of the REC in degrees for life size pictures of human heads at 1.25m distance [10]

2 2.1

Method Participants

Six participants took part in the study. Two of them were female and four of them male. Al participants were employees of the university and knew about the KSERA project, but they were naive with respect to the specific details of the experiment. The right eye was the dominant eye of all participants. Participants had normal or corrected-to-normal vision and were between 1.65 m and 1.89 m tall.


2.2

283

Design and Task

The participants had to perform two tasks. The centring task and the nose-bridge task. In the centring task, the participant had to adjust the head orientation of Nao until it appeared to make eye contact. Thus, the measured head orientation corresponds to the edge of the Region of Eye-Contact (REC). There were four directions in which the participant had to adjust the head orientation (up, down, left, right) and for each of these centring tasks the robot’s initial gaze direction was directed away in the opposite direction (see Figure 3). It is important to note that all gaze directions obtained in this way are perceived as making eye contact. In the nose-bridge task participants had to adjust the head orientation until the robot appeared to look straight at the observer’s nose bridge.

Fig. 3. Different initial gaze directions for the different tasks. (1) Nose-Bridge task, (2) Right task, (3) Down task, (4) Left task, (5) Up task.

A within-subjects design was used for the experiment. There were three conditions: (1) a standing condition, (2) a sitting condition, and (3) a eye height condition (see Figure 4). For each condition, the REC was measured from four directions (see Figure 3) and each direction was repeated four times. The location of the nose-bridge was also measured from four directions (upper left, upper right, lower left and lower right), but each direction was repeated only once. This results in a total of 3 (conditions) x 4 (repetitions) x 4 (directions) + 3 (conditions) x 1 (repetition) x 4 (directions) = 60 trials. 2.3

Experimental Setup

We used a 19” TFT screen to present the task instructions to the participants. The humanoid robot Nao (Aldebaran Robotics, France) was used as the anthropomorphised looker. We recorded the relative yaw and pitch angle in degrees,

284


the completion time for each task, and the size and position of the detected face. We used OpenCV’s implementation of Viola & Jones’ object detection algorithm [17] to detect the face from the image of Nao’s main camera that was recorded at the moment that the “enter” key was pressed. Nao was either located on a small 40 cm high table (standing and sitting condition), or an 80 cm high table (eye-height condition). During the sitting and eye-height condition participants were seated in a chair that was adjusted to the same height for all participants. In every condition Nao was placed at 160 cm from the participant (see Figure 4).

Fig. 4. Experimental setup. Upper pane shows the setup for the standing condition. Nao is located on a small table while the participant is standing. Middle pane shows the setup for the sitting condition. Nao sits on a small table while the participant is sitting on a chair. The lower pane shows the setup for the eye-height condition. The participant is sitting while Nao is on the tabletop. Nao is located at eye-height.


2.4

285

Procedure

Every participant’s body height was noted, and a test was done to determine the visual acuity and the dominant eye. Participants were asked to sit down on the chair and perform a few practise trials. The experimenter was present during these practise trials to explain the different tasks. If the participant understood what to do for the different tasks, the actual experiment was started, and the participant was left alone. The experimenter could monitor the experiment’s progress through Nao’s camera. In the centring task, Nao moved its head to one of the initial gaze directions and told the participant in which direction to move Nao’s head (see Figure 3). The participant moved the robot’s head by pressing the corresponding keys on the keyboard. As soon as the participant perceived eye-contact (s)he pressed the “enter” key and the robot’s head pose and front camera image was recorded. In the nose-bridge task Nao looked away from the participant in one of four directions. The participant then moved Nao’s gaze toward their nose-bridge adjusting both the yaw and pitch angles of Nao’s head. Participants pressed the ’enter’ key when satisfied. This was repeated until all trials were completed. The order of presentation was randomised for each participant.

3

Results

In Figure 5 the REC is shown for each of the four directions of approach of the robot’s head (cornerpoints of the solid line) along with the 95% Confidence

Fig. 5. Region of eye contact (REC, solid line) from the participants’ point of view. The centre of the REC (triangle) does not coincide with the origin, but it is horizontally aligned with the gaze directed at the observer’s nose bridge (square).

286


intervals (corner points of dotted lines). The width of the REC at its widest point was 3.7 ± 0.7 deg (SD = 1.3 deg) and its vertical span was 5.0 ± 0.8 deg (SD = 1.5 deg). We used the centre of the image coming from Nao’s forward camera as its presumed gaze direction. Ideally, the nose bridge of the participant should be in the centre of the image in the nose bridge task. We found that this centre of the REC (triangle in Figure 5), lies approximately 0.52 degrees (2,5 pixels in a 320x240 window) to the left and 0.39 degrees (1.9 pixels in a 320x240 window) of the image centre. The robot’s head oriention when looking straight at the participant’s nose bridges is indicated by the square at (0.5◦ , 1.8◦ ). The centre of the REC differed from the measured nose bridge location: the pitch angle was significantly less (t(71) = −4.85, p < 0.5), but the yaw angle was not significantly different (t(71) = 0.30, p = 0.77).

Fig. 6. The vertical (diamonds) and horizontal (squares) dimension of the REC plotted for every pose of the participant. The whiskers represent the 95% confidence intervals.

The effect of the conditions (standing, sitting or eye height) are shown in Figure 6. The height of the REC was larger than the width for all conditions, but the conditions had no significant effect on the size of the REC (F (2, 71) > 0.2, p > 0.82).

4

Discussion and Conclusions

The most important conclusion from the results of the experiment is that the perception of the gaze-direction and, specifically, eye contact with Nao as a looker, is not that different compared to a human looker. For a distance of 1.60m, we found a width of about 4◦ and a height of about 5◦ , which lies in-between the values of 8.1◦ for 1m distance and 3.9◦ for 5m distance as observed by Gamer and Hecht [6]. The similarity between the REC of Nao and a human looker is encouraging, because the Nao robot only has very basic facial features. This suggests that the realism of the facial features is not essential for the accuracy with which people judge spatial orientations of a (robotic) head. As a consequence,


287

we expect that the REC is similar for other robots as long as their heads have a well-defined spatial orientation. Unlike more sophisticated humanoid robots, the Nao robot does not have orientable eyes within the head. This may seem a serious drawback, but our results show that head rotations are sufficient to mimick eye contact with comparable tolerances. Using only head rotations for mimicking eye contact is not as unrealistic as one might initially believe. From a distance real human eyes are hardly visible let alone that their spatial orientations are discernable. Still, when the eyes are visible, people can very accurately discriminate pupil positions within the sclera of the eye [4]. It is possible that realism of facial features plays a much more important role for judgements of eye turn, also because the spatial orientations of real human eyes are typically not the same when focused on a target. Chen [3] reported an asymmetry in the REC in the downward direction for human lookers. To check whether such an asymmetry is present we compared the centre of the REC with the gaze direction obtained when the robot is looking straight at the participants’ nose bridge. We found that the computed REC (assuming symmetry) lies below the nose bridge gaze direction. Thus, if the nose bridge is considered the true centre of the REC, this difference implies that the REC is larger in the downward direction than in the other directions. Chen [3] proposed that this asymmetry is due to the eyelids making eye-movements more salient in the upward direction. However, this cannot explain the asymmetry of the REC of Nao, because it does not have eyelids. More likely, the asymmetry reflects the fact that the nose bridge is not the centre of the REC. We used a face detection algorithm to centre the robot’s gaze on a person with an adjustable offset. It may be that the centre of the head in the image of Nao’s camera is misaligned with its perceived head orientation. To test this we determined the location of the central pixel in Nao’s video frame when it was perceived as looking at the participant’s nose bridge. We found a significant difference betweem the image centre and the position of the nose bridge in the image. This suggests the camera was misaligned about 0.5◦ in both pitch and yaw. The obtained REC constrains the parameters for Nao’s eye contact behaviour. In terms of eye contact, it is better to use the centre of the REC Nao’s gaze direction because it divides the downward and upward direction in equal parts, making the chance of accidentally breaking eye contact as small as possible. The REC enables the use of different head orientations for signalling eye contact. We can use this property to signal emotions using head orientation. For example, looking down is perceived as having a sad emotional state, where looking up is perceived as having an “up” or alert emotional state. The results from this study provide a good estimate of the tolerances for eye contact with the Nao. They show that these tolerances are similar whether the person is standing, sitting or at eye height with Nao. Our results also show that care should be taken to use the centre of Nao’s camera images as looking straight ahead. Preferably, one should calibrate the straight ahead direction for

288


each robot in order to obtain the best results. The results from this study also tell us how to break eye contact. The obtained tolerances give us the precision requirements for Nao’s gaze behaviours and they allow us to signal emotion using head tilt without losing eye-contact with the observer. Since the Nao’s facial features are very basic, we expect that our results extend to other robots as long as they have heads with clear spatial orientations. Acknowledgements. The research leading to these results is part of the KSERA project (http://www.ksera-project.eu) and has received funding from the European Commission under the 7th Framework Programme (FP7) for Research and Technological Development under grant agreement n° 2010-248085.

References [1]

Breazeal, C.: Toward sociable robots. Robotics and Autonomous Systems 42(3-4), 167–175 (2003), doi:10.1016/S0921-8890(02)00373-1 [2] Gillesen, J.C.C., Barakova, E.I., Huskens, B.E.B.M., Feijs, L.M.G.: From training to robot behavior: Towards custom scenarios for robotics in training programs for ASD. In: 2011 IEEE International Conference on Rehabilitation Robotics (ICORR), pp. 1–7 (2011), doi:10.1109/ICORR.2011.5975381 [3] Chen, M.: Leveraging the asymmetric sensitivity of eye contact for videoconference. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Changing Our World, Changing Ourselves - CHI 2002, p. 49. ACM Press, New York (2002), doi:10.1145/503376.503386 [4] Cline, M.G.: The Perception of Where a Person Is Looking. The American Journal of Psychology 80(1), 41 (1967), doi:10.2307/1420539 [5] Dautenhahn, K.: Socially intelligent robots: dimensions of human-robot interaction. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 362(1480), 679–704 (2007) [6] Gamer, M., Hecht, H.: Are you looking at me? Measuring the cone of gaze. Journal of Experimental Psychology. Human Perception and Performance 33(3), 705–715 (2007) [7] Johnson, D.O., Cuijpers, R.H., Juola, J.F., Torta, E., Simonov, M., Frisiello, A., Bazzani, M., Yan, W., Weber, C., Wermter, S., Meins, N., Oberzaucher, J., Panek, P., Edelmayer, G., Mayer, P., Beck, C.: Socially Assistive Robots: A comprehensive approach to extending independent living. International Journal of Social Robotics (submitted) [8] Kleinke, C.: Gaze and eye contact: A research review. Psychological Bulletin 10, 78–100 (1986) [9] Mutlu, B., Hodgins, J., Forlizzi, J.: A storytelling robot: Modeling and evaluation of human-like gaze behavior (2006) [10] van der Pol, D.: The effect of slant on the perception of eye-contact. Master Thesis (2009) [11] van der Pol, D., Cuijpers, R.H., Juola, J.F.: Head pose estimation for a domestic robot. In: Proceedings of the 6th International Conference on Human-robot Interaction (HRI 2011), pp. 277–278. ACM, New York (2011), doi:10.1145/1957656.1957769


289

[12] Robins, B., Dautenhahn, K., Dickerson, P.: From Isolation to Communication: A Case Study Evaluation of Robot Assisted Play for Children with Autism with a Minimally Expressive Humanoid Robot. In: Second International Conferences on Advances in Computer-Human Interactions, ACHI 2009, pp. 205–211 (2009), doi:10.1109/ACHI.2009.32 [13] Sidner, C.L., Kidd, C., Lee, C., Lesh, N.: Where to look: a study of human-robot engagement. In: Proceedings of the 9th International Conference on Intelligent User Interfaces, pp. 78–84. ACM, New York (2004) [14] Tapus, A., Mataric, M., Scassellati, B.: The grand challenges in socially assistive robotics. Robotics and Automation Magazine, 1–7 (2007) [15] Torta, E., Oberzaucher, J., Werner, F., Cuijpers, R.H., Juola, J.F. (2012). The attitude toward socially assistive robots in intelligent homes: results from laboratory studies and field trials. Journal of Human-Robot Interaction 1(2), 76-99. DOI: 10.5898/JHRI.1.2.Torta. [16] Torta, E., van Heumen, J., Cuijpers, R.H., Juola, J.F.: How Can a Robot Attract the Attention of Its Human Partner? A Comparative Study over Different Modalities for Attracting Attention. In: Ge, S.S., Khatib, O., Cabibihan, J.-J., Simmons, R., Williams, M.-A. (eds.) ICSR 2012. LNCS, vol. 7621, pp. 288–297. Springer, Heidelberg (2012) [17] Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: Proc. IEEE CVPR 2001, vol. 01 (2001) [18] Wada, J., Shibata, T., Saito, T., Tanie, K.: Analysis of factors that bring mental effects to elderly people in robot assisted activity. In: Proceedings of the 2002 IEEE/RSJ Conference on Intelligent Robots and Systems, Lausanne, Switzerland, pp. 1152–1157 (2002)