The e ect of vertical viewpoint on expression perception: the Noh mask and the human face. Michael J. Lyons, Ruth Campbelly, Andre Plantez Mike Colemany, Miyuki Kamachi ATR Human Information Processing Research Labs yDept. of Human Communication Science, University College London zATR Media Integration & Communication Research Labs contact:
[email protected] Muscular changes accompanying facial emotions lead to displacement of the facial features and this is usually considered the most salient perceptual cue to emotional expression in the facial image. Detection of such change by the visual system requires sensitivity to ne metric relations on the facial surface. Rigid-body transformation of the head, speci cally out-of-plane rotation, distorts metric properties of the facial surface as they appear in the two-dimensional projection of the visual world on the retina. It is interesting to ask whether the distortion due to rigid displacement of the head interferes with the interpretation by visual system of the non-rigid displacement of internal facial features due to facial expression. Our interest in this question was stimulated when we learned that certain masks used in Japanese Noh drama (particularly those used to portray young female roles such as Ko-omote, Wakaonna, and Magojiro) appear to change expression as the vertical inclination of the mask is changed. Tilt the mask forward and it appears to smile; tilt it backwards and it appears sad. Is this phenomenon evidence that facial expression perception may not be invariant to changes in viewpoint? We examined the Noh mask e ect in a series of psychological experiments. For the rst experiment, an authentic Noh mask was photographed at inclinations varying from ,30 to +30 in 5 increments. Two groups of subjects, 20 Japanese and 20 British, were shown brief (400 ms) exposures of the images and asked to respond whether the expression on the mask appeared happy or sad. Mean responses showed a main e ect of inclination and an interaction of culture and inclination but no signi cant e ect of subject gender. There was an approximately linear relationship between angle of inclination and judgement of negative-positive a ect in the display. This e ect is moderated by culture: Japanese participants rated the displays more positively at negative inclinations (head back) and more negatively at extreme positive inclinations (head down). We hypothesized that the non-monotonic dependence of apparent emotional a ect on inclination in Japanese viewers may be a result of con icting cues from the head pose and internal facial feature shape. In the second experiment, head pose cues were reduced by deleting mask areas outside a frame enclosing the internal features. Twenty Japanese and 20 British subjects participated in a study identical to the rst one, but with the modi ed stimuli. Mean responses again showed a main e ect of inclination and a an interaction of culture and inclination: Japanese participants rated more positively at negative inclinations. However, the negative dip at positive inclinations was eliminated, in support of our hypothesis about the e ect of head pose.
Stimuli for experiment 3 consisted of images of a Japanese female model posing a neutral expression somewhat similar to that seen on the Noh mask. A Cyberware Colour 3D scanner was used to acquire separate shape and colour information of the model's head. Views of the model's head were taken with vertical viewing angles varying from ,30 to +30 spaced at 5 increments. Twenty Japanese and 20 British subjects participated in a study identical to the rst two, but for the change in stimuli. The only signi cant e ect shown by the mean responses was a main e ect of vertical viewing angle, and no signi cant cultural di erence was observed. The viewpoint e ect was somewhat less pronounced than for the modi ed Noh mask stimuli (experiment 2). We can conclude (a)There is a systematic relationship between judgement of positive vs negative a ect from a facial image, as a function of the vertical inclination of the head. Head down - happier appearance to the facial features, head back - sadder. (b) When the image is of the speci c cultural artifact, the Noh mask, culture-speci c factors emerge. In particular, while naive viewers may base their judgements simply on the curvature of the mouth, less naive (Japanese viewers) are sensitive to visual cues to pose of head (when these are visible), and possibly to other con gural cues to expression. Because Japanese and British subjects did not di er in their ratings of a laser-scanned head, cultural di erences apparent for Noh mask viewing are probably not related to di erences in the use and extensions of the terms 'happy' and 'sad' in Japanese and English, but have their origins in perceptual factors. Examination of the 3D shape of the Noh mask and the human face reveal the source of the e ect. The lips of the mouth may be approximated by semi-circular arcs. When the head is tilted forward the edges of the semi-circle point upwards relative to the center of the arc giving the impression of a smiling mouth. When the head is tilted back, the edges of the semi-circle move downwards relative to the center, suggesting a sad mouth with drooping corners. The appearance of other facial features such as the eye and eyebrow regions are also a ected by changes in vertical viewpoint. Comparison of the Noh mask and human face show that the curvature of the mouth region of the mask is exaggerated. This increases the e ect of inclination on the apparent mouth shape. Small movements of the actor's head produce eeting changes in the mask's apparent expression. Rigid motion of the head produces illusory perception of non-rigid displacement of the internal features of the mask - giving it animated appearance. It seems probable that the artists who set the form of these masks, during the Kamakura period (1192-1333), had sucient intuitive understanding of these visual e ects to modulate them by manipulation of the natural 3D shape of the face.
References [1] K. Komparu, \The Noh Theatre: Principles & Perspectives" Weatherhill/Tankosha, New York & Tokyo, 1983. [2] A. Kappas, U. Hess, C. L. Barr, and R. E. Kleck, \Angle of Regard: The e ect of vertical viewing angle on the perception of facial expressions," Journal of Nonverbal Behavior 18(4): 263 - 280 (1994).
