Design Expressive Behaviors for Robotic Puppet ... representational behaviors for the robotic puppet. ..... Robot, Master Thesis, Department of Computer and.
Seventh International Conference on Control, Automation, Robotics And Vision (ICARCV’02), Dec 2002, Singapore
Design Expressive Behaviors for Robotic Puppet Shusong Xing
I-Ming Chen
School of Mechanical and Production Engineering Nanyang Technological University, Singapore 639798 in behavior-based method are motor primitives for humanoid robot control [7][12][13][14].
Abstract This paper describes the generation of expressive gestures on a robotic puppet controlled using behaviorbased method. Recent findings on biological motion mechanisms provide an inspiring model for controlling high dimensional robot with motor primitives. The modular features of primitives make them the building blocks for constructing complex behaviors. The inborn similarity between gesture and speech enable us to study behavior with those methods for language. We designed a motor controller using a small set of primitives. The effectiveness of this method is validated on the robotic puppet with different expressive gestures generated by superimposing and sequencing the primitives.
1 Introduction Similar to the non-verbal behaviors exploited as an essential media in inter-personal communication, a robot’s expressive behaviors enable it to convey information to humans in a natural and intuitive manner. The pattern of expressing information with body gestures offers an alternative to traditional information output interfaces driven by monitors, menus and icons. Such a robot capable of affecting people in a comfortable fashion may find applications in the domains of entertainment and education. In this paper we describe the implementation of a robotic puppet (Fig. 1), whose behavior set is created under the frame of the behavior-based method.
Figure 1. The robotic puppet
2 Motivation Primitives are modular motor control circuits including spinal fields and center pattern generators [12]. In robotics, they refer to the basic motor control programs from which the entire movement repertoire can be generated. Recent investigations [3][15][16] on the vertebrates like frogs found the existence of force-field motor primitives. The central nervous system may generate a wide repertoire of motor movements through the vectorial superposition of a small number of motor primitives stored within the neural circuits in the spinal cord. Some primitives can inhibit others when they are active. The principle of biological motor control suggests an appealing model for designing humanoid robots, that is, the set of all behaviors the robots have may be built from a limited number of low-level primitives. This new perspective on robot design will
The behavior-based method gets inspirations from motion control mechanism of biological systems [2]. It provides a distributed and bottom-up architecture for setting up robot controllers. As a rather high-level abstraction, each behavior is a functional block that may further comprise low-level control schemes, such as PID control [14]. The behavior-based method has been widely applied to mobile robots in noisy and dynamic environment because of its situated, responsive and emergent characters [2][10]. The application field of behavior-based method is not only restricted to robot navigation. Recently developments
378
for the robotic puppet and those expressive behaviors could convey information as a kind of non-verbal language.
dramatically simplify the difficulties encountered in robot control. The motor primitive concept shows clearly that neurobiology offers insights to the design of humanoid robots.
Cerebral Motor Activities
To control a highly articulated robot with a large number of degrees of freedom (DOF) is a challenging task because of the kinematics, dynamics calculation and the redundancy of the system. The modular and parallel characteristics of motor primitives enable us to decompose the whole problem into some smaller ones. In most conditions, only part of the total DOFs of a robot are directly involved in the motion control. Attracted by this advantage, primitives become one of the major methods to address the control problem of humanoid robots. Mataric[12][13][14] has used behavior primitives to control a robot arm in order to overcome the difficult problems like inverse kinematics. In [5][12], sequential action data were collected, categorized and recognized with the expectations of finding out behavior primitives and then using them in the task of imitating the behaviors of human. Taking another way, Ijspeert [9] designed a network of central pattern generators to make adaptive behaviors appear on a simulated amphibian. They exploited knowledge of neurophysiology and cognition in their works that took advantages of the flexibility and efficiency of biological motion control model, which provides a desirable reference for studying complex articulated mechanisms.
Primitives
Primitives
Primitives
Limbs, hands
vocal cords
face
Gestures
Speech
Expressions
Figure 2. The generation of gestures and speech. The studies on gesture communication aim to discover new and obvious gestures in order to convey information transparently. Finding and defining sets of useful gestures will be probably remain an application specific effort until gesture-based interaction has been understand in depth. Human only uses a small portion of the total working space of the body. The most obvious and common gestures used in daily communication are easily to be ignored. The mixture of motion trajectories, social context and noisy background increase the complexity of distinguishing and differentiating, especially when the gestures are intertwined with speech. The findings of the essential role that primitives act as in building a vocabulary of behaviors provides an alternative perspective to study gestures with a bottomup method. The same origin where both the gestures and speech come is the solidest explanation to the obvious similarity and intertwined relationship between them. So it is reasonable that the study on robot behaviors is connected with the study on languages and some methods for studying language may be used in the study of behaviors [6].
For our robotic puppet, behavior primitives are used not only as the abstract modules for robot motor control, but also as the building blocks of complex behaviors because our goal is to create a rich repertoire of representational behaviors for the robotic puppet.
3 Expressive Behaviors Within interpersonal communication, the voice stream of speech is the main medium of information transmission but not the only one. Nonverbal signals, including body postures and facial expressions, are important means that can convey meanings and ideas at the linguistic, semantic and emotional levels, especially in multimodal communication. Motor theory [1] regards that the language originated from the transfer of motor activities from previously controlling body movements to the articulatory organs. The elementary motor activities when directed to the articulatory organs produce an equivalent set of elementary articulatory programs generating elementary speech sounds. A close link exists between the structures of language and those of human motor system. This suggests that we may use primitives to setup a vocabulary of behaviors
The performance effects of the gestures directly influence the results of our endeavors to bringing life the human-like puppet figure. To make the robotic puppet express information by exhibiting different body gestures, the functions and potentials of behaviors as a kind of major non-verbal cues in communication shouldn't be ignored. Gestures often provide a context that makes a verbal expression more precise. It can contribute additional substantive contents. Within human-machine interaction, body gestures of humanoid robots can describe information of actions, attitudes, object attributes and relationships. Usually, the
379
end fixed on the puppet body, the wires come from the motors mounted on top of the frame. This arrangement of wires can not make all 30 DOFs of the robot under control. The wires attached to each of upper arms can generate rotary movement along two degrees of freedom of the puppet’s shoulders. The wires on each leg can control two rotary degrees of freedom. The gravitation force exerts motion control to one more degree of freedom on each part of the puppet figure. So altogether 17 of the 30 DOFs are under constraint.
following four kinds of gestures are distinguished [4][11]: •
•
• •
Symbolic gestures represent some widely recognized conventionalized meanings with body configurations and movements. For example, a thumbs-up gesture indicates agreement. Deictic gestures are those that point objects in the environment with arms and figures. They may also be used to indicate unseen, abstract or imaginary things. Iconic gestures are some pictorial gestures intuitively representing physical entities in the world. Metaphoric gestures represent abstract objects or concepts.
The robotic puppet has 30 DOFs and only eight motors. Even considering the gravitational force, the number of wires is still far away from putting all the 30 DOFs of figure body under full constraint. Thus, the robotic puppet has insufficient constraint to the suspended parts and belongs to incompletely restrained wire-suspended mechanisms [17].
While symbolic gestures can be interpreted without context references, deictic, iconic and metaphoric gestures can be interpreted meaningfully only with additional information from other modalities that occur simultaneously.
4.2
The hardware setup of the robot controller is shown in Figure 3. It structurally consists of two parts: an embedded PC running Linux for high-level control and a motor control board for motor command execution. Data and command transfer is performed between them through the RS-232 protocol.
4 Experimentation 4.1
Robot Controller
Robotic Puppet
The robotic puppet system is a highly articulated wiredriven robot (Fig. 1). Different from common humanoid robots that take a serial motor-linkage structure, the design of robotic puppet follows the way of a puppeteer controlling a marionette. The puppet figure is suspended to the frame using eight overhead wires. Each wire is driven by one DC servomotor. There are eight servomotors mounted on the top of the frame in total. Though the structure is simple, this system provides us a platform to study representational behaviors of machines.
Sensor
Embeded PC
I2C Bus
I2C Bus Microcontroller motor
The robotic puppet consists of nine moving parts. The torso and the head form one piece and keep stationary in current operation. There is no more constraint added to the joints. Each of the four rotating joints has three rotary DOFs. So the puppet figure totally has 30 degrees of freedom: the torso part has six DOFs and the joints have 24 DOFs. Within the robot's threedimensional working space, the positions and orientations of the different parts of the puppet figure can be controlled by the robotic puppet mechanism through changing the length of the wires.
Figure 3. Structure of motor controller hardware. An agent program running on the embedded PC is responsible for acquiring information, keeping internal states for the robot and generating motor control commands (Fig. 4). The Sensor Filter accepts only part of stimuli in the environment. It will discard those signals that will not cause any actions and state transitions. A set of predefined gestures is stored in Gesture Representation module. They describe the desired trajectories of movements in the space. Each gesture is composed of a sequence of fundamental movements that can be implemented with primitives. When a valid stimulus
At present, the arrangement of wires on our robotic puppet doesn't follow the way of the marionettes. In puppetry, the suspending wires fall down vertically and attach to the tips of the limbs of the puppet figure. In our design, the wires are not laid vertically. With one
380
commands from the agent program, but also store and generate concurrent and reactive motor actions when events happen. This design gets inspiration from the findings of Bizzi [3] that the spinal motor system of fog is an active participant in several aspects of the production of movement besides relaying motor commands as a passive role. This suggests that interactions among low-level behaviors and primitives are a source of complex behaviors.
arrives, one gesture in Gesture Representation is triggered and the desired trajectory of the gesture is sent to Gesture Planner. The Gesture Planner's task is to build motor action sequences using the primitive templates. A primitive template abstractly represents the dynamic properties of a motor primitive. The Gesture Planner allocates available body parts and determines specific parameters of primitives based on the dynamics and kinematics of primitives and the desired gestures. The obtained parameter values are sent to the low-level Primitives for execution. The primitives in Primitives module activated by Gesture Planner generated fundamental movements. The three basic operations on the primitives used by Gesture Planer are superposition, inhibition and sequencing. When more than one primitive are activated simultaneously by the Gesture Planner, one primitive may superposition or inhibit another one depending on their dynamic interactions.
Sensor Filter
5
One of the major considerations to use primitives on the robotic puppet is that we want to build a rich repertoire of representational behaviors through combining and sequencing primitives. Then the robot can carry out different tasks in its working space by selecting appropriate behaviors from its behavior collection. Hence, the primitives should be general and the set of the primitives should be small.
5.1
Gesture Representation
Gesture Planner Environment
Experiments
Primitives
Though fog's primitives can be found on its spinal through anatomical methods, no standard ways have been explicitly setup for selecting the appropriate primitives for robots. As attempts to derive primitives automatically for humanoid robot control, Mataric [12] analyzed the movement data captured from human and Edsinger [5] collected data from a robot arm. The complex dynamics and kinematics of robotic puppet imposing constraints to motor control make the process of discriminating fundamental primitives from various behaviors difficult. Two kinds of primitives are studied based on the observations to human behaviors and the convenience of implementation on the robotic puppet:
Primitive Templates
Primitives
Motor Actions
Figure 4. The architecture for gesture production. On the motor control board (Fig. 3), there are eight PICmicro microcontrollers in use. Each of them is dedicated to the position and velocity control of one servomotor. They are connected with each other through I2C bus, which make the control board extensible. The average speed to transmit one byte between two microcontrollers is about 100 microseconds because of the delay caused by the interrupt processing program on each microcontroller. The motor control board is easily to be extended to include heterogeneous devices. Sensors and other peripherals compatible with I2C bus protocol can be easily integrated into the controller. This networked controller structure is suitable for the implementation of concurrent and distributed low-level behaviors. Motor control commands descended from the embedded PC through the serial port at an actual speed of 70 bytes per second are executed by the microcontrollers. A little different from the hierarchical control structure, the microcontrollers not only execute low-level motion
• •
The primitives generate repetitive motions. Repetitive motions like walk, construct a big portion of human behaviors. The primitives move a limb to a goal position. A large set of motions and gestures can be achieved as the result of continuously and coordinately using this kind of primitives on different limbs.
From these two kinds of primitives, four primitives are designed to represent fundamental movements. The motion caused by each primitive is associated with a subset of the total 30 DOFs of the robotic puppet. It simplifies the combination of primitives activating different body parts of the puppet figure. At this moment, the primitives designed to construct complex behaviors are: •
381
Walk: move the legs back and forth rhythmically.
•
with opening arms reveals a responsive attitude. In Figure 6, such a metaphorical gesture is made by the robotic puppet through superposition Walk primitive and Move-arm primitive. The processes of operating objects are often expressed with gestures. These gestures can also be composed with the primitives. In Figure 7, the puppet stretch out its arms to represent an acceptive action. Move-arm primitives are used on both arms to achieve this effect.
Swing-arm: moves the arms back and forth, this primitive is synchronized with Walk primitive on low-level. Move-arm: moves one arm to the goal position. Collision-avoid: avoid to collide with other wires. It is implemented with the force-field method [2].
•
• 5.2
Expressive behaviors
Some complex behaviors can be composed with these primitives. The major operations on the set of primitives are superposition, inhibition and sequencing. Following examples demonstrate that the robotic puppet can exhibit deictic, metaphoric and iconic gestures to convey information. In Figure 5, the robotic puppet takes a deictic gesture pointing to a direction with the Move-arm primitive. The Move-arm primitive is activated and it rotates the arm to point to a direction.
Figure 7. This gesture represents object operation.
Figure 5. The puppet takes a pointing gesture.
Figure 8. The puppet takes an iconic gesture. With these primitives, more gestures with different meanings can be represented and constructed. The puppet robot can use its collection of express behaviors in human-machine communication.
6 Conclusions and future work The goal of the work described in this paper is to design a robotic puppet that can use its behaviors to convey information. The importance of body gestures in humanmachine communication is discussed. Our puppet robot
Figure 6. The puppet shows a responsive gesture. Behaviors can reveal the thought and attitude of a person. For example, a gesture of approaching forward
382
Simulated Salamander,” Biological Cybernetics, Vol. 84 (5), pp 331-348, 2001. [10] Jaeger, H., Christaller, T., “Dual Dynamics: Designing Behavior Systems for Autonomous Robots,” In Proceedings of Int. Symposium on Artificial Life and Robotics, (Beppu, Japan), pp. 76-79, 1997. [11] McNeill, David, Language and Gesture, Cambridge University Press, 2000. [12] Mataric, M. J., “Sensory-Motor Primitives as a Basis for Imitation: Linking Perception to Action and Biology to Robotics,” Imitation in Animals and Artifacts, MIT Press, 2001. [13] Mataric, M. J., Zordan, V. B. and Williamson, M. M., “Making Complex Articulated Agent Dance, An Analysis of Control Methods Drawn from Robotics, Animation, and Biology.” Autonomous Agents and Multi-Agent Systems, 2(1), pp. 23-44, 1999. [14] Mataric, M. J., Williamson. M. et al, “BehaviorBased Primitives for Articulated Control,” From Animals to Animats 5, MIT Press, pp. 165-170, 1998. [15] Mussa-Ivaldi, F. A., Bizzi, E., “Motor Learning through the Combination of Primitives,” Phil. Trans. R. Soc. Lond. B 355, pp. 1755-1769, 2000. [16] Mussa-Ivaldi, F. A., “Modular Features of Motor Control and Learning,” Current Opinion in Neurobiology, vol. 9, pp. 713-717, 1999. [17] Yanai, N., Yamamoto, M. and Mohri. A., “Inverse Dynamics Analysis and Trajectory Generation of Incompletely Restrained Wire-Suspended Mechanisms,” In Proceedings of IEEE Int. Con. on Robotics & Automation, (Seoul, Korea), pp. 3489-3494, May, 2001.
is a 30 DOF incompletely restrained wire-suspended system. Taking inspirations from biological motor control systems, the behavior-based method provides a framework for designing motor controller for the robotic puppet. Based on the idea that the entire behavior repertoire is built on basic primitives, the motor controller including a small set of primitives is implemented to solve the complex control problem of the robotic puppet. The effectiveness of the motor controller is demonstrated with the expressive gestures generated by combining primitives. In current implementation, the stationary torso simplified the combination of primitives by dividing the articulated puppet figure into four subsets of the 30 DOFs. The influence of the movement of one limb on global dynamics can be ignored. Another problem is that the gestures of the robotic puppet are not precise because the number of motors is insufficient to exert control to all the 30 DOFs of the robotic puppet even considering the gravitational force that contributes constraints on at most nine DOFs. Some parts of the figure body fall down under gravitational force. The form and gestures of the body are critical to the effects of behavioral information representations. More motors will be mounted and the wires will be re-arranged based on precise dynamics analysis.
References [1] Allott, Robin, The Motor Theory of Language Origin, Sussex, England, 1989. [2] Arkin, R. C., Behavior-based Robotics, MIT Press, 1998. [3] Bizzi, E., Tresch, M. C. et al, “New Perspectives on Spinal Motor Systems,” Nature Reviews Neuroscience, vol. 1, pp. 101-108, 2000. [4] Cassell, J. et al, “Animated Conversation: RuleBased Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational Agents.” Computer Graphics Annual Conference Series, pp. 413-420. SIGGRAPH, 1994. [5] Edsinger, A., A Gestural Language for a Humanoid Robot, Master Thesis, Department of Computer and Electrical Engineering, MIT, 2001. [6] Gibbon, D., Mertins, I. and Moore R. K., Handbook of Multimodal and Spoken Dialogue Systems, Kluwer, 2000. [7] Giszter, S. F. et al, “A Neurobiological Perspective on Humanoid Robot Design,” IEEE Intelligent Systems, pp. 64-69, 2000. [8] Gorman, J., “The Cable Array Robot: Theory and Experiment,” IEEE Int. Conference on Robotics & Automation, (Seoul, Korea), pp. 2804-2810, 2001. [9] Ijspeert, A. J., “A Connectionist Central Pattern Generator for the Aquatic and Terrestrial Gaits of a
383
