A Theoretical Approach to Human-Robot Interaction Based on the Bipolar Man Framework Francesco Amigoni, Viola Schiaffonati, Marco Somalvico Dipartimento di Elettronica e Informazione Politecnico di Milano Piazza Leonardo da Vinci, 32 20133 Milano, Italy E-mail:
[email protected],
[email protected],
[email protected] Abstract The consideration of a general theoretical scenario accounting for the relationships between humans and robots may enhance the design and the development of human-robot interfaces. By starting from peculiar examples, the aim of this paper is to present an abstract framework that accounts for the current tendencies within the field of human-robots interaction. This framework is based on a precise “philosophical” position expressed by the concept of bipolar man, which has an impact on both the analysis and the design of human-robot interfaces.
1
Introduction
The theoretical study of the human-robot interaction offers nowadays a fundamental background to set a general scenario that may enhance the design and the development of human-robot interfaces. The aim of this paper is to present an abstract framework based on the concept of bipolar man [1] able both to potentially describe the current human-robot interaction on the basis of some very general ideas and to evidence the fundamental role of man, even in presence of complex modern and sophisticated robots. According to this perspective, a man is considered as a unique subject who can perform his intellectual and interactive activities in two different poles: the man-body pole and the man-machine pole. In the first case, the man directly carries on his activities by his body; in the second case, the man indirectly carries on his activities by robots. Moreover, this framework allows for a natural extension toward the social and pluralistic dimension of the human-robot interaction, namely it can be widened to encompass the idea of bipolar society composed of different bipolar men. Although our proposal is based on the traditional classification of human-robot interaction, this paper adopts a more abstract and general point of view with respect to other published papers. Moreover, our contribution accommodates an epistemological perspective on the human-robot interaction. For example, it pictures an interesting scenario that is placed on a higher abstraction level than the detailed taxonomy of cooperation forms between humans and manipulators presented in [5].
We show how the bipolar man framework is able to unify in a unique general setting three different tendencies we deem are important in modern human-robot interaction: • the enlargement of the communication flow between humans and robots, that increasingly uses nonverbal communication, as required by the ubiquitous computing and active artifacts applications [14]; • the use of robots as fundamental parts of extremely powerful interfaces between humans and machines, as in the case of the inverse robot employed in virtual reality [9] and telemanipulation applications; • the extension of the human-robot interaction to the social dimension, as needed by the robots devoted to the care of the elderly [11]. The adoption of the bipolar man framework has an impact on both the analysis and the design of humanrobot interfaces. On the one hand, it helps in understanding and classifying the existing systems, their properties, and the relations among them. On the other hand, it stimulates profitable ideas for the conception of novel and improved systems. The paper is structured as follows. In Section 2, we analyze, by means of examples, some important and characterizing tendencies in modern human-robot interaction. Section 3 presents the bipolar man framework, its extension to sociality, its ability to appropriately capture the issues of the identified tendencies, and its role in the analysis and in the design of human-robot interfaces. Finally, Section 4 concludes the paper.
2
Examples of Human-Robot Interaction
In this section, we report three significant examples of the current human-robot interaction. The first two examples are aimed to evidence the expansion of the communication channel between human and machines, both when the robots are the machines toward which the humans direct their communication (Subsection 2.1) and when they are the fundamental part of the interface between humans and machines (Subsection 2.2). The third one is devoted to enlighten the new role of sociality in the human-robot interaction (Subsection 2.3). All these features will be summarized in Section 3 within the theoretical framework we will present there.
2.1 The active artifacts This first is centred around the enlargement of the communication channel between humans and robots by new and unconventional communicative acts. In many recent research projects, among which the best known is probably the “Things that Think” project at MIT [14], the goal is embedding intelligence in simple everyday objects, like doors and chairs. These objects, also called active artifacts, are usually small (disappearing) robots that can operate in the environment under the easy control and supervision of humans. To facilitate the interaction between the humans and these robotic artifacts, new kinds of communication have been developed and employed. These include nonverbal communication, bodily communication, tangible interfaces, and embodiment communication. Some of these new kinds of communication can be clearly observed in the active artifact presented in [13], which is an autonomous chair that can move toward a user who manifests the intention of sitting down. The intention is expressed by moving an hand (with a wagging of the fingers similar to that of “come here”). The hand movements are perceived by a sensor (e.g., by a camera) and the chair drives autonomously to the point where the user called it. This is a paradigmatic example of nonverbal communication between humans and robots that is based on a functional relation centred around the concept of affordance, according to which artifacts acquire properties only when humans make some uses of them [7]. In this example, the robot (the autonomous chair just described) cooperates with the human who wants, for example, to sit down. The enlargement of the communication flows (e.g., including nonverbal communication) facilitates and makes more natural the use of such active artifacts. We deem that this tendency is reinforcing as the ubiquitous computing and the ambient intelligence are spreading more and more. 2.2 The inverse robot for virtual reality applications The enlargement of the information flow between humans and robots discussed in the previous subsection has an impact also on those applications that require an evolution of the human-machine interaction from a contemplative character to a participative one. In the contemplative interaction, the man communicates to the machine his desires and his intentions by highly structured commands and receives some information back from the machine. With the passage to the more immersive participative interaction, the man communicates to the machine also by means of natural language, of borderline acts, and of gestures; in a similar way the machine uses high-level communication acts (such as facial expressions) to influence the human being. The participative interaction is usually required in virtual reality settings when a computer simulates a scene in which a man is acting in an environment (possibly including other men). In this schema, the man is both a part of the simulation environment (as a model reproducing him) and an observer (as a real person) of the simu-
lated happening. In order to simulate a real man, the computer must contain a model of the man. Hence, the information that the real man conveys to the computer does not concern only the representation of his desires, but also the representation of the model of himself involved in the simulated scene (e.g., the information about his posture). Conversely, the information that the computer conveys to the real man involves the model of the simulated man acting in the simulated scene (e.g., the information about the orientation of the simulated head). It is thus clear that the interface between man and computer plays a fundamental role. This interface is an inverse robot, namely a robot with its sensors and actuators (see Fig. 1) that represents an example of robot-based interface. In this case, the interaction between humans and robots is a part of a more general interaction between humans and machines.
Figure 1. The role of inverse robot in virtual reality. The inverse robot interface receives from the man the information about what he wants and conveys it to the simulation machine. Moreover, the inverse robot conveys to the man the model of what his simulated model experiences in the simulated setting. In this framework, both man and simulation machine play the role of direct subjects (in particular, we will denote simulation machine as direct machine). As illustrated in Fig. 1, the inverse robot has actuators corresponding to sensors of the real man and, conversely, has sensors corresponding to actuators of the real man. The natural sensors of man receive their input from the artificial actuators of the inverse robot. For example, two LCD displays (artificial actuators) are placed on a pair of goggles in front of the man’s eyes (natural sensors) to form a socalled head-mounted display. Similarly, the natural actuators of man provide their output to the artificial sensors of the inverse robot. For example, a glove (artificial sensor) is placed on a man’s hand (natural actuator) to perceive the flexion of fingers. On the other hand, the artificial actuators of the inverse robot are connected to the virtual sensors of the man’s model within the direct machine (e.g., the LCD displays show what the virtual eyes of the simulated man see). In addition, the artificial sensors of the inverse robot are connected to the virtual representation of the scene within the direct machine (e.g., the movements perceived by the glove influence the simulated scene by, for example, moving objects). The role of inverse robot can thus be intended as devoted to set up a correspondence between the natural sensors and actuators of man and the virtual sensors and actua-
tors of the man’s model within the direct machine. Moreover, the inverse robot provides a clear conceptual separation between the direct machine, where the simulation takes place, and the inverse robot, which is the interface between the user and the direct machine. (For more details about this conception of inverse robot, see [2].) Besides virtual reality, the same considerations about the role of inverse robot apply also to the traditional field of telemanipulation systems (for example, to the field of surgical robots used in medical applications). 2.3 The elderly-care robots Service robotics represents nowadays an increasing area of research, which has significant scientific, economic, and social impacts. It includes various robotic systems presenting difficult technical challenges in their development due mainly to the unstructured environment in which they operate [12]. Among the several researches, health care for assistance to elderly and disabled people is one of the most promising for the positive repercussions in the everyday life. In particular, as a consequence of the extension of longevity, a strong effort has been put in technologies that increase independence and quality of life among older people. The “Nursebot Project” is a very significant research, conceived in 1998 and involving a multidisciplinary team from three different universities [10]. It has the aim to develop mobile robots for assisting the elderly; in particular, one of such robots has been implemented and tested in a retirement community. The robot has basically two functions: to remind people about routine activities and to guide them through the environments, which are strong challenges as regarding to the human-robot interaction. By dealing in particular with the elderly, the robot must exhibit, from the one side, a very friendly user interface and, on the other side, a high degree of autonomy. Moreover, the same robot has to adapt to be used by different people with different goals, thus a sociality aspect has to be considered in the human-robot interaction. In order to fulfill these requirements, from the hardware perspective, particular attention has been reserved to the design of the robot, which is equipped with a head unit that gives a sensation of friendliness and safeness. From the software perspective, the robot features a telepresence interface, a speech interface (to recognize and synthesize voice), a system to detect and track faces, and a navigation system. The tele-presence interface is particularly important in allowing nurses to monitor and interact with the user: therefore, instead of replacing a nurse, the robot facilitates communication between the patient and the nurses or doctors. It is worth noting that this example enlightens not only the importance of sociality but also that of communication, since the robot acts as an interface for the communication between humans. In such a way, the robot increases also the user’s contact with the outside, giving to her or him the idea of a minor isolation. In addition, the tele-presence offers the control of the robot also to a remote user, such as a relative or a friend that can drive the robot around the user’s room. Also the
speech interface and the face detection functions allow for a more natural interaction. In the first case, the system is controlled in real-time by a speech dialog manager device able to generate the appropriate response and to avoid other forms of communication (like keyboards and computer screens) particularly unfriendly for the elderly people. In the second case, the face detection enables to direct the robot’s sensors toward the user’s face, who is particularly important when he or she is an elderly person with cognitive and speech impairments. Finally, the navigation system is specifically designed for interacting with people, exploiting functions such as the adaptive velocity to the user and the ability of an accurate localization. It appears clearly that a new tendency in humanrobot interaction is related to robots able to interact with several humans at the same time, such as in this case where the robot interacts with different elderly people, nurses, doctors, and relatives.
3 The Bipolar Man Framework In this section, we present the bipolar man framework (Subsection 3.1) and it extension to sociality (Subsection 3.2). Then we discuss the possible contributions of our approach to human-robot interaction (Subsection 3.3). 3.1 The bipolar man The three examples described in the previous section show some modern and significant tendencies in humanrobot interaction. In order to summarize in a unitary scenario these different issues, we propose to adopt a theoretical framework for the anthropological status of a single man (and, in the following, of a group of men) when interacting with robots. This framework is based on the concept of bipolar man [1]. The bipolar man approach is applicable to a broader scenario and shows the central role of man by describing in general the interaction between man and information machines (of which computers and robots are paradigmatic examples). According to the bipolar man framework, a man is a unique subject - called man-mind subject to emphasize the unity and homogeneity of his nature - who can perform his intellectual and interactive activities in two different poles: the man-body pole and the man-machine pole (see Fig. 2), which are associated with different modalities of performance. In the first case, the man-mind subject directly carries on his intellectual and interactive activities within his natural body; in the second case, the man-mind subject indirectly carries on some of his intellectual and interactive activities within artificial information machines. It is worth noting that this imply that the information machines are not autonomous in carrying on the activities traditionally performed by humans. There is just a delegation of some of these activities to sophisticated artifacts. In this paper, we focus on the case in which the man-machine pole is a robot. Within this framework, it is interesting to evaluate which intellectual and interactive activities can be per-
formed by the man-machine pole, namely by the robot. We claim (according to a well-established epistemological mainstream) that the activities performed within the man-machine pole are those that can be modeled, namely those that can be described in rational terms and that belong to the “fabricative intelligence”. These activities can be expressed by algorithms and, as a consequence, can be performed by a robot (see [1] for further details). In general, interactive (involving perception and action) and processing activities can be delegated to robots. Examples include the grasping of objects and the navigation between two points in an environment. From another point of view, but in accordance with our approach, the robotic man-machine pole can be conceived as an extension of the human sensorial and motor capabilities. We explicitly note that the man-body pole and the manmachine pole do not simply denote the human and the robot, respectively, but stress the central role of man who delegates some of his activities to the robot that, in this epistemological perspective, is not regarded as an autonomous subject.
Figure 2. The man-mind subject (dashed line) composed of man-body pole (left) and man-machine pole (right). Within a man-mind subject, the main interaction is the intrabipolar interaction between the man-body and the man-machine poles. This interaction is evidenced by the arrows of Fig. 2 (Fig. 1 is a particular case of this very general schema). The features of the intrabipolar interaction can be better understood at the light of the evolution of man-machine communication. As already outlined in Subsection 2.2, in the past this communication required a pre-processing of the input information performed by man in order to prepare it in a form suitable to be communicated to the machine (the robot, in our case). Thus, the form of the information to be communicated to the machine was very different from the form of information used to communicate among humans. The advances of intelligent and adaptive interfaces [3] [4] [8] promote an increasingly graceful communication environment: the flow of messages from the manbody pole to the man-machine pole can be expressed more easily. For example, the communication toward the robot can be more naturally and handily performed if natural language is used, maybe enriched by some borderline (often unconsciously) acts, such as prosody, mimicry, gestures, and body language. Although these acts are not formally captured in any natural language structure, they are very important in revealing the real cogni-
tive intentions of the person using them. In order to cover these requirements, a non-invasive and natural communication to the robot must be composed of two parts: the conscious representation of the human intentions in a multimedia-based natural language and the unconscious representation of intentions through non orthodox communication acts (see [6] for these perceptual user interfaces). In general, the intrabipolar communication can vary from simple signal sending (like start/stop) to the complex communication just envisaged. The choice strongly depends on the tasks the robot is asked to carry on. The first two examples given in the previous section represent significant instances of modern and sophisticated intrabipolar interaction where, to allow a better interaction among humans and robots, the communication channel between the two poles is significantly improved. The bipolar man framework unifies the issues presented in the Subsections 2.1 and 2.2. In the first case, the intrabipolar communication is empowered by the adoption of nonverbal communication (Fig. 3). This means that more natural actuators (e.g., hands) of the man-body pole are employed in the communication with the man-machine pole that, in turn, exploits increasingly complex artificial sensors. In the second case, the (inverse) robot acts as an improved interface between humans and machines (Fig. 4). In Fig. 4, the man-machine pole is composed of two parts: the inverse robot interface and the simulation machine. This situation evidences that the man-machine pole is not static but can be augmented (e.g., by the simulation machine of Fig. 4) according to the pragmatic exigencies of the man-mind subject, which is an aspect deserving more attention.
Figure 3. The enlargement of the communication flow between human and robot discussed in Subsection 2.1.
Figure 4. The inverse robot acting as the interface discussed in Subsection 2.2. 3.2 The bipolar society The anthropological scenario offered by the bipolar man concept can be enriched in an interesting and realistic way by encompassing the social dimension of human activities. Several human activities are carried on by
groups of people who interact together and form different kinds of societies. According to our view, a society (conventionally intended as a group of interacting people) can be seen as composed of several interacting bipolar men and can be called bipolar society (see Fig. 5). We have before identified a man-body pole and a man-machine pole as the two different poles of a single man. In a similar way, we can now identify a society-body pole and a society-machine pole (the dot and dash boxes of Fig. 5) as the two different poles of a bipolar society, when such society is thought as a single subject composed of bipolar men who perform their activities both within their bodies and within information machines. As before, we will consider the case in which robots are the man-machine poles and, consequently, the case in which the societymachine pole is composed of a set of (interacting) robots.
Figure 5. The bipolar society (the big dashed box) composed of society-body pole (the dot and dash box on the left) and society-machine pole (the dot and dash box on the right). When we consider a bipolar society, we put the attention on the interactions that occur among the different man-mind subjects and that represent the interbipolar interaction. Similarly to the intrabipolar interaction, the interbipolar interaction has received a lot of attention in the last few years, but many issues are still unexplored. When we consider the interaction between two manmind subjects, we can have three different situations according to the nature of poles that interact: two man-body poles interacting, a man-body pole and a man-machine pole interacting, and two man-machine poles interacting. The first situation characterizes the society-body pole as a unified entity and, being typical of human interactions, will be not further discussed. The second situation is similar to the intrabipolar interaction (within a single man-mind subject) previously presented. The third situation, when the man-machine poles of different subjects interact together, is in our opinion the most significant and innovative one since it characterizes the societymachine pole as an integrated entity. The interaction among the man-machine poles (i.e., the robots) of differ-
ent subjects can be profitably cast in the field of multirobot systems (that can be seen as particular multiagent systems [15]). In this case, a number of robots interact, coordinate, cooperate, and compete. For example, many efforts have been devoted to develop robots that efficiently perform cooperative work, such as mapping an environment or pushing a box. The social aspect embedded in the bipolar society is particularly emphasized in the case of the elderly-care robots of Subsection 2.3 which, having to interact with several people, represent an original example of a society-machine pole in which different people share the same robot. In fact, the patients, the nurses, the doctors, and the relatives are different man-mind subjects that communicate and interact together through the mediation of their man-machine poles, in particular by delegating some of their communicative activities to a single shared robot. This situation is depicted in Fig. 6, where it is evidenced the fact that the different subjects share the same robot.
Figure 6. The social interaction among patients, nurses, doctors, and relatives in the elderly-care robotic application discussed in Subsection 2.3. 3.3 The importance of the bipolar man framework In this subsection, we outline the contributions that the bipolar man framework can give to the analysis and the design of human-robot interfaces. Three examples of the improvements to the analysis provided by the proposed framework have been already discussed in the examination of the three systems of Section 2. It turns out that by adopting the bipolar man framework we are able to classify different cases of human-robot interaction both when a single human and a single robot are involved and when more humans and more robots are involved. The precise and clear distinction between humans and robots and the taxonomy of all the possible interactions between them are the main contributions of the bipolar man framework to the analysis of human-robot interaction. Moreover, our approach states, from a conceptual and epistemological point of view, the boundaries for the actions of robots (see [1]).
The contributions our framework can provide to the design of human-robot interfaces are still under investigation. However, even in this preliminary phase, some ideas can be assessed. For example, it is possible to conceive scenarios that extend those presented in Figs. 4 and 6 by allowing more users to interact with a single machine by means of different inverse robots (see Fig. 7). Such a system could be useful in situations when a pool of surgeons, each one interacting with an inverse robot, cooperatively work on the same patient in particularly complex operations conducted by advanced telemanipulation systems or in entertainment systems when different players are acting in the same virtual world. As this example suggests, the contribution of the bipolar man framework to the design of interfaces for human-robot interaction lies mainly in its power of envisaging interesting scenarios and situations that can be taken as architectural bases for the development of new systems.
Figure 7. The social interaction that can be useful for cooperative surgery and entertainment.
4
Conclusions
In this paper we have illustrated a general and abstract framework in which the analysis and the design of humanrobot interaction can be proficiently settled. The framework is based on the idea of bipolar man, which considers the modern man as a unique subject who performs his intellectual and interactive activities in two different poles: in his natural body and in artificial information machines, like robots. We have presented three significant cases that summarize some of the modern tendencies in human-robot interaction to assess the appropriateness of our approach. It has resulted that the bipolar man framework can account for the enlargement of the communication channels between humans and robot, for the use of robots as powerful interfaces between humans and machines, and for the social interaction that is required by some modern robotic applications. The future work will be devoted to further validate and detail our approach in other cases in order to evaluate its role as a solid and effective tool for understanding and developing human-robot interfaces. In particular, we
are studying the requirements that the emerging nanorobotic technologies pose in term of interfaces for humanrobot interaction.
References [1] F. Amigoni, V. Schiaffonati, M. Somalvico “Processing and Interaction in Robotics”, Sensors and Actuators A: Physical, vol. 72(1), 1999, pp. 16-26. [2] F. Amigoni, V. Schiaffonati, M. Somalvico “The Bipolar Man Framework for Human-Centred Intelligent Systems”, to appear in the Proceedings of the Sixth International Conference on Knowledge-Based Intelligent Information and Engineering Systems (KES2002), Crema, Italy, September 16-18 2002. [3] M. Bordegoni, G. Faconti, S. Feiner, M. Maybury, T. Rist, S. Ruggieri, P. Trashanias, M. Wilson “A Standard Reference Model for Intelligent Multimedia Presentation System”, Computer Standards and Interfaces, vol. 18(6,7), 1998, pp. 477-496. [4] P. Brusilovsky “Adaptive Hypermedia”, User Modeling and User-Adapted Interaction, vol. 11, 2001, pp. 87-110. [5] C. Burghart, S. Yigit, O. Kerpa, D. Osswald, H. Woern “Concept for Human Robot Co-operation Integrating Artificial Haptic Perception”, in M. Gini et al. (editors), Intelligent Autonomous Systems 7, IOS Press, 2002. pp. 38-45. [6] “Communications of the ACM, special issue on The Intuitive Beauty of Computer-Human Interaction”, vol. 43(3), 2000. [7] J. Gibson “The Ecological Approach to Visual Perception”, Houghton Mifflin Company, 1979. [8] M. Maybury (editor) “Intelligent Multimedia Interfaces”, AAAI Press, 1993. [9] L. Perry, “An Investigation of Current Virtual Reality Interfaces”, ACM Crossroads, vol. 3(3), 1997. [10] M. Pollack et al. “Pearl: Mobile Robotic Assistant for the Elderly”, to appear in AAAI Workshop on Automation as Eldercare, August 2002. [11] N. Roy et al. “Towards Personal Service Robots for the Elderly”, in Proceedings of the Workshop on Interactive Robotics and Entertainment (WIRE), Pittsburgh, PA, 2000. [12] R. Schraft “Mechatronics and Robotics for Service Applications”, IEEE Robotics and Automation Magazine, vol. 1(4), 1994, pp. 31-35. [13] K. Terdada, T. Nishida “Active Artifacts: for New Embodiment Relation between Human and Artifacts”, in M. Gini et al. (editors), Intelligent Autonomous Systems 7, IOS Press, 2002. pp. 333340. [14] “Things that Think”, http://www.media.mit.edu/ttt/, last access June 20 2002. [15] G. Weiss “Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence”, MIT Press, 1999.
