Interaction Design with Commands Embedded in Actions for Human

2nd International Conference on Autonomous Robots and Agents December 13-15, 2004 Palmerston North, New Zealand

Interaction Design with Commands Embedded in Actions for Human-Robot Cooperative Task Kazuki KOBAYASHI The Graduate University for Advanced Studies 2-1-2 Hitotsubashi, Chiyoda Tokyo 101-8430, Japan [email protected]

Seiji YAMADA National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda Tokyo 101-8430, Japan [email protected]

Abstract This paper proposes a novel interaction model for human-robot cooperative task. The point of the interaction model is an introduction of CEA (Commands Embedded in Actions) by which a human work-load is reduced because it need less inputs and outputs of a user of a robot than that of using direct commanding methods in a conventional interaction model. We build a procedure of the design for the proposed interaction model, and apply the method of interaction design to a cooperative sweeping task by a human and a small mobile robot. We then conduct experiments to confirm that reduction of a human work-load on our interaction design in the sweeping task. Human cognitive loads are examined to evaluate human work-load while a human interacts with a robot, and loads between CEA and direct control of a robot are compared. The results of the experiments show that the CEA minimizes a human cognitive load in comparison with other direct commanding methods.

Keywords: interaction design, cooperative task, sweeping, cognitive load 1 Introduction There are robots spreading among people as a progression of its technologies. We can purchase pet robots like AIBO [1] or cleaning robots like Roomba, and interact with them in a home environment. We will see tour-guide robots [2] in a museum in the near future. Robots thus have transferred their scene from industrial environments to home environments. How a home robot interacts with people is one of most important issue to be accepted by people who want to share their time and spaces with robots. Various researches have been studied in a field of human-robot interaction. Most of the researches have dealt with methods of communication between a human and a robot such as gesture [3, 4, 5], speech [6, 7, 8], using control devices such as joysticks or computers [9, 10, 11], multiple methods [12] and others [13, 14, 15, 16]. These types of interaction focusing into a function of a robot are described in Fig.1 in which the methods of communication correspond to the arc of command, and tasks of a human correspond to the arc of action. This figure assumes that a robot works based on the cycle of sensing, interpretation, and execution. The arc of human’s command means that a human gives a robot information which a robot cannot sense or interpret, and the arc of human’s action means human’s tasks which a robot cannot execute it because of the difficulty of equipping necessary hardware for given tasks. For instance, a commercial cleaning robot such as Roomba

301

Figure 1: Conventional Interaction

Figure 2: Suggested Interaction can sweep out a room autonomously, however a human actually needs to help the robot by removing obstacles because of equipping no hardware for handling the obstacles. Two tasks are assigned to the human: to control the robot by remote controller and to remove the obstacles. The improvement of the arc of human’s command such as gesture recognition, speech recognition and so on

2nd International Conference on Autonomous Robots and Agents December 13-15, 2004 Palmerston North, New Zealand

are insufficient for an actual task of a human in terms of his/her work-load: the human have to achieve two tasks. We therefore focus on a human’s given tasks represented by the arc of human’s action and deal with this kind of work as a cooperative task between a human and a robot to reduce the load of a human. We then introduce a new interaction model of Fig.2 in which no direct interaction between a human and a robot are employed. A human can control a robot by executing his/her actions to environment. We call this CEA (Commands Embedded in Actions) by which output and input of a human can be reduced. A human does not need to execute direct commanding to a robot and understand a way for communication with it. The existence of a robot is practically transparent for a human, and it leads reduction of a human work-load. There are studies related our suggested interaction model for a human-robot cooperative task. Their cooperative task is to carry a long or big object by a human and a robot based on manipulator [17, 18, 19], and outdoor cooperative tasks by a human and a humanoid [20]. These studies are typical instances of CEA. The robots can work well for helping a human by sensing force of a shared object without direct commanding methods. However, interchanging force is only allowed on such cooperation. In contrast, interchanging various information including force is allowed by CEA between a human and a robot. Our aim is to build up a general framework for human-robot cooperation. Therefore, CEA is a novel approach in the research area of human-robot interaction. After this section, in Section 2, we describe the procedure of interaction design for a human-robot cooperative task. In Section 3, we apply our interaction design to a cooperative sweeping task between a human and a robot. A developing of a behavior-based robot is described in Section 4. Section 5 describes experiments to compare CEA using suggested interaction design with direct control using conventional interaction design such as voice or hand manipulation in terms of human’s cognitive load. The experimental results are indicated in Section 6. Finally, Section 7 concludes this paper.

2 Method of Interaction Design We propose the procedure of interaction design for a human-robot cooperative task illustrated in Fig.2 as follow: 1. Divide a given task into a human’s task and a robot’s task. (a) Determine a robot’s task as maximize robot’s autonomy within given cost of hardware design. (b) Determine a human’s task as assign the task which a robot cannot execute autonomously. (c) Determine cooperative behavior and information interchanging between a human and a robot.

302

2. Embed commands in actions. Commands including the information of a robot are communicated to a robot as a human executes his/her actions. Actions are determined as minimize the change in space and time from a typical human’s action for an assigned task. 3. Design functions of a robot. Add the functions to a robot to sense or interpret CEA. CEA can be designed through above procedure. Although details depend on a given task, the procedure can assist interaction design for reducing human workload. CEA has advantages described as follow: • No additional cognitive load in execution: Since CEA has minimum additional actions to typical human’s actions to achieve an assigned task, a human does not need additional cognitive load by direct communication with a robot and smoothly does cooperation with a robot by executing only typical actions. • No understanding a way for communication: Since direct communication is not necessary in CEA, a human does not need to understand communication protocol like gesture commands, special speech commands. CEA thus releases a human from learning protocol and training to communicate with a robot.

3

Design on Cooperative Sweeping

We deal with a human-robot cooperative sweeping by a human and a small mobile robot as a cooperative work. A goal of the cooperative task is to sweep out a desk including the region of under an object. In this section, we first describe an experimental environment and a specification of small robot. Then we apply the procedure of our interaction design to the cooperative sweeping.

3.1

Environment and robot

Fig.3 shows an experimental environment where a human and a robot work cooperatively. This environment simulates a place used by a human routinely such as a desktop. A desk swept out by a robot has a flat surface and a wall which encloses the region for keeping a robot not to fall. We use a small mobile robot KheperaII. The robot has eight infrared proximity and ambient light sensors with up to 100mm range, a processor Motorola 68331 (25MHz), 512 Kbytes RAM, 512 Kbytes Flash ROM, and two DC brushed servo motors with incremental encoders. The program written by C-language runs on the RAM.

2nd International Conference on Autonomous Robots and Agents December 13-15, 2004 Palmerston North, New Zealand

Layer 3 : Interaction 44 cm

(Cooperative Behavior)

Projected Image

Layer 2 : Sweeping (Autonomous Behavior)

Wall

Sensor Value

Layer 1 : Obstacle Avoidance

Object

33 cm

Action

(Autonomous Behavior)

Figure 5: Subsumption architecture KheperaII

1. A human picks up an object when a robot approaches an object. Then a robot comes into the region under the object picked up.

Hand

2. While a robot is in under an object, it keeps turning at the object’s edge.

Figure 3: Experimental environment

3. A robot goes out of the region when a human approaches an object to it.

4

Figure 4: Cooperative sweeping by CEA

3.2 Interaction Design The procedure of our interaction design is applied for cooperative sweeping as follow: 1. Divide a sweeping task into a human’s task and a robot’s task. (a) A robot’s task is to sweep out a desk autonomously with strategy of a random turn. The hardware resource of a robot is equal to that of a commercial sweeping robot. (b) A human’s task is to move an object because a robot cannot move an object by itself. (c) Cooperative behavior is that a robot sweeps out the region under an object when a human moves the object. 2. Embed commands in actions A command sent by a human makes a robot sweep out the region under an object. This command is made by human actions to achieve his/her task. Fig.4 shows CEA in which the human’s action has no changes in the trajectory from the typical one, and it has little additional time to keep picking up. A human does not need to move an object to another place. 3. Design functions of a robot. An extra infrared sensor which measures the distance in vertical direction is added on a robot’s general I/O turret to sense CEA. The detail of designed CEA in Fig.4 is described as follow:

303

Behavior Design of Mobile Robot

Our robot performs obstacle avoidance, going forward (when no obstacles are on its front) and turning for random direction. We use sweeping with selecting directions at random because it has lower cost than a calculated sweeping using a map. Many methods for region covering have been developed [21], and most of them need precise position of a robot. However, our robot cannot use reliable methods like dead reckoning. We hence consider that the random sweeping is adequate to our study because the most of consumer sweeping robots adopt this kind of method. A robot is implemented by behavior-based approach, and we adopt subsumption architecture [22] to manage following behaviors: (1) obstacle avoidance, (2) autonomous sweeping while no object is sensed, and (3) interactive sweeping while an object is above a robot. Fig.5 shows the robot’s behaviors into the three layers in subsumption architecture. Each layer asynchronously checks the applicability of behaviors and executes applicable ones. Higher layers suppress lower layer’s behaviors, and lower layers have more reactive behaviors. The behaviors of each layer consist of multiple actions. When the system obtains multiple outputs, it generally selects the highest layer’s action. Each layer has output frequency of action to control the robot smoothly. We set the frequency as obstacle avoidance: an action by 5msec, sweeping: 10msec, interaction: 5msec, obstacle avoidance and interaction occur most frequently.

5

Experiments

We conduct experiments to confirm that our interaction design reduces work-load of human. We examine human cognitive load to evaluate human work-load, and compare the load between CEA and direct control of a

2nd International Conference on Autonomous Robots and Agents December 13-15, 2004 Palmerston North, New Zealand

Box

KheperaII

Sweeping Area

EC (Embedded) PC

Projector

SC (Sound)

HC (Hand)

Figure 7: Three types of interactions EC : A robot performs sweeping by CEA. SC : A robot sweeps with receiving a sound command by hand clapping.

USB Camera

HC : A robot sweeps with receiving a command by hand as blocking the robot’s line. Figure 6: Experimental system robot. The experiments are performed on our experimental system described in the next subsection.

5.1 Experimental system Fig.6 shows the experimental system which can indicate a robot’s trajectory. This system consists of a sweeping area and a projection system. In experiments, a human interacts with a robot on the sweeping area in Fig.3 indicating a swept location by the projection system including a personal computer, a projector, and a USB camera. The projection system detects a robot’s location using a picture of two beams of infrared LEDs equipping on the robot. The robot’s location is calculated by image processing in the picture from the camera, and then an image indicating the trajectory is created with the location. This image is ultimately projected on the sweeping area. The projected image also includes small square cells. A cell is lit when a robot enters the cell in real time. These small cells therefore express the trajectory of a robot. The sweeping area having a width of 44 cm and a height of 33 cm is divided into 16 × 12 cells. Cells of 3 × 3 approximately correspond to the area of a robot.

5.2 Cognitive load measurement We measure the cognitive load of a human interacting with a robot by CEA, and compare the load to that of direct control. Two direct control methods are chosen as typical one without extra devices such as remote controller and a human can create a command by moving his/her body. These control methods are shown in Fig.7, and the detail is described as follow:

304

The robots receiving such commands are prepared by adding extra sensors such as microphone or modifying the program of a robot. When a robot controlled by direct command receives a command from a human, it turns from 90◦ to 180◦ randomly. A robot controlled by CEA also turns from 90◦ to 180◦ randomly when it runs out from under an object. Measurement starts when a robot enters the region of under an object, and it continues until all cells of the region are swept. A box whose size is 4 × 4 cells is employed because sweeping time of the region is appropriate for subjects and the measurement. In the EC, subjects keep to pick up an object until all the cells of under the box are swept. In the SC and HC, first, a human relocates a box to a corner of the sweeping area, and then send a command for a robot to be turn by making sounds or approaching their hand to it when it is likely to run out from the region of a box. We use a dual task method to measure human cognitive load. Subjects have to do mental arithmetic as a secondary task while controlling the robot as a primary task [9, 23]. They count backwards by three from a randomly selected three-digit number vocally. We obtain the average number of correct answers per second, and evaluate the human’s cognitive load for controlling each robot. Subjects are required to calculate as quickly as possible, and to prioritize the controlling a robot over the counting. They practice controlling robots and the counting well before experiments begin. The order of EC, SC and HC for each subject is determined at random, and these experiments are recorded three times respectively for a subject. Subjects are also measured counting ability without operations of a robot before a measuring of EC, SC, or HC respectively. The counting ability is the number of correct answers of the counting for 30 seconds. Fig.8 shows the experimental appearance.

2nd International Conference on Autonomous Robots and Agents December 13-15, 2004 Palmerston North, New Zealand

Figure 8: Experimental Appearance

Therefore, CEA plays the significant role in a humanrobot cooperative task.

Figure 9: Results of scores and differences

5.3 Results Subjects are eight men and four women between the age of 22 and 32. Each subject has three scores: EC, SC, and HC. A score is the average of normalized number of correct counting answers per second for a subject. The normalization is to divide the correct answers per second by correct answers per second without operations of a robot. Fig.9 shows averaged scores, standard deviations, and differences tested by Dunnett’s test. Each EC, SC and HC is the average of all subject’s scores. The number 1.0 means counting ability of each subject without operations of a robot. EC has the highest average. The difference between EC and HC has a significant difference (p