Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems October 9 - 15, 2006, Beijing, China
Basic Study on Sensory Aspects of a Master/Slave System for Force Telecommunication* Masayuki Hara
Young-Mi Jung
Graduate School of Engineering Yokohama National University Yokohama 240-8501, Japan
Graduate School of Engineering Yokohama National University Yokohama 240-8501, Japan
[email protected]
[email protected]
Jian Huang
Tetsuro Yabuta
Department of Mechanical Engineering Yokohama National University Yokohama 240-8501, Japan
Department of Mechanical Engineering Yokohama National University Yokohama 240-8501, Japan
[email protected]
[email protected]
Abstract – The advancement of multimedia services including audio/visual media has been rapid. The applications of cellular phones, in particular, have increased by using these technologies. In addition, if force telecommunication were realized, force information would become a next leading factor in the multimedia service. In this field, a robotic master/slave system is a leading candidate. This system has been studied as a teleoperation system between an operator and an environment; the operator controls only the master device whereas the slave device is never controlled. For multimedia applications, such systems require flexibility and bidirectionality because force telecommunication must consider the interactions between users. This paper details the fundamental issues of force communication from a viewpoint of human perception. With regard to the control system, this study uses an integrated hybrid control master/slave system that helps to find a key factor of equal bilateral force telecommunication in the sensory aspect. The employed system has a function to change the proportion of force display to position display by varying a weight parameter. We discuss the basic control performance of a force telecommunication system and evaluate qualitative feelings of users by using a psychological evaluation method. Index Terms – Master/Slave System, Force Sense, Force Telecommunication, Parallel Control
have seen the proliferation of cellular phones with new models being developed every year. These phones are used by various generations worldwide. While, with regard to visual communication, the emergence of the television brought a revolutionary phase to the field of entertainment. Currently, audio and visual communication technologies are applied to Internet applications such as the Internet telephone and digital distribution of music, movies and images. As evident from the above examples, communication technologies clearly support and benefit our modern way of life; their lack would have surely hampered the conveniences of modern society. Force display and its transmission are the focal technologies in the next-generation multimedia services. Recently, a haptic interface system that can provide a force/tactile sensation has gained increasing attention [1] [2]. The PHANToM series, which adopts this technology, has already been commercialized. If a force/kinesthetic sensation can be precisely interchanged among users with force display devices through the Internet, three-dimensional physical services with the sense of weight, touch and deformation in the physical world can be made available to users. Also, it is expected that these devices will enable the sharing of these
I. INTRODUCTION In the information society, it is widely considered that the development of communication technologies is a crucial factor enabling mutual communication with people at remote locations. Ever since human beings have employed language as a communication tool, they have developed and enhanced various devices to facilitate mutual communication. The most typical communication device is a telephone. A. G. Bell constructed a prototype of the telephone in 1876, which used a cable-based device. This device later adopted a wireless technology, resulting in today’s cellular phone. Recent times *
Fig. 1 Example of force telecommunication service
This work is partially supported by a Grant-in-Aid for Scientific Research (B2 No.16300066) and a Grant-in-Aid for JSPS Fellows (No.1700543800) of Japan Society the Promotion of Science.
1-4244-0259-X/06/$20.00 ©2006 IEEE
1706
sensations among users. For example, it would become possible to virtually play table tennis with a person at a remote location, as shown in Fig. 1. Hence, force display technologies have the potential to open a new avenue to the application field of information technology. Among the current systems, the master/slave system is considered to be a leading candidate for force telecommunication. However, recent studies on these systems have focused on teleoperation, which assumes the exchange of force information between an operator and an environment. These studies usually discuss the stability of system and compensation for communication delay [3]–[7]. Few studies have focused on force telecommunication between users in the sensory aspect [8]. In this study, we examine a master/slave system for force communication between users. The objective of this study is to clarify technical issues and evaluate the performance when using the master/slave system as man-to-man communication tool; this study, then, used an integrated system with both position and force servomechanisms based on parallel control. II. FUNDAMENTAL ISSUES OF FORCE COMMUNICATION A. Geometrical Issue The allocation of direction in which the force acts is a significant factor in force communication. In the conventional system, it is hoped that the slave robot behaves in the same manner as the master robot. Then, both coordinate systems are generally defined as the same coordinate system based on each robot base in order to unify their attitudes. However, the following peculiar cases are assumed due to target tasks when employing the master/slave system as a force communication device. As shown in Fig. 2 (a), when the users operate the device in mutually opposite direction, the force sensations flip even though the attitude information of robot is correctly interchanged between the devices. That is, the device pulls the receiver forward even though the sender pushes the device. Ideally, the force communication system has to realize the situation that the device pushes the receiver. Then, the
receiver should feel “pushed” sensation by the device. In this case, the position servo-loop works stronger than the force servo-loop. Such an inverted situation should be avoided for interchanging both the position and force sensations. However, this characteristic is effective when employing the master/slave system for teaching or rehabilitation. In this case, the instructor, who sends the force sensations, generally operates the device in order to lead or support the follower. Vice versa, the follower, a student or a patient, behaves passively according to the instructions because the follower cannot act voluntarily until the instructions are received. That is, the above geometrical issue can be ignored when the users face in the same direction and perform the task. It should be noted that the received force sensations are dependent on the target task and the allocation of users. A mirror image, then, appears in the force sensations between the master and slave devices; this issue is unavoidable due to the target task. In addition, the human cognition is related to this issue. It is thought that users receive entirely different sensation from the device by whether they consider the device as an avatar of communication partner or a mere mechanical interface. So, in the multimedia applications, the coordinate systems of both devices should be defined individually to match the target task considering the complicated problem on human cognitive level. B. Competition between Operators In general, a typical master/slave system has only one operator interacting with a mostly passive environment. The operator controls only the master device and manipulates the remote object by using the slave device, which behaves in the same manner as the master device acts. In this case, as shown in Fig. 3 (a), each role cannot be changed because the slave device passively functions in accordance with the master device. However, this master/slave relationship may collapse when the environment is replaced by another operator. In this case, each user would play the role of the master and attempts to operate their own device independent of the other’s actions Operator
Inverted Force Information
Remote Control Push
Pull
Passive
Active
Object Active Operator
Passive Operator
Master Robot Slave Robot (a) Operator to object (Conventional case)
(a) Task 1: Force telecommunication
Operator ll Pu sh Pu
Instructor
Operator Conflict
Teaching
Follower
Master Robot Master Robot (b) Operator to operator Fig. 3 The master/slave relationship due to communication object
(b) Task 2: Teaching and rehabilitation Fig. 2 Difference in the transfer direction of a force sensation due to task
1707
as shown in Fig. 3 (b). Therefore, the force/motion conflict occurs when the users simultaneously manipulate the devices in opposite direction. Although this situation is natural with regard to human behavior, conventional applications do not assume such a situation because the slave device never moves voluntarily. Thus, it is necessary to consider the master/master system and introduce flexible factor for human competition in the force communication applications. C. Technical Issues Concerning Servomechanisms A robot control system has two major servomechanisms, which are the position and force servomechanisms. In the conventional master/slave system, the position servo system is widely used while the force servo system is only applied in the force reflection type, where it is doubtful that bilateral performance is guaranteed completely. The main purpose of the conventional system is to apply remote control systems such as a surgical assist robot and an ocean/space probe robot. Hence, the position feedback control is more dominant factor than the force feedback control because the precise position control of the slave robot is crucially required in above applications, especially, in the field of microsurgery. However, with regard to force communication, it is considered that the force feedback control plays a more significant role as well as the position feedback control. This is the reason why the force sensation, which is provided by another user, must be transmitted with precision and same magnitude sensorily. If different forces are displayed between users, sharing of the force sensations becomes impossible. Thus, the force servomechanism also plays the key role in sharing the force information between the devices, although it is not big deal in the conventional master/slave systems. The servo system has another essential issue. In general, it is supposed that human beings immediately generate the appropriate motion (position) and force when contacting with the environment. While interacting with the environment, human beings can move with an object and adjust the force to get balance with the reaction force from the object. It implies that human beings can simultaneously control the position and Position Feedback
Position Force
+
-
Position
+
-
Muscle Force
Object
Force Feedback
the force, i.e., we have both the position and force servomechanisms that can work together as demonstrated in Fig. 4 (a). However, the robot cannot control both factors in parallel because they can basically perform only one function; the control system has to uniquely select the position or force servomechanism as shown in Fig. 4 (b). So, it is skeptical whether the conventional master/slave system, which is equipped with only the position servomechanism, really enables exchanging and sharing equivalent force/kinesthetic sensations between users. Therefore, the master/slave system for force communication should be constructed with both the position and force servomechanisms, i.e., a position/force hybrid control system. III. EXPERIMENTAL SYSTEM A. Software System 1) Parallel Control Type: To date, many researchers have expanded the frameworks of the master/slave system [5]. The most basic systems are the symmetric, force reflection and parallel control types. Among these systems, the parallel control type is considered most suitable for bilateral force communication because force communication requires the symmetric structure in the force/position display. Only parallel control type has a possibility that satisfies with this requirement by adding an indirect force feedback loop as shown in Fig. 5. In the figures, X and F denote the position and the force, respectively; u is the input to the robot. Subscripts M and S mean the master and the slave robot, respectively. Therefore, this study focuses on the parallel control master/slave system. However, it is doubtful whether this system precisely transmits the force sensations to another user because of fundamentally employing only position servomechanism. In this type, the force feedback loop works indirectly by using the position information converted from the force information through the position generator. Thus, this study integrated the position and force servomechanisms into one control system by introducing the weight factor. The schematic block diagram of the hybrid system is shown in Fig. 6, where w ( 0 ≤ w ≤ 1 ) is the weight parameter that decides the magnitude of the position and force feedback loops. This control system becomes an equal hybrid position/force control system with no bias for the position and force feedback loops when w is set to 0.5. That is, it functions as a complete position or force servo system when w is set to 0 or 1, respectively. This study attempted to clarify the servo effects +
(a) Human servomechanism Position Feedback Position Position
+
Position Servo Controller
Robot
FM
Separated Force Force
+ -
Controller
uM Master
XM
Robot
Master Controller
Position Servo
Position Generator
+ + -
Force Feedback
-
+ +
Slave Controller
FS
uS Slave
+
+ Fig. 5 Parallel control type with the position servomechanism
(b) Robot servomechanism Fig. 4 Schematic models of servomechanisms
1708
XS
+
+
PID Controller (Position)
JT
∆X D
uMX
Master XM
Compliance Generator &&D + D∆X& D + K∆XD = F M∆X
PID Controller (Position)
∆X D
JT uSX
Position Feedback Loop
1− w
FM
-
+
1− w
+
+
+
+
w
uMτ PID Controller (Force)
J
FD
T
+
+
Compliance Generator && + D∆X& + K∆X FD = M∆X
+ +
XS w
Force Feedback Loop τM
FS Master
τS
FD
+
+
-
+
J
T
uSτ PID Controller (Force)
-
-
Fig. 6 Schematic diagram of the improved parallel control system with the hybrid servomechanism
to users with this system by changing the weight parameter. Then, the weight parameter w was set to the same value in both the devices for the purpose of simplification; it is also possible to demonstrate the force/position reflection type by setting w to inverse value in both the devices. 2) Compliance Generator: In this study, virtual dynamics are applied to the position/force generator in order to give the flexibility to the robot system. On the side of position servomechanism, ∆X D is given as follow: ∆X D (k ) = G1 ∆F (k ) + G2 ∆X D (k − 1) + G3 ∆X& D (k − 1) (1) G n = M T + D + KT (2) G1 = T G n (3) G 2 = M G nT + D G n (4) G3 = M G n (5) Vice versa, the output of compliance generator FD can be easily obtained with following forward calculation. FD (k ) = D∆X (k ) + D∆X (k ) + K∆X (k ) (6) Here, M , D and K are the inertia, viscosity and stiffness parameters, whose values are shown in Table I. T is the sampling time, which was set to 5 ms in this system; this study ignored the communication delay which appears in the practical telecommunication. ∆X D and FD denote the position error and the force calculated with (1) and (6). k is a time parameter in the discrete-time system. 3) J T Type Hybrid Control: A hybrid control method was proposed by M. H. Raibert and J. J. Craig in [7], which employed a J −1 type transmission control. However, J −1 type generates undesired behavior near the singular point [8]. With Lyapunov’s analysis, T. Yabuta clarified that the J T type is more suited to the hybrid control than the J −1 type [9]. Hence, this study applied the J T type hybrid control,
1709
Master Slave
TABLE I COMPLIANCE PARAMETERS M (Inertia) D (Viscosity) 6.00 Ns/m 1.33 × 10-2 kg 6.00 Ns/m 1.33 × 10-2 kg
K (Stiffness) 6.00 × 102 N/m 6.00 × 102 N/m
Amplifier
: Drive Voltage : Angle Data : Force Data
Controller
Force Sensor
Computer
Master Robot
Force Sensor
Slave Robot
Fig. 7 Schematic diagram of the experimental system
which demonstrated better performance than the J −1 type by a preliminary experiment. B. Hardware System This study used two robot finger systems as the master/slave system, whose schematic diagram is shown in Fig. 7. The robot finger with 3’DOF has an encoder at each joint. In addition, it is equipped with a three-directional strain gauge type force sensor on its fingertips. These functions enable both the position and force feedback controls. IV. EVALUATION EXPERIMENT A. Objective Evaluation At first, this study examined the position/force display performance between two operators with the system of Fig. 6. During the experiment, the operators freely manipulated their
Position (mm)
own devices in the vertical direction. The operators could individually select their motions whether they resisted or followed to the behavior by another operator. In this experiment, the compliance generator was removed in order to examine only servo effect. Fig. 8 and Fig. 9 respectively show the results in the symmetric types with the position and force servomechanisms, i.e., with the conditions w = 0 and w = 1. Fig. 10 shows the result in the hybrid type with the condition w = 0.5. In these figures, (a) and (b) respectively indicate the position and force display performances. The results of Fig. 8 demonstrate the good position display performance with the large disagreement of force between two devices. On the other hand, the results of Fig. 9 demonstrate the inverse results of Fig. 8 in the position and force display performances. The hybrid type shows the good performance in both the position and force displays as shown in Fig. 10. Hence, it implies that the hybrid type is a leading candidate for the bilateral force communication. In addition, the compliance generator also provided the similar performance although the results are not presented in this paper.
0
5
Position (mm)
O p e ra to r II 10
T im e (s )
15
20
15
20
(a) Position display performance Force (N)
O p e r a to r I 0
5
10
T im e ( s )
15
20
15
20
(a) Position display performance Force (N)
O p e r a to r I I
5 0 -5 -1 0
O p e r a to r I 0
5
10
T im e ( s )
Position (mm)
(b) Force display performance Fig. 9 Performance of symmetric type with the force servomechanism 120 100 80 60 40 20
O p e r a to r I I
O p e r a to r I 0
5
10
15
20
15
20
T im e ( s )
(a)
Position display performance
Force (N)
10
O p e r a to r I I
5 0 -5 -1 0
O p e r a to r I 0
5
10
T im e ( s )
(b) Force display performance Fig. 10 Performance of hybrid type
stimulus, which has the magnitude of 100. The subjects could answer the magnitude of the operational feeling by larger values than 100 when they had perceived to easily manipulate the device. Similarly, the reaction sensitivity was estimated by larger values when the subjects were easy to sense the reaction force from another device. In these experiments, we provided the standard stimulus at first; the stimulus for comparison was presented in following turn. Each subject tried five sets in each comparison. In addition to servo effects, this study also examined the compliance effect. Fig. 11 shows the actual evaluations; the next chapter discusses the details. V. DISCUSSIONS
O p e ra to r I
10
O p e r a to r I I
5 0 -5 -1 0
O p e r a to r I I
10
B. Subjective Evaluation This study also examined the performance of servomechanisms based on the subjective evaluation. We attempted to evaluate the manipulating comfort toward each control system with the magnitude estimation, which is a psychological evaluation method. In detail, we conducted the evaluation experiments related to the operational feelings of master/slave device and the reaction force sensitivity from the communication partner; the number of subjects was five. As for the operational feeling, we instructed the subjects to manipulate the device up or down; then, the other device was kept free. On the other hand, as for the reaction force sensitivity, the subjects were instructed to resist to the reaction force from another operator. Then, the operator who conducts the experiment made an effort to generate the same force value of approximate 5 N. In each experiment, this study instructed the subjects to qualitatively evaluate their feelings by comparing the symmetric types (w = 0 or 1) with the hybrid type (w = 0.5). In this study, the operational feeling and the reaction force sensitivity when manipulating the device applying the hybrid type was defined as standard 120 100 80 60 40 20
120 100 80 60 40 20
O p e r a to r I 0
5
10
T im e ( s )
(b) Force display performance Fig. 8 Performance of symmetric type with the position servomechanism
A. Symmetric Type with the Position Servomechanism In this type, the results show that the subjects were easy to sense the reaction force with the worse operational feeling. They mentioned that the operational feeling was very hard. It is assumed that this is due to the lack of force feedback loop. In this case, the subjects had to generate the force against the friction of each actuator to move both the devices. It implies that it was difficult to smoothly operate the device. Further, the large disagreement of forces between the master and slave devices was appeared as shown in Fig. 8 (b). These evidences show that this type is unsuitable for the force communication.
1710
Magnitude of Evaluation
150
Position Type
Force Type
In the process, we noticed some technical issues. This study attempted to examine the issues by using the hybrid master/slave system. In particular, we focused on the relationships between the force/position display performance of the control system and the force/kinesthetic sensations given from the master/slave devices. This paper challenged to clarify the relationships by using a psychological evaluation method, the magnitude estimation. The force/position display performances revealed that to realize force communication requires the force feedback loop and the point of action of force because the users cannot exactly generate and perceive the reaction force without them; it is impossible to achieve them with one of the position and force servomechanisms. Further, according to the subjective evaluations, evaluation of the hybrid type, especially, compliance type was higher in both the operational feeling and the reaction force sensitivity. Therefore, it implies that the hybrid type is the most suitable for force telecommunication. However, force telecommunication also has the technical issues such as the miller image in cognition level; our current study attempts to examine it with a psychological method. Further, it is assumed that other problems emerge in the human sensory aspect during the practical applications because it has the communication delay and stability issues, etc. Therefore, in future works, a standard measure for force communication must be developed based on our results.
Compliance Type
Standard Value
100 50 0
A
B
C
D
E
Subjects Magnitude of Evaluation
(a) Operational feeling of the master/slave device 150
Position Type
Force Type
Compliance Type
Standard Value
100 50 0
A
B
C
D
E
Subjects (b) Reaction force sensitivity received from another device Fig. 11 Subjective evaluations of the master/slave system by five subjects
B. Symmetric Type with the Force Servomechanism The results are exactly opposite to the case of applying the position servomechanism. In this type, the subjects could operate the device very smoothly because the force feedback loop works to eliminate the friction and force error between devices. However, it seems to be very difficult to generate the large force as shown in Fig.9 (b). This is the reason why the operator cannot generate the reaction force when there is no point of action of force, especially, in the static task. Namely, these results imply that only force servomechanism cannot provide the force sensations to users. However, in the dynamic task, it may have the possibility to realize dynamic force because the point of action changes every moment. C. Compliance Effect As shown in Fig. 11, most subjective evaluations are slightly higher than that of the hybrid type. It means that the subjects instinctively felt the compliance effect although the force/position display performances are approximately same with normal type. This implies that the compliance generator produced the flexibility when manipulating the device; conversely, the subjects could perceive good force sensation due to the stiffness of dynamics when receiving the reaction force. These results also indicate the possibility that human beings change their perceptual strategy of position and force according to active and passive tasks. This hypothesis needs to be verified with a more detailed investigation. However, these results exhibit that the hybrid type master/slave system is the most suitable for the force communication and compliance factor is very effective. VI. CONCLUSION Our study approached the master/slave system from the viewpoint of force telecommunication between human beings.
REFERENCES [1] V. D. Daniulaitis, M. Osama, H. Kawasaki, Y. Tanaka and T. Hori, “Medical Palpation of Deformable Tissue Using Physics-Based Model for Haptic Interface RObot (HIRO),” Proc. 2004 IEEE/RSJ Int. Conf. Robotics and Systems, Sendai, Japan, November 2004, pp. 3907–3911. [2] M. Hara, C. Asada, T. Higuchi and T. Yabuta, “Perceptual Illusion in Virtual Reality using Haptic Interface,” Proc. 2004 IEEE Int. Conf. Robotics and Systems, Sendai, Japan, pp. 3901-3906, October 2004. [3] N. A. Tanner and G. Niemeyer, “Improving Perception in Time Delayed Teleoperation,” Proc. 2005 IEEE Int. Conf. Robotics Automat., Barcelona, Spain, pp. 356-361, April 2005. [4] S. Sirouspour, “Robust Control Design for Cooperative Teleoperation,” Proc. 2005 IEEE Int. Conf. Robotics Automat., Barcelona, Spain, pp. 1145-1150, April 2005. [5] D. A. Lawrence, “Stability and Transparency in Bilateral Teleoperation,” IEEE Robotics Automat., vol. 9, no. 5, pp. 624-637, 1993. [6] M. C. Cavusoglu and F. Tendick, “Kalman Filter Analysis for Quantitative Comparison of Sensory Schemes in Bilateral Teleoperation Systems,” Proc. 2003 IEEE Int. Conf. Robotics Automat., Taipei, Taiwan, pp. 2818-2023, 2003. [7] M. C. Cavusoglu and W. K. Chung, “Quantitative Comparison of Bilateral Teleoperation Systems Using H_infinity Framework,” Proc. 2004 IEEE/RSJ Int. Conf. Robotics and Systems, Sendai, Japan, pp. 22292234, 2004. [8] A. Balijepalli and T. Kesavadas, “Value-Addition of Haptics in Operator Training for Complex Machining Tasks,” ASME J. Computer and Information Science in Engineering, vol. 4, pp. 91-97, 2004. [9] M. H. Raibert and J. J. Craig, “Hybrid position/force control of manipulators,” ASME J. Dynam. Syst. Meas. Contr., vol. 102, pp. 126133, 1981. [10]O. Khatib, “A unified approach for motion and force control of manipulators: The operational space formulation,” IEEE Trans. Robotics Automat., vol. Ra-3, no. 1, pp. 43-53, 1987. [11]T. Yabuta, “Nonlinear Basic Stability Concept of the Hybrid Position/Force Control Scheme for Robot Manipulators,” IEEE Trans. Robotics Automat., vol. 8, no. 5, pp. 663-670, 1992.
1711
