Manipulator Collision Detection and Collided Link Iden

Manipulator Collision Detection and Collided Link Identification based on Neural Networks Abdel-Nasser Sharkawy 1, 2 (), Panagiotis N. Koustoumpardis2, and Nikos Aspragathos2 1Mechanical

Engineering Department, Faculty of Engineering, South Valley University, Qena 83523, Egypt

[email protected] 2

Department of Mechanical Engineering and Aeronautics, University of Patras, Rio 26504, Greece

[email protected], [email protected]

Abstract. In this paper, a multilayer neural network based approach is proposed for the human-robot collisions detection during the motions of a 2-DoF robot. One neural network is designed and trained by Levenberg-Marquardt algorithm to the coupled dynamics of the manipulator joints with and without external contacts to detect unwanted collisions of the human operator with the robot and the link that collided using only the proprietary joint position and joint torque sensors of the manipulator. The proposed method is evaluated experimentally with the KUKA LWR manipulator using two joints in planar horizontal motion and the results illustrate that the developed system is efficient and very fast in detecting the collisions as well as the collided link. Keywords: Collision Detection, Collided Link Identification, Neural Networks, proprietary sensors.

1

Introduction

Safety is necessary factor when the robots and humans share the same workspace because the proximity of the operator to the robot can lead to potential injuries. Therefore, a system for collision avoidance or detecting the collision should be incorporated to collaborative manipulators. Collision can be avoided by monitoring the environment using vision as presented in [1, 2] or using proximity sensors as in [3]. Although these methods can be used for collision avoidance, modifications in the manipulator body are required for the sensors installation. Some researchers contributed to the safety system in HRI by the collision detection and reaction methods such as disturbance observer method [4, 5], nonlinear adaptive impedance control law [6] and detection methods based fuzzy logic and neural networks [7, 8]. In the previous paper [9] a neural network (NN) was presented for collision detection in one-joint motion. It is well known that there is a dynamic coupling between the joints of the serial manipulator particularly in high speeds and accelerations. Therefore, the scaling of our previous approach by using one independent NN for each joint could not be generally applied. In this paper, one NN coupling two joints is designed and trained

2

by Levenberg-Marquardt (LM) algorithm where a priori knowledge of the dynamic model of the robot is not required. In this method just the joint position and torque sensors are used which are proprietary to the KUKA LWR manipulator that is used for the experiments. For the training of the NN, the measurements of the collision force mapped to the joints torque is required. Then the trained NN can estimate the external torque and hence the collision is detected, and the collided link is identified. The generalization ability of the trained NN is presented.

2

Neural Network Design

In this work, the KUKA LWR manipulator is used as shown in Fig. 1. The arm is configured as a SCARA type robot (2-DoF for planar horizontal motions), to avoid the effects of gravity during the control of the motions. Joint 1 represents KUKA’s A3 (4th joint) and joint 2 represents KUKA’s A5 (6th joint). The collisions are performed randomly by the human hand touching the end-effector and the link between the two joints as shown in Fig. 1. One NN is designed and trained using the external torque estimated from KUKA Robot Controller (KRC). Then the trained NN is used to calculate the external torque, detect the collisions and identify the collided link. If the external collision torque by the trained NN exceeds a threshold value, then the collisions are detected. The collision threshold is defined as the maximum of the absolute values of the training error between the external torques estimated from KRC and the trained NN of contact-free motion. Joint 1 (A3) F (case 2)

F (case 2) F (case 1)

Joint 2 (A5) F (case 1) External Force Sensor

(a) (b) Fig. 1. a) Experimental setup with Kuka LWR manipulator. b) External force F is applied during the motion of joints where the black spot means the center of mass for each link.

A sinusoidal motion 𝜃𝑑𝑖 (𝑡), 𝑤ℎ𝑒𝑟𝑒 𝑖 = 1,2 with variable frequency is commanded simultaneously to joint 1 and joint 2 respectively. These two serial joints compose a coupled system where if any external force exerts on any link, then the torque of each joint is affected. If the collision occurs on the end-effector (case 1, Fig. 1), the effect appears clearly on the measured joint torques of the two joints. But when there is collision between the two joints (case 2, Fig. 1), a small effect occurs in the torque of joint 2 because of the inertial force applied on the link whereas the effect on the torque of joint 1 appears clearly. The measured torque of joint 1 is always higher than of joint 2 whether there is collision or no.

3

After many experiments and trials as discussed in our previous paper [9], the main inputs for the neural network that give us the good performance and the minimum mean squared error (mse) are the current position error 𝜃̃𝑖 (𝑘) between the desired and actual joint position, the previous position error 𝜃̃𝑖 (𝑘 − 1), the commanded joint velocity 𝜃̇𝑑𝑖 and the measured joint torque 𝜏𝑖 , where 𝑖 = 1,2. The actual joint velocity from the previous paper [9] is replaced by the commanded joint velocity since after more experiments it is found that the performance of the NN (lower training error) is better using the commanded joint velocity. Three layers are used to compose the NN as shown in Fig. 2; the input layer, the non-linear hidden layer and the output layer that calculates the external torques ′ ′ 𝜏𝑒𝑥𝑡1 , 𝜏𝑒𝑥𝑡2 of joints 1 and 2 that are compared respectively with the external torques 𝜏𝑒𝑥𝑡1 , 𝜏𝑒𝑥𝑡2 given by KRC. The external torques 𝜏𝑒𝑥𝑡1 , 𝜏𝑒𝑥𝑡2 are used only for training the network and the external collision force can be measured by any external sensor and transformed to the joint torque by the Jacobian. It should be noted that one independent NN for each joint is also applied but it has less performance than using one NN coupling the two joints since the motion of the two joints is coupled system as discussed.

Fig. 2. The multilayer neural network scheme for the two joints.

3

NN Training and Testing

The NN is trained using Levenberg-Marquardt learning which has the trade-off between the fast learning speed of the classical Newton’s method and the guaranteed convergence of the gradient descent [10, 11]. A sinusoidal motion with variable frequency is commanded on the two joints of KUKA LWR robot (Fig. 1), as described previously, where the frequency is linearly increasing from 0.05 Hz to 0.17 Hz for joint 1 and from 0.05 Hz to 0.23 Hz for joint 2. These frequencies produce angular velocities ω up to 1.068 rad/s for joint 1 and 1.445 rad/s for joint 2. The range of joints motion is [-80, 10] deg for each joint. The training data are divided into two sets: in the first set the robot joints perform the motion without any external force (collision) applied to the robot body and in the second set the same motion is performed with the user performing random collisions

4

suddenly and stochastically with his hand on the end-effector and between the two joints. During the experiments the robot is commanded to move with position control mode and no reaction strategy is implemented. Using the data collected in the motion with and without collision, the neural network system with the eight inputs is trained. The total number of input-output pairs collected from the experiments are 170131. From these data 80% are used for training, 5% for validation and 15% for testing. After trying many different weights’ initializations and number of hidden neurons, it is found that the best case is 120 hidden neurons and 1000 iterations, which gives the minimum mse and the adequate collision threshold. The training process is very fast and stable and applied using Matlab on an Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz processor. The trained NN is evaluated with the same data-set that are used for the training to get an insight about the approximation. The difference between the measured external ′ ′ torques (𝜏𝑒𝑥𝑡1 , 𝜏𝑒𝑥𝑡2 ) and the external torques (𝜏𝑒𝑥𝑡1 , 𝜏𝑒𝑥𝑡2 ) estimated by the NN system is calculated and it is derived that the approximation error of the collision torque for each joint is higher than the error of contact-free motion. The averages of the absolute error values are very small and are 0.0954 Nm and 0.0486 Nm for joint 1 and joint 2 respectively. By providing the dataset without any collisions to the trained NN, the resulted collision threshold values are 𝜏𝑡ℎ1 = 1.5529 𝑁𝑚 and 𝜏𝑡ℎ2 = 0.8006 𝑁𝑚 for joint 1 and ′ | > 𝜏𝑡ℎ𝑖 for the ith joint. joint 2 respectively and the collision is assumed, when |𝜏𝑒𝑥𝑡𝑖 The external collision torques from KRC and the trained NN for both joints are compared as shown in Fig. 3. The spikes with blank circles refer to the collisions between the two joints whereas the others are the collisions on the end-effector.

(a) Joint 1 (b) Joint 2 Fig. 3. The two external collision torques from KRC and NN.

4

Performance of the Trained NN

The proposed method is evaluated and tested in the experimental setup by commanding the robot to perform two joints motion with constant velocity profiles, which is a common case in robot applications. The generalization of the trained NN is tested for joint ranges outside the ones that have been used for training, to show its effectiveness. An external force sensor (ATI F/T Nano 25) is used to compare its readings that are converted into joint torques via Jacobian with the external torques estimated by the trained NN and KRC.

5

4.1

Trained NN Evaluation

The trained NN is evaluated with a speed equal to 0.5 rad/s on both joints with two cases of the collisions; the first case is that the force sensor is fixed on a predefined location at the end-effector (Fig. 1a) and the collisions occur only on the force sensor perpendicularly to the end-effector body and the second case is that the force sensor is fixed on the link between the two joints and the collisions occur only on the force sensor perpendicularly to the link between the two joints. Fig. 4 and Fig. 5 illustrate the comparison between the three torques for both joints form KRC, Trained NN and external force sensor. From Fig. 4 where there is a collision on the end-effector only, both joints’ torques are affected by the collision and also the close approximation between the three external torques is very good particularly from the trained NN and KRC since the external torque from KRC is used for training the NN. Before and after collision the joint torques measured by the external sensor are zero and during the sudden collision increased quite rapidly. This wave form illustrates the collision phenomenon most accurately since the robot dynamics do not affect this measurements as it is the case in the estimated joint torques by KRC. The actual collision detection time is calculated as the elapsed time from the detection of the collision by the external force sensor to the moment when the external torque by the trained NN exceeds the threshold so the actual collision detection time for joint 1 and 2 is 14.8 msec and 27.01 msec respectively. From Fig. 5 where there is a collision between the two joints only, joint 1 external torque is highly affected by the collision, whereas joint 2 external torque has small effect at time when the force exerts and this effect sometimes does not appear clearly if the collision force is small. The close approximation between the three torques is good particularly from the trained NN and KRC. The good approximation between the external joint torques measured by KRC and external sensor with the estimated by the trained NN proves the success of our trained NN. The actual collision detection time for joint 1 is 21.4 msec. As shown from the results the trained NN is easily identifying the collided link since when the collision torques of joint 1 only exceeds the threshold, it means that the collisions occur on the link between the two joint whereas when the collision torques of both joints exceeds the threshold, it means that the collisions occur on the end-effector. To confirm the validity and efficiency of the proposed method under a wide range of operating conditions and acquire a performance measure, another 61 trials of collisions (NC) are evaluated with various magnitudes, points of collision, directions and different velocities of the motion. Table 1 provides the performance in terms of the number of the correctly detected collisions (CC), the number of false negatives (FN) which is the number of collisions not detected by the method and the number of the false positives (FP) which are the collision alerts provided by the method when there is not an actual collision. It is noted from Table 1 that when there are collisions at the end-effector, the proposed method succeeds with 92.59 % and 81.48 % to detect the collisions affected on joints 1 and 2 respectively which could be considered very good success rate. When there are collisions between the two joints, the trained NN succeeds but with a lower rate (73.5 %) to detect the collisions affecting on joint 1 only and not joint 2. Also the number of the false positive collisions is low (16 %) which means that our method has

6

less sensitivity to the external disturbances and unmodelled parameters. In another meaning the efficiency of the presented method or the success average percentage to detect the collisions is very good (82.52 %).

(a) Joint 1 (b) Joint 2 Fig. 4. The three external collision torques from KRC, trained NN and using the force sensor when there is collision on the end-effector.

(a) Joint 1

(b) Joint 2

Fig. 5. The three external collision torques from KRC, trained NN and using the force sensor when there is collision between the two joints. Table 1. Summary of the performance of the proposed method obtained for the different collision scenarios and angular speeds of the joints Trained Neural Network Method Collision Scenario Joint Efficiency % NC CC FN FP Collisions on the end-effector Collision between the two joints

Joint 1

27

27

0

2

92.59 %

Joint 2 Joint 1

27 34

26 34

1 0

3 9

81.48 % 73.5 %

0

0 98.86 %

0 1.136 %

0 16 %

--------82.52 %

Joint 2 Average Percentage

4.2

87.035 % 73.5 %

Trained NN Generalization

The joint motion, for collecting data used for training the NN and its evaluation, limited in the range from -80 deg to 10 deg for each joint. Furthermore, two experiments are executed outside of these ranges (the first from 10 deg to 40 deg and the other one from -110 deg to -80 deg range) with random collisions on the end-effector and between the two joints to show the generalization of the trained NN. The results are shown in Fig.

7

6 and Fig. 7, where it is observed that the trained NN approximates adequately the two KRC and trained NN external torques but the error is a little bit larger compared to the previous case (inside the training range). However, the trained NN presents satisfactory performance which proves the generalization ability. It should be noted that the spikes have blank circle refers to the collisions between the two joints whereas the others are the collisions at the end-effector.

(a) Joint 1 (b) Joint 2 Fig. 6. The two external collision torques from KRC and trained NN during out of range two joints motion (from 10 to 40 deg).

(a) Joint 1 (b) Joint 2 Fig. 7. The two external collision torques from KRC and trained NN during out of range two joints motion (from -110 to -80 deg).

4.3

Discussion of The Results

The proposed method is easily applied and the training is very fast. The fuzzy based part of [7] is compared with our method. The estimated external torque 𝜏𝑒𝑥𝑡 given by the robot controller is used here only for training the NN, whereas in [7] the external torque measured by an ATI F/T Nano 25 force sensor is used for verification and the training. In the present method, one trained NN is used only for collisions detection since the two joints motion is a coupled system whereas in [7] two independent trained fuzzy systems are used. Our method succeeds to detect the collisions when there are collisions on the end-effector with a percentage of 87% whereas in [7] the Fuzzy system achieves 85% and for collisions between the two joints our method achieves 73.5 % whereas in [7] Fuzzy system has 50%. In [8], the proposed method by Lu et.al depends on two external force sensors, whereas our method does not require any external sensors and is used with any robot without model knowledge but having joint torque sensors. Our method presents also

8

low detection time for the collisions. It should be noted that because of the different data and operating conditions used in our paper and the other two papers, it is difficult to compare quantitatively the time required for the detection of the collisions.

5

Conclusion and Future Work

In this paper, a method is proposed for human-robot collision detection based on the multilayer neural network approach trained by Levenberg-Marquardt algorithm. One NN system is implemented and trained for a 2-DoF robot which is a coupled system. The training is stable and very fast. The inputs to the NN are derived from the joint position and torque sensors and the method is able to detect the collision of the robot with the human hand very quickly and identify the collided link. The evaluation of the proposed method shows that our method, compared with the other methods, is efficient in detecting the collisions, since it succeeds with very good and higher percentage to detect the collisions whether on the end-effector or between the two joints. The number of the false positive collisions is low which means that the presented method has low sensitivity to the external disturbances and unmodelled parameters. The trained NN is tested using out of training range motion and presents satisfactory performance which proves the generalization ability. Because of the promising results in this paper, it is considered the extension of the proposed approach to implement the collision detection system for three joints of the manipulator taking into consideration the effect of the gravity during the motion. Acknowledgments. Abdel-Nasser Sharkawy is funded by the “Egyptian Cultural Affairs & Missions Sector” and “Hellenic Ministry of Foreign Affairs Scholarship” for Ph.D. study in Greece.

References 1. 2.

3.

4.

5. 6.

7. 8.

9. 10. 11.

A. Mohammed, B. Schmidt, and L. Wang, “Active collision avoidance for human – robot collaboration driven by vision sensors,” Int. J. Com. Integr. Manuf., vol. 30, no. 9, pp. 970–980, 2017. F. Flacco, T. Kroeger, A. De Luca, and O. Khatib, “A Depth Space Approach for Evaluating Distance to Objects with Application to Human-Robot Collision Avoidance,” J Intell Robot Syst, vol. 80, no. Suppl 1, pp. S7–S22, 2015. T. L. Lam, H. W. Yip, H. Qian, and Y. Xu, “Collision Avoidance of Industrial Robot Arms using an Invisible Sensitive Skin,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 4542–4543. S. Haddadin, A. Albu-sch, A. De Luca, and G. Hirzinger, “Collision Detection and Reaction : A Contribution to Safe Physical Human-Robot Interaction,” in 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008, pp. 3356–3363. C. Cho, J. Kim, S. Lee, and J. Song, “Collision detection and reaction on 7 DOF service robot arm using residual observer,” J. Mech. Sci. Technol., vol. 26, no. 4, pp. 1197–1203, 2012. S. Morinaga and K. Kosuge, “Collision Detection System for Manipulator Based on Adaptive Impedance Control Law,” in Proceedings of the 2003 IEEE International Conference on Robotics &Automation, 2003, pp. 1080–1085. F. Dimeas, L. D. Avendano-valencia, and N. Aspragathos, “Human - Robot collision detection and identification based on fuzzy and time series modelling,” Robotica, pp. 1–13, May 2014. S. Lu, J. H. Chung, and S. A. Velinsky, “Human-Robot Collision Detection and Identification Based on Wrist and Base Force / Torque Sensors,” in Proceedings of the 2005 IEEE International Conference on Robotics and Automation, April 2005 , pp. 796–801. A.-N. Sharkawy and N. Aspragathos, “Human-Robot Collision Detection based on Neural Networks,” Int. J. Mech. Eng. Robot. Res., vol. 7, no. 2, pp. 150-157, March 2018. K. Du and M. N. S. Swamy, Neural Networks and Statistical Learning. Springer, 2014. M. T. Hagan and M. B. Menhaj, “Training Feedforward Networks with the Marquardt Algorithm,” IEEE Trans. NEURAL NETWORKS, vol. 5, no. 6, pp. 2–6, 1994.