Local Object Shape From Tactile Sensing - Semantic Scholar

Local Object Shape From Tactile Sensing Ning Chen, Ray Rink Department of Electrical Engineering University of Alberta, Edmonton Alberta, Canada T6G 2G7 [email protected]

Abstract Studies on contact problems such as shape recovery from tactile sensing involve complex geometrics and kinematics. In the rst part of the paper we introduce a convenient matrix expression of 3-D surfaces, which can be easily applied to manipulate contact constraints and kinematics equations using homogeneous transformation. In the second part of the paper we apply the notation introduced in the rst part and derive an active tactile sensing strategy which consists of a shape sensing algorithm and a touch-based robot control scheme.

1 Introduction Most robot manipulation and exploration tasks involve direct contact between a robot and the object it interacts with. In these situations direct contact constrains the motion of both bodies (the robot and the object/environment) which creates a more dicult problem than free body kinematics. The problem of direct contact between two rigid bodies was studied as early as 1940's in the area of gear manufacturing and design. The relative motion investigated in those studies is usually of special type, of interest to gear and cam design and analysis only. In the area of robotics there is essentially no restriction on the relative motion between a robot and its environment, and the uncertainties during robot motion have to be considered as well. Many contact problems and their applications remain interesting and challenging issues to robotics researchers. To date the most in uential studies on robot contact kinematics are perhaps those by Cai and Roth [1] [2] and Montana[3]. In Cai and Roth (1986)[1], basic contact constraints on point contact between planar curves were introduced. This study brought many valuable results from the previous gear studies into robotics. In Cai and Roth (1987) [2] the results were

Hong Zhang Department of Computing Science University of Alberta, Edmonton Alberta, Canada T6G 2H1 [email protected] extended to point contact of 3-D rigid bodies. Independently Montana (1988)[3] studied the same problem but described his result in a totally dierent manner. In comparison Cai and Roth gave a general solution to the problem of point contact of rigid bodies, using higher order contact constraints, although the higher order constraints are often noisy and less useful in practice. Montana, on the other hand, considered only the zeroth and rst order contact constraints and provided his result under a totally dierent but somewhat inconvenient kinematics framework. These studies are based on dierential geometry. When applied to the shape sensing (or curvature nding) problems, they often rely on an assumption of certain type of dierential relative motion between contacting bodies. For instance, as an application given in [3], a set of small rolling-probing motions was suggested for every single sampling point, in order to nd the local curvature at that point. This however is inecient and also dicult to apply because, during recognition or manipulation, large relative motion between robot end-eector and the object is necessary. Also, due to uncertainties the exact type of contact, rolling, for example, is dicult to maintain. In this study we will not assume the type of relative motion between the robot and object (rolling, sliding, or rolling and sliding). We will introduce a matrix expression of 3-D surfaces to simplify a local surface pitch to a second order polynomial. Such a surface matrix expression also facilitates the convenience of manipulating 3-D surface geometry with robot kinematics using homogeneous transformations. Instead of attempting to express the entire object surface as an overall surface function with respect to a single frame, the active sensing scheme presented in this paper expresses the unknown object as a set of local contact frames and the surface parameters (the surface matrix) associated with each frame. Although the accuracy of such an algorithm still depends on the spatial sampling rate over the object surface, it does not depend on the assumption of in nitesimal relative

motion implied by previous studies. Therefore the restriction on the speed of robot motion over the object surface is greatly released.

2 Surface matrix and its properties 2.1 De nitions At a contact point on a 3-D surface, we establish a Cartesian coordinate system such that its Z axis coincides with the inward pointing normal at this point. X and Y axes are orthogonal to each other and lie in the tangent plane. We name such a coordinate frame a contact frame. Consider a general 3-D surface which is expressed in the contact frame as

f (x; y; x) = 0: (1) The general surface function can be approximated locally at the contact frame as the following second order polynomial equation (notice that this is dierent from a \small area" in the dierential sense). c11x2 + 2c12xy + 2c13xz + 2c14x+ c22y2 + 2c23yz + 2c24y+ c33z 2 + c34z + c44 = 0 (2) Eq.(2) can be rewritten into a quadratic form matrix equation 0 c ;c ;c ;c 1 0 x 1 11 12 13 14 B C BB y CC = 0, c B [x; y; z; 1] @ c21;; cc22;; cc23;; cc24 C A @zA 31 32 33 34 c41; c42; c43; c44 1 (3) or X T SX = 0; (4) where we de ne a surface matrix S of a 3-D surface as 0 c ;c ;c ;c 1 11 12 13 14 B c ;c ;c ;c C S= B , @ c2131; c2232; c2333; c2434 CA c41; c42; c43; c44 cij =cji ;i=1to4;j =1to4 (5) and X = [x; y; z; 1]T ; (6) is the generalized robot position vector. Some example of the surface matrices are

0 0; 0; 0; a 1 B 0; 0; 0; b CC, S = Sp = B @ 0; 0; 0; c A s 0 10;;00;; 00;;d 0 1 B 0; 0; 0; 0 CC, etc, Sc = B @ 0; 0; 1; ?r A

0 1; 0; 0; 0 1 BB 0; 1; 0; 0 CC, @ 0; 0; 1; ?r A 0; 0; ?r; 0

0; 0; ?r; 0 where Sp is a general plane in 3-D, Ss is a sphere of radius r, and Sc is a cylinder of radius r and center line aligning with y axis. Note that all the surface matrices are described in their contact frame by de nition. We will refer to a second order surface with a surface matrix S as surface S in the rest of the discussion.

2.2 Basic operations on surface matrices In this section we demonstrate some of the useful operations required for later discussions. We will omit some of the proofs because of lack in space. 1. Addition/subtraction: The surface S as a result of the addition/subtraction of two surfaces S1 and S2 is

S = S 1  S2 : (7) This property indicates that a complex surface can be expressed as the sum of some of the simple primitive surfaces, such as those given as examples in section2.1. 2. Frame transformation: The surface S as a result of the coordinate transformation from the original surface S0 can be expressed as S = T T S0 T; (8) where T is the homogeneous transformation from the original frame to the new frame. Proof: Suppose before transformation the surface function is X0T S0 X0 = 0: (9) Due to the frame transformation, the generalized position vectors in the two dierent frames are related by X0 = TX: (10) Substitute (10) into the generalized function in the original frame and rearrange, we have X T (T T S0 T )X = 0; (11)

and it is easy to shown that S = T T S0 T is the transformed surface matrix which preserves all the properties and operations of a surface matrix. 3. Surface normal: The surface normal n = (nx ; ny ; nz ) at a point x = (x; y; z ) is given by n = 2SX(row1;2;3) : (12) Proof: Suppose F is the polynomial equivalence of surface S, we have ~n = rF; where r is the gradient operator. This property can be proved by carrying out the dierentiation.

3 Contact kinematics Let us label robot end-eector (or in many cases a robot nger) as f and the object as o. At each instance we distinguish between the contact point Pf on the robot and the contact point Po on the object. In addition to a xed global frame fW g, we introduce two contact frames fCf g and fCO g which at each instance has contact point Pf and Po as their origin respectively, and their Z axes share the inward pointing normal of the robot and the object, respectively. The X and Y axes lie in the tangent plane and are orthogonal to each other. The basic contact constraints we start with are the well known fact that at the point of contact both bodies share the same position in space and share the same surface normal, i.e., w Pf =w Po ; and (13) w nf = ?w no ; (14) where the superscripts on the right side denote the frame that the quantity is with respect to. Such a rule of notation is also used throughout the rest of the paper. The above basic equations hold for any motion as long as the contact is not broken. To introduce kinematics, let us consider the change in contact within a time interval
t. Suppose the contact point moves from Pf1 to Pf2 on the robot end-eector side and from Po1 to Po2 on the object side within the same time interval. Since Pf1 and Po1 and Pf2 and Po2 are contact point pairs before and after
t, they share the same surface normals before and after. Therefore,
nf =
no : (15)

n1 n1

P1

n1’ P2 n 1’

∆ n11’= n1 x ω ∆ t

∆ n12

n2 ∆ n1’2 =2S v ∆ t

∆ n11’ ∆ n1’2

Figure 2: The change in surface normal Because translation does not aect the change in surface normal (see Fig.2 for illustration), the change in normal vector is composed of the change due to rotation, (w  n
t), and that due to the contact point sliding on the surface, which from (12) is 2Sv
t, where v is the sliding velocity at the contact. Therefore (15) can be rewritten as

wf  nf + 2Sf vf = ?cf Rco (wo  no + 2Svo ); (16) where vf and vo are sliding velocities on the endeector and the object side, respectively. wf and w0 are angular velocities of the robot end-eector and the object, respectively. nf and n0 are normals at contact point of the robot end-eector and the object, respectively. Sf and So are the surface matrices of the robot end-eector and the the object, respectively. Cf RCO is the rotation from the contact frame fCf g to contact frame fCo g. The sliding velocity on the robot/dextrous hand (vf ) is always available from tactile sensing. However, the sliding velocity along the object surface is dicult to obtain in most cases because there are no sensors on the object. To eliminate the sliding velocity vo from (16), we imagine a pair of points Qf and Qo , which do not belong to either contacting bodies and move along the the robot and the object surface respectively, such that they are always at the instantaneous contact point. It is a well known result that the absolute velocity of Qf and Qo are equal (see, e.g., [4]), i.e., w (17) VQf =w VQo : If we denote the relative velocity between two arbitrary points P1 and P2 as Vp1 =p2 , (17) can be written as Vpf + VQ=Pf = Vpo + VQ=Po : (18) From (18) the sliding velocity at the object can be obtained as

vo = vQ=Po = VPf =Po + vf :

(19)

robot/dextrous hand

{S}

end−effector/finger

{S}

sr

z

z

{W}

{O}

y

y

pf p

object

p o rp

o

x

x

(a)

wr s

{W}

wr

{O}

o

(b)

Figure 3: (a) Active shape sensing using tactile feedback (b) The transformation graph

4 An active tactile sensing strategy for shape recovery Touch provides only extremely local measurements to an object. It requires a robot to maintain rolling and/or sliding contact with the object while collecting and inferring information of the unknown object. This type of combined robotic manipulation and tactile sensing, or the so called active tactile sensing [5], is essential to tactile perception. In this section we develop a shape sensing algorithm using the results from the previous sections. A touchbased robot control scheme is designed in the second part of this section which integrates the shape sensing with robot servo control.

4.1 A shape sensing algorithm The shape sensing algorithm proposed in this section is to describe a 3-D surface to surface pitches with respect to the instantaneous contact frames (or the spatial sampling points). We rst locate the contact frame using robot direct kinematics and tactile sensing, then solve the surface matrix using contact information available from sensor readings.

4.1.1 Finding the contact frame We choose the world frame fW g as the robot base frame. The object frame is named as fOg. We also introduce a sensor frame fS g which is attached at a con-

venient position on the robot eector (or nger tip). In active tactile sensing the motion of the robot/ nger is usually available from direct kinematics using the robot/ nger joint angle measurements. In other words the position w rs(t) and the orientation w Rs(t) of the sensor frame is known and time dependent due to the robot motion. Tactile sensing provides the relative position of the contact with respect to the sensor frame, i.e., the position s rf (t) and velocity s vf

of the contact point on the robot/ nger side are measurable from tactile sensor. The surface normal at the contact point can be measured by nding the direction of contact force from tactile sensing, or, in the case of active tactile sensing where the nger shape is known, the surface normal at the contact point can simply be calculated from the end-eector/ nger shape given the contact position, by using (12). Once the surface normal is de ned, the orientation of the contact frame s R is also de ned. c From the transformation graph shown in Fig.3(b), we have Pf =w rs +w Rs :srP ; and (20) Po =w ro +w Ro :o rP : (21) Equating (20) and (21) (the common position constraint) the position of the contact frame with respect to the object frame can be obtained as o rPo =w RTo (w rs ?w ro +w RS :S rPf ): (22) where W Ro is the orientation of the object frame fOg. Similarly, using the common normal constraint (12), the orientation of the contact frame can be found as o (23) RPo =W RTo :W RS :sRC :Cf RCo ;

4.1.2 Finding the local surface parameters in the contact frame

There are ten independent elements within a surface matrix S. To solve all the unknowns in a surface matrix with respect to a contact frame (or a spatial sampling point), we can use N measurements at a time. Eg., we use the current tactile sensor reading and the past N-1 measurements. Rewriting (16) for all N measurements and noting that we express all quantities with respect to the contact frames, Cf (t?i) fW (t ? i) + 2Sf (t)v(t ? i)g =

?Cf (t?i) fCf RCo 2S (t ? i)(V (t ? i) + v(t ? i)g (24) i = 1; 2; :::; N , where t is the time index, W (t ? i) = !(t ? i)  n(t ? i)

0 ?! (t ? i) 1 y B ! (t ? i) C x = B @ 0 CA,

.

0

(25)

and !(t ? i) is the relative angular velocity expressed in Cf (t ? i). Transforming all object surface matrices to the current contact frame Cf (t) (see operation (8)), we have

Finally, the least square solution for (30) can be obtained as [6]

S (t ? i) =Cf (t) TCTo (t?i)S (t)Cf (t) TCo(t?i) (26) Substituting the above equations into (24) yields xi = ?Syi ; i = 1 to N: (27) where xi = fCf (t) TC?oT(t?i) :Cf (t?i)(W (t?i)+S0 (t?i)v(t?i))g4N (28) and yi = fCf (t) TCo(t?i) :Cf (t?i)(V (t ? i) + v(t ? i))g4N : (29) Note that we now have all the N matrix equations expressed in terms of the surface matrix S in the current contact frame (or S(t)). We need, however, to drop the equations corresponding to the forth element of xi in (27), because the surface normal operation given in (12) provides only three non-trivial rows. Assemble (27) to the following form suitable for least square solution. Ac = x; (30) where c = [c11; c12; c13; c14; c22; c23; c24; c33; c34; c44]101; (31) A = [A ; A ; :::; A ] ; (32) 0 y ; y ; y1 ; y2 ; 0; 0;N0;30N;0;100 1 i1 i2 i3 i4 Ai = @ 0; yi1; 0; 0; yi2; yi3; yi4; 0; 0; 0 A , 0; 0; yi1; 0; 0; yi2; 0; yi3; yi4; 0 (i=1 to N ) (33) b = [b1; b2; :::; bN ]T3N 1; (34) yij (i=1 to N, j= 1 to 4) are elements of yi given in (29), and di (i=1 to N) is assembled from the rst three rows of xi which is given in (28). Note that trivial equations have been eliminated after the matrix assembly. The de nition of contact frame also introduces a boundary condition fz = 0; at x = y = 0g. To apply such a boundary condition an extra row [0; 0; :::; 0; 1] needs to be added to the A matrix, and an extra element [0] to the b vector. This brings the nal dimension to ((3N + 1)  10) for A and dimension ((3N +1)  1) for b. A minimum of three measurements (N  3) is required to solve the surface S which has a total of 10 independent variables.

4.2 A tactile-based control scheme

c = (AT A)?1 AT x:

(35)

As discussed before, due to the characteristics of active tactile sensing, we need to develop a control scheme which servos the robot joint motion based on the shape and other information (contact force and position) obtained from tactile sensing, such that the contact is maintained and a surface tracking strategy applied. Such a control is based on the current and previous tactile measurements, and is to guarantee the availability of the next sensing. The control objective of the active sensing can be generally stated as \to move the sensor at a desired speed V along direction  in the tangent plan of contact, while maintaining a desired contact force F". Note that V,  and F can be variables depending on the chosen surface tracking strategies. With tactile sensing the actual pose of the contact frame can be obtained from tactile sensing and robot direct kinematics, as shown in (22) and (23). The feedback contact states can be collected into a vector corresponding to the actual pose of the contact frame. Xa = [xa; ya ; KFa ; xa ; ya ; za ]T ; (36) z where xa and ya are the actual contact position in the tangent plane. Fa is the normal contact force. Kz is the stiness factor. xa ; ya and za de ne the actual contact orientation. In planning the robot motion for the next time interval, we consider the predicted local object shape, which is obtained from the shape sensing algorithm and undated every sampling period, and compensate the deviation between the desired and actual contact states. Such a control scheme can be implemented by the following steps:

 Initialize matrix A, vector b and Jacobian J ().  Calculate xd = Vx (t + dt) and yd = Vy (t + dt).  Calculate Xd (t + 1) = [xd ; yd ; zd ; xd ; yd ; zd ]T = f (xd ; yd ).  Calculate d = J (Xd (t + 1) ? Xa (t)).  Calculate (t + 1) = (t) + d:

"motor.data"

Z (mm) 374 373 372 371 370 369 368 367 366 365 364 -100 -110 -140

-120 -135

-125 X (mm)

Figure 4: The active shape sensing experiment In step three, f (x; y) = [fz (x; y); fx (x; y); fy (x; y); fx  fy ]T is a vector of functions derived from the surface matrix, i.e, fz (x; y) is obtained from solving z in (3), giving xd and yd ; and fx (x; y) and fy (x; y) are obtained from the surface normal equation (12), giving the position vector [xd; yd ; zd ]T .

5 Experimental results Active shape sensing experiments were carried out on our experimental system consisting of a PUMA 260 robot arm, a spherical probe, and a piezoresistive tactile array sensor mounted to the probe (Fig.4). The tactile sensor used for the experiments has a total of 16x16 stress sensing elements over an area of 1 inch square. The radius of the probe is 40mm. The sensory data processing and the real-time tactile servo were performed in a Sun 3/VME based multi-processor robot control system developed at the Robotics Research Laboratory at the University of Alberta[7]. Fig.5 illustrates an example of the object surface obtained by using the tactile-based control scheme together with the shape sensing algorithm given in section 4, when probing a cylinder like object. The object is unknown to the robot. For this experiment, a robot servo rate of 400Hz and a tactile data processing rate of 200 Hz was applied. The relative contact velocity, caused by both rolling and sliding, is controlled at 25mm/sec. The experiment demonstrated satisfactory control speed and response. More experments are also being performed and further analysis of the experimental data will be supplied shortly.

6 Conclusion A framework to discuss contact kinematics using surface matrix is introduced in this paper. Such a

-130

-130

-120

-115

-110

Y (mm)

-140

Figure 5: The trajectory of contact when probing a

cylinder-like object. The object is unknown to the robot.

mathematical tool is convenient to manipulate 3-D surface geometry, contact constraints and robot kinematics equations. An active tactile sensing strategy is developed, where both a shape sensing algorithm and a control scheme are designed. Preliminary experimental results show satisfactory control accuracy and response during real-time robot active surface sensing.

References [1] C. Cai and B. Roth. On the planar motion of rigid bodies with point contact. Mechanism and Machine Theory, 21(6):453{466, 1966. [2] C. Cai and B. Roth. On the spacial motion of a rigid body with point contact. In 87'IEEE International Conf. on Robotics and Automation, pages 686{695, 1987. [3] D. J Montana. The kinematics of contact and grasp. International Journal of Robotics Research, 7(3):17{30, 1988. [4] O. Bottema and B. Roth. Theoretical Kinematics. Elsvier, Amsterdam, 1979. [5] W.D Hillis. Active touch sensing. Master's thesis, MIT, Dept of Electrical Engineering and Computer Science, 1981. [6] S.L. Campbell and C.D Meyer. Generalized inverses of linear transformations. London:Pitman, 1979. [7] Ning Chen, Hong Zhang, and Ray E. Rink. Edge tracking using tactile servo. 95'IEEE/RJS International Conf. on Intelligent Robots and Systems, 2:84{99, 1995.