Racket Control for a Table Tennis Robot to Return a Ball

SICE Journal of Control, Measurement, and System Integration, Vol. 4, No. 1, pp. 001–008, January 2011

Racket Control for a Table Tennis Robot to Return a Ball Chunfang LIU ∗ , Yoshikazu HAYAKAWA ∗,∗∗ , and Akira NAKASHIMA ∗ Abstract : This paper mainly proposes a racket control method for returning a table tennis ball to a desired position with a desired rotational velocity. The method determines the racket’s state, i.e., the racket’s striking posture and translational velocity by using two physical models: the racket rebound model and the aerodynamics model. The algorithm of determining the racket’s state is derived by solving nonlinear equations and solving a two-point boundary value problem of a differential equation. But this is not suitable for a real-time process because of large computing time. The paper proposed a modified algorithm which could be used for a real-time process by introducing a simple aerodynamics model. Numerical simulations and experimental results show effectiveness of the proposed methods. Key Words : racket control, racket rebound model, simple aerodynamics model, two-point boundary value problem, table tennis robot.

1. Introduction

High Speed Camera (900fps)

It is very important for a robot to work well in unstructured and/or dynamic environments, where complementary cooperation between physical models and on-line data processing of external sensors is one of the key technologies. In this sense, we are interested in developing a robot which plays table tennis with human, that is a typical situation of dynamic environments, where the robot has to be smart enough to overcome a variety of coming balls. To realize such a robot, we choose a strategy, which is different from professional players’ strategy. Figure 1 is a table tennis system which we have developed. Our approach is that the following tasks are performed accurately and in real-time. 1) Detect the ball’s position and velocity just after the ball is shot from the catapult machine by vision sensors. 2) Estimate the trajectory of the ball by using the ball’s data obtained in the task 1). 3) Control the robot with the racket for returning the ball to a target destination in the opposite court. This paper aims to achieve the task 3), i.e., determining a suitable posture and velocity for the racket to return the ball to a desired landing position with a desired translational/rotational velocity on the opposite court. There have been many researches on controlling the racket for table tennis robot returning balls. Miyazaki et al. [1] proposed a racket control method by using three input-output maps which corresponded to racket rebound model and aerodynamics model. The maps were constructed based on experimental data, not on physical models. Acosta et al. [2] determined the ∗ ∗∗

Department of Mechanical Science and Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan RIKEN-TRI Collaboration Center, RIKEN, 2271-103, Anagahora, Shimoshidami, Moriyama-ku, Nagoya 463-0003, Japan E-mail: [email protected], [email protected], a [email protected] (Received xxx 00, 2011) (Revised xxx 00, 2011)

Manipulator

Ball Catapult Ball

Racket

y

z

ΣB x Fig. 1

Table

Table tennis system .

posture of the racket under the assumption that when the racket struck the ball, the reflection angle of the ball was same as the incidence angle and also there was about 30% energy loss after the rebound. The trajectory of the ball was estimated by the parabolic throw formula which considered the friction of the ball in the air. Yang et al. [3] fixed the racket’s pitch angle and decided the racket’s yaw angle by using the direction of the coming ball and the specular reflection. However, all of the above studies have not considered exact physical models of either racket rebound or aerodynamics. Moreover, the rebound ball’s rotational velocity has not been treated as one of control variables. In the authors previous researches, a real-time method for measuring the ball’s position, translational and rotational velocities by applying two high speed cameras (900Hz) has been proposed [4],[5]. And three physical models have been established: the aerodynamics model (ADM) which considers both the drag and Magnus forces [6], the table rebound model (TRM) and the racket rebound model (RRM) [7]. Both rebound models characterize the variation of the ball’s translational and rotational velocities before and after rebounding. In [8], based on RRM and ADM, a racket control method has been proposed, where the racket’s state, i.e., the racket’s translational velocity, the yaw angle, and the pitch angle at the moment of striking the ball, are determined to return the ball to a desired position on the opposite court. Then the effectiveness of the proposed method was verified by numerical simulations. In this paper, the method proposed in [8] is summarized briefly, and the racket control planning is discussed rigorous-

c 2011 SICE JCMSI 0001/11/0401–0001 ⃝

2

SICE JCMSI, Vol. 4, No. 1, January 2011

ly from a theoretical viewpoint, i.e., concerning a table tennis robot, a two-point boundary value problem (TPBVP) is discussed more rigorously than [8]. Moreover, unlike [8], the effectiveness of the proposed racket control is verified by not only numerical simulations but also experiments in this paper. Section 2 shows two physical models RRM and ADM briefly. In Section 3, by regarding RRM as a set of nonlinear equations with respect to the racket’s state, an existence condition for real solutions of the set of nonlinear equations is shown and moreover the solutions are expressed in the closed form. On the other hand, associated with ADM, when considering the racket control, it is shown that the TPBVP plays important role. In Section 4, an on-line control method for the racket’s state is proposed, where the method for determining the robot’s state at the striking time is shown. Section 5 shows numerical simulations and experimental results and Section 6 is a conclusion.

as follows. [ ] [ ] v1 − V v0 − V = RRRM (α, β) , ω1 ω0 where [ RRRM (α, β) =   cos β  0 RR =   − sin β

RR 0

0 RR

][

sin β sin α cos α cos β sin α

Avv Aωv

Aωv = −kω rS 12 ,

Avω Aωω

sin β cos α − sin α cos β cos α

Avv = diag(1 − kv , 1 − kv , −er ),

2. Physical Models of Ball Motion There are two physical models, the racket rebound model [7],[9] and the aerodynamic model [6]. The racket rebound model (RRM) expresses a relation between ball’s velocities at the moment just before and after a racket strikes the ball. The aerodynamic model (ADM) is a differential equation which shows the motion of the ball flying in the air with the rotational velocity. In this section those physical models will be shown briefly. The base coordinate system ΣB is set at a corner of the table (see Fig.2) through this paper. The size of the table is 2.74 × 1.525 [m].

(1)

][

RR 0

0 RR

]T ,

    ,

Avω = kv rS 12 ,

Aωω = diag(1 − kω r2 , 1 − kω r2 , 1),

and S 12

  0  =  −1  0

1 0 0

0 0 0

    .

Note that RR is a rotational matrix from ΣB (the base coordinate) to ΣR (the racket coordinate) with the yaw α and the pitch β. The parameter r = 2 × 10−2 [m] is the radius of the ball, and the parameters kv = 6.15 × 10−1 , kω = 2.57 × 103 and er = 7.3 × 10−1 are obtained by experimental data. See [7],[9] for details about how to identify those parameters and how well RRM works. 2.2 Aerodynamics Model (ADM) When a ball flies in the air with the rotational velocity ω, the ball motion is given as follows.

Fig. 2

2.1

The base coordinate system .

Racket Rebound Model (RRM)

Suppose that the racket strikes the ball with the racket’s translational velocity V ∈ R3 and the posture of the yaw angle α ∈ [− π2 , π2 ] and the pitch angle β ∈ [0, π]. Denote the ball’s translational and rotational velocities just before and after the racket strikes the ball respectively by v0 ∈ R3 , ω0 ∈ R3 and v1 ∈ R3 , ω1 ∈ R3 (See Fig.3).

ρ ρ ˙ p(t) ˙ + C M (t) Vb ω × p(t), ˙ p(t)=−g ¨ − C D (t) S b ∥p(t)∥ m m (2) where p(t) ∈ R3 is the ball’s position at the time t, g = [0, 0, g]T with the acceleration of gravity g = 9.8 [m/s2 ], S b = 12 πr2 , and Vb = 43 πr3 . Notice that ω = [ω x , ωy , ωz ]T ∈ R3 is the ball’s rotational velocity which is here assumed constant during the ball flies in the air. The parameters m and ρ are the ball’s mass and the air density; m = 2.7 × 10−3 [kg] and ρ = 1.184[kg/m3 ](25◦ C). C D (t) and C M (t) in (2) are respectively drag coefficient and Magnus coefficient, which are given by C D (t) = aD + bD h(p(t), ˙ ω),

RR (α , β ), V

C M (t) = a M + b M h(p(t), ˙ ω), (3)

where h(p(t), ˙ ω) = √

Fig. 3

The racket rebound model .

Then the relation between (v0 , ω0 ) and (v1 , ω1 ) can be shown

p˙ x ωy − p˙ y ω x ( p˙ x ωy − p˙ y ω x )2 + ( p˙ 2x + p˙ 2y )ω2z

(4)

and aD = 0.505, bD = 0.065, a M = 0.094 and b M = −0.026. The identification for those parameters has been proposed and verified in [6].

3

SICE JCMSI, Vol. 4, No. 1, January 2011

3. 3.1

Models’ Properties

Solution to RRM

Concerning RRM described by (1), we will consider the following problem: given v0 , ω0 , v1 and ω1 , find V , α, β which satisfy (1). Note that (1) is a set of 6 nonlinear equations with respect to V , α, β. It is easy to see from the first three equations that V is given by { } V =v0 + RR (I − Avv )−1 RTR (v1 − v0 ) − Avω RTR ω0 . (5) Substituting (5) into the last three equations of (1) leads to { } ω1 =S (v1 − v0 ) + RR Aωv (I − Avv )−1 Avω + Aωω RTR ω0 =S (v1 − v0 ) + ω0 ,

(6)

where the last equality can be derived from the fact that Aωv (I − Avv )−1 Avω + Aωω = I, and the matrix S is defined as follows. S := −RR Aωv (I − Avv )−1 RTR =

kω r S s (α, β), kv

where

 0   S s (α, β)= − cos α cos β  − sin α

cos α cos β 0 − cos α sin β

(7)

kω r kv (v1

η = S s (α, β)ξ.

− v0 ) (9)

From the above observation, it is easy to see that if RRM described by (1) has a solution of (V , α, β), then it is necessary that ξT η = 0 which is equivalent to (v1 − v0 )T (ω1 − ω0 ) = 0.

(10)

Now we will solve (9) with respect to α and β under the condition that ξT η = 0 holds. By using the notations η = [η x , ηy , ηz ]T and ξ = [ξ x , ξy , ξz ]T , (9) is rewritten as

(16)

Notice here that the condition (15) implies that the following inequality always holds. √ −b ± b2 − ac ≤ 1. −1 ≤ (17) a In fact, it is easy to see that b2 ≤ a(η2x + η2y + η2z ) ≤ a2 under (15), which means |b| ≤ a, i.e., both a − b ≥ 0 and a + b ≥ 0 hold. On the other hand, it is easy to see that b2 − ac ≤ (a + b)2 and√b2 − ac ≤ (a − b)2 , which by noting that a ± b ≥ 0, means b2 − ac ≤ a±b. Now√it is straightforward to prove that √ √ 2 b2 − ac ≤ a + b implies −b+ ab −ac ≤ 1 and b2 − ac ≤ a − b √

implies −1 ≤ −b− ab −ac , which are equivalent to (17). Once α is determined by (16), β is obtained by (12) as 2

ξz sin β − ξ x cos β =

 sin α   cos α sin β   0 (8)

is a skew symmetric matrix. By introducing new variables ξ and η as ξ = and η = ω1 − ω0 , then (6) is equivalent to

where ξ T η = 0 is used. Thus α is given by √ −b ± b2 − ac . sin α = a

ηy , cos α

(18)

which has a real solution β if and only if √ |ηy | ≤ ξ2x + ξz2 cos α, which is equivalent to 2  √  −b ± b2 − ac  η2y  .  ≤ 1 −  a ξ2x + ξz2

(19)

By summarizing the above observation, we get the next proposition. Proposition 1: RRM described by (1) has a solution of (V , α, β) if and only if (1) ξT η = 0, (2) ∥η∥ ≤ ∥ξ∥, (3)

η2y ξ 2x +ξz2

≤1−

(

)2 √ −b± b2 −ac . a

η x =ξy cos α cos β + ξz sin α,

(11)

When these conditions are satisfied, V , α, and β are given by (5), (16) and (18).

ηy =−ξ x cos α cos β + ξz cos α sin β,

(12)

ηz =−ξ x sin α − ξy cos α sin β.

3.2 Two-Point Boundary Value Problem for ADM

(13)

Here we will consider the two-point boundary value problem (TPBVP) of ADM described by (2), i.e., given p(t1 ) and p(t2 ) with t1 < t2 , find p(t) and p(t) ˙ for all t ∈ [t1 , t2 ] which satisfy (2). It is well known that unlike the initial value problem, a TPBVP of an ordinary differential equation does not always have a (unique) solution. Let us consider the following TPBVP [10]: { p(t) ¨ = f (t, p, p), ˙ (20) p(t1 ) = p1 ∈ Rn , p(t2 ) = p2 ∈ Rn , t1 < t2 ∈ R.

(11) and (13) lead to ξy2 cos2 α = (η x − ξz sin α)2 + (−ηz − ξ x sin α)2 , which is equivalent to the following quadratic equation with respect to sinα a sin2 α + 2b sin α + c = 0,

(14)

where a = ξ2x + ξy2 + ξz2 , c = η2x + η2z − ξy2 .

b = ξ x ηz − ξz η x ,

The next theorem is known as one of sufficient conditions for existence of a solution of the TPBVP.

Notice that the quadratic equation has real roots if and only if b2 − ac ≥ 0, which can be shown to be equivalent to ∥η∥2 := η2x + η2y + η2z ≤ ξ2x + ξy2 + ξz2 =: ∥ξ∥2 ,

(15)

Theorem 2 [10] : Suppose that there exist constants K, L > 0 such that f (t, p, p) ˙ satisfies ∥f (t, pa , p˙ a ) − f (t, pb , p˙ b )∥ ≤ K∥pa − pb ∥ + L∥p˙ a − p˙ b ∥,(21)

4

SICE JCMSI, Vol. 4, No. 1, January 2011 (V ,α , β )

for any pa , pb ∈ Rn and any t ∈ R. Then if t1 and t2 satisfy (t2 − t1 ) t2 − t1 +L < 1, 8 2

Striking position

2

K

z x Ball Racket z ( p0 , v 0 , ω 0 ) y t = t0 ∑B x

(22)

the TPBVP described by (20) has a unique solution. In the case of ADM described by (2), note that

Fig. 4

ρ ρ f (t, p, p) ˙ = −g − C D (t) S b ∥p(t)∥ ˙ p(t) ˙ + C M (t) Vb ω × p(t) ˙ m m where C D (t) and C M (t) are functions of p˙ and ω as shown in (3) and (4). Therefore, f (t, p, p) ˙ is a function of only p˙ and does not depend on p, so f (t, p, p) ˙ can be denoted by f (p). ˙ By the mean-value theorem, there exists a p˙ such that f (p˙ a ) − f (p˙ b ) =

∂f (p) ˙ (p˙ a − p˙ b ), ∂p˙

which implies that by supporting that ∥p(t)∥ ˙ ≤ 10[m/s] and ∥ω∥ ≤ 400[rad/s], we can use Theorem 2 and it is easy to see that in (21), K → 0 and

∂f (p) ˙

≈ 3.17, L = max

p,ω ˙ ∂p˙ where

)} { ( ∂f (p) ˙ ρ ∂h p˙ p˙ T = − S b ∥p∥ ˙ bD p˙ + CD I + ∂p˙ m ∂p˙ ∥p∥ ˙ 2 { } ρ ∂h + Vb b M ω × p˙ + C M (ω×)T . m ∂p˙

Therefore it can be concluded that the TPBVP of ADM has a unique solution if t2 − t1 < L2 ≈ 0.63[s].

4.

Racket’s Striking Posture and Velocity

Before discussing how to control the racket’s striking posture and velocity, recall how the ball motion is generated by two physical models, RRM and ADM. Let the time be t = 0 when the racket strikes the ball, and denote the time just before and after striking the ball by t0 and t1 , respectively, i.e., t0 = 0− and t1 = 0+ . Suppose the ball’s state at the time t0 is given as p(t0 ) = p0 , p(t ˙ 0 ) = v0 , ω(t0 ) = ω0 .

(23)

According to RRM described by (1), the racket’s state (V , α, β) determines p(t1 ) = p1 , p(t ˙ 1 ) = v1 , ω(t1 ) = ω1 ,

(24)

where note that p1 = p0 . Then ADM described by (2), where the initial conditions p(t1 ), p(t ˙ 1 ) are given by (24), provides a ball motion (p(t), p(t)) ˙ for t ∈ [t1 , t2 ] (as shown in Fig. 4). Note that the time t2 is a landing time when the ball lands in the opposite court, i.e., p(t2 ) = p2 = [p x2 , py2 , 0]T .

( p1 , v1 , ω 1 ) t = t1

y

(25)

Now, in order for the table tennis playing robot to return a ball to a desired destination in the opposite court, we will propose how to control the racket’s state (V, α, β) in real-time process. Note that the ball’s state just before striking, p0 , v0 , ω0 , are known. Suppose that the desired destination p2 and the landing time t2 are requested.

The landing point

( p2 , v 2 , ω 2 ) t = t2

Rebound ball’s motion .

By using the proposed method of racket control, i.e., the racket’s state (V, α, β), it can be achieved not only that p(t2 ) is equal to p2 but also that two elements of ω(t) = ω1 = ω2 are regulated as you like, say the y-element ωy (t) = ωy1 = ωy2 and the z-element ωz (t) = ωz1 = ωz2 are controlled at any desired values. In fact, first of all, by considering (2) (ADM) with the boundary conditions p(t1 ) = p1 = p0 , p(t2 ) = p2 (see Subsec. 3.2), we would use one of the well-known numerical methods of the TPBVP, e.g., the initial value methods, the finite difference methods, etc. [11] and also we can use a command “bvp4c” for the TPBVP in MATLAB. But those methods are not suitable for real-time process because they need a lot of computing time. Then we will propose a method of solving the TPBVP which gives an approximate solution in real-time process, i.e., with a reasonable computing time (< 30 [ms]). Instead of ADM described by (2), we propose to use a simple aerodynamics model (SAM)    p¨ x (t) = −D| p˙ x (t)| p˙ x (t),    p ¨ y (t) = −D| p˙ y (t)| p˙ y (t), (26)      p¨ z (t) = −g, where D = C D mρ S b is constant. Notice that (26) has a solution slightly different from the exact solutions of ADM described (2). But we can find out analytical solutions for SAM. Given p x (t1 ) = p x1 , py (t1 ) = py1 , pz (t1 ) = pz1 and p x (t2 ) = p x2 , py (t2 ) = py2 , pz (t2 ) = 0 where p x2 > p x1 and pz1 > 0 are assumed, it is straightforward to see that p x2 − p x1 py2 − py1 −pz1

= = =

ln(Dv x1 (t2 − t1 ) + 1), sgn(vy1 ) D1 ln(D|vy1 |(t2 − t1 ) + 1), (27) − 21 g(t2 − t1 )2 + vz1 (t2 − t1 ). 1 D

where p˙ x (t1 ) = v x1 , p˙ y (t1 ) = vy1 and p˙ z (t1 ) = vz1 . Therefore v x1 vy1 vz1

= = =

exp{D(p x2 −p x1 )}−1 , D(t2 −t1 ) exp{D|py2 −py1 |}−1 sgn(py2 − py1 ) , D(t2 −t1 ) pz1 1 2 g(t2 − t1 ) − t2 −t1 ,

(28)

where sgn(py2 − py1 ) = sgnvy1 is used. Second, v0 , v1 , ω0 and the desired ωy1 , ωz1 determine ω x1 as shown in the condition (10). Finally, we can obtain the racket’s state (V, α, β) as shown in Subsec. 3.1. All those processes take the computing time less than 30[ms]. Notice that in [8], associated with ADM (2), besides SAM (26), a simpler approximation model is also discussed where the drag and Magnus terms are ignored, but SAM usually gives more precise solutions. Therefore in this paper only SAM is focused on.

5. Numerical Simulations and Experimental Results In this section, it will be verified via numerical simulations and experimental results how well the racket’s state (V , α, β) obtained by the proposed method can work.

5

SICE JCMSI, Vol. 4, No. 1, January 2011

Numerical Simulations

1

z[m]

0

-0.5

0.5

1

1.5

2

2.5

1.5

2

2.5

1.5 1 0.5 0 0

0.5

1

x[m]

(a) ball trajectory 4 2 0 -2

Table 1

0

x[m]

y[m]

The ball’s state (p0 , v0 , ω0 ) just before the racket strikes the ball is set in three cases; p0 = [−0.15, 0.70, 0.25] [m] is common for the three cases, and v0 , ω0 are shown in Table 1(a). Furthermore, the desired (p x2 , py2 , ωy1 , ωz1 , t2 ) are also shown in Table 1(b). From the base coordinate system (see Fig.2), it is easy to see that the comming ball and the returning ball of the case 1) are in backspin, ones of the case 2) are in topspin, and ones of the case 3) are in the side spin, respectively. Note that the desired p x2 , py2 [m] and t2 [s] are set in common for three cases.

0.5

x[m]

5.1

Ball’s state and desired destination .

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

t[s]

1) 2) 3)

0.7 0.6

0

0.1

0.2

0.3

t[s] 1

z[m]

(a) Ball’s state just before struck by racket [v x0 , vy0 , vz0 ] [m/s] [ω x0 , ωy0 , ωz0 ] [rad/s] 1) [-2.5, 0, 0.1] [0, 150, 0] 2) [-4.5, 0, 0.3] [0, -150, 0] 3) [-2.75, -0.8, -0.5] [0, -50, 150]

y[m]

0.8

0.5 0

0

0.1

0.2

0.3

t[s]

(b) The desired (p x2 , py2 , ωy1 , ωz1 , t2 ) [p x2 , py2 ] [m] [ωy1 , ωz1 ] [rad/s] t2 [s] [2.055, 0.768] [-100, 0] 0.6 [2.055, 0.768] [100, 0] 0.6 [2.055, 0.768] [0, -100] 0.6

(b) ball time-history Fig. 5

The case 1) of backspin .

1

z[m]

0

-0.5

0

0.5

1

1.5

2

2.5

1.5

2

2.5

x[m] 1.5

y[m]

Associated with the three cases shown in Table 1, the proposed method provides the racket’s state as shown in Table 2; “the proposed” means the racket’s state which is obtained by the proposed method, i.e., using SAM, and “the true” means the racket’s state which is obtained by the command “bvp4c” in MATLAB and achieves the desired (p x2 , py2 , ωy1 , ωz1 , t2 ) exactly.

0.5

1 0.5 0 0

0.5

1

x[m]

(a) ball trajectory Racket’ state via the proposed method and the true state . β [rad] 0.792 0.811 1.66 1.63 1.25 1.24

x[m]

α [rad] -0.011 -0.013 -0.012 -0.014 0.31 0.28

2 0 -2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

t[s] 0.8

y[m]

1) the proposed the true 2) the proposed the true 3) the proposed the true

V [m/s] [1.48, 0.095, 1.41] [1.51, 0.11, 1.34] [0.65, 0.064, 1.72] [0.67, 0.075, 1.87] [1.48, 0.23, 1.30] [1.50, 0.38, 1.33]

4

0.7 0.6

0

0.1

0.2

0.3

t[s] 1

z[m]

Table 2

0.5 0

0

0.1

0.2

0.3

t[s]

Table 3 shows results of p x2 , py2 , ωy1 , ωz1 , t2 when the racket’s states via the proposed method are used. Table 3 1) 2) 3)

(b) ball time-history Fig. 6

The case 2) of topspin .

The obtained (p x2 , py2 , ωy1 , ωz1 , t2 ) .

[p x2 , py2 ] [m] [2.09, 0.76] [1.92, 0.78] [1.98, 0.66]

[ωy1 , ωz1 ] [rad/s] [-100, 0] [100, 0] [0, -100]

t2 [s] 0.620 0.561 0.582

Comparing Table 3 with Table 1(b), the average errors on p x2 , py2 and t2 are 0.082[m], 0.045[m] and 0.025[s], respectively. In addition, it can be seen that ωy1 and ωz1 can be achieved exactly. Those results are due to that RRM described (1) is exactly solved, but ADM described (2) is approximated as SAM described by (26) to be solved in real-time process. Moreover, the returning ball’s trajectory and time-history are shown in Figs. 5-7 for the cases 1)-3), respectively. In each figure, the solid line shows the result obtained by the proposed method, and the dash line shows the result which achieves the desired destination exactly.

5.2 Experimental Results 5.2.1 Experimental Settings In the experiment, a table tennis ball is shot from a ball catapult machine. The ball’s position, translational velocity and rotational velocity just after being shot by the catapult machine are measured by using two high speed monochrome cameras (Hamamatsu Photonics) with the frame rate 900[Hz] and the on-line measurement [5]. Then ADM described by (2) and the table rebound model [7] with the measured ball’s position and velocity are used to estimate the ball’s trajectory to determine the striking position p0 at which a robot strikes and returns the ball with the racket. The striking position p0 = [p x0 , py0 , pz0 ] is determined by using the ball’s estimated trajectory under the condition of the smallest z-axis absolute velocity |vz0 | in the range of -0.3[m]≤ p x0 ≤ 0. The robot is a robot manipulator with 7 DOF (PA10, Mit-

6

SICE JCMSI, Vol. 4, No. 1, January 2011

q˙6¯ = J¯3¯† (q)V

z[m]

1

0.5

J¯3¯†

= (J¯3¯ W ) (J¯3¯ W −1 J¯3¯T )−1 . W −1 = diag {1, 0.6, in which 2, 2, 2π, 2π} is a weight matrix with respective to the joints’ limit velocity.

0

-0.5

0

0.5

1

1.5

2

(32) −1 T

2.5

x[m]

y[m]

1.5

5.2.2 Some Results Here some experimental results are shown in the cases that the catapult machine shot a backspin ball and a topspin ball. The striking position p0 = p1 and the ball’s velocities v0 , ω0 were estimated on-line by using the high speed cameras’ data as in Table 4 (a). The desired destination p2 in the opposite court, the landing time t2 and the desired rotational velocities ωy1 , ωz1 are set as in Table 4 (b). Then the racket’s state obtained by the proposed method is shown in Table 4 (c).

1 0.5 0 0

0.5

1

1.5

2

2.5

x[m]

(a) ball trajectory x[m]

4 2 0 -2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

t[s] y[m]

0.8 0.7 0.6

0

0.1

0.2

0.3

Table 4

t[s]

Experimental results .

z[m]

1

(a) Ball’s state just before struck by racket back spin top spin [p x0 , py0 , pz0 ][m] [-0.30, 0.66, 0.23] [-0.14, 0.73, 0.26] [v x0 , vy0 , vz0 ] [m/s] [-2.97, -0.0077, 0.34] [-4.17, -0.12, -0.0023] [ω x0 , ωy0 , ωz0 ] [rad/s] [-28.02, 229.76, -28.58] [7.53, -241.09, -6.28]

0.5 0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

t[s]

(b) ball time-history Fig. 7

The case 3) of sidespin .

subishi H.I.) and the racket is attached on the robot’s end effector. The method of controlling the posture and velocity of the racket at the striking position is shown in Sec. 4. Now we will show how to control the robot’s joints’ angles at the striking position. Let q ∈ R7 be the vector of joint angles with its i-th element qi being the i-th joint angle. Then the kinematic equation is given as [pT , θ T ]T = F (q) where p ∈ R3 is a position of the racket’s center and θ = [α, β, γ]T ∈ R3 is the racket’s posture. Recall that the references of p, α and β are given as shown in Sec. 4, but γ is free. Therefore the above kinematic equation is reduced to [ ] p = F5¯ (q), (29) θ2¯ where θ2¯ = [α, β]T ∈ R2 and F5¯ (q) = [I5 05×1 ]F (q). Notice that given [pT , θ2T¯ ] ∈ R5 , the equation (29) is a set on nonlinear equations to q ∈ R7 and so it could have infinite number of solutions if there exists a solution q. Here, in order to obtain a solution q of (29), the Newton-Raphson method is used, i.e., ( ) q (k+1) = q (k) + J5¯† (q (k) ) [pT , θ2T¯ ]T − F5¯ (q (k) ) , (30) ∂F

where J5†¯ is a pseudo inverse of the Jacobian J5¯ , i.e., J5¯ = ∂q5¯ ∈ )−1 ( ∈ R7×5 . The iterative process (30) R5×7 and J5†¯ = J5T¯ J5¯ J5T¯ starts with q (0) which is a waiting position for the comming ball and the iterative process stops when the stop condition is satisfied, i.e., ∥[pT , θ2T¯ ]T −F5¯ (q (k) )∥ ≤ e where e > 0 is a preassigned small number. Now we will show how to achieve the racket’s velocity V . Again the kinematic equation [pT , θ T ]T = F (q) is reduced as p = F3¯ (q) where F3¯ (q) = [I3 03×3 ]F (q). Therefore it is ∂F derived directly that V = J3¯ (q)q˙ where J3¯ = ∂q3¯ ∈ R3×7 . Then q˙ 7 = 0 is always selected for simplicity, so by introducing q˙6¯ = [I6 06×1 ]q˙ ∈ R6 , the above equation is reduced as V = J¯3¯ (q)q˙6¯ ,

(31)

where J¯3¯ = J3¯ [I6 06×1 ] ∈ R mined by T

3×6

. Therefore, q˙6¯ can be deter-

(b) The desired (p x2 , py2 , ωy1 , ωz1 , t2 ) [p x2 , py2 ] [m] [ωy1 , ωz1 ] [rad/s] back spin [2.055, 0.768] [-125.66, 0] top spin [2.055, 0.768] [104.72, 0]

t2 [s] 0.6 0.5

(c) Racket’ state via the proposed method V [m/s] α [rad] β [rad] back spin [1.26, -0.18, 1.75] -0.068 0.708 top spin [1.15, 0.019, 0.64] -0.028 1.814

In order to evaluate how well the hitting position is estimated and how well the ball is controlled by the racket to return to the desired destination, two middle speed colored cameras (Library Inc.) are used, whose frame rate is 150 [Hz]. Figure 8 shows the ball’s trajectories of x-z plane and x-y plane in the case of back spin ball. The markers “+” show the comming ball’s trajectory sensed by the middle speed camera system. The circles “o” show the returning ball’s trajectory sensed by the middle speed camera system, and the solid curve is a smooth curve approximating the circles. Figure 9 shows the time-history of the returning ball’s position, where the circles “o” are data sensed by the middle speed camera system and the solid curve is a smooth curve derived from the data. The horizontal solid line is the desired landing position. From Figs. 8 and 9, it can be evaluated that the achieved destination p2 in the opposite court is [p x2 , py2 ] = [2.10, 0.78] [m] and the landing time t2 is t2 = 0.622 [s]. In the case of top spin ball, Fig. 10 shows the ball’s trajectories of x-z plane and x-y plane and Fig. 11 shows the time-history of the ball where “+”, “o”, the solid curve and the horizontal solid line express the same things as in Figs. 8 and 9. From those figures, in the case of top spin ball, it can be evaluated that the achieved destination p2 in the opposite court is [p x2 , py2 ] = [1.90, 0.744] [m] and the landing time is t2 = 0.451 [s]. The experimental results above have almost same features as observed in the numerical simulations, i.e., the achieved p x2 and t2 are larger than the desired ones in the case of back spin ball, while the achieved p x2 and t2 are smaller than the desired ones in the case of top spin ball.

7

SICE JCMSI, Vol. 4, No. 1, January 2011 2.5

0.6 0.5

2

Racket x-axis[m]

z-axis[m]

0.4 0.3 0.2 0.1

1 0.5 0

Table

0

1.5

-0.5 0

-0.1 -0.5

0

0.5

1 x-axis[m]

1.5

2

0.1

0.2

0.3

0.4

0.5

0.3

0.4

0.5

0.3

0.4

0.5

time[s]

2.5 0.8

(a) x-z plane y-axis[m]

0.75

0.8

y-axis[m]

0.7 0.65

0.75

0.6 0

0.7

0.1

0.2

time[s] 0.6

0.65 Racket

0.5

0

0.5

1 x-axis[m]

1.5

2

z-axis[m]

0.4

0.6 -0.5

2.5

(b) x-y plane

0.3 0.2 0.1 0

Fig. 8

-0.1 0

Ball trajectory (Experiment): the case of backspin ball .

0.2

time[s]

Fig. 11

2.5

0.1

Ball time-history (Experiment): the case of topspin ball .

x-axis[m]

2 1.5

where the horizontal solid lines show qre f and q˙re f and the dots show q(t) and q(t) ˙ which is calculated by q(t+h)−q(t) with a samh pling period h. The vertical solid line shows the striking instant, i.e., t = 0. Figure 13 also shows the time-histories of the angles and velocities of the joints #1, #3 and #5 in the case of top spin ball. From both figures, it can be seen that q(0) and q(0) ˙ have been controlled well.

1 0.5 0 -0.5 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.4

0.5

0.6

0.7

time[s] 0.8

y-axis[m]

0.75 0.7 0.65 0.6 0

0.1

0.2

0.3

time[s] -3

80

0.6

0.3 0.2

Velocity of joint 1 [deg/s]

Joint 1 [deg]

0.4

z-axis[m]

70

-3.5

0.5

-4 -4.5 -5

0.1 -5.5

0 -0.1 0

0.1

0.2

0.3

0.4

0.5

0.6

-6 -0.1

0.7

time[s]

Fig. 9

0 time[s]

0.05

0.3 0.2

120

20

100

19.5 19 18.5

0.1 18

Table 0.5

1 x-axis[m]

1.5

2

17.5 -0.1

2.5

-0.05

0 time[s]

0.05

3 2.5 2 1.5

0.65

1

1.5

2

0.5 -0.1

2.5

60 40 20 0 -0.05

0 time[s]

0.05

0.1

Ball trajectory (Experiment): the case of topspin ball .

Associated with the requested racket’s state (V , α, β), the requested joint angle and velocity of the robot manipulator at the striking instant, qre f and q˙re f , are determined as shown in (30) and (32). It is not a concern in this paper how to control the manipulator to achieve q(0) = qre f and q(0) ˙ = q˙re f , but it must be shown how near the attained q(0) and q(0) ˙ are to the requested qre f and q˙re f . Figure 12 shows the time-histories of the angles and velocities of the joints #1, #3 and #5 in the case of back spin ball,

-50 -100 -150 -200 -250 -300

-0.05

0 time[s]

0.05

0.1

-350 -0.1

-0.05

0 time[s]

0.05

0.1

(c) The time-histories of q5 (left) and q˙ 5 (right)

(b) x-y plane Fig. 10

0.1

0 Velocity of joint 5 [deg/s]

Joint 5 [deg]

y-axis[m]

Racket 0.7

1 x-axis[m]

0.05

50

4 3.5

0.5

0 time[s]

80

-20 -0.1

0.1

4.5

0.8 0.75

0

-0.05

(b) The time-histories of q3 (left) and q˙ 3 (right)

(a) x-z plane

0.6 -0.5

20

0 -0.1

0.1

Velocity of joint 3 [deg/s]

Joint 3 [deg]

z-axis[m]

Racket

0

30

20.5

0.5

-0.1 -0.5

40

10 -0.05

0.6

0

50

(a) The time-histories of q1 (left) and q˙ 1 (right)

Ball time-history (Experiment): the case of backspin ball .

0.4

60

Fig. 12

The joints’ time-histories in the case of back spin ball .

6.

Conclusion

This paper has considered the method for a table tennis robot to control the racket’s state at the striking point; the translational velocity and the posture of the yaw and pitch angles, which can return a spinning ball to a desired position on the opposite court with desired translational and rotational velocities at a desired landing time. The problem consists of two subproblems; a set of nonlin-

8

SICE JCMSI, Vol. 4, No. 1, January 2011 26

50 Velocity of joint 1 [deg/s]

25.8

Joint 1 [deg]

25.6 25.4 25.2 25 24.8 24.6 -0.1

-0.05

0 time[s]

0.05

40 30 20 10 0 -0.1

0.1

-0.05

0 time[s]

0.05

0.1

-1

120

-1.5

100 Velocity of joint 3 [deg/s]

Joint 3 [deg]

(a) The time-histories of q1 (left) and q˙ 1 (right)

-2 -2.5 -3 -3.5 -4 -0.1

-0.05

0 time[s]

0.05

80 60 40 20 0 -20 -0.1

0.1

-0.05

0 time[s]

0.05

0.1

(b) The time-histories of q3 (left) and q˙ 3 (right) 30.5

100 Velocity of joint 5 [deg/s]

30

Joint 5 [deg]

29.5 29 28.5 28 27.5 27 -0.1

-0.05

0 time[s]

0.05

0.1

0 -100 -200 -300 -400 -0.1

-0.05

0 time[s]

0.05

0.1

(c) The time-histories of q5 (left) and q˙ 5 (right) Fig. 13

The joints’ time-histories in the case of top spin ball .

ear equations which comes from the racket rebound model (RRM) and a two-point boundary value problem of a nonlinear differential equation which comes from the aerodynamics model (ADM). The paper clarified the fundamental properties of those subploblems and then derived the method of solving the problem, but the two-point boundary value problem of ADM needs too much computing time to be treated in real-time manner. Therefore the paper proposed the real-time algorithm by using an approximate aerodynamic model (SAM). Some numerical simulations and experimental results have verified effectiveness of the proposed algorithms. Notice that the method proposed in this paper does not belong to a class of visual feedback control systems. In fact, the high speed cameras (900Hz) have narrow fields so that the cameras can observe less than 10 image frames just after the ball is shot from the catapult machine. After the cameras estimate the ball’s translational and rotational velocities at this instant, it usually takes about 600[msec] and more until the manipulator with racket strikes the coming ball. During this interval, the proposed method controls the manipulator based on the physical models without any visual feedback. From this sense, unlike the visual feedback control with high speed cameras [12], the proposed method is not robust against the physical models’ errors. One of future researches is to integrate the proposed method with the visual feedback control.

2010. [4] C. Liu, Y. Hayakawa, and A. Nakashima: A registration algorithm for on-line measuring the rotational velocity of a table tennis ball, Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, pp. 2270-2275, 2011. [5] C. Liu, Y. Hayakawa, and A. Nakashima: An on-line algorithm for measuring the translational and rotational velocities of a table tennis ball, SICE Journal of Control, Measurement, and System Integration, Vol.5, No.4, pp. 233-241, 2012. [6] J. Nonomura, A. Nakashima, and Y. Hayakawa: Analysis of effects of rebounds and aerodynamics for trajectory of table tennis ball, Proceedings of SICE Annual Conference 2010, pp. 1567-1572, 2010. [7] A. Nakashima, Y. Ogawa, Y. Kobayashi, and Y. Hayakawa: Modeling of rebound phenomenon of a rigid ball with friction and elastic effects, Proceedings of the 2010 American Control Conference, pp. 1410-1415, 2010. [8] C. Liu, Y. Hayakawa, and A. Nakashima: Racket control for robot playing table tennis ball, Proceedings of the 12th International Conference on Control, Automation and Systems, pp. 1427-1432, 2012. [9] A. Nakashima, Y. Ogawa, C. Liu, and Y. Hayakawa: Robotic table tennis based on physical models of aerodynamics and rebounds, Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, pp. 2348-2354, 2011. [10] S. R. Bernfeld and V. Lakshmikantham: An Introduction to Nonlinear Boundary Value Problems, Chapter 1, Academic Press, Inc., 1974. [11] U. M. Ascher, R. M. M. Mattheij, and R. D. Russell: Numerical Solution of Boundary Value Problems for Ordinary Differential Equations, SIAM, 1995. [12] T. Senoo, A. Namiki, and M. Ishikawa: Hybrid trajectory generation of an articulated manipulator for high-speed batting, Journal of the Robotics Society of Japan, Vol. 24, No. 4, pp. 515-522, 2006 (in Japanese).

Chunfang LIU (Student Member) She received her M.S. degree from Jilin University, China, in 2008. She is currently a Ph.D. student at Nagoya University. Her research interests include computer vision and robot control.

Yoshikazu HAYAKAWA (Member, Fellow) He received the B.E. degree in mechanical engineering, and the M.E. and Dr. Eng. degrees in information and control science, all from Nagoya University, Nagoya, Japan, in 1974, 1976, and 1982, respectively. Since 1979, he has been with the Faculty of Engineering, Nagoya University, where he is presently a Professor of the Dept. of Mechanical Science and Engineering. His research interests are in system and control theory, robot control, and vibration control. He is a member of JSME, IEEE, etc.

Akira NAKASHIMA (Member) References [1] M. Matsushima, T. Hashimoto, M. Takeuchi, and F. Miyazaki: A learning approach to robotic table tennis, IEEE Transactions on Robotics, Vol. 21, No. 4, pp. 767-771, 2005. [2] L. Acosta, J. J. Rodrigo, J. A. Mendez, G. N. Marichal, and M. Sigut: Ping-pong player prototype, IEEE Robotics & Automation Magazine, Vol. 10, No. 4, pp. 44-52, 2003. [3] P. Yang, D. Xu, H. Wang, and J. Zhang: Design and motion control of a ping pong robot, Proceedings of the 8th World Congress on Intelligent Control and Automation, pp. 102-107,

He received the B.E., M.E. and Ph.D. degrees in Engineering from Nagoya University in 2000, 2002 and 2005, respectively. He is presently an Assistant Professor at the Dept. of Mechanical Science and Engineering, Nagoya University. His research interests are multi-fingered robot hands, visual servo control and nonholonomic mechanics. He is a member of RSJ and IEEE.