A hybrid dynamic motion prediction method with collision detection

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection

A hybrid dynamic motion prediction method with collision detection I. PASCIUTO*†, A. VALERO†, S. AUSEJO† and J.T. CELIGÜETA† † CEIT and Tecnun (University of Navarra), San Sebastian, Spain

Abstract In this paper we present a hybrid dynamic motion prediction method which combines data-based with knowledge-based methods. On the one hand it relies on a database of captured motions for reference while on the other it includes knowledge in the prediction as a dynamic motion control law. The motion prediction is carried out imposing the conditions that the predicted motion must fulfill the goals defined for the motion; resemble the reference motion in the database; follow a motion control law; and handle collisions with the environment. Dynamics are included in the formulation both in the motion control law to be followed and in the constraints the motion is subject to. The method is applied to the prediction of an automobile clutch pedal depression and is validated against repetitions of real motions performed by subjects of the same anthropometric characteristics as the subject used in the prediction. Keywords: Biomechanics, Posture and motion, Human motion prediction.

1. Introduction Digital Human Models (DHMs) are more and more frequently employed in product development processes to test virtual environments in the early stages of the design (Monnier et al. 2006; Monnier et al. 2008; Abdel Malek and Arora 2008). To simulate the interaction of various DHMs, representing different user populations, with a variety of environments, human motion prediction is an interesting and useful tool. Motion prediction aims at predicting within a reasonable accuracy the motion that a subject would perform to accomplish a specific goal. In contrast with animation (Badler et al. 1993), which essentially guides a particular subject with a specific user-defined motion, motion prediction focuses on representing the motions that a generic specimen of a population would perform. Due to the redundancy of degrees of freedom (DoFs) in the human body, generally there are infinite sets of values of the DoFs which fulfill a specific task. Of all these sets of values, only some can be defined as realistic, and may be grouped into the various strategies and styles adopted by subjects while carrying out the task. The challenge for motion prediction is to simulate motions that not only adjust to the requirements for realism but also represent the variability observed in real motions.

*Corresponding author. Email: [email protected]

Motion prediction methods can be classified into data-based and knowledge-based methods, according to whether they rely or not on a database of captured motions. Data-based methods (Zhang and Chaffin 2000; Wang 2002; Monnier et al. 2003; Park et al. 2004; Park 2008) consist in obtaining, among the motions which compose the database, the most appropriate one to be taken as reference and in modifying it in order to fulfill the new motion constraints. The motion in the new scenario (a new subject in a new environment) is predicted starting from a real motion carried out in a reference scenario (a reference subject in a reference environment). The main advantage of data-based methods lies in the intrinsic realism of the reference motion, which should be maintained during the modification process. Furthermore, a database of captured motions allows to identify the various strategies and styles (Monnier et al. 2003; Park 2008) which the subjects adopt during the fulfillment of the considered task. Each identified strategy and style may then be predicted, hence representing the variety of ways in which the generic specimen of the population may carry out the task. The main drawback of data-based methods is the restriction of being able to reasonably predict only tasks which are present in the database. To predict a different task, first a corresponding database of motions must be generated.

1

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection On the other hand, knowledge-based methods (Kim et al. 2006; Chung et al. 2007; Ren et al. 2007; Abdel Malek and Arora 2008; Xiang et al. 2009) do not rely on a database of captured motions and must confer realism to the motion through the creation of an appropriate objective function, representing the motor control law that drives the motion. Although knowledge-based methods are theoretically applicable to any task to be performed, their drawback lies in the difficulty of identifying the correct objective function to describe the motion control law. Even simple reaching motions seem to require the combination of several objective functions (Yang et al. 2004), leaving the prediction of complex motions to be more than a challenge at the present time. For specific task-driven motions, as those involved in the virtual testing of products, data-based motion prediction methods may be considered the most suitable option. Current data-based methods are generally kinematic. However, for certain applications, such as ergonomic studies, a dynamic prediction may be required. In this paper we present a hybrid dynamic motion prediction method, which both relies on a database of captured motions and includes knowledge in the prediction as a dynamic motion control law. In addition, collisions between the DHM and the virtual environment are detected and modeled as contact forces. 2. Method The hybrid dynamic motion prediction method proposed in this paper is optimization-based and is composed of the following steps: human model definition; database generation; reference motion selection; end-effector trajectory modification; resolution of the optimization problem describing the conditions imposed to the predicted motion. Each of these steps is hereafter described in detail. 2.1. Human model definition To perform a human motion prediction, a DHM is required. In this paper we consider a multibody model of the human body, described with the formalism of relative coordinates. 2.2. Database generation Our prediction method relies on a structured database of motions among which the reference motion is identified. The database is generated reconstructing captured motions; organizing them according to the task, strategies, styles and scenario characteristics; and analyzing them to identify the relevant features of the motion. An optoelectronic system is used to capture the motions’ kinematics and the objects of the environment with which the subjects interact are equipped with force and pressure sensors to allow a

dynamic motion reconstruction through Inverse Dynamics. Once the motions are reconstructed, key-frames of the motions are identified. A key-frame is a frame in which a key event occurs in the motion. Keyframe identification is paramount in motion prediction to define the goals that characterize the key events in the motion and to determine at which frames they must occur in the predicted motion. In this paper we consider that key-frames in the reference motion are maintained in the predicted motion, i.e. key events occur at the same instant in time. The criteria for key-frame identification encountered in literature are used for kinematic motion prediction (Bindiganavale and Badler 1998; Monnier et al. 2008), and only consider the kinematic magnitudes of the motion, for instance zero-velocity conditions for the end-effector. In dynamic motion prediction, the key-frames identified with kinematic criteria still represent key events of the motion, but other key events may need to be identified analyzing dynamic magnitudes. Regarding dynamics-related key-frames, we considered the force sensors’ values to identify the frames at which the contact between the subject and the environment occurs. The frames corresponding to the first and last non-zero force values are the key-frames corresponding to the beginning and the end of the contact. 2.3. Reference motion selection To predict the motion of a subject performing a specific task in a given environment, the reference motion must first be selected from the database. The criterion adopted in this paper is based on the comparison between the prediction scenario and the scenarios of the motions that form the database for the considered task. Among these motions, the one that most resembles the prediction scenario is retrieved. The parameters that are used to carry out the comparison are the subject’s gender, age and anthropometrics; the position of the objects in the environment with which the subject interacts as well as their geometric and dynamic characteristics. 2.4. End-effector trajectory modification Generally, the trajectory followed by the segment in charge of fulfilling the task (usually an endeffector) in the reference motion does not lead to the task’s accomplishment in the predicted scenario. To guide the motion of the end-effector and ensure the fulfillment of the task, a new trajectory for the end-effector must be defined. A way of obtaining an end-effector trajectory for the predicted motion is to modify the end-effector’s reference trajectory in order to meet the targets in the prediction scenario.

2

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection Two modification methods are proposed in the literature (Zhang 2002): one imposes that the velocity along the predicted trajectory be proportional to that of the reference motion (VP, velocity proportional); the other imposes that the acceleration along the predicted trajectory be the same as in the reference motion (AP, acceleration preserving). Generally both methods yield very similar results, nonetheless VP is favored in this study as it presents the desirable feature of maintaining zero-velocity conditions at key-frames. On the other hand, VP fails to generate a reasonable trajectory when the initial and final positions of the end-effector are very close to each other, since numerical problems arise. In this case AP is applied instead. The end-effector trajectory modification must be applied in each time period, which is the lapse of time between two consecutive key-frames. Instead, if the entire trajectory across a time period is specified, we apply a different method. The method imposes that the distance run by the end-effector between two frames holds the same proportion respect to the whole distance to be run in the time period as in the reference motion. 2.5. Optimization As mentioned in the introduction, the human body is a highly redundant system, presenting more DoFs than those strictly necessary to operate in the workspace. For this reason, the imposition of the end-effector trajectory is not enough to define the motion of all the body segments, since infinite sets of values of the DoFs allow its fulfillment. Optimization may be used to define a criterion that identifies the most realistic set of DoFs as the set which minimizes a certain objective function subject to a list of constraints. As opposed to the optimization problems encountered in data-based motion prediction methods (Monnier 2003; Park et al. 2004) which carry out an optimization for each frame in the motion, we consider the motion as a whole and search the optimum across the entire motion. Instead of using the DoFs in each frame as design variables, as in (Popović and Witkin 1999), we chose to obtain a B-Spline representation of the DoFs’ profiles. B-Spline curves are a linear combination of linearly independent piecewise polynomials, named basis functions, through coefficients named control points. The basis functions are a function of the independent variable (in our case: time) and are non-zero only in limited intervals in time. This leads to the property of local support for B-Splines: the value of each control point affects the curve in a lapse of time, which generally is only a portion of the whole duration. On the other hand, control points are defined in the space of the dependent variables (in our case: DoFs).

A B-Spline representation of a curve C(t) is given by: =



(1)

Ni p are the p-order basis functions, each associated to a control point, CPi. Our method consists in obtaining a B-Spline representation of the ndof profiles of the DoFs in the reference motion and in supposing that the same number of control points is suitable to describe also the DoFs’ profiles in the predicted motion. We consider this supposition reasonable due to the similarity criterion used to select the reference motion from the database. Given the set of ndof profiles which represent the reference motion, we encounter the minimum number nCP of control points needed to ensure that each of the ndof B-Splines approximates the reference DoFs’ profiles within a specified tolerance (Piegl and Tiller 1997). The obtained set of control points constitutes the initial approximation for the optimization problem. The advantage of a B-Spline parameterization of the motion is that the whole motion is described by a relatively small number of control points, which are the design variables of the optimization problem hereafter described. The solution of optimization problem yields the values of the control points which minimize an objective function subject to constraints. The objective function is in charge of maintaining a resemblance to the reference motion and in defining a motion control law to be followed. On the other hand, the constraints we impose can be classified into task-related constraints and balance constraints. The former ensure that the goals in the motion are fulfilled. The latter enforce the dynamic equilibrium of the multibody model. The objective function is defined as: 1 (2) = Ψ Ψ 2 The vector Ψ is composed of equations representing the following conditions: • Resemble the DoFs’ profiles in the reference motion: (3) − ∀ • Resemble the end-effector trajectory obtained through the previously described modification method: (4) − • Follow the motion control law defined as resembling the joint power profiles in the reference motion: (5) ∀ − The motion control law described by Eq. (5) was chosen among several motion control laws encountered in literature (Yang et al. 2004; Kim et al. 2006; Ren et al. 2007; Abdel-Malek and Arora

3

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection 2008) since, according to (Pasciuto et al. 2010), it seems to be the one which yields the most realistic results. The constraints the minimization is subject to are: • Task-related: match the end-effector trajectory in key-frames and when its trajectory is defined across a time period: (6) − = 0 • Balance: ensure the dynamic equilibrium of the multibody model by imposing balance conditions between the forces in the root segment and the external forces due to its interaction with the environment: = 0 = 0

(7)

The dynamic magnitudes, used in the definition of both the objective function (Eq. 5) and the constraints (Eq. 7), are evaluated through the application of the Newton-Euler recursive method (Craig 2005). The method performs two iterations across the system: the first starts from the root segment and, moving outwards, it obtains the segments kinematics; the second starts from the distal segments and, moving inwards, it obtains the forces and moments in the joints by imposing the dynamic equilibrium of each segment. To evaluate the external forces acting on the segments, the collisions of the DHM with the environment are detected and modeled. Collisions are detected according to the position of the DHM’s segments respect to the position of the objects with which collisions are expected. If a collision takes place, the collision is modeled as a force dependent on the value of the displacement and on the velocity of the segment. Both the displacement and the segment velocity are a function of the control points representing the DoFs’ profiles. 2.6. Test case The method described in the previous section has been applied to a clutch-pedal depression motion. The human multibody model used to represent the subjects performing the motion is that of a left leg, modeled following RAMSIS specifications (Human Solution GmbH). The model is composed of 4 segments (pelvis, thigh, shank, foot) and presents 13 DoFs: 3 translational and 10 rotational, distributed across the joints as shown in Fig. 1. The upper body of the DHM is taken into account by conferring its inertial properties to the pelvis segment.

Figure 1: RAMSIS model of a left leg with 3 translational DoFs and 10 rotational DoFs.

A database of captured clutch-pedal operation motions is available from the DHErgo project (www.dhergo.org). The motions were carried out by subjects belonging to 4 different populations: young males and females (ages 20-35); elderly males and females (ages 65-80). The subjects in each population, approximately belonging to the 50th percentile, differ less than 10cm in height and 6kg/m2 in BMI. Using our method we have predicted the motion of a new subject in a modified environment. The new scenario actually matches an experimental configuration in which three motion repetitions were captured. These repetitions are used for validation. The characteristics of the new and reference scenarios are detailed in Table 1. Table 1: Characteristics of the reference and new scenarios.

Subject

Vehicle*

Gender Age Height Weight Pedal rest pos. x y z Travel length Travel angle

Reference Female 30 1.68m 59kg -0.761m -0.073m -0.195m 0.161m 23º

New Female 22 1.63m 63kg -0.758m -0.074m -0.096m 0.158m 8º

*see Fig.2 The end-effector trajectory is imposed as a constraint in the first frame and from the pedal depression key-frame to the end depression keyframe. In all other frames it is included in the objective function. The force due to the clutch-pedal depression is modeled as a function of the displacement (Fig. 3), based on the pedal’s force vs. displacement curve, which corresponds to a real clutch-pedal.

4

End Effector Trajectory X Axis [m]

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection

-0.7 Reference Modified Predicted

-0.75

-0.8

-0.85

-0.9 0

20

40

60

80

100

Time [%]

Figure 2: Experimental car mock-up.

Figure 4: End effector trajectory along the x axis: reference, modified and predicted trajectories. End Effector Trajectory X Axis [m]

120

100

60

40

20

0.02

0.04

0.06

0.08

0.1

-0.9 0

20

40

Figure 3: Force-displacement curve for clutch-pedal.

The displacement depends on the position of the foot, which depends on the DoFs of the model. On the other hand, the force due to the interaction with the seat is modeled as a function of the penetration of the pelvis in the seat and its velocity. Due to the local support property of B-Splines, both the constraints and the optimization function are evaluated only in specific frames in the motion. Denoting with nFrames the number of frames in the motion to be predicted and with nCP the number of control points of the B-Spline representation, the frames included in the optimization problem are a sample every nFrames/nCP frames. Additionally, all key-frames are included as well. 3. Results The results obtained by applying the described hybrid dynamic motion prediction method to the test case are shown in the following figures. All figures represent the values of the corresponding magnitudes obtained with the motion prediction (in red), the values in the reference motion (in blue), and the values in the motions used for validation (in black). The time axis is normalized so that 50% of the motion corresponds to the key-frame in which the pedal depression begins and 100% corresponds to the end depression key-frame.

60

80

100

Time [%]

0.12

Displacement (m)

Figure 5: End effector trajectory along the x axis: predicted and validation trajectories. 90

GKNL Flex-Ext Angle [deg]

0

-0.85

Reference Predicted Validation1 Validation2 Validation3

80 70 60 50 40 0

20

40

60

80

100

Time [%]

Figure 6: Knee flexion-extension joint angle profile. GHUL Flex-Ext Ang Vel [deg/s]

0

-0.8

60

Reference Predicted Validation1 Validation2 Validation3

40 20 0 -20 -40 0

20

40

60

80

100

Time [%]

Figure 7: Hip flexion-extension angular velocity profile. 150 GHUL Flex- Ext Torque [Nm]

Force (N)

80

Predicted Validation1 Validation2 Validation3

-0.75

Reference Predicted Validation1 Validation2 Validation3

100 50 0 -50 -100 0

20

40

60

80

100

Time [%]

Figure 8: Hip flexion-extension torque profile.

5

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection maximum error across the motion is limited to less than 10 N.

Force in Pelvis X Axis

150 100 50

Error Child Inertia External

5. Conclusions

0 -50 -100 -150 0

20

40

60

80

100

Time [%]

Figure 9: Forces acting on the pelvis along the x axis.

4. Discussion Fig.4 and Fig.5 show the trajectories followed by the end-effector along the x axis. The curve in green represents the modified trajectory which the end-effector must resemble (in the first part of the motion) or follow exactly (in the second part). This trajectory was obtained with the described endeffector trajectory modification method. The predicted motion (in red) matches the pedal trajectory in the second half of the motion, as imposed by the constraint. In the first half it must resemble the modified trajectory while other conditions must be met as well, hence the resemblance is slighter. Fig.5 compares the predicted trajectory to those followed in the validation motions, and the values obtained are quite similar. The differences in the validation trajectories are partially due to the fact that different points in the foot were used by the validation subject to depress the pedal. Fig.6 shows the knee flexion-extension joint angle profile. The predicted motion resembles the shape of the reference profile and the values obtained are generally similar to those in the validation motions. However in the predicted motion the knee seems to flex more than in actually performed motions before reaching the pedal (approximately at 30%). Fig.7 and Fig.8 show the angular velocity and the flexion-extension torque profiles at the hip joint. In both, the predicted profiles seem to follow the natural oscillations encountered in actually performed motions and the predicted values are always contained in the range of values of the real motions. The forces acting on the pelvis along the x axis are shown in Fig.9. The oscillations in the first part of the motion are due to the inertia forces of the segment and of the child: these oscillations are balanced by the seat. The second part of the motion is characterized by the force acting on the hip joint, mainly due to the pedal reaction force during depression. The magenta curve shows the sum of all the forces acting on the pelvis, i.e. the error of Eq. (9). The frames at which the equilibrium constraint was imposed present a strongly reduced error ((o) 10-4 N), but at intermediate frames the constraint is not met exactly. However the

In this paper we presented a hybrid dynamic motion prediction method, which detects and models the collisions between the DHM and the environment. Dynamics are included both in the definition of the objective function as well as in the constraints the motion is subject to. This hybrid method relies on a database of captured motions as well as on the definition of a motion control law and is able to predict the motion of a new subject performing a task in a new environment. The results we obtained do not differ substantially from actually performed motions: on the one hand, the ranges of values of predicted magnitudes are similar to those encountered in real motions; on the other, the oscillations in values that characterize real motions are present in the predicted motion as well. Acknowledgement The authors would like to thank INRETS for the database of captured motions used in the present study and Human Solutions GmbH for providing the RAMSIS specifications to define the DHM. This research was funded by the European project DHErgo, “Digital Humans for Ergonomic design of products” (FP7/2007-2013), grant agreement nº 218525. References Abdel-Malek K, Arora, J, 2008. Physics-based digital human modeling: predictive dynamics. In: Duffy,V.G. (Eds.), Handbook of Digital Human Modeling. CRC Press- Taylor & Francis Group, Boca Raton, Florida, USA, pp. 5.1-5.33. Badler N.I, Phillips C.B, Webber B.L, 1993. Simulating Humans: Computer Graphics, Animation and Control. Oxford University Press. Bindiganavale R, Badler N.I, 1998. Motion abstraction and mapping with spatial constraints. Modelling and Motion Capture Techniques for Virtual Environments. Springer, Berlin/Heidelberg, pp. 70-82. Chung H.J, Xiang Y, Mathai A, Rahmatalla S, Kim J, Marler T, Beck S, Yang J, Arora J, Abdel-Malek K, 2007. A robust formulation for prediction of human running. In Proceedings of the 2007 Digital Human Modeling for Design and Engineering Symposium, Seattle, Washington. Craig J.J, 2005. Introduction to Robotics: Mechanics and Control. Pearson Prentice Hall. Kim J.H, Abdel-Malek K, Yang J, Marler R.T, 2006. Prediction and analysis of human motion dynamics performing various tasks. International Journal of Human Factors Modelling and Simulation Vol. 1, Nº 1, pp. 69-94.

6

I. Pasciuto, A hybrid dynamic motion prediction method with collision detection Monnier G, Renard F, Chameroy A, Wang X, Trasbot, J, 2006. A motion simulation approach integrated into a design engineering process. In Proceedings of the SAE International Conference and Exposition of Digital Human Modelling for Design and Engineering. July 4-6, Lyon, France. Monnier G, Wang X, Trasbot J, 2008. A motion simulation tool for automotive interior design. In: Duffy,V.G. (Eds.), Handbook of Digital Human Modeling. CRC Press-Taylor & Francis Group, Boca Raton, Florida, USA, pp. 31.1-31.14. Monnier G, Wang X, Verriest J.P, Goujon S, 2003. Simulation of complex and specific task-orientated movements - application to the automotive seat belt reaching. In Proceedings of the SAE International Conference and Exposition of Digital Human Modeling for Design and Engineering. 17-19 June, Montreal, Canada. Park W, Chaffin D.B, Martin B.J, 2004. Toward memory-based human motion simulation: development and validation of a motion modification algorithm. IEEE Transaction on System, Man and Cybernetics – Part A: Systems and Humans Vol. 34, Nº 3, pp. 376-386. Park W, 2008. Data-based human motion simulation methods. In: Duffy,V.G. (Eds.), Handbook of Digital Human Modeling. CRC PressTaylor & Francis Group, Boca Raton, Florida, USA, pp. 9.1-9.17. Pasciuto I, Valero A, Ausejo S, Celigüeta J.T, 2010. A dynamic motion prediction method based on a motion database and motion knowledge. In: Proceedings of the 1st International Conference on Applied Bionics & Biomechanics. Venice, Italy. October 14-16 2010. Piegl L.A, Tiller W, 1997. The NURBS Book. Springer Verlag. Popović Z, Witkin A, 1999. Physically based motion transformation. In Proceedings of the ACM SIGGRAPH 99, Computer Graphics Proceedings. Ren L, Jones R.K, Howard D, 2007. Predictive modelling of human walking over a complete gait cycle. Journal of Biomechanics Vol. 40 Nº 7, pp. 1567-1574. Wang X, 2002. Prediction of lower-limb movements of clutch pedal operation from an existing motion database. In Proceedings of 5th SAE Digital Human Modeling Conference, Munich, Germany, pp. 271-283. Xiang Y, Chung H.J, Mathai A, Rahmatalla S, Kim J, Marler T, Beck S, Yang J, Arora J.S, AbdelMalek K, 2009. Optimization-based dynamic human walking prediction. International Journal for Numerical Methods in Engineering, Vol. 79, Nº 6, pp. 667-695. Yang J, Marler R.T, Kim H.J, Arora J.S, AbdelMalek K, 2004. Multi-objective optimization for upper body posture prediction. In Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis

and Optimization Conference. Albany, New York, EEUU. Zhang X, 2002. Deformation of angle profiles in forward kinematics for nullifying end-point offset while preserving movement properties. Journal of Biomechanical Engineering Vol. 124, pp. 490-495. Zhang X, Chaffin D, 2000. A three-dimensional dynamic posture prediction model for simulating in-vehicle seated reaching movements: development and validation. Ergonomics, Vol. 43, Nº 9, pp. 1314-1330.

7