Discussing Validation of 3D Character Animation Demonstrating ...

Discussing Validation of 3D Character Animation Demonstrating Ushiro-Ukemi Pedagogical Progression Mauro Cesar Gurgel de Alencar Carvalho1,2,3, Bruno Martins Carvalho1, Felipe Leal de Paiva Carvalho3, Heidi Dias Oliveira Junior2, Gerson Gomes Cunha2, Luiz Landau2, and Estélio Henrique Martin Dantas3 2

1 LaCiMovi – Movement Science Laboratory, Pedro II School, Rio de Janeiro, Brazil LAMCE–Laboratory of Computational Methods in Engineering–COPPE–UFRJ, Brazil 3 LABIMH – Laboratory of Human Bio-kinetics – UNIRIO – , Rio de Janeiro, Brazil [email protected], [email protected], [email protected], [email protected], {landau,gerson}@lamce.coppe.ufrj.br, [email protected]

Abstract. This study was designed to verify the evidence of validity, through: 1) literature review about ushiro-ukemi pedagogical progression. 2) An expert performed the movement and it was recorded to develop the animation. 3) Finally, an evaluation of 3D character animations processes by other experts’ opinions. 13 experts have evaluated three affirmatives through the Likert scale, and answered one question, also in ordinal scale, about the quality of the animation. The calculated medians for the first, second and third affirmatives were 5, 5 and 5 (fully agree), but, only for the 3rd affirmative about sitting ushiro-ukemi, the median was 4 (agree). The percentage of answers that scored higher than agree varied from 84,6 to 100%. The median of the 3D character animation was very good (4) and the percentage of acceptance was 100% (> good). The found definitional evidence of validity for these animations ensured their application in a learning material. Keywords: 3D character animation, Judo, sport skill, validation.

1 Introduction There are many available definitions for validation [1; 2; 3] or validity [4; 5; 6; 7; 8], varying according to the used approach or application. The references from the U. S. Department of Defense and related softwares for military use have shown detailed aspects about VV&A. The terms validation, verification and accreditation (VV&A) are commonly used in Computer Science, Software Engineering to attest the developed product fidelity, functionality and credibility. The Department of Defense [9] and the Department of Navy [10], both from USA, have presented a glossary and a manual with many close definitions about the Modeling and Simulation (M&S) Processes, validation, verification and accreditation. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 320–324, 2011. © Springer-Verlag Berlin Heidelberg 2011

Discussing Validation

321

Verification. The process of determining that an M&S implementation and its associated data accurately represent the developer's conceptual description and specifications. An informal question often applied to verification is "Was the M&S built right?" Validation. The process of determining the degree to which an M&S and its associated data are an accurate representation of the real world from the perspective of the intended use of the M&S. The informal question often asked for validation is "Was the right M&S built?" Accreditation. The official determination that an M&S application and its associated data are acceptable for use for a specific purpose. Accreditation seeks to answer the question "Is this the right M&S to use?" (DoN, 2004). Cook and Skinner [11] have also presented definitions and lited 75 different strategies to develop validation process for M&S. They divided these strategies in 4 groups: Informal, static, dynamic and formal, however these approaches are specifically directed for military M&S softwares. According to Howe and Mahar [12], definitional stage is the first stage during the validation process, though it is necessary to define the developed 3D animations as ushiro-ukemi representations.

2 Goals This study aimed to identify the 3D animations as representations of ushiro-ukemi pedagogical progression, to verify if 3D executed movements may be seen in many angles, to present and validate the overlapping technique for animating the biped performing ushiro-ukemi, and to evaluate the quality of the animation.

3 Sample The subjects were 13 experts in Physical Education and Judo (black-belt).

4 Materials and Methods 4.1 Video Videos were shot from front, back and sides views where a Judo teacher performed the laying, sitting, squatting and standing ushiro-ukemi. 4.2 Website Creation A website was idealized and built with a link, where the experts would be able to provide this validation: www.lacimovi.net/validacaoanime/index.htm. In this the face validation process was divided in three parts: introduction, animation process and questionnaire. The introduction part informed the experts about the validation proposal, the questionnaire goals and asks for their personal data. This part had four webpages,

322

M.C.G. de Alencar Carvalho et al.

one page for each movement of the ushiro-ukemi pedagogical progression: laying, sitting, squatting and standing (www.lacimovi.net/validacaoanime/01_deitado.asp is the first). Each page presented an animated image (.gif) showing the animation process developed at 3DS Max, where the video was set as viewport background and the biped movement was adjusted over the Judo teacher’s movement.

a

b

c

d

Fig. 2. (a) laying, (b) sitting, (c) squatting and (d) standing ushiro-ukemi animation at 3DS Max

This part presented three statements and one question to experts about the laying ushiro-ukemi. The affirmatives were evaluated through the Likert scale and the question according to the ordinal scale applied by Read and MacFarlane [13]. After answering this page, the experts were directed to the next page where the sitting ushiro-ukemi animation process and questionnaire was presented. The same procedure was followed by the squatting and standing ushiro-ukemi.

5 Results The first affirmative stated: “This 3D animation allows the (*) ushiro-ukemi visualization through various angles”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 91,3 (laying); 92,1 (sitting); 90,9 (squatting), and 92,1 (standing). The second affirmative stated: “This animation technique where the virtual human (biped) is overlapping the video allows presenting the (*) ushiro-ukemi flowing along time according to teacher’s movement”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 100 (laying); 92,3 (sitting); 90,9 (squatting), and 100 (standing). The third affirmative stated: “This 3D animation technique allows positioning the biped, adjusting its movement proportion every keyframe, according to the (*) ushiro-ukemi performed by the teacher”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 84,6 (laying); 84,6 (sitting); 90,9 (squatting), and 92,3 (standing). The fourth item was a question about the quality of the animation: “how do you evaluate this animation of the (*) ushiro-ukemi?” The animation scored “very good” for the sitting stages, but “excellent” for the others. The squatting stage scored 91,7 for the percentage of agreement, but 100 for the others. The % of agreement represents the amount of “fully agree” and “agree” answers. The median was applied to represent the answers central tendency measure, because they are based on an ordinal scale. was followed by the squatting and standing ushiro-ukemi.

Discussing Validation

323

6 Discussion and Conclusion The closest approach to this study goals was found at http://vva.msco.mil/Special_Topics/ HBR-Validation/default.htm, where DoD [14] discuss specifically the considerations for Human Behavior Representations (HBR). DoD affirms that the most commonly used strategy is face validation, obtained through experts opinions. However, while DoD (op. cit.) affirms that this validation technique is less reliable and complete, Cook and Skinner [11] deny that assuming that face validity, classified as informal, may lead to a misinterpretation. “While informal techniques have the advantage that they are relatively easy to perform and understand, their application is anything but unstructured. (…) Informal V&V techniques can be very effective if applied with structure and guidelines, and they are relatively low cost. Informal V&V techniques are effective for examining both the model and the simulation” [11]. This supports the strategy applied in this study for validating 3D character animation representing the pedagogical progression of ushiro-ukemi. Besides, during the animation process some steps were carefully taken to control some variables, to ensure a better representation of the ushiro-ukemi and to strengthen the internal and external validity [8]: a)

b)

c) d) e) f) g) h) i)

the original movement (in video) was performed by a Judo teacher (experienced performer), according to Kano’s [15] and Inokuma & Sato [16] descriptions and served as gold standard; animation was developed in one of the best programs for modeling and animation, 3D Studio Max, which already had a skeleton similar to humans, as a default, to develop the animation; a website was created to facilitate experts’ access and to permit the face validation of the animation to occur; the evaluators were chosen by their expertise in Physical Education and also in Judo (black belt holders); the site contains an introductory explanation to the experts about the task they would perform and also the proposed goals; three statements and a question was asked for each stage of the pedagogical progression; the answers were automatically sent to three different e-mail accounts to prevent loss of data; results were statistically treated for a proper description and interpretation; The medians and the percentage of agreement have shown enough evidence of validity for the 3D character animation, for the applied animation technique, for controlling the 3D animation positioning and speed, the type of the applied light and change the background color, and also for the quality of the animation.

The present study has shown evidence of face validity for the developed animation representing the ushiro-ukemi pedagogical progression. The applied technique was helpful for short duration movements, but it may not be advantageous (time x effort)

324

M.C.G. de Alencar Carvalho et al.

for long duration movements. These models may be applied in Augmented Reality systems for visualization and teaching Physical Education and Sports [17; 18].

References 1. Sargent, R.G.: Verification, validation, and accreditation of simulation models. In: Proceedings of the 2000 Winter Simulation Conference, pp. 50–59 (2000) 2. Balci, O., Ormsby, W.F., Carr, J.T., Saadi, S.D.: Planning for verification, validation, and accreditation of modeling and simulation applications. In: Proceedings of the 2000 Winter Simulation Conference, pp. 829–839 (2000) 3. DoD – Department of Defense of United States of America: Key concepts of VV&A (September 15, 2006a), http://www.vva.dmso.mil 4. Docherty, D.: Measurement in Pedriatric Exercise Science. Human Kinetics, 291–292 (1996) 5. Burton, A.W., Miller, D.E.: Movement skill assessment. Human Kinetics, 110–117 (1998) 6. Thrischler, K.: Medida e avaliação de Educação Física e Esportes. Manole, 66–72 (2003) 7. Rowe, D.A., Mahar M. T.: Validity. In: Measurement theory and practice in Kinesiology. Terry Wood and Weimo Zhu, Human Kinetics, 9–26 (2006) 8. Thomas, J.R., Nelson, J.K., Silverman: Métodos de pesquisa em atividade física. ARTMED, 280–296 (2007) 9. DoD – Department of Defense of United States of America: Glossary (December 5, 2006b), http://www.vva.dmso.mil/Glossary/default.htm 10. DoN – Department of Navy of United States of America: Modeling and Simulation Verification, Validation, and Accreditation Implementation Handbook, vol. I VV&A Framework (2004) 11. Cook, D.A., Skinner, J.M.: How to Perform Credible Verification, Validation, and Accreditation for Modeling and Simulation. CrossTalk. The Journal of Defense Software Engeneering (May 2005) 12. Mahar, M.T., Rowe, D.: A Practical Guidelines for Valid and Reliable Youth Fitness Testing. Measurement in Physical Education and Exercise Science 12(3), 126–145 (2008) 13. Read, J.C., Macfarlane, S.: Using the Fun Toolkit and Other Survey Methods to Gather Opinions in Child Computer Interaction. In: IDC 2006 (June 7-9, 2006) 14. DoD – Department of Defense of United States of America: Glossary (2001), http://vva.msco.mil/Special_Topics/HBRValidation/default.htm 15. Kano, J.: Kodokan Judo. Kodansha International (1994) 16. Inokuma, I., Sato, N.: Best Judo. Kodansha International (1986) 17. Carvalho, M.C.G.A., Dantas, E.H.M., Landau, L., Cunha, G.G.: Teaching Judo breakfall (ushiro-ukemi): an application of human modeling, movement simulation and augmented reality techniques. In: The International Workshop on Applied Modeling & Simulation 2006, Búzios, Brazil, p. 173 (2006) 18. Carvalho, M.C.G.A., Dantas, E.H.M., Landau, L., Cunha, G.G.: Augmented Reality applied to Ushiro-ukemi visualization. In: Annals of the 5th International Judo Federation World Research Symposium, Rio de Janeiro, Brazil, September 12, p. 35 (2007)