Sourceless Human Body Motion Capture David Fontaine, Dominique David, Yanis Caritu Département systèmes pour l’information et la santé LETI, CEA Grenoble
[email protected] [email protected]
Abstract We introduce a new human machine interface based on the human body posture tracking. The using of multiple low cost and miniaturized sensors distributed on the body is able to provide the movement without any external sources. This information can then be used for real time character animation, interaction with virtual worlds and medical applications. Thanks to ease of use and low cost required components, such system will be compatible with consumer applications.
Key words Motion capture, Human interfaces, sourceless tracking, accelerometer, magnetometer, virtual reality.
Figure 1: direction of the gravitational field The magnetic field, B, is oriented to the north with an angle depending of our position on the Earth. (figure 2). Three orthogonal magnetometers give us the direction of the magnetic field.
1. Introduction Motion Capture systems are the best interfaces to navigate and interact with the 3D virtual worlds. For complete immersion, the user should sense and interact with the synthetic environment in the same manner in which interaction with the natural world takes place. Such systems are not accessible to the general public because of their price and complexity. They use a lot of different technologies and approaches based on various physical principles and providing different performance characteristics (ref [1]). Our approach consists in taking benefits from the micro sensor technologies to design new motion tracking microsystems which don’t need any artificial external sources. The sensors issued from the LETI technologies (accelerometers and magnetometers) enabled us to design a three degrees of freedom orientation tracking system. The idea is to track the orientation of each segment to determine the body posture. The miniaturization of these sensors enables to integrate them in the clothes. The goal is to bring motion capture to the general public and investigate into what is its contribution for 3D interfacing.
2. Orientation tracking Microsystem We describe here the basic orientation tracker which will be distributed on the body. 2.1. Giving Orientation without sources nor integration 2.1.1.
Principle
The principle is orientation estimation based on three axis sensing of Earth gravitational and magnetic fields. The gravitational field, G, has always the same direction and value (figure 1), so three orthogonal accelerometers attached to a static object will give us the direction of the local vertical.
Figure 2: direction of the earth magnetic field Both fields are necessary because G doesn’t give the rotation around the z axis. And the three axis are essential to restitute the three angles in full 360°. This solution doesn’t integrate gyrometers unlike other devices based on the same principle (ref [http1] and [http2]). It permits to reduce considerably the cost and the dimensions. The limitations are that the accelerometers are sensitive to gravity as well as linear accelerations. In a system which accelerates, the three linear accelerometers produce an output vector which doesn’t give the direction of the vertical. The other limitation is that the Earth magnetic field is perturbed by any ferromagnetic object. 2.1.2.
Data fusion
Because of these limitations, sensors used independently can’t reach a very high accuracy, but the combination of the six sensors (3 accelerometers and 3 magnetometers) reduces the errors: when a sensor gives erroneous information another sensor compensates by giving good information, for example, the accelerometers are perturbed by an acceleration due to displacement and the magnetometers are not. The algorithm of data fusion is based on the reducing of the quadratic error between the measured vector and evaluated vector computed from the sensor models with the angles we are looking for. The evaluated data are computed with these equations: Ax= -sinθ.g Ay=+sinψ.cosθ.g Az=-cosψ.cosθ.g
and Mx=cosθ.cosϕ.ck+sinθ.sk My=(sinψ.sinθ.cosϕ+cosψ.sinϕ)ck-sinϕ.cosθ.sk Mz=(- cosψ.sinθ .cosϕ+ sinψ .sinϕ)ck+ cosψ.cosθ.sk Where ψ, θ, ϕ are the euler angles and k the orientation of the magnetic field with the horizontale (sk=sin k, ck= cos k).
z ϕ
ψ
θ
y
x
An iterative method changes the angles until the error is minimum. One of the optimization consists in detecting when a group of sensors gives erroneous information and then reducing their contribution in the algorithm. To detect when accelerometers measure anything else that the gravitational field we calculate the norm of the acceleration: Ax2+Ay2+Az2, the difference between 1 and the norm gives us the importance of the error. 2.2. Hardware considerations 2.2.1.
Accelerometers
Surface micromachined acceleration sensors have been commercially available for some years now, with the automotive sector being the largest market. Initial applications include crash sensors for airbags. These devices resolve acceleration down to the milli-g level (1g=9.8m/sec2). Miniaturization and precision enlarged the applications to inertial navigation coupled with GPS navigation. The accelerometers mounted in our tracker measure in the range of ±1g or ±2g. We use the analog Devices and STMicroeletronics accelerometers which include on chip signal conditioning and the Tronics Microsystems accelerometer which requires an external capacitive detection. 2.2.2.
Magnetometers
The micro fluxgate magnetometers are designed in the LETI Laboratories. It consists of two coils wrapped around a common high-permeability ferromagnetic core whose magnetic induction changes in the presence of an external magnetic field. A drive signal must be applied to the primary winding and the signal is detected to retrieve the magnetic field value. They benefit from performances required in space applications. The low cost alternative is the giant magnetoresistance. To improve their performance characteristics they need compensation circuits for drift correction. But in applications such as navigation for general public, using conditions don’t impose elaborated compensations. 2.2.3.
3D mounting of the sensors
Only few sensors are available in 3D package, so we need at least two orthogonal planes to have the third dimension. The solution used for our prototype is a 1.5 cm cube where the 6 sensors are mounted on three of the inside faces (figure 3).
Figure 3: sensors mounted on 3 planes 2.2.4.
Miniaturization of the orientation tracker
3D packages are almost available for Honeywell GMR, but are rare for accelerometers but it will change in the near future and the system will be mounted on one plane reducing the manufacturing cost and the dimensions. On the other hand, the progress in micromachining and the heterogeneous assembling of materials make possible the mounting of the six sensors on a single chip with the conditioning electronics. 2.3. Performance of the orientation tracker The orientation sensing is very efficient in the static mode, i.e. when the movements are quite slow. When the movements are fast, the initial and final orientation are well found but not during the movement. The resolution is about 0.5° and the precision is better than 1° in the static mode. 100 80 60 40 20 0 -20
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
-40 -60 -80 -100
Psi
Theta
Phi
Figure 4: response for Psi angle This figure shows a good linearity and precision and also a little variation for the other angles. To show the dynamic response we are developing an automatic system which gives known orientations and translations. The goal of this system for the application of real time motion tracking is not to be very precise but easy to use and inexpensive. When this system is used in interaction situation, the user sees what he does on a screen and corrects himself the errors due to the tracker.
3. Human body motion capture 3.1. Human body model First we must decide exactly what we are trying to animate. Although the ideal computer generated character would include muscle and tissue that deforms during movement, skin and clothing that wrinkles and stretches, hair that flows and expressive facial features, we will have to restrict our attention to animating simple approximations to real bodies. A skeleton can be represented by a collection of simple rigid objects connected together by joints. A detailed approximation to the human skeleton may have as many as 200 degrees of freedom although often fewer suffice. Figure 4 shows the basic skeleton used in Kaydara Filmbox software.
3.2. Method The graphical rendering is made with Kaydara Filmbox, it is a real-time capture system designed for the capture, editing, and blending of motion data. With complete support for motion-capture and a full suite of 3D character animation tools, it enables us to acquire live animation data from multiple and simultaneous motion capture and tracking devices. We developed a plug-in to interface our device to this software, the angles measured are then directly affected to the character bones. The number of trackers depends on the application, for the whole body, the minimum required is ten trackers but fifteen is much better (figure6).
Figure 7: placement of the trackers Figure 5: skeleton Each Bone is animated by assigning a rotation measured from the orientation tracker. The segments are arranged in tree structured hierarchy. This hierarchy ensures that segments inherit the rotations applied to joints higher in the tree, a rotation applied at, say, the shoulder joint, causes the entire arm to rotate, and not just the upper arm segment. This is an example of a tree structured hierarchy (figure 5). Hips
Some applications require the motion capture of the upper body only and the number of tracker is reduced. Let’s see the errors that can occur and what solutions are envisaged. 3.2.1.
Data from sensors are false (due to the Earth magnetic field which changes from a place to another). Magnetic sensors must be re-calibrated when the environment changes. The system can also detect a bad calibration. 3.2.2.
Left Up Leg
Right Up Leg
Spine
Left Leg
Right Leg
Spine 1
Left Foot
Right Foot
Spine 2
Left Toes
Right Toes
Spine 3
Left Shoulder
Right Shoulder
Neck
Left Arm
Right Arm
Head
Left Fore Arm
Right Fore Arm
Left Hand
Right Hand
Trackers sliding over the skin
The alignment with the bones can change due to relative movements of muscles and skin. This misalignment can result in large errors of angles. The trackers must be placed close to the bones where there is less muscles and fat. On the other hand, the rotations of the joints must also be limited to stay in the human limitations. 3.2.3.
Figure 6:skeleton hierarchy
Calibration errors
Simplified skeleton modeling
The simplified representation of the skeleton and the limited number of orientation trackers don’t take into account all the movement of the clavicle and the spine curve. To correct these errors we must integrate some elements of inverse kinematics. 3.2.4.
Fast movement detection
When the accelerometer is at the end of the arm, accelerations due to displacement is more important resulting in large angles errors. We have different methods to reduce this
phenomenon. We can use human models and constraints which give us a lot of information. 3.3. Data acquisition The actual system works with a PCI acquisition board from Measurement Computing, this board has 64 analog inputs, so it can acquire signals from ten cubes (each containing six sensors). The Software receives the converted voltage from each sensors, the signals are then calibrated to get the corresponding accelerations and magnetic field values. This system has a lot of connections, there are two solutions: dissimulating the wires in specific clothes or developing a wireless system. The first solution is to integrate sensors and wires in clothes, the analog signals are then collected to a wearable box containing the analog/digital converter, the serial data are then sent to the PC via a wired or wireless connection. This solution is less expensive compared with the second solution consisting in integrating the analog conversion and HF emitter in each cube. But it allows us to place the tracker where we want and to select which segment has to be tracked. The decision depends on the application. 3.4. Applications Motion tracking is an important aspect of virtual reality. Important notions in VR are real time interaction with virtual world and immersion (ref [3]). These features are made possible with the help of interfaces which act on our senses and capture our behaviors. Examples of VR interfaces are Head Mounted Displays and Data Gloves. For a better immersion in virtual worlds these devices must be forgotten by the user to act like he would in the real world. Detecting the body posture can trigger off actions by the computer. Video games uses VR but the degree of immersion is more or less important. Television uses Motion Capture for virtual actors (ref [4]). There are also applications in professional field: formation to dangerous operations, surgeons training, studies on behaviors for medical, simulation of military operations. We could imagine a large number of applications.
contained, it doesn’t require important calibrations and has not external parts mounted in the environment of the user unlike optical and magnetic motion capture systems. It is immune to occlusions, it needs no clear line of sight to anything else. There is still much work to be done on the subject, but the directions are clearly identified. First, the dynamic response must be improved. Errors occur during large movements of the arm because accelerometers detect accelerations due to these displacements. We have to evaluate the performance of this system compared with others. Then, the logical course of action is to design a wearable system which would be miniaturized and wireless.
5. Conclusions The mouse and the keyboard will no longer suffice as the predominant method for interfacing with synthetic environments and virtual worlds. We believe that the new human interfaces will include some type of body tracking system for sensing the user’s actions. We have shown that orientation-only motion capture using accelerometers and magnetometers without external sources is possible. The use of wide-spread sensors like accelerometers and magnetometers reduces the price and enables to bring this technology to the consumer market and not only professional motion capture studios. It supposes to design new services and usage modes.
6. References [1] Welch, G.; Foxlin, E., “Motion tracking: no silver bullet, but a respectable arsenal '', IEEE Computer Graphics and Applications, Vol. 22, Issue 6, Nov/Dec 2002. [2] Bachmann, E., “Inertial and Magnetic Angle Tracking of Limb Segments for Inserting Humans into Synthetic Environments”, Ph.D. dissertation, Naval Postgraduate School, Monterey, CA., 2000. [3] Fuchs, P; Moreau, G; Papin, J.P., “Le traité de la réalité virtuelle”, Les Presses de l’Ecole des Mines de Paris, Paris 2001. [4] David J. Sturman., “Computer Puppetry ”, IEEE Computer Graphics and Applications, January/February 1998. [http1] : http:\\www.isense.com [http2] : http:\\www.xsens.com [http2] : http:\\www.kaydara.com
Figure 8: the trackers in action
4. Discussion and future works We proposed in this paper the advancement of our works on this system. We have demonstrated the ease of use and self
