Accuracy in Optical Tracking with Fiducial Markers: An Accuracy Function for ARToolKit
Abstract Optical tracking with fiducial markers is commonly used in Augmented Reality (AR) systems. AR systems that rely on the ARToolKit [1] are prominent examples. The information obtained by the tracking subsystem are widely used in AR, e.g. in order to calculate how virtual objects should be located and oriented. The results of extensive accuracy experiments with single markers are reported and made operational by the definition of an accuracy function. The results show a specific distribution of tracking accuracy dependent on distance as well as angle between camera and marker. This insight is applicable for designing the set-up of AR applications in general that rely on optical tracking.
1. Experimental Set-Up In our experiment, we used ARToolKit to track a single cardboard marker. The marker (55 mm side length, ARToolKit pattern “sample1”) was fixed on a stable construction which allows a variation around the y-axis. The camera was mounted in a mechanical jig and adjusted in a way that the center of the lens and the center of the marker were on the same height. The experiment was carried out with a standard commercial web cam (Philips PCVC750K). We used a resolution of 640x480 pixels and a refresh rate of 15 Hz. The experiment consists of several test sequences. In every sequence 250 sequential measurements were logged. Between test sequences, we changed the distance of the marker to the camera and we rotated the marker around the y-axis. We adjusted the set-up in the manner that the center of the marker was projected in the center of the video image. All other conditions were held constant. We used artificial lighting in order to ensure this constancy. The distance of the marker to the camera was changed from 20 cm up to 100 cm with steps of 10 cm (9 variations). The angle of the marker
Ralf Dörner Hochschule Harz, Wernigerode, Germany
[email protected] was changed only around the y-axis in the range from 0° up to 85° in steps of 2.5° (35 variations) – while the angles around the x and z-axis were constant at 0°. Thus, our experiment had 315 test sequences with 250 individual measurements in each test sequence resulting in 78.750 single datasets.
2. Results The results are depicted in Fig. 1 to Fig. 4. 18 16 14 Systematic error [cm]
Daniel F. Abawi, Joachim Bienwald J. W. Goethe-University Frankfurt, Germany {abawi, jbienwald}@gdv.cs.uni-frankfurt.de
12 10 8 6 4 2 0 20
30
40
50
60
70
80
90
100
Camera distance [cm]
Figure 1. Distance errors for different angles
3. The Accuracy Function We aim to make the results of our experiment operational by expressing them as an “accuracy function”. Our accuracy function should not be considered as a substitution of the confidence value calculated by ARToolKit. Both aim to assess different characteristics. The confidence value of ARToolKit measures for each detected marker how well the system could match the marker to a specific pattern. Our accuracy function assesses how accurate the tracking parameters delivered by ARToolKit are in the specific distance and angle to the marker if the marker was correctly recognized in the first place.
Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004) 0-7695-2191-6/04 $20.00 © 2004 IEEE
6 5 4 3 2 1 0 0
10
30
40
50
60
70
80
Figure 4. Standard deviations of the camera angle for different distances 100 cm
90 cm
80 cm
70 cm
Y 50 cm
40 cm
20 cm
30 cm
X
8
Camera distance
90°
6
20 cm
80°
30 cm
4
40 cm 2
70°
50 cm 60° 60 cm
0 20
30
40
50
60
70
80
90
100
Camera distance [cm]
70 cm
20
Camera distance
15
40°
90 cm 100 cm
30° 20°
[
10° 0° Camera angle
X Low standard deviation of camera distance Y Low systematic error of camera distance Z Low standard deviation of camera angle [ Low systematic error of camera angle
10 5 0 0
10
20
30
40
50
60
70
80
90
-5 -10
Z
50°
80 cm
Figure 2. Standard deviation of the distance for different angles
Systematic error [°]
90
Camera angle [°]
10 Standard deviation [cm]
20
60 cm
12
The function shown in Fig. 5 gives an assessment of the accuracy for the estimation of distance and angle of the ARToolKit for a single marker. Furthermore, in situations where ARToolKit is able to track more than one marker at the same time, this function enables us to choose the marker with the most accurate tracking parameters.
Standard deviation [°]
Analyzing the results of our experiments we identify the following parameter ranges where standard deviation and systematic error are low: N The systematic error of ARToolKit in aspect of the estimated distance of the marker to the camera is low in the range of 20 cm up to 70 cm (cf. Fig. 1). N The distance estimation of ARToolKit has a small standard deviation in the range of 20 cm up to 50 cm (cf. Fig. 2). N The systematic error of the angle estimation is small in the range from 30° up to 40° (Fig. 3). N The angle estimation of ARToolKit has a small standard deviation in the range of 40° up to 85° (cf. Fig. 4). These four intervals are depicted in Fig. 5. This is our accuracy function; the more of the four intervals a specific area lies within, the more accurate the results of ARToolKit’s estimation are (darker areas in Fig. 5). The areas where ARToolKit’s marker recognition could not identify patterns well enough to consider accuracy are also depicted in Fig. 5.
Accuracy of detection: high
low no detection
Figure 5. Accuracy as a function of camera distance and relative camera angle
References
-15 -20 Camera angle [°]
Figure 3. Angle errors for different distances
[1] H. Kato, and M. Billinghurst, “Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System”, Proceedings of IWAR 99, pp. 85-94, 1999.
Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004) 0-7695-2191-6/04 $20.00 © 2004 IEEE
