Real-time and Markerless Vision-Based Tracking for Outdoor Augmented Reality Applications Didier Stricker, Thomas Kettenbach Fraunhofer Institute for Computer Graphics Rundeturmstraße 6, 64283 Darmstadt, Germany fstricker,
[email protected] Abstract In this paper, a new concept for markerless optical tracking called “tracking with reference images” is introduced . This concept provides a flexible and praticable framework and is especially relevant for outdoor augmented-reality applications, for which no tracking solution currently exists. The implementation is achieved using an image matching technique, which compares the current live video image with one or more of the reference images. The complete system has been tested outdoor in the context of a mobile AR-application for archeology. The system runs on a laptop at around 10 Hz and provides views of virtual monuments superposed to their ruins.
1 Optical tracking with reference images Markerless optical tracking is a very complex task. One can apply an image processing operator, like the LucasTomasi or Harris corner detector, to the video images and recover the camera trajectory out of the motion of the 2D features. This approach works well off-line because all features are accessible at a time, what enables filtering or minimization of the errors by e.g. bundle-adjustment techniques. Beyond it, one important characteristic of this kind of approaches is, that the motion is given in an arbitrary coordinate system and with an arbitrary scale because all information are got only from the images. On contrary, realtime Augmented-Reality (AR) applications require absolute coordinates, usually provided by the markers of an optical tracking-system. We propose here to substitute the markers with another support, namely with standard images of the environment. These images can be considered as the (necessary) reference component of the tracker and are called for this reason “reference images”.
0.7
0.9
0.2 User view
Augmented view 0.3
Database of reference images
Matching results
Figure 1. Tracking with reference images
The principle of the tracker is illustrated figure 1. At a given view point, a set of reference images is selected. The user view, ie the current live video image, is compared to all reference images and a correlation score is computed. The best score is retained and the 2D transformation between the video image and the reference image is evaluated. Considering that the reference images are calibrated the current camera position and orientation are deducted.
2 Real-time image matching 2.1 Overview The image matching technique represents the core of the system. The transformation between the live video image and the reference image must be computed accurately and in real-time. It exists a lot of different approaches [1], which differ basically by: (1) the kind of the considered transformation - local transformation, global linear and non-linear transformation - (2) the data, which are used - e.g. corners, contours, pixel intensity - (3) the search space and stategie - e.g. graph search, linear bipartite assignment.
Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR’01) 0-7695-1375-1/01 $17.00 © 2001 IEEE
2.2 Implementation and Results The current implementation of the matching-module bases on the Fourier transformation and is limited to 2D similarities (translation, rotation, scale). The Fourier image registration technique has been choosen because of its robustness to light changes and good results even in case of small image overlaps [2, 3]. Furthermore, the computations are done with a fix number of operations, which makes it particulary suitable for real-time processing. We consider actually for performence reason only one reference image per view point. One important issue is threreby to estimate if the image registration is valid or not. This has been achieved with help of the computation of the mean and variance of the inverse FFT of the cross-power spectrum at the last step of the process (see [2, 3] for details). If the correlation peak is well defined, the tracking is considered as valid.
(a)
(b)
Figure 3. (a) Augmented-View of the Hera Temple. (b) Trials on-site
The complete system runs on a laptop PC (PIII 800 MHz, GeForce graphics card) at 10 Hz with an image resolution of 320x240 pixels (see figure 3(b)).
4 Conclusion In this paper we point out the necessity to provide a support to optical tracking-systems. Usually this support is given by the markers and has been replace successfully with images of the environment. The presented system has been demonstrated outdoor and was able to determine in a reliable way the head motions at given viewing areas.
Figure 2. Real-time motion estimation based on image matching
The figure (2) shows a result of the tracking on-site. The images of a video sequence are matched sequentially and build a panorama of the scene.
3 Application: ArcheoGuide The tracking system has been originally developed in the project ArcheoGuide ( The Augmented Reality based Cultural Heritage On-site GUIDE ). The ArcheoGuide-system consists of a mobile AR unit that allows the visitors to see a computer-generated reconstruction of monuments without cutting them off from the real surroundings of the site. The project has for goal to explore new ways to learn about a cultural site. Arriving at an “view point”(marked on the ground), the visitors can wear on the Head Mounted Display and contemplate views of the virtual monuments on theirs ruins [4]. As a first trial site, we have selected the ancient Olympia in Greece, the birthplace of the Olympic games.
Acknowledgments This work is funded by the EU ( Project Archeoguide, IST-1999-11306). The authors would like to thank all the consortium partners Intracom S.A. (Greece), the Computer Graphics Center (ZGDV) (Germany), the Centro de Computao Grfica (CCG) (Portugal), A&C 2000 Srl (Italy), Post Reality S.A. (Greece) and the Hellenic Ministry of Culture (Greece).
References [1] L. G. Brown. A survey of image registration techniques. ACM Computing Surveys, 24(4):325–376, Dec. 1992. [2] D. Casasent and D. Psaltis. Position-, rotation-, and scaleinvariant optical correlation. Applied Optics, 15(7):1795– 1799, July 1976. [3] B. Reddy and B. Chatterji. An fft-based technique for translation, rotation, and scale-invariant image registration. IP, 5(8):1266–1271, August 1996. [4] D. Stricker., P. Daehne, F. Seibert, I. Christou, L. Almeida, R. Carlucci, and N. Ioannidis. Design and development issues for archeoguide: An augmented reality based cultural heritage on-site guide (icav3d’01). In International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging, Mykonos, Greece, May 28-30 2001. IEEE.
Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR’01) 0-7695-1375-1/01 $17.00 © 2001 IEEE
