A 3D City Info for Mobile Users - CiteSeerX

A 3D City Info for Mobile Users

1

Ismo Rakkolainen1 , Jani Timmerheid1 and Teija Vainio2 Digital Media Institute, Tampere University of Technology, P.O.BOX 553, 33101 Tampere, Finland 2 Hypermedia Laboratory, 33014 University of Tampere, Finland

Keywords 3D graphics, mobile multimedia, VRML, usability, navigation, city info Abstract For many applications, a three-dimensional representation improves the usability of data. Soon 3D rendering and broadband wireless communications will be embedded into various wireless handheld devices. We have earlier built a web-based 3D city info connected to a database. We customized the system for mobile users, and conducted field trials and usability studies on the added value of 3D graphics on navigation and wayfinding in cities. Our results show that search and visualization of location-based information of a city becomes more intuitive with life-like 3D. 1

Introduction Where is a certain shop? Where do I find an item nearby? Where am I now? How do I get from railway station to my hotel? These are typical ques tions of inhabitants and visitors of a city. Mobile devices will be able to provide answers. The web pages of hotels, restaurants, and other services often show their locations on a 2D map. Some information can also be delivered to mobile communicators, PDAs, or WAP devices. Would 3D graphics add value to mobile navigation and serv ice browsing? Some usability tests show that 3D environments help understanding spatial relations better and thus can improve task performance and navigation. In principle, 3D graphics has many advantages over video or 2D graphics. Three-dimensional, interactive environments are an intuitive and userfriendly way to view location-based information. After the initial download, an infinite number of interactive renderings can be created. 3D graphics

can scale and compress better than equal quality video or animations. Potential new user interface paradigms are additional advantages of 3D graphics. Both the high computational and data transfer costs have prevented the use of high-quality 3D graphics in mobile devices. The 3D rendering speed of a new PC has increased more than 100fold in three years. Moore's law is for wimps! Multimedia and 3D instructions are being embedded into mobile chips, too. The other bottleneck, network bandwidth, will be removed with third generation (3G) cellular systems using Universal Mobile Telecommunication System (UMTS), or technologies like wireless LAN, GPRS, EDGE, and Bluetooth. They will make the access speeds of mobile devices comp arable to, or higher than the present home modem connections. Also wireless Internet connections being on the market already will make mobile Internet possible in limited areas. The emergence of wireless broadband communication and powerful 3D graphics to multimedia cellular phones and other mobile devices is just a matter of time. There is already a 3D engine [36] available for EPOC platform, and a Cortona CE VRML (Virtual Reality Modeling Language) browser for Windows CE. New kinds of networked 3D applications will soon become feasible. We have earlier built a web-based 3D city info connected to a database. We customized the system for mobile users, and conducted field trials and usability studies on the added value of 3D graphics on navigation and wayfinding in cities. Our results show that search and visualization of location-based information of a city becomes more intuitive with life-like 3D. The users recognise landmarks and find the route in cities easier with the 3D model than with a symbolic 2D map. Chapter 2 lists some related work. Chapter 3 describes our earlier implemented database-driven 3D city info system [16] for wired Internet users.

The 3D city model provides an intuitive 3D interface and an inherent spatial index. Chapter 4 describes the modified 3D city info system for mobile users. Chapter 5 describes the setup and results of the usability test. Chapter 6 discusses some future work issues. In Chapter 7, we present the relevance of results and some possible exploitation concepts for 3D graphics in mo bile 3G city communication. 2

Related Work Some early mobile 3D experiments are Raposo et al. [31], who filtered and reduced the VRML data to be transmitted over GSM network, and the MOMENTS project [24], [1], which visualized 3D vector graphics, small VRML animations, and other multimedia on mo bile data terminals over GSM. Augmented reality uses 3D graphics for overlaying synthetic objects onto real scenes, e.g., [2], [3], [34], [35]. The required see-through HMD (head-mounted dis play) may not be convenient for mobile 3G users. CyPhone-project has created visions of the future UMTS devices [8]. Paula-project [28] is developing new ways of user interaction and devices for future broadband telecommunication networks. Xanavi Inc. has developed Birdview [42], a GPS-based in-car, non-networked navigation system. It allows drivers to see where they are on a 3D-looking map. There exists some mobile city guides without 3D graphics, e.g., [11], [5], [6], [7], [13], [29]. The Deep Map project [12] is developing a multidimensional GIS system for an electronic tourist guide related to the city of Heidelberg. It will also feature 3D worlds and mobile use. We have earlier built a system to visualize realtime GPS data with VRML worlds [30]. Movements of mo bile units could be viewed in a VRML world. The mo bile unit consisted of a laptop, a GPS receiver with radio differential correction, and a GSM phone. A demonstration can be found at http://juna.dmi.tut.fi/gpsdemo/. 3

The 3D City Info We have implemented a web-based 3D city info [16] for wired users. It is a joint project of several universities. The implemented system is at http://www.uta.fi/hyper/projektit/tred/. It is based on VRML [37], as it is the de-facto 3D format on the web.

3.1

Database VRML cities usually don't provide information on the real services found in the cities. If they do, it is embedded into static HTML or VRML files [14], [23]. Updates require regeneration or manual editing of the files. This can be avoided by putting the dynamic data to an easily updateable database. We combined a relational database of services to the VRML city model and have found no other such implementations. The system is platform in dependent and requires only a standard Java-enabled web browser and a VRML browser. The user can make queries about public and private services in Tampere city center, e.g., the locations of hotels or restaurants. The search criteria are by name, by address or by type. The user can search data either by using a hierarchical list of services or by using free text search. The system visualizes the query results both in the VRML world and on the interconnected 2D map. Our 3D city info uses a novel mechanism to connect a database to a VRML model. A Java 2 applet communicates with the database through a standard JDBC API [18]. It provides crossDBMS connectivity to a wide range of SQL databases. The applet communicates with the VRML world through the EAI interface. Figure 1 shows the system overview.

City info relational database

JDBC Web browser Java applet Search

2D map

EAI VRML browser

Figure 1. Wired 3D City Info system overview.

The system is easy to combine with commercial databases, e.g., yellow pages, or other such service catalogues. Standard relational database technologies are used on the server side. This enables easy updates to the data content. Structural changes to database, like adding new kind of in formation, are fast to implement. Modeling The effort involved in capturing the content for the city is a major task currently. Some GIS databases or modeling methods exist or are being researched to produce city models semi- or fully automatically, e.g., based on imagery, range data, or satellite surveillance data. It is though not in the scope of our research. Municipal CAD data provided only the basements of the buildings. Some elevation data was inaccurate and had to be corrected manually. The walls and roofs had to be modelled manually based on stereo aerial photographs and other images. Also other available materials, like architectural drawings, was used. The modelled area consists most of the city center. Some other parts of the city are in separate models. The geometry of the city center is accurately modelled (better than 10 cm). The VRML version of the city model had to be made highly simplified to enable reasonable rendering and download times. It contains some 5100 polygons. The size of the geometry is 120 KB and the textures are about 5 MB.

The models look highly realistic due to detailed texturing (see Figure 2). The realistic style is chosen because the interface design should convey the naturalness of the real world in able to provide appropriate tools for navigation, orientation and feedback for the user [22]. The model renders fast with modern PCs. It downloads in moderate time over ISDN or faster lines.

3.2

Figure 2. A comparison of the real view and the VRML model of a building.

4

A 3D City Info for Mobile Users

In this paper, we adapted the 3D City Info for mobile users and built a demonstration of future mobile services. We field-tested the usability of 3D visualization. We have built a fully working mobile laptop version of the 3D City Info with an integrated GPS receiver. The city info database and model run locally on a 650 MHz PIII laptop capable of fast 3D graphics. An online database connection could be built easily with, e.g., a GSM card, but for our purposes a local database is adequate. The fully working laptop version was not used for the usability test. For the field tests, the ease of use and simulation of the actual user situation of the future devices was highly important. A PDA is much easier to use and lighter to hold in a city environment than a laptop. Therefore we used a handheld PDA, HP Jornada 548. Our 3D model is too large to run even nearly smoothly on any PocketPC. The frame rate with a Cortona CR VRML browser was one frame each 8 seconds. Even bigger a problem is to run Java 2 on that platform at the moment. Also GPS is not available for all PDAs currently. These problems will be removed gradually, as hardware improves. As 3D graphics is not yet very fast in PDAs, we customized a "mockup" field-test version of our system. All real functionality was disabled and only HTML code and images, which simulate the real system, was used. The disfunctional simulation of the system also saved implementation time and helps in evaluation and further development of the system. The image-based version of the system was adequate for the purposes of the well-focused and limited usability test. The mobile user in the field tests could locate him/herself manually in some predetermined fixed checkpoints. In this way we gained better positioning accuracy, as GPS signals in cities may be blocked or reflected.

4.1

Interface Design As the mobile devices typically have a smaller display than desktop PCs, we modified the user interface to fit into a smaller device (see figure 3). It has two of the three components (database query, 2D map, 3D world) visible simultaneously. The view can be changed easily by pressing a button on the left on the screen.

Purposeful, oriented movement during navigation (i.e., skilled wayfinding) improves with increased spatial knowledge of the environment. Spatial knowledge can be described as three levels of information: landmark knowledge, procedural knowledge, and survey knowledge [9]. In landmark knowledge, the landmarks are defined as the predominant objects in the environment [4], [20]. In our 3D City Info project, our specific aim is to support users to navigate better and to learn the structure of the graphical space with the help of the real-time connection between the 2D map and the 3D model based on landmark knowledge. Levine [19] has defined that for efficient map usage, the map must be congruent with the environment it is representing. According to the forward-up equivalence principle, the upward direction on a map must always show what is in front of the viewer [19]. We apply these statements to our system. Also the body-scale is easier to recognize in a near-photorealistic virtual city. 5

Figure 3. The user interfaces of the mobile 3D City Info and an HP Jornada 548 PDA. The principles we investigate in our system are based on navigation and wayfinding in virtual worlds. Darken [9] classifies wayfinding tasks into naïve search, primed search, and exploration. We concentrated to naïve search, in which the user has no a priori knowledge of the target [9].

Usability Test The purpose of the field test was to find out the wayfinding process in the 3D model and in the real world with the aid of the mobile information system. Our primary concern is to compare landmark knowledge in interfaces that are combined with the map and the 3D model. Some usability tests were done in desktop PCs in a laboratory to test the overall usability of the mobile version, others with a mobile device in a real city environment. In mobile devices we were using mock-ups with web pages and images in stead of a full 3D model, but in desktop PCs the usefulness of the real 3D model was tested. In wayfinding, it is important to recognize the environment with the help of landmarks. It is essential for navigation tools, such as a 2D map and a 3D world, to provide these landmarks too. Our hypothesis is that in a 3D world compared to a 2D map this kind of landmark recognition is easier and in 3D world the landmark recognition can be considered as a remarkable feedback to the users. By using mobile devices with a positioning technology, the user can verify his location in the real world both from the 3D model and the 2D map. The metaphors, which are used in photorealistic 3D worlds, are more recognizable to users than in 2D maps, which are symbolic, simplified, flat 2D representations of reality.

The test persons were not very familiar with the city of Tampere, and had not much experience on computers or the Internet. The task for the persons in the mobile usability test was to search a site from the database, view the route to there and then go there. The users located themselves initially and during some fixed checkpoints during the route finding walk. They could view the 2D map and 3D view of the predefined points. The user was asked to "think aloud" during the walk, which was recorded and user was also interviewed after the task. According to the usability study, the test persons would prefer to use the 3D model combined with a map than to use only the map or the model alone. More likely they also recognized their own position and the landmarks from the photorealistic model than from the map. The visual similarity with the reality helped them to find the places in real life. 6

Future Work

The 3D City Info is an ongoing project with more content and features upcoming. The environment and database will be e xpanded. A realtime GPS-based bus tracking system is also being developed in Tampere, and the bus position information may be added to the system in the future. The major problem with even medium detailed city models is their big size, which affects both the rendering speed and the download time. The mo dels of cities and their details can be nearly infinite. High-quality online-visualization of virtual cities in real-time is a demanding task even for a state-of-the-art PC, although hardware is improving in a dramatic pace. The rendering and data transfer problems are even more severe for mobile devices. The major bottleneck is the data transfer, especially with slow home modems. Our 3D model is currently too detailed for them. The download time could be shortened with many optimizations, e.g., by providing a model with no textures, but then the visual quality is partially lost. Level of details improves the rendering speed but does not affect the download time. 6.1

Improved 3D for the Web Also VRML as a format has many severe limitations. It can't handle huge worlds smoothly over the Internet. The VRML browser downloads the whole model even if it needs only a part of it. An

improved performance of web3D technologies is needed, es pecially for mobile communications. Many working groups of the Web3D Consortium [39] address the problems. A framework for streaming geometry in VRML is also presented [15]. The Systems part of MPEG-4 [25] is largely based on VRML and it incorporates a streaming and comp ressing scheme. Some solutions are to render the geometry progressively according to the available hardware and bandwidth, as in, e.g., [21]. Also streaming the binary-comp ressed progressive meshes, model simplification, geometry smoothing or other techniques could be used. Some solutions are presented in [17], [26], [27], [32], [33], [41]. The next version of VRML, X3D, is based on XML. It's Core profile contains a reduced set of VRML nodes and enables light implementations of X3D players. They are particularly suitable for mobile devices. The players need only a web browser supporting Java, and download and install themselves automatically. A broad range of mobile devices will feature Java. An X3D city model would provide future compatibility. The transformation of the model to X3D Core or MPEG-4 should be rather easy. 7

Conclusions

In this paper, we have described a 3D city info application for mobile users. It enables to view the locations of services and real places in an intuitive and user-friendly way. Our system is an early prototype of the future mobile services. They can benefit from the user's location and visualize time-, location-, and context-based information. The usability tests show that 3D models help the users to recognize landmarks and find a route in cities easier than with a symbolic 2D map. The mobile 3D City Info provides an intuitive 3D interface for navigation and to the services. A mobile user could ask for personalized information based on his/her location. The advantages and applications of visualizing the real-time real world with 3D models are numerous. A 3D representation of a real place makes it easier to perceive, e.g., proportions, dis tances, and landforms, and to recognize landmarks. 3D objects and environments are highly interactive and allow unrestricted movement. They also enable augmented or wholly immersive virtual spaces with HMDs. Mobile phones will transform into wireless Internet multimedia terminals. They may look like the devices in Figure 4. Some of the coming mo-

bile devices can locate themselves. Mobile phones embedded with Global Positioning System (GPS) and online maps have already emerged to market.

models were mostly made in Geoinformation Laboratory of Tampere University of Technology and Tampere Polytechnic School of Art and Media. User interface was partly made by Outi Kotala at the Hypermedia lab of University of Tampere. The work is largely based on the wired version of the 3D City Info made by Arttu Heinonen and Simo Pulkkinen. Their work helped immensely in the implementation of this paper. Thanks to Jani Pääjärvi and Mikko Mäkelä for the "Papyrus" device image (Fig 4b). We wish to thank profes sor Pauli Kuosmanen for his helpful comments. References

Figure 4. Visions of the future 3D/3G devices with GPS. The one below has a flexible, scrollable display. Some devices may also contain a digital compass, inertial sensors, miniatyre video cameras for position and orientation tracking [43], etc. Location-, time-, and context-aware visualization of information will become possible. Whitted [40] has pointed out that for mobile devices, the polygons per second is not as important a measure as frames per Watt-hour. Future developments in hardware will make powerful 3D graphics usual in compact mobile networked computers. New types of 3D Internet applications using GIS, spatial, location, and orientation data will emerge in the near future. We believe that 3D graphics will be important for the future concepts of mobile personal communications. Acknowledgments This research was funded by Academy of Finland's Paula-project and Tekes' TreD-project. The

[1] C. Belz, H. Jung, L. Santos, R. Strack, and P. Latva-Rasku, Handling of Dynamic 2-D/3-D Graphics in Narrow-Band Mobile Services. In Virtual Worlds on the Internet, IEEE Computer Society 1998, 147-156. [2] M. Billinghurst, S. Weghorst, and T. Furness, Wearable Computing for Three Dimensional CDCW. Proceedings of the International Symposium on Wearable Computing (October 1997). [3] M. Billinghurst and H. Kato, Collaborative Mixed Reality, Proceedings of the International Symposium on Mixed Reality (ISMR '99), Yokohama, Japan, March 1999, 261-284. [4] D. Charitos, Designing Space in Virtual Environments for Aiding Wayfinding Behavior. Department of Architecture, University of Strathclyde. The fourth UK VR-SIG Conference, Brunel University, 1st November 1997. [5] CitiKey software, http://www.CitiKey.com/. [6] CitySync, http://www.citysync.com/. [7] V. Coors and B. Wiedmann, Using Wearable GIS in outdoor applications. Proceedings of the Symposium on Interactive Applications for Mobile Comput ing, IMC'98, Rostock, Germany, November 1998. http://www.egd.igd.fhg.de/~imc98/ Proceedings/imc98-SessionMA4-1.pdf. [8] CyPhone project page, http://www.ee.oulu.fi/~tino/pihvi.html. [9] R. Darken, Wayfinding in Large-Scale Virtual Worlds. Conference Companion ACM SIGCHI '95, pp. 45-46. [10]R. Darken and J. Sibert, Navigating Large Virtual Spaces. International Journal of Human-Computer Interaction, January-March 1996, 8 (1), pp.49-72. [11]N. Davies, K. Mitchell, K. Cheverst, and G. Blair, Developing a Context Sensitive Tourist Guide, Proceedings of the First Workshop on Human Comput er Interaction for Mobile Devices, Glasgow, Scotland, May 1998. http://www.dcs.gla.ac.uk/ ~johnson/papers/mobile/HCIMD1.html.

[12]The Deep Map project, http://www.eml.villabosch.de/deutsch/Forschung/deepmap/deepmap.html [13]S. Feiner, B. MacIntyre, and T. Höllerer. A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment. Proceedings of the International Symposium on Wearable Computing (October 1997). [14]The Glasgow VRML model. http://iris.abacus.strath.ac.uk/glasgow/. [15]A. Gueziec, G. Taubin, B. Horn, and F. Lazarus. A Framework for Streaming Geometry in VRML. IEEE Computer Graphics and Applications, 19 (2), March/April 1999, pp. 68 - 78. [16]A. Heinonen, S. Pulkkinen, and I. Rakkolainen, An Information Database for VRML Cities. The Proceedings of IV2000 Conference / Web Graphics and Visualization, London, England, July 2000. [17]H. Hoppe, Progressive Meshes. Proceedings of the SIGGRAPH 1996 Conference, pp. 99-108. [18]JDBC API, Sun Microsystems, http://www.javasoft.com/products/jdbc/. [19]M. Levine, I. Marchon, and G. Hanley, The Placement and Misplacement of You-Are-Here Maps. Environment and Behavior, 1984. 16(2): p. 139157. [20]K. Lynch, The Image of the City. 1960, Cambridge: MIT Press. 194. [21]MetaStream, http://www.metastream.com/. [22]S. Mills and Noyes J. Virtual reality: An overview of User-related Design Issues Revised Paper for Special Issue on "Virtual reality: User Issues" in Interacting With Computers, May 1998. Interacting with Computers 11 (1999) pp. 375-386. [23]The Mischek Tower, http://www.mischek.at/2d/english/index%2De.htm. [24]MOMENTS project, http://www.zgdv.de/ departments/z3/mmivs/projects/Moments/moments.html [25]MPEG-4 standard, http://drogo.cselt.stet.it/mpeg/ standards/mpeg-4/mpeg-4.htm. [26]R. Pajarola and J. Rossignac, Compressed Progressive Meshes, GVU technical report GIT-GVU-00-04 , ftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-04.pdf. [27]W. Pasman and F.W. Jansen, Latency layered rendering for mobile augmented reality. Proceedings of the 21th symposium on information theory, May 25-26, 2000, Wassenaar, The Netherlands. http:// wwwcg.twi.tudelft.nl/~wouter/publications/Pasman 00a.pdf. [28]Paula project page, http://paula.oulu.fi/. [29]Psion PLC. Street Planner '99 software, http://www.psion.com/. [30]I. Rakkolainen, S. Pulkkinen, and A. Heinonen, Visualizing Realtime GPS Data with VRML Worlds. Proceedings of the ACM-GIS'98 workshop, pp. 52-56 (November 1998).

[31]A. Raposo, L. Neumann, L. Magalhães, and I. Ricarte, Visualization in a Mobile WWW Environment. Proceedings of the WebNet 97 - World Conference of the WWW, Internet, and Intranet, November 1997. [32]J. Rossignac and P. Borrel, Multi-Resolution 3D Approximations for Rendering, in Modeling in Computer Graphics, Springer-Verlag, 1993, pp. 455-465. [33]B.-O. Schneider and I. Martin, An Adap tive Framework for 3D Graphics in Networked and Mobile Environments. Proceedings of the Symposium on Interactive Applications for Mobile Comput ing, IMC'98, Rostock, Germany, November 1998. [34]T. Sephton, J. Black, G. El Naggar, and A. Fong, A Mobile Computing System for Testing Wearable Augmented Reality User Interface Design. http:// www.funhousefilms.com/MARS/ISWC99e.htm [35]A. Smailagic and R. Martin. Metronaut: A Wearable Computer with Sensing and Global Communication Capabilities. Proceedings of the International Symposium on Wearable Computing, Oct. 1997. [36]T3D software, http://www.neuon.com/apps/t3dtest/. [37]The Virtual Reality Modeling Language Specification. http://www.vrml.org/Specifications/VRML97/ [38]VRML Enterprise technology working group. http://www.vrml.org/WorkingGroups/dbwork/. [39]Web3D Consortium, http://www.web3d.org/. [40]T. Whitted, Personal Image Generators. In Digital Illusions, Dodsworth C., editor, Addison-Wesley, Reading MA, 1997, 261-273. [41]WorldStreamer software, Mediaclick Oy, http://www.virtuaaliturku.fi/. [42]Xanavi Birdview navigation system, http://www.xanavi.co.jp/en/nav/index.html. [43]S. You, U. Neumann, and R. Azuma, Hybrid Inertial and Vision Tracking for Augmented Reality Registration. Proceedings of IEEE Virtual Reality 1999, 260-267.