Reality-based generation of virtual environments for ... - CiteSeerX

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth Vol. 1, No. 1, March 2008, 88
106

Reality-based generation of virtual environments for digital earth A. Gruen Institute of Geodesy and Photogrammetry, ETH Zurich, Switzerland Digital Earth essentially consists of 3D and moreD models and attached semantic information (attributes). Techniques for generating such models efficiently are required very urgently. Reality-based 3D modelling using images as prime data source plays an important role in this context. Images contain a wealth of information that can be advantageously used for model generation. Images are increasingly available from satellite, aerial and terrestrial platforms. This contribution briefly describes some of the problems which we encounter if the process of model generation is to be automatised. With the help of some examples from Digital Terrain Model generation, Cultural Heritage and 3D city modelling we show briefly what can be achieved. Special attention is directed towards the use of model helicopters for image data acquisition. Some problems with interactive visualisation are discussed. Also, issues surrounding R&D, professional practice and education are also addressed. Keywords: photogrammetry; 3D modelling; digital terrain models; 3D city modelling; cultural heritage; texture mapping; visualisation

Introduction The notion of ‘Digital Earth’ has recently found much attention, not only in the scientific communities, but also in political, economical and ecological circles and in the general public. While a static model of a Digital Earth is necessarily the starting point for any data analysis, we must nowadays consider more and more the importance of change, of natural and man-made processes, which also means that we should turn our R&D interests more towards the dynamic use of such models. In this context, the ‘Digital Earth’ may also be called a ‘Virtual Earth’, incorporating the active interaction of people with the digital models and even the immersion of people and other living subjects into augmented reality structures. In any case, the generation, administration, analysis, representation and manipulation of three-dimensional (3D) models, describing elements of and processes on Digital/ Virtual Earth are very essential tasks and require the attention and input of all geo-related sciences. In recent years, imaging techniques have made tremendous progress technically and have infiltrated almost all areas of science, technology and art. The fascination of images is that ‘they say more than a thousand words’. Images contain a wealth of information, which is unmatched by any other information-carrying device. Images are utilised not only for artistic, entertainment and leisure-related activities, but more and more for technical and professional applications. All of today’s key sciences-life Email: [email protected] ISSN 1753-8947 print/ISSN 1753-8955 online # 2008 Taylor & Francis DOI: 10.1080/17538940701782585 http://www.informaworld.com

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

89

sciences, nano sciences, information technology (IT) and environmental sciencesdepend very much on imaging technologies of various kinds. Geospatial sciences also increasingly make use of imaging technologies. The success of Google Earth and Microsoft Virtual Earth lies in the fact that they give fast access to worldwide highresolution image-related information. Images may come from a wide range of different sensors and platforms. The recent exploration activities on Mars and other planets and space bodies rely heavily on the use of image-based sensors, working both in orbit and on the surface. Earth observing satellite platforms carry increasingly high-resolution imaging sensors with stereo capabilities. Digital aerial cameras of various types are collecting images at an unprecedented speed and amount. For instance, Pictometry is currently collecting oblique aerial images with eleven aircrafts and as many 5-camera systems over 900 European towns. This will result in 1.1 GB of imagery for every km2 at a resolution of 12
15 cm and a positioning accuracy of 50 cm. Various camera types are used in terrestrial modes by non-expert people. In Japan, alone 43.5 Million 3G phones have been sold in 2006. 45 out of 98 models had global positioning system (GPS) capabilities for self- and remote location tasks and almost all included digital cameras. What this all amounts to in terms of protection of privacy and public and private security issues is not very clear yet. It has lately become known for instance that Google Earth has been used by terrorists to target British troops in Iraq. Hardcopy Google Earth maps are sold openly in Basra markets. Since quite some time, imaging techniques are not restricted any more to the use of photographic cameras or not even to the visible part of the electromagnetic spectrum. Nowadays, photogrammetry and remote sensing are better defined as image-based modelling techniques, which allow for the extraction of both geometrical and semantic information from images. Efficient (accurate, reliable and fast) processes of transforming raw image data into value-added 3D model information are nowadays of utmost importance for the creation of geospatial databases. In this article, we will focus on the use of aerial and satellite images for 3D model generation. We must note however, that there is a large amount of work being done in image-based modelling for applications not directly related to the issues discussed here: Industrial inspection and quality control, robotics and navigation, animation for movies and TV shows, computer games, medical and forensic applications, sports and biomechanics, bio- and nano-sciences, etc. Image-based techniques are expanding at a remarkable pace. Most of these applications do have a spatial relation, although they are not always ‘geo’- related in the traditional sense. Data acquisition and processing For data acquisition, we use a great number of different sensors: Film cameras, special charge coupled device (CCD) and complementary metal
oxide
semiconductor (CMOS) cameras, still video cameras, camcorders, linear array cameras of various types, among them the latest multi-line-scanners and digital panoramic cameras, laser scanners, microwave and ultrasound sensors, X-ray

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

90

A. Gruen

devices and electronic imaging devices, and all sorts of combined and hybrid systems. On the data processing side, we had to distinguish in the past three lines of system development, according to the three major sensor platforms and application areas: Satellite remote sensing, aerial photogrammetry and closerange photogrammetry, leading to different kind of systems with greatly varying functionality. Today, we see a tendency towards integrating all functions under one unique system: the Digital Station. Given the flexible nature of such platforms, also radiometric manipulations, special remote sensing software, 3D modelling functions, database functionality, data analysis procedures, visualisation and animation functions and all kind of third party software can be integrated or connected to. This makes a Digital Station a truly universal system for data processing, administration, analysis and representation. Needless to say that in reality not all desired functionality is available yet and there is much room for future improvements and developments. Figure 1 shows the workflow of the photogrammetric techniques used to turn aerial and satellite images into hybrid 3D models (‘hybrid’ means mixed models including geometry and texture).

Figure 1. Workflow and products of the photogrammetric/remote sensing processes for 3D modelling from aerial and satellite images.

International Journal of Digital Earth

91

Research and development Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

Some key issues With the great changes in technology come many challenges for research and development: 1. New sensors: The development and use of new sensors requires the study and test of innovative sensor models, and the investigation of the related network structures and accuracy performance. Of particular interest are here highresolution satellite and aerial linear array cameras, terrestrial panoramic cameras and laser scanners. 2. Sensor and data integration: The combination of different sensors and their related datasets requires new approaches in sensor modelling and in the combined processing of different kind of data. Combinations of cameras and GPS/INS, cameras of various types and laser scanners, optical images and radar data and the like are of particular interest. Also, the appropriate use of a priori data information, e.g. from geographic information system (GIS), for image analysis is of relevance. 3. On-line and real-time processing: The need for very fast processing requires algorithmic redesigns in many areas. Sequential estimation methods present a suitable tool to tackle some of those problems. 4. 3D modelling: Our environment is essentially three-dimensional. With digital technology the traditional approach of 2.5D modelling should be overcome in favour of consequent 3D modelling. Here many new problems are emerging in measurement, surface modelling, topology generation and data model definition. 5. Image understanding; The overriding research issue today is the automation of all image analysis functions, from the orientation/georeferencing processes to image matching and feature and object extraction. While we have seen some progress lately in those procedures where mainly geometric issues are dealt with, the great problem to be solved is automated image interpretation. Methods of artificial intelligence have not delivered the promised performance and we observe a certain stagnation in the development of image understanding algorithms. Therefore many researchers have turned towards semi-automated approaches, where the human capacity in image content interpretation is paired with the speed and precision of computer-driven measurement algorithms. Another approach to advance is the use of hybrid sensors, in order to collect as many different cues as possible. The status of automated processing The automation of photogrammetric processing is obviously an important factor when it comes to efficiency and costs of data processing. The success of automation in image analysis depends on many factors and is a hot topic in research. Progress is slow and the acceptance of results depends on the quality specifications of the user. Also, the image scale plays an important role in automation. Potentially, the smaller the scale the more successful automation will be. Therefore it is a bit difficult to make firm statements which would be valid in all cases. However, in general one can state that:

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

92

A. Gruen . orientation and georeferencing can be done in parts automatically; . digital surface models (DSM) generation can be done automatically, but may need substantial postediting; . orthoimage generation is a fully automatic process; . object extraction and modelling is possible in semi-automated modes at best. . Since object extraction and modelling constitute very important elements in many applications we will give some specific comments on that in the following.

Object extraction and modelling In commercially available digital photogrammetric software, object extraction functionality is restricted to manual or semi-automated measurements together with the capability of attribute data acquisition. The main applications are 3D modelling of city and industrial areas. Commercial systems assist the human operator in measuring 3D objects in combination with registration of attribute data in a semi-automated mode, e.g. Leica Photogrammetry Suite, Z/I Image Station or Virtuozo IGS Digitize. These systems provide libraries containing objects, e.g. buildings or streets, which allow for object modelling according to certain rules concerning object topology. For the 3D modelling of buildings and other man-made objects we have developed and tested CyberCity Modeler (CC-Modeler, see Gruen and Wang 1998, also Figure 7). This is a semi-automated technique, where the operator measures manually in the stereomodel a weakly structured pointcloud, which describes the key points of an object. The software then turns this pointcloud automatically into a structured 3D model, which is compatible with computer aided design (CAD), visualisation and GIS software. Texture can be added to the geometry to generate a hybrid model. A DTM can also be integrated. An example using CyberCity Modeler for 3D modelling of terrain and buildings in an archaeological application was conducted for the pre-hispanic site of Xochicalco, Mexico, where an ancient urban center was reconstructed photogrammetrically from aerial images (Gruen and Wang 2002, see Figure 9). Main applications and products Currently we have four types of products in which geo-related 3D modelling is of particular importance: . . . .

digital terrain (surface) models; 3D city models; cultural and natural heritage models; special models (for forest/vegetation, water, etc.).

The importance of terrain and city models is obvious. They form the basis for any geo-related studies and developments in our natural and man-made environments. In Heritage applications we distinguish four areas, for which model generation is essential:

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth . . . .

93

larger sites, ensembles; single structures/buildings; statues and other objects; artefacts, smaller findings.

The modelling of large sites has received much attention in recent years. This was triggered on the one side by the increased interest of UNESCO and other supranational and national organisations, and on the other side by the new technologies available for recording, processing, administration and visualisation of the data. As can be see from the UNESCO World Heritage List (whc.unesco.org), many recent additions can actually be classified as ‘Large Sites’, both in terms of culture and nature. In a press release (No. 2002-77: ‘For UNESCO, Space Technologies should be Harnessed for Sustainable Development’) UNESCO has stressed the use of satellite imagery for monitoring World Heritage sites. Lately, many conferences are devoted to this issue (e.g. the International Symposium on ‘Conserving Cultural and Biological Diversity: The Role of Sacred Natural Sites and Cultural Landscapes’, 30 May2 June in Aichi, Japan, on occasion of the World Exhibition 2005). Conservation and management of these sites rely heavy on the availability and timeliness of data. We have conducted a number of projects in the past that have shown the potential of some of the new recording, processing and modelling techniques (see also www.photogrammetry.ethz.ch under PROJECTS). Among those are: . Large sites: Mount Everest (Gruen and Murai 2002), Ayers Rock/Australia, Kunming Region/China (Zhang et al. 2002), Bamiyan/Afghanistan (Gruen et al. 2004a, 2004b, 2005), Geoglyphs of Nasca/Peru (Lambers and Gruen 2003, Reindel and Gruen 2005), Tucume/Peru (Sauerbier et al. 2004), Inka settlement Pinchango Alto/Peru (Eissenbeis et al. 2005), Machu Picchu/Peru (in work), Petroglyph site of Chichictara/Peru, Xochicalco/Mexico (Gruen and Wang 2002). . Single structures/buildings: Rock church Bet Georgis of Lalibela/Ethiopia (Buehrer et al. 2001), Bayon Tower, Angkor/Cambodia (Visnovcova et al. 2001) . Statues and other objects: Weary Herakles Antalya Museum/Turkey, Khmer Head, Rietberg Museum Zurich/Switzerland (Akca et al. 2007), Pfyffer Relief, Luzern/Switzerland (Niederoest 2003), A. Escher Statue, Zurich/Switzerland, St. Gallen Globe, Zurich/Switzerland. For statue-type objects and smaller items we have made very good experiences with the use of active (especially structured light, but also time of flight) optical systems (Akca et al. 2007). Virtual Museums are a major user of some of these techniques. Terrain model/natural heritage site generation The terrain is a key element in all geo-related applications and investigations. Therefore, 3 D modelling of terrain is an ever-relevant issue. The status of terrain modelling varies worldwide very much. Although there exist already worldwide DSMs, e.g. Shuttle Radar Topography Mission (SRTM)-based, they show the terrain

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

94

A. Gruen

only in 2.5D representation, have many gaps and are partially very inaccurate. The need for more detailed modelling is obvious in many applications. Mapping efforts are underway in many countries. Sometimes, especially for smaller regions or countries (e.g. Switzerland) LiDAR is used, giving an accuracy of 0.5 m in open terrain and 1.5 m in vegetation-covered terrain. In larger projects, satellite stereo images are combined with aerial images in order to generate new DSMs over vast areas. In China, for instance, the project ‘Mapping of the Western Territories’ is underway, which calls for the generation of about 5032 sheets of 1:50 000 scale topographic maps within a 5-year period, covering an area of 2 Million km2, which is about 20% of the total area of China. It is very clear that such efforts can only be successful if the required data can be generated in an automated or at least semi-automated way. Although image matching has a long history of R&D, the problem is not fully solved yet. New methods for image matching, as implemented for instance in our software SAT-PP (Gruen and Zhang 2003, 2004, Zhang 2005, Zhang and Gruen 2006), have led to some progress and deliver much better results than the current commercial packages. In many pilot projects, we have recently shown that DSMs can be generated fully automatically with an accuracy of 1
5 pixels from stereo satellite imagers (Aster, SPOT-5, IKONOS, Quickbird, ALOS/PRISM). Similarly controlled results are only sparsely available for aerial cameras and do not allow yet for generalised conclusions. In a model helicopter flight over an archaeological site in Pinchango Alto, Peru we have automatically processed digital still video images and compared with terrestrial laserscanning data. We achieved a height RMSE of 6 cm, given an image pixel size of 4 cm. Figure 2 shows the automatically generated photo-textured DSM. Sometimes DSMs may be derived from already existing analogue data. We produced a textured 3D model of Mount Everest by scanning existing contour plans 1:10 000 and converting them into a raster DSM of 10 m grid width. The scanned aerial photographs of scale 1:35 000 were geo-referenced and texture-mapped onto the DSM with a resolution of 1m (Gruen and Murai 2002). Figure 3 shows a view onto our Everest model, which stretches over an area of 2525 km2. Certain studies may require the analysis of changes occurring over time. Imagebased techniques allow to go back in time and process existing older images. Our

Figure 2. Photo-textured DSM of the archaeological site of Pinchango Alto, Peru, generated automatically by image matching.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

95

Figure 3. Mount Everest model (size 25 25 km2), derived from analogue data (contour plans and aerial photographs); view onto the summit from North.

next example (Figure 4) shows a 3D model of the archaeological site of Tucume, Peru, which was produced from aerial images from the year 1949. Sometimes images from different platforms, sensors and times may have to be combined in 3D modelling. Our Bamiyan, Afghanistan project is such an example (Gruen et al. 2004a, 2004b, 2005). It is a combination of multi-platform, multiresolution and multi-temporal photogrammetric data. We have used SPOT 5-HRG stereos for DTM generation of a large area of ca. 4938 km2 and IKONOS images for texture mapping. Rollei and Sony Cybershot images have been used in a terrestrial mode to model the 1 km long rock facade with all the Buddhist caves and the now empty Buddha niches. Old glass plates of format 13 18 sqcm have been

Figure 4. View onto the 3D model of the Tucume adobe complex, Peru, as it existed in 1949. To the left is Huaca Larga, a huge adobe building of 545 m length, with an Inka stone building on top.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

96

A. Gruen

used for the computer reconstruction of the Great Buddha. Standard small format tourist photos have been applied to reconstruct some of the now destroyed frescos. The geometric resolution of the recovered 3D data spans from 20 m (SPOT 5) to 5 cm (Buddha model), while the texture information is between 2.5 m (SPOT 5) and 2 mm (fresco) resolution. A maximal factor 400 exists between the different geometry resolutions, while there is a factor 1250 in the texture. Table 1 gives an overview about the different images used from different cameras and times. Figure 5 shows a view onto the IKONOS-textured DTM of larger Bamiyan, while Figures 6(a) and (b) show the current situation of the empty niche and the computer-reconstructed Great Buddha. The whole triangulated surface model contains more than 4 millions triangles, while the texture occupies ca. 2 GB. The fusion of the multi-resolution (and multitemporal) data is a very complex and critical task. Currently, there is no commercial software able to handle all these kinds of data at the same time, mainly for these reasons: . The data are a combination of 2.5 and 3D geometry, limiting the use of packages for geodata visualisation, usually very powerful for large site textured terrain models . The amount of data is too big for graphical rendering and animation packages, generally able to handle textured 3D data . The high-resolution texture information exceeds the memory capacity of most current graphic cards. 3D City Modelling It is only over the last 15 years that photogrammetric approaches to building extraction and modelling have evolved. What started out as a pure research issue has now found firm grounds in the professional practice. After the first phase of efforts to extract buildings fully automatically the tight specifications of users have led to the development of efficient manual and semi-automated procedures. Actually, the need to extend modelling from simple to much more complex buildings and full ensembles and to even generate complete city models (including DTM, roads, bridges, parking lots, pedestrian walkways, traffic elements, waterways, vegetation objects, etc.) puts fully automated methods even further back in the waiting line of technologies for practical use. In a sense, the user requirements have outpaced the Table 1. Multi-resolution data (geometry and images) used in the Bamiyan Project. Source of data Satellite images Terrestrial images

SPOT 5-HRG IKONOS Rollei Sony Cybershot [Kostka 1974] Tourist photos (Frescos)

Year

Image resolution

Geometry resolution

Texture resolution

2003 2001 2003 2003 1970 60s & 70s



20 mm 4 mm 10 mm 20 mm

20 m 5m 1m 50 cm 5 cm N.A.

2.5 m 1m 50 cm 10 cm 1 cm 2 mm

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

97

Figure 5. View onto the Bamiyan DTM of 49 38 km2 extension, derived from SPOT 5HRG stereos and partially textured with IKONOS images.

capabilities and performance of automated methods. However, to make it clear, automation in object extraction from images is still and will continue to be a key research topic. There are many fully automated approaches to building extraction, but only very few which were designed as semi-automated ones from the very beginning. Very often, procedures are declared as automatic but require so much post-editing that their status as automatic methods becomes questionable.

Figure 6. (a) (left). 3D model of the current empty niche of the Great Buddha, derived from Sony Cybershot images. (b) (right). Reconstructed 3D model of the Great Buddha from old photogrammetric glass plates.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

98

A. Gruen

Applications of city models are manifold. Currently, the major users in Europe are in city planning (see Figure 7), facility mapping (especially for chemical plants and car manufacturers), telecommunication, construction of sports facilities and other infrastructure buildings. Others include environmental studies and simulations, location-based services (LBS), risk transports and analysis, car navigation, simulated training (airplanes, trains, trams, etc.), energy providers (placement of solar panels), real estate business, virtual tourism, and microclimate studies. Interesting markets are expected in the entertainment and infotainment industries, e.g. for video games, movies for TV and cinema, news broadcasting, sports events, animations for traffic and crowd behaviour, and many more. Divers applications require different levels of detail in modelling, a great variety of different objects to be extracted and the handling of different data types and manipulation functions. Therefore, when designing an efficient method for object extraction and modelling the following requirements should be observed: Extract not only buildings, but other objects as well Generate truly 3D geometry and, if a GIS platform is used, topology as well Integrate natural image texture (for DTM, roofs, facades, and special objects) Allow for object attributation Keep level of detail flexible Allow for a wide spectrum of accuracy levels in the cm- and dm- ranges Produce structured data, compatible with major CAD and visualisation software . Provide for internal quality control procedures, leading to absolutely reliable results . . . . . . .

We see currently three major techniques, which are used in city model generation:

Figure 7.

3D model of campus Hoenggerberg of ETH Zurich.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

99

a. Digitisation of maps. This gives only 2D information. The height of objects has to be approximated or derived with great additional efforts. It does not provide for detailed modelling of the roof landscape. This roof landscape is usually very important to the user, because city models are mostly shown from an aerial perspective. Also, map data is often outdated. b. Extraction from aerial laser scans. Laser scans produce regular sampling patterns over the terrain. Most objects in city models are best described by their edges, which are not easily accessible in laser scans and often cannot be derived unambiguously. Some objects of interest do not distinguish themselves through height differences from their neighborhood, can thus not be found in laser data. Finally, the resolution of most laser scan data is not sufficient for detailed models. c. Photogrammetric generation. Aerial and terrestrial images are a very appropriate data sources for the generation of city models. They allow to derive both the geometrical and the texture models from one unique dataset. The photogrammetric technique is highly scalable, it can adapt to required changes in resolution and accuracy in a flexible way. The processing of new images guarantees an up-to-date model. Images are a multipurpose data source and can be used for many other purposes as well. As a result of this brief analysis, the photogrammetric approach must be considered the most relevant technique. A scheme for image-based reconstruction of a hybrid city model is shown in Figure 8. Hybrid refers to the fact that both vector and raster data can be represented by the model. According to this scheme roof landscapes, DTM, transportation elements, land-use information, etc. can be extracted from aerial images. Combining roofs and DTM will result in building vector models. These models could be refined by using terrestrial images, taken with camcorders or still video cameras. Aerial images, terrestrial images and digitised maps can all contribute to the texture part of the hybrid model. It is also well-known that to a certain extent texture information can compensate for missing vector data. GENERATION OF HYBRID CITY MODEL Aerial images

Extraction

Roofs

DTM

Traffic, etc.

Landuse Geometrical reconstruction

Building models

Vector models

Pixel data Hybrid model

Figure 8.

Geometrical city model Facade models

Roof models

Terrestrial images

Traffic, DTM Landuse etc. Aerial images

Maps

Hybrid city model

Image-based reconstruction of a hybrid city model.

Textural reconstruction

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

100

A. Gruen

Since fully automated extraction methods cannot cope with most of the aforementioned requirements, semi-automated photogrammetric methods are currently the only practical solutions of choice. There is one relevant semi-automated approach for city modelling available: CyberCity Modeler (CC-Modeler, see Gruen and Wang 1998, Gruen et al. 2003). This is a method and software package which fits planar surfaces to measured and weakly structured point clouds, thus generating CAD-compatible objects such as buildings, trees, waterways, roads, etc. Usually these point clouds are taken from aerial images, but it is also possible to digitise them from existing building plans. This product is marketed by CyberCity Inc., Urdorf, Switzerland, a spin-off company of ETH Zurich. In the meantime, in excess of 1 Million buildings have been generated worldwide and it has also been shown that CC-Modeler can effectively be used for 3D modelling of archaeological sites (see Xochicalco, Mexico, Figure 9). Model helicopter photogrammetry The use of model helicopters for aerial image acquisition has reveived much attention lately. Our model helicopter belongs to the class of mini-UAVs (unmanned autonomous vehicles). UAVs are designed to operate without a human pilot onboard. Mini helicopters have typically a size of up to 2 m (rotor diameter), a flight time of up to one hour and can operate up to 1 km from the base station. Mini helicopters are highly maneuverable, owing to their capability to hover and to change flight direction around the center of rotation. Our helicopter is equipped with GPS/INS for positioning and attitude measurements and has a stabilising platform. Any low cost still-video and video camera can be attached and turned in horizontal and vertical directions. Owing to the flexibility of such systems they can cover vertical and oblique aerial and terrestrial applications using only one platform for image acquisition. Furthermore, because of the small size of the system and the precise maneuverability it is possible to fly even into restricted spaces.

Figure 9. 3D model of the pre-hispanic site of Xochicalco, Mexico; derived semiautomatically by CC-modeller from an aerial image stereo-pair.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

101

These vehicles can be used in a great variety of applications. Lately, we have applied such a system to Cultural Heritage modelling, in agriculture and geology applications. In our opinion, these systems do have a great potential for all sorts of recording tasks for relatively small areas. Here we will briefly report about an archaeological modelling task executed over the site of Pinchango Alto in Peru. Pinchango Alto is located close to Palpa, next to the famous Nasca geoglyphs. It dates back to the Late Intermediate Period (AD 1000
1400), a still poorly understood pre-Inkaic period. Access to and working on the site is rather difficult, because it is located in a remote and harsh area. In a 2004 field campaign we used our model helicopter carrying a CMOS camera to acquire a series of 90 vertical aerial images for photogrammetric recording and 3D modelling of the site and the surrounding terrain. While the GPS/INS unit enables semiautomated navigation along a predefined flight path, the stabiliser ensures a stable flight attitude and thus highly reliable image acquisition and overlap. The processing and analysis of the acquired images encompassed image pre-processing, semiautomatic triangulation and automated DTM generation. A hybrid 3D model of the site was produced and visualised. The results were analysed concerning in particular the potential of DSM generation from model helicopter images as compared to terrestrial laserscan data. For details of the whole mission see Eissenbeis et al. 2005. Figure 2 shows the automatically derived Digital Terrain Model. Visualisation of 3D models Visualisation of 3D models is an essential function. A model which cannot be seen at all or which can only be seen with great time delay is losing much of its value. Software packages for terrain visualisation are abundantly available. In http://www.Tec.army.mil/TD/tvd/survey/survey_toc.html for example we can access a list of about 550 software packages just for terrain visualisation. More packages are lingering in various research labs. Although the conceptional aspects of computer graphics algorithms are quite straightforward, it is always the implementation and the quality of the key components of the computer platform which define the performance. Geovisualisation packages are complex software systems, with strong dependencies on the hardware as well. In order to represent an efficient system, all components have to perform well individually, but also their interaction must be solved in an acceptable manner. When analysing visualisation software a major consideration is whether interactive, or even real-time performance is required or not. The fascination of real-time performance is intriguing enough so that most users, once they have been exposed to it, will not want to do without it. Also, for many analysis applications, real-time performance is just a must for the sake of economy and efficiency of operation. One can classify visualisation software on the basis of its real-time performance, given a certain computer configuration. In this context one can distinguish high-end, middle class and low-end systems (e.g. Skyline http://www.skylinesoft.com, IMAGINE VirtualGIS http://www.erdas.com/software/ProductModules.asp, and Cosmo Player http://cai.com/cosmo, in this order). While low-end software is increasingly

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

102

A. Gruen

available as freeware over the Internet, the other levels of quality can only be reached by paying, in parts dearly, for the software. There are many more quality criteria which can be applied, like acceptance of public data formats for input, rendering quality of geometry and texture, shading model, anti-aliasing, trajectory definition, illumination and animation functions, video/DVD production capabilities, etc. Observing that computer power and graphics board performance is increasing dramatically over the last years one can nowadays expect even from laptops a rendering performance which was unheard of only 3-5 years ago. High-end performance of photo-textured 3D models can be obtained on laptops if the software supports that. One critical parameter here is the level-of -Detail (LoD) property. LoD capability ensures that at each and every frame of an image sequence only the foreground portion of the 3D model is represented at highest resolution. Other zones of model depth are represented at lower resolution. This reduces the amount of computations substantially, an advantage, which becomes the more prominent the bigger the model is. This LoD property applies both to vector and image raster data. The interested user should also pay attention to the fact whether the different resolution layers can be produced effortlessly by him-/herself or whether a major effort is needed. We have quite some experience with commercial visualisation software. In Gruen and Roditakis 2003 we have reported our experiences with commercial visualisation packages like Cosmoplayer, ERDAS VirtualGIS, TerrainView, Skyline and MAYA using our Mount Everest dataset. None of them, when used in interactive mode, showed really satisfying performance. Although very many terrain visualisation packages are available worldwide, there is still the need for development of more efficient software, combining ease of use with speed and quality of rendering for large and very large datasets. However, one must note that much progress has been achieved in the past few years. Yet, the interested user of such packages is strongly advised to check the performance beforehand by using his/her typical datasets. While certain packages may perform quite well when only 2.5D terrain data is used, they may exhibit problems with truly 3D data, especially if vertical building faces are textured. Conclusions and outlook In the past years, image-based modelling techniques like digital photogrammetry and remote sensing have opened many new areas of applications. We have shown here how high-resolution satellite images, aerial and terrestrial images can be used in order to generate hybrid 3D models for many divers applications. The digital nature of many of those images and the progress in automatic photogrammetric processing allows for very efficient procedures and for new kinds of results. Additional options for recording and processing are available through the use of aerial and terrestrial laser scanners, panoramic cameras and combined systems. Of particular interest is a UAV (unmanned aerial vehicle)-a model helicopter, which works in an autonomous mode, based on integrated GPS/INS, stabiliser platform and digital cameras, and which can be used to get images from otherwise hardly accessible areas. This system, together with advanced software for automated processing will allow us in the near future to generate at least an initial

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

103

model of the object fully automatically on-line in the field or immediately after data collection in the campaign office. With the recent expansion of photogrammetry’s data acquisition tools (sensors) and processing techniques we see many more novel applications emerging in the close-range area. The generation of reality-based data for virtual environments, animation, video gaming and the like constitutes a huge potential for future work. In fact, there are many recent movies (The Lord of the Rings, Matrix, etc), where photogrammetry has been used extensively, especially for the recording of 3D movements of bodies (motion analysis) and the tracking of face expressions. The pressing need for geo-related modelling of our 3D environment (3D city and terrain modelling) from aerial and high-resolution satellite images and laser scanners will have a tremendous impact in natural hazard damage monitoring, risk analysis, car navigation with 3D models, location-based services, virtual tourism, and in many more applications. With the new generation of high-resolution satellite sensors (SPOT, ALOS/ PRISM, IKONOS, Quickbird) the issue of three-dimensional modelling is gaining much more prominence. Therefore, photogrammetric techniques are also becoming more important in satellite image applications. On the other hand, radiometric analyses are also attaining more attention in photogrammetry. We observe that the originally different techniques in remote sensing and photogrammetry are converging today strongly. Within the image-related sciences computer vision, robot vision, remote sensing, visualisation, simulation, animation and spatial information science, modern photogrammetry has managed to position itself as an indispensable member, whose specific procedures and techniques are required for problem solving. The impacts of these developments and requirements on the profession are manyfold, out of which questions arise such as: . How do we handle the inflation of data, in particular images? Our processing capabilities are already today trailing way behind the data generation rate. . How does the increased system complexity affect our daily production work? It would be a great mistake to assume that photogrammetry can be handled as a black-box approach. Most of the procedural aspects of photogrammetry are much too complex that they can be left to untrained personal. Only a good education of the operators can ensure that photogrammetry is executed properly and that the results are of high quality and reliability. . How do we cope with the competition from neighbouring disciplines? Depending on our own capabilities, flexibilities and attitudes, our discipline will either disappear or emerge with greater strength than ever before. We have shown that photogrammetry and remote sensing have expanded their techniques very much in recent years. This has opened many new fields of applications. There is no good reason why this process should not continue in the years to come. Already now, but even more in the near future, we are and will be flooded by huge amounts of images, emerging from satellite, aerial and terrestrial platforms. A good deal of those images will have to be processed using photogrammetric techniques. This is why we see a very bright future for photogrammetry, in research, development and with respect to practical applications as well.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

104

A. Gruen

All these presented technologies, together with Spatial Information Systems, 3D modelling, visualisation and animation techniques are still in a dynamic state of development, with even better application prospects for the generation and analysis of datasets for Digital/Virtual Earth in the near future. References Akca, D., Gruen, A., Breuckmann, B. and Lahanier, Ch., 2007. High definition 3D-scanning of arts objects and paintings. Proceedings ‘Optical 3-D Measurement Techniques’, 9
12 July, Zurich, pp. 50
58. CD-ROM. Buehrer, Th., Gruen A., Zhang, L., Fraser, C. and Ruther, H., 2001. Photogrammetric reconstruction and 3D visualisation of Bet Giorgis, a rock-hewn church in Ethiopia. Proceedings of the CIPA 2001 International Symposium on Surveying and Documentation of Historic Buildings, Monuments and Sites, 18
21 September, Potsdam, Germany. Eissenbeis, H., Lambers, K., Sauerbier, M. and Zhang, L., 2005. Photogrammetric documentation of an archaeological site (Palpa, Peru) using an autonomous model helicopter. Proceedings CIPA International Symposium. 26 September
1 October, Torino, Italy. CD ROM. Gruen, A. and Murai, Sh., 2002, High-resolution 3D modeling and visualization of Mount Everest. ISPRS journal of photogrammetry and remote sensing, 57, pp. 102
113. Gruen, A., Remondino, F. and Zhang, L., 2004a, Photogrammetric reconstruction of the Great Buddha of Bamiyan. Photogrammetric record, 19 (107), 177
199. Gruen, A., Remondino, F. and Zhang, L., 2004b. The Bamiyan Valley: landscape modeling for Cultural Heritage visualization and documentation. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), Vol. 34, Part 5/W1. Proc. of the International Workshop on Processing and Visualization using High Resolution Imagery. Pitsanulok, Thailand, November 2004. CD-ROM. Gruen, A., Remondino, F. and Zhang, L., 2005. The Bamiyan project: multi-resolution imagebased modeling. Proceedings of the International Workshop ‘Recording, Modeling and Visualization of Cultural Heritage. 22
27 May, Centro Stefano Franscini, Monte Verita, Ascona, Switzerland. CD-ROM. Also Taylor & Francis 2006, E. Baltsavias, A. Gruen, L. van Gool and M. Paterakis, eds. pp. 45
55. Gruen, A. and Roditakis, A., 2003. Visualization and animation of Mount Everest. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences) 34, Part 5/W10. Proceedings of the International Workshop on Visualization and Animation of Reality-Based 3D Models. 24
28 February, Tarasp-Vulpera, Switzerland, on CD-ROM. Gruen, A. and Wang, X., 1998. CC-modeler: a topology generator for 3D city models. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 32, Part 4, pp. 188
196, ISPRS Commission IV Symposium on GIS-Between vision and application. Stuttgart/Germany, 7
10 September, ISPRS Journal of Photogrammetry and Remote Sensing, 1998, 53, 286
295. Gruen, A. and Wang, X., 2002. Integration of landscape and city modeling: The pre-hispanic site Xochicalco. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), Vol. 34, Part 5/W3, International Workshop: Visualization and Animation of Landscape. Kunming, China, 26 February
1 March. CD-ROM. Gruen, A. and Zhang, L., 2004. Automatic DSM generation from Linear Array imagery data. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 35, Part B3, XXth ISPRS Congress. Istanbul, Turkey, 12
23 July, pp. 128
133. Gruen, A. and Zhang, L., 2003. 3D processing of high-resolution satellite images. Proceedings Asian Conference on Remote Sensing 2003, Busan, Korea, 3
7 November. CD-ROM.

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

International Journal of Digital Earth

105

Gruen, A., Zhang, L. and Wang, X., 2003. Generation of 3D city models with linear array CCD-sensors. Proceedings International Conference on Optical 3D Measurement Techniques. Zurich, September, Vol. II, pp. 21
31. Lambers, K. and Gruen, A., 2003. The geoglyphs of Nasca: 3D recording and analysis with modern digital technologies. Acts of the XIVth UISPP Congress, University of Liege, Belgium, 2
8 September 2001. Section 1: theory and methods
general sessions and posters (BAR International Series 1145), Archaeopress, Oxford, pp. 95
103. Niederoest, J., 2003. A birds’s eye view on Switzerland in the 18th century: 3D reconstruction and analysis of a historical relief model. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 34, Part 5/C15, 589
594. Reindel, M. and Gruen, A., 2005. The Nasca-Palpa project: a cooperative approach of archaeology, archaeometry and photogrammetry. Proceedings of the International Workshop: Recording, Modeling and Visualization of Cultural Heritage, Centro Stefano Franscini, Monte Verita, Ascona, Switzerland, 22
27 May. CD-ROM. Taylor and Francis, 2006. E. Baltsavias, A. Gruen, L. van Gool and M. Paterakis, eds, pp. 21
32. Sauerbier, M., Kunz, M., Fluehler, M. and Remindino, F., 2004. Photogrammetric reconstruction of adobe architecture at Tucume, Peru. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 36, Part 5/W1. Proceedings of the International Workshop on Processing and Visualization using High Resolution Imagery. Pitsanulok, Thailand, November 2004. CD-ROM. Visnovcova, J., Zhang, L. and Gruen, A., 2001. Generating a 3D model of a Bayon tower using non-metric imagery. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 34, Part 5/W1, Proceedings of the International Workshop: Recreating the Past
Visualization and Animation of Cultural Heritage. Ayutthaya, Thailand, 26 February
1 March, pp. 30
39. Zhang L., 2005, Automatic digital surface model (DSM) generation from linear array images. PhD Dissertation, Mitteilungen Nr 88, Institute of Geodesy and Photogrammetry, ETH Zurich. Zhang, L. and Gruen, A., 2004. Automatic DSM generation from linear array imagery data. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 34, Part B3, XXth ISPRS Congress, 12
23 July, Istanbul, Turkey, pp. 128
133. Zhang, L. and Gruen, A., 2006. Multi-image matching for DSM generation from IKONOS imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 60 (3), 195
211. Zhang, L., Gruen, A., Feiner, J., Louy, O. and Schmid, W., 2002. Photo-textured digital terrain models as a basis for regional and local planning. IAPRS. International Archives of Photogrammetry, Remote Sensing (and Spatial Information Sciences), 34, Part 5/W3. International Workshop on Visualization and Animation of Landscape, Kunming, China, 26 February
1 March.

About the author Since 1984 Professor Dr Armin Gruen has been Professor and Head of the Chair of Photogrammetry and Remote Sensing, ETH Zurich, Switzerland. He is Member of the Editorial Boards of several scientific journals, has published more than 350 articles and papers and is Editor and Co-editor of 20 books and Conference Proceedings. He is currently Chairman of the ISPRS International Scientific Advisory Committee (ISAC) and Member of the Executive Committee of the International Society for Digital Earth. His main current research interests include: Automated object detection and reconstruction with digital photogrammetric and videogrammetric techniques,

Downloaded By: [ETH-Bibliothek] At: 08:31 5 February 2008

106

A. Gruen

building and line feature extraction, 3D city modelling, image matching for DTM generation, industrial quality control using vision techniques, motion capture and face reconstruction for animation, imaging techniques for generation and control of VRs/VEs, Mobile Mapping, Cultural Heritage recording and modelling, 3D processing of high-resolution satellite images.