Introduction to the Special Issue on Image Information Mining ...

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 7, NO. 1, JANUARY 2010

3

Introduction to the Special Issue on Image Information Mining: Pursuing Automation of Geospatial Intelligence for Environment and Security

T

HE EARTH is facing unprecedented climatic and environmental changes, which require global scale monitoring and result in a host of new Earth observation (EO) satellite missions. Addressing the need for geospatial intelligence for the environment and security requires data from EO satellites, operating as single missions or constellations, and can thus enable a higher revisit frequency. Sensors on board these satellites will represent a host of new measurable phenomena that will increase the variety, amount, and resolution (spatial, spectral, and temporal) of data. This increase in EO missions is expected to result in unprecedented amounts of data to be processed. Moreover, as the data sharing principles of Global Earth Observation System of Systems move forward, more of this EO data will be made available to the global community. At the same time, the need for timely delivery of focused information for decision making is increasing. Therefore, EO data, which are not immediately usable, require chained transformations before becoming the needed “information products” (easily understandable and ready to use without further manipulations) offered as on-demand or systematic services. Different entities may perform these transformations using their own processes, which require specific knowledge, experience, and, possibly, data or information from domains other than EO. Today, these information products are primarily developed in a semiautomated fashion by experts or specialized companies operating in specific application domains. By using systems which can learn and/or apply knowledge, there should be increased efficiencies from the reduction of the information extraction time through the automation of such processes. Automatic or semiautomatic image information mining (IIM) techniques facilitate rapid identification of the relevant subsets among the large quantity of images, can support the expert’s interpretation, or even directly provide extracted information. This approach can also enable the processing of the petabytes of archived or new data, which currently are systematically processed only in limited quantities. In this context, IIM is an interdisciplinary approach in automating remotesensing analysis that draws on signal/image analysis, pattern recognition, artificial intelligence, machine learning, information theory, databases, semantics, ontologies, and knowledge management. It includes novel concepts and methods to help

Digital Object Identifier 10.1109/LGRS.2009.2034822

humans to access and discover information in large image archives to rapidly gather information about courses of action. This special issue focuses on automation of geospatial intelligence for the environment and security in which IIM concepts are a central component. Papers for this special issue were solicited in the areas of theory and applications, leading to more automation in geospatial information extraction and understanding from high spatial resolution optical and synthetic aperture radar (SAR) EO images and heterogeneous sources. Specifically, the call sought contributions in the following topics: 1) automatic image preprocessing (georeferencing, orthorectification, radiometric calibration, etc.); 2) challenges for meter resolution optical and SAR EO images; 3) geospatial intelligence: synergies across images, maps, and geo information; 4) models, semantics, and spatial syntax for image understanding; 5) information mining from heterogeneous sources; 6) human–machine communication for spatial reasoning; 7) knowledge discovery and sharing; 8) scenarios and constraints in environment, security, and intelligence applications; 9) system architectures for geospatial information processing. Over the past several years, the IEEE Geoscience and Remote Sensing Society, through its Data Archiving and Distribution Technical Committee, has sponsored several sessions at IEEE International Geoscience & Remote Sensing Symposium, and the European Image Information Mining Coordination Group (IIMCG) has supervised five conferences jointly organized by the European Space Agency (ESA) and European Union Satellite Centre to provide a forum for IIM researchers. In 2008, the IIMCG honored Dr. Klaus Seidel of the Swiss Federal Institute of Technology Zurich (ETH) for his pioneering work with large data sets dating back to ERS-1, which is ESA’s first EO satellite launched in 1991. Dr. Seidel’s persistence in finding new methods to deal with large EO data sets became his lifelong passion, and he was honored for his many years of service and for his pioneering work in establishing IIM as a new field of research. His lifelong commitment to research in information extraction from EO data sets played an instrumental role in the development of the Center of Competence on Information

1545-598X/$26.00 © 2009 IEEE

4


Extraction and Image Understanding for Earth Observation, which is a cooperation of Centre National d’Etudes Spatiales, German Aerospace Center (DLR), and Telecom ParisTech. Students at the Center of Competence are responsible for six of the contributions of this special issue. The focus of this special issue is to describe some of the recent advances made in the field of IIM for EO, particularly as they relate to geointelligence for environment and security. It is composed of 19 papers selected according to the standard review process of the IEEE Geoscience and Remote Sensing Letters. These contributions cover the topic from theory to applications and have been organized into four sections— Methods of Image Information Mining, Methodologies for Automated Processing of High-Resolution Optical Earth Observation Imagery, Methodologies for Automated Processing of High-Resolution SAR Earth Observation Imagery, and Environment, Security, and Intelligence Applications. The section on Methods of Image Information Mining contains nine contributions to the theory of information extraction and to knowledge-based approaches for automated processing of imagery. The letter by Cerra et al. presents an approach to image analysis based on the algorithmic information theory. The approach seeks to overcome the problem of understanding when assumed data models and estimated parameters are used. The contribution by Muñoz and Datcu presents an architecture for a Knowledge-centered EO computer system (KEO). The software design is based on a modular and user-oriented architecture, and the communication is focused on standard exchange of messages. There is a beta version of the KEO system available for testing. In the letter from Shah et al., the use of features extracted from a combined independent component analysis-wavelet transformation is introduced. Experimental results suggest that the new feature sets are more effective compared to traditional approaches in capturing both spectral and texture information for IIM. The next two letters both use the Latent Dirichlet Allocation (LDA) model, with the first at the object level and the second at the image tile level. The contribution by Sun et al. proposes a methodology to automatically detect geospatial objects present in high-resolution remote-sensing images. The approach enables detection, recognition, and segmentation of geospatial objects in a new image via a learned taxonomic semantic approach. The letter from Liénou et al. focuses on the annotation of large satellite images using semantic concepts defined by the user. Given a training set of images for each concept, learning is based on the LDA model. The research uses panchromatic Quickbird images with 60-cm resolution. Aksoy and Cinbi¸s describe a framework for modeling directional spatial relationships among objects and using this information for contextual classification and retrieval. Experiments show that the Bayesian decision rule that incorporates spatial information significantly decreases the amount of commission among spectrally similar classes. Retrieval experiments also produce more intuitive results and higher precision than other approaches in dynamic query scenarios with spatial constraints. In the contribution of Barb and Shyu, a knowledge discovery algorithm that uses content-based methods to link low-level image features with high-level visual semantics in an effort to automate the process of retrieving semantically

similar images is proposed. Durbha et al. address the problem of performing rapid IIM, which is required during the time of a disaster when previously classified objects are now manifested in the imagery with damage. The letter proposes a wrapperbased approach for feature selection and generation, and the results indicate that the number of features is reduced for a particular class used in the semantic model generation. In the final letter in this section, Molch examines the representation of semantic categories integrating Ikonos and Quickbird imagery in the knowledge-based information mining system. The work demonstrates change detection using different sensors. A processing sequence is demonstrated, which accounts for sensor-related differences along with an evaluation of the application of IIM technologies in operational rapid mapping scenarios. The section on Methodologies for Automated Processing of High Resolution Optical Earth Observation Imagery has three contributions of high relevance for mining the new generation of meter resolution EO images. The letter by Bovolo et al. investigates the effects of pansharpening when it is used for automated time series analysis for change detection in highresolution imagery. Although pansharpening improves edge detection and visual clarity, it can induce errors from artifacts induced by the fusion process. The authors describe a quantitative analysis strategy that exploits similarity measures to rank pansharpening approaches according to their impact on time series change detection performance. In their letter, Pacifici and Del Frate propose a pulse-coupled neural network (PCNN) for image change detection in high-resolution optical imagery. The PCNN architecture has the characteristics of being unsupervised and context sensitive. This latter property may be particularly useful when very-high-resolution images are considered. The Methodologies for Automated Processing of High Resolution SAR Earth Observation Imagery has three contributions dealing with techniques to automate despeckling of imagery from the TerraSAR-X high-resolution SAR satellite. The letter contributed by Soccorsi et al. presents a method which is capable of generating despeckled images using nonquadratic regularization processing applied to SAR complex-valued data. The novelty of the proposed method is that the optimization is performed directly on complex-valued data which enables dealing with the nature of the SAR coherent image system. Gleich et al. compare the despeckling of SAR images within the bandelet and contourlet domains. The experimental results show that the combination of Bayesian inference and bandelet transform outperforms the contourlet-based despeckling algorithm using synthetic data and objective measurements. In the final despeckling contribution, Molina et al. compare three different despeckling methods based on a Bayesian approach and Gibbs random fields. All approaches worked well in despeckling the imagery; however, each method had its own advantages for speckle removal, resolution preservation, or speed. The Environment, Security, and Intelligence Applications section has five contributions of application in urban environments, from fire risk management to detecting Saharan dust. The letter by Chaabouni-Chouayakh and Datcu describes a


semiautomatic tool for urban area interpretation using SAR data. It utilizes a coarse-to-fine approach based on the fusion of different features. The coarse step consists of segmenting the image into homogeneous regions, while the fine step includes the detection of bright and dark linear structures that match, respectively, with building edges and roads. Test results and evaluation on two TerraSAR-X subscenes over Las Vegas are presented. Thiele et al. present an approach aimed at reconstruction of gable-roofed buildings by knowledge-based analysis, considering the magnitude and interferometric phase signature. The analysis determines the signature changes by varying illumination and building geometry, and the reconstruction results are assessed by using a high-resolution light detection and ranging (LIDAR) surface model of Dorsten, Germany, as ground truth. The letter from Tuia et al. presents a scheme for the classification of very-high-resolution urban images using support vector machines and composite kernels. Morphological features extracted from the panchromatic image and multispectral bands are used simultaneously to account for both spectral and spatial information. The experimental results indicate a significant increase of the classification accuracy when the spatial information is used. The letter by Diagne et al. proposes an algorithm to detect the presence of forest fires using data from both geostationary and polar orbiting satellites. The problem is approached in a “global” way, providing the basis for an automated system that is not dependent on the local area properties. The algorithm is implemented in a “Multisource Fire Risk Management System” for Senegal, and a field campaign is carried out in order to validate the system. The final contribution of this special issue, which is the letter from McPherson and Reagan, seeks to improve aerosol retrievals from spaceborne LIDAR systems. In addition to investigating the overall scattering properties of Saharan dust, the letter highlights various retrieval approaches useful in determining the scattering properties of aerosols.

5

ACKNOWLEDGMENT The Guest Editors would like to thank the IEEE Geoscience and Remote Sensing Society for its support in publishing this special issue; Prof. W. J. Emery, Editor-in-Chief when we began this special issue, for his constant support and encouragement during the preparation of this special issue; the reviewers of the papers, who played the most important role in the final selection and quality of the contributions published in this special issue; and, finally, A. Larkin of the IEEE staff for her valuable assistance in all phases of the preparation of this special issue.

M IHAI DATCU, Guest Editor German Aerospace Center (DLR) Remote Sensing Technology Institute (IMF) 82234 Wessling, Germany Paris Institute of Technology Telecom Paris 75013 Paris, France ROGER L. K ING, Guest Editor Mississippi State University Department of Electrical and Computer Engineering Mississippi State, MS 39762-9618 USA S ERGIO D’E LIA, Guest Editor Head/Service Support and Ground Segment Technology European Space Agency ESA Centre for Earth Observation (ESRIN) Earth Observation Programme (EOP-GTR) 00044 Frascati, Italy

6


Mihai Datcu received the M.S. and Ph.D. degrees in electronics and telecommunications from the University “Politechnica” of Bucharest (UPB), Bucharest, Romania, in 1978 and 1986 and the “Habilitation à diriger des recherches” from Université Louis Pasteur, Strasbourg, France, in 1999. Since 1981, he has been a Professor in electronics and telecommunications with UPB. From 1991 to 1992, he held Visiting Professor appointments with the Department of Mathematics, University of Oviedo, Oviedo, Spain, and, from 2000 to 2002, with the Université Louis Pasteur, Strasbourg, France, and the International Space University, Strasbourg. From 1992 to 2002, he had a longer Invited Professor assignment with the Swiss Federal Institute of Technology, Zürich, Switzerland. Since 1993, he has been a Scientist with the German Aerospace Center (DLR), Wessling, Germany. He is currently a Senior Scientist and the Image Analysis Research Group Leader with the Remote Sensing Technology Institute, DLR, a Coordinator of the Centre National d’Etudes Spatiales (CNES)–DLR–TELECOM ParisTech Competence Centre on Information Extraction and Image Understanding for Earth Observation, and a Professor with the Paris Institute of Technology/GET Telecom Paris, Paris, France. In 1994, he was a Guest Scientist with the Swiss Center for Scientific Computing (CSCS), Manno, Switzerland. In 2003, he was a Visiting Professor with the University of Siegen, Siegen, Germany. He is involved in advanced research programs for information extraction, data mining, knowledge discovery, and data understanding with the European Space Agency, CNES, NASA, and in a variety of European projects. He develops algorithms for model-based information retrieval from high-complexity signals and methods for scene understanding from synthetic aperture radar (SAR) and interferometric SAR data. He is engaged in research related to information theoretical aspects and semantic representations in advanced communication systems. His interests are in Bayesian inference, information and complexity theory, stochastic processes, model-based scene understanding, image information mining for applications in information retrieval, and understanding of high-resolution SAR and optical observations. Dr. Datcu is a member of the European Image Information Mining Coordination Group. Roger L. King (M’73–SM’95) received the B.S. degree from West Virginia University, Morgantown, in 1973, the M.S. degree in electrical engineering from the University of Pittsburgh, Pittsburgh, PA, in 1978, and the Ph.D. degree in engineering from the University of Wales, Cardiff, U.K., in 1988. He began his career with Westinghouse Electric Corporation but soon moved to the U.S. Bureau of Mines Pittsburgh Mining and Safety Research Center. Upon receiving the Ph.D. degree in 1988, he accepted a position with the Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, where he is currently a Giles Distinguished Professor and the Director of the Center for Advanced Vehicular Studies, Bagley College of Engineering. He has also served as the Associate Dean for Research and Graduate Studies, Associate Director of the GeoResources Institute, and as the Director of the U.S. Department of Transportation funded National Consortium on Remote Sensing in Transportation—Environmental Assessments. Dr. King is a Registered Professional Engineer in the State of Mississippi. He received numerous awards for his research, including the Department of Interior’s Meritorious Service Medal. Over the last 30 years, he has served in a variety of leadership roles with the IEEE Industry Applications Society, the Power and Energy Society, and the Geosciences and Remote Sensing. He has served for four years as the Chair of the IEEE GRSS Data Archiving and Distribution Technical Committee, and he is presently a member of the IEEE GRSS AdCom. He also served as the Co-Technical Chair for IGARSS’09 in Cape Town, South Africa. Sergio D’Elia started his career in November 1966 as an Electronic Hardware Designer. He then became a Software Analyst and, finally, Head of Service Support and Ground Segment Technology. From 1966 to 1975, he was with the research and development branches of various leading Italian research centers and industries (Consiglio Nazionale delle Ricerche, Centro Sperimentale Metallurgico, Contraves, and Selenia), with responsibilities focused on electronic design of subsystems in support of various research topics, radar applications, or scientific satellite exploitation. From 1975 [when he joined European Space Agency (ESA)] to 1986, he was with the ESA’s Information Retrieval Service, managing part of the software development of QUEST, which is ESA’s information retrieval system. Since 1999, he has been managing the Service Support and Ground Segment Technology Office, Earth Observation Programme, ESA, Frascati, Italy, where he contributes to the definition of the European Research and Technology Development requirements for the Earth observation ground segment and manages a team that is in charge of implementing the related plan (concerned areas include user interfaces, information systems, payload planning, information mining, feature extraction, archiving, processing, distribution, automation, service support, GRID, etc.).