For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
Development of a Robust Spatial Engine for Management of Environmental Data S. Kazemi and S. Lim School of Surveying and Spatial Information Systems The University of New South Wales, Sydney, NSW 2052, Australia
[email protected] ABSTRACT Strain for maintaining up-to-date maps at a range of scales for spatial intelligence decision is ever growing. Egovernment agencies and private organizations require highly developed applications and software tools in responding to disaster events using location-based services. This underpins reliance on accurate, economical and viable digital spatial information products. Albeit the fact that automated generalisation systems meets these requirements at a reduced cost, although there is a strong demand for maintaining multiple data models and digital maps at different scales. Also, it should to be noted that automated spatial data generalisation will decrease the loading time for web mapping applications. This paper aims to report on development of a knowledge-based solution “Generalization Expert System (GES)”. A brief description of key steps undertaken in building GES and its components are presented in this paper. GES is being developed in Java and Python for the delivery of simplified spatial data. GES can assist emergency planners or decision makers to produce maps where natural or technological disasters are occurred, and facilitate timely evacuation, rehabilitation, damage assessment and modelling. Also GES offers a convenient way to access and navigate the various GIS datasets and performs generalisation processes. Its capabilities will be demonstrated in a case study through simplifying GEODATA TOPO-250K Series 2 data into 1:500000 and 1:1000000 scales over Canberra Australia. Maps produced by GES will assist emergency managers and field crew involved in recovery/reconstruction to understand the scope of the damage and identify locations that require support and rescue for the affected areas. Although a number of advanced applications have been developed with high technical skills but GES has a simple and user-friendly GUI that brings benefit to users with less technical skills and knowledge of spatial data management. 1.0 INTRODUCTION Spatial intelligence decision making relies on accurate, economical and viable digital spatial information products which underpin e-government initiatives and location-based services. Automated generalisation systems reduce the cost for maintaining multiple data models and digital maps at different scales. To facilitate the automated generalisation, this paper presents a detailed generalisation framework for deriving multi-scale spatial data involving assessment of existing generalisation systems (ESRI ArcGIS™, Intergraph DynaGen™, and LaserScan Clarity™), an international survey of cartographic generalisation practices, and developing a knowledgebased expert system on feature generalisation (see Kazemi and Lim 2007; Kazemi et al., 2007). In order to overcome insufficient handling of scale inconsistency in modern-day spatial updating endeavors, a “Generalisation Expert System (GES)” has been built in Java-Python, delivering automated generalisation of thematic data at a range of scales. The accomplishment is based on a pioneering approach to spatial data revision that assimilates a scale-independent data model and interactive cartographic generalization. The approach takes the advantage of national mapping agencies, state mapping agencies and a number of software vendor’s inputs in relation to the knowledge acquisition process of cartographic practices by undertaking a cartographic generalisation survey capturing the cartographers’ knowledge about the principles of cartographic generalisation and their experience with existing generalisation platforms. The findings from the survey are incorporated to outline the requirements for knowledge-based spatial database in which the cartographic knowledge and heuristic rules are formulated as a series of rules for feeding into GES to deliver
1
For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
coherent capabilities and automate the generalisation of features as much as possible for “derivative mapping” on real-time for sustainable spatial information management applications. One of the applications of GES is natural and technological disasters management and response. GES offers the emergency planners/decision makers to produce simplified maps which are exposed to risks or affected by disasters events, and facilitate timely evacuation, rehabilitation, damage assessment and modelling. Also GES offers a convenient way to access and navigate the various GIS datasets and performs generalisation processes. 2.0 METHODOLOGY It is well known that cartographic map production is a time consuming process with complex procedures. Current map production systems provide reasonable complex tools and procedural cartographic protocols still require cartographers interactions. This includes the selection of information, symbolization of features, maintaining topological relationships among features, and visualization of graphical conflicts as a result of scale variation that all aims to improve graphical quality and legibility of maps. For many of cartographic generalization procedures there is no algorithmic solution that requires an “ill-structured approach”. Expert systems use collection of “rules of thump” that are mainly heuristics criteria, methods, or principles for deciding which among several alternative courses of action promises to be taken to solve a particular problem. In the framework of this study, a rule-based generalization expert interfaced with a Geographic Information System software application “ArcGIS” is applied. Fundamentals of the system incorporate cartographer’s knowledge and intuition to generalize lines and polylines. Rules are symmetric so they can be processed in either a forward or a backward approach. GES uses operations, algorithms and knowledge-based rules that are elaborated in the next sections of this paper. Key components and methods of this research are shown in Figure 1 and include the following: A detailed review and evaluation of current generalisation methods and solutions. Developing a conceptual methodology to generalise GEODATA TOPO-250K Series 2 data into 1:500000 and 1:1000000 scales Cartographic knowledge acquisition by undertaking an international cartographic generalisation survey and analysis of the survey results to build a rule-based expert system. Construction of GES in Java-Python-C programming languages delivering automated generalisation of spatial databases. Demonstration of GES application for spatial data mining and generalisation for emergency disaster management.
2
For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
Figure 1. Overview of research methodology. 2.1 GES Architecture & Key Features A graphical overview of GES architecture is shown in Figure 2a, and its graphical user interface, key features, tools, and windows are presented in Figure 2b. The system is built in Java-Python-C programming environments with an input from the cartographic knowledge acquisition process “international cartographic generalisation survey” resulting development of the rule-based expert system “GES”. Mechanistic components and its structure are briefly explained in the following sections:
3
For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
Figure 2a. Architecture of Generalization Expert System (GES)
Figure 2b. Graphical user interface of GES and its key features, tools, and windows 2.1.1 Communication among components GES consists of three main components that are implemented in various programming languages. Communications between these components are achieved through different ways: • Direct calling uses build-in functions 4
For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
Java methods call C executable programs Python algorithm methods call C executable programs Indirect calling uses Unix/Linux Shell Scripts as a bridge Java Graphical user interface (GUI) calls Python algorithms using “gui_run_algorithm.sh” The returned data are stored in temporary plain text files in directory “tmp”.
• •
GES uses the Shapefile C Library (open source software) to read and write shape and dbf files (binary). In “bin” directory, there are four executable programs: shpparser.exe, dbfparser.exe, shpwriter.exe and dbfwriter.exe. They were developed from the Shapefile Library and then compiled to executable beta. For example, after opening the Java GUI, by clicking the “Open” button to open a Shapefile, the GES will call the “shpparser.exe” to parse the input shape file and generate a plain text output for the shape data, and to call the “dbfparser.exe” to parse the corresponding dbf file and generate plain text output for the dbf data. Both text outputs are stored in the directory “tmp”. When the “Run” button is clicked, GES python component will read these temporary shp and dbf plain text data and do line simplification/generalization algorithms to generate new outputs dumped to the “tmp” directory. Data geometry such as road class, types, length, etc are read and stored in the DbfTable objAect which are to be used for line merging later. 2.1.2 Java client for generalisation definition The java interfaces through a GUI are provided to set up the parameters for generalisation processes, visualising of input data and results for interactive completion of generalisation for the cases where cartographic intervention is required as shown in Figure 1b. Spatial data generalisation requires a large number of cartographic-defined parameters. In the GES, majority of those parameters are established by mapping specification, defining what is to be achieved, coupled with a set of constraints that introduce limits on allowable change (Neuffer et al., 2004). Main features/functions include: • Choice of user-defined knowledge rules for defining thresholds and algorithms • Choice of displaying different parts of GIS data eg lines, vertices, and labels • Choice of display colors for different parts of GIS data • Displaying both input and output GIS maps in the same window for comparison • Highlight selected shape files with prompt text • Zoom and/or move of GIS map. 2.1.3 C client for GIS data interoperability It utilises the Shapefile C Library (Version 1.2) to parse input shape files data and generate output shape files. The Shapefile C Library provides the functions for developing simple C programs for reading, writing and updating (to a limited extent) ESRI Shapefile, and their associated attribute files (.dbf). Four C standalone miniprograms have been developed: • shpparser.c/shpparser.exe – parse input shape data (binary) and then output as plain text file (ASCII) • dbfparse.c/dbfparse.exe – parse input dbf data (binary) and then output as plain text file (ASCII) • shpwriter/shpwriter.exe – write out final results into shape file (binary) • dbfwriter/dbfwriter.exe – write out final results into dbf file (binary). 2.1.4 Python component To run various line simplification and generalisation algorithms against input Shapefiles. 2.1.5
Physical structure of GES
Main Directory ges
Sub-Directory
Usage
Bin
Executable C program, Unix Shell Scripts
5
For presentation at the first Conference on Environmental Engineering Systems, Planning and Management Engineering, Organised by the University of Tehran’s Engineering Faculty and Iranian Space Agency, 11-12 December 2007, Tehran, Iran .
Classes config data
Compiled Java byte code classes for GUI Knowledge rules and GesLog.log Example input shp/dbf files Documentation for GES. Currently only include documents for python doc components eg algorithm.html, datastructure.html and shapes.html. lib The Shapefile C library python Python code, including three modules and driver class Src Java source code for GUI Temporary directory for storing temporary output files e.g. Tmp tempdata_shp.out and tmpdata_dbf.out start_ges.sh – a shell script for running the GES Java GUI 2.1.5 Inputs parameters The inputs include shape and dbf files (binary), and knowledge rules (e.g., “ges_rule1.dat” in the “config” directory). Refer to example below: ges_rule1.dat (in the “config” directory) Specifies all rules (algorithms and thresholds) to be applied to the GES program. Lines starting with # are the comment lines and will be ignored by the program. Please do not change the text on the left side of each equal mark, ONLY change values on the right side of equal marks, eg Yes/No. 0.20, 0.0. # This the knowledge rule file for GES # # Flag for whether or not to print out processing information to the GesLog.log file (Yes or No) # Default value is Yes. Write_Log_File = Yes # Flag for whether or not to mark key points before processing (Yes or No) # Default value is No. Mark_Key_Points = Yes # Flag for whether or not to merge polylines which share the same key properties (Yes or No) # Default value is No. Merge_Polylines = Yes # Flag for whether or not to allow overlapped line segments before simplification (Yes or No) # Default value is Yes. Line_Overlap = Yes # Threshold of Point Tolerance for vertex reduction. # If Line_Tolerance
