AN OPEN SOURCE FRAMEWORK FOR INTEGRATION OF VESSEL POSITIONS DETECTED IN SPACEBORNE SAR IMAGERY IN OPERATIONAL FISHERIES MONITORING AND CONTROL Guido Lemoine, Guillermo Schwartz-Juste, Naouma Kourti, Iain Shepherd, Christian Cesena European Commission Joint Research Centre, TP 267, I-21020 Ispra (VA), Italy, Email:
[email protected] ABSTRACT The European Commission’s Joint Research Centre has implemented a vessel detection system (VDS) based on the near real time use of synthetic aperture radar (SAR) imagery from operational orbiting satellites. The modular system consists of a number of distinct processing steps that are combined in an automated work flow capable of providing detected target positions within 60 minutes after at-sensor image acquisition and correlating these with timely position reports from various sources. All system modules have been implemented with Open Source software components and are extended with functionality derived from Java APIs. This paper details the use of the Open Source components and reports performance parameters for the end-to-end value adding processing chain, which has been extensively tested during monitoring pilot campaigns in 2003 and 2004. 1.
INTRODUCTION
European fisheries management policies have traditionally been focused on the established of annual total allowable catch (TAC) and catch quotas for certain species and maritime areas. Monitoring and management of these quotas is based on (paper-based) landing reports and skipper’s logbooks. Long term depletion of various fishing stocks has led to the introduction of new electronic reporting requirements which allow fisheries management authorities to monitor the movement of individual fishing vessels and their fishing activities. By far the most important European Union (EU) wide monitoring system is currently the Vessel Monitoring System (VMS) which was initially introduced in EU legislation in 1993 and recently amended [1] to cover all fishing vessels above 18 m overall length and all fishing vessels above 15 m overall length from 1 January 2005 onwards. The key characteristic of the VMS is that it transmits the vessel’s GPS position at 1-hourly or 2-hourly interval to the coastal EU Member State where the vessel is registered. The VMS has global coverage, except for the poles. The use of VMS has provided new insights into the dynamics of fisheries all over the world and has enabled new measures to manage fisheries activities at
a more detailed scale. Stock recovery measures by establishing closed areas for certain periods of the season have become far easier to control with the use of VMS. Calculation of effort parameters such as total days at sea and origin of catch are more easily verifiable against logbook entries than they were before the introduction of the VMS. VMS has even been introduced as legal supporting evidence in infringement cases of European or national fisheries regulations. In order to support the VMS system and extend the reach of the monitoring, the European Commission (EC) has shown a keen interest in the use of satellite imagery for the detection of fishing vessels. The EC’s Joint Research Centre (JRC) started feasibility studies for vessel detection systems (VDS) based on Synthetic Aperture Radar (SAR) data use in 1999 [2]. On the basis of these studies’ results, the project “Improving Fisheries Monitoring Integrating Passive and Active Satellite based Technologies” (IMPAST) was launched in early 2002. The IMPAST project’s aim is to set-up and demonstrate an operational processing chain that allows Fisheries Monitoring Centres (FMC) of the EU Member States to integrate vessel positions detected in satellite imagery with their timely VMS records in near real time in order to support verification and control activities. The main purpose of the VDS is to support verification actions of VMS reports. The overall goal of the combined use of VMS and VDS, together with other fisheries information collections, is to enable the EU Member States and the European Commission to better manage and control fish stocks in European and international waters where the European fishing fleet is active. The specific objective of the VDS system is to support the analysis of possible anomalies in VMS position reports and estimate presence of non-VMS carrying fishing vessels in EU and international waters. In order to be meaningful in a control context VMS verification must be performed with a minimum time delay between at-sensor image acquisition and correlation of VDS and VMS positions. Verification is focused on both the VMS positions not identified as VDS (potential anomalous reports) and on VDS positions not matched with VMS (potential noncompliance with VMS regulation). A final goal of the
operational system is to allow surveillance efforts to be redirected in real time to areas for which the system flags occurrence of anomalies or non-compliance. In order to support this goal, VDS reports must be made available in a format and/or within a readily accessible analysis environment for decision makers in the FMC. The particular situation in the EU requires coordination between national FMCs, each with the responsibility to monitor its own fishing fleet, especially in maritime areas that straddle more than one FMC’s Exclusive Economic Zone, or in international waters where several EU and non-EU vessels may be active (Fig. 1).
Figure 1. VDS positions detected in RADARSAT-1 ScanSAR Narrow B imagery acquired near the South-western border of the Icelandic EEZ. Vessels in this area are active in Red Fish fisheries and are mostly from Iceland (IS), Greenland, Russia, Norway (NO), Germany, France, Spain (ES) and Portugal (PT). FMCs from IS, NO, ES and PT are partners in the IMPAST project, and receive VDS records in near real time. The extract at the top is for June 27, 2003 (19:27 GMT), the one on the bottom is for June 27, 2004 (08:22 GMT). Besides the use of VDS in near real time, regular collections of VDS positions can possibly contribute to fisheries pattern analysis to plan long term monitoring and control activities. The emphasis in this paper is on the near real time use of VDS, for which the user
requirement for maximum time delay between satellite sensor acquisition to FMC supply is defined as better than 2 hours. However, the software components described in this paper already support a number of post-analysis tasks or can be easily enhanced to support additional requirements, for instance, spatial aggregation and cross-correlation with other fisheries data. One of the key factors limiting the more wide-spread acceptance of remote sensing derived information is the “expert threshold”. This threshold exists both as the high level of expertise required to work with satellite imagery, which holds especially for SAR imagery, and the need to work with complicated software and processes to arrive at an information product that is usable in the thematic context of the end-user. The typical manager or operator at an FMC has limited knowledge of satellite data use, except for GPS and weather satellite data, and computer use is mostly focussed on operational tasks. A common system set-up at FMCs consists of a central data base repository that is fed by live feeds of VMS data and a visualisation environment, typically a GIS, that supports analysis and reporting tasks. One of the tasks in the IMPAST project was to seamlessly integrate VDS records in such an environment, or offer an integrated alternative with limited, or no significant, additional costs to the FMCs. Thus, the value adding services components between image acquisition and final data use are, to a large extent, concerned with hiding the complexity of SAR data use and delivering information tailored to the various thematic end applications of the FMC user. Open Source (OS) software components have developed to a maturity that allow their integration in a number of processing steps in operational application context. This is especially true within the scope of the IMPAST project, which is primarily a prototyping exercise. It is, however, the opinion of the authors that there are no significant limitations in the use of OS even for a fully operational VDS production system. The use of OS has many advantages [3] and relatively few disadvantages in comparison to commercial alternatives. For the author's organisation OS is especially attractive as it allows us to demonstrate key functionality to a wide range of external contacts at European Commission level, EU Member State authorities and technical partners, without having to limit the reference implementation to a specific software product, computing platform or particular system configuration.
2.
METHOD
The VDS system structure proposed under the IMPAST project consists of distributed functional components that are chained together through communication interfaces. The main four functional components are the image supplier (IS), the VDS service provider (VP), the VMS-VDS correlator (VC), the Fishing Monitoring Centre (FMC) and the image server (IMS). While, in principle, each of these functional components can be hosted by separate entities, combinations of functional components within one entity are possible as well. Functional roles are depicted in Fig. 2 and outlined as follows:
Figure 2. VDS system architecture depicted as a workflow of functional components and tasks. Image supplier (IS): the IS receives SAR signal data from the satellite sensor at the ground station, processes this into SAR imagery in a rush service set-up and supplies these to the VP in near real time via a dedicated communication link. The IS will arrange satellite and ground station resources on the basis of the programming plan provided for the VDS operations. Due to the limited ground cover of an IS’s receiving antenna, it may be necessary to employ the services of more than one IS to complement coverage over extended maritime areas, especially if this is towards the outer extents of the IS ground mask. The coverage of all European waters with RADARSAT-1 imagery effectively requires 3 different ground stations: KSAT in Tromsø, Norway, West Freugh in the United Kingdom and ITU SAGRES in Istanbul, Turkey. VDS provider (VP): the VP receives SAR imagery from the IS and executes the VDS algorithm for the detection of targets at sea. The key component for the VP is an efficient detection algorithm, where efficiency is defined in terms of high quality of detection and fast processing speeds. The VDS results are forwarded to the VMS-VDS correlator (VC). Since the VP has a central role in the scheduling of tasks, it takes care of
all direct communication on planning, programming, ordering and data transmission with the IS. Scheduling is based on the proposals for area selections and timing by the participating FMCs. An obvious advantage of combining the roles of the IS and VP in one entity is the absence of the need for separate planning and tasking analysis, which is already a core business task of the IS. Furthermore there is no strict requirement for high bandwidth communication of image data. One of the user requirements of IMPAST, however, was to allow the decoupling of these two roles in order to demonstrate independent choice of the two services. VMS-VDS correlator (VC): the VC function deals with correlating the VDS positions that are received from the VP with timely VMS positions reports that are available with the participating FMCs. The correlation consists of a matching algorithm that sorts the VDS targets and VMS positions into matching pairs. The original set-up proposed in the IMPAST project foresees this function to be housed within the FMC, mostly due to the need to treat the VMS data with strict confidentiality. Practical experience in the IMPAST project, and related experimental VDS campaigns, has shown that VC implementation at the FMC is not always easy (see next section). Secure internet communication combined with web server technology allow this task to be hosted elsewhere as well. In any case, correlation results need to be made available, again in a secure way, by the VC to the participating FMCs and the central access point at the IMS. Fishing Monitoring Centre (FMC): the "end user" of the project results, supplies timely VMS positions to the VC and receives both correlated and uncorrelated VDS results for integration into its monitoring task. VMS positions are obtained by active polling of vessels in the area, when this is possible. VMS polling is set up for the known extensions of the image frames and should be planned for a time frame within 10 minutes of the known image acquisition times. As part of its normal functioning, coastal FMCs receive VMS positions from other flag states’ FMCs for vessels that operate in its jurisdiction. Participating FMCs are requested to poll for their vessels in the area, so that these VMS can be integrated in the correlation. Alternatively, each FMC’s VMS positions are correlated in separate runs. The FMC has access to the central repository of image data and VDS and VMS-VDS correlation results at the IMS. The FMCs are expected to define a number of follow up actions, based on the results of the VMS-VDS correlation. This follow up may consist in integration of the results into the FMC's interface to VMS analysis tools, to highlight potential anomalies. The near real
time supply of the results, however, is aimed at optimizing surveillance efforts, e.g. by enabling the redirection of inspection aircraft or vessels to areas where potential anomalies have been highlighted. IMage and Data Server (IMS): the IMS serves as the central repository of campaign results. This service provides access to the campaign imagery, VDS results and VMS/VDS correlation results. More advanced IMS implementations may include interfaces to VDS algorithms, so that vessel detection can be re-run, for instance with alternative parameter settings, on subselections of image data sets. Such a function would be of use if a FMC wants to better discriminate between noise targets and real targets in an area where surveillance takes place. 3.
RESULTS
The results of the IMPAST project consist of the various software implementations of the functional modules outlined in the previous section, and performance parameters derived from experimental runs.
Figure 3. The VDS workflow (same as in Figure 2) emphasising the communication interfaces and software components used in the implementation. Red software components are in the Open Source domain. Green components are developed by JRC on the basis of Open Source APIs. The software implementation at the IS is typically proprietary and linked to its existing SAR receiving and processing infrastructure. A detailed discussion of the IS configuration is beyond the scope of this report, except for the SAR image data rush processing service and forwarding mechanism. The KSAT (Tromsø, Norway) ground station’s SAR processor is capable of producing fully processed SAR imagery between 15 and 30 minutes after reception, in dependence of selected image mode and processing parameters.
As part of the IMPAST project, the JRC has installed a dedicated satellite communication uplink facility at KSAT. The uplink was necessary in the early phase of the project, since existing FTP communication performance was not sufficient to allow NRT use of the imagery. The uplink facilitates the use of internet protocol based data transfer over a satellite (IPSat) connection. The hardware and software for the satellite link have been supplied and installed by Alcatel, the technical communication partner in the IMPAST project. Typical transfer rates achieved with this set-up are in the order of 10-15 Mb per minute, with total transfer times of 4-10 minutes depending on the overall file size of the image, which depends, in its turn on the selected image mode. In September 2003, KSAT upgraded its FTP service bandwidth. Extensive testing in the autumn of 2003 demonstrated that the upgraded FTP link performed equally to, and sometimes better than, the satellite uplink. The latter requires manual operator intervention, causing an additional delay, whereas the FTP process is completely automatic. For the production of VDS, a target detection algorithm needs to be in place at the VDS provider (VP). JRC has developed and implemented the SUMO algorithm [4]. The SUMO algorithm is based on a fast version of a classical CFAR (constant false alarm rate) target detection algorithm. It is capable of analysing both RADARSAT-1 and ENVISAT ASAR imagery of all image modes, including the latter’s alternating polarisation mode. The main advantage of the algorithm is the performance rate: a complete image can be run through the algorithm in approximately 1 minute on a Pentium PC. SUMO is implemented in Java, and can be made available for all platforms supporting the Java Virtual Machine. A fast implementation of a Kdistribution based detection algorithm [5] has been implemented as well. The detection algorithm requires a robust source of land perimeters to mask land targets in the imagery. The original concept of the SUMO algorithm to work with manually digitized outlines of land no longer sufficed in an operational context, which should be applicable on a global scale. Currently, the best public source of vectorial coastline data is the Global Self-consistent, Hierarchical, High-resolution Shoreline (GSHHS) data base [6]. The full, high and coarse resolution versions of the data base were imported into a PostgreSQL/Postgis data base and spatially indexed. Java extraction routines that make use of spatial queries were implemented that allows arbitrary selections from the global set. Query performance time is typically less than 1 second for areas the size of an image frame. GSHHS extracts are then fed, via a web service, into
the SUMO land masking routine, in a similar way as done with manually digitised land perimeters. The accuracy of the GSHHS shoreline data depends on several factors, such as the original source and scale of the shoreline information, tidal differences, etc. Furthermore, uncertainty in the orbital parameters may cause a SAR image frame to be shifted up to several hundred meters relative a fixed geographical reference. SUMO allows a fixed distance from shore to be defined in order to avoid detection of near coast targets that are possibly due to inaccurate shoreline data. Limitation of the GSHHS quality was highlighted in the English Channel (large tidal difference) and Bothnian Gulf (many very small islands, most of which are not mapped). The implementation of the land perimeter feed as a web service allows a straightforward exchange of shoreline data sources to, for instance, larger scale data holdings that are more appropriate in a regional application. In order to service the automatic near real time supply of VDS positions to FMCs the SUMO algorithm is wrapped in a controller application, again implemented in Java. Automatic runs are scheduled according to the planned image acquisition schedule (e.g. using the ‘at’ scheduling function on Unix systems). The controller polls for the presence of a new image in a dedicated project directory, independent from the transmission mechanism used between IS and VP (either satellite uplink or FTP). Upon detection of the new image, it configures and runs the detector algorithm, produces tabular and graphical VDS output and forwards results via E-mail and secure HTTP (HTTPS). All results are stored to the IMS data base back-end. The choice of Java is again ideal for this purpose as it contains all necessary application programming interface for advanced image processing, graphics and network communication. A complete run of the controller initiated SUMO algorithm takes between 2 and 3 minutes. HTTPS transfer is normally instantaneous as typical VDS tracks are only several kilobytes in size, while E-mail reception delays depends on mail configuration at server and client sides and general internet transfer rates. The time delays caused by the image reception, image processing, image transfer and vessel detection modular functions add up to a theoretical throughput performance of between 20 and 40 minutes, primarily dependent on image file size. This performance has been corroborated by numerous experimental results between June and December 2003 and ongoing experiments. The minimum delay has been 21.8 minutes for a Standard 6 image mode (S6) RADARSAT-1 product of 123 Mb over the Baltic Sea on November 7, 2003. For a Wide 2 image mode (W2)
RADARSAT-1 product of 276 Mb over the same area, a minimum delay of 37.1 minutes was accomplished on November 24, 2003. The correlation process at the VC matches pairs of VDS and VMS positions, taking into account possible displacement of VMS positions in the time lapse between VDS detection time (i.e. image registration) and VMS detection time (poll response). The correlation is triggered upon reception of VDS tracks and interfaces with the VMS data base to extract timely VMS positions for matching. The matching result is then forwarded to the IMS so that they can be accessed by all FMCs that have vessels in the image area. A major drawback of the original IMPAST approach to place the correlation process at FMCs is the need for individual server set-ups at each participating FMC and the need to synchronise both the correlator algorithm and correlator output. In its effort to integrate correlation in a web service environment, the JRC has implemented a Java version that is fine-tuned to a number of practical problems found during experimentation. These problems relate mostly to format handling, which, although standardized through commonly agreed data formats, were not implemented and maintained correctly by all FMC servers. This problem was resolved by establishing web uploading functions that automatically convert received VMS formats into a common standard and store these in the IMS spatial database. In this way, a number of important pre-filtering functions, such as spatial and temporal selection, sorting and interpolation can be handled by highly efficient data base functions. JRC has built up considerable expertise with the Open Source PostgreSQL/Postgis (www.postgrsql.org) configuration to support this part of the system. The correlation algorithm itself is implemented as a web service, which produces its result as XML formatted output. The correlation web service is further extended to handle also positional records from other sources such as coastal radar, surveillance observations and static maritime objects that are stored in the data base. The use of XML structured output is of considerable importance in the context of the VDS. A major obstacle has been the integration of the results in some visual analysis environment for the rapid analysis of the results. The use of Scalable Vector Graphics (SVG, an open XML standard that describes graphical objects and operations) solves this problem rather elegantly. SVG can be used to produce interactive thematic maps on demand. Other data layers in the SVG interface can be compiled from several XML data streams, each of which can be derived from separate web services (e.g.
the map layers in Fig. 4). The correlation is integrated as a dynamically calculated map theme.
Figure 4 The SVGClient interface of the geotools OS package (geotools.codehaus.org) adapted to the correlation server application. Based on JavaScript, this runs in a web browser which has the SVG viewer plugin installed. The interface closely resembles the well known ArcView interface, with colour coded layer handling and zoom and labeling functions. This example shows instantaneous VDS and VMS correlation results over the Baltic Sea area near Öland island As well as VDS records and correlation output, FMCs might benefit from the image itself. Access to SAR imagery can be provided to FMCs through an extension of the web server functionality. Implementation of this functionality must address the efficient access to subselections of large image files. Various compression algorithms are available that combine the use of reduced storage space on the server with fast access to full resolution imagery. The IMPAST project has experimented with a propriety solution implemented by ISO-Informatique for both the server and client side. The client side required a plug-in to handle the compressed format. An Open Source implementation was set-up with the use of the Geospatial Data Abstraction Library (GDAL, www.remotesensing.org/gdal) and Univ. of Minnesota (UMN) MapServer software modules (mapserver.gis.umn.edu). GDAL supplies routines to automatically warp imagery to a geographical projection. UMN MapServer provides a fully functional GIS interface, in which georeferenced imagery can be integrated in a straightforward way. A significant advantage of UMN MapServer is that it can be fed with geographical data from the PostgreSQL data store that is already in place. It provides extensibility via an PHP API. Example screen shots are given in Fig. 5.
Although this is not within the main scope of this paper, it is important to state that the VMS and VDS collections, and other positional records, form an uniform benchmark data set for performance analysis [7]. We routinely feed the output of other detection algorithms into the data base. Cross-correlation between VDS sets, and between VDS and VMS, can then be generated almost instantly for analysis of performance of detection. In the IMPAST project, VDS benchmarking for the SUMO algorithm shows consistent detection rates which depend on both the image mode and the overall size of vessels. Detection in open ocean areas, where vessels tend to be at least 50 m in overall length, demonstrate detection rates of as high as 90%. A similar performance is found for standard image mode in continental seas like the Baltic Sea, where vessel size is rather around 20 m overall length. Wide mode imagery over these areas tend to have somewhat lower detection rates (70-80%), due to its lower image quality. In an operational set-up, the tradeoff between wide area, low resolution and smaller area, high resolution imagery must be evaluated against the expected occurrence of vessel types in the fisheries under surveillance. This information is easily obtained from historical VMS records. More detailed analysis of detection performance is found in [8] and [9] and is subject to further work in the IMPAST project. 4.
CONCLUSIONS
The IMPAST project has implemented a vessel detection system (VDS) based on the near real time use of synthetic aperture radar (SAR) imagery from operational orbiting satellites. The modular system consists of a number of distinct processing steps that are combined in an automated work flow and which communicate via web services. Consistent performance with at sensor to FMC delays of better than 1 hour has been realized, well within the original user requirement. The JRC VDS implementation is completely based on Open Source components and extended with Java-based APIs. The processing chain has been devised in such a way that it largely hides the complexity of SAR data processing from the intended end-users (the FMC operators) and presents results in a portable, extensible manner.
5.
ACKNOWLEDGEMENT
The work reported in this paper is co-financed by the European Commission’s FP5 R&D programme under contract Q5RS-2001-02266-IMPAST. All RADARSAT-1 imagery shown are © Canadian Space Agency. ENVISAT imagery is © European Space Agency. Both types of SAR imagery were processed at Kongsberg Satellite Services AS (KSAT) of Tromsø, Norway. The authors would like to thank the Open Source community for their contributions to the OS components and data sets used in this work. REFERENCES [1] Commission Regulation (EC) No 2244/2003 of 18 December 2003 laying down detailed provisions regarding satellite-based Vessel Monitoring Systems Official Journal L 333, 20 December 2003, p. 17 –27. http://europa.eu.int/eur-lex/en/search/index.html
[2] N. Kourti, I. Shepherd, G. Schwartz, P.Pavlakis, Integrating Spaceborne SAR imagery into Operational systems for Fisheries Monitoring, Can. J. Rem. Sens. 27, No. 4, 291-305, 2001. [3] R. Stallman, The GNU Manifesto, http://www.gnu.org/gnu/manifesto.html [4] G.Schwartz, N. Kourti, Algorithm Development Searching for Unidentified Maritime Objects (SUMO), JRC Ispra, Tech. Note No. I.01.115, 11 p., 2001. [5] H. Greidanus, Applicability of the K-distribution to RADARSAT Maritime Imagery, in Proc. Int. Geosci. Rem. Sens. Symp. (IGARSS’04), 2004. [6] National Oceanic and Atmospheric Association (NOAA), GSHHS —a global self-consistent, hierarchical, high-resolution shoreline database.. http://www.ngdc.noaa.gov/mgg/shorelines/shorelines.html, 1999. [7] H. Greidanus et al., Benchmarking Operational SAR Ship Detection, in Proc. Int. Geosci. Rem. Sens. Symp. (IGARSS’04), 2004. [8] G. Lemoine et al, Near Real Time Vessel Detection using Spaceborne SAR Imagery in Support of Fisheries Monitoring and Control Operations, in Proc. Int. Geosci. Rem. Sens. Symp. (IGARSS’04), 2004 [9] J. Chesworth and G. Lemoine, IMPAST: Improving Fisheries Monitoring by Integrating Passive and Active Satellite Technologies, Results and Analysis from second year's campaign, JRC Ispra, Techn. N. No I.04.3, 45 p., 2004
