www.ietdl.org Published in IET Radar, Sonar and Navigation Received on 5th March 2007 Revised on 5th June 2007 doi: 10.1049/iet-rsn:20070032
ISSN 1751-8784
Data correction for visualisation and classification of sidescan SONAR imagery C.G. Capus1 A.C. Banks2 E. Coiras3 I. Tena Ruiz4 C.J. Smith5 Y.R. Petillot1 1
Ocean Systems Laboratory, School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, EH14 4AS, UK Institute of Oceanography, Hellenic Centre for Marine Research, Gournes Pediados, P.O. Box 2214, Heraklion, Crete 71003, Greece 3 NATO Undersea Research Centre, Viale San Bartolomeo 400, La Spezia 19138, Italy 4 SeeByte Ltd., Orchard Brae House, 30 Queensferry Road, Edinburgh, EH4 2HS, UK 5 Institute of Marine Biological Resources, Hellenic Centre for Marine Research, Gournes Pediados, P.O. Box 2214, Heraklion, Crete 71003, Greece E-mail:
[email protected] 2
Abstract: Of all the remote sensing modalities available for underwater applications, acoustic methods, covering frequency ranges from a few Hz to several MHz, are by far the most flexible and widely used. The authors propose a method for preprocessing sidescan sonar data for visualisation, detection and classification purposes. Sidescan imagery is highly sensor specific and is typically affected by factors that have either a range dependency or an angular dependency. Each of these is altered in a different way given variations in sensor altitude over the seabed. Working from the physics and geometry of the sonar process, the proposed method estimates separate correction factors for range and angular dependencies directly from the image data. Once calculated, these factors can be applied over large data sets to provide radiometric correction over the entire survey area. Simpler image processing algorithms are more effective because the image statistics are improved with more stable means and variances across the sonar swath. The method requires a good bottom detection algorithm for estimation of sensor altitude at each transmission time and incorporates a resampling scheme for the calculation and application of the angular-dependency correction factors. Results showing improved classification performance for two large area surveys are presented. The method proposed provides a more complete solution than previously reported resampling schemes and offers significant improvements in terms of accuracy, robustness, usability and execution times.
1
Introduction
Acoustic sensing provides an excellent means for data gathering in the marine environment. Electromagnetic waves attenuate rapidly and their operating range is limited to tens of metres, or less in turbid water. Sidescan imaging sonars offer the user rapid environmental assessment over large areas with centimetric resolution and long ranges, up to hundreds of meters. The sidescan sonar is an active system consisting of a long acoustic array providing a beam which is wide perpendicular to the array and
& The Institution of Engineering and Technology 2008
narrow parallel to the array’s long axis [1, 2]. These sensors are typically mounted to port and starboard sides of a towfish or underwater vehicle moving through the water. At each transmission time the generated acoustic beams will illuminate a narrow strip of seabed, Fig. 1. Over time a large area image of the seabed is obtained as a concatenation of successive strips. The main parameters affecting the resolution of the generated images are the frequency of the acoustic wave used, which determines the across-track resolution, and the antenna length, which determines the along-track resolution. Overall seabed IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 155 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 1 Sidescan sonar survey using a towfish
coverage is determined by the speed of the acquisition platform. Advances in transducer and processing technologies, combined with the increased availability of relatively cheap unmanned and autonomous underwater vehicles (AUVs), are providing increasingly capable, portable and low cost survey sensors. Sidescan is now so commonly used it is important that it is able to benefit from the full range of image understanding and processing techniques available. Various types of remote sensing imagery suffer from banding and striping arising from particular sensor data acquisition characteristics, sensor calibration problems and platform motion [3 – 19]. Schowengerdt [20] provides a useful overview of various image processing methods for correcting the imagery that have been proposed. An interesting comparison exists particularly between sidescan sonar and synthetic aperture radar (SAR) data and those methods that have corrected airborne SAR imagery for use in very precise differential interferometry applications where the main problem in processing is in compensating for aircraft movement and thus sensor motion [12, 14, 15]. Effects of normalisation schemes on detection systems have been considered [21], with forward-backward normalisation filters proposed for improved mine detection performance with sidescan sonar [22] and pseudoinvariant and temporally invariant cluster methods proposed for improved change detection in satellite data [23]. Effective compensation schemes incorporating beam directivity patterns have been demonstrated for multibeam echosounder data [24]. In this paper, we address the difficult problem of correcting sidescan sonar imagery in the presence of changes in altitude of the acquisition platform over the seabed, a common occurence when using towed sidescan systems. The method is shown to be suitable for shallow water and low altitude surveys with improved classification performance for the corrected data. 156 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
Working from the physics and geometry of the sonar process, calculation and application of correction factors incorporates a resampling procedure. The method presented here provides a more complete solution than previously reported resampling schemes [25, 26] and offers significant improvements in terms of accuracy, robustness, usability and execution times. Sidescan compensation for keel-mounted sensors, incorporating synchronised ship orientation information, has been considered previously [27]. In the current case, correction factors are generated directly from the sonar imagery without any assumed knowledge of sensor position and attitude information. This is important for application to legacy data and allows the algorithm to be used where navigation data are missing, corrupted or have been poorly synchronised with the sensor returns. Performance is improved through a data reduction procedure which allows generation of correction factors in under a minute given a suitable exemplar image and allows data correction to be performed considerably faster than the data acquisition rate.
1.1 Data sets Results for the correction of two sets of data are presented. The first is from a low altitude towfish survey over a sandy sediment containing maerl beds. Maerl is a collective term for several species of coralline algae, forming aggregations of gravel-like sediments over the natural sediment surface. They are typically found in clear waters from shallow depths down to 100 m (in the Mediterranean). The data set area is characterised by a muddy sand seabed with maerl patches and occasional maerl– rock reefs with heights up to 1 m [28]. By increasing the complexity of the bottom, maerl beds provide an important habitat for marine animals and plants, thereby increasing the biodiversity of an area and due to their fragility can provide an early indicator of damaging environmental change. Maerl beds exist extensively and the current survey was taken off the north coast of Crete in October 2003 as part of the AMASON research project [29]. Marine science interest arises in part from desire to protect the maerl species themselves and in part from interest in the floral and faunal species supported by the maerl colonies. A high resolution video image of a maerl region is shown in Fig. 2. Sidescan data were gathered aboard HCMR’s survey vessel RV Philia, towing a Geoacoustics 196D towfish, with a winch-mounted coaxial cable, SS941 transceiver and GeoPro LC processor (all Geoacoustics, UK) connected to a differential global positioning system (DGPS). Surveys were carried out at a frequency of 410 kHz, 98 m range per channel giving a swath
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 2 Eastern Mediterranean maerl (North of Crete, Greece at 50 – 100 m depth) shown in sections of a mosaiced video taken by the HCMR Max Rover ROV in 2004 a and b c20 m sections representative of the distribution and texture of the maerl existing on the underlying sandy bottom sediment c Zoomed c3 m section showing a concentration of maerl on and around a rocky outcrop. Mosaics created by HCMR using the IFREMER MATISSE software
width of 196 m. The published figures for vertical (408) and horizontal (0.38) beamwidths and pulse length of 88 ms at 410 kHz, equate to across-track resolution of 15þcm and along-track resolution of 25 cm at 50 m range [30]. The second illustrative data set comes from the BP02 experiments carried out by the SACLANT Undersea Research Centre, La Spezia, Italy, now renamed the NATO Undersea Research Centre (NURC). Provided good correction factors can be estimated, these data show the procedure is appropriate for application to highly textured sidescan imagery. The correction procedure is applied to data gathered using a REMUS AUV [31] with a Marine Sonics sidescan sonar operating at 900 kHz. The survey was conducted in very shallow coastal waters at an average altitude of around 3.5 m. The altitude is much more stable in this survey than in the towed sidescan survey. However the data do suffer from a number of other artefacts that affect the image quality.
1.2 Layout Following this introduction, Section 2 discusses sidescan data interpretation. Intensity variations in sidescan imagery arising from the sonar process are discussed alongside the the difficulties associated with sensor altitude variation. Section 3 describes the methodology
& The Institution of Engineering and Technology 2008
in detail and Section 4 describes efficiency measures which can aid execution times. Section 5 presents radiometric correction and classification results from the maerl and very shallow water survey data sets. Section 6 concludes the paper.
2
Sidescan data interpretation
Fig. 3 shows a typical sidescan sonar image over a region of flat seabed. Each horizontal line in the image corresponds to echoes from a single transmission. The bright vertical line in the centre of the image is due to direct coupling between the transmit and receive stages. There is little response from the water giving rise to the dark central region, often termed the water column, being the ensonified section of water between the sensor and the first bottom return. There is then a sharp response in both channels on reception of the first ground returns. The presentation in Fig. 3 is a time – time plot. Pixel spacing in the along-track direction is determined by the pulse repetition rate of the sonar. In the across-track direction pixel spacing is determined by the sampling rate of the data acquisition hardware. Conversion from time to range is made using an estimate of the sound speed under the pertaining survey conditions. IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 157 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 3 Typical sidescan sonar image showing water column central black region and first returns Sensor altitude variation is indicated by changes in the width of the water column
2.1 Beam pattern Sidescan sonar images characteristically display acrosstrack intensity variations resulting from the transducers’ vertical beam patterns. The beam pattern is determined by the operating characteristics and physical design of the sensor arrays. For an array of N regularly spaced elements separated by a distance d, each transmitting in phase with acoustic wavelength l, the normalised beam pattern in the far field is given by [2] sin (N(pd=l) sin (u )) 2 B(u ) ¼ N sin ((pd=l) sin (u ))
Figure 4 Polar plot showing the theoretical beam pattern for a six element array with elements spaced at l/2 (dB scale)
grazing angle is the angle at which the acoustic pulse hits the seafloor and is determined by sensor orientation and seabed topography. Backscatter intensity is strongly dependent on grazing angle and for sonar systems operating at up to around 200 kHz is well modelled for a wide range of generic seabed types. Fig. 5 illustrates the variation in backscatter intensity with grazing angle at 100 kHz for various sediments generated from Applied Physics Laboratory, University of Washington (APL-UW) models [32]. The curves are generally well separated for grazing angles between 58 and 708, which will cover more than 95% of the swath width for ground ranges out to 12 times the survey altitude. This indicates that a
(1)
For a six element array, with d ¼ l/2, this provides the beam pattern shown in Fig. 4. For an array of length L, far field behaviour can be assumed at ranges beyond R ¼ L2=l. In the far field, for an array with an element spacing less than the acoustic wavelength ((d/l) , 1.0), there is characteristically a single high intensity mainlobe perpendicular to the array, with several lower intensity sidelobes.
2.2 Backscatter and grazing angle The beam pattern clearly has an angular dependency and in this regard can be grouped with the influence on backscattering strength of the grazing angle. The 158 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
Figure 5 Backscatter intensity from APL-UW models for various generic seabed types ensonified at 100 kHz
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org correction factor calculated for one sediment type, for example the gravel seabed, can provide a relatively stable output for each of the other sediment types over most of the sonar swath. Many imaging sidescan sonars now operate at frequencies traditionally considered very high for sonar operation, around 400 kHz – 1 Mhz. While this is not expected to pose any particular problems for the proposed correction algorithm, it is noted that backscatter models at these frequencies are currently underdeveloped and are limited for the most part to a single sediment type [33– 35].
2.3 Range dependent factors Other major influences on image intensity are range dependent, for example transmission and absorption losses. Spherical spreading is assumed for transmission losses giving an expected variation in intensity proportional to 1/R 2, where R is the slant range [2]. It is assumed that absorption losses, a function of 1/R with dependencies on water temperature, pressure and salinity, will not vary dramatically over the survey period. During data acquisition, range dependent losses are usually compensated by an applied time-varying gain (TVG). The TVG function increases amplification of later returns and is often highly variable with settings defined by the sonar operator. As signal processors, we are often required to work with such data without knowledge of the parameters used in the data gathering. It is for this reason that we are interested in calculating correction factors directly from the stored data.
2.4 Altitude variation Correction for intensity variations in sidescan imagery usually proceeds under the assumptions of a flat seabed and a constant altitude for the acquisition platform [6]. Under these conditions the range and angular dependent factors can be treated together. For shallow water surveys and for surveys over sloping seabeds, these assumptions do not generally hold and it is desirable to treat angular dependencies and range dependencies separately. Fig. 6 illustrates the influence of altitude on the expression of the sonar beam pattern on the seabed. The range at which a beam at a particular angle strikes the seabed is directly proportional to the sensor altitude. To achieve separation between angular and range dependent factors, we aim to calculate two correction factors, using estimates generated directly from the image data. This requires estimation of the sensor altitude at each sonar ping time. The altitude
& The Institution of Engineering and Technology 2008
Figure 6 Range at which a particular beam angle strikes the seabed is directly proportional to sensor altitude
information can also be estimated directly from the data by a bottom-tracking algorithm, which detects the first ground return in each line of data, Fig. 3. Calculation of the correction factors is robust to some bad estimates for ping altitudes. On correction, however, large errors will manifest as excessively dark or light bands in the imagery with poor compensation for the sonar beam pattern. For the majority of data sets, this is not a significant problem, assuming care is taken in selection of parameters for the bottomtracking algorithm. Where a few bad estimates remain, for example due to large objects in the water column, manual adjustments can be made if required.
3
Method
3.1 Imaging model For a point, p, the received intensity can be represented by I( p) ¼ G( p)R( p)B(u, p)g(u, p)Z( p)
(2)
where, G( p) represents the value of the TVG function applied at point p, R( p) covers the transmission losses due to spreading and absorption, B(u, p) is the intensity of the transmitted sound wave in the direction of p (the beam pattern) and g(u, p) is the backscatter intensity at p. The latter two parameters have angular dependencies, the first two parameters are dependent on the range from the transmitter to point p. Compensation for these four factors will leave Z( p) which represents, assuming a flat seabed, the variation in backscatter intensity due to surface roughness or texture.
3.2 Correction model The correction model proposed for compensation for G( p), R( p), B(u, p) and g(u, p) uses three multiplicative correction factors. The first is a quadratic function describing the variations in intensity associated with sonar altitude, C R( p) and this is coupled with an estimation of fixed gains across the sonar swath, C G( p), to provide compensation for all of the range dependent factors. The angular dependencies, due mainly to the IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 159 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org vertical beam pattern and grazing angle, are wrapped up in a third compensation factor, C u( p). Correction for point p is represented by J( p) ¼ I( p)CR ( p)CG ( p)Cu ( p)
(3)
where, I( p) is the intensity of a point in the original image and J( p) is the intensity of the corresponding point in the corrected image. Calculation of both C R( p) and C u( p) requires knowledge of the sensor altitude at each acquisition time. The first stage in the correction procedure, working from image data alone, is estimation of the sensor altitude. The bottom-tracking algorithm used for this purpose in this paper is based on the selection of maximum derivatives. These correspond to maximum change points in the original data which are used because the transition from water column returns to ground returns provides a very rapid and dramatic change in signal intensity. An alternative efficient bottom-tracking algorithm could be substituted.
3.3 Symmetry Variation in the range dependent correction factor C G( p) across the sonar swath is largely determined by the applied TVG function. Under certain circumstances it is advantageous to assume that this is symmetric across the port and starboard channels. Symmetry will provide the most robust performance where estimates are generated from data gathered over flat seabeds and is particularly useful where artefacts may corrupt one or both channels. Under other conditions, particularly for surveys carried out across a steeply sloping seabed or where independent or extreme gain settings have been used for each channel, it is preferable to produce estimates which treat each channel independently. The variation with altitude, C R( p), is always applied symmetrically. Ordinarily, the angular factors cannot be considered symmetrical. First, transducers are invariably imperfectly matched, meaning that perfectly symmetrical sidelobes are a rarity. Secondly, we wish to include the case of surveys running across a sloping seabed and in this case the geometric distortions of the beam pattern will be different in the port and starboard channels.
3.4 Estimation of the correction factors Estimation of the correction factors involves an iterative process which separates out intensity variations with an angular dependency from those displaying purely range dependencies. 3.4.1 Estimation by along-track averaging: Alongtrack averages of the sidescan data are used to define the characteristic variations in intensity across the sonar 160 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
swath. In practice, median values are used instead of means as these reduce the influence of textural variations, objects and artefacts within the images. The following notation is used: for an N M image I(n, m), n(0 n , N) provides the row index and m(0 m , M) gives the column index for a given pixel. The median over all rows in the image with respect to range is denoted by the symbol ˜I(m), m(0 m , M). Over a flat homogeneous seabed and at constant sensor altitude, intensity variation in range is inseparable from angular variation, This is not the situation where there is a change in sensor altitude, under which circumstances we can use data resampling to highlight the angular variations. Surveys over a seabed with slopes in the along-track direction can be dealt with by the porposed algorithm. We assume that slopes in the across-track direction are consistent over the survey area. 3.4.2 Resampling: In the presence of changes in sensor altitude, estimates of the angular intensity variation can be generated by aligning the data points by beam angle rather than by acquisition time, the latter being directly proportional to range. The realignment is achieved through a resampling scheme driven by the estimated altitude at each ping time. Resampling is achieved using a polyphase filter [36] with the sampling rate conversion determined by the rational factor dref/d(n), where dref and d(n), derived from the bottom detection, are sample numbers for the first returns in a fixed reference line and the nth data lines, respectively. Following data realignment, we can calculate the median over all rows with respect to angle. We denote the realigned image I 0 (n, m), with along-track median ~I 0(m). Figs. 7a and 7b show the data from Fig. 3 before and after resampling to realign by beam angle. Figs. 7c and 7d show the corresponding along-track median estimates ~I (m) and ~I 0(m). Note that the latter provides much finer representation of the beam pattern close to the water column, but becomes noisier towards the edges as fewer samples are available to contribute to the medians following resampling. The missing data are coloured black in the resampled image. 3.4.3 Intensity variation with altitude: Since we are now dealing with changes in sensor altitude, we must also account for differences in intensity arising from this additional range dependency. In the simplest case, it is expected from spreading losses that intensity will vary as 1/R 2 but in the recorded data this simple relationship is unlikely to hold due to the influence of other factors, notably the unknown TVG function. However, since range is directly proportional to
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 7 Side scan image a b c d
Before resampling After resampling to align data by beam angle Along-track median estimate for a, ˜I Along-track median estimate for b, ˜I0
sensor altitude, we can trace these intensity variations relative to the fixed reference altitude, dref and we proceed by fitting the quadratic given in (4) CR (d(n)) ¼ C1
2
dref d(n)
þC2
dref þ C3 d(n)
(4)
Recall that dref is the altitude for a reference data line. Ideally dref will be the lowest survey altitude, but in practice it is more likely to be the lowest altitude achieved over level terrain suitable for estimating the correction factors. A problem in fully automated determination of dref may arise due to grounding of the sensor during data acquisition, effectively giving a zero lowest altitude. Since we are working on sampled data, we must also ensure that there is a sufficient number of samples available to represent a rapidly oscillating
& The Institution of Engineering and Technology 2008
beam pattern at the reference altitude. In these circumstances, higher values for dref can be chosen. Fig. 8 illustrates the variation in mean intensity with altitude for the image in Fig. 3. The quadratic fit to these data points provides an estimate for the coefficients of C R. The large variations seen at altitudes 125 –135 result from platform instability during winching. We do not aim to compensate for these and they have only a small impact on the data fit. 3.4.4 Initialisation: For a given image, initial acrosstrack range and angular correction factors are derived from ~I (m) and ~I 0(m), respectively. A triangular weighting of ~I 0(m) is used, based on the assumption that angular effects will dominate closer to the transducer, whereas range effects will be increasingly dominant IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 161 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 8 Calculation of residual error from mean intensity variation with altitude a Mean intensity at each ping time is calculated from across-track means b Least squares quadratic fit to data (black line) provides estimate for correction factor C R
towards the edges of the image. C0G (m) ¼
1 ~I (m)
(5)
C0u (m) ¼
1 W (m)~I 0(m)
(6)
m W (m) ¼ (Wmax Wmin ) 1 1 þ Wmin M=2
Choosing limits of Wmax ¼ 1:9 and Wmin ¼ 0:1 ensures that WT (m) has an average weight of 1.0 and does allow 162 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
some expression of range dependent factors near the water column and some expression of angular factors at the edges of the image. The range dependent correction factor is initialised to a value of 1.0 over all samples. C G(m) and C u(m) are jointly normalised to unity so that overall image intensity is handled by the remaining correction factor, C R. 3.4.5 Application of correction factors: The generated correction factors are applied line by line to the sonar data. The altitude at each ping, as determined by the bottom-detection, is used to determine any resampling required for the angular
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org estimate. This altitude also provides the required input for the quadratic range correction factor. The gain estimate is applied directly. 0
J(n, m) ¼ I(n, m)CR (d(n))CG (m)Cu (m)
(7)
0
where, Cu is the correction estimate Cu resampled at (d(n)/dref) the original rate. It should be noted that wherever the survey altitude, d(n), drops below the reference altitude, dref , additional points must be extrapolated from the current estimate for Cu. The superscript notation allows us to consider part corrected imagery. For example, application of the angular and altitude-dependent correction factors to produce a part corrected image can be written as 0
J R,u (n, m) ¼ I(n, m)CR (d(n))Cu (m)
(8)
This part corrected image, J R,u (n, m), is used to produce an improved estimate for the remaining correction factor, C G. For the sake of clarity, given the number of parameters involved, it is useful to omit the dependencies on n and m for some of the more complex expressions. For example, in the remainder equivalent to of this paper J R,u0 should be considered 0 J R,u (n, m) and Cu equivalent to Cu (m). 3.4.6 Iteration: As noted above, refinement of the initial estimates for the correction factors follows an iterative procedure. Error in intensity variation with altitude is used to drive the convergence process. Fig. 9 summarises the procedure and shows the order in which the correction factors are re-evaluated. At each iteration, updated median estimates for C G and Cu R,u are generated from the part corrected images, Jt1 and JtG,R , respectively, CtG ¼
1 , ~ R, u J t1
Ctu ¼
1 ~ J 0G,R
(9)
t
The original data are corrected using the most recent estimates for C R and Cu . An updated gain factor, CtG , is generated from the part corrected image and this is used to generate a fresh angular estimate, Ctu . Using the fully corrected image, JtR,G,u , residual error is estimated from the remaining variation in intensity with altitude. 3.4.7 Error minimisation and convergence: Calculating a mean intensity across all ground samples for each line of data in the corrected image, JtR,G,u , provides a new set of mean intensity values for each altitude. At convergence, correction is expected to
& The Institution of Engineering and Technology 2008
Figure 9 Convergence algorithm
produce an image with a mean intensity of 1.0 across all ground samples for all altitudes. We can quantify the error through a least squares quadratic fit to the variation in mean level with altitude for the fully corrected image. The coefficients of the resulting error function, which has the same form as C R, are used to adjust the values of the three quadratic coefficients to produce CtR for the next iteration, (10). R þ dERt CtR ¼ Ct1
(10)
The factor d controls the adaption rate. Driving the convergence in this manner, it is only the values of the coefficients of C R that are directly altered. Updates to CG and Cu follow automatically from re-estimation of ~I and I~0 on the part corrected images. With the generation of each complete set of correction factors, a convergence test is performed. Changes in the mean intensity of the fully corrected image are used. In practice good convergence is usually achieved within 50 iterations, with changes in mean image intensity typically dropping to below 0.005% within 200– 300 iterations. The initialisation of C0u given in Section 3.4.4 provides a straightforward and convenient first estimate but is not essential. Convergence is rarely significantly delayed even when the procedure is started with a uniform initialisation for all correction factors.
4
Data reduction
The most processor intensive stage in the estimation procedure is data resampling. The speed of the algorithm can be dramatically increased by reducing IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 163 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org the number of data lines which require to be resampled. This is achieved by sorting the lines according to sensor altitude and averaging over all lines having the same altitude. For estimation on a Pentium 4, 3.2 GHz machine using a 2000 2000 pixel image, execution time can be reduced from around 1 h to under a minute. The exact degree of improvement is determined by the amount of altitude variation in the image used for estimation. Once the correction factors have been estimated, their application to the complete data set can be performed even more rapidly. The angular correction factor is resampled only once for each unique sonar altitude, with results stored as they are generated. Data correction follows the procedure outlined in (7). Typical processing times for a 2000 2000 pixel image, including resampling, would be between 2 and 15 s depending on the degree of altitude variation seen.
5
Results
5.1 Maerl beds 5.1.1 Data correction: The gain and angular correction factors calculated for the Geoacoustics data are presented in Fig. 10. These have been derived from the image given in Fig. 3. Difficulties in classification and segmentation of these data arise largely from extreme residual TVG effects. These manifest as a characteristic ramp, with onset at around 700 samples from the centreline in both the port and starboard channels in the estimate for the range dependent correction factor.
Figure 10 Range and angular correction factors calculated for the maerl bed data from the image given in Fig. 3
The estimate for Cu has one very intense lobe close to the water column in each channel. Figs. 11a and 11b show the outcome of application of these correction factors to a new image from the maerl data set. A target mean grey level is chosen for the corrected reference image. This target level is used as a scalar multiplier on all corrected data to raise values to suitable levels for display purposes. In the current case a target value of 48 has been chosen, with images displayed in an 8-bit image format (256 grey levels). The grey level for the original data has also been raised slightly for purposes of comparison. Note that highlights and shadows are well preserved, which is
Figure 11 Maerl data set a Raw data b Corrected data: maerl shows up as brighter backscatter regions. The image statistics are improved in the corrected data
164 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 12 Classified data using co-occurence statistics a Raw data b Corrected data Dark grey ¼ background sediment; light grey ¼maerly sediment; white ¼ rocks and other objects giving particularly high returns
particularly important for target detection [22] and some sonar classification methods [37].
appear as an even mid grey intensity and impact on classification performance.
5.1.2 Segmentation: Working on the raw data, segmentation of the sidescan images is made difficult because the variation in grey levels over the sonar swath is greater than the variation between seabed types. A powerful textural feature set has been used, based on the co-occurence statistics [38, 39], and two exemplar images of 200 200 pixels have been used for each of three seabed classes, clear sediments, maerl and rock. At best, around 80% accuracy can be achieved working on the unprocessed images, Fig. 12a. Misclassification between maerl and rock is most prominent across the rapid fluctuations in beam intensity near to the water column. Further from the centre line there is considerable confusion between clear and maerly sediments. Following data correction, the segmentation is considerably improved, Fig. 12b.
Further artefacts arise from the vehicle dynamics. In particular, the REMUS AUV rolls into turns. With each small course correction there is a slight roll, which affects the expression of the sonar beam pattern on the sea floor. Since this is not accounted for in the correction procedure, errors arise at each course alteration.
5.2 REMUS AUV data set 5.2.1 Data quality: There are many difficulties associated with these data, including strong surface returns and interference from other sensors which appear as intensely bright pixels in the images. A sample image, showing sections of the main texture types and including surface returns and sensor noise is given in Fig. 13. These strong signals tend to saturate the 8-bit data format used by this sonar, but a more serious difficulty arises from high responses in the strong sidelobe returns near to the water column. These produce larger saturated regions containing no texture information at all. Following correction they
& The Institution of Engineering and Technology 2008
Figure 13 Sample image showing strong texture features, surface returns and interference from other sensors IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 165 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org
Figure 14 Range and angular correction factors calculated for the REMUS AUV data
5.2.2 Correction factors: Correction factors estimated from two small, manually selected regions, 400 lines of data in total, are presented in Fig. 14. The complicated multilobed beam patterns are well represented in both channels. The gain correction factor is raised near to the water column as a result of the data saturation creating high levels which do not vary only with beam angle. Two concessions are made to aid the processing of these highly variable data. The first is to force the gain correction factor, C G, to be symmetrical over the port and starboard channels. This provides some additional noise resistance. The second change is to apply a small smoothing kernel (10 samples) to C G only. This is done primarily to mitigate against the influence of the strong surface returns in the original data. No other steps are taken to deal with the
Figure 15 Sample image showing good correction with sensor in level flight, but with roll-induced error and sensor power fluctuations, arrowed
surface returns or other artefacts. In the angular estimate, data resampling spreads the surface return over a wider range so that it is less influential. In general, we would avoid using any smoothing function on this estimate because of the complex beam pattern whose representation would be corrupted by the smoothing algorithm. The estimates for the quadratic parameters for altitude compensation for these data stabilise readily, within a few hundred iterations. For particularly noisy data a more advanced convergence procedure, for example, Markov chain Monte Carlo (MCMC)
Figure 16 Detail from large area mosaic of REMUS AUV data a Raw data b Processed data Banding is reduced in the corrected data, seabed textures are well preserved
166 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org simulation, could be substituted, with final estimates calculated from long term ergodic averages. For these data the differences between values after 500 iterations and ergodic averages over the final 1000 samples of a 2000 iteration run are small and have very little impact on overall performance. 5.2.3 The corrected imagery: The complex interplay of sensor power variations and interference, vehicle roll and an extremely complicated beam pattern presents a severe test and in various places the algorithm is seen to fail. Fig. 15 indicates what happens when the vehicle makes a small course adjustment. Over most of the image the correction is good, but with the course alteration, the vehicle roll affects beam pattern expression, leading to a poor match with the estimated angular variations. This image also contains some darker regions associated with sensor power fluctuations. These cannot be corrected by the current algorithm. On the plus side the surface returns and high intensities from interfering sensors do not unduly affect the correction algorithm, simply being passed through into the corrected data. A larger section of the data is presented in Fig. 16 which shows zoomed sections of mosaics built from several adjacent tracks before and after correction. The banding due to the strong beam pattern is significantly reduced in the corrected imagery, although some errors are apparent due to vehicle roll during course corrections. A detailed segmentation for the entire data set can be found in [40]. This used a variant of Pace and Gao’s spectral feature set [41] and incorporated a less well developed resampling correction scheme. A Markov Random Field (MRF) fusion engine was employed to further improve the classmaps by combining information from overlapping regions in adjacent sonar tracks.
6
Conclusions
This paper has detailed a fast, automated data correction procedure for sidescan sonar imagery. Estimates for three correction factors for fixed gains, angular-and range-dependent effects are generated from a suitable exemplar image. Estimated correction factors for two data sets gathered with different sonars under different conditions have been applied to large survey areas. Improved segmentation performance over the corrected images has been demonstrated. Variations in the image statistics over the sonar swath derived from sensor beam patterns and altitude changes are much reduced, with more stable means and
& The Institution of Engineering and Technology 2008
variances. This enables simpler image processing algorithms to be applied effectively to the data. The large area mosaics are visually more appealing and classification is improved through the ability to make use of a greater proportion of the sonar swath. Since the method cannot compensate for low SNR we may still expect a larger proportion of errors at the extreme edges of the noisy imagery. Shadow and highlight regions are well preserved in the corrected data. Platform stability is sometimes an issue and tackling the increased complexity in modelling sensor roll, pitch and yaw remains a topic for investigation. It is believed that these challenges could be met given sufficiently frequent, accurate and synchronised attitude data. Sidescan sonar remains a key sensor of choice for relatively low cost and high resolution large area surveys. The algorithms presented here are simple to apply and can assist marine scientists in rapid environmental assessment and have further applications in object detection, identification and classification. The inclusion of more complex high frequency ocean acoustics models will provide further information from sidescan sonar returns.
7
Acknowledgments
This research has been in part funded by the European Union under the project EVK-CT2001-00059 AMASON: Advanced Mapping with Sonar and Video. The authors wish to thank IFREMER for the use of their MATISSE video mosaicing software during AMASON and the NATO Undersea Research Centre for the Remus data set and for permission to use it in this paper.
8
References
[1] MAZEL C.: ‘Side scan sonar record interpretation’. Technical Report, Klein Associates, Inc., New Hampshire, USA, 1985 [2] URICK R.: ‘Principles of underwater sound’ (McGraw-Hill, New York, 1983, 3rd. edn.) [3] CUMMING I., BAST D.: ‘A new hybrid-beam data acquisition strategy to support scansar radiometric calibration’, IEEE Trans. Geosci. Remote Sens., 2004, 42, (1), pp. 3 – 13 [4] CERVENKA P. , DE MOUSTIER C. : ‘Sidescan sonar image processing techniques’, IEEE J. Ocean. Eng., 1993, 18, (2), pp. 108– 122 [5] SHIPPEY G., BOLINDER A., FINNDIN R.: ‘Shade correction of side-scan sonar imagery by histogram transformation’. Proc. OCEANS ‘94, September 1994, vol. 2, pp. 439 – 443 IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 167 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org [6] ANSTEE S.: ‘Removal of range-dependent artifacts from sidescan sonar imagery’. Technical Report DSTO-TN-0354, DSTO Aeronautical and Maritime Research Laboratory, Australia, 2001 [7] ALGAZI V., FORD G.: ‘Radiometric equalization of nonperiodic striping in satellite data’, Comput. Graph. Image Process., 1982, 16, pp. 287– 295 [8] CORSINI G., DIANI M., WALZEL T.: ‘Striping removal in MOS-B data’, IEEE Trans. Geosci. Remote Sens., 2000, 38, (3), pp. 1439 – 1446 [9] CRIPPEN R.: ‘A simple spatial filtering routine for the cosmetic removal of scan-line noise from Landsat TM P-type imagery’, Photogramm. Eng. Remote Sens., 1989, 55, pp. 327 – 331 [10] GADALLAH F., CSILLAG F.: ‘Destriping multidetector imagery with moment matching’, Int. J. Remote Sens., 2000, 21, (12), pp. 2505 – 2511 [11] HORN B., WOODHAM R.: ‘Destriping LANDSAT MSS images by histogram modification’, Comput. Graph. Image Process., 1979, 10, pp. 69– 83 [12] MOREIRA A. , MITTERMAYER J., SCHEIBER R.: ‘Extended chirp scaling algorithm for air and spaceborne SAR data processing in stripmap and ScanSAR imaging modes’, IEEE Trans. Geosci. Remote Sens., 1996, 34, (6), pp. 1123 – 1136 [13] POROS D., PETERSEN C.: ‘Methods for destriping landsat thematic mapper images - a feasibility study for an online destriping landsat thematic mapper image processing system (TIPS)’, Photogramm. Eng. Remote Sens., 1985, 51, pp. 1371 – 1378 [14] REIGBER A. , PAPATHANASSIOU K.: ‘Correction of residual motion errors in airborne repeat pass interferometry’. Proc. Int. Geoscience and Remote Sensing Symp, Sydney, Australia, 2001, pp. 3077– 3079 [15] REIGBER A., SCHEIBER R. : ‘Airborne differential SAR interferometry: first results at l-band’, IEEE Trans. Geosci. Remote Sens., 2003, 41, (6), pp. 1516 – 1520
[19] WEINREB M., XIE R., LIENESCH J., CROSBY D.: ‘Destriping GOES images by matching empirical distribution functions’, Remote Sens. Environ., 1989, 29, pp. 185– 195 [20] SCHOWENGERDT R.: ‘Remote sensing: models and methods for image processing’ (Academic Press, London, 1997) [21] STRUZINSKI W., LOWE E.: ‘A performance comparison of four noise background normalization schemes proposed for signal detection systems’, J. Acoust. Soc. Am., 1984, 76, (6), pp. 1738– 1742 [22] DOBECK G.: ‘Image normalization using the serpentine forward-backward filter: application to high-resolution sonar imagery and its impact on mine detection and classification’. Proc. SPIE, Orlando, FL, March 2005, vol. 5794, pp. 381 – 391 [23] CHEN X., VIERLING L., DEERING D.: ‘A simple and effective radiometric correction method to improve landscape change detection across sensors and across time’, Remote Sens. Environ., 2005, 98, pp. 63– 79 [24] HELLQUIN L., BOUCHER J.-M., LURTON X.: ‘Processing of highfrequency multibeam echo sounder data for seafloor characterization’, IEEE J. Oceanic. Eng., 2003, 28, (5), pp. 78– 89 [25] CAPUS C. , TENA RUIZ I. , PETILLOT Y. : ‘Compensation for changing beam pattern and residual TVG effects with sonar altitude variation for sidescan mosaicing and classification’. Proc. 7th. European Conf. Underwater Acoustics, Delft, The Netherlands, July 2004, pp. 827 – 832 [26] PRESTON J., CHRISTNEY A., BERAN L., COLLINS W.: ‘Statistical seabed segmentation - from images and echoes to objective clustering’. Proc. 7th. European Conf. Underwater Acoustics, Delft, The Netherlands, July 2004, pp. 827–832 [27] CLARKE J.H.: ‘Seafloor characterization using keelmounted sidescan: proper compensation for radiometric and geometric distortion’. Canadian Hydrographic Conf., Ottawa, Canada, May 2004
[16] SRINIVASAN R. , CANNON M. , WHITE J.: ‘Landsat data destriping using power filtering’, Opt. Eng., 1988, 27, pp. 939– 943
[28] SMITH C. , RUMOHR H., KARAKASSIS I. , PAPADOPOULOU K.-N. : ‘Analysing the impact of bottom trawls on sedimentary seabeds with sediment profile imagery. benthic dynamics: in situ surveillance of the sediment-water interface’, J. Exp. Mar. Biol. Ecol., 2003, 285 – 286, pp. 479 – 496
[17] TOUTIN T., CARBONNEAU Y. : ‘MOS and SEASAT image geometric corrections’, IEEE Trans. Geosci. Remote Sens., 1992, 30, (3), pp. 603– 609
[29] AMASON: ‘Advanced mapping with sonar and video‘, available at: http://www.ece.eps.hw.ac.uk/Research/ amason.html, 2001
[18] WEGENER M.: ‘Destriping multiple detector imagery by improved histogram matching’, Int. J. Remote Sens., 1990, 11, (5), pp. 859– 875
[30] GeoAcoustics: ‘Dual frequency sidescan sonar: specifications’, available at: http://www.geoacoustics.com/ specifications.htm, 2007
168 / IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169 doi: 10.1049/iet-rsn:20070032
& The Institution of Engineering and Technology 2008
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
www.ietdl.org [31] VON ALT C., ALLEN B., AUSTIN T., ET AL .: ‘Hunting for mines with REMUS: a high performance, affordable, free swimming underwater robot’. Proc. MTS/IEEE Int. Conf. OCEANS’01, 2001, vol. 1, pp. 117 – 122 [32] APL-UW : ‘High-frequency ocean environmental acoustic models handbook’ (Applied Physics Laboratory, University of Washington, 1994) [33] WILLIAMS K. , JACKSON D. , THORSOS E., TANG D., BRIGGS K. : ‘Acoustic backscattering experiments in a well characterized sand sediment: data/model comparisons using sediment fluid and biot models’, IEEE J. Ocean. Eng., 2002, 27, (3), pp. 376– 387 [34] GREENLAW F., HOLLIDAY D., MCGEHEE D.: ‘High-frequency scattering from saturated sand sediments’, J. Acoust. Soc. Am., 2004, 115, (6), pp. 2818 – 2823 [35] BROTHERS R., ROBB G., LEIGHTON T., HEALD G., WHITE P., DIX J.: ‘Very high frequency backscattering from a rough sediment’. Proc. 7th. European Conf. Underwater Acoustics, Delft, The Netherlands, July 2004, pp. 613 – 618
& The Institution of Engineering and Technology 2008
[36] PROAKIS J., MANOLAKIS D. : ‘Digital signal processing: principles, algorithms and applications’ (Prentice-Hall, New Jersey, 1996, 3rd. edn.) [37] MIGNOTTE M., COLLET C., PE´EREZ P., BOUTHEMY P.: ‘Markov random field and fuzzy logic modeling in sonar imagery: application to the classification of underwater floor’, Comput. Vis. Image Underst., 2000, 79, pp. 4 – 24 [38] HARALICK R., SHANMUGAM K., DINSTEIN I.: ‘Textural features for image classification’, IEEE Trans. Syst. Man Cybern., 1973, SMC-3, (6), pp. 610 – 621 [39] UNSER M.: ‘Sum and difference histograms for texture classification’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, PAMI-8, (1), pp. 118– 125 [40] REED S., TENA RUIZ I., CAPUS C., PETILLOT Y.: ‘The fusion of large scale classified side-scan sonar mosaics’, IEEE Trans. Image Process., 2006, 15, (7), pp. 2049– 2060 [41] PACE N., GAO H. : ‘Swathe seabed classification’, IEEE J. Ocean. Eng., 1988, 13, (2), pp. 83– 90
IET Radar Sonar Navig., 2008, Vol. 2, No. 3, pp. 155 – 169/ 169 doi: 10.1049/iet-rsn:20070032
Authorized licensed use limited to: Heriot-Watt University. Downloaded on November 12, 2009 at 13:23 from IEEE Xplore. Restrictions apply.
