quite severe with large data sets, such as multispectral or hyperspectral data, typical of remote sensing applications. [1, 2]. Various tasks in remote sensing deal ...
IMAGE REGISTRATION IN MULTISPECTRAL DATA SETS Hani Mahdi* and Aly A. Farag Department of Electrical and Computer Engineering, University of Louisville, Louisville, KY 40292, USA. *On sabbatical leave form Faculty of Engineering, Ain Shams University, Cairo, Egypt E-mail: {hmahdi,farag}@cvip.Louisville.edu - http://www.cvip.louisville.edu ABSTRACT The paper discusses the matching of two multispectral data sets using the Genetic Algorithm (GA) as a search technique for the global optimum estimates of the transformation parameters. It uses the simplest form of the affine transformation, where the scaling and rotation are ignored, to explain how the GAs, which are used in the registrations’ processes for the different bands, can cooperate. This cooperation between the GAs improves the speed of the matching process. The cooperation depends on the exchange of the intermediate results between the GAs. Therefore, the proposed approach demands the parallel realization of GAs. In addition, the proposed approach can be considered as a fusion method for the different decisions resulting form the registrations for the different bands. Landsat 7-Band and 4-Band aerial data set types are considered. The aerial data set type is used to explain how to the proposed approach speeds the matching process. 1.
reference? Data registration involves estimating the translation, rotation and scaling parameters from one data set (model) to another (data). In the case of ATR, a 3D model of a target can be establis hed and be used as a registration tool between different types of data sets [3]. Although this paper considers multispectral data sets , it focuses on the simplest form of the registration, i.e., where rotation and scaling are ignored. This scenario, as simple as it is in real world multimodality multisensory remote sensing applications, still provides a challenge for how to match these multispectral data sets over time [4]. The focus of this paper is to introduce a solution of this matching problem for multispectral data sets , using cooperative parallel genetic algorithms, one for each band. Examples of such data sets are shown Figures 1 and 2. The following section considers the traditional genetic algorithm and related works using GAs in the registration problems. Section 3 introduces the preliminary results of the registrations using both the traditional and the proposed cooperative GAs..
INTRODUCTION 2.
Image matching is a well-known approach in motion analysis, stereo reconstruction, automatic target recognition (ATR), and other image analysis applications. The typical approach is to compare one data set “model” to a population “data,” in order to decide on the best match. This comparison is based on a metric, such as the mean square difference (error). This matching process becomes quite tedious with color representation (R, G, B) where the three-bands are involved. The problem becomes quite severe with large data sets, such as multispectral or hyperspectral data, typical of remote sensing applications [1, 2]. Various tasks in remote sensing deal with the following scenario: given a collection of aerial measurements (taken over time) of a certain area, using various sensors (e.g., photographs, multispectral, hyperspectral, and other sensors), and various acquisitions parameters (e.g., height from earth, angle of projections, and number of bands). How one can register these data sets together to a common This work has been supported by Air Force Office of Scientific Research, AFSOR Grant f49620-01-1-0367.
0-7803-7622-6/02/$17.00 ©2002 IEEE
MATCHING USING GAs
Classical exhaustive search techniques for image matching use different objective functions, e.g. the correlation and the minimum square difference (error) metrics between the “model” and “data”. The exhaustive search is very time consuming, and impractical in remote sensing applications. Different searching techniques are used to get the optimal transformation parameters [5]. Among the widely used methods are Powell’s direction, downhill simplex, and steepest gradient descent. All the traditional searching techniques assume that the initial guess of the solution is in the global optimu m range. Different non-traditional methods are used for registering a gray (single band) model onto a gray (single band) data. Among these methods are the Genetic Algorithms (GAs) and the Simulating Annealing. A good survey for using GAs in the registration problem can be found in [6]. Using GAs in the registration problem referred first to Fitzpatrick’ work [7]. Very few works have addressed the usage of genetic algorithm in the regis tration of multispectral images and fewer considered the parallel GAs in the registration of multispectral data sets. This registration problem of a multispectral model onto a
II - 369
IEEE ICIP 2002
multispectral data set can be considered as a fusion problem as well as a registration problem, since the registration processes for different bands can yield different decisions. These decisions must be fused together for a final decision. A genetic algorithm (GA) is an iterative stochastic searching technique that has been shown to be useful in many difficult optimization problems. The GA is a global optimization technique that returns the global maximum value of an objective function. This function is called the fitness function. GA codes each potential solution as a string of genes what is called a chromosome. Each gene represents a variable in the problem; and each chromosome represents a potential solution in the solution space. A set of chromosomes are referred to as a population or a gene pool. According to the objective/fitness function, each chromosome gets a qualification/fitness score. The chromosomes in the pool are used to generate a new population (generation) of the chromosomes. The selection of chromosomes for reproduction is carried out according to their fitness. Essentially, two operations are used to produce the new generation; namely crossover and mutation. In crossover, chromosomes are selected in pairs randomly and combined to produce a pair of chromosomes in the new generation. In mutation, single chromosomes are selected randomly and altered randomly to produce a chromosome in the new generation. There are many versions (varieties) of GAs realization [8]. In practice, the usage of GA as a searching algorithm requires not only the selection of the type of GAs, but also the selection of the terminating criterion (condition) of the iterations, the representation (coding) schema of the searched variable(s) into gene(s), the determination of the selection strategy of the chromosomes that are used for generating the new generation, and the percentage of mutation and to whom the mutation will be applied to. Using GAs for matching a multispectral model onto a remote sensing data set, is a challenging problem. The difficulties arise from the natural huge data size and the possibility of inherent high frequencies in the data set. Different techniques are proposed to overcome these problems. Among these is the partitioning of the data set to manageable sizes [6]. Another technique is proposed to use a low resolution version of the model and the data set to get coarse estimates of the transformation parameters and then improves these estimates using the full resolution versions [9]. In this paper, the parallel implementation of the GAs for different multispectral bands is used to get more computationally efficient estimates of the transformation parameters, as will be discussed later.
The randomness of the GA operations assures having the chromosome, that is associated with the peak (global maximum) value of the objective function, in the population. However, this assurance assumes that the algorithm will run endless. In other words, there is no terminating condition for the GA. This is not a practical solution and generally one of two trends are used for termination the searching process. The first considers an upper limit for the iterations’ number. The second considers the history of the best estimated values between the generations as a termination condition. A combination of these two methods can be used. In all these three possibilities, the termination can happened while the best chromosome in the population is associated with a local maximum. Using the GA s in matching multispectral data sets, can add another possible terminating condition. This condition is the agreement between the best estimated variables’ values obtained from the different GAs that are used for matching the different bands. 3.
PRELIMINARY RESULTS
In the following, the matching process of multispectral model onto the data returns an estimate for the central location of the model in the data. This location will be indicated by (h, v), where h and v denote the horizontal and vertical distances (in pixels) from the model’s center to the upper left corner of the data image, respectively. Two data set types are considered. The first data set type is the Landsat 7-Band (169x169 pixels ) data set for an agriculture field, Figure 1(a). The model for this data set type is taken as the part centered at (55, 75), Figure 1(b). Its width and height are taken to be 99 and 111 pixels , respectively. The second data set type is the 4-Band (9200x6500 pixels ) aerial imager of the San Francisco area, Figure 2(a). The model for this data set type is taken as the part centered at (3333,555) , Figure 2(b). Its width and height are taken to be 777 and 999 pixels , respectively. The GA parameters are taken the same for both data set types except that for the first data set type the central location is coded using 8 bits for each variable, while for the second data set type the location is coded using gene with 16 bits. The searching space for the first data set type is taken assuming that the whole model is in the image, i.e. h ∈ Rh = [50,119] and v ∈ Rv = [56,113] . The search space is then about 4000 locations. While for the second data set, the range is taken to be the (about h ∈ Rh = [3000,4000]and 1,000,000 pixel),
v ∈ Rv = [0,1000]. The population size N is taken to be 200. In our realization, the best 10% of the chromosomes (known as the elite population) are taken as they are to the new generation without modification. Moreover, 10% of
II - 370
the new generated chromosomes including the elite population are subjected randomly to the mutation. The objective function is taken to be the inverse of the root mean square difference (error). Next we will introduce a measure for the difficulty of the searching problem using the GA . The range of any variable can be divided by its required resolution. In our case, the required resolution is 1, i.e. the center location must be estimated in integer values. The number of occurrences in the range of any variable is the population size N . According to the Poisson distribution, the probability of having x integer occurrences in an interval x −λ is given by λ e / x ! , where λ is the average number of occurrences in this interval. For the above examples λ = N / R , where R is Rh or Rv . The probability p of
replaced with the best estimated values obtained from other GAs. The chromosomes to be replaced are those with the least fitness. In this parallel realization, the termination of the iterations is the task of the supervisor process. The supervisor process receives the best value from each GA and distributes it to all other GAs. It uses these values to calculate two variables; TDNC and TAGR. TDNC (They Did Not Changed) states that the best values of the different GAs did not changed over a number of iterations. TAGR (They AGReed) denotes that the best values of the different GAs are identical. Table 3 shows the best values of each GA after each iteration. One iteration is executed after the supervisor process evaluates the TAGR as a true value and order the termination of the iterations. Table 3. The best values evolution using the cooperation.
have at least 1 value in this interval of the correct pixel (1 − e− λ ) . Therefore, the location is given by probabilit ies to be initially in the correct interval of the central location are 0.912 and 0.033 for the first and second data set types, respectively. The inverse ratio of these probabilities gives a measure to the ratio of the relative difficulties between the two considered matching problems.
Iteration Number h Band 1 v
We select for stopping the iterations a condition relating to the stability of the best solutions between generations. For the agriculture data, the registrations for all bands estimated correctly the center value (55,75) but after different number of iterations, as shown in Table 1. For the San Francisco data set, the registrations for all bands reached near, inexact, and two different values for the center location value (3333,555), due to the difficulty of the problem. The number of iterations and the resulting estimates are shown in Table 2. In the following, the paper introduces a proposal to accelerate the GAs decisions and fuse these decisions. Table 1: Different bands need different iteration numbers. Band Number 1 2 3 4 5 6 7 Number of 9 8 11 6 5 10 9 Iterations Table 2: Different decisions and iterations’ numbers. Band Number Number of Iterations Estimated h
1 12
2 23
3 19
4 11
3333
3333
3333
3333
Estimated v
553
557
557
557
The proposal assumes the parallel realization of GAs and is based on the exchange of the best estimated values after each iteration. Half of the population of each GA will be
0 3325 467
1 3325 600
2 3333 549
3 3333 557
4 3333 557
5 3333 557
Band 2
h v
3333 490
3333 490
3333 561
3333 553
3333 557
3333 557
Band 3
h v
3337 541
3337 541
3333 545
3333 557
3333 557
3333 557
Band 4
h v
3325 529
3333 553
3333 553
3333 557
3333 557
3333 557
Comparing the final results in Tables 2 and 3, it is easy to conclude that this cooperation actions fused the final estimates. Moreover, it saves 45 iterations, this is a hug amount of calculations because each iteration consist of comparing 200 templates ( the population size) and each of these templates include about 775,000 pixels. 4.
CONCLUSIONS AND FUTURE WORKS
The paper introduced a proposed cooperation mechanism between the GAs, that are used for matching the different bands of multispectral data sets. The proposed procedure is computationally efficient and it facilitates the fusion of the different decisions resulting from the registrations for the different bands. However, this cooperative procedure demands the parallel implementation of GAs and a supervisor process. Local computing facilities at the CVIP Lab enable us the usage of up to 64 parallel process at the same time. The introduced intuitive computationally efficient procedure need to be proved theoretically. REFERENCES [1] M. G-H. Mostafa, A. A. Farag and E. A. Essock, “Multimodality Image registration and Fusion using Neural Network,” Fusion2000, Paris, France, vol. 2, pages WeD3-3 -- WeD3-9, July 2000.
II - 371
[2] D. Landgrebe, “Information Extraction Principles and Methods for Multispectral and Hyperspectral Image Data,” in C. H. Chen, editor, Information Processing for Remote Sensing, Chapter 1,World Scientific 1999. [3] R. Chellappa, Q. Zheng, P. Burlina, C. Shekhar, and K. B. EOM, “On the Positioning of Multisensor Imagery for Exploitation and Target Recognition,” Proceedings of IEEE, vol. 85, no. 1, pp. 120-138, 1997. [4] R. R. Brooks and S. S. Iyengar, Multi Sensor Fusion: Fundamentals and Applications with Software, Prentice Hall, 1998. [5] William H. Press, Saul A. Teukolsky, William T. Vetterling, Brian P. Flannery, Numerical Recipes in C++: The Art of Scientific Computing, 2nd edition, Cambridge University Press, 2002.
[6] P. Chalermwat, "High Performance Image Registration for Remote Sensing," Doctoral Dissertation, Fall 1999, T. El-Ghazawi Director, George Mason University, Fairfax, Virginia. [7] J. M. Fitzpatrick, J. J. Grefenstette, and D. Van-Gucht, “Image Registration by Genetic Search,” Proceedings of IEEE Southeast Conference, 460-464, 1984. [8] W. Banzhaf, P. Nordin, R.E. Keller, and F. D. Francon, Genetic Programming: An Introduction, Morgan Kaufmann Publishers, 1998. [9] Prachya Chalermwat, Tarek El-Ghazawi, Jacquline LeMoign, “GA -Based Parallel Image Registration on Parallel Clusters,” IPPS/SPDP Workshops, pp. 257-265, 1999.
(a) Data bands (169 x 169).
(b) The generated model bands from the data in (a) centered at (55,75) with dimension (99x111),. Figure 1. The gray levels of the 7 bands of the Landsat data set and model (shown to scale).
(a) Data bands (9200 x 6500)
(b) The generated model bands from the data in (a) centered at (3333,555) with dimension (777x999), Figure 2. The gray levels of the 4-band Optec data (Green, Blue, Red, and Near Infrared) of the Golden gate area in San Francisco and the model (NOT shown to scale).
II - 372
