Pattern Recognition Letters 26 (2005) 625–632 www.elsevier.com/locate/patrec
Design of committee machines for classification of single-wavelength lidar signals applied to early forest fire detection Armando M. Fernandes a,*, Andrei B. Utkin b, Alexander V. Lavrov b,1, Rui M. Vilar a a
Departamento de Engenharia de Materiais, Instituto Superior Te´cnico, Av. Rovisco Pais 1, 1049-001 Lisbon, Portugal b INOV––Inesc Inovac¸a˜o, Rua Alves Redol 9, 1000-029 Lisbon, Portugal Received 17 February 2004
Abstract The application of committee machines composed of single-layer perceptrons for the automatic classification of lidar signals for early forest fire detection is analysed. The patterns used for classification are composed of normalised lidar curve segments, pre-processed in order to reduce noise. In contrast to the approach used in previous work, these patterns contain application-specific parameters, such as peak-to-noise ratio (PNR), average amplitude ratio (AvAR) and maximum amplitude ratio (MAR), in order to improve classification efficiency. Using this method a smoke signature detection efficiency of 93% and a false alarm percentage of 0.041% were achieved for small bonfires, using an optimised committee machine composed of four single-layer perceptrons. The same committee machine was able to detect 70% of the smoke signatures in lidar return signals from large-scale fires in an early stage of development. The possibility of using a second committee machine for detecting fully developed large-scale fires is discussed. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Lidar; Forest fire; Automatic detection; Committee machine; Single-layer perceptron
1. Introduction * Corresponding author. Tel.: +351 21 841 81 37; fax: +351 21 841 81 20. E-mail address:
[email protected] (A.M. Fernandes). 1 On leave from Russian Science Center ‘‘Applied Chemistry’’, St. Petersburg 197198, Russia.
Extending the principles of radar to the optical range, lidar (light detection and ranging) technology has found applications in the determination of the position and velocity of targets, measurement of the concentration of particles and
0167-8655/$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2004.09.012
626
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
chemical compounds in the atmosphere and in the study of the oceans (Measures, 1984). The successful application of a single-wavelength direct lidar technique to early forest fire detection was first demonstrated by the authors (Utkin et al., 2002a, 2003; Lavrov et al., 2003). This technique presents considerable advantages compared to currently used passive detection methods based on infrared and visible cameras, such as higher sensitivity, the possibility of accurately locating the fire, and the ability to detect the fire even when the flames are out of the line-of-sight from the observation point or against a lighted background. However, the widespread application of lidar to forest fire surveillance requires an automatic system for forest fire signature recognition to be made available. The present paper describes a system based on neural networks for the automatic detection of smoke plumes by lidar. The identification of smoke signatures in lidar curves is a complex task, because the shapes of the peaks originating from smoke plumes are significantly affected by random phenomena, such as weather conditions, aerosols in the atmosphere, etc. Neural networks have been effectively used for the analysis of signals that are strongly influenced by stochastic phenomena, such as radar (Haykin and Deng, 1991), sonar (Gorman and Sejnowsky, 1988), sodar (Pal et al., 1999), and lidar (Bhattacharya et al., 1997) signals. However, for a neural network to be capable of automatically recognising smoke plume peaks in lidar curves strongly affected by atmospheric noise, due to the wide range of possible shapes that those peaks may present, it should be composed of a large number of neurons, and considerable effort would be required to train and optimise the neural network structure. In previous papers (Fernandes et al., 2002; Utkin et al., 2002b), the authors showed that single-layer perceptrons (SLP) provide satisfactory results in the identification of smoke plume peaks in lidar curves, despite the fact that SLPs are restricted to solving linearly separable problems. In order to continue using SLPs (to keep training simple) and to be able to solve problems that are not linearly separable (Knerr et al., 1990, 1992), committee machines (CM) (Haykin, 1999, p. 351) composed of several SLPs were used in the present
work for the identification of smoke plume signatures in lidar curves resulting from small bonfires and large-scale fire experiments. The SLPs that constitute the CMs are formed of a single neuron, using a hyperbolic tangent as activation function. The SLP contains one input node with fixed input equal to +1 for bias representation and the activation function takes values between 1 and 1. The number of SLPs in a CM depends on the acceptable false alarm percentage. The SLPs from a CM are trained with the same smoke signatures and different atmospheric noise patterns and are expected to acquire expertise in solving distinct aspects of the whole classification task. In the present situation each SLP eliminates atmospheric noise patterns and passes on smoke signatures to the next SLP. An alarm is generated every time all the SLPs classify a pattern as a smoke signature. This multistage structure helps CMs to automatically select candidate atmospheric noise patterns for training each SLP, because the atmospheric noise patterns used for this purpose are chosen randomly from among those that previous SLPs erroneously considered smoke signatures. Using this pattern selection procedure it is possible to ensure that all types of atmospheric noise patterns are adequately represented by the few hundred atmospheric noise patterns chosen. This allows training sets to be composed with similar numbers of both types of patterns, an objective that would be impossible to achieve otherwise, because the number of atmospheric noise patterns that can be extracted from the experimental lidar curves is orders of magnitude greater than the number of smoke signature patterns. In a previous publication (Fernandes et al., 2002), the application of CMs to the classification of lidar signals was described, but the level of false alarms remained excessive for effective forest fire surveillance. The present work aims to improve pattern classification efficiency and to decrease false alarm probability by using, beside the CMs, a different type of training pattern that includes different combinations of application-specific parameters such as the peak-to-noise ratio (PNR), average amplitude ratio (AvAR) and maximum amplitude ratio (MAR), further to lidar
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632 0.04
Lidar Signal (a.u.)
curve segments. These segments correspond to regions where smoke signatures are likely to occur, due to the presence of a local maximum. Using them instead of the complete lidar return signal allows neural networks with a smaller number of input nodes and weights to be used for classification. Application-specific parameters have been employed in applications such as time series prediction (Deco et al., 1997) with excellent results because these parameters, as in the present case, are designed to contain the underlying structure of the data to be analysed. The different combinations of application-specific parameters with the two types of lidar curve segments help to reduce the tendency of SLPs to focus on the same pattern features, a tendency that leads to increased misdetection probability (Fernandes et al., 2003). The parameters PNR, AvAR and MAR measure the difference between the central peak amplitude and the amplitude of surrounding noise, facilitating feature identification by SLPs. The values of these parameters are proportional to the probability of occurrence of a smoke signature in a pattern. The addition of a parameter to a pattern does not significantly increase training time, because it means one weight is added to the SLP. Moreover, the calculation of PNR, AvAR and MAR is extremely fast, due to their simple definitions, so it does not significantly increase the time required for classifying each pattern. Finally, it is shown that CMs trained with smoke signatures resulting from small bonfires effectively detect smoke signatures from large-scale forest fires.
627
0.03
0.02
0.01
0.00 3
4
5
6
7
8
9
Distance (km)
Fig. 1. Lidar curve with smoke signature at 6.1 km.
and a laser beam divergence of 0.5 mrad. The laser energy was varied in the range 2–20 and 6–60 mJ for 532 and 1064 nm radiation, respectively. A second series of experiments was performed with 1064 nm laser radiation, a larger laser beam divergence (2 mrad) and pulse energy of 150 mJ. The wind velocity during the experiments ranged from nearly 0 to 50 km per hour. Lidar curves were collected during daytime, in sunny or cloudy weather, as well as during the night and at sunset. Each lidar curve resulted from the accumulation of between 32 and 256 laser pulse returns and is composed of 2000 backscattered-power measurements (see Fig. 1). Varying the number of pulse return signals enables different signal-to-noise ratios to be obtained, for a given set of experimental conditions. 2.1. Types of lidar curve segments
2. Pattern description Lidar curves containing smoke signatures were experimentally obtained, on one hand, from bonfires with a burning rate of 0.02 kg/s and, on the other hand, from large-scale fires with burning rates of about 10 kg/s, as part of the Gestosa 2003 test campaign (Viegas, 2000, 2003). The tests were carried out using a lidar based on a Nd:YAG laser. The experimental technique used was described in (Lavrov et al., 2003; Utkin et al., 2003). A first series of tests was carried out using two radiation wavelengths, 532 and 1064 nm,
The segmentation of lidar curves is possible because the smoke signature shape is independent of distance. It also simplifies calculating the exact distance to the fire in the event of an alarm. The lidar curves were therefore divided in regions-of-interest (ROI) consisting of 41 points, whenever a local maximum coincided with the centre of the ROI. The width of the ROIs was chosen so that, on one hand, the ROIs contain enough information to allow a low false alarm probability and, on the other hand, the SLPs have a small number of weights and are easy to train. The influence of
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
2.2. Parameters to characterise smoke plume signatures As previously mentioned, the patterns used in the numerical experiments consist of either the ROI or transformed lidar curve segments, associated with the problem-specific parameters, peakto-noise ratio (PNR), average amplitude ratio (AvAR) and maximum amplitude ratio (MAR) (see Fig. 2). The peak-to-noise ratio is an extension of the signal-to-noise ratio (SNR) concept to situations where the central peak results from atmospheric noise and not from a smoke plume. It was defined as the ratio between the peak amplitude (P in Fig. 2a) and the standard deviation of the ROI point values (D in Fig. 2a), excluding seven
PNR=P/∆
Lidar Signal (a.u.)
Linear Fit
P
∆ 0
10
20
30
40
Point Index
Regi on of In te rest
M1
Lidar Signal (a.u.)
the number of points in the patterns on the results achieved has been analysed elsewhere (Fernandes et al., 2003). The ROIs were normalised to minimum and maximum values of 0.9 and 0.9, respectively, in order to make them independent of the scale and of the time-dependent background originating from electronics and atmospheric noise. The normalisation also speeds up neural network training by avoiding weight values that are, in modulus, much larger or smaller than unity. The analysis of the ROI enables identification of smoke plumes based on the characteristic shape of smoke plume peaks in lidar curves, but some difficulties are expected due to the noisiness of the lidar signal. To overcome this limitation transformed lidar curve segments have been created on the basis of the normalised ROIs previously defined. Each segment consists of nine points whose values are proportional to the percentage of points from an ROI whose value exceeds a certain threshold between 0.9 and 0.9. In other words, the transformed lidar curve segments are the cumulative frequency distribution of values of an ROI. They enable the classification to be performed on the basis of the distribution of segment values, but some of the noise contained in the 41-point segments is eliminated. Again, the fact that the transformed segments contain a lower number of points than the 41-point segments helps to reduce the training time, by reducing the number of weights to be calculated.
M AR=A1 /A 2
M2
A1
A2
m1 m2
0
10
20
30
40
50
60
70
80
Point Index Region of Interest
A3
M3
AvAR=A3/A4
M4
Lidar Signal (a.u.)
628
A4
m3
m4
0
20
40
60
80
100
120
Point Index
Fig. 2. Definition of parameters PNR, MAR and AvAR.
points that, typically, correspond to the width of peaks resulting from small bonfire smoke plumes. The amplitude of the central peak and the standard deviation are calculated relative to the background, defined by a root-mean-square interpolation of the pattern points, excluding the
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
MAR ¼
M 1 m1 A1 ¼ M 2 m2 A2
The average amplitude ratio (Fig. 2c) is the ratio between the amplitude of the central peak in the ROI (A3) and the backscattered-power value in the central peak position (A4) of the line that connects the maxima of the 41 points to the left and to the right of the ROI (M3 and M4): AvAR ¼
A3 A4
A3 and A4 are calculated relative to the background defined by a straight line connecting the minima of the 41 points to the left and to the right of the ROI (m3 and m4). The purpose of the PNR is to compare the central peak amplitude with the average amplitude within the ROI points. The aim of the parameter MAR is to detect sudden increases in signal amplitude, by comparing amplitude within the ROI points with the amplitude of points to its right. The AvAR compares the actual ROI central peak amplitude with its expectable amplitude by calculating the ratio between the central peak amplitude and an average amplitude calculated taking into consideration the maximum values to the left and to the right of the ROI.
3. Single-layer perceptron training procedure The training of SLPs was carried out by minimising the sum-of-squares cost function, with an algorithm based on backpropagation and called polynomial approximation with periodically restarted conjugate gradient (PPRCG), described in previous papers (Fernandes et al., 2002; Utkin et al., 2002b). This algorithm uses conjugate gradient and polynomial interpolation to calculate the descent direction and the optimal learning rate
for each training epoch, respectively. The conjugate gradient is calculated from the second-order derivatives of the error function, estimated using a recursive method that is computationally less demanding than the explicit determination of second-order derivatives (Yu et al., 1995). In order to estimate the classification error reliably, a 10fold cross-validation method was used (Haykin, 1999, p. 217). In this method the available training patterns are divided into 10 groups. Then 10 neural networks are trained with the patterns in nine groups and validated with the patterns in the remaining group. The cross-validation error is defined as the sum of the errors of the 10 neural networks in the classification of the validation groups. The outcome of the 10-fold cross-validation method, besides the cross-validation error, is a neural network trained with all patterns in the groups. In order to avoid overfitting (characterised by an increase in the cross-validation error), the early stopping method (Prechelt, 1998; Haykin, 1999, p. 215) was applied. Early stopping was executed using the Neyman–Pearson criterion and relative operating characteristic (ROC) curves built using the results of different numbers of training epochs (Fernandes et al., 2003). ROC curves (Freking et al., 1998) are plots of the percentage of detected smoke signatures as a function of the false alarm
Neyman-Pearson False Alarm Level
100 Detected Smoke Signatures (%)
seven central points (linear fit in Fig. 2a). The maximum amplitude ratio (Fig. 2b), is defined as the ratio A1/A2, where A1 is the difference between the maximum (M1) and the minimum (m1) in an ROI, and A2 is the difference between the maximum (M2) and minimum (m2) values of the 41 points located to the right of the ROI:
629
x epochs y epochs z epochs
95
90
85
80 0
2
4
6
8
10
12
14
16
18
False Alarms (%)
Fig. 3. Example of the early stopping method in which it can be seen that ‘‘y epochs’’ (y is larger than x and smaller than z) provide the best true detection percentage for the established Neyman–Pearson criterion.
630
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
percentage. They are constructed by varying the neural network detection threshold, defined as the output value above which a pattern is classified as a smoke signature. According to the Neyman– Pearson criterion (Haykin, 1999, p. 28), the detection threshold and the largest number of training epochs before overfitting must maximise the percentage of detected smoke signatures, subject to the constraint that the percentage of false alarms should not exceed a prescribed value (see Fig. 3). The maximum percentage of false alarms allowed is chosen taking into consideration that decreasing the false alarm percentage causes a decrease in the percentage of detected events.
4. Committee machine design and results Since smoke signatures resulting from small bonfires present narrow peaks, which differ considerably from the complex shaped smoke signatures resulting from large-scale fires (see Fig. 4), two CMs were designed to detect smoke signatures resulting from small- and large-scale fires, respectively. The first CM is composed of four SLPs, with all classifying patterns made up of 41-point lidar curve segments, together with the PNR for the first two SLPs, PNR and MAR for the third and PNR, MAR and AvAR for the fourth. The second CM is composed of three SLPs. One SLP classifies Sm all Bonfire Large-scale Fire
Normalized Lidar Curve Segments
1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 0
5
10
15
20
25
30
35
40
Pattern Point Index
Fig. 4. Comparison of lidar curve segments of a smoke signature from a small bonfire and a smoke signature from a large-scale fire.
patterns formed by 41-point lidar curve segments and MAR, while the other two classify patterns formed by transformed lidar curve segments and AvAR or MAR. The number of SLPs in the CM structures was chosen so that the percentage of false alarms obtained by cross-validation does not exceed 0.012%. This represents a 10-fold improvement as compared to the results previously reported (Fernandes et al., 2002). Of the combinations tested, the combinations of lidar curve segments with application-specific parameters chosen for the SLPs were the ones that enabled the lowest percentage of misdetections by crossvalidation to be achieved. The first CM training set consisted of 141 smoke signatures from small bonfires and 670 atmospheric noise patterns. The second CM was trained with 69 large-scale fire smoke signatures presenting a shape so complex that they are misclassified by the first CM, and 400 atmospheric noise patterns. The lidar curve segments composing the atmospheric noise patterns for training both CMs were selected from a total of 30 864. The cross-validation error for misdetection percentage was calculated by adding together each SLP misdetection percentage obtained by crossvalidation, because undetected smoke signatures are discarded and do not pass to the next SLP. By contrast, atmospheric noise patterns classified as smoke signatures pass from one SLP to the next, so the CM cross-validation error for the false alarm percentage is the product of the individual SLPsÕ false alarm percentages obtained by crossvalidation. The first CM presented cross-validation error values of 13% for misdetections and 0.0076% for false alarms, while the second CM led to 33% and 0.011% for misdetections and false alarms, respectively. The larger percentage of misdetections for the second CM compared to the first one is explained by the greater width and complexity of large-scale fires compared to small bonfire smoke signatures. Even though using 10-fold cross-validation and test sets simultaneously is not a standard procedure, two test sets were used for checking the generalisation ability of the first CM and the representativeness of the training and validation sets. It was not possible to build a similar test set
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
for the second CM due to the insufficient number of representative smoke signatures from largescale fires. For a test set containing 880 small bonfire smoke signatures and 29 442 atmospheric noise patterns the first CM presented classification errors of 7.4% and 0.041% for misdetections and for false alarms, respectively. The high classification efficiency achieved demonstrates that the 141 small bonfire smoke signatures and 670 atmospheric noise patterns used for training are representative of the larger sets of patterns, composed of 880 smoke signatures from small bonfires and 29 442 atmospheric noise patterns, and that it is possible to use a small number of patterns for training, an important result because it enables the sampling required for new CMs to be reduced. The ability of a CM trained to identify small bonfire fire signatures to detect large-scale fires was tested by feeding into the first CM a test set composed of 232 smoke signatures resulting from large-scale fires. The CM correctly classified 70% of the patterns. This relatively good performance is explained by the similarity between peaks recorded at the edge of a large fire smoke plume and those corresponding to small bonfires. The 30% large-scale fire smoke signatures misdetected by the CM were predominantly recorded at an advanced stage of the fire (more than 5 min after its start). They failed to be correctly classified because the corresponding peaks are broad and present a complex shape, due to the large size and complex internal structure of the smoke plume, in particular when smoke is spread by wind. These facts confirm that small bonfire smoke signatures can be used for training an automatic system for forest fire detection. Applying the first CM to the classification of lidar signals obtained by accumulating 32 laser pulses at 10 Hz laser pulse frequency leads to about one false alarm every 3.4 min. It is possible to increase the time lapse between false alarms by checking alarms through the accumulation and analysis of a new lidar curve in the same direction.
5. Conclusions Using a method to prevent overfitting based on ROC curves and the Neyman–Pearson criterion, it
631
was possible to build a committee machine composed of four single-layer perceptrons that was optimised for automatic early forest fire detection. This committee machine was trained using small bonfire smoke signatures and atmospheric noise patterns and presented a 93.6% detection efficiency with only 0.041% of false alarms. It misdetected 30% of smoke signatures from large-scale fires because the shape of these smoke signatures is very different from those of the bonfires used to build the training set. A second committee machine trained to detect large-scale fires was built, and led to 33% misdetections and 0.011% false alarms. Both committee machines presented high classification efficiency due to the inclusion in the patterns of the parameters peak-to-noise ratio (PNR), average amplitude ratio (AvAR) and maximum amplitude ratio (MAR), which facilitate the single-layer perceptronsÕ task of identifying relevant features. The two CMs developed are able to detect different types of smoke signatures, so their cooperative use leads to better detection efficiency. The major drawback of such a system is the false alarm percentage, which corresponds to the union of the false alarm sets of each CM. It is possible to use only the first CM when the lidar scanning time is short enough to enable early detection of fire, before the smoke plume increases and spreads due to wind. Variation of the wavelength, pulse energy, and divergence of the laser beam used did not noticeably affect the smoke signature shape and, consequently, the CM efficiency. This provides some flexibility concerning the specification of the lidar, helping to reduce development costs.
Acknowledgments A.M. Fernandes gratefully acknowledges PhD grant SFRH/BD/2943/2000 from Fundac¸a˜o para a Cieˆncia e a Tecnologia. This research is partially supported by a grant from Ageˆncia de Inovac¸a˜o, Portugal (project FOGO!). The authors are grateful to the Portuguese Air Force, to Professor Xavier Viegas (University of Coimbra), to ADAI staff and to Mr. Anto´nio Fernandes for their help in organising the field experiments.
632
A.M. Fernandes et al. / Pattern Recognition Letters 26 (2005) 625–632
References Bhattacharya, D., Pillai, S.R., Antoniou, A., 1997. Waveform classification and information extraction from LIDAR data by neural networks. IEEE Trans. Geosci. Remote Sensing 35 (3), 699–707. Deco, G., Neuneier, R., Schurmann, B., 1997. Non-parametric data selection for neural learning in non-stationary time series. Neural Networks 10 (3), 401–407. Fernandes, A., Utkin, A.B., Vilar, R., Lavrov, A., 2002. Recognition of smoke signatures in lidar signal with a perceptron. In: SCI2002––Sixth World Multiconf. on Systemics, Cybernetics and Informatics, Orlando, USA, vol. 9, pp. 504–509. Fernandes, A., Utkin, A., Lavrov, A., Vilar, R., 2003. Classification of lidar signals by committee machines applied to automatic forest fire detection. In: JCIS2003––Seventh Joint Conf. on Information Sciences, Cary, NC, USA, pp. 1585– 1588. Freking, A., Biehl, M., Braun, C., Kinzel, W., Meesmann, M., 1998. Receiver operating characteristics of perceptrons: Influence of sample size and prevalence. Technical Report WUE-ITP-98-049, Institut fur Theoretische Physik, Universitat Wurzburg, Germany. Gorman, R.P., Sejnowsky, T.J., 1988. Learned classification of sonar targets using a massively parallel network. IEEE Trans. Acoust. Speech Signal Process. 36 (7), 1135–1140. Haykin, S., 1999. Neural Networks: A Comprehensive Foundation. Prentice Hall, London. Haykin, S., Deng, C., 1991. Classification of radar clutter using neural networks. IEEE Trans. Neural Networks 2 (6), 589– 600. Knerr, S., Personnaz, L., Dreyfus, G., 1990. Single-layer learning revisited: Stepwise procedure for building and training a neural network. In: Neurocomputing: Algo-
rithms, Architectures and Applications NATO ASI Series, vol. F68, pp. 41–50. Knerr, S., Personnaz, L., Dreyfus, G., 1992. Handwritten digit recognition by neural networks with single-layer training. IEEE Trans. Neural Networks 3, 962–968. Lavrov, A., Utkin, A.B., Vilar, R., Fernandes, A., 2003. Application of lidar in ultraviolet, visible and infrared ranges for forest fire detection. Appl. Phys. B 76, 87–95. Measures, R.M., 1984. Laser Remote Sensing. John Wiley & Sons, New York. Pal, P., Mukherjee, A., Acharya, S., Das, J., 1999. Continuous detection of atmospheric patterns from SODAR signals. Signal Process. 74, 153–168. Prechelt, L., 1998. Automatic early stopping using cross validation: Quantifying the criteria. Neural Networks 11, 761–767. Utkin, A.B., Lavrov, A.V., Costa, L., Simo˜es, F., Vilar, R., 2002a. Detection of small forest fires by LIDAR. Appl. Phys. B 74 (1), 77–83. Utkin, A.B., Fernandes, A., Simo˜es, F., Vilar, R., Lavrov, A., 2002b. Forest-fire detection by means of lidar. In: ICFFR2002––IV Internat. Conf. on Forest Fire Research, Luso, Portugal. Millpress, Rotterdam, p. 58. Utkin, A.B., Fernandes, A., Simo˜es, F., Lavrov, A., Vilar, R., 2003. Feasibility of forest fire smoke detection using lidar. Internat. J. Wildland Fire 12 (2), 159–166. Viegas, X., 2000. GESTOSA 2000––Experimental fires in shrub vegetation in Central Portugal. International Forest Fire News 23, 102 (on-line version http://www.uni-freiburg.de/ fireglobe/iffn/country/pt/pt_5.htm). Viegas, X., 2003. Gestosa 2003 News (on-line version http:// www.uni-freiburg.de/fireglobe/iffn/country/pt/pt_5.htm). Yu, X.-H., Chen, G.-A., Cheng, S.-X., 1995. Dynamic learning rate optimization of the backpropagation algorithm. IEEE Trans. Neural Networks 6 (3), 669–677.
