Infrared Flame Detection System Using Multiple Neural Networks Javid J. Huseynov†*, Shankar Baliga*, Alan Widmer*, Zvi Boger‡
[email protected],
[email protected],
[email protected],
[email protected]
Abstract – A model for an infrared (IR) flame detection system using multiple artificial neural networks (ANN) is presented. The present work offers significant improvements over our previous design [1]. Feature extraction only in the relevant frequency band using joint time-frequency analysis yields an input to a series of conjugate-gradient (CG) method-based ANNs. Each ANN is trained to distinguish all hydrocarbon flames from a particular type of environmental nuisance and ambient noise. Signal saturation caused by the increased intensity of IR sources at closer distances is resolved by adjustable gain control.
I. INTRODUCTION The detection of natural phenomena in safety applications carries the burden of ensuring both the reliability of the detection and the fast response of the system in operation. The study of fire occupies a unique niche in the world of science and engineering because an unwanted fire is considered a failure in the sense that it is not a desirable outcome and is to be avoided [2]. Therefore, the early detection of hydrocarbon flames in an industrial facility serves as means of avoiding such failures. In addition to the robust identification of the undesired phenomenon, a flame detection system should also be able to distinguish it from sources of environmental nuisance and ambient noise. To ensure the adherence of a flame detection system to these requirements prior to its deployment, various regulatory compliance standards, such as FM 3260 [3], EN 54-10 [4] and ULC/ORD-C386 [5] were developed. These standards subject the detection system to a set of reproducible flame and environmental nuisance sources to test and certify its performance upon deployment. Strict flame performance regulations have in the past guided the designs of a variety of expert systems based on a limited set of classical time- and frequency-domain signal processing algorithms. These systems offered simplicity and the definitive reproducibility of performance in standard compliance testing, mostly designed to avoid failures in detection of flames (false negatives). Yet, beyond the regulatory standards, these systems offer very little flexibility in terms of avoiding the identification of nuisance and ambient noise sources as flames (false positives). _________________________________________________ † School of Information and Computer Science, University of California Irvine, Irvine, CA 92697 * General Monitors, Inc., 26776 Simpatica Circle, Lake Forest, CA 92630 ‡ OPTIMAL – Industrial Neural Systems, Ltd., Be’er Sheva 84243, Israel
Particularly, in the IR sensor-based expert systems, the addition of environmental nuisance (such as heat, hot surface radiation, sunlight, welding) in the same IR wavelength and frequency spectrum as hydrocarbon flames makes the rejection of false positives nearly impossible. Some system designers [6-8, 23, 24] attempted to ease the problem with false positives by adding extra sensors at various wavelengths and restricting the system with a set of more elaborate expert rules. However, in the current market, this solution is extremely pricy in terms of both development and manufacturing. As opposed to expert systems, adaptive systems are poised to push the performance in both false positive and false negative cases far beyond the scope of regulatory standards at no extra cost. The clear advantage of adaptive detection systems based on fuzzy logic [9] or neural networks [1, 1016] is in their reliance on the pattern of the signals rather than on fixed magnitude or phase. For example, an ANNbased detection system can be trained on the data collected from one or more IR sensors observing a variety of falsepositive and false-negative cases. Successful training can automatically derive a set of unique input-output correlation rules that would, otherwise, have to be derived by an expert. Hence, the adaptive approach offers a greater classification capability in a much shorter development time. In [1], we proposed a complete detection scheme, based on a single large-scale neural network, for classifying IR flame sources from non-flames. While showing a satisfactory performance in the product through extensive testing, this solution had a relatively slow response time (> 10 seconds) and did not meet some regulatory standards. In this paper, we propose a new multiple-ANN-based scheme, which offers faster response time (< 6 seconds). This improved design was implemented in the first industrial flame detector [16] using ANN, which was certified per FM 3260 and ULC/ORD-C386 and was successfully tested to meet EN 54-10 regulatory performance standards for flame detectors and is currently deployed in field operation. In section II, we describe some related work as well as our earlier design and discuss its weaknesses. In Section III, we present a new design based on multiple small neural networks. Section IV summarizes our results and provides a conclusion. II. PREVIOUS WORK Over the last two decades, the application of neural networks for the classification of sources has been a subject of research in fire detection systems. Perhaps, the earliest example was the work by Okayama et al [10] using odor
sensors together with a backpropagation neural network to distinguish smoldering fires from environmental noise such as coffee powder and perfume. In subsequent work [11], they experimented with inputs from carbon monoxide, temperature and smoke sensors to obtain fire probability. Expanding on Okayama’s research, Milke et al [12, 13] applied neural networks to a wider range of sensors and fire test cases, such as those from flammable liquid, paper, cotton, cardboard, etc. Their experiments correctly identified 62% of smoldering fire sources and 87% of nuisance and ambient non-fire sources. Ishii et al [14] used a time-delayed neural network (TDNN) model to distinguish fires from non-fires in a limited number of tests. TDNN relies on historical information, and in the model presented in [14] the TDNN records a transient nature of fire and non-fire conditions. In our approach, for extracting the ANN input features, we also look at the signal over a data window of the past 2.5 seconds. However, in our design each data window is processed independently, and no correlations are established between the consecutive data windows for the ANN input. In [15], Chen et al proposed using the Fourier Transformbased IR spectroscopy (FT-IR) of gas phase products along with ANN to distinguish flaming and smoldering fires from environmental nuisance. Their ANN model was formulated using the Linear Vector Quantization (LVQ) approach with a hidden Kohonen layer. Their training model implemented with NeuralWorks software [17] produced a 96% success rate in classification and was extensive in analyzing the contribution of various gas types to flaming and smoldering fires. But it wasn’t designed for implementation in a product that must comply with regulatory agency requirements as in our case, i.e. the false negative error rate in our case has to be at 0%. In our previous work [1], we presented a model based on a single large-scale backpropagation ANN based on the Conjugate-Gradient (CG) method [18], utilizing PCA-CG algorithm [19] for training. This design had an average 95% classification success rate using a single network on all test cases. While passing the majority of the reproducible compliance tests, this solution, however, also had some inherent limitations which were revealed during the testing: 1.
2.
3.
4.
The training data collected from some nuisance sources outside of the flame frequency band did not contribute to an improvement in classification. High-resolution frequency analysis on a longer data collection window resulted in slower response time but did not contribute to significantly better classification. The scheme using a single large-scale ANN to classify all possible flame and non-flame scenarios proved to be less efficient than having several small ANNs, each classifying a limited subset of non-flame nuisance targets from all flame targets. The saturation of the IR signal from a detector placed at distances less than 40 ft from the target yielded an ANN-based classification scheme completely useless at those distances.
5.
The combination of flame and non-flame sources occurring simultaneously must be classified as a flame. A single large-scale ANN proved inefficient at this task much of the time.
In this work, without changing the training algorithm, we propose restricting the feature frequencies used as inputs to the ANN to be closer to the flame flickering frequency band (5 – 15 Hz). We also propose to shorten the length of the data collection and processing window, which subsequently reduces the response time of the system by a factor of two. We introduce a new classification scheme based on multiple neural networks applied in series all using the same input data during the classification. Each network is trained with data from only two major types of target phenomena, i.e. flame vs. sunlight, flame vs. hot surface without light emission, flame vs. heated surface with light emission, etc. Finally, we propose a scheme to resolve the signal saturation issue using an adjustable gain control mechanism applied to the system to produce usable feature inputs for the ANN. III. THE NEW CLASSIFICATION MODEL A. Input feature generation As mentioned earlier, in [1] we proposed a signal processing scheme that makes use of a Short-Time Fourier Transform (STFT) [20] applied to the data window W(n) of length N = 512 samples, and time shift ∆ = 25 samples as shown in Figure 1.
Figure 1. IR signal processing
The data sampling rate was set at 10 ms, and the Hamming data windowing method of the form 1⎧ ⎛ 2πn ⎞⎫ W Hm (n) = ⎨1.08 − 0.92 cos⎜ ⎟⎬ 2⎩ ⎝ N − 1 ⎠⎭ was applied upon every time shift to the newly collected time window, in order, to diminish the spectral leakage effect [21]. This design produced a feature frequency spectrum in 0 – 50 Hz band at 0.2 Hz resolution from each of the four sensors. All of this frequency data, combined with the averages of the raw signal x(n) in the time domain, formed a vector of 1028 ANN input features.
However, our previous work [1] and experiments have shown that the hydrocarbon flame flickering produces an infrared signal with distinguished magnitudes in the 5 – 15 Hz frequency band. Thus, training the ANN on frequencies far outside of this band does not contribute to a better classification of flame and non-flame targets. To the contrary, it adds extra nuisance and ambient noise to the training data and significantly complicates the training process. In particular, the large amounts of ambient IR noise collected during vibration tests in the frequency band of 20 – 50 Hz contained no useful feature information for training and classification. Another empirical observation was the fact that the signal was being over sampled using a 512-point STFT (0.2 Hz resolution). A 256-point STFT (0.4 Hz resolution) would be sufficient for extracting classifiable input features. The main benefit here is the fact that reducing the size of data window W(n) from 512 to 256 samples also reduces the response time of the detection system by a factor of two. Besides this fundamental improvement, the reduction in the size of STFT also facilitates the efficiency of computation in the embedded system and creates an opportunity for using more than one ANN in the reduced cycle time.
In our experiments, we identified three major types of false positive sources, some of which overlap in features: • Heated surfaces with light emission (arc welding, industrial heaters); • Modulated and stationary hot surfaces without light emission (hot plates, hot air flow, hot air gun); • Direct, modulated, reflected sunlight and bright light surfaces (incandescent and luminescent lamps, arc welding);
Figure 3. Multiple ANN application.
Figure 2. ANN Input Feature Generation.
In the new signal processing design for generating ANN input features, shown in Figure 2, we have used four 256point Fast Fourier Transforms (FFT) [22], one for each IR sensor channel. In the output real FFT vector, we have chosen only 50 values in the index range 3 to 52 (1.2 – 20.8 Hz). Combined between four sensors, this model of preprocessing produces a 200-input ANN feature vector.
B. ANN Classification Improvements In previous work [1], we proposed using a single largescale ANN based on PCA-CG algorithm [14] with 1028 input features. This approach significantly complicated the training process and the flexibility of the classification. Briefly, the system with a single comprehensive ANN failed to distinguish flame sources from a heater at distances closer than 10 feet and certain direct sunlight conditions. It was impossible to train a single network which would perform well on all the cases involving various false positives differing in physical nature but overlapping in the IR domain with certain hydrocarbon flames.
So we decided to subdivide the single large-scale ANN into multiple small sized ANNs, each separately trained to distinguish all flames from only a given set of false positive nuisance and noise sources. The new design is presented in Figure 3 for arbitrary N number of ANNs. Here, each ANN uses the same set of input features and makes flame vs. false decision. In order for the final decision to be flame, all ANNs have to produce a positive flame decision, otherwise, the final decision is false. C. Gain Control An important problem that usually arises in signal processing is signal saturation [21]. An excessive intensity of the analog signal from a sensor may yield an analog signal which is cut off (or saturated) at the range limits of the analog-to-digital converter, as shown in Figure 4. The saturated signal looks like a square wave, and its FFT does not produce valid frequency spectrum information, which ultimately invalidates the input to ANN and the whole scheme. The scaling of the converted signal within the converter’s range is dependent on the electronic gain of the circuit, which is controllable by the embedded software. Hence, to alleviate the signal saturation effects, we have come up with an expert algorithm, which constantly tracks the raw signal amplitude between the limits of VMIN + ∆ and VMAX − ∆ as shown in Figure 4. As soon as the amplitude of the signal
falls above or below this range, the system will adjust the electronic gain and rescale the signal back to the range.
The structure of the ANN hidden and output layers was the same as in previous design [1]: 5 hidden neurons and a single output neuron, using a unipolar activation function. The training and testing root-mean square (RMS) error rates in nuisance cases for each ANN are presented below in Table 1. The training algorithm converged in an average of 100 – 150 epochs. TABLE 1 RMS ERROR RATES FOR EACH ANN IN NUISANCE CASES
ANN
Figure 4. Signal Saturation and Gain Control.
This improvement to the design enabled the detection system to take advantage of the neural network classification of the intensive phenomena at closer distances. For example, n-heptane flame burning in a 1 square foot pan at 40 feet causes immediate signal saturation at the nominal gain, which made it impossible for the ANN to classify this particular phenomenon under the previous design [1]. In the new design, because of the gain control mechanism, the signal saturation happens at 10 feet or closer with minimal adjusted gain. Subsequently, in the current design, the ANNs can effectively distinguish such an intense flame phenomenon from non-flames at or above 10 feet.
IV. EXPERIMENTAL IMPLEMENTATION AND TEST RESULTS The design described in previous section has been implemented in the FL4000 flame detector from General Monitors [16]. The following subsections describe the training implementation, results, and classification performance of the final instrument.
A. ANN Training Four independent ANNs have been trained using four data sets with some overlapping data. As in previous work, the data was collected from the IR sensors observing various flame and non-flame nuisance and noise conditions regulated per FM 3260, EN 54-10 and ULC/ORD-C386 requirements. These conditions included n-heptane, propane, butane flames at distances from 0 to 250 feet, direct, reflected and modulated sunlight, arc welding, modulated heater with light source, modulated hot surface with and without a light source, flashlight, incandescent and luminescent light, vibration and other non-flame nuisance. The training program ran in MATLAB 7.1 software on a Windows PC. Each of the four ANNs had inputs of 30,000 samples (30,000 x 200 matrix), out of which 70% were randomly chosen as a training set, 10% for cross validation and 20% for testing. The target was a single column indicating either flame (1) or non-flame (0) condition.
Training Error Testing Error
(1) Flame vs. Welding
(2) Flame vs. Light
(3) Flame vs. Heat
3.2% 5.6%
2.8% 4.7%
3.9% 6.3%
(4) Flame vs. Hot Surface 1.8% 3.6%
B. Classification Performance Results The trained ANN model for classification was implemented in the embedded system on a Texas Instruments (TI) F2812 Digital Signal Processor (DSP), using C program with a virtual floating point arithmetic library from TI. Extensive performance tests were conducted on the final instrument to comply with the regulatory standards. The test results show that, in flame response performance, our ANNbased design is equal or superior to some known expertsystems for IR flame detection [23, 24]. Particularly, our design offers the on-axis range of 230 feet to detect a nominal n-heptane flame, burning in a 1 square foot pan, with a response time of less than 6 seconds. The highest onaxis range ever offered by an expert system [23] for the same source is 210 feet with a response time of over 10 seconds. TABLE 2 COMPARATIVE PERFORMANCE IN FALSE POSITIVE CASES: ANN-BASED DESIGN VS EXPERT-SYSTEM
Nuisance (False Positive) Sources Arc welding @ 190A DC Halogen Lamp (500 W) Fluorescent Lamp (25 W) Incandescent Lamp (60 W) Radiant Heater (1,500 W) Sunlight
Min. Immunity Distance ANN-based Expert (feet) (feet) 15 40 2 8 0 3 1 3 1 3 0 0
In addition, our nuisance testing results show advantages of the ANN-based design over expert systems in eliminating false positives (non-flame sources identified as flames). In Table 2, the comparative performance results in rejecting nuisance sources as non-flames are presented for the ANNbased design vis-à-vis the best achievements [23, 24] of the expert systems. The table lists the minimum “immunity” distances at which the nuisance source is identified as a false positive.
[3] TABLE 3 COMPARATIVE PERFORMANCE IN FALSE POSITIVE CASES: CURRENT VS PREVIOUS ANN-BASED DESIGN
Nuisance (False Positive) Sources Halogen Lamp (500 W) Fluorescent Lamp (25 W) Incandescent Lamp (60 W) Radiant Heater (1,500 W) Sunlight
Min. Immunity Distance Multiple Single ANN (feet) ANN (feet) 2 10 0 4 1 12 1 10 0 > 0 (fail)
As shown in Table 3, our current multiple-ANN design also showed visible improvements over our previous design with single large-scale ANN [1] in eliminating false positives. The flame response time was improved by two times, while the range remained the same.
[4]
[5] [6]
[7] [8]
[9]
[10]
[11]
V. CONCLUSION Several improvements to the previous design [1] of an industrial IR flame detection system using ANN are presented. Banding feature frequencies close to the flame flickering range of 1 – 20 Hz removes unnecessary noisy input from the ANN, contributes to faster training and improved classification success rate. The input reduction also enables using more than one ANN in series for classification, further contributing to lower classification error rates. Additionally, the signal saturation problem has been addressed using a gain control mechanism, which improves the quality of collected data for input to ANN at distances closer to the detected phenomenon. The detection system described in this paper has been fully implemented in the industrial IR flame detector [16] by General Monitors, which was certified per North American FM 3260 and ULC/ORD-C386 standards, and was tested to meet the European EN 54-10 regulatory standard for industrial flame detection. Presented results show that the flame detector using our ANN-based classification method achieves longer range (up to 230 ft) of flame detection at shorter response times than those currently provided by the expert systems. At the same time, it provides for exceptional discrimination against non-flame sources of environmental nuisance and electronic noise. Presented improvements over our previous design [1] include the two-fold reduction in flame response time and much shorter false positive “immunity” ranges. VI. REFERENCES
[12]
[13]
[14]
[15]
[16]
[17] [18]
[19]
[20]
[21] [22]
[23] [24]
[1]
[2]
Javid J. Huseynov, Zvi Boger, Gary Shubinsky, and Shankar Baliga. “Optical Flame Detection Using Large-Scale Artificial Neural Networks”, Proceedings of the International Joint Conference in Neural Networks (IJCNN’05), Montreal, Canada, August 2005 W. W. Jones. “Development of a Multi-Criteria Algorithm for Fast and Reliable Fire Detection”, Proceedings of the Thirteenth International Conference on Automatic Fire Detection (AUBE’04), pp. 184-195, 2004
Factory Mutual (FM) Global. “Radiant Energy-Sensing Fire Detectors for Automatic Fire Alarm Signaling”, Available: http://www.fmglobal.com European Standard EN 54-10:2002. “Fire Detection and Fire Alarm Systems”, Part 10, European Committee for Standardization, 2002, Available: http://www.cenorm.be Underwriters Laboratories Canada (ULC) Standard ULC/ORDC386, “Flame Detectors”, Available: http://www.ulc.ca Shankar Baliga, Herb Rabe, Brett Bleacher. “Digital MultiFrequency Flame Detector,” U.S. Patent No. 6, 150, 659, November 21, 2000 Fred Schuler. “Dual Wavelength Fire Detection Method and Apparatus,” U.S. Patent No. 5, 850, 182, December 15, 1998 Ephraim Goldenberg, Tal Olami, Jacob Arian. “Method For Detecting A Fire Condition”, U.S. Patent No. 5, 373, 159, December 13, 1994. Marc Pierre Thuillard, “Method for Analyzing the Signals of a Danger Alarm System and Danger Alarm System for Implementing Said Method”, U.S. Patent No. 6,011,464, January 4, 2000. Y. Okayama. “Approach to Detection of Fire in Their Very Early Stage by Odor Sensors and Neural Net,” Proceedings of Third International Symposium of Fire Safety Science, pp. 955-964, 1991. Y. Okayama. “A Primitive Study of a Fire Detection Method Controlled by Artificial Neural Net”, Fire Safety Journal, pp. 535553, 17, 1991 J.A. Milke and T.J. McAvoy. “Analysis of Signature Patterns for Discriminating Fire Detection with Multiple Sensors,” Fire Technology, Second Quarter 1995. J. A. Milke, “An Application of Neural Networks for Discriminating Fire Detectors”, Proceedings of the International Conference on Automatic Fire Detection (AUBE’95), pp. 213-222, 1995 H. Ishii, T. Ono, Y. Yamauchi and S. Ohtani. “Fire Detection System by Multi-Layered Neural Network with Delay Circuit,” Proceedings of the Fourth International Symposium of Fire Safety Science, pp. 761-772, 1993 Yonggang Chen, Michael A. Serio and Sandeep Sathyamoorthy. “Development of a Fire Detection System Using FT-IR Spectroscopy and Artificial Neural Networks,” Proceedings of the Sixth International Symposium of the International Association for Fire Safety Science, pp. 791-802, 1999 Javid J. Huseynov, Shankar Baliga, Gary Shubinsky, Zvi Boger. “Flame Detection System,” U.S. Patent No. 7,202,794, April 10, 2007 NeuralWorks Professional II/PLUS. NeuralWare Inc., Carnegie, PA, Available: http://www.neuralware.com E. M. Johansson, F. U. Dowla, D. M. Goodman. “Backpropagation learning for multi-layer feed-forward neural networks using the conjugate gradient method,” International Journal of Neural Systems, 2(4), pp. 291-301, 1992 Hugo Guterman. “Application of Principal Component Analysis to the design of neural networks,” Neural, Parallel and Scientific Computations, vol. 2, pp. 43-54, 1994 Tomasz P. Zielinski. “Joint time-frequency resolution of signal analysis using Gabor transform.” In: IEEE Transactions on Instrumentation and Measurement, Volume 50, No.5, October 2001 David Swanson. Signal Processing For Intelligent Sensor Systems, Marcel Dekker, Inc., 2000 James W. Cooley and John W. Tukey, “An Algorithm for The Machine Calculation of Complex Fourier Series”, Math. Comput., 19, pp. 297 – 301, 1965 X3301 Multi-Spectrum Infrared Flame Detector, Det-tronics Inc., Available at: http://www.detronics.com SharpEye 20/20I Triple IR Flame Detector, Spectrex Inc., Available at: http://www.spectrex-inc.com
