EXPERIENCES AND TESTING OF AN AUTONOMOUS ON-LINE OIL QUALITY MONITOR FOR DIESEL ENGINES Carl Byington, Ryan Brewer, Vijit Nair, and Adam Mott Impact Technologies, LLC 200 Canal View Blvd. Rochester, NY 14623 (585) 424-1990
[email protected] Abstract: The paper summarizes the author’s application and testing of an oil quality sensor for diesel engine applications. Maintaining healthy fluid systems is critical to keeping machinery in a high readiness state. The authors describe the oil sensing principles and recent experiences proving the sensor’s ability to autonomously assess oil quality and classify specific diesel oil contaminants. The sensor includes both analog and digital electronics enabling the sensor to perform fluid interrogations, operate contaminant classifier algorithms, trend specific estimated contaminants, implement higher-level communications protocols and ultimately enable prognostics of future oil quality or contaminant level. The sensor design enables direct insertion into a drain plug or fluid circulation line, as done in the current testing effort. The authors provide an overview of the experimental approach, pressure-fed lubrication system, seeded fault contamination plan, and test results. The main result of the 6-month test program was demonstrated classification of multiple contaminants below the condemning limit of each.
Key Words: Lubricant Contamination, Lubricant Analytical Techniques, Condition Monitoring, Internal
Combustion Engine and Oils, Equipment Monitoring.
1
Introduction: Condition-based Maintenance (CBM), an emerging maintenance philosophy, employs active monitoring to determine the health of a component or system and enables maintenance based upon the diagnosis and predicted remaining useful life [1]. CBM provides the potential for reduced life cycle maintenance costs, improved safety, and increased operational readiness. Because contamination and in-service degradation of hydraulic fluids, gear oils, lubricants, and other in-service fluids are among the most common causes of machinery failure, lubricant health monitoring is an ideal addition to a CBM system. Traditionally, lubricant condition monitoring is accomplished through periodic sampling and analysis via a laboratory and/or field test kits. However, these types of analyses are time consuming, costly, error prone, and have long lag-times. Likely error sources include sample port location, container cross-contamination, test methodology accuracy/repeatability/interference, and man-in-the-loop issues. An automated, in-situ oil quality monitoring system addresses these concerns, providing continuous, real-time lubricant degradation information allowing maintainers to prevent unnecessary system wear, optimize maintenance intervals, and address equipment problems early. Ultimately, the technology provides further benefits through reduced equipment downtime, and lower operational costs. Several methods exist for real-time condition monitoring of lubricants; most of these methods target one of three main categories: quality, debris, or elemental techniques [2]. Several technologies employ inductive transducer elements to detect particle contamination [3]. However, this technology shows limited promise for particle sizes below 500µm, which is insufficient considering that particles between 5 and 20 microns cause 60% of all engine wear [4]. This method is also insensitive to contaminations such as fuel and water, which are prevalent in diesel systems and severely degrade lubricant performance. Sensing techniques that measure the conductivity of engine oil [5] are limited due to the large number of failure modes of oil and only a single parameter with which to infer oil quality. Other monitoring techniques have been developed that discriminate oil aging mechanisms due to specific contaminations [6], but are incapable of detecting other primary failure modes. Some systems use
2
multiple complex sensors and require significant modifications to existing oil circulation and diagnostic/control systems [7] and, therefore, hamper direct integration into a commercial application. The Smart Oil SensorTM (SOS) described in this paper provides a real-time analysis of in-service fluids that is online, in-situ, real-time, and inexpensive. The goal of this paper is to demonstrate that the SOS technology overcomes the aforementioned drawbacks inherent in other oil analysis techniques by implementing a novel, broadband impedance spectroscopy based approach. The paper also presents the results of extensive testing. Basis of Measurement: The Smart Oil SensorTM system employs a patent-pending, low-powered, broadband impedance measurement coupled with multi-sensor fusion and a model-based analysis package designed to be capable of predicting fluid quality and degradation for a range of fluid systems. The electrochemical impedance spectroscopy (EIS) approach involves injecting a complex alternating current signal into a system over a wide frequency spectrum and measuring the system’s response to determine oil quality. The impedance of the system is determined by analyzing the differences between the injection (excitation) and response signals. By scanning across a wide-range of frequencies, the sensor obtains a measurement that is rich with information and better reflects the actual impedance of the oil. Kozlowski et al. [4] used a similar method to model the electrochemical impedance of a cell for predicting the state-of-charge and state-of-health in batteries. By injecting a broadband signal, rather than a single tone, the SOS can complete a fluid interrogation in less than 30s. By comparison, a traditional, tone-at-a-time EIS measurement could take upwards of 50 minutes to complete [8], which is unacceptable for online use due to process variable changes that can occur within that time. The presence of detrimental contaminants and the degradation of the oil base stock and additive package cause oil quality changes. These changes affect the dielectric properties, conductivity, bulk resistance, capacitance, and other key properties of the oil. The SOS uses electrochemical models to represent the 3
impedance response of the lubricant, and model-based parameter estimation methods to identify the model elements. Machine learning techniques link the changes in these parameters to oil quality changes, such as a change in a particular type of contamination. Figure 1 shows the basic electrochemical impedance measurement circuit and typical impedance response of a fluid under changing contaminant conditions. Smart Oil SensorTM Design: The SOS (Figure 2) is a stand-alone unit that includes sensing element, signal conditioning, data processing, and communications elements in an integrated package. The EIS measurement element is comprised of two concentric cylinders of uniform separation. The authors selected the sensor head geometry to maximize sample surface area while minimizing impedance to fluid flow. The SOS houses signal conditioning and data processing electronics capable of functions such as dynamically reconfigurable gain and filter selection, interrogation signal generation, high-speed data acquisition, data analysis, and off-board communications.
An IP57 rated housing protects the
sensor electronics. The current sensor design supports both RS-232 and Controller Area Network (CAN) communications. Through these interfaces, the sensor can communicate fluid measurements and sensor status or receive configuration and firmware updates. Including CAN protocol support simplifies integration into existing diagnostic and control systems. Feature Extraction and Classification: After performing an EIS measurement, the sensor executes a series of algorithms to extract fluid quality information from the gathered impedance data. This process includes feature extraction and classification processes. Feature extraction is the method used to estimate electrochemical impedance model parameters, such as bulk-resistivity and interfacial properties. The resulting features provide the link between an impedance measurement and actual fluid property changes. The sensor’s embedded processor performs feature 4
extraction with a linear least square fitting algorithm. This tried-and-true method allows for easy implementation on an embedded platform without sacrificing significant performance or accuracy. The authors leveraged their experiences in multi-sensor data fusion, classification, and data mining [1], [9], [10], [12] to build a classifier that best meets the application requirements for a real-time, in-situ, stand-alone sensor. The authors experimented with several classifier architectures based on linear discriminant analysis, Bayesian probabilistic models, robust fault detection and isolation [11], and neural networks. To evaluate the classifier methodologies, the authors considered compatibility with the data set, required training sample size, classification accuracy, and execution time. The Bayesian probabilistic classifier best suited the requirements for this particular application, achieving a high accuracy with reasonable efficacy. Bayesian classifiers apply feature independence assumptions to underlying probability distributions. They model the features using a multivariate, normal probability density function to compute the class posteriori probability for each feature vector. Hence, given a feature vector, X = x , the probability that it belongs to a class, C = i , is given by Bayes theorem (Note: Upper case letters denote variables while lower case letters denotes the current value, or observation, of these variables): P(C = i | X = x ) =
P( X = x | C = i )P(C = i ) P( X = x )
(1)
The class that yields the maximum a-posteriori probability (MAP) determines classification. A Bayes classifier uses the class posterior probabilities given by the feature vector as discriminant. A Naïve Bayes classifier makes the simplifying assumption that the features are independent, given the class [13]. Hence, the classifier uses the discriminant function:
5
f ( x ) = ∏ P (X j = x j | C = i ) P (C = i )
(2)
j
The classifier architecture selected for the identification of fuel, water, and soot within diesel oil consist of three tiers that utilize real-time data from the sensors and historical classifications. The first and second tier Bayesian classifiers are connected through a cross-coupled architecture. These cross-coupled classifiers oscillate through the search space as they hunt for a mutually acceptable classification. Such a simple architecture decreases the complexity of the system design while increasing the stability and accuracy of the classification. The Bayesian knowledge fusion system [9] reinforces the belief in the health assessment as updated by the current statistical probability of failure. The third tier updates the classification by fusing predictions from the historical trending information. Hence, the system will not overreact to contamination spikes and anomalous measurements thereby preventing rapid switching between estimated contaminant types and levels. Methods of Determining Classifier Accuracy: Classifier performance is often interpreted using a
confusion matrix. A confusion matrix contains information about the actual (estimated) and predicted classifications generated by a classification system. The confusion matrix is a stochastic, n × n sized matrix of conditional probabilities, where each of its elements ( pij ) defines its probability of predicting a class i given an example of an actual class j [14]. Hence, the accuracy of the classifier is given by the equation:
∑x A= ∑∑ x ii
= ∑ pii
i
i
ij
(3)
i
j
where xij is the number of instances that the actual class i was predicted as class j . Such an estimate does not take into account the error severity, which is particularly important in this application. Instances where the sensor significantly misclassifies the contaminant level or estimates the wrong
6
contaminant are considered severe errors. To account for error severity, the authors devised a costsensitive accuracy estimate that weighs each misclassification with a weighted coefficient ( wij ) that reflects the distance of the misclassification from the actual class. The authors also used the Kappa coefficient as a metric to evaluate the classifier’s performance. The kappa corrects the degree of agreement between the classifier’s predictions and reality by considering the proportion of predictions that might occur by chance. In a multi-class classifier, Eq. (4) gives the Fleiss’ kappa [15].
K=
N ∑ xii − ∑ xi. x.i i
i
(4)
N − ∑ xi. x.i 2
i
where N is the total number of instances and xi. and x.i are the column and row counts, respectively, of the confusion matrix.
Experimental Evaluation: The authors designed a lubrication test stand that recreates typical in-line flow environmental conditions to test the SOS’s ability to take accurate readings of contaminated oil in a dynamic environment. The test stand, shown in Figure 3, replicates real-world pressure, temperature, and flow scenarios, and provides a means to perform seeded fault contamination testing. The SOS was tested inline on the test stand to detect and track water contamination, fuel dilution, and soot contamination in MIL-PRF 9000H lubricant. All testing was conducted at a flow rate of 1.1 m/s (3.5 ft/s) past the sensor head at 172kPa (25 psi) and 51.6°C (125°F). In the first phase of testing, the authors contaminated the system in a monotonically increasing manner up to the condemning limit of each type of contaminant. The test sponsor defined condemning limits as 1% soot by mass, 5% fuel by volume, and 2000 ppm (0.2%) water by volume. Contamination sweeps, rather than random variations of contaminant levels, were used to develop progression trends while 7
minimizing the amount of oil used during experimentation. Slow addition of each contaminant to the test stand over a 2-3 hour period allowed for uniform distribution of the contaminant within the test stand. Figure 4 shows a cube plot that illustrates the Design of Experiments (DoE) for the contamination test plan. As shown in the figure, six separate test sequences were performed in which the order of contaminant addition was varied. This approach fostered the development of a knowledge base that could account for not only the linear effects of isolated contaminants but also the nonlinear effects caused by interactions between multiple contaminants and the interactions between contaminants and oil additives. An additional phase of testing filled the gaps in the knowledge base to improve classifier accuracy in previously untested regions. By collecting data in several regions in the middle of the search space (multiple contaminants, below their condemning limits) the classifier can more easily identify previously unseen classes.
Determination of Optimum Resolution: The authors analyzed the data collected from the aforementioned test plan to verify the performance of the classifier and determine the optimal classification resolution. The resolution of the classifier determines the maximum change in health state that the classifier can detect. A classifier with a high resolution can detect fine changes in the health state and is most beneficial to the user. However, classification accuracy is inversely proportional to classifier resolution. Thus, there is an essential tradeoff between resolution and accuracy. To determine the optimum resolution, the authors divided the contamination range into 10 levels of resolution (N) and evaluated the performance of the classifier on several data sets for these predetermined levels. Figure 5 shows the decrease in the classifier accuracy resulting from an increase in resolution. While classifier performance is very high (above 90% accuracy), for three classes per contaminant, the lack of resolution provides a user little prognostic capability. With only ‘healthy’,
8
‘warning’, and ‘critical’ states, a user cannot predict when contamination levels will reach a condemning limit. Alternatively, with 8-10 classes per contaminant there is a significant decrease in classifier accuracy, which leaves a user second-guessing the sensor outputs. For these reasons, the authors selected five classes per contaminant for the diesel lubricant multi-contaminant classifier.
Evaluation of Classifier Accuracy: The authors evaluated the performance of the three-tiered Bayesian classifier using data collected on the scaled lube system test stand. The authors evaluated the classifiers using a cross validation approach, which trains the classifier on 70% of the data and uses the remaining 30% for testing. Figure 6 shows a confusion matrix for the classifier. In a confusion matrix, the rows indicate the actual classification of the system (based on estimated contamination) and the columns indicate the classifier’s result. For an ideal classifier, the confusion matrix should have all instances lie on the diagonal. Misclassifications appear as deviations from the diagonal. As seen by the dominance of diagonal instances in Figure 6, the classifier predicts most of the classes very accurately. For example, the SOS classifies Virgin Oil (row 1) with 95% accuracy. However, for high fuel contamination (row 10), the classifier achieved only 61% accuracy. This is largely due to a limited sensitivity to fuel contamination, which is inherent in the current design; therefore, high levels of other types of contamination can overwhelm the fuel contamination classification.
Lab-based Verification of SOS Results: The project sponsors conducted a sensor verification test in which they contaminated the test stand with known as well as unknown levels of fuel, water, and soot contaminantion, sampled oil from the test stand at regular intervals, and dispatched the samples to a laboratory for analysis. Testing ran over a 2-day period and the test stand was drained, cleaned and refilled at the start of each day. The resulting laboratory analysis reports confirmed the ability of the Smart Oil SensorTM to detect contamination and trend multiple contamination level simultaneously. The independent analysis lab performed gas chromatography according to ASTM D3524 to measure fuel dilution. Figure 7 shows that the actual fuel levels measured by the lab fall between the fuel dilution 9
bounds predicted by the sensor with a high level of accuracy and consistency. Note: Samples labeled as set ‘A’ denote day 1 of testing and set ‘B’ denote day two. The actual contamination levels that fall outside of the predicted bounds were never off by more than one class. SOS results trend very well with the laboratory results and are well within the margin of error (2% per ASTM specification) of the lab analysis method. Two independent laboratories performed water concentration analyses using a coulometric Karl-Fischer test (ASTM D6304). Figure 8 shows the actual water contaminant level detected by the labs and the bounds of the SOS classification. The plot highlights the variability that can occur between analysis labs. The SOS classifications trend well with both lab reports; however, conclusions on measurement accuracy are limited by lab inconsistency. The discrepancy between the labs is most likely due to improper correction of the measurement for zinc dialkyl dithio phosphate (ZDDP) interference. Because the SOS relies on laboratory analysis to verify classifier measurements, selection of analysis methods and laboratory are critical to the overall accuracy of the sensor. The authors also employed two laboratories to estimate the soot levels in the oil samples (one using Wilkes Soot meter and the other using FTIR analysis) and compared these to the SOS predicted results. The soot contamination levels reported by the two labs conflict not only with the SOS predictions but also with each other. Both labs actually reported significantly lower soot levels than were actually added. The trending of the reported soot levels (for known seeded fault contaminations) is in contrast to the expected progression. This highlights the susceptibility of error in lab analysis potentially due to insufficient agitation of the soot samples at the lab, non-uniform mixing and sampling of soot in the test stand, low resolution of methods like FTIR analysis and difference in Carbon Black (used for seeded fault contamination) and actual soot.
Future Work: While the sensor demonstrated the capability of identifying and trending multiple contaminants in a test stand environment, the authors are addressing several issues that will improve the 10
commercial success of the sensor. Improvements such as expanding the interrogation frequency range, improving temperature compensation algorithms, and increasing the sensor’s sensitivity to particular contaminants will enhance classifier accuracy and resolution. Further improvements such as reducing electronics size, improving temperature limitations, and decreasing sensor head size will allow the sensor implementation in a much broader array of application. The authors have already extended their initial work by employing the oil sensing capability in the applications shown in Table 1. Moving beyond diesel lubes, the sensor performs extremely well while monitoring water content and lubricant quality within gearbox systems. ‘Real-world’ application testing of the sensor in diesel and gearbox applications will be used to further verify the sensor’s capabilities and identify potential limitations.
Conclusion: In this paper, the authors describe the Smart Oil SensorTM technology that employs broadband spectroscopy approaches, electrochemical techniques, and advanced multi-sensor data fusion methods to present a near real-time, inline oil analysis device. The results from the 6 months of continuous testing and data analysis demonstrate the flexibility and robustness of the classifier and highlight the accuracy it can achieve. Testing has also identified areas of improvement that need addressing to improve classifier accuracy and resolution. The laboratory analysis of oil samples corroborates the assertion that the Smart Oil SensorTM is able to detect and track levels of oil contamination due to fuel, water, and soot. With further testing, the SOS can be adapted to a wide array of other possible contaminations and fluid types. The authors have also provided additional installation and test applications in which they have integrated the oil quality sensor for further development and validation. These evaluations, in concert with dedicated laboratory ground-truth data collection, will provide the means to evolve the technology and demonstrate its ability to track and predict oil contamination in such environments. Fundamentally, the
11
SOS technology provides a key enabler to achieve effective fluid system monitoring, thereby safely extending machine life, minimizing environmental impact, and reducing life cycle costs.
Acknowledgments: This effort is partially based upon work supported by the Office of Naval Research (ONR) under Contract No. N00014-04-C-0389. The authors would like to thank Dr. Ignacio Perez (ONR) as well as Ken Scandell, James Soisson and Vicky Larimore (NSWC) for their input and support.
References [1]
Byington, C.S. and Garga, A.K., (2001) “Data Fusion for Developing Predictive Diagnostics for Electromechanical Systems,” in Handbook of Multisensor Data Fusion, CRC Press, Boca Raton, FL, D. L. Hall and J. Linas, eds., pp 23-1 – 23-31.
[2]
Schalcosky, D.C., and Byington, C.S., (2000), “Advances in Real Time Oil Analysis,” Practicing Oil Analysis Magazine, 11, pp 28-34.
[3]
Yonghui, Y., et al., (2003), “An Integrated On-line oil Analysis Method for Condition Monitoring,” Measurement Science and Technology, 14, pp 1973-1977.
[4]
Kaufman, M., (2000), The Motor Oil Bible, http://www.motor-oil-bible.com.
[5]
Wang, S.S., et al., (1994), “The Development of In Situ Electrochemical Oil-Condition Sensors,” Sensors & Actuators: B. Chemical, 17, 3, pp 179-185.
[6]
Scherer, M., Arndt, M., et al., (2004), “Fluid Condition Monitoring Sensors for Diesel Engine Control,” IEEE Sensors, 1, pp 459-462.
[7]
Liu, Y., et al. (2000), “Research on an On-line Wear Condition Monitoring System for Marine Diesel Engine,” Tribology International, 33, 12, pp 829-835(7).
[8]
Lvovich, V. F. (2003), “Method and Apparatus for On-line Monitoring of Quality and/or Condition of Highly Resistive Fluids,” U.S. Patent No. 6,577,112.
[9]
Byington, C.S. and Kacprzynski, G., (2006), “Reasoning Methods,” PHM/CBM Design Course, Impact Technologies, LLC, Orlando, FL, November 3, 2004. 12
[10] Kozlowski, J.D., Byington, et al. (2001), “Model-based Predictive Diagnostics for Primary and Secondary Batteries,” Proc. of the 16th Annual Battery Conference on Applications and Advances, Long Beach, CA, pp 251-256. [11] Hailu, H. (2006), “A Frequency Domain Robust Model Invalidation Approach to Robust Fault Detection and Isolation with Applications to Identification of Contaminants in Lubrication and Transformer Oils,” Thesis (MS), Penn State University. [12] Smith, M., Byington, C., et al. (2006), “Layered Classification for Improved Diagnostic Isolation in Drivetrain Components,” Proc. of the 2006 IEEE Aerospace Conference, Big Sky, MT, pp 8. [13] Rish, I. (2001), “An Empirical Study of the Naïve Bayes Classifier,” Proc. of the International Joint Conferences on Artificial Intelligence, Acapulco, Mexico, pp 41-46.
[14] Provost, F., Fawcett, T., and Kohavi, R., (1998), “The Case Against Accuracy Estimation for Comparing Classifiers,” in Proc. of the 15th International Conference on Machine Learning, Madison, WI, pp 445-453. [15] Ben-David, A., (2006), “What's Wrong with Hit Ratio?” IEEE Intelligent Systems, 21, 6, pp 68-70.
13
Tables
Marine Diesel Oil Quality Sensing: The authors performed development and validation of the oil sensing technology to monitor the health of lubrication systems on shipboard diesel engines. The sensor was tested on a scaled lubrication test stand. Results with water, soot, and fuel contamination indicated that clearly discernable parameters could be derived for fluid quality classification. Gearbox Corrosion Monitoring and Prognostic System: The authors have applied the SOS to infer gearbox lubricant quality and model the progression of gearbox corrosion. The sensor, tested on a Scaled Gearbox Test Rig, could detect water contamination within the lube system (a leading cause of corrosion damage) and other oil quality changes. This allowed us to estimate the risk of gearbox component corrosion and predict remaining useful life. Commercial Mobility Applications: The authors are currently testing the SOS on a fleet of diesel-powered fuel oil delivery trucks. The sensor is installed on a long haul truck powered by a 12-liter Volvo D12D diesel engine and other test vehicles to evaluate its survivability and oil quality measurement capability. In one embodiment, oil quality information is displayed to the driver and wirelessly transmitted to web servers by a rugged tablet PC. The data can then be used as part of a maintenance and logistics database or served via the web.
Table 1: Summary of Installations and Near Term Applications for the Smart Oil SensorTM
14
Figure Captions Figure 1: Smart Oil Sensor Principle - Use of Broadband Impedance Spectroscopy Figure 2: The Smart Oil Sensor™ Figure 3: Lubrication System Test Stand Figure 4: Cube Plot Demonstrating DoE (each color indicates a different test sequence) Figure 5: Classifier Accuracy vs. Resolution Figure 6: Confusion Matrix Showing Results with Correct and Incorrect Classifications Figure 7: Fuel Dilution Level Verification using Gas Chromatography Figure 8: Water Contamination Verification using Karl Fischer Titration
15
Figures
Figure 1: Smart Oil Sensor Principle - Use of Broadband Impedance Spectroscopy
16
Figure 2: The Smart Oil Sensor™
17
Figure 3: Lubrication System Test Stand
18
Figure 4: Cube Plot Demonstrating DoE (each color indicates a different test sequence)
19
Figure 5: Classifier Accuracy vs. Resolution
20
Figure 6: Confusion Matrix Showing Results with Correct and Incorrect Classifications
21
Fuel Classification 6 SOS Classification
Lab Results
Fuel (%vol)
5
4
3
2
1
0
A1
A2
A3
A4
A5
A6
A7
B1
B2
B3
B4
B5
Samples
Figure 7: Fuel Dilution Level Verification using Gas Chromatography
22
B6
B7
Water Classification 0.4 SOS Classification
Lab 1 Results
Lab 2 Results
Water Content (%vol)
0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 A1
A2
A3
A4
A5
A6
A7
B1
B2
B3
B4
B5
Samples
Figure 8: Water Contamination Verification using Karl Fischer Titration
23
B6
B7
