Design, development, testing and deployment of ... - Science Direct

Available online at www.sciencedirect.com

Procedia Engineering 10 (2011) 2882–2885

ICM11

Design, development, testing and deployment of the world’s first fully automated individual body armor x-ray inspection system Lawrence J. D’Aries, M.S., CEng, MInstNDT Quality Engineering and System Assurance Directorate (QE&SA), Radiographic Laboratory, US Army Armament Research, Development and Engineering Center (ARDEC), Picatinny Arsenal, NJ 07078-5000 USA

Abstract With the onset of conflict and great troop deployments into South West Asia in 2003 during Operation Iraqi Freedom, the Radiography Laboratory at the US Army ARDEC was approached to investigate the feasibility of designing an automated inspection system to examine the integrity of all the Small Arms Protective Insert (SAPI) ceramic armor plates currently in service. This amounted to on the order of 1 million units so the need for a high-throughput, fully automated system was apparent; this included automated defect recognition (ADR) software and automated material handling (AMH). Among the most challenging imaging artifacts is the fine local variation that arises as a combination of imager noise, photon noise and as a result of the arrangement of the fibers themselves in the plate backing and cover material. In many of these cases, the crack indications are actually smaller than the level of interference imposed by the fine local variations. © 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and peer-review under responsibility of ICM11

Keywords: Non-Destructive Inspection; X-ray; Ceramics; Algorithms __________________________________________________________________________________________________________

1.Introduction With the onset of conflict and great troop deployments into South West Asia in 2003 during Operation Iraqi Freedom, the Radiography Laboratory at the US Army ARDEC was approached to investigate the feasibility of designing an automated inspection system to examine the integrity of all the Small Arms Protective Insert and Enhanced Small Arms Protective Insert (SAPI/ESAPI) ceramic armor plates currently in service. Since this amounted to on the order of 1 million units the need for a high-throughput, fully automated system was apparent; this included the need for automated defect recognition (ADR) software and automated material handling (AMH). It was determined that a compact, pulsed X-ray source coupled with a high efficiency solid state detector panel would provide the required throughput of on the order of 15 sec per plate.

1877-7058 © 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and peer-review under responsibility of ICM11 doi:10.1016/j.proeng.2011.04.478

Lawrence J. D et al. / Procedia Engineering 10 (2011) 2882–2885

2883

2.Conception As with any fully automated NDE system as this the software is the greatest challenge. This software would be required to determine the health of these ceramic plates and with a very high degree of confidence. In addition, there is great variation in the design of these plates from various manufacturers as they are designed to a performance specification and not a design specification. 3.Technical development From the very beginning of the SAPI development program the requirement released to prospective developers/designers/contractors was one of performance specification. The proposed design of a ceramic plate was only specified by a physical size (weight and dimensions) and by its ability to defeat a ballistic threat. For example, the current ESAPI design specifies the ability to stop a 7.62 x 54mm (rimmed) steel core (armor piercing) round at a specified distance and a specified number of repeat hits. A size and weight specification is also provided. At that point the developers/designers come up with unique configurations. Externally there is not much deviation between and among plate designs. Internally, however, there are great differences. Some plate designs even employ metal “arrestor” areas which have a specific function in aiding the plate to defeat a ballistic threat. 4.Software Because of this performance specification there are naturally many varying designs for these ceramic plates. As such, the quality inspection of so many varying designs makes automated serial inspection a great challenge. When doing automated serial inspection the one thing always sought for such an application is uniformity from one item to the next. When this uniformity is just not present then software must take into consideration “all the possibilities”. Hence, for example, if there are 18 different manufactured designs in existence then the software must be expecting such and must recognize which design it is inspecting and have a plate design library available for analysis. It also follows that if such design is not recognized because it is absent from the design library then a valid inspection cannot take place at that time. Among the most challenging imaging artifacts is the fine local variation that arises as a combination of imager noise, photon noise and as a result of the arrangement of the fibers themselves in the plate backing and cover material. In many of these cases, the crack indications are actually smaller than the level of interference imposed by the fine local variations; this necessitated the use of novel techniques that would have to be employed into this software to accomplish this; certainly a transform analysis would be needed to be performed and applied. The Hough transform which is a feature extraction technique used in image analysis and machine vision applications [1] was thus employed. The following is a list of steps that are employed in the software as it processes the image from a ceramic plate:  Remove artifacts of measurement  Remove expected structure related to design characteristics  Minimize the effects of measurement noise  Identify and describe areas having related pixels  Remove noise areas  Remove any areas reflecting expected structure  Test remaining areas against pass/fail criteria 5.Hardware Basic system architecture consists of a Golden Engineering XP200 pulsed x-ray source operating at 150kV. Pulse rate is 25 pulses per second nominal and pulse length is about 60 nsec with 6 pulses required for an inspection. Detector is a Varian 4030E progressive scan 14-bit 3.94lp/mm resolution amorphous silicon detector panel.

2884

Lawrence J. D et al. / Procedia Engineering 10 (2011) 2882–2885

6.Performance testing and verification A prototype Armor Inspection System (AIS) was delivered and field testing began in 2006. Thousands of SAPI plates were run through the system and with great success. Testing took place at three undisclosed Army locations in the continental United States (CONUS). First deployment took place in 2008 to an OCONUS destination. New SAPI plate designs (ESAPI, XSAPI, etc.) have since been fielded or are on the horizon and will soon be fielded. As such it is critical that these new designs are addressed by the inspection system. Thus, the need for constant upgrading in capability is an ongoing effort. Performance testing and verification have been a constant aspect of this program from the beginning. Once there were many tens to hundreds of thousands of ceramic plates having been inspected by the system at Ali Al Salem Air Base in Kuwait, a large enough data set was available for a major analysis effort. Statistical inference analysis of this data set indicated a major sample size was required to achieve a high confidence result. As such, a data set on the order of a few thousand points was selected and a test was performed where three radiographers would review the pass/fail decision made by the AIS. Results of this exercise proved to be very promising. In the first operational assessment (January, 2007) there was one radiographer doing the blind manual review. The second assessment was done in August of 2007 and there were three radiographers performing the evaluation. If there was not a unanimous decision among the three radiographers then a meeting was held and a consensus reached. Once this agreement was reached then the AIS was judged against the radiographers which were considered the ground truth here. Thus, the ground truth was either accepted or rejected; these results were considered the gold standard and were what was used to evaluate the performance of the AIS system. Table 1. Employed condition codes defined

_________________________________________________________________ Condition code

Actual

Truth

Result

_________________________________________________________________ True Reject: False Reject (Beta/II): True Accept: False Accept (Alpha/I):

Cracked Plate Uncracked Plate Uncracked Plate Cracked Plate

Correctly Incorrectly Correctly Incorrectly

Identified as Identified as Identified as Identified as

Cracked Cracked Uncracked Uncracked

_________________________________________________________________ The first operational assessment consisted of 2166 plates which included both Small Arms Protective Insert (SAPI) and Enhanced Small Arms Protective Insert (ESAPI) plates. The most critical result is the false accept rate (this is the case where the AIS accepts a plate but the radiographers reject that plate). This is also referred to as Alpha or a Type I error. A situation where the AIS rejects a plate but the radiographers accept it is Beta or a Type II error. Both Alpha and Beta should be as low as possible but a Type I error can lead to an injury or death because we could have a situation where a soldier is wearing a damaged plate, whereas a Type II error would be disposing of an undamaged plate which at worst will only lead to higher costs. Alpha is thus a more important assessment figure than Beta. After the first assessment Alpha was found to be 2.5% and Beta was 4.1%. Specificity = True Accept / True Accept + False Reject (ȕ 

(1)

6HQVLWLYLW\ 7UXH5HMHFW7UXH5HMHFW)DOVH$FFHSWĮ 

(2)

Accuracy = Specificity + Sensitivity / 2

(3)

Thus, the specificity (1) of the AIS system was 95.9% and the sensitivity (2) was 97.5%. It can be shown that the overall accuracy (3) of the AIS system in this assessment is thus 96.7%. The second operational assessment consisted of 2661 plates with a resultant sensitivity of 94.4%, specificity of 95.3% and thus accuracy of 94.9%. This result is not quite as good as the first assessment but this may be in part due to the second assessment having a much smaller number (108 vs. 316) of cracked plates. It should also be noted that there were only 6 plates that were found to be false accepts and 5 of these 6 had density issues which the

Lawrence J. D et al. / Procedia Engineering 10 (2011) 2882–2885

2885

radiographers chose to reject even though this damage did not appear to be crack like. Thus, only 1 false accept is a clear cut miss for the AIS system. This single missed cracked plate out of 108 actually cracked plates would lead to an Alpha of less than 1% and thus a sensitivity of over 99%. Now, Alpha and Beta are related to each other inversely…when you take measures to decrease Alpha you naturally wind up increasing Beta….by increasing the system ability to find cracks you are more likely to find cracks that do not actually exist so you wind up rejecting more uncracked plates. The goal is thus to try to decrease BOTH Alpha and Beta at the same time. This is achievable but must be done by more than simply just changing inspection parameters to result in a change in Alpha, and thus inversely in Beta. What was required here was a paradigm shift, some new way of thinking through this situation where an optimization of recognizing plate characteristics such as crack arrestors, plate covering materials and adhesive application processes could be implemented. With application of clever machine vision techniques, some of them novel, the researchers were able to greatly improve the AIS performance since the two operational tests were done in 2006. Through improvements that have now spanned a few generations of software releases the AIS accuracy has only improved. In fact, current Alpha is about 0.3% and Beta is about 1.6%. This translates to a sensitivity of 99.7%, a specificity of 98.4% and an overall AIS system accuracy of 99.0%. These current results show that the AIS can offer greater accuracy than any radiographers manually reading images and making decisions, and it can do it much faster. It can be expected that, at most, 0.3% (or 1 in 333) actual cracked plates will be accepted by the AIS. In addition to the US Army efforts the other services are also in various stages of spinning up inspection programs for their body armor. The US Navy has been inspecting plates stateside for a number of years now. The US Air Force and the US Marine Corps are both transitioning from manual inspection as it was done in the past years to an automated inspection protocol. 7.Summary A successful body armor inspection program ensures that armor worn by troops is both functional and protective. Individual body armor is critical to the protection of troops in harm’s way that risk everything serving their country on a daily basis. References [1] Jain A. Fundamentals of Digital Image Processing, New York: Prentice-Hall; 1989, Chapter 9