Standoff, Wide-Area Explosive and Narcotic Detection Using Shortwave Infared Hyperspectral Imaging

Shortwave infrared (SWIR) hyperspectral imaging (HSI) is a useful technique for soldiers and first responders to combat the ever-changing threat landscape. SWIR HSI provides the ability to rapidly analyze a large, complex scene, and detect small constituents located within. Two key threat scenarios that are currently being addressed by SWIR HSI are the detection of explosive or chemical warfare agent threats before potential harm can occur, and to intercept narcotics entering prisons and jails. Tunable filter–based SWIR HSI sensors can be configured to use either single-bandpass or multiple-bandpass wavelength tuning. These filter types enable hazardous material detection in stationary, moving sensor, or moving target scenarios, while providing near real-time results. Tunable-filter HSI also provides users the ability to add new materials to a target library, allowing the sensors to adapt to changing threats or applications. This article provides an overview of SWIR HSI using liquid crystal tunable filters and provides detection results for key threat applications.

Explosive and narcotic materials are a potential danger to human life. The risk of improvised explosive devices and the rise in deaths related to narcotic materials have created a need for improved sensing capabilities that can target these materials at safe distances, before the threat can be introduced to the general population (1,2). In addition, the threats themselves can change over time. These changes include variations in material type, composition, or method of introduction. With a dynamic threat landscape, a sensing approach must be adaptable to interdict threats before they cause harm.

Unfortunately, the current generation of spectroscopic detection approaches has limitations. Commonly, these include close-to-target proximal requirements, reduced area search rate, eye safety concerns from focused excitation (such as Raman spectroscopy), and long analysis times that have hindered the ability of spectroscopic instrumentation to fully combat this problem and be adopted by first responders (3–6).

One approach that has the potential to address these problems is shortwave infrared (SWIR) hyperspectral imaging (HSI) (7–10). SWIR HSI is beneficial for detecting small constituents located within complex scenes, where the target concentration is commonly low. SWIR HSI typically uses the wavelength range 1000–1700 nm and incorporates indium gallium arsenide (InGaAs) focal plane arrays (FPAs) as detectors. InGaAs FPAs can be uncooled, which reduces system size, weight, and power constraints, and makes a SWIR HSI sensor more portable.

SWIR region absorption bands are derived from overtones and combination bands of O-H, N-H, C-H, and S-H bend and stretch vibrations. C-H bond vibrations are the most prevalent in explosive and narcotic materials, with each containing numerous bonds typically found within -CH₂ or -CH₃ functional groups. These bands are typically broad, leading to complex spectra for which it can be difficult to assign specific chemical components to specific spectral features. The use of multivariate analysis techniques may be used to extract this information.

In addition, the SWIR region is insensitive to target area temperature interference (such as heat emitted from a target does not interfere with analysis), which can be a problem for midwave-length infrared (MWIR) and long-wavelength infrared (LWIR) sensors. The response in the region is only to photons that are absorbed or reflected by a target. SWIR sensors are capable of utilizing solar illumination for outdoor target excitation. For indoor applications, low-cost broadband illumination or SWIR light-emitting diodes (LEDs) may be integrated onto a sensing platform.

Although there are many configurations for HSI instrumentation (such as pushbroom and snapshot [11]), full-frame imaging using a tunable filter has certain advantages that can address current sensing shortcomings, such as higher imaging fidelity and target library expansion. One filtering approach utilizes liquid crystal tunable filters (LCTFs), where a specific wavelength or passband is output through the filter. An LCTF is composed of multiple stages (up to four) where the output of each stage is voltage modulated so the overall bandpass of the filter is the desired wavelength or passband.

There are two filtering modes for LCTFs: discrete band filtering and conformal filtering (CF). Discrete band tuning is when the filter is set to pass a single wavelength. Switching between wavelengths occurs at speeds under 100 ms and a full hypercube (a sequential scan of wavelength specific images) may be collected in typically under 1 min, depending on the filter tuning increment. Each hypercube contains a fully resolved, unique spectrum for each pixel on the FPA, which provides this approach the capability to detect particles on a single-pixel level. After the spectral information about the target is known, a condensed subset of imagery can be collected at wavelengths related to the absorption bands of the target to reduce the sensor's time to detect.

Figure 1 shows the concept of SWIR HSI using a discrete-band LCTF. In the figure, a vehicle with a threat deposition (handprint) on the door is illuminated with solar irradiance. The light reflected off the vehicle is collected by the sensor and images can be taken at each wavelength in the SWIR region, allowing measurements of absorbance for all materials in the field of view. Each material in the field of view (FOV) is spatially separated on the FPA, which provides the ability to resolve small constituents in an expansive scene.

Figure 1: Overview of SWIR HSI using a liquid crystal tunable filter (LCTF) with broadband irradiance.

The second filtering approach, CF, is designed to pass multiple optical passbands, not a single discrete wavelength, where the desired passband is optimized to discriminate a desired target or target class from the background. The passband is calculated based on the concepts of compressive sensing and multivariate optical computing (MOC) (12–14), where a regression vector is coded into the transmission of the filter. The regression vector is aligned with the absorption properties of the target material, which improves target discrimination in the presence of a complex background. CF can also be used for real-time motion-sensitive applications, like drone detection or people screening, since it does not require sequential wavelength tuning.

Figure 2 shows the concept of conformal filtering and how CF can identify two homemade explosive precursors against complex background materials and a near-neighbor confusant. The target board scene contains a mixture of target materials (homemade explosive [HME] 1 and 2), substrates (white and red car coupons, paper, and drywall), environmental clutter (sand and vegetation), and confusant materials (near neighbor HME confusant, drug 1 and 2, and chemical warfare agent simulant 1 and 2). The score image shows the response of the filter, with brighter pixels having a higher score value and being more likely to be considered a detection. The spectral traces show the SWIR absorption spectra for the two HMEs and the near neighbor confusant material. The final image is the detection image, with materials overlaid in green being considered a detection, which indicates that the two HMEs are being detected while the near neighbor confusant and all other scene constituents are not.

Figure 2: Overview of conformal filter breadboard testing, with the sensor trained to identify two HMEs amongst a complex target set, including a near neighbor confusant.

Experimental

ChemImage LCTF-based SWIR HSI sensors have been constructed in two application configurations: benchtop and standoff. The benchtop configuration has been designed to screen articles of mail in either transmittance or reflectance modes. The sensor is mounted onto a tabletop assembly with transmittance and reflectance oriented halogen lights for target illumination. A piece of mail is placed on a glass plate, above the transmittance lighting, and the sensor is positioned above the target facing toward the glass. The field of view of the sensor is set to image the full glass plate area (a rectangular area approximately 8.5 in. x 11 in.).

The discrete band LCTF standoff system was designed for indoor and outdoor use, for applications such as vehicle screening or room clearing. The sensor is mounted on an FLIR PTU/tripod, and uses eight on-sensor 50-W halogen bulbs, 4x 100-W halogen flood lamps, and solar illumination depending on use scenario and standoff distance. In addition, a conformal LCTF breadboard was used for moving-target applications.

For each configuration, the collection optics pass light through the LCTF and onto the FPA (FLIR Tau 640x512 InGaAs, uncooled). The main optical variation between the two configurations is that the standoff configuration uses an adjustable lens array to manually adjust the focal range for standoff distances between 1 and 20 m. The benchtop configuration uses two fixed lens settings, which allows a selection between high or low zoom, with a sensor to target distances less than 0.5 m. Each sensor applies the same image corrections to ensure maximum performance: background (dark) subtraction, bad pixel correction, and flatfielding.

Imagery analysis and detection results are generated using a real-time SWIR automated detection algorithm (SWIR-ADA). SWIR-ADA provides autonomous analysis of imagery, utilizing the spectral responses of threat materials in comparison to environmental clutter to establish target selection criteria. The result of a SWIR-ADA scene analysis is an image of the scene with false color overlays in areas where a threat material is located. To achieve at or near real-time results, detection mode uses a spectral library of known threat materials, identifies a subset of wavelengths that isolate the dominate absorption bands for each material or class of material, and then collects imagery for that subset of wavelengths.

Results and Discussion

Using the benchtop system to scan articles of mail for narcotics or threat materials is an application of high interest to correctional institutes. Some narcotics of interest include heroin and methamphetamine as well as Suboxone, a prescribed medication that combats opioid addiction. Suboxone is commonly found as an orange sublingual strip and can be easily placed into letters and covered with markers or crayons, or hidden in glue seams in envelopes, masking the drug to the naked eye. These tainted letters are then potentially mailed into the prison populace and consumed.

Figure 3 (left) shows an example of Suboxone (buprenorphine and naloxone) detection in a concealed sample. A strip of Suboxone (approximate mass of 10 mg) was halved and placed into the seam of an orange envelope and then resealed. The tainted envelope was then placed onto the sensor and scanned for approximately 7 s. During this scan time, the sensor tuned to and acquired images at wavelengths that corresponded to specific absorbing and nonabsorbing bands of Suboxone. The resulting image shows a color overlay on the envelope seam where the Suboxone strip was located. Figure 3 (right) shows a sample where Suboxone had been mixed with an orange crayon, which was used to color two areas on a page. The detection image shows a color overlay on the two spots where the Suboxone was deposited.

Figure 3: Detection of Suboxone concealed in an envelope (left) and in a crayon drawing (right) using the benchtop SWIR HSI sensor.

The standoff system is designed for an expanded array of applications, with vehicle residue screening and room clearing being two primary focuses. For testing purposes, materials of interest are used in these scenarios, and their presence is indicated by a specific color overlay: ammonium nitrate (red), urea (orange), and aspirin (green). Ammonium nitrate and urea are homemade explosive precursors, and aspirin is used as a narcotic simulant.

Figure 4 shows four detection scenarios for standoff SWIR HSI. Each scene contains target material, confusants (NaCl), and realistic environmental clutter: vehicle trunk clearing, residue detection on steering wheels, clandestine lab investigation, and room clearing. In Figure 4a, materials are presented as bulk spills in the trunk of a car. Figure 4b shows residue transfers onto a steering wheel. Figure 4c shows a simulated clandestine lab, where unknown bulk white powders are found on a table and fingerprint residue transfers are on the backpack. Figure 4d shows room clearing, where bulk spills are on the floor. For each example, the true positives (TP) and true negatives (TN) are labeled.

Figure 4: Detection images from four application scenarios using the standoff SWIR HSI sensor: (a) vehicle trunk clearing, (b) residue detection on steering wheels, (c) clandestine lab investigation, and (d) room clearing.

In Figure 5, the conformal breadboard was used for real-time moving target detection of a drone carrying a bulk amount of urea. Conducting real-time measurements is possible because of its dual polarization optical design, where the collected light is split by a polarized beamsplitter, and sent through two conformal filters aligned to the beamsplitter's polarization output. Each filter's output was imaged onto two FPAs, and provides the ability to analyze the target area at the FPA frame rate. The drone was at variable standoff distances during the experiment, but typically was between 20 and 50 m. The prototype sensor was able to track and detect the urea while the drone was in motion.

Figure 5: Moving target detection of a drone carrying a bulk bag of urea using the conformal filter breadboard.

Conclusions

Hyperspectral imaging is a valuable tool to detect and identify threat materials, such as explosives or narcotics. In these applications, SWIR HSI has the potential to detect small constituents in scenes with a greater amount of environmental clutter. These sensors have been implemented in benchtop and standoff configurations, using an LCTF to provide high fidelity imagery with at or near real-time detection speeds.

References

(1) K.E. Creppage, J. Yohannan, K. Williams, J.M. Buchanich, T.J. Songer, S.R. Wisniewski, and A. Fabio, "The Rapid Escalation of Fentanyl in Illicit Drug Evidence in Allegheny County, Pennsylvania, 2010–2016," Public Health Reports (2018).

(2) K.D. Strang, M.E. Korstanje, and N. Vajjhala, Research, Practices, and Innovations in Global Risk and Contingency Management (IGI Global, Hershey, Pennsylvania, 2018), Chapter 17.

(3) L. Mokalled, M. Al-Husseini, K.Y. Kabalan, and A. El-Hajj, Int. J. Sci. & Eng. Research5(6), 337–348 (2014).

(4) E.L. Izake, Forensic Sci. Intl.202, 1–8 (2010).

(5) A.W. Fountain III, S.D. Christensen, R.P. Moon, J.A. Guicheteau and E.D. Emmons, Appl. Spectrosc. 68, 795–811 (2014).

(6) R.M. Wentworth, J.H. Neiss, M.P., Nelson, and P.J. Treado, Standoff Raman Hyperspectral Imaging Detection of Explosives (IEEE Antennas and Propagation Society International Symposium, 2007), pp. 4925–4928.

(7) D.E. Moon, S.K. Tazik, J.G. Wierszewski, N.R. Gomer, and M.P. Nelson, Proceedings of SPIE10657, 106570T (2018).

(8) N.R. Gomer, S.K. Tazik, C.W. Gardner, and M.P. Nelson, Proceedings of SPIE 10183, 1018303 (2017).

(9) N.R. Gomer, C.W. Gardner, and M.P. Nelson, Proceedings of SPIE9855, 98550M (2016).

(10) O. Klueva, M.P. Nelson, C.W. Gardner, and N.R. Gomer, Proceedings of SPIE 9455, 94550O (2015).

(11) T. Adã o, J. Hruška, L. Pádua, J. Bessa, E. Peres, R.l Morais, and J. João Sousa, Remote Sens.9(11), 1110 (2017).

(12) M.P. Nelson, J.F. Aust, J.A. Dobrowolski, P.G. Verly, and M.L. Myrick, Anal. Chem.70(1), 73–82 (1998).

(13) M.P. Nelson, L. Shi, L. Zbur, R.J. Priore, and P.J. Treado, Proc. SPIE9824, (2016).

(14) E. Candès and M. Wakin, IEEE Signal Processing Magazine 25(2), 21–30 (2008).

Nathaniel Gomer, Shawna Tazik, Jeffrey Beckstead, Charles Gardner, and Matthew Nelson are with ChemImage Corporation in Pittsburgh, Pennsylvania. Direct correspondence to: gomern@chemimage.com