Spectral Sensing Using a Handheld NIR Module Based on a Fully Integrated Sensor Chip

Near-infrared (NIR) spectroscopy is widely used for the classification of materials and the quantification of their properties. Today, there is a high demand for extending the use of this technique to portable applications, and eventually, the integration with consumer appliances and smartphones. To reach this goal, the overall size of the NIR sensor, its production cost, robustness, and resistance to vibrations are of particular importance. This paper describes an approach to spectral sensing in the NIR (850–1700 nm) using a handheld sensor module based on a fully integrated multipixel detector array with a footprint of around 2×2 mm². The capabilities of the spectral sensor module were recently evaluated in two application cases: Quantification of the fat percentage in raw milk and the classification of plastic types. Fat quantification was achieved with a root mean square error (RMSE) of prediction of 0.14% and classification of plastic types was achieved with a prediction accuracy on unknown samples of 100%. The results demonstrate the feasibility of the direct NIR sensing approach used by the integrated sensor, which has potential to be used in a variety of applications.

For decades, near-infrared (NIR) spectroscopy has played an important role in countless applications, ranging from monitoring industrial processes to assessing the chemical composition and quality of products. However, traditionally spectrometers are large, expensive, complex, and include moving parts, making them sensitive to vibrations and shocks. The challenge today lies in reducing the size and cost of these spectroscopic devices while maintaining their robustness and sensitivity, which is essential to expand their application beyond dedicated stations in industrial settings and analytical laboratories.

The design of current portable NIR sensor systems is mainly focused on the miniaturization of conventional spectrometers using gratings or interferometers. Although they represent valid options for portable NIR spectroscopy, the size and cost of most commercially available systems is still relatively large. The level of miniaturization, cost, and production scalability required for consumer applications can only be reached with wafer-scale integration—analogous to how complementary metal-oxide- semiconductor (CMOS) cameras came to pervade industrial and consumer applications. There has been substantial progress in this direction for the visible spectral region (400–700 nm) up to 1100 nm, utilizing mature silicon technologies (1). However, progress in the integration of NIR spectral sensors has been relatively slow.

Recently, a novel approach to NIR spectral sensing was proposed, using a miniaturized fully-integrated multipixel array of resonant-cavity-enhanced (RCE) InGaAs photodetectors (2). Their millimeter-scale footprint and wafer-scale fabrication make them appealing for portable and embedded NIR sensing. The approach utilized by this multipixel sensor was different from conventional spectrometry because the sensor does not measure the full spectrum, but rather a limited number of spectral regions with limited resolution (50–100 nm). The target biochemical information was directly extracted from the power reflected or transmitted by the sample in these regions, without any intermediate spectral reconstruction step. The multipixel sensor has a footprint of 1.8 × 2.2 mm² and consists of an array of 16 pixels with tailored spectral responses in the 850–1700 nm wavelength range. Each pixel was fabricated within a single monolithic element having a thin absorbing layer and a tuning layer inside an optical cavity. In this approach, the detector and filter elements were directly co-integrated at the wafer-level, providing a robust system that could be fabricated at high volumes using standard semiconductor processing methods (2).

In this article, we report the results of sensing experiments using a standalone, handheld sensing module called SpectraPod, based on the direct NIR sensing concept. The module, which has been recently described in the literature (3), incorporates the 16-pixel integrated chip, an internal light source, and basic optics for light collection. The performance of the handheld module was evaluated in two exemplary real-world application cases: Quantification of the fat percentage in raw milk and plastic type classification. The purpose is to assess whether an integrated spectral sensor with very limited resolution and number of spectral points can deliver the performance required in practical application cases.

Measurements of the fat content in milk and the plastic type classification were chosen as being representative of a wider class of sensing problems that fall into either the regression or classification category. Furthermore, the accurate quantification of the fat content in milk impacts its economic value and provides insights into the cow’s health (4). Meanwhile, plastic is one of the most widely used materials today, and improving its sorting and recycling processes is crucial to reducing the enormous amount of plastic waste generated globally (5–7). NIR spectral sensing has the potential for a positive impact in both application cases, and the miniaturization of the sensor will facilitate access to and adoption of the technology.

Materials and Methods

Sample Preparation for Milk Fat Quantification Experiment

Commercially available sets of raw milk with calibrated fat concentrations were obtained (QSE GmbH) and pair-wise mixed (8) to increase the number of concentration levels.

The final sample set consisted of samples with 17 different levels of fat, ranging from 2.1–6.0 g/mL. Eight samples were assigned to the test sample set, whereas the remaining nine were the calibration set used to train the prediction algorithm. The milk samples were kept in a water bath set to 24 °C until their transfer into cuvettes for optical measurement at room temperature. The experiment was completed within one day because of the limited lifetime of the samples.

Sample Preparation for Plastic Classification Experiment

For this study, 90 plastic samples with various thicknesses, colors, and textures collected from common household waste were used in this study. Of the 90 samples, 28 were selected in a randomized way prior to measurement as the test set, such that this group consisted of seven samples of type 1 plastic (polyethylene terephthalate [PET]), three samples of type 2 (high-density polyethylene [HDPE]), seven samples of type 4 (low-density polyethylene [LDPE]), six samples of type 5 (polypropylene [PP]), two samples of type 6 (polystyrene [PS]) and three samples of type 6 foam (PS foam). All the remaining 62 samples were incorporated in the training set. More information on the samples, acquisition procedure, and data analysis for the plastic classification experiment can be found in the literature (3).

Spectral Data Acquisition

The NIR diffuse reflectance or transmission were measured using the novel, standalone handheld spectral sensing module. The core technology of the module is the ChipSense (MantiSpectra) sensor chip, an array of 16 detectors that have tailored broadband spectral responses and exhibit a high peak responsivity (>0.25 A/W) and low noise, enabling measurements at the few pW level (2,9). The multipixel detector array is contained inside the handheld module along with an adjustable internal tungsten lamp, a diffuser, an 850 nm high-pass filter, and a lens to focus the incoming light onto the sensor array. Various illumination and signal collection configurations are feasible via the attachment of several extensions, including reflectance, interactance, transmission, and a fiber-coupled input. The module is operable via an in-house developed software application that controls the instrument and acquisition settings, including the integration time, number of scan averages, and lamp power.

The module used in the milk fat quantification experiment contained a first generation of chips (“chip A”), while a second generation (“chip B”) was used for the plastic classification experiment. The response curves of the two sensor chips are shown in Figures 1b and 1c, which were measured as a function of illumination wavelength using a narrow spectral line produced by a monochromator (8 nm line width, approximately 1.34 μW power). Chip A and chip B present 15 and 16 functional pixels, respectively, whose photocurrent values were directly used in chemometrics analysis without the need for spectral reconstruction.

FIGURE 1: The standalone handheld module and response curves. (a) Schematic of the handheld module (dimensions: 8.2 L × 8.2 W × 3.4 H cm). The response curves of (b) chip A and (c) chip B.

Samples from both experiments were illuminated by the internal lamp and an integration time of 145 ms per pixel was used. Cuvettes containing the milk samples were held by a cuvette holder extension and measured in reflectance mode using five acquisition cycles. The plastic classification experiment samples were placed directly against the measurement window and measured in reflectance using one acquisition cycle. Translucent and transparent plastic samples were measured with a diffuse reflectance standard (>95% reflectivity; Ocean Insight) held against the sample.

Data Analysis

The photocurrents from each experiment were dark-corrected, and outlier analysis was done using Q2 residuals and Hotelling’s T² (10). Subsequently, the measurements were pre-processed using either sum-normalization (normalization by the sum of all the spectral data points in each measurement) or standard normal variate (SNV) prior to modeling. Partial least squares regression (PLSR) was used to model fat percentage in milk using the calibration sample set and PLS–discriminant analysis (PLS-DA) was used for classification of the plastic type. In both cases, 5-fold cross-validation (CV) was used to optimize the number of latent variables (LVs) used in the model. The errors in regression were quantified using root mean square error for calibration (RMSEC) and root mean square error for cross-validation (RMSECV), which were calculated using the calibration data set and the RMSE of prediction (RMSEP) calculated using the test set. All data analysis algorithms were implemented in Python using packages from NumPy (11), Matplotlib (12), and Scikit-learn (13).

Results and Discussion

Quantification of the Fat Percentage in Milk

Figures 2a and 2b shows the SNV-processed signals measured by the module and the modeling results of fat percentage, respectively. The model was developed using calibration samples and subsequently evaluated using test samples for external validation. Using five LVs, the PLSR model achieved a RMSEP of 0.14%, R²_P of 0.98, and a high ratio of performance to deviation (RPD) of prediction of 7.3. The RPD value considers both the standard deviation of the reference and error of the prediction, and generally, an RPD value below 1.5 indicates poor prediction performance, whereas an RPD above three is accepted as an acceptable working model (14,15). With the high RPD of 7.3, we demonstrate that a strong regression model for predicting fat percentage in raw milk was attainable using only 15 values measured from the multipixel sensor array, without measuring a full spectrum nor using spectral reconstruction.

FIGURE 2: (a) The measured photocurrent values normalized by SNV, and (b) the predicted fat percentage in milk, obtained via PLS regression using the normalized photocurrents. The number of latent variables (LVs), root mean square error of cross-validation (RMSECV), prediction (RMSEP), coefficient of determination for prediction (R²_P), and the ratio of performance to deviation (RPD) are indicated in subfigure (b).

Classification of Plastic Types

The dark-corrected photocurrents obtained via the module and the average sum-normalized photocurrent values for each type of plastic are shown in Figures 3a and 3b, respectively. The variations in intensity seen in Figure 3a are mainly because of the differences in sample thickness and color. The spectral differences between the different plastic types are highlighted in Figure 3b, as shown by the difference in the shape of the normalized photocurrent peaks. The distinct reflectance signals measured by the handheld module in the 850–1700 nm range are primarily because of the overtone vibrations of the plastic molecules. C-H, O-H, N-H, and C-O bonds have distinct absorption peaks in this NIR region, which give rise to the spectral characteristics of the plastic samples and enable the identification of their type (5,6,16).

FIGURE 3: (a) The measured photocurrent, and (b) the average sum-normalized photocurrent of each plastic type measured by the module are shown.

The model for classification of plastic types was developed using PLS-DA, with sum-normalized sensor data as input. The classification model used 10 PLS latent variables and achieved 100% accuracy in both cross-validation and test set prediction (confusion matrices in Figure 4a–b). These results show that fast and reliable determination of plastic types can be achieved using a fully integrated multipixel NIR sensor.

FIGURE 4: Results of plastic type classification using PLS-DA analysis with 10 LVs. Confusion matrices display in absolute frequency the outcomes of classification (a) on the calibration, and (b) on the test sample set. Here, 100% accuracy was achieved for both cross-validation and test set prediction.

Limitations and Future Outlook

Overall, satisfactory sensing performance was demonstrated for these two applications using the handheld module, which shows the potential of the direct sensing approach using a simple integrated multipixel sensor chip. This study clearly indicates that the small multipixel array can capture the relevant spectral information despite not measuring the spectrum. NIR spectra are complex and consist of multiple overlapping peaks that result in broad bands (17). As a result, the spectral features commonly encountered in NIR spectral sensing are broad and measurement using high spectral resolution is not always necessary to achieve the sensing goal.

However, the approach of direct sensing using multipixel sensor arrays may provide reduced performance if the chemical features are narrow (<50 nm) and are located in a wavelength region sparsely covered by the sensor’s response peaks. Generally, miniaturized sensing solutions such as the one presented in this article offer a practical approach to industrial and consumer sensing applications, they are not intended to replace high-end laboratory spectrometers.

Conclusion

In this article, we demonstrated the successful application of a handheld NIR sensor, based on a fully integrated array of RCE photodetectors to quantify fat percentage in milk and classify plastic types. This supports the direct NIR sensing approach used by the handheld module, where only a limited number of broad and overlapping spectral bands are measured compared to the measurement of the full spectrum in traditional spectrometers. It thereby shows the potential of multipixel spectral sensing as an analytical technique easily applicable outside the laboratory.

Competing Interests

Fang Ou, Petar Ševo, Maurangelo Pa- truzzella, Kaylee D. Hakkel, and Francesco Pagliano are employees and Maurangelo Patruzzella, Francesco Pagliano, and Andrea Fiore are shareholders and co-founders of MantiSpectra B.V., the company commercializing the sensor. The remaining authors declare no competing interests.

References

(1) R.A. Crocombe, Appl. Spectrosc. 72(12), 1701–1751 (2018). DOI: 10.1177/0003702818809719.

(2) K.D. Hakkel, M. Petruzzella, F. Ou, A. van Klinken, F. Pagliano, T. Liu, et al., Nat. Commun. 13(1), 1–8 (2022), DOI: 10.1038/s41467-021-27662-1.

(3) F. Ou, A. van Klinken, P. Ševo, M. Petruzzella, C. Li, D.M.J. van Elst, K.D. Hakkel et al., Sensors 22(18), 7027 (2022). DOI: 10.3390/S22187027.

(4) B. Aernouts, E. Polshin, J. Lammertyn, and W. Saeys, J. Dairy Sci. 94(11), 5315–5329 (2011). DOI: 10.3168/jds.2011-4354.

(5) H. Masoumi, S. Safavi, and Z. Khani, Int. J. Mech. Mechatron. Eng. 6(5), 213–220 (2012). DOI: doi.org/10.5281/zenodo.1076916.

(6) M. Rani, C. Marchesi, S. Federici, G. Rovelli, I. Alessandri, I. Vassalini et al., Materials 12(7), 2740 (2019). DOI: 10.3390/ma12172740.

(7) S. Zhu, H. Chen, M. Wang, X. Guo, Y. Lei, and G. Jin, Adv. Ind. Eng. Polym. Res. 2(2), 77–81 (2019). DOI: 10.1016/j.aiepr.2019.04.001.

(8) A. Bogomolov and A. Melenteva, Chemom. Intell. Lab. Syst. 126, 129–139 (2013). DOI: 10.1016/j.chemolab.2013.02.006.

(9) D. M. van Elst, A. van Klinken, F. Ou, M. Petruzzella, K.D. Hakkel, F. Pagliano et al., Proc. SPIE PC12013, MOEMS and Miniaturized Systems XXI, PC1201305 (2022). DOI: 10.1117/12.2605960.

(10) J. Kuligowski, G. Quintás, C. Herwig, and B. Lendl, Talanta 99, 566–573 (2012). DOI: 10.1016/j.talanta.2012.06.036.

(11) T. E. Oliphant, A Guide to NumPy (Trelgol Publishing, USA, 2006).

(12) J.D. Hunter, Comput. Sci. Eng. 9(3), 90–95 (2007). DOI: 10.1109/MCSE.2007.55.

(13) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel et al., J. Mach. Learn. Res. 12, 2825−2830 (2011).

(14) X. Li, Z. Xu, W. Cai, and X. Shao, Anal. Chim. Acta. 880, 26–31 (2015). DOI: 10.1016/j.aca.2015.04.026.

(15) O. Escuredo, L. Meno, M.S. Rodríguez-Flores, and M.C. Seijo, Sensors 21(24), 8222 (2021). DOI: 10.3390/S21248222.

(16) Y. Zheng, J. Bai, J. Xu, X. Li, and Y. Zhang, Waste Manag. 72, 87–98 (2018). DOI: 10.1016/j.wasman.2017.10.015.

(17) M. Manley, Chem. Soc. Rev. 43(24), 8200–8214 (2014). DOI: 10.1039/C4CS00062E.

Fang Ou, Anne van Klinken, Kaylee D. Hakkel, Maurangelo Petruzzella, Don M. J. van Elst, Petar Ševo, Chenhui Li, Francesco Pagliano, Rene P. J. van Veldhoven, and Andrea Fiore are with the Department of Applied Physics and Eindhoven Hendrik Casimir Institute, at Eindhoven University of Technology, in Eindhoven, The Netherlands. Ou, Petruzzella, Ševo, and Pagliano are also with MantiSpectra B.V., in Eindhoven, The Netherlands. Direct correspondence to: f.ou@tue.nl ●