Calibration Transfer Chemometrics, Part II: A Review of the Subject

We continue with our review of the methods and mathematics involved with the subject of calibration transfer. Calibration transfer involves several strategies and mathematical techniques for applying a single calibration database consisting of samples, reference data, and calibration equations to two or more instruments. In this installment, we continue by providing a review of the chemometric and tactical strategies used for the calibration transfer process.

The basic methods that have been published related to calibration transfer mathematics and the techniques involved in this process are discussed as a continuation of our previous column (1).

Successive Projections Algorithm

The use of the successive projections algorithm (VSPA) to select variables for building robust transferable multiple linear regression (MLR) models has been demonstrated (2). Robust MLR models have been compared to partial least squares (PLS) models using piecewise direct standardization (PDS) to correct the secondary or child instrument spectra. In data sets tested, the mean prediction errors at the child instrument for the robust VSPA-MLR and the PLS-PDS models were comparable, and slightly better for VSPA-MLR. The VSPA approach appears to be a valid alternative to the commonly used PLS-PDS technique.

An example of the VSPA for selecting variables for building robust transferable MLR models has also been demonstrated. The transfer sample set is selected using a Kennard–Stone (KS) algorithm or modified VSPA that is applied on the rows of the instrument response matrix. In published examples, the mean prediction errors of the secondary or child instrument are similar for VSPA-MLR as compared to PLS-PDS models. For some applications, the VSPA-MLR method produces lower prediction errors.

Compressed Wavelet Domain

Direct standardization in the compressed wavelet domain is termed WTDS. Calibration transfer in the compressed wavelet domain permits a reduction in processing time for the analysis of large spectral data sets with some success in calibration transfer (3).

In a published example, wavelet transform methods such as denoising, compression, and multiscale analysis have been developed for near-infrared (NIR) calibration transfer. Combined methods have been empirically tested to improve the precision of PDS. Methods compared have included

• wavelet denoising and piecewise direct standardization (WDPDS);

• wavelet compression and piecewise direct standardization (WCPDS); and

• wavelet multiscale and piecewise direct standardization (WMPDS).

Optimal results have been reported for these combined methods on various data (4).

Canonical Correlation Analysis

A method based on canonical correlation analysis (CCA) for calibration model transfer was developed for calibration transfer (5). In this work, two real NIR data sets were tested in a comparative study between CCA and PDS. A comparison of methods was most positive for the CCA technique.

Positive Matrix Factorization

Another approach to calibration transfer has been developed based on a revised factorization technique called positive matrix factorization (PMF). PMF was developed and initially applied to environment data analysis (6). It has important differences from PCA. Because it is a least squares approach to solving the factor analysis problem, it can use subjective weights for individual data points and thereby allows the inclusion of uncertain data points-for example, missing data or analysis values below the detection limit.

Spectral Regression

A calibration transfer method for NIR spectra based on spectral regression has been examined (7). A comparative study of a spectral method and PDS for standardization on two benchmark NIR data sets was demonstrated with the results using spectral regression similar to those obtained using PDS with background correction.

A procedure for the transfer of the regression equation in NIR spectroscopy, from a first (parent) instrument to a second (child) instrument, is presented in an example paper (8). The procedure uses PLS regression twice: in the first step to compute the relationship between the spectra of transfer samples of the two instruments, and in the second step to compute the regression equation (relationship between chemical variables and spectral variables) of the first instrument. In the example paper, spectra were recorded with four different instruments, in four different laboratories with transfer results given, and reported as standard error of prediction. The spectral regression method prediction results for the transfer instrument were reported to be comparable to the original calibration.

Wavelet Packet Transform Standardization

A standardization algorithm, called wavelet packet transform standardization (WPTS), based on a wavelet packet transform (WPT) and entropy criteria, was proposed to transfer spectra between two NIR spectrometers (9). By comparison, the example WPTS produces a predicted value transfer performance comparable to wavelet transform standardization 1 (WTS1), which is considered one of the best existing transfer methods, and better than wavelet transform standardization 2 (WTS2) combined with standard PDS (WTS2-PDS). The WPTS method also has a low computational requirement as an added benefit.

Stacked Partial Least-Squares Regression

A paper demonstrated the use of stacked partial least-squares (SPLS) regression and stacked dual-domain (SDD) regression analysis combined with commonly used techniques for calibration transfer to improve predictive performance from transferred multivariate calibration models (10). The paper reported that the predictive performance for gasoline samples was improved when combined with three conventional calibration transfer methods-namely, PDS, orthogonal signal correction (OSC), and model updating (MUP). These methods require transfer standards to be measured on both instruments.

Multiplicative Scatter (or Signal) Correction

An applications paper compares two multiplicative signal correction (MSC) algorithms for the standardization of data from two NIR spectrometers (11). Absorbance spectra were measured in the 1000–2200 nm range for a set of 45 jet fuel samples. Data from one instrument were standardized to match data from a second instrument using windowed MSC (W-MSC) and moving window MSC (MW-MSC). For W-MSC user-defined windows were selected and for MW-MSC the window size was optimized based on a two-step procedure: assigning a cut off window to avoid over-processing and selection of a specific window size based on sample leverage.

Multiplicative scatter (or signal) correction provides some mitigation of the scattering phenomenon by regressing the individual spectrum to the mean spectrum of a collection of spectra, for both calibration and prediction datasets (12–15). The identical algorithm must be completed for both calibration and prediction sets. When conditions or datasets are changed the algorithm is recomputed for the entire set. The algorithm assumes that the ideal spectrum is the mean spectrum, which is an oversimplification, but it provides a basis for making some correction. Light scatter and spectral measurements are complex and poorly understood so correction algorithms, although useful, do not present an ideal theoretical correction for the scattering phenomenon, but only an approximation.

For the MSC algorithm, each spectrum in a set of spectra represents the dependent variable (as a vector) in a least squares regression case where the mean spectrum is the independent variable (also a vector). From this computation, both the slope and intercept MSC variables are computed. So, the general least squares formula in standard notation is as shown in equation 1:

For this least squares linear regression equation, a_i is an individual spectrum i from the dataset, {a_i is the mean spectrum of the set of spectra, o_i is the bias or offset, and e_i is the residual spectrum.

The estimated or corrected MSC spectrum is computed using the computed slope (b_i) and intercept (o_i) constants as shown in equation 2:

MSC approximates a regression estimate of the mean spectrum with the remaining unexplained residual spectrum representing the spectral differences between the spectra within a set. This residual difference is assumed to contain much of the chemical information differences between samples in the spectrum set.

Standard Normal Variate

The standard normal variate (SNV) data preprocessing method partially compensates for some spectral slope and particle size differences within a series of spectra measured using diffuse reflection (16–19). The algorithm is applied individually to each spectrum for both calibration and prediction spectra. This processing step is applied to each data point for each spectrum using the mean and standard deviation of the spectrum in absorbance space (units). The computation is straightforward as the mean absorbance difference for each data point to the mean absorbance of the spectrum ratioed to the standard deviation of the absorbance for the spectrum. The computation formula is as shown in equation 3:

where a_ij(SNV) is the computed standard normal variate for the datapoint (element) a_ij, a_i is the mean of the spectrum i, and n is the number of data points or elements in the spectrum.

Detrend

Detrend is a general term used to describe the removal of a basic trend from data to more closely determine the signal attributes associated with analyte concentration or basic information content in a signal (16). To remove the trend (that is, detrend) data in a time series a linear least squares fit is determined for the data and then subtracted from the data. For unusual shapes, such as spectra, a polynomial function of a selected order is fit to all the data points in the spectrum and subtracted from the spectrum. The polynomial is used to fit all data points in the spectrum. This technique will remove large background interference, or specific trending variation, and is often used in combination with the SNV preprocessing function. A detrend algorithm consists of a polynomial or linear fit to each spectrum and a subtraction function of the fit function from the spectrum.

Classification Methods

Spectrometer instruments are characterized by classifying their spectra into previously defined clusters. The spectra are mapped to the clusters and a classification is made based on the similarity of extracted spectral features to one of the previously defined clusters. Calibration models for each cluster are provided to compensate for within or between instrumental variation. A simplified method of calibration transfer maps clusters to each other, so that a calibration transferred between clusters models only the difference between the two clusters, substantially reducing the complexity of the models used for transfer (20).

Synthetic Calibration Spectra

A spectrum simulation method is described for use in the development and transfer of multivariate calibration models from NIR spectra (21). By use of previously measured molar absorptivities and solvent displacement factors, synthetic calibration spectra are computed using only background spectra collected with the spectrometer for which a calibration model is desired. The resulting synthetic calibration set is used with PLS regression to form the calibration model. This methodology is demonstrated for use in the analysis of physiological levels of glucose (1–30 mM) in an aqueous matrix containing variable levels of alanine, ascorbate, lactate, urea, and triacetin.

Dispersive to Fourier Transform Transfer

NIR calibration models have been developed for the determination of content uniformity in pharmaceutical tablets (22). Data used in one study were acquired from five NIR instruments manufactured by two different vendors. One of these spectrometers was a dispersive-based NIR system while the other four were Fourier transform (FT) based. The transferability of the optimized PLS calibration models developed on the primary instrument (A) located in a research facility was evaluated using spectral data acquired from secondary instruments B, C, D, and E. Instruments B and E were located in the same research facility as spectrometer A while instruments C and D were located in a production facility 35 miles away. The same set of tablet samples was used to acquire spectral data from all instruments. To minimize the effect of instrument variability, calibration transfer techniques such as PDS and wavelet hybrid direct standardization (WHDS) were used.

A principal component regression calibration model developed on a scanning NIR spectrophotometer (with a standard cuvette) was transferred to a Fourier transform NIR spectrophotometer (with a fiber-optic probe) for the determination of sodium chloride in aqueous solutions by using the piecewise direct standardization method (23). Only two transfer samples were needed in obtaining a good calibration transfer.

Dispersive to Handheld Transfer

The objective of one study was to assess the potential of calibration transfer from the Foss NIRSystem 6500 instrument to the Polychromix Phazir instrument (24). The results show that good calibration models were obtained for various feed properties (that is, fat, fiber, protein, and starch) developed on a Foss NIRSystem 6500 instrument, based on a spectral database of 9164 samples transferred to a microelectromechanical (MEMS)-based Polychromix Phazir handheld spectrometer.

Temperature Compensation

Calibration transfer between spectra measured at different temperatures was investigated for NIR spectroscopic determination of sodium chloride in aqueous solutions (25). A PCR calibration model developed at 23.0 °C was transferred to a model at 28.5 °C by using a PDS method.

Two-Dimensional Method Transfer

A method for calibrating two-dimensional (2D) responses measured on multiple instruments or on a single instrument under different operating conditions has been demonstrated (26). From computer simulation and experimental data, it was found that the design of the standard sample spectral profile is crucial for the parameter estimations and response standardization. This is because of the phenomenon that instrument response is ample dependent.

Compensating for Extraneous Phenomena (or Unmodeled Variation)

The analysis accuracy and precision of any multivariate calibration method will be severely degraded if unmodeled sources of spectral variation are present in the unknown sample spectra. A synthetic generalized method was developed and published for correcting errors generated by the presence of unmodeled components or spectral variation (27). This method requires that the spectrum of the unmodeled component be simulated or measured and mathematically added in variable amounts to the original calibration spectra. After this adjustment, a revised calibration model is generated. This new calibration accommodates the unmodeled source of spectral variation. This method minimizes the effect of long-term drift on prediction errors and using the synthetic method eliminates the need for the expensive generation of new calibration samples.

The method is referred to as prediction-augmented classical least-squares (PACLS), and does not require that all interfering spectral characteristics be known during the calibration process. PACLS can correct calibration models if unmodeled spectral components can be identified, simulated, and added during a PACLS prediction step (28). The addition of a classical least-squares–partial least-squares (CLS–PLS) hybrid algorithm, and a prediction-augmented classical least-squares–partial least-squares (PACLS–PLS) hybrid algorithm improves the calibration process even in the presence of unmodeled sources of spectral variation (29,30).

An additional prediction-augmented classical least-squares–partial least-squares (PACLS–PLS) hybrid algorithm has been demonstrated for use in calibration transfer between instruments (31). Dramatic spectral differences caused by deviations in instrument response are incorporated into the algorithm through the use of sample spectra measured on both parent and child spectrometers.

Creating Transferable Calibrations in Mid-IR

Sensitivity studies showed that detector nonlinearity, frequency accuracy, incident angle, and variation in purge gases are important parameters to obtain transferable calibrations for Fourier transform infrared (FT-IR) calibrations (32). As the complexity of the modeled variables and calibration increases, calibration transfer becomes more challenging. Outlier detection methods are useful to determine instrument similarity and goodness of fit for a specific spectrum to a calibration database. Calibrations that maintained limited variance in instrument factors, subtracted purge gas spectra, and used data from specific robust frequencies were successfully transferred, and were also robust against spectrometer drift.

Summary

This column completes the overview of calibration transfer mathematical approaches. As new techniques are developed and published we hope to provide updates on this topic.

References

(1) J. Workman Jr. and H. Mark, Spectroscopy 32(10), 18–24 (2017).

(2) F.A. Honorato, R.K.H. Galvão, M.F. Pimentel, B. de Barros Neto, M.C.U. Araújo, and F.R. de Carvalho, Chemom. Intell. Lab. Syst. 76(1), 65–72 (2005).

(3) J. Yoon, B. Lee, and C. Han, Chemom. Intell. Lab. Syst. 64(1), 1–14 (2002).

(4) J.X. Wang, Z.N. Xing, and J. Qu, Spectroscopy 28(6), 36–41 (2013).

(5) W. Fan, Y. Liang, D. Yuan, and J. Wang, Anal. Chim. Acta 623(1), 22–29 (2008).

(6) Y. Xie and P.K. Hopke, Anal. Chim. Acta 384(2), 193–205 (1999).

(7) J. Peng, S. Peng, A. Jiang, and J. Tan, Spectrochim. Acta, Part A 78(4), 1315–1320 (2011).

(8) M. Forina, G. Drava, C. Armanino, R. Boggia, S. Lanteri, R. Leardi, and R. Bigoni, Chemom. Intell. Lab. Syst. 27(2), 189–203 (1995).

(9) T.A.N. Chao and L.I. Menglong, Anal. Sci. 23(2), 201–206 (2007).

(10) W. Ni, S.D. Brown, and R. Man, Anal. Chim. Acta 661(2), 133–142 (2010).

(11) K.E. Kramer, R.E. Morris, and S.L. Rose-Pehrsson, Chemom. Intell. Lab. Syst. 92(1), 33–43 (2008).

(12) P. Geladi and E. Dåbakk, J. Near-Infrared Spectrosc. 3(3), 119–132 (1995).

(13) M.S. Dhanoa, S.J. Lister, R. Sanderson, and R.J. Barnes, J. Near-Infrared Spectrosc. 2(1), 43–47 (1994).

(14) H. Martens, J.P. Nielsen, and S.B. Engelsen, Anal. Chem. 75(3), 394–404 (2003).

(15) J.-W. Jin, Z.-P. Chen, L.-M. Li, R. Steponavicius, S.N. Thennadil, J. Yang, and R.-Q. Yu, Anal. Chem. 84(1), 320–326 (2012).

(16) R.J. Barnes, M.S. Dhanoa, and S.J. Lister, Appl. Spectrosc.43(5), 772–777 (1989).

(17) A. Candolfi, R. De Maesschalck, D. Jouan-Rimbaud, P.A. Hailey, and D.L. Massart, J. Pharm. Biomed. Anal. 21(1), 115–132 (1999b).

(18) T. Woodcock, G. Downey, D. Kelly, and C. O'Donnell, J. Agric. Food Chem. 55(22), 9128–9134 (2007).

(19) K. Tanaka, Y. Kuba, T. Sasaki, F. Hiwatashi, and K. Komatsu, J. Agric. Food Chem. 56(19), 8787–8792 (2008).

(20) K.H. Hazen, T.B. Blank, S. Monfre, snd T.L. Ruchti, "Method of Characterizing Spectrometer Instruments and Providing Calibration Models to Compensate for Instrument Variation," U.S. Patent 6864978 (2005).

(21) Y. Sulub and G.W. Small, Appl. Spectrosc. 61(4), 406–413 (2007).

(22) Y. Sulub, R. LoBrutto, R. Vivilecchia, and B.W. Wabuyele, Anal. Chim. Acta 611(2), 143–150 (2008).

(23) J. Lin, S.C. Lo, and C.W. Brown, Anal. Chim. Acta 349(1–3), 263–269 (1997).

(24) J.A.F. Pierna, P. Vermeulen, B. Lecler, V. Baeten, and P. Dardenne, Appl. Spectrosc. 64(6), 644–648 (2010).

(25) J. Lin, Appl. Spectrosc. 52(12), 1591–1596 (1998).

(26) B.R. Kowalski and Y. Wang, "Calibration transfer for second order analytical instruments," U.S. Patent 5559728 (1996).

(27) D.M. Haaland, Appl. Spectrosc. 54(2), 246–254 (2000).

(28) D.M. Haaland and D.K. Melgaard, Appl. Spectrosc. 54(9), 1303–1312 (2000).

(29) D.M. Haaland and D.K. Melgaard, Appl. Spectrosc. 55(1), 1–8 (2001).

(30) C.M. Wehlburg, D.M. Haaland, D.K. Melgaard, and L.E. Martin, Appl. Spectrosc. 56(5), 605–614 (2002).

(31) C.M. Wehlburg, D.M. Haaland, and D.K. Melgaard, Appl. Spectrosc. 56(7), 877–886 (2002).

(32) I.S. Adhihetty, J.A. McGuire, B. Wangmaneerat, T.M. Niemczyk, and D.M. Haaland, Anal. Chem. 63(20), 2329–2338 (1991).

Jerome Workman Jr.

Jerome Workman Jr. serves on the Editorial Advisory Board of Spectroscopy and is the Executive Vice President of Engineering at Unity Scientific, LLC, in Milford, Massachusetts. He is also an adjunct professor at U.S. National University in La Jolla, California, and Liberty University in Lynchburg, Virginia.

Howard Mark

Howard Mark serves on the Editorial Advisory Board of Spectroscopy and runs a consulting service, Mark Electronics, in Suffern, New York. Direct correspondence to: SpectroscopyEdit@UBM.com