Calibration Transfer Chemometrics, Part I: Review of the Subject

Calibration transfer involves multiple strategies and mathematical techniques for applying a single calibration database consisting of samples, reference data, and calibration equations to two or more instruments. The instruments used for initial calibration development and transfer may be of like or meaningfully different optical designs. In a two-part series of articles, we take a broad overview of the chemometric and tactical strategies used for the calibration transfer process. Here in part I, the emphasis is on methods that modify the spectra or regression vectors to correct differences between instruments involved in the calibration transfer process. Part II will review calibration transfer mathematics and approaches.

Calibration transfer in its purest form allows a calibration to be moved from one instrument to another with statistically retained accuracy and precision (1,2). Ideally, a model developed on one instrument would provide a statistically identical analysis when used on other instruments, without bias or slope adjustments, without additional product samples for standardization, and without product recalibration.

A variety of textbooks and papers exist describing the general methods and history of calibration transfer. Among these are references describing the details of the background, chemometrics, types of instruments, and differences in sample chemistry (3,4). One comprehensive review of calibration transfer presents an overview of the different mathematical methods used for calibration transfer as well as a critical assessment of their validity and applicability (5). The focus of this review was on a description of methods used for transfer of calibrations for near-infrared (NIR) spectra (5).

We note there are at least four basic strategies that can be used before the calibration modeling step for calibration transfer:

• instrument matching or making instruments truly alike,

• global modeling or developing models that include varying instrumental parameters, sample preparation, and environmental factors in the data before computing the calibration method,

• model updating, which adds samples from each transfer instrument before computing a revised calibration, and

• sensor selection or finding regression variables that are somewhat insensitive to variation in measurement conditions, yet remain sensitive to the analyte signal.

Most of the methods discussed in this column refer to making one set of spectral responses from one instrument similar to a second set of responses from a second instrument for transfer; these methods would generally be categorized as instrument matching or standardization methods. 

Another prominent review of calibration transfer methods describes multiple chemometric techniques used (6). In this review paper, calibration transfer methods are classified as being represented by three main technical approaches:

• by developing more robust calibrations, derived by combining data from multiple instruments and by using spectral pretreatments of that data,

• by modifying calibrations using bias and slope adjustments (the traditional method), and

• by adjusting spectra, using such methods as piecewise direct standardization (PDS) combined with a patented method that requires spectra from transfer sample sets measured on each instrument involved in the transfer experiment (7).

Again, most of the methods described within this column refer to adjusting spectra or coefficients to match instruments.

A comprehensive book chapter presents a review of calibration transfer methods and a critical assessment of the more common methods in terms of applicability and capabilities (8). This chapter focuses on methods for calibration transfer using NIR spectra. It is noted that the methodology described is widely applicable to other analytical measurement techniques (8). Comprehensive reviews of chemometric methods, including calibration transfer, have been published since 1980 as part of the Fundamental Reviews in Analytical Chemistry (9). One of the most cited of those fundamental chemometrics reviews delves into multiple topics related to calibration methods and calibration transfer techniques (10).

Comparison of Transfer Methods

A popular and patented technique for transferring calibration models from a reference analytical instrument to a target analytical instrument is demonstrated as a U.S. patent (11). For this technique, a set of diverse transfer samples are measured on a reference instrument, producing a reference instrument response for each sample. These measurements are repeated for the target instrument, to produce a target instrument response for each transfer sample. One then generates transfer coefficients capable of performing a multivariate estimation of the reference instrument responses for the transfer samples from the target instrument responses for those samples. The transfer coefficients may then be used to convert a target instrument response for an unknown sample into the equivalent response for the reference instrument. The patent describes piecewise, classical, and inverse transfer techniques. The target instrument responses for the transfer samples are combined with reference instrument responses for the full set of calibration samples, to derive a multivariate prediction model for the target instrument (11).

A research paper describes the transfer of calibration models between NIR spectrometric instruments using three different standardization sets representing different sample chemistry types to be used as transfer standards. The first set contained samples that are very similar to the agricultural samples from three different sets to be analyzed, the second set contained generic biological and chemical standards, and the third transfer sample set contained pure organic and inorganic chemicals (12). Another research paper describes several standardization strategies for calibration transfer methods as well as an analysis of problems associated with the choice of the standardization samples (13).

Instrument Alignment and Correction

There are a number of research publications that describe strategies and techniques for instrument alignment and correction (14–18). In these approaches, the instrument wavelength or frequency (X) and photometric (Y) axes are carefully maintained as compared to physical standards so that instrument drift and the differences between instrument wavelength and photometric axes scales are minimized. Full spectrum instruments and research associated with basic instrument measurement technology, instrumentation alignment, mathematical calculations, measuring agreement between instruments, and applying wavelength and photometric standards are described in a series of papers (19–25). The current state of commercial NIR spectrophotometers and their comparative accuracies are presented in this series. The performance of different instrument types, such as monochromator, Fourier transform, and diode-array–based instruments are compared in detail with data comparisons shown in tabular and graphical forms (19–25). An example of the practical issues associated with multivariate calibration transfer (or calibration transfer) for NIR spectroscopic instruments using reflection spectroscopy for flour analysis was discussed in a recent paper (26). A trademarked and patent pending method that aligns instrument parameters to first principles physics has been described, namely True Alignment Spectroscopy (TAS, Unity Scientific) (27). In this technique, all instrument parameters are aligned to the "true" physical values of first principles standards, such as emission lamps and pure solids, gases, or liquid materials. This method does not rely on a single instrument response function as the standard like what is applied using a "master" instrument concept, but rather relies on basic physical science and the measurement of constant physical standards. Figure 1 illustrates the differences between the concept of first principles alignment and master instrument alignment.

Figure 1: The use of a first physical principles alignment technique provides constant and accurate alignment of all instruments to unchanging physics at any time. The use of a master instrument provides uncertain similarity alignment to the master instrument response that will vary over time.

The Master Instrument Concept

The utilization of NIR spectroscopy by the forage and feed industry was initially limited by the requirement to individually calibrate each instrument. Calibration transfer became a necessity for practical application and implementation of NIR over time because of the collection of large, comprehensive databases of more than 1000 or more samples. In the initial application of the "master instrument" concept, 60 sealed samples were used for measurements on primary ("master") and target ("slave") instruments for the transfer process. Note that master and slave terms are anachronisms, and have been changed to parent and child instruments, or primary and secondary instruments, respectively. In this method, one monochromator is selected to be the master instrument. Calibration equations are developed on it and transferred to one or more slave instruments. In one publication, six transfer instruments were evaluated (28). The standard errors of difference (SED) including bias among the six transfer instruments for crude protein, acid detergent fiber, neutral detergent fiber, lignin, and in vitro dry matter disappearance were less than the comparative SED among laboratories for reference chemical analysis. It was concluded that satisfactory calibration transfer could be accomplished with 10 samples. This was the most successful and practical method for calibration transfer in the mid-1980s.

Filter Instrument Calibration Transfer

There have been many approaches to correcting instruments for calibration transfer. In one instance, filter instruments were used and a method was demonstrated for computing calibrations that are robust against filter differences and bandwidths of filters. Common methods for wavelength selection involve using absorbance maxima for analyte spectra, using correlation plots relating the change in absorbance with respect to a change in concentration, and using computerized wavelength search algorithms relating to specific wavelengths and their correlation to changes in concentration.

The technique of selecting isonumeric wavelengths allows the development of general wavelength positions and regression coefficients that are robust against small variations in center wavelength positions for filters because of manufacturing variations. This technique also applies to the wavelength variations that exist in full scanning instruments (29). In the isonumeric method, small changes in wavelength center positions do not affect regression coefficients or the predicted results. To illustrate, given the standard regression equations for every multivariate model expressed in algebraic notation as follows:

where A_j is the measured absorbance at wavelength j, C₁ and C₂ are the concentrations for samples 1 and 2, and b_ij represents the regression coefficients for sample i and wavelength j. The bias is the offset value for the regression.

For the regression coefficients to be immune to wavelength shifts, the following conditions must hold for the regression coefficients b₁₁ and b₁₂:

An equation derived through a proof in the reference paper leaves the final equation as follows:

where Y is the reference laboratory value and Ŷ_j is the predicted value for any sample. The end result of a wavelength search under the conditions that the change in wavelength does not change the regression coefficients, allows the derivation of a calibration equation that is robust against small wavelength shifts within and between instruments.

Direct Standardization and Piecewise Direct Standardization

The second and third most used approaches to multivariate calibration transfer involve the application of direct standardization (DS) and piecewise direct standardization (PDS) (30–33). These approaches are described in detail with application descriptions, examples, and equations within the cited references. The DS and PDS approaches are also often combined with small adjustments in bias or slope of the predicted values to compensate for small differences not accounted for by using standardization algorithms. Note that the frequency with which standardization approaches must be applied to child instruments is dependent on the frequency of the calibration updates required and the drift of the child instruments with respect to the parent (or original calibration instrument).

PDS was applied to a set of gasoline samples measured on two different NIR spectrometers (34). PDS was applied to a set of two-component samples measured on the same ultraviolet–visible (UV–vis) spectrometer with the use of a cuvette cell with a 10-mm pathlength and a fiber-optic probe with a 2-mm pathlength. Piecewise direct standardization proceeds by determining a structured transformation matrix using the spectra of a few samples measured with both devices. This transformation matrix can then be used to transform any spectrum measured on one device to that obtained on another device, thereby making the calibration model transferable between devices (35). More detailed chemometric improvements to the basic technique of PDS for NIR spectrometric instruments are further described for multivariate calibration transfer in a 1996 paper (36).

For the PDS computation, we apply the following equations:

where A_Pj is the response of the standardization samples measured on the parent instrument at wavelength j; A_Cj is the response for the standardization samples measured on the child instrument at wavelength j; and b_j is the transformation or correction vector of coefficients for the jth wavelength.

The transformation matrix for PDS is computed using equation 6:

where k is the total number of wavelengths in the measured standardization samples.

Finally, the response prediction equation for any unknown sample measured on the child instrument is estimated using equation 7:

For the DS method, the test sample set is measured on the parent and child instruments as typically absorbance (A) with respect to wavelength (k). The spectral data have k individual wavelengths. A transformation matrix (T) is used to match the child instrument data (A_C) to the parent instrument data (A_P). And so equation 8 demonstrates the matrix notation. Note that for DS a linear relationship is assumed between the parent and child measurement values.

In equation 8, A_P is the parent data for the test sample set as an n × k matrix (n samples and k wavelengths), A_C is the child instrument data for the test samples as an n × k matrix, Tˆ is the k × k transformation matrix, and E is the unmodeled residual error matrix.

The transformation matrix (Tˆ) is computed as equation 9:

where A⁺_C is the pseudoinverse approximated using singular value decomposition of the n × k spectral data matrix for a set of transfer or standardization samples measured on the child instrument, and A_P is the n × k spectral data matrix for the same set of transfer or standardization samples measured on the parent instrument. The transform matrix is used to convert a single spectrum measured on the child instrument to be converted to "look" like a parent instrument spectrum.

The response vector for a new measured sample designated as a is predicted using the original model equation as follows:

where superscript T is the matrix transpose and â represents the estimated or predicted value for the measured sample a.

For the PDS method, the DS method is used piecewise or with a spectral windowing method to more closely match the spectral nuances and varying resolution and lineshape of spectra across the full spectral region, and there is no assumption of linearity between the parent and child prediction results. The transformation matrix is formed in an iterative manner across multiple windows of the spectral data in a piecewise fashion. Many other approaches have been published and compared, but for many users these are not practicable or have not been adopted for various reasons; basic method comparisons are demonstrated in the literature (37).

Orthogonal Signal Correction

An example of orthogonal signal correction (OSC) was applied to NIR spectra that were used in a calibration for the water content in a pharmaceutical product (38). Partial least squares (PLS) calibrations were then compared to other calibration models with uncorrected spectra, models with spectra subjected to multiplicative signal correction, and a number of other transfer methods. The performance of OSC was on the same level as for piecewise direct standardization and spectral offset correction for each individual instrument and PLS models with both instruments included.

The goal of OSC is to remove variation from the spectral data, A, that are orthogonal to Y. This orthogonal variation is modeled by additional components for A and results in the decomposition as shown in equation 11:

where t_o and p_o represent the scores and loadings for the orthogonal component and e is the residual. By removing the Y-orthogonal variation from the data via A – t_op_o', OSC maximizes correlation and covariance between the X and Y scores to achieve both good prediction and interpretation. As for OSC, there are several algorithms that have been reported in the literature (40). These algorithms differ in the approach used to obtain t_o. For new spectral data, A_new, a new score vector is calculated and multiplied with the loading vector (p') computed previously. In the final step, the product of the two is subtracted from A_new.

Procrustes Analysis

In analytical chemistry, it is necessary to form instrument-dependent calibration models. Problems such as instrument drift, repair, or use of a new instrument create a need for recalibration. Since recalibration can require considerable costs and cause time delays, methods for calibration transfer have been developed. One paper shows that many of these approaches are based on the statistical procedure known as procrustes analysis (PA) (41). Transfer by PA methods is known to involve translation (mean-centering), rotation, and stretching of instrument responses (41). The standard PA steps include translation, uniform scaling, rotation, and fine adjustments to superimposing the signals or spectra. An excellent tutorial on the general use of the procrustes technique in chemistry is found in the literature (42).

Finite Impulse Response

In an example paper, four different calibration transfer techniques are compared (43). Three established techniques, finite impulse response (FIR) filtering, generalized least squares weighting (GLSW), and PDS, were evaluated. A fourth technique, baseline subtraction, was reported to be the most effective for calibration transfer. Using as few as 15 transfer samples, the predictive capability of the analytical method was maintained across multiple instruments and major instrument maintenance (43). Previously, a standard-free method using the FIR filter was successfully used to transfer the NIR spectra of caustic brines, analgesics, and terpolymer resins. This example paper carries the FIR transfer method one step further, leading to an improved algorithm that makes the transfer more robust and general by avoiding transfer artifacts in the filtered spectra (44).

In a second example paper, FIR filtering was used for a set of spectra to be transferred, using a spectrum on the target instrument to direct the filtering process (45). Often, the target spectrum is the mean of a calibration set. The method is compared against direct transfer and piecewise direct transfer on NIR reflectance spectra in two representative data sets. Results from these studies suggest that FIR transfer compares favorably with piecewise direct transfer in terms of accuracy and precision of the match of transferred spectra to the predictive calibration models developed on the target instrument. Unlike piecewise direct transfer, FIR transfer requires no measurement of standard samples on both the source and target spectrometers. Details and limitations of the FIR transfer method are presented in the literature (45).

Maximum Likelihood Principal Component Analysis

A calibration transfer method, called maximum likelihood principal component analysis (MLPCA), is analogous to conventional principal component analysis (PCA), but incorporates measurement error variance information in the decomposition of multivariate data (46). A very detailed description of the derivation, computations, and results discussion is given in the literature (47).

Using Wavelength Standards for FT-NIR Alignment

A series of papers describe the use of a powdered mixture of Er₂O₃, Dy₂O₃, Ho₂O₃, and talc measured at a constant resolution of 2 cm^-1 on four combinations of spectrometers and sampling accessories. The wavenumber scale of each mixture spectrum was corrected with lines in the vibration–rotation spectrum of water vapor to an accuracy better than 0.02 cm^-1. The claim for this work is that the reported values for the DR standard are improved by an order of magnitude compared to the values reported by the National Institute of Standards and Technology (NIST). These values for the standard are used for testing or calibrating alignment for the wavelength axis of a spectrometer. A discussion of the various methods, and results, is provided in the literature (48–50) and will be further described in greater detail in subsequent columns on the subject of reference standards.

Summary

There are multiple mathematical techniques being applied for transfer of multivariate calibrations. Some of these techniques were described within this column with appropriate references. In the next installment in this series, we will continue to provide a basic top-level review of calibration transfer methods.

References

(1) J.J. Workman Jr., in 40 Years of Chemometrics–From Bruce Kowalski to the Future (American Chemical Society, Washington, D.C., 2015), pp. 257–282.

(2) H. Mark and J. Workman, Spectroscopy 28(2), 24–37 (2013).

(3) H. Mark and J Workman, Chemometrics in Spectroscopy, 1st Edition (Elsevier/Academic Press, Boston, Massachusetts, 2010).

(4) J. Workman, in The Concise Handbook of Analytical Spectroscopy: Physical Foundations, Techniques, Instrumentation and Data Analysis, J. Workman, Ed. (World Scientific Publishing-Imperial College Press, in 5 Volumes, UV, Vis, NIR, IR, and Raman, New Jersey and Singapore, 2016), volume 3, pp. 185–188, volume 4, pp. 159–162 (2016).

(5) R.N. Feudale, N.A. Woody, H. Tan, A.J. Myles, S.D. Brown, and J. Ferré, Chemom. Intell. Lab. Syst. 64(2), 181–192 (2002).

(6) T. Fearn, J. Near-infrared Spectrosc. 9(4), 229–244 (2001).

(7) J. Shenk and M. Westerhaus, U.S. Patent 4,866,644 September 12, 1989.

(8) S.D. Brown, Chemom. Intell. Lab. Syst. 64(2), 181–192 (2002).

(9) B.R. Kowalski, Anal. Chem. 52(5), 112R–122R (1980).

(10) B. Lavine and J. Workman, Anal. Chem. 80(12), 4519–4531 (2008).

(11) B.R. Kowalski, D.J. Veltkamp, and Y.D. Wang, U.S. Patent 5,459,677 October 17, 1995.

(12) E. Bouveresse, D.L. Massart, and P. Dardenne, Anal. Chim. Acta 297(3), 405–416 (1994).

(13) O.E. De Noord, Chemom. Intell. Lab. Syst. 25(2), 85–97 (1994).

(14) J. Workman, B. Kowalski, and P. Mobley, "The Design of an Expert Calibration System for Spectroscopic Based Process Analytical Chemistry," presented at 40th Annual ISA Analysis Division Symposium, Toronto, Ontario, Canada, April 1995.

(15) J. Workman, in Spectrophotometry, Luminescence and Colour, Science and Compliance, C. Burgess and D. G. Jones, Eds. (Elsevier, Amsterdam, 1995), pp. 369–384.

(16) D. Tracy, R. Hoult, and A. Ganz, U.S. Patent 5,303,165 April 12, 1994.

(17) K. Busch and D. Rabbe, U.S. Patent 6,774,368 August 10, 2004.

(18) J. Workman Jr. and H. Mark, Spectroscopy 30(7), 32–37,48 (2015).

(19) J. Workman Jr. and J.P. Coates, Spectroscopy 8(9), 36–42 (1993).

(20) A. Folch-Fortuny, R. Vitale, O. de Noord, and A. Ferrer, J. Chemom. 31(3), e2874 (2017).

(21) J. Workman Jr. and H. Mark, Spectroscopy 28(5), 12–25 (2013).

(22) J. Workman Jr. and H. Mark, Spectroscopy 28(6), 28–35 (2013).

(23) J. Workman Jr. and H. Mark, Spectroscopy 28(10), 24–33 (2013).

(24) J. Workman Jr. and H. Mark, Spectroscopy 29(6), 18–27 (2014).

(25) J. Workman Jr. and H. Mark, Spectroscopy 29(11), 14–21 (2014).

(26) C. Eskildsen, P. Hansen, T. Skov, F. Marini, and L. NØrgaard, J. Near Infrared Spectrosc. 24(2), 151–156 (2016).

(27) J. Workman Jr., NIR News 27(3), 12–15 (2016).

(28) J.S. Shenk, M.O. Westerhaus, and W.C. Templeton, Crop Science 25(1), 159–161 (1985).

(29) H. Mark and J. Workman Jr., Spectroscopy 3(11), 28–36 (1988).

(30) Y. Wang, D. Veltkamp, B.R. Kowalski, Anal. Chem. 63(23), 2750–2756 (1991).

(31) Z. Wang, T. Dean, and B.R. Kowalski, Anal. Chem.67(14), 2379–2385 (1995).

(32) L. Zhang, G. Small, and M. Arnold, Anal. Chem. 75(21), 5905–5915 (2003).

(33) B.R. Kowalski, D. Veltkamp, and Y. Wang, U.S. Patent 5,459,677 October 17, 1995.

(34) Y. Wang and B.R. Kowalski, Appl. Spectrosc. 46(5), 764–771 (1992).

(35) C.S. Chen, C.W. Brown, and S.C. Lo, Appl. Spectrosc. 51(5), 744–748 (1997).

(36) E. Bouveresse and D.L. Massart, Chemom. Intell. Lab. Syst. 32(2), 201–213 (1996).

(37) X. Luo, A. Ikehata, K. Sashida, S. Piao, T. Okura, and Y. Terada, J. Near Infrared Spectrosc. 25(1), 15–25 (2017).

(38) J. Sjöblom, O. Svensson, M. Josefson, H. Kullberg, and S. Wold, Chemom. Intell. Lab. Syst. 44(1), 229–244 (1998).

(39) S. Wold, H. Antti, F. Lindgren, and J. Öhman, Chemom. Intell. Lab. Syst. 44(1), 175–185 (1998).

(40) H. Goicoechea and A. Olivieri, Chemom. Intell. Lab. Syst. 56(2), 73–81 (2001).

(41) C.E. Anderson and J.H. Kalivas, Appl. Spectrosc. 53(10), 1268–1276 (1999).

(42) J.M. Andrade, M. Gómez-Carracedo, W. Krzanowski, and M. Kubista, Chemom. Intell. Lab. Syst. 72(2), 123–132 (2004).

(43) R.P. Cogdill, C.A. Anderson, and J.K. Drennen, AAPS PharmSciTech. 6(2), E284–E297 (2002).

(44) H. Tan, S.T. Sum, and S.D. Brown, Appl. Spectrosc. 56(8), 1098–1106 (2002).

(45) T. Blank, S. Sum, S.D. Brown, and S. Monfre, Anal. Chem. 68(17), 2987–2995 (1996).

(46) D.T. Andrews and P.D. Wentzell, Anal. Chim. Acta 350(3), 341–352 (1997).

(47) P. Wentzell, D. Andrews, D. Hamilton, K. Faber, and B.R. Kowalski, J. Chemom. 11(4), 339–366 (1997).

(48) H. Yang, T. Isaksson, R.S. Jackson, and P.R. Griffiths, J. Near Infrared Spectrosc. 11(4), 229–240 (2003).

(49) H. Yang, T. Isaksson, R.S. Jackson, and P.R. Griffiths, J. Near Infrared Spectrosc. 11(4), 241–255 (2003).

(50) T. Isaksson, H. Yang, G.J. Kemeny, R.S. Jackson, Q. Wang, M.K. Alam, and P.R. Griffiths, Appl. Spectrosc. 57(2), 176–185 (2003).

Jerome Workman Jr

Jerome Workman Jr. serves on the Editorial Advisory Board of Spectroscopy and is the Executive Vice President of Engineering at Unity Scientific, LLC, in Milford, Massachusetts. He is also an adjunct professor at U.S. National University in La Jolla, California.

Howard Mark

Howard Mark serves on the Editorial Advisory Board of Spectroscopy and runs a consulting service, Mark Electronics, in Suffern, New York. Direct correspondence to: SpectroscopyEdit@UBM.com