Data Analytics, Statistics, Chemometrics, and Artificial Intelligence

Mathematics is a formal logic system, perhaps the ultimate formal logic system. Here we describe the elegance of the foundations of the mathematics that chemometrics is based on.

We explore how different algorithms and different numbers of factors affect the results.

Spreadsheets are often used to perform GMP-related calculations, but can lead to serious problems and unnecessary risk. We explain why the use of spreadsheets is heavily discouraged in a regulated laboratory environment.

We provide a scorecard of chemometric techniques used in spectroscopy. The tables and lists of reference sources given here provide an indispensable resource for anyone seeking guidance on understanding chemometric methods or choosing the most suitable approach for a given analysis problem.

Chemometrics in Spectroscopy is a collection of column articles that the authors published in Spectroscopy over a period spanning more than two decades. Each article is generally arranged as a chapter in the book, and chapters dealing with the same or similar topics are arranged closely as a section block rather than following the original sequence in the magazine. Although each article or series of articles only discusses one specific topic, collectively, the articles form a comprehensive reference that is a valuable source for readers wanting to learn chemometrics, especially with its applications in spectroscopy.

A newly discovered effect can introduce large errors in many multivariate spectroscopic calibration results. The CLS algorithm can be used to explain this effect. Having found this new effect that can introduce large errors in calibration results, an investigation of the effects of this phenomenon to calibrations using principal component regression (PCR) and partial least squares (PLS) is examined.

In the current data-integrity–centric world, is outsourcing your spectroscopy work a good idea? To answer this question, one must consider several factors.

Calibration transfer involves multiple strategies and mathematical techniques for applying a single calibration database to two or more instruments. Here, we explain the methods to modify the spectra or regression vectors to correct differences between instruments.

A big question in forensic science today is, “How do we best report uncertainty?” The answer to which approach is “best” turns out to be surprisingly complex, for many reasons.

The archnemesis of calibration modeling and the routine use of multivariate models for quantitative analysis in spectroscopy is the confounded bias or slope adjustments that must be continually implemented to maintain calibration prediction accuracy over time. A perfectly developed calibration model that predicted well on day one suddenly has to be bias adjusted on a regular basis to pass a simple bias test when predicted values are compared to reference values at a later date. Why does this problem continue to plague researchers and users of chemometrics and spectroscopy?

When using any regression technique, either linear or nonlinear, there is a rational process that allows the researcher to select the best model.

The statistical methods used for evaluating the agreement between two or more instruments (or methods) for reported analytical results are discussed, with an emphasis on acceptable analytical accuracy and confidence levels using two standard approaches, standard uncertainty or relative standard uncertainty, and Bland-Altman "limits of agreement."

This article describes the application of chemometric methods and statistics for reporting clinical quantitative measurement methods. The equations and terminology are consistent with the Clinical and Laboratory Standards Institute (CLSI) guidelines. These chemometric and statistical methods describe the accuracy and precision of a test method compared to a reference method for a single analyte determination. Part I will introduce these concepts and Part II will discuss the statistical underpinnings in greater detail.

Columnists Howard Mark and Jerome Workman, Jr. take a final look at the topic of principal components, which has been the subject of six previous installments.

This column is a continuation of the set we have been working on to explain and derive the equations behind principal components (1–5). As we usually do, when we continue the discussion of a topic through more than one column, we continue the numbering of equations from where we left off.

For a system of homogeneous equations to have a solution other than the trivial solution, the determinant of the system of equations must be zero.

Howard Mark and Jerome Workman, Jr. continue their discussion of the derivation of the principal component algorithm using elementary algebra.

Howard Mark and Jerome Workman, Jr. continue their discussion of the derivation of the principal component algorithm using elementary algebra.

Paper is easy to archive, but what about archiving electronic records? What do you do with all the electronic records that are generated? In this month's installment, columnist Bob McDowall explores the issue of electronic records management and looks at the recent guidance issued by the OECD for GLP laboratories.

Using information provided by guidance documents from outside the spectroscopy laboratory can be very useful when trying to meet the regulations that we must follow.

In this month's installment, columnists Howard Mark and Jerome Workman, Jr. present the derivation of the principal component algorithm using elementary algebra.

This column is the continuation of a series (1-5) dealing with the rigorous derivation of the expressions relating the effect of instrument (and other) noise to its effects on the spectra we observe. Our first column in this series was an overview. While subsequent columns dealt with other types of noise sources, the ones listed analyzed the effect of noise on spectra when the noise is constant detector noise (that is, noise that is independent of the strength of the optical signal). Inasmuch as we are dealing with a continuous series of columns, on this branch in the thread of the discussion, we again continue the equation numbering and use of symbols as though there were no break. The immediately previous column (5) was the first part of this set of updates of the original columns.

In the second part of this series, columnists Jerome Workman, Jr. and Howard Mark continue their discussion of the limitations of analytical accuracy and uncertainty.

This tutorial reviews the mathematical models for dealing with interelement effects in optical emission and X-ray fluorescence spectrochemical analysis. Line overlaps and matrix effect corrections are examined.

In this month's installment of "Chemometrics in Spectroscopy," the authors again explore that vital link between statistics and chemometrics, this time with an emphasis on the statistics side.

Combining the three techniques of LC, MS, and NMR into one integrated system provides optimal use of NMR intrument time by using information-rich MS data to automatically guide the NMR operation. Here, the authors explore just this type of system.

In this month's installment of "Chemometrics in Spectroscopy," the authors explore that vital link between statistics and chemometrics, with an emphasis on the chemometrics side.

At this point in our series dealing with linearity, we have determined that the data under investigation do indeed show a statistically significant amount of nonlinearity, and we have developed a way of characterizing that nonlinearity. Our task now is to come up with a way to quantify the amount of nonlinearity, independent of the scale of either variable, and even independent of the data itself.

At this point in our series dealing with linearity, we have determined that the data under investigation do indeed show a statistically significant amount of nonlinearity, and we have developed a way of characterizing that nonlinearity. Our task now is to come up with a way to quantify the amount of nonlinearity, independent of the scale of either variable, and even independent of the data itself.

This column presents results from some computer experiments designed to assess a method of quantifying the amount of non-linearity present in a dataset, assuming that the test for the presence of non-linearity already has been applied and found that a measurable, statistically significant degree of non-linearity exists.