Are We There Yet? Is There Such a Thing as an Expert Calibration System for Vibrational Spectroscopy?

The development of an expert calibration system (ECS) for spectroscopic-based process analytical chemistry would be a significant advancement aimed at automating the creation of high-quality calibration models for standard zero-order and first-order calibrations as well as multidimensional imaging applications. The concept of ECS would seek to reduce the reliance on users’ extensive knowledge of chemometrics all the while leveraging their domain knowledge and understanding of specific sample physical and chemical properties. By providing automated tools and guidance, an ECS would aim to streamline the calibration process, improve calibration transfer, enhance operator efficiency, and improve the overall consistency and reliability of analytical results produced using advanced chemometrics and machine earning techniques. In this article, we resurrect the discussion from nearly three decades ago.

At the 1995 Instrument Society of America (ISA) Conference in Toronto, Canada, there was a paper presented and published entitled, “The Design of an Expert Calibration System for Spectroscopic Based Process Analytical Chemistry,” authored by J. Workman, B. Kowalski, and P. Mobley (1). This presentation suggested that an automated expert calibration system (ECS) would be of keen interest to those analysts using spectroscopic analytical techniques, especially for process analysis applications. Based on the then state-of-the-art computer-based technology, an overview was described for such a system. The paper explained a complete ECS program that would include data transformations, data preprocessing, linear and non-linear calibration methods, model output validation, calibration performance monitoring, and model transfer optimization. The paper further suggested that additional features for an ideal ECS would include online help screens, computational time estimates, automatic report generation (with a description of pertinent details), and multiple user levels based upon operator expertise and sophistication (1). Because of the age of this presentation, we review its content here with the intention of expanding upon these concepts in future articles.

Process analytical chemistry plays a crucial role in various industries, facilitating the optimization of processes, improving productivity, ensuring product quality, and minimizing waste. The adoption of optical spectroscopic methods, such as ultraviolet (UV), visible (vis), near-infrared (NIR), mid-infrared (MIR), and Raman spectroscopy, has become increasingly common because of their speed, precision, non-destructive nature, and ability to perform multicomponent analyses. These methods rely on multivariate quantitative analysis techniques to establish relationships between instrument responses (absorbance, reflection, transmittance, and intensity) and analyte parameter concentrations.

Background

Process analytical chemistry has witnessed remarkable progress, driven by the demand for rapid, accurate, and reliable analytical information in industrial settings. Optical spectroscopic methods have emerged as the preferred techniques for process analysis because of their versatility and efficiency. These methods allow for real-time monitoring of various parameters, enabling proactive process control and optimization.

The traditional approach to calibration involves establishing relationships between instrument responses and analyte concentrations using chemometric techniques, such as Beer’s law, combined with regression analysis. However, this process requires considerable expertise in chemometrics, which may not be readily available to all users. Additionally, manual calibration procedures are time-consuming and prone to errors, leading to suboptimal results.

The Concept of an Expert Calibration System (ECS)

The ECS concept aims to bridge the gap between user expertise and calibration requirements by providing automated, user-friendly tools for calibration model development and utilization. Unlike traditional methods that rely heavily on manual intervention and expert judgment, the ECS concept utilizes a knowledge-based approach to guide users through the calibration process. By incorporating user input regarding sample properties and application requirements, the ECS generates calibration models tailored to specific analytical needs.

Would modern artificial intelligence (AI) tools, including various new machine learning (ML) tools, offer some remedies for the challenges associated with calibration modeling, transfer, and maintenance? We have begun to ask these questions once again and to explore the options and precautions of moving forward in this direction (2–5). However, what are the key requirements of an ECS that need to be addressed irrespective of the software tools utilized?

Key Components of the ECS

The ECS concept comprises several essential components, including steps for basic spectroscopic model generation and optimization. These steps include: data preprocessing, model building, validation, monitoring, and model transfer optimization (Table I and Figure 1). These components work together to ensure the creation of robust and reliable calibration models (6). Data preprocessing involves various techniques such as spectral correction, baseline correction, and noise reduction to enhance the quality of input data. Model building encompasses the selection of appropriate calibration algorithms, optimization of model parameters, and validation of model performance. In addition, successful model building requires making provisions for seamless calibration transfer across instrumentation platforms. For imaging applications, a similar set of steps is required for creation, analysis, and validation of image generated analytical results (Figure 2). For these processes a set of well-constructed heuristic rules is required for ECS design. Table II shows the basic types of rules required for creating such a software functionality. If AI is involved, such knowledge is also required as input parameters and functional structure for the AI system.

FIGURE 1: Expert calibration system standard calibration modeling workflow (1,6).

FIGURE 2: Expert calibration system imaging calibration modeling workflow (7).

Validation and Monitoring

Validation and monitoring are critical aspects of the calibration process, ensuring the integrity and reliability of calibration models. Validation involves assessing the accuracy and precision of calibration models using independent data sets or reference methods. Monitoring, on the other hand, involves ongoing evaluation of model performance to detect any deviations or drifts over time. By continuously monitoring model performance, the ECS can alert users to potential issues and facilitate timely recalibration or adjustments.

Model Transfer Optimization

Model transfer optimization is essential when deploying calibration models across different instruments or environments. Variations in instrument characteristics or sample matrices may affect the performance of calibration models, necessitating adjustments or fine-tuning. The ECS employs advanced algorithms and decision-making criteria to optimize model transferability, ensuring consistent and reliable results across diverse operational conditions.

Implementation Challenges and Solutions

The development and implementation of the ECS presents several challenges, including data integration, algorithm selection, and user interface design. Addressing these challenges requires a multidisciplinary approach involving expertise in chemometrics, data science, and software engineering. By leveraging state-of-the-art computational techniques and knowledge engineering principles, the ECS can overcome these challenges and deliver robust, user-friendly solutions.

Conclusion

The development of an ECS would represent a significant step forward in process analytical chemistry, offering automated tools and guidance for calibration model development. By streamlining the calibration process and reducing reliance on user expertise, the ECS would enhance the efficiency, reliability, and usability of spectroscopic-based process analyzers. With ongoing advancements in technology and methodology, the ECS holds promise for revolutionizing process analytical chemistry and driving innovation across various industries. The implementation of machine learning and other AI software tools will still require sound experimental design and domain knowledge to create a useful system able to make calibration modeling, calibration transfer, and calibration maintenance more efficient, but also to minimize errors in these processes (2–5). Furthermore, machine learning and deep learning AI programming will be more effective for spectroscopic methods if the spectral data is of high quality and is properly aligned for X- and Y-axis registrations, such that the signals in the spectral data are comparable or standardized.

References

(1) Workman, J.: Kowalski, B.; Mobley, P. The Design of an Expert Calibration System for Spectroscopic Based Process Analytical Chemistry. In Proc. Instrument Society of America (ISA) Conference, Toronto, Canada, 1995, April, 97–106.

(2) Analytically Speaking Podcast Page. Ep. 9: Automating Advanced Chemometric Methods for Data Processing. February 1, 2023. https://www.spectroscopyonline.com/view/ep-9-automating-advanced-chemometric-methods-for-data-processing (accessed 2024-03-08)

(3) Workman, Jr., J; Mark, H. Artificial Intelligence in Analytical Spectroscopy, Part I: Basic Concepts and Discussion. Spectroscopy 2023, 38 (2), 13–22. DOI: 10.56530/spectroscopy.og4284z8

(4) Workman, Jr., J.; Mark, H. Artificial Intelligence in Analytical Spectroscopy, Part II: Examples in Spectroscopy. Spectroscopy 2023, 38 (6), 10–15. DOI: 10.56530/spectroscopy.js8781e3

(5) Workman, Jr., J. An Interview with AI About Its Potential Role in Vibrational and Atomic Spectroscopy, May 17, 2023 Online. https://www.spectroscopyonline.com/view/an-interview-with-ai-about-its-potential-role-in-vibrational-and-atomic-spectroscopy (accessed 2024-03-08)

(6) ASTM E1655-17 Standard Practices for Infrared Multivariate Quantitative Analysis Book of Standards Volume: 03.06, Developed by Subcommittee: E13.11 DOI: 10.1520/E1655-17

(7) Workman, Jr. J. Exploring the Spectrum of Analytical Techniques for Material Characterization. Spectroscopy 2023, 38 (s11), 6–13. DOI: 10.56530/spectroscopy.on6669u1

Jerome Workman, Jr. serves on the Editorial Advisory Board of Spectroscopy and is the Senior Technical Editor for LCGC and Spectroscopy. He is the co-host of the Analytically Speaking podcast and has published multiple reference text volumes, including the three-volume Academic Press Handbook of Organic Compounds, the five-volume The Concise Handbook of Analytical Spectroscopy, the 2nd edition of Practical Guide and Spectral Atlas for Interpretive Near-Infrared Spectroscopy, the 2nd edition of Chemometrics in Spectroscopy, and the 4th edition of The Handbook of Near-Infrared Analysis.

Howard Mark serves on the Editorial Advisory Board of Spectroscopy, and runs a consulting service, Mark Electronics, in Suffern, New York. Direct correspondence to: SpectroscopyEdit@mmhgroup.com ●