Validation of Spectrometry Software - Audit Trails for Spectrometer Software

To understand audit trails within software, I'd like to take an unusual approach and start from a common starting point that should be understandable by all in the paper world: the laboratory notebook. Then this column looks at the translation to an electronic environment, any applicable regulations and guidance, and then addresses how audit trail functionality has been implemented in the software applications for spectrometers. Finally, we'll look at some of my views on how audit trails should be implemented.

R.D. McDowall

The Audit Trail in the Laboratory Notebook

What do we do now? Here's where we go back to paper and the humble laboratory notebook. The laboratory book is issued to an individual, each is uniquely numbered, and the pages are bound to make it secure. Furthermore, the pages are numbered sequentially to ensure the data integrity of the whole book: anyone can tell if pages have been removed simply by looking at the pagination. The book is also a linear record of laboratory activities; convention is that we start at the first page and write up our activities until we complete the book. This provides a time sequence of events as later work appears in the notebook after earlier activities.

Work and observations written up in the notebook must be recorded in ink and not pencil and should be error free — but usually never is. Therefore, errors must be corrected manually by the notebook owner. This takes the form of the writer striking through the error but without obscuring the original entry; then the new entry is made close to the original one and the user then initials and dates the new entry. Typewriter correction fluid must never be used, nor the original entry scribbled through. In some quality systems, the reason for change is also added (for example, transcription error, wrong data entered, or an appropriate entry). This is a useful addition even if not mandated by a regulation because it might indicate that there is a problem with the process for which an individual might need some additional training.

The bottom pages of the laboratory notebook are then signed by the spectroscopist to acknowledge that he or she is responsible for the content of the work and the conclusions reached. A supervisor will then check the work and if the calculations are correct and the conclusions reached are correct, the supervisor signs the bottom of the page to verify this.

All of these mechanisms are designed to ensure the integrity of the work performed.

Conversion to an Electronic Environment

Now comes the interesting bit — how does this linear paper-based approach translate when working in a dimensionless electronic environment? It all depends how the spectrometer software has been designed and implemented.

First, there is the problem that software does not work in a linear way like the laboratory notebook, as data and files usually are created and maintained in either directories containing files or within tables of a database. Data are not stored in a linear manner similar to the notebook entries, and unless you want to get into detail, how are you going to demonstrate that a correct sequence of events has occurred? Typically, here is where the trusty laboratory notebook comes to the rescue. We use the spectrometer and the associated software as a very expensive typewriter and print out the data and stick this into the laboratory notebook. However, this is not helpful when the aim is to work electronically and reduce the amount of paper produced in an analysis.

The audit trail within the software is the key when you are working in an electronic environment. In a way it can be considered as the equivalent of the binding and pagination of the laboratory notebook. The problem is that poor design of many audit trails currently available in spectrometry software will hold back the development of electronic working in many laboratories, as we shall see later.

The Regulations

Before progressing further with this discussion, let us just check and see what the regulations say about audit trails. The main one to consider for our discussion is 21 CFR 11 (1), and §11.10(e) is specific for audit trails:

Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes will not obscure previously recorded information. Such audit trail documentation will be retained for a period at least as long as that required for the subject electronic records and will be available for agency review and copying.

In essence, the regulation requires a computerized system to have an independent monitor of user actions covering the life to death of data and records together with any changes that are made to them by individual users. The audit trail also must be secure and not alterable. This is the equivalent of the explicit and implicit security built into a laboratory notebook.

In 2003, the FDA issued the guidance for industry on Part 11 Scope and Application (2), in which audit trails are one of the areas in which there is enforcement discretion when inspecting companies as follows:

Persons must still comply with all applicable predicate rule requirements related to documentation of, for example, date (for example, § 58.130[e]), time, or sequencing of events, as well as any requirements for ensuring that changes to records do not obscure previous entries.

Even if there are no predicate rule requirements to document (for example, date, time, or sequence of events in a particular instance), it can nonetheless be important to have audit trails or other physical, logical, or procedural security measures in place to ensure the trustworthiness and reliability of the records.

We recommend that you base your decision to apply audit trails, or other appropriate measures, on the need to comply with predicate rule requirements, a justified and documented risk assessment, and a determination of the potential effect on product quality and safety and record integrity. We suggest that you apply appropriate controls based on such an assessment. Audit trails can be particularly appropriate when users are expected to create, modify, or delete regulated records during normal operation.

While on the one hand appearing to let go of the requirement for a system to have an audit trail, on the other hand, the FDA still emphasized the need to have the information available either inside or outside of the software application as well as the need to comply with GMP or GLP. In my view, the enforcement discretion allowed around Part 11 by the Agency does not minimize a laboratory's need to ensure data integrity regardless of how it is achieved: by the software, by a handwritten log, or by a combination of the two approaches.

Some Current Audit Trail Implementations

Let's start from what audit trail options we currently have in spectrometry software. The first is fairly brief — nothing! This is not a useful option.

Spectral Data File Contains the Audit Trail: One of the common implementations of an audit trail in spectrometer software is the incorporation of the functionality into the data file created during an analysis. This is fine for the data generation and the modification but is not compliant with Part 11 because when the file is deleted, there is no record since the file recording the deletion is no longer in existence (in other words, the file cannot record itself being deleted). This is a problem with software using directory structures versus a database. In a software application with a database, there should be a separate table for recording all audit trail entries associated with creation, modification, and deletion of data and records.

Editable Audit Trail Entries: One of the Part 11 requirements is that the audit trail should be secure. Therefore, when assessing some spectrometer software applications, it is amazing to find that some audit trail entries can be opened and edited using text editors. Obviously, this is a new definition of secure that I was not aware of. The audit trail needs to be viewable but not changeable. Some nonspectroscopic software will checksum or encrypt the audit trail entries so that they cannot be changed.

Single Audit Trail: There is usually a single audit trail within a spectrometer system. This is fine, but the audit trail usually covers all the functions of the software such as user account management; user logins; creation, modification, and deletion of new methods; and creation, modification, and deletion of data. Everything is thrown in here — if you are looking at a specific item is there a search facility?

Searchable Audit Trail: This is a feature that might be lacking in some software applications. There must be a function to search the information about a specific analysis within all the audit entries. This should be an essential feature for any audit trail.

Understandable Audit Trail Entries: Once you have searched the audit trail and found entries associated with your query, it would be reassuring to know that the entries are easy to read and understandable. Ideally, the user's real name (Bob McDowall) should be entered into the audit trail rather than the user log on identity (FM8120BM) so that you can easily identify the individual who carried out an action .

If the reason for change is needed, there should be an option of a drop-down list of predefined reasons for change. This would speed up data entry and avoid typographical errors and make it easy to review the entries. In addition, the original result and the change should be present explicitly in the audit trail rather than requiring a user to guess what happened.

The aim here is for the audit trail to help reconstitute the analysis changes and be unambiguous.

Archive and Restore: Before we start, let's not confuse backup and recovery, which is short-term recovery from disaster, with archive and restore of data. This is the removal from the system of work that is completed along with the associated audit trail entries. You might be able to archive the data OK, but the problem is if you want to restore them, the data can go back into the system OK but often, the restored audit trail information cannot be inserted back into the existing audit trail. This can make it difficult to reinterpret the data.

Can We Do Better?

Of course we can. Perhaps the best way to look at ways to ensure the quality and integrity of spectrometry data is to go back to basics and consider the design of the system and the audit trail in particular. In my view, there should be more than one audit trail in a system. Here are some of my thoughts about this.

System Audit Trail: This would audit the whole system and include user account management (creation and modification of user types and user accounts), users logging on to the system, as well as security alerts. The creation and archiving of the work packages within the application would be covered by this audit trail as well as the copying of methods and reports between work packages. Set up of spectral libraries also would be covered by the system audit trail.

"Work Package" Audit Trail: Depending on where you work within an organization, your view of a work package will be different. In R&D, your view might be project- or program-based in comparison to a production view of batches. Regardless of the approach, each work package created in the system should have an individual audit trail. This would cover the setup and establishment of methods for analysis, plus the acquisition, analysis, calculation, and reporting of data. The advantage of this approach is that all the audit trail entries are associated intimately with the data they relate to. Searches of the audit trail will be more specific and quicker than from a single one.

Any Data Changes?: We could go further if the design of the software is well thought through. Rather than trudge through a badly designed audit trail, perhaps we can have simple visual signs for a supervisor that manual changes have been made to data to help them focus their effort where it is best needed?

Summary

I have looked at the way that an audit trail is implemented in many spectrometer software applications and made suggestions for improvements to aid electronic working. Unless there are effective audit trails to help scientists using spectrometer systems, we cannot work electronically and are stuck with paper as the archive medium.

R.D. McDowall is principal of McDowall Consulting and director of R.D. McDowall Limited, and "Questions of Quality" column editor for LCGC Europe, Spectroscopy's sister magazine. Address correspondence to him at 73 Murray Avenue, Bromley, Kent, BR1 3DJ, UK.

References

(1) 21 CFR 11, Electronic Records; Electronic Signatures Final Rule, 1997.

(2) Guidance for Industry, Part 11 Scope and Application, FDA, Rockville, Maryland, 2003.