Why Is My Application’s Audit Trail Rubbish?

It has been 20 years since 21 CFR 11 became effective-and with increased emphasis on data integrity, an audit trail is essential for spectroscopy software. Why then do many audit trails fail to meet regulatory requirements?

Time flies when you are having fun. Blink and you could have missed the 20th anniversary of 21 CFR 11 regulation on electronic records and electronic signatures becoming law (1). The regulation was requested by the pharmaceutical industry in 1990 to streamline production by eliminating paper and using electronic signatures. The United States Food and Drug Administration (US FDA), however, turned Part 11 into an electronic records and electronic signatures regulation. Since 2002, there have been virtually no citations against the regulation because the underlying predicate rule (good manufacturing practice [GMP] or good laboratory practice [GLP]) has been cited instead. Part 11 appears to have sunk without trace and one could reasonably question the point of the regulation. Apparently, the sole function of the regulation these days is to send questionnaires to software suppliers to ask if they have a Part 11 technically compliant audit trail in their applications (but most laboratories rarely investigate the response).

It is my contention that many of the audit trails in software applications used in regulated laboratories today fail to meet regulatory expectations. Let's explore why in this column.

Unable Able Laboratories

Audit trails were brought into sharp focus in 2005 with the Able Laboratories fraud case. Able was a generic pharmaceutical company based in New Jersey who used some "innovative" analytical techniques to pass batches of product such as copy and paste, manipulation of sample weights and standard weights, dilutions, and purity factors as well as dramatic repositioning of baselines in chromatograms. The FDA had inspected the company several times and had not picked up anything suspicious because the inspectors only focused on paper records and reports. The FDA only knew about the problems in Able from a whistle-blower inside the company.

Citation 1 of the Able Laboratories 483 form states the following (2): "The Quality Unit failed to: review electronic data as part of batch release, review computer audit trails in the <Redacted> Data Acquisition System and provide adequate training to analytical chemists."

The underlying GMP regulation 21 CFR 211.194(a)(8) for second person review of laboratory data was used to justify this citation (3).

EU GMP Annex 11 Update

Able Laboratories was an input to the revision of Clause 9 on audit trails when European Union (EU) GMP Annex 11 on computerized systems was updated in 2011 (4):

Consideration should be given, based on a risk assessment, to building into the system the creation of a record of all GMP-relevant changes and deletions (a system generated "audit trail").

For change or deletion of GMP-relevant data the reason should be documented.

Audit trails need to be available and convertible to a generally intelligible form and regularly reviewed.

The Annex 11 audit trail requirement is slightly different in that it focuses on data modifications and deletions rather than the creation, modification, and deletion that is the requirement of 21 CFR 11. However, there is a further Annex 11 requirement in clause 12.4 that covers data creation: "Management systems for data and for documents should be designed to record the identity of operators entering, changing, confirming or deleting data including date and time."

Section 12.4 applies to all computerized systems used in a regulated environment even if there is no audit trail in the application.

The last element of our Annex 11 discussion is contained in clause 1, which is focused on risk management. It states, "Risk management should be applied throughout the lifecycle of the computerized system taking into account patient safety, data integrity and product quality . . . ."

In addition, the documentation requirements for an audit trail are contained in Chapter 4 on documentation clauses 4.7–4.9 inclusive (5). This is particularly important as the reason for change which we will discuss later.

What Is Wrong with Audit Trails?

The remainder of this column looks at some of the issues of why audit trails are poorly implemented in laboratory applications. There are several reasons for this situation as we shall see.

Pharma Is Only a Small Business for Us

The excuse that some software companies use for not implementing technical controls in their software is that pharmaceutical companies only represent a small portion of their business and it is not worth their while implementing such functions. This is rubbish. Because the pharmaceutical industry is regulated, software must comply with these regulations. It raises two questions:

• Why is the company selling to the pharmaceutical industry in the first place?

• Why did the laboratory buy the product?

Let's focus on the second question. It might be that the software and instrument are essential for the laboratory analysis. This still raises another question: Did the laboratory evaluate the software for technical compliance with the regulations? If not, the laboratory has made a regulatory rod to whip their own back because procedural controls and work arounds will raise the cost to implement and maintain a system. The current view of regulatory authorities is that technical controls are far superior to procedural ones, and more difficult to get around to falsify data. Therefore, if this system is preferred, then at least get a written contract with the company to develop these controls by a specified date. It is always better if there are monetary penalty clauses for noncompliance in this contract. Consider the entire cost of qualification and validation plus added procedures and training to manage integrity gaps in the system design to understand the real cost of a system. These costs are usually hidden from view. After these procedural costs are added, more expensive, technically capable systems begin to show their worth.

How Much Regulatory Knowledge?

One key failing in suppliers offering spectroscopy software is the level of regulatory knowledge in the company. It is important to realize that both EU Annex 11 and CFR Part 11 need other regulations to fully understand how they work as we shall see:

• 21 CFR 11 contains the requirements for managing electronic records and electronic signatures, but it does not say which records and signatures are needed. That is the role of the predicate rule, which is a legal term meaning pre-existing. The applicable predicate rules are 21 CFR 211 for GMP (6) and 21 CFR 58 for GLP (7). These regulations may have differing requirements-for example, the former regulation does not require a reason for a change but the latter one does. How would a supplier implement software to accommodate these differences?

• Annex 11 must be interpreted by clauses 4.7–4.9 of EU GMP Chapter 4 on documentation (5). These clauses set out clearly what need to be done for compliant documentation and include a reason for change when correcting errors, and so forth.

In a recent software audit that I performed, the company had brought in consultants to train them. The training slides were simply a copy and paste of the regulation with no interpretation for software, and there was no check of the understanding of the trainees, which did not instill confidence in the company or the application. Therefore, the regulatory knowledge of the supplier is critical to implementing compliant software including audit trails.

The Audit Trail Was Turned Off

The biggest design flaw of a regulated application is the ability for users to turn the audit trail function on or off. This is a frequent citation in many data integrity warning letters. The easy solution is that the audit trails within an application must be functional from initial installation and cannot be turned off. The audit trail would operate silently recording who did what and when. The only user option would be to configure the audit trail to have a reason for change.

At this point many readers working in research would ask, why do we need an audit trail? We don't work to GMP. The answer is that sometimes research data can end up in a regulatory submission to the FDA, and these data are subject to Part 11 regardless of whether there is a regulation or not as stated in 21 CFR 11.3 (1).

The Audit Trail Is An Integral Part of the Application

The audit trail must be designed into the software application, not added on as an afterthought. A database is the only way to ensure a secure and robust audit trail that cannot be tampered with. In the early days of Part 11, some audit trails were simply text files or the audit trail was added into the data system file. The latter point is interesting: If a user deletes a file, how can the audit trail monitor its own deletion? The simple answer is that it can't. This is one of the reasons why you need a database. The audit trail table can be hidden in the database and entries can be check summed or encrypted so that they cannot be modified by users.

How Many Audit Trails?

There are two main design philosophies for audit trails in a regulated application:

• One large audit trail covering all entries from log on or off to data modification. This approach requires very effective search routines to be able to pull all entries related to a specific analysis and then mark them as reviewed.

• Two audit trails. One at the application level dealing with system-level activities and one at the work package level so that each analysis has the relevant entries directly and close with the analytical work. Search routines will still be required, by the way.

As long as you can retrieve the entries to your analysis easily, either way would be acceptable. As we shall see later, the need to identify and mark data if there are any changes to ease review probably means the two-audit-trail approach is better.

Coverage of System Actions

It is important to realize that an audit trail should not just record changes or deletions of records, but should also document key activities within the software (such as configuration of the software, account management, and so on) as well as documenting that work has occurred (for example, file creation and interpretation of data). As the UK Medicines and Healthcare Products Regulatory Agency (MHRA) data integrity guidance notes, "The items included in audit trail should be those of relevance to permit reconstruction of the process or activity" (8).

This process is important to make the second-person review easier and transparent, as well as audits and inspections, by implication. In addition, there is the requirement in EU GMP Chapter 1.9 for the work to have taken place (9) and tasks such as data creation also need to be documented to comply with Annex 11 section 12.4 (4).

Are Entries Intelligible and Understandable?

From the Chapter 4 clauses 4.7–4.9 (5), the requirements for each audit trail entry include the following:

• Date and time stamp: The format of this stamp must be unambiguous and many also require the time zone, especially for multinational companies.

• User name: The name of the user performing an action.

• User identity (optional): Some audit trails take the user identity only, which is not ideal because an auditor or inspector will need to see the list of user identities to identify a specific individual.

• Entries in the audit trail must be intelligible as well as understandable: plain text about the action, not system messages. The latter are typically a result of when a developer does not understand the regulatory requirements.

• The old and new value for a change must be recorded.

• A reason for changing data should be available: Entries can be free text or context-sensitive user-defined text. The latter is preferred for consistency but can take time for the laboratory to develop.

• Audit trails need to be associated with the activities supported, and it must be possible to search the entries for specific events.

• Audit trails must be secure from change.

• Audit trails must be retained for the record retention period. If the data are archived and retrieved then so too must the audit trail entries.

• An export of audit trail entries should be possible for inspections but also data integrity audits and investigations to try to identify actions contributing to poor data management practices.

Review by Exception

Are all budding Indiana Joneses still reading this overview and looking forward to digging into all those interesting audit trail entries? For some systems, the only way to review audit trail records is to dig through mounds of entries looking for the odd gem. Is this process the most soporific and error-prone activity in the laboratory? No, wait, that's statistical analysis, but audit trail review comes in at a close second!

Here's where we need the suppliers to come to our aid. To simplify an audit trail review, we need the application to identify where there has been data modification or deletion. It will know because there will be an audit trail entry, so it should be easy for the application to highlight such activity. Some applications add annotations to identify the type of change or use colors, such as green no changes, yellow a modification, and red a deletion. This is OK provided users are not color blind.

Why review audit trail entries if there have been no changes? The aim here is to use Clause 1 of EU GMP Annex 11-apply risk management throughout the life cycle (4). This step is where you can apply risk management, but only if this functionality has been validated and shown to work correctly.

Review of audit trail entries by exception also needs to be balanced by the need to consider the second person review and the risk of data falsification:

• Has the analysis been carried out correctly, with, for example, consistent date and time stamps on records?

• Have the data been interpreted correctly according to sound science and applicable procedures?

• For critical applications, instead of review by exception, perhaps a sampling process of audit trail entries could be adopted.

However, many applications we use in the laboratory today are simply not up to the audit trail job-20 years after the publication of 21 CFR 11. Why is this the case? Probably because users only focus on the analytical functions of software and not the compliance features. This is wrong. Also, many laboratories still operate applications that are many years old and have not been upgraded because of cost or low usage; these applications will have poor audit trail functionality.

In 2015 the UK regulatory, the MHRA, noted in their data integrity guidance document (8) that

If no audit trailed system exists a paper based audit trail to demonstrate changes to data will be permitted until a fully audit trailed (integrated system or independent audit software using a validated interface) system becomes available. These hybrid systems are currently permitted, where they achieve equivalence to integrated audit trail described in Annex 11 of the GMP Guide. If such equivalence cannot be demonstrated, it is expected that facilities should upgrade to an audit trailed system by the end of 2017.

The problem is that many of the applications that they may upgrade to have audit trails may not be compliant with the regulations.

Documenting the Review

Ideally, there needs to be function associated with the audit trail to record electronically that pertinent audit trail entries associated with an analysis have been reviewed. This function is not possible in most spectroscopic applications now and procedural controls need to be used instead. Typically, the second-person review procedure will state that the signature of the reviewer means that appropriate audit trail entries have been reviewed and that this situation should only be a stop-gap until a technical control is available.

E-Compliance Requirements Initiative

In March 2017, an initiative on "e-Compliance Requirements" was launched to help the regulated industry adequately define a set of e-compliance requirements for various types of computerized systems, including audit trails. The objective is to define requirements in a unified way to provide clear compliance expectations for system suppliers and developers. The e-Compliance Requirements Initiative (eCRI) is an international initiative, and the organization can be contacted at ecri@kereon.ch or ecri.kereon.ch.

Conclusions

The aim of this column has been to highlight the poor state of audit trails in spectroscopy software used in regulated laboratories. The goal here is to educate both users and suppliers that better audit trails are required to meet regulations.

Acknowledgments

I would like to thank Yves Samson and Mark Newton for comments made reviewing this column.

References

(1) Code of Federal Regulations (CFR), 21 CFR 11 "Electronic records" (Electronic signatures, final rule, in Title 21 1997, Food and Drug Administration, Washington, D.C.).

(2) Able Laboratories Form 483 Observations, 2005, available from: http://www.fda.gov/downloads/aboutfda/centersoffices/officeofglobalregulatoryoperationsandpolicy/ora/oraelectronicreadingroom/ucm061818.pdf.

(3) Code of Federal Regulations (CFR), 21 CFR 211 "Current Good Manufacturing Practice for Finished Pharmaceutical Products" (Food and Drug Administration, Silver Spring, Maryland, 2008).

(4) European Commission Health and Consumers Directorate-General, EudraLex: The Rules Governing Medicinal Products in the European Union. Volume 4, Good Manufacturing Practice Medicinal Products for Human and Veterinary Use. Annex 11: Computerised Systems (Brussels, Belgium, 2011).

(5) European Commission Health and Consumers Directorate-General, EudraLex: The Rules Governing Medicinal Products in the European Union. Volume 4, Good Manufacturing Practice Medicinal Products for Human and Veterinary Use.Chapter 4 Documentation, (Brussels, Belgium, 2011).

(6) FDA, "Amendments to the Current Good Manufacturing Practice Regulations for Finished Pharmaceuticals," Federal Register73(174), 51919–51933 (2008).

(7) Code of Federal Regulations (CFR), 21 CFR 58 "Good Laboratory Practice for Non-Clinical Laboratory Studies" (Food and Drug Administration, Washington, D.C., 1978).

(8) MHRA GMP Data Integrity Definitions and Guidance for Industry, 2nd Edition (Medicines and Healthcare products Regulatory Agency, London, 2015).

(9) European Commission Health and Consumers Directorate-General, EudraLex: The Rules Governing Medicinal Products in the European Union. Volume 4, Good Manufacturing Practice Medicinal Products for Human and Veterinary Use. Chapter 1 Pharmaceutical Quality System. (Brussels, Belgium, 2013).

R.D. McDowall

R.D. McDowall, is the director of R.D. McDowall Limited, as well as the editor of the "Questions of Quality" column for LCGC Europe, Spectroscopy's sister magazine. Direct correspondence to: SpectroscopyEdit@UBM.com