Dear Esteemed Vendor?

We have all had them — letters from a vendor of a product you are using in your laboratory that begin with the dreaded words "Dear Esteemed Customer ...." Reading those three words, you know what's coming: a price increase, your instrument will not longer be supported, your software is obsolete, the company has been taken over, or most of the above items. So what's with the title of this column, you may ask?

Compliance with regulations or standards is an important part of my consultancy and on a few recent projects, I have had the pleasure of reviewing many examples of vendor qualification and validation documentation on behalf of clients. You might be interested to know that lifting some examples of this material is better than going to the gym. Why bother pumping iron when I can pump paper instead? However, reviewing many of these documents over a relatively short period of time, I was surprised with the following:

• The poor quality of many of these documents including the lack of scientifically sound approaches.

• The misunderstanding or misinterpretation by some vendors of what the regulations actually require and, in the worst cases, providing their customers with regulatory noncompliances to explain to an inspector — for example, failure to allow users to review and approval documents before execution.

• Acceptance by laboratory staff of these documents without any critical review (it's the "if it's been written by the vendor it must be good mustn't it" syndrome).

So this column has two objectives:

• First, to present my feedback to vendors about the quality of their documentation that they foist upon their customers, who appear to accept anything without much scrutiny or critical review. And to reciprocate all those "Dear Esteemed Customer" letters I have received over the years, now it's my turn and so welcome to "Dear Esteemed Vendor."

• Second, to educate spectroscopists and analytical chemists so that they do not accept a document at face value. Instead, they should apply their knowledge and have the confidence to be able to critically review these materials. That way they can decide if they are being given value for money or being sold, what can be called politely, agricultural fertilizer.

So we will discuss if vendor documentation for instrument qualification or computer validation meets the GXP regulations and is scientifically sound. Not prejudging the issue (very much), I will propose suggestions for you to evaluate such documentation and for improving vendor qualification/validation material. I will take this discussion wider than just spectrophotometers and also include some other instruments and systems such as balances, centrifuges, and high performance liquid chromatography (HPLC) systems that are commonly used in a laboratory.

I want to make it clear that not all vendor qualification and validation documents are poor as there are some exceptions but unfortunately not many. Regrettably, there are many more examples which require improvements to provide customers with value for money coupled with adequate regulatory compliance, rather than generate a pile of paper simply by ticking a box. Looked at from the perspective of a regulated laboratory user, it is unlikely that you will have a single vendor for all your instrumentation, therefore you will have multiple vendors providing you with documentation for installation qualification and possibly operational qualification. So this is not just a single source problem to manage but it is a multivendor and multidocument problem to manage. From my perspective a single vendor can have a few good documents and then some poor ones so beware after a baseline has been established as that is when a poor document could catch you out.

The Dawn of Realization ...

Over the past 20 years, it has dawned both on analytical scientists working in regulated laboratories and the regulators themselves that analytical instruments and computerized systems need to be fit for their intended use and that there must be documented evidence available to demonstrate this. This started in the early 1990s; the publication by Furman and colleagues (1) presented the FDA's viewpoint with the concept of modular and holistic qualification, Freeman and colleagues (2) from the Pharmaceutical Analytical Science Group (PASG) gave the UK pharmaceutical industry approach with the introduction of the 4Qs model to analytical instrumentation, and Burgess and colleagues (3) looked at some of the practical issues involved with qualifying instrumentation in regulated laboratories. The UK Laboratory of the Government Chemist (LGC) developed an approach for qualification of analytical instruments (4) and then two guidance documents for HPLC systems and UV–vis spectrophotometers in the late 1990s under the Valid Analytical Measurement (VAM) program (5,6).

However, many of the early attempts at analytical instrument qualification (AIQ) were conducted in-house but were often time consuming. So laboratories often would ask the vendors of their instruments if they could help. In principle this is a great idea, as the vendor knows their instrument, and therefore, they would be well placed to help qualify the instrument and to validate the computer system that controls it. The theory is great but the practice is often a let-down. Vendors typically have a wide market and GLP and GMP compliance is in a relatively narrow industry sector, such as the pharmaceutical or biotechnology sector. By sales, regulated laboratories can be a large market but it is small compared to the needs of the overall market requirements, from research to manufacturing, from petrochemicals to testing, from no-quality system to ISO 17025 and then to GMP/GLP. Furthermore, most vendors are certified to ISO 9001 for the delivery and support of a product or a service which can mean that documents written for compliance with ISO might not be compliant with GXP regulations. So unless a vendor specializes in this specific market sector, they can be at a loss as to understand what is required for a regulated laboratory especially as the regulatory framework has changed over the past ten years at an ever-increasing speed.

... Brings a Market Opportunity!

Over time, laboratories have realized the burden, both in terms of staff time and lost work, involved in the qualification of instruments and validation of the computerized systems is too great and so various ways have been tried to reduce the impact. The development of in-house groups that will focus on qualification and validation to the laboratories (these can often be an outgrowth of existing metrology groups) was one way of helping. However, these units are often vulnerable during reorganization or merger as they are not seen as a core operation to the laboratory. This is unfortunate, as it removes from the organization the very skills required to assess and critique vendor documentation.

The other way, providing there is a suitable budget, is to outsource to a third party such as the vendor or to a company dedicated to offering qualification and validation services for all vendor instrumentation. For the purposes of this article, I'll not consider directly the qualification services company as I wish to focus solely on vendor products and services. However, many of the principles discussed here apply equally to all documentation for qualification and validation regardless of the origin of the material.

Vendor services to the regulated laboratory can be very lucrative, especially when times are tough, as instruments still need to be qualified initially and thereafter on a regular basis and after a major repair. So if the instrument market is down, qualification documentation coupled with delivery by professional services or the field engineers can provide a good source of revenue for an instrument company.

Roles and Responsibilities ...

It is important to understand the roles and responsibilities involved in this area. We typically have three roles to consider in qualification and validation: the system owner, quality assurance, and the vendor.

• System Owner, typically a senior scientist who is in charge of the system in a regulated laboratory, is responsible and accountable for all qualification and validation activities carried out on the system to ensure that it is fit for purpose. The work can include qualification of an instrument, validation of the software, or both. If a portion of this work is outsourced to a third party, in this case a vendor, the vendor is responsible for carrying out the work to an agreed standard but the system owner remains accountable.

• Quality Assurance, responsible for ensuring compliance of the work and documentation with the regulations and company procedures. In Europe, there is a split documented in the regulations between QA and QC (7) that is not reflected in the U.S. GMP regulations (the Quality Control Unit or in later documents just the Quality Unit).

• Vendor who will be carrying out installation qualification and operational qualification using their knowledge of the instrument and software to install and demonstrate that the system works as expected. They will provide the documentation an the personnel to perform the qualification work.

In addition, there are two other roles with the following responsibilities:

• Laboratory Management must ensure that resources are available to qualify and validate instruments and systems as well as maintain the systems in compliance with the regulations. Resources can include time, budget, staffing, and so forth. This is consistent with the requirements of ICH Q10 on Pharmaceutical Quality Systems (8) and reflects one of the major changes in the emphasis of the GMP regulations recently.

• Information Technology will also be involved if the laboratory system is to be connected to the network and can be involved if they have to supply the PC that the software will be installed on.

... and Regulations

The basic problem is that many vendors think that they know the GXP regulations but given the evidence that I have seen when reading their documentation, this is not true. So let us look at the GMP regulations.

In Europe, we have the GMP regulations divided into three parts plus annexes to cover specific subject areas. Part 1 covers manufacturing of products, Part 2 is GMP for active pharmaceutical ingredients and is simply ICH Q7, and Part 3 is the draft section on the Site Master File. In addition, there are 20 annexes, of which the two main ones to consider are annex 11 on computerized system validation and annex 15. It is annex 15 (9), entitled qualification and validation, that is critical to this discussion, as the vendors do not appear to have read this part of EU GMP, which has been effective since September 2001. The main clauses for this discussion are presented in Table I but please take the time to read the full annex.

Table I: A selection of regulations for qualification and validation documentation

Similarly, there are the U.S. GMP regulations (10) that require scientifically sound work and a document for the work being carried out under the laboratory controls sub-part of these regulations. The major sections of the U.S. GMP regulations for this discussion are presented in Table I.

So let U.S. summarize the explicit requirements from the regulations that we need to consider further in this column. Qualification and validation documentation must meet the following:

• We need a written protocol or document that tests the instrument or system against predefined specifications (EU and U.S.).

• This protocol must be reviewed and approved by the Quality Control unit in the U.S. or QA in Europe (in practice it will be QA in both regions) (EU and U.S.).

• The specifications and work carried out both must be scientifically sound (EU and U.S.).

• At the end of the work, there must be a report and release statement that is reviewed and approved before the next stage of work is carried out (EU).

These requirements are not the basis for negotiation, they are the law. All regulated GMP laboratories must follow them. If vendors are used they also must follow them, but, as we shall see, they rarely do, either by design or ignorance. In this the vendors are aided and abetted by naïve laboratory staff and management who also fail to appreciate some of the legal requirements of their work.

At this point, we have not looked at 21 CFR 11 on electronic records and electronic signatures (11), however, I will return to this later in this column. I also will discuss one or two aspects of USP <1058> on analytical instrument qualification (AIQ) (12) in this column but you will need to consider the whole of this general chapter when planning any work along with any other ones applicable to the instrument you want to qualify; for example, <41> for balances, <1119> NIR, <621> chromatography, and so forth. The next "Focus on Quality" column will look at USP <1058> as it is now two years since it has been effective. It is also important to realize that there is no equivalent chapter to <1058> in the European Pharmacopoeia.

The 4Qs Model and the Role of a Vendor

The 4Qs model is the one that is outlined in USP <1058> (12). For those of you not familiar with the general chapter, I'll outline the four stages of the model.

• Design qualification (DQ) is the documented collection of activities that define the functional and operational specifications of the instrument and criteria for selection of the vendor, based upon the intended purpose of the instrument.

• Installation qualification (IQ) is the documented collection of activities necessary to establish that an instrument is delivered as designed and specified, and is properly installed in the selected environment, and that this environment is suitable for the instrument.

• Operational qualification (OQ) is the documented collection of activities necessary to demonstrate that an instrument will function according to its operational specification in the selected environment

• Performance qualification (PQ) is the documented collection of activities necessary to demonstrate that an instrument consistently performs according to the specifications defined by the user, and is appropriate for the intended use.

The main area of the 4Qs model where a vendor can be most useful is the IQ installing their own instrument and checking it out to see it works as a system. However, as part of the outsourcing model a laboratory also can rely upon a vendor's specification for the DQ and typically will also use them for the OQ. A vendor OQ is only useful if the tests conducted match the laboratory requirements, a subject we shall return to later in this column.

A question now arises: How good is a vendor specification?

Does Design Qualification = Vendor Specification?

Of the four stages of the 4Qs model, the one that is performed the worst or indeed, is not performed at all, is the design qualification (DQ). The definition of DQ from earlier is "Is the documented collection of activities . . . ." Note the word activities, not activity. There is more than one in this process! However, in many laboratories, the approach is to use the manufacturer's specification. For simple instrumentation, this might be acceptable. The use of the vendor specification is enshrined in USP <1058> as being the "right" thing to do; however, read the next few paragraphs before you agree. Failure at later stages of the 4Qs model can be attributable directly to a failure to undertake an adequate DQ. So ask yourself a simple question: Have you ever purchased an instrument or system that failed to do its job? I have and so has virtually every other senior analyst. When you analyze the reason for this, it is down to the failure to specify exactly what you wanted or that you were too naïve to write a specification.

So Let's Ask the Question: A Vendor Specification Is Good, Right?

Wrong!

Unfortunately, this can be a misconception that you can easily and often fall into. It is also the same problem with USP <1058> that relies upon using a vendor specification for an apparatus in group A and instruments in group B. Does this help? In many situations, the answer might be yes but you need to check the specification and how it was measured. This is best illustrated by the example of a benchtop centrifuge that my colleagues and I were asked to qualify as part of a laboratory AIQ project. The centrifuge had a single-speed rotor with a timer that a user could set to determine how long the samples would be centrifuged. The manufacturer's specification for the rotor speed was 3500 ± 1 rpm. This is very tight and precise specification. When we asked the vendor how this was measured, we were informed that it was the pulse train from the stepper motor without a rotor attached — the rotor speed was not even measured directly! Let me ask you another simple question: How many of you centrifuge samples with no rotor? I suggest that in this case, the vendor's specification is useless and meaningless and these are the politest comments that the Editor will allow me.

Therefore, YOU need to define the intended purpose of the instrument or system you are going to purchase by defining your key operational parameters to enable it to do its job. Then compare these with the vendor's specification. Do not start from the vendor specification and work backwards. Also, forget USP <1058> about using a vendor specification, as this is simply the regulated industry being dumb and stupid. There is only one person responsible for defining the requirements for an instrument (group B) or a system (group C) and that person is currently reading this article. Aren't you?

Another thing you should consider about vendor specifications is that instrument specifications are, not surprisingly, designed to sell instruments and to demonstrate that vendor A's instrument is better than all other vendors. It is also, in my view, a damage limitation exercise in that if the instrument does not work, here's the specification to demonstrate that it does. However, as the centrifuge example from earlier shows, unless you have the test method associated with each specification parameter, then each parameter listed by a vendor is meaningless.

To be balanced, we also should consider the question from the vendor perspective. They do not work to GMP but comply with ISO 9001: 2008. Internal documentation written by their engineers is to comply with ISO 9001 accreditation and not GMP regulations. Furthermore, these documents typically are not written or intended for customer use but industry wants to reduce time and effort associated with instrument qualification.

So what should a user or system owner do? First, write your own requirements for key instruments and systems. Second, remember that the definition earlier that states that DQ is not simply a single activity. Therefore, I strongly suggest for larger instruments (group B) and complex systems (group C) that the vendor signs off the design qualification (typically the URS or a variant thereof) to state that the system to be purchased will perform all the listed functions. This simple process will save you a lot of agony and will prevent you having to find more space in that dark, dark cupboard in the depths of your laboratory where you consign unwanted instruments and systems (13). You might think this a waste of time but my colleague was working with one of our clients on the purchase of a NIR spectrometer. He developed a user requirements specification (URS) based upon the intended use of the system and the selected vendor stated that the system would meet that specification. Instead of saying OK, accepting the statement, and doing nothing, he requested that the vendor signed the URS to this effect. When the system was delivered, it did not meet the URS requirements. What happened? The vendor was contractually required to develop software to meet the specification. Initially, this was a custom macro but this would be included in the next release of the NIR software and therefore, would go from category 5 to category 3 software. Caveat emptor!

Does a Vendor Understand the Regulations?

This is a two-edged sword and to be fair to vendors, they have a balancing act to perform. First, a vendor will have many customers working to GMP regulations and while there will be a broad consensus of what is meant by a specific regulation, there will be many nuances when you come to the detail of implementation. These nuances will be influenced by the organization's overall view of compliance and if the regulators have made adverse comments about their compliance practices in previous inspections. Alternatively, a vendor might have worked extensively with just one organization to develop the initial product, and this will have a single company's interpretation of the regulations rather than a broad spectrum approach. Also, vendors will learn from their customers about regulatory compliance or they might be large enough and have enough sales into regulated healthcare laboratories to justify a compliance post and have the expertise in-house.

Leading on from this is: How does a vendor understand and interpret regulations? Here is one example where the interpretation is way off the mark about 21 CFR 11, and I quote verbatim from the vendor documentation:

• FDA defines a closed system as "An environment where there is communication among multiple persons, where system access is restricted to people who are part of the organization that operates the system."

Is the vendor right? Of course not, but we need to demonstrate this conclusively. Some of you might know one of the corollaries of Murphy's Law, Cahn's Axiom, which simply states "when all else fails, read the manual," or in our case, the regulation:

• Looking in 21 CFR 11 Sub-part A Definitions and you'll find definition 4: Closed system means an environment in which system access is controlled by persons who are responsible for the content of electronic records that are on the system (11).

Let us compare and contrast the vendor and the FDA definitions:

• The vendor definition is plainly wrong and is not the FDA definition from Part 11 — how, I cannot explain. My version of the FDA definition was obtained by that leading-edge technology of copy and paste from the Agency website. Furthermore, the vendor definition is nowhere close to being correct. Quoting verbatim from that well known spectroscopist, Homer Simpson, "D'oh!"

• The vendor interpretation is also wrong as you can have a closed system where access by people outside the organization is allowed as long as it is controlled by the laboratory or organization staff. A typical example would be by a vendor's service engineer, where temporary access to perform maintenance or qualification is granted by laboratory staff. You would have thought that a vendor would know all about that, wouldn't you? Double d'oh!

So if you want to outsource, you need to check the documentation you are getting before you part with any money. Are you getting quality documentation or simply agricultural fertilizer?

Accepting the "Standard" Vendor IQ and OQ Approach

When a laboratory purchases an instrument or laboratory computerized system, the salesperson is always ready to sell additional goodies such as qualification services. You can imagine the scene as he or she slithers across the floor toward you: "We have IQ and OQ materials for you to help you get the system operational quickly." Mesmerized like a deer in the headlights of a car, a grateful user rapidly adds the standard packages to the quote and everything goes through the company sales process. This is followed by delivery and qualification of the system.

Fast forward a few months and the door to the laboratory bursts opens and in walks the validation police: It's the corporate quality assurance audit! As a fully paid-up member of the validation police (sorry, I meant to say that I'm also an auditor), when I am reviewing a standard vendor qualification protocol in a laboratory, I will identify operating ranges for some of the key parameters from the document. Then I'll ask the system owner if there are methods that go outside of these ranges. For example:

• If the wavelength accuracy is calibrated using a holmium solution, does the laboratory measure samples below 240 nm? Usually, I find that the laboratory may be measuring down to 210 nm, which is outside of the qualified range of the spectrophotometer or HPLC detector.

• If a flow rate for an HPLC pump has been qualified over a range of 1.0–2.0 mL/min, how many analytical procedures use flow rates outside of this range? Quite a large number in some cases is the truth.

• Some qualification protocols might only make a single measurement; for example, the oven temperature of an HPLC column compartment may be measured at 40 °C. This is fine as long as the laboratory only uses this temperature. What happens if you use the column compartment outside of this value? You are not covered by the qualification is the answer, so why do vendors do this? More importantly, why do dumb and stupid users accept this rubbish?

So the information that the laboratory needs and the questions to be asked are before you purchase the documentation are

• What are your operating ranges (defined in your user requirements specification) and how do these compare with the vendor's standard qualification package?

• If your operating range is outside of the standard package, what are you or the vendor going to do about it? You will either have to do additional work yourself or get the vendor to do it on your behalf.

• Interpolation rather than extrapolation is the key requirement here. It is defensible to have wider qualified operating ranges rather than operate outside of them. Never extrapolate.

Is the Work Scientifically Sound?

Now here's where you have to review the vendor qualification material critically to see if it meets the regulatory requirement of scientifically soundness. Let us take a look at two examples that are not, in my view, scientifically sound approaches:

• Qualification of an HPLC column compartment is scientifically unsound. Apart from the single temperature determination as discussed earlier, the instructions for executing the test in the same document state that the temperature probes must be placed on the heat exchangers in the column compartment. These are the two solid metal stubs in the column compartment that have a dual function: to heat the column and also serve as a rest for the column when in situ. Placing the temperature probes on the heat exchangers themselves rather than on the end of the column biases the result, as the column temperature in the column compartment is not qualified — only the heat exchanger. Based upon personal experience, it is difficult to qualify column compartments based upon convection or heat transfer such as this. A much better approach is to use a forced air oven such as that found in a gas chromatograph to maintain the temperature of HPLC columns.

• Using caffeine as a wavelength standard for spectrophotometer and HPLC detector wavelength accuracy is scientifically unsound, in my view. Caffeine has been selected by some vendors as a wavelength standard as it has peaks at 205 nm and 273 nm that bracket many of the wavelengths used in a laboratory. In principle, this is a good idea. However, have you ever thought about what is required from the material that we are using as the external source for wavelength accuracy? Table II lists the key requirements for a wavelength standard. The problem is that caffeine fails to meet several of these key requirements.

Figure 1: Spectra of (a) caffeine (0.00068% w/v in water) and (b) holmium perchlorate (NIST SRM 2034) over the 200–300 nm wavelength range.

First, it is not traceable as a wavelength standard. If in doubt, read the calibration certificates provided by the various vendors:. The spectrometer that is used to measure caffeine is calibrated using traceable standards, but the caffeine itself cannot be traced to an agreed international standard. The best one can do is purchase a standard from the United States Pharmacopoeia, but this is sold only as an analytical standard for pharmacopoeial analysis. The certificate of analysis makes clear that any other use is at the user's own risk, and there is no mention of wavelength in the specification.

Second, caffeine does not have sharp bands but very broad ones, as you'll see in Figure 1, which leads to another unscientifically sound problem with caffeine: How accurately can you measure the peak maxima? The best that I can find on the certificates offered by various vendors is ±1 nm, which is 33–50% of the acceptance criteria required by the pharmacopoeias in Table II and leaves little room for instrument error.

Oh by the way, caffeine is inherently fluorescent (14)! So are you happy with using caffeine as a standard?

Looking on the bright side, caffeine is better than some materials used in the past as wavelength standards, e.g., uracil.

Vendor Qualification and Validation Documentation

So with the roles and responsibilities defined and the key items from the regulations outlined, how do the various vendor qualification and validation documentation stacks up? Let me give you a clue: It's not good and this is not an all-inclusive list of problems with vendor documentation. So what you should be looking for is the following:

1. Who Wrote This Document?

When you write a report about developing a method or method validation, it will have your name on the front cover as the technical author along with the names of the other individuals who reviewed and approved it. This does not happen with the vast majority of vendor qualification documents. Why? Most qualification and validation documents are not authored by a named individual or group of individuals — the best we might get is an engineering part number. Is there a problem? Yes, as the regulations all ask for a combination of education, training or experience for anybody performing regulated work. So from the system owner's perspective in the laboratory, who is paying money for this service and why can't they have the names of the author and reviewers within the company?

2. Where Do I Sign to Preapprove a Document?

To comply with the GMP regulations outlined in Table I, each vendor qualification or validation document must be approved by the user company before execution. Some vendors allow this with space in the documentation. However, most vendors do not, which is a poor understanding by them of the applicable regulations in Table I. But it is even a bigger problem for users who allow vendors to get away with it and compound the noncompliance problem.

If a laboratory knows the document needs approval and there is no space for preapproval signatures, they will review the document and then sign a separate cover page to provide the pre-execution approval. However, here comes the catch 22: To do this, you need to get your hands on a copy of the qualification document before the engineer turns up on-site. Will the company allow you to review and document before you purchase with the risk of rejection and possible ripoff of ideas or do you have to stump up the cost of the document only to find you have purchased rubbish? One company I was involved with was provided with an example document for a review. However, this was eight years old. The most interesting fact was the supply of qualification data on a 3.5-in. diskette, which would be most entertaining given the current stock of laboratory PCs! How are we to review such rubbish accurately?

Perhaps the best way to get around ths problem is to have an agreement as part of the overall system purchase in which the laboratory sees the copy of the current protocol and, unless there are clear compliance or scientific reasons not to continue, then the vendor qualification purchase should go ahead.

3. Where Are the Specifications Being Tested?

Tests being carried out are usually adequately documented but the specifications against which each test passes or fails are not listed in the majority of vendor documents. Furthermore, how are the tests and the acceptance criteria traced back to your original specifications? Ideally, if there were a single section containing all the specifications along with the acceptance criteria, this would enable the users to link the specifications to their own user requirements specification to demonstrate traceability. However, in general, there is not this information and you have to hunt throughout the document to find what is being tested and do all the traceability yourself.

4. How Does the Engineer Document the Test?

There are two issues to consider here:

• Are all the materials for the test provided by the vendor with certificates of preparation or are any materials to be made up on-site? In the latter situation, where will the work be recorded? In the vendor document or in a laboratory notebook of a user and if so, is there space to cross reference this in the qualification document? Why is this an issue, you may ask? Put simply, you have to document this for a regulated analysis so why should any vendor be exempt from the law?

• Recording the work performed by the service engineer has improved over the years. The days when just check or tick boxes were used to complete the testing without identifying the individual who actually did the work should be in the bin now. However, check the document before work starts just to make sure that this is the case with a specific vendor.

5. Lack of Explicit Acceptance Criteria

One of the biggest problems in qualification and validation documentation is stating how tests have passed the acceptance criteria simply because many documents (user and vendor written) have no explicit acceptance criteria. Ideally, there should be a section in each document that summarizes the test and what the acceptance criteria are and if a test has passed these acceptance criteria or not.

6. No Overall Sign-Off of the Work by the Vendor Engineer

If an analyst in the laboratory is involved in executing any qualification or validation document, there will be a place for the individual to state that the instrument or computer system has passed or failed the test. They will put their name against this work or indeed any analytical work that the carry out in the laboratory. However, in the vendor documents, often there is no explicit statement and no overall sign off by the service engineer that the work has been completed and the instrument has passed. There is a general assumption that if the tests are OK then the instrument has been passed. However, the writers of this documentation do not appreciate that their potential audience nor had the document been written for auditability.

In one very poor example I have reviewed, the only people signing off the document were laboratory management. Here I think the vendor's legal eagles have reviewed the document and commented that their service engineer could not possibly sign off the document, as the vendor might get sued. In my opinion, if a regulated organization contracts with an instrument company to do the work, they should expect the instrument company's engineer to sign to say the work has been carried out and that the instrument passes or fails.

7. Is There an IQ Report Before the OQ Begins?

As required by EU GMP regulations there needs to be a report written and approved at the end of each phase of the work — for example, a report at the end of the IQ and another at the end of the OQ. Do we get this? No. What happens in practice is that the IQ leads straight into the OQ and at the end of the work there is a pile of paper left by the service engineer on the bench as they exit the laboratory. This is a area that both vendors and users need to be more aware of and ensure that the regulations are complied with. The report can be integrated into the existing documentation but approval of the work needs to take place before the next stage of the work begins.

So here are some of the issues system owners and senior scientists need to address with vendor supplied qualification documentation to ensure compliance with the regulations to which they operate. Otherwise would you prefer to explain this to an inspector?

Software Validation Issues

And then we have software validation for USP <1058> group C systems. I will spend more time on this in my next column when I review <1058> two years after the implementation date. The approach to software validation varies from the sublime to the ridiculous. Sublime where group B instruments with embedded software are qualified as part of the overall instrument effort and ridiculous with group C systems and the reliance on a vendor to do most of the software validation work, which is naïve to say the least.

Implicit or indirect testing of embedded software in group B instruments is a practical approach and is good for the most part. There are some omissions with embedded software, which I will discuss in my next column, but overall the approach is sound. The problem comes with group C systems. Here I will discuss the vendor issues and leave the major omissions and problems with the approach to my next column.

1. Misclassification of Software Category

One of the biggest problems is the vendor misclassifying software category whether this is due to the deviousness of the marketing department or a misunderstanding of the current GAMP software classifications I will leave for you to decide. For example, one spectrometry software application is classified by its vendor as COTS without defining exactly what COTS means. It implies that the software is commercial off the shelf software or GAMP category 3 software (15) however in the same paragraph of the validation document it notes that the software can also be configured and that a user can define macros to manipulate data and results which are software category 4 and 5, respectively. Therefore it important that the system owner defines the software category according to their company standard operating procedures and CSV policies and that they should not rely on vendor self-classification. This is particularly important as companies transition from GAMP 4 to GAMP 5 software classifications which are substantially different.

Table II: Key requirements for a wave- wavelength standard

2. Vendor Software OQ Repeats the In-House Testing

Some enterprising vendors will adapt their release testing documents into the operational qualification that they will sell to users. What is required in the operational qualification is that the software works and where appropriate can control the instrument, users do not need time wasted on repeating in house release testing. This is especially true when the software will be configured afterward by the users to meet their own business needs, such as for electronic records protection and use of electronic signatures. I will return to this topic in a later column.

3. Poorly Written and Untestable Software Requirements

To help users, some vendors will provide a validation package from plan, through specification of requirements to testing. Treat these packages with great suspicion until you have had the time to examine the whole package critically. If a vendor does not allow you to see the document before you buy, my advice is do not purchase it. See the package before you buy — always.

The reason for this is presented below. Here are two examples of software requirements from a vendor validation package that can be yours for a princely sum:

• The software fulfils regulatory requirements.

• The software is suited for its intended purpose.

These requirements are untestable rubbish for which you, unsuspecting reader, have paid good money to acquire. Let us look in more detail at why I have described them as untestable rubbish. There is a software engineering standard number 1233 from the Institute of Electrical and Electronic Engineers (IEEE) entitled IEEE Guide for Developing System Requirements Specifications (16). In this document, there is a section on writing testable or verifiable user requirements. Space does not permit me in this column to explain in detail the whole process, but this is described in more detail in my book on validation of chromatography data systems (17). The essence for a requirement to be either testable or verifiable is that there should be a capability described that states what you want the system to do coupled with a condition that will makes the requirement testable or verifiable. For example "access to the system will be via password" is the capability and "consisting of eight alphanumeric characters" provides the conditions to make the requirement testable.

Now look at the one of the vendor examples presented above. The software fulfils regulatory requirements. So which regulatory requirements do we mean: GLP or GMP or even 21 CFR 11? To go further, does the requirement imply all requirements in each of these regulations or only the ones that relate to the laboratory? So back to my assertion. How will you test this requirement? You cannot is the simple answer. At best, the software fulfills regulatory requirements is a design concept that will have to be developed much, much further into a large number of specific and testable requirements. These requirements will cover areas in the software such as security and access control, data file integrity, review, approval and reporting of data including electronic signatures and audit trails.

Conclusions

The use of vendor qualification material, if it matches the laboratory needs, is a valuable resource. However, the material is often deficient in compliance with regulatory requirements such as not being scientifically sound nor provides for preapproval signatures to identify two areas. It is the responsibility of the system owner and the users in the laboratory to ensure that any material purchased meets both user needs and regulatory requirements. I hope this column will help users be more critical of vendor material and also help vendors develop better approaches to delivering this material. The relationship between vendor and user should be a partnership and not adversarial.

Acknowledgments

I would like to thank Chris Burgess for help and comment in the preparation of this article.

R.D. McDowall is principal of McDowall Consulting and director of R.D. McDowall Limited, and "Questions of Quality" column editor for LCGC Europe, Spectroscopy's sister magazine. Address correspondence to him at 73 Murray Avenue, Bromley, Kent, BR1 3DJ, UK.

R.D. McDowall

References

(1) W.B. Furman, T.P. Layloff, and R. Tetzlaff, JOAC Int. 77, 1314–1317 (1994).

(2) M. Freeman, M. Lang, D. Morrison, and R.P. Munden, Pharm. Technol. Eur. 10, 45–48 (1995).

(3) C. Burgess, D.G. Jones, and R.D. McDowall, Analyst 123, 1879–1886 (1998).

(4) P. Bedson and M. Sargent, J. Accred. Qual. Assurance 1, 265–274 (1996).

(5) Guidance on Equipment Qualification of Analytical Instruments: High Performance Liquid Chromatography (HPLC), June 1998, LGC/VAM/1998/026.2.

(6) Guidance on Equipment Qualification of Analytical Instruments: UV-Visible Spectro(photo)meters (UV-Vis) Version 1.0 – September 2000, LGC/VAM/2000/079.

(7) EU GMP Chapter 1 on Quality Management.

(8) ICH Q10 Pharmaceutical Quality Systems.

(9) EU GMP Annex 15 on Qualification and Validation.

(10) FDA GMP for Finished Pharmaceutical Products (21 CFR 211).

(11) FDA 21 CFR 11 on Electronic Records and Electronic Signatures .

(12) United States Pharmacopoeia general chapter <1058> on Analytical Instrument Qualification (AIQ).

(13) R.D. McDowall, Spectroscopy 24 (12), 67–72 (2009).

(14) Standards in Absorption Spectrometry, UV Spectrometry Group, C. Burgess and A. Knowles, Eds. (Chapman and Hall, 1981), pp. 63–64.

(15) R.D. McDowall, Spectroscopy 25 (4), 22–31 (2010).

(16) IEEE Software engineering standard 1233 from the Institute of Electrical and Electronic Engineers (IEEE) is entitled IEEE Guide for Developing System Requirements Specifications.

(17) R.D. McDowall, Validation of Chromatography Data Systems, Meeting Business and Regulatory Requirements (Royal Society of Chemistry, Cambridge, UK, 2005).