One of the most challenging things to complete in a regulated environment, such as the Pharmaceutical Industry, is the implementation of a new automated system. The choices of automated solutions open to the industry are wide and varied, however, the final documentation package will need to defend the new system to a regulatory inspector’s expectation that the system operates as designed and tested.

This brief article attempts to highlight several areas that have proved to be a painful experience during past projects. It is by no means a comprehensive review of the requirements for validating a new system, but hopefully provides some insight to potential pitfalls that can delay and/or impact the delivery of a new automation system in this regulated environment.

V Model CSV Image

Project Funding and Planning

The primary challenge in this area is to determine the reasonable timeline for the project and the associated funding required to bring the system on-line. Too generous of a timeline will generally add unneeded cost to the project, however, too tight of a timeline can dramatically impact the outcome of unanticipated events that need resolution prior to concluding the project. Project sponsors and/or oversight groups will usually consider the impact to both the delivery date and associated costs when determining the next course of action.  

Completing risk analysis during the planning stage to assist with identifying potential setbacks to the proposed project timeline, and the cost of remediation, can be beneficial to support adjustments to the project plan prior to final approval. It is important to include all business groups associated with the new system in the planning stage. Specifically, the Engineering (IT) group, Business Owners, and Quality Assurance must have the opportunity to review and provide input to the project plan prior to approval.

Finally, the time requirements and number of internal resources must be adequately planned. Internal resources can easily be underestimated when considering the need to provide oversight for outsourced support (generally multiple groups) and “stacked” project plans. A common concept is to “stack” the project plan deliverables. i.e. design phase, testing phase, project turnover, etc., into multiple swim lanes. What looks like a reasonable approach to shorten a project timeline can quickly overwhelm Project Leads as inevitable delays occur within these swim lanes. Bringing on additional resources mid-project is difficult, and a more favorable practice is to plan and include all internal resources at the beginning of the project.

User Requirements Specification (URS)

This will become the highest priority document in the new automated system’s lifecycle. The URS will “live” with the system, get revised as new changes are implemented to the system, and address any regulatory question regarding the system’s intended functions. The process of generating a comprehensive URS should not be rushed or held to a rigid timeline.

Input from the entire user community must be collected and organized into a comprehensive URS. Perspective on what is needed may be different based on the users intended function with the new system, i.e. the Quality Assurance needs may be different from that of the Business Owner needs. There may be a tendency from the user community to provide “general” needs, however, the project team must push to a deeper level of user requirements. This can be accomplished by providing “leading questions” of the user community, such as:

  • How should the system look and feel to the user
    • Layout of the user interface to the system
    • Manner in which the user moves through various sections of the system
  • What type data and/or processing is needed from the new system
  • Are there standard and/or ad-hoc reporting functions required of the system
    • What type of data is required in reports
    • How should reports look to the eventual end-user
  • Are there interfaces with other systems needed for data, etc.

The development team should be able to provide system mock-ups to allow the user community to evaluate their input and provide any additional preferences for the new system. This can generally occur as a series of events until a reasonable URS can be generated. At times this can appear to be an unending cycle, however, resist the tendency to complete this step and move on without sufficiently covering all intended users and defining all of the system’s requirements. Additional changes as a result of discovering new functions, etc., will have a much wider impact as the project progresses.

Formal Change Control

Regardless of the methodology used for developing and designing the new system, there will come a time that all further changes will need to go through a formal change control process to evaluate and approve changes to the system.  I would recommend that time frame be tied directly to the approval of the system URS.  

Project change control must be every bit as rigorous as existing change control procedures and processes. Sufficient oversight must be maintained to evaluate any proposed changes against system requirements and determine the impact to system functionality and documentation packages. Project personnel must understand that non-compliance to the change control process could potentially lead to significant delays in the project timeline.  

Previous experience has included significant delays during the testing phase as multiple deviations generated during test script runs had to be investigated and remediated. Many of these deviations were a direct cause of changes being added to the system without the oversight of the change control process. Subsequently, revisions to design specifications providing the basis of test script development were not implemented. The resulting implications included multiple test script deviations, including several test script failures, requiring remediation and retesting of impacted sections. The impact of these delays were felt across the entire project team.

Training of Project Personnel

Training of project personnel can be easily overlooked, but this is an area that should be considered carefully. Many teams are made up of subject matter experts that may feel that further training is not of much benefit, however, bringing everyone together to review project details can be very helpful. Much of this can be dealt with during project kick-off meetings covering team structure, validation methodologies, project planning, etc. Brief refresher training for personnel will assist with reducing issues downstream.

Particular focus should be directed to testing personnel, specifically requirements for good documentation practices during test script development and execution. This is very important with the continued adoption of automated testing applications. These applications generally allow management of the testing phase in a digital environment, which can be much better than managing a “truckload” of paper documentation. However, the training for these systems usually only covers the use of the application. Items such as development of meaningful test scripts and proper documentation of test script execution are not generally covered and can directly impact the testing phase of the system and the eventual GMP compliance status of the new system.

There also must be a uniform understanding of the terms and conventions used during the project. As an example, from a previous project, the term “address” is commonly known as a Box/House Number, Street, City, State, and Zip code in most all of the US. However, this is not necessarily the same understanding of an individual from a different region in the world, or someone whose perspective might be influenced by the data field in the system, i.e. Address Field = house/box number and street. The result of the field issue didn’t present itself until testing was conducted on an associated report that required the address of a particular entity. Remediation of this particular deficiency touches many areas within the project design, testing, etc.

Insufficient Testing Environment

The planning and execution of acceptance testing of a new system can be complex and, at times, very complicated. Much of this can be properly managed through the development of a System Test Plan to provide oversight and direction of this important project phase.

The concern of note in this area is the lack of a sufficient testing bed, in some cases known as the testing sandbox. Some more sophisticated systems encountered have mirrored the operational database in a Development and Testing environment. System changes can then be evaluated in the Development environment, tested in the testing sandbox, then migrated into the operational database.  

Certainly, system testing can be sufficiently managed outside of such a setup, however, previous experience has shown that some serious system discrepancies were not identified until the system became operational. Setup of a testing environment noted above is costly, both in project timeline and funding, however, the benefits of such an environment will generally far outweigh the cost of setup.

Validation Package Documentation

The challenge here is to ensure a completed documentation package has been prepared and delivered, generally through a turn-over package at the end of the project. The basis of defending the validation of a new GMP automation system is included in this final documentation package. Management of this project deliverable is generally accomplished by way of a documentation listing and project repository.  

The documentation list must include the title, revision, and effective date of the associated document as it progresses through the project timeline. A revision history is also beneficial; however, this can be covered in a history log as part of each document revision. At first sight, this looks a bit like restating the obvious, however, previous experience has shown that these activities are easily overlooked during the course of dealing with other project items. In order to avoid any difficulty with bringing validation documents into alignment with the delivered automation system, these actions should be managed closely throughout the project timeline.

Hopefully the items noted in this article will trigger consideration of potential troublesome areas that may be encountered during the validation of new GMP automation systems.  Feedback and ongoing dialogue regarding these items is most welcome.