International Organization for Standardization 

• ISO 13485:2016 Medical devices ‒ Quality management systems ‒ Requirements for regulatory purposes
• ISO 19011:2018 Guidelines for auditing management systems
• ISO 14971:2019 Medical devices ‒ Application of risk management to medical devices
  Note: EN ISO 14971:2012 applies only to manufacturers placing devices on the market in Europe; for the rest of the world, ISO 14971:2007 remains the applicable standard.
• ISO 9000:2015 Quality management systems – Fundamentals and vocabulary
• ISO 9001:2015 Quality Management Systems ‒ Requirements
• ISO 23640:2011 In vitro diagnostic medical devices ‒ Evaluation of stability of in vitro diagnostic reagents
• ISO 15223-1:2021 Symbols to be used with information to be supplied by the manufacturer - Part 1 General Requirements
• ISO 2859-10:2006 Sampling procedures for inspection by attributes ‒ Part 10: Introduction to the ISO 2859 series of standards for sampling for inspection by attributes

Global Harmonization Task Force (GHTF), available on the IMDRF site

• GHTF/SG1/N071:2012 Definition of the terms 'Medical device' and 'In vitro diagnostic (IVD) medical Device
• GHTF/SG1/N70:2011 Label and instructions for use for medical devices
• GHTF/SG3/N19:2012 Quality management system – medical devices – nonconformity grading system for regulatory purposes and information exchange
• GHTF/SG3/N18:2010 Quality management system – medical devices – guidance on corrective action and preventive action and related QMS processes
• GHTF/SG3/N17:2008 Quality management system – medical devices – guidance on the control of products and services obtained from suppliers
• GHTF/SG3/N15R8 Implementation of risk management principles and activities within a quality management system
• GHTF/SG3/N99-10:2004 (Edition 2) Quality management systems – process validation guidance
• GHTF/SG4/N84:2010 Guidelines for regulatory auditing of quality management systems of medical device manufacturers – Part 5: Audits of manufacturer control of suppliers

World Health Organization

• WHO documents: https://extranet.who.int/prequal/key-resourcesdocuemnts/vitro-diagnostics/all
• IMDRF MDSAP (Medical Device Single Audit Program): MDSAP_AU_P0019.004: Medical Device Regulatory Audit Reports Policy

Other Reference Documents (European Commission)

The standards listed at the following website are harmonized standards and thus conformity with the relevant essential requirements is assumed:
http://ec.europa.eu/enterprise/policies/european-standards/documents/harmonised-standards-legislation/list-references/iv-diagnostic-medical-devices/index_en.htm

Common Technical Specifications on IVD are published under: "Other amending and implementing legislation" at http://ec.europa.eu/health/medical-devices/documents/index_en.htm

The responsibilities of the manufacturer include:

• agreeing upon the objectives and the scope of the inspection with the WHO prequalification team
• informing the WHO prequalification inspection team of any issues that may affect an effective and efficient inspection process
• cooperating with the inspectors to ensure that the objectives of the inspection are achieved
• appointing an employee to be responsible for coordinating and facilitating the inspection process on behalf of the manufacturer
• informing relevant employees and personnel about the objectives and scope of the inspection
• determining which members of staff will accompany members of the inspection team
• ensuring that the inspectors are aware of health, safety and other applicable requirements
• providing on-site resources, such as a meeting room, for the inspection team, in order to ensure an effective and efficient inspection process
• providing full access to the manufacturing facilities, documents and records and other evidence as requested by the inspectors, in a timely manner, to ensure an effective and efficient inspection process and so that the inspection timetable can be adhered to.

If findings are identified during the inspection, information about these will be provided to the manufacturer during the inspection and at the closing meeting.

An on-site inspection will seek to confirm the adequacy and effectiveness of the manufacturer’s documented quality management system (QMS), with emphasis on the control of the production processes and compliance with state-of-the-art practices, including WHO technical guidelines.

Documents and records from all levels of the QMS will be reviewed. Post-market surveillance data, as well as marketing and training material, may be included in the review.

Interviews of personnel at all levels and discussions with persons selected by an inspector will form part of the inspection process.

Evidence will be collected on-site, as follows:

• by examination of documents, including standard operating procedures and records
• by visual observation of production and quality control activities relevant to te product(s) in scope
• by visual observation of environmental conditions and infrastructure
• by confirmation of statement of fact that is acquired through interviews and may include
  • random sampling of product for laboratory quality control testing
  • photographs.

Findings identified during the document review or interviews will be notified to the accompanying representative of the manufacturer and may be verified by acquiring additional information where possible. The manufacturer will be given an immediate opportunity to comment on findings. Based on this evidence, a finding, even if corrected immediately, will be noted and form part of the final inspection report.

Onsite inspection overview

The WHO inspection will be conducted in a format that follows the production process. Note that ISO13485 is used as a basis for the inspection and that other products and QMS-related standards and references may be used by the manufacturer to ensure good practice in the manufacture of the IVDs or MCDs.

The QMS areas inspected may include, but is not limited to the following processes:

• management: to ensure that an adequate and effective QMS is in place, including management review
• product documentation, including design and development: to ensure that the manufacturer has established sufficient documented systems and adequate communication of the systems (including change control) to all personnel, to ensure a quality product outcome
• production and process controls: to ensure that the manufacturer has established sufficient systems such as testing, infrastructure, facilities, equipment and personnel to ensure a quality outcome; demonstrated independence between the production and quality unit and that the quality unit controls release of product batches
• corrective and preventive actions, internal audits: inspection of this section is to confirm that the manufacturer collects and analyses actual and potential quality problems through investigation and appropriate action
• purchasing controls: especially important when significant components are outsourced – the manufacturer must ensure that raw material, intermediates, components and services provided by suppliers are of an appropriate standard
• documentation and records: to ensure that relevant documents and records are defined, established and controlled, for example, by being updated and properly authorized; and to ensure procedure and process documents are readily available and in routine use by staff as needed
• customer-related processes: in this context, customers include purchasers and users of the product and relevant regulatory bodies
• training of personnel and training records: to ensure that qualifications and training of personnel are appropriate to the tasks required of them
• infrastructure and work environment: to ensure the adequacy of facilities, manufacturing, equipment, monitoring and quality control equipment; calibration and maintenance.

Verification of data supporting IVD product dossier submission

During the inspection, an inspector will sample the quality records and reports that support the data submitted with the product dossier for WHO prequalification. This may include, but is not limited to, data recorded for the batches submitted for WHO performance evaluation or testing, data collected in performance studies, quality control data, and batch manufacturing records. The manufacturer must ensure that the entire product dossier submitted to WHO is available on-site during the time of the inspection.

The dates and time allocated for the inspection are to be agreed upon by all participants under the guidance of the WHO inspector and will be documented in an inspection plan. The inspection should be scheduled for a time when:

• the production line of the product undergoing prequalification assessment is active (if several products are inspected during the same inspection, the production line for at least one must be active)
• quality control activities are being performed
• the key personnel for the quality management system (QMS), quality control and production line will be present.

Usually, the inspection plan will be provided one to two weeks prior to the inspection. It will include details of the type of inspection to be conducted and the sites and products to be inspected, based on submitted information about the product and the QMS,  including the quality manual. The inspection plan is a guide that may be altered to permit changes in emphasis based on information gathered during, or immediately prior to, the inspection.

The inspection plan will include the following information:

• the type of inspection (e.g., initial, re-inspection)
• the scope and criteria of the inspection
• the inspection team members
• the date of the inspection
• the address of the site inspected
• an approximate time and duration of each inspection activity, including meetings to be held with the manufacturer's management team.

The time allocated to the inspection will be determined according to the number and complexity of the product(s) in scope of the inspection, the number and complexity of manufacturing technologies, the size of the site, and other information gathered prior to the inspection. The WHO Lead Inspector will allocate responsibilities to the inspection team members according to their expertise and inspection requirements

Documentation regarding subcontractors, outsourced processes and critical suppliers

The manufacturer must have the necessary documentation available to demonstrate that the processes to control product supply are effective and meet the relevant quality expectations. This includes, but is not limited to, documentation relating to providers of critical raw material, interim components, packaging or other services used to produce the IVD or MCD. The manufacturer shall be responsible for the sufficient control of any critical supplier, including outsourced processes. If this requirement is not sufficiently met, a nonconformity against the respective ISO 13485 requirement will be issued and an inspection of the subcontractor site(s) may be necessary.

Overview

The purpose of the inspection report is to provide:

• the manufacturer with information on nonconformities identified during the inspection: identified nonconformities must be investigated to determine and implement an effective CAPA plan
• the manufacturer with information upon which to base improvements to the QMS
• a permanent record of the findings of the inspection
• recommendations to the WHO team on actions to be taken following the inspection.

A more detailed description of the purpose of the inspection report can be found in guidance such as IMDRF/MDSAP WG/N24 FINAL: 2015 – Medical device regulatory audit reports and ISO 19011:2011 Guidelines for auditing management systems.

The inspection team will prepare the inspection report and the WHO Lead inspector is responsible for its accuracy and content.

A final inspection report will be issued by WHO to the manufacturer within 30 calendar days from the last day of the inspection. This time may extend during periods of high workload. If a final inspection report cannot be issued within a 30-day time-frame, the manufacturer will be notified accordingly. The manufacturer understands and agrees that WHO will have absolute, exclusive control over the manner in which the inspection is carried out, including publication of the results of the inspection, irrespective of the outcome. The manufacturer also understands and agrees that WHO reserves the right to share the manufacturer's prequalification application and related information, as well as the results of the inspection and the full inspection reports, including any drafts thereof and including (subject to appropriate obligations of confidentiality) any confidential information to which WHO may gain access in the course of the prequalification process and/or inspection, with the relevant authorities of any interested Member State of the Organization, with interested national regulatory authorities (NRAs), and with relevant inter-governmental organizations. As used here, "Confidential information" means:

• confidential intellectual property, know-how and trade secrets (including, e.g. formulas, processes or information contained or embodied in a product, unpublished aspects of trade mark, patents, etc.);
• commercial confidences (e.g. structures and development plans of a company).

Contents of an inspection report

An inspection report includes:

• the type, scope and criteria of the inspection
• details of the inspection team
• details of the areas covered in the inspection
• any limitation to the inspection
• details of nonconformities (and their severity grading)
• comment and conclusions about the effectiveness of the manufacturer's QMS in meeting quality objectives
• summary of conclusions
• authorized signature and date of the report.

The report will include the following comment: "This report contains the collective views of the inspection team performing this inspection and does not necessarily represent the decisions or the stated policy of the World Health Organization."

Nonconformities

During the on-site inspection, nonconformities may have been identified with respect to:

• QMS inspection criteria
• verification of data supporting the product dossier claims.

Both types of nonconformities, as well as the objective evidence contributing to the nonconformity, including the inspection criterion that was not met, will be individually stated and described in the manufacturing site inspection report.

The severity of nonconformities will be classified according to the Global Harmonization Task Force SG3 N19:2012 document: with level 1 being the lowest level on nonconformity and level 5 the highest. The QMS shall be considered critically deficient, if the following occur:

• 1 or more level 5 nonconformity(s) or
• 7 or more level 4 nonconformities.

If any nonconformities are included in the final inspection report, a corrective and preventive action (CAPA) plan shall be submitted to WHO by the manufacturer within 30 calendar days after receipt of the final inspection report. For each identified nonconformity, the CAPA plan shall include:

• a root-cause analysis
• the corrections required for the identified nonconformities (if applicable)
• the corrective actions required to remove the cause and to prevent recurrence (if applicable)
• the preventive actions required to prevent occurrence of similar nonconformities (if applicable)
• a timeline for implementation of corrections, corrective and preventive actions
• an approach and timeline to the evaluation of the effective implementation of the proposed CAPA plan
• objective evidence in support of actioned CAPA

The manufacturer must provide the CAPA and any related objective evidence to WHO in an editable format, such as a Word document or Excel spreadsheet. This will facilitate communication regarding the approval or rejection of the CAPA, or a request for additional information and documents. 

WHO review of corrective action plan

The inspection team will review the CAPA plan and determine if:

• the CAPA submissions are acceptable to WHO
• if the CAPA submissions are not acceptable to WHO, the inspection team will request an improved CAPA plan and may ask the manufacturer for further evidence of the effective implementation of the CAPA plan. The manufacturer must provide this requested information within 30 calendar days of receipt of the report of the first CAPA review by WHO. The manufacturer will typically have a maximum of two opportunities to supply WHO with the necessary information to assess the effective implementation of the CAPA plan. In special circumstances pertaining to the nature and/or number of the nonconformities, WHO may agree in writing to grant the manufacturer an extention in which to supply the requested information.
• if the effective implementation of the CAPA plan cannot be evaluated by document review, a follow-up inspection will be required before the nonconformities may be closed out.
• In certain cases, WHO may agree, at its sole discretion, to permit the manufacturer to correct specific nonconformities after prequalification, provided that the manufacturer commits in writing to correct them by an agreed deadline. Such a "commitment to prequalification" will be monitored by the WHO Inspector and may be verified during a re-inspection. Failure to comply with those commitments within agreed timelines may result in the delisting of the product from the relevant WHO list of prequalified products.

Before finalizing the manufacturing site inspection, all nonconformities identified in the inspection report must have been satisfactorily addressed (as determined by WHO) by the manufacturer. However, sometimes, a satisfactory outcome is not reached if, for example:

• the manufacturer does not submit (whether within the specified timeline, or at all) a CAPA to WHO
• the manufacturer is unable to implement all agreed corrections and corrective actions during the agreed time period
• more than six months have elapsed after the initial inspection and the manufacturer has still not provided WHO with satisfactory responses to the identified nonconformities.

In any of the above cases, the manufacturer’s application for WHO prequalification of the product(s) concerned may be terminated by WHO and the manufacturer will have the option to reapply for prequalification in 12 months from the date of termination. This may require a new application , potentially including a new evaluation for prequalification of the product.

Any subsequent application for WHO prequalification will not be accepted unless and until the manufacturer submits sufficient evidence demonstrating that the nonconformities have been effectively corrected. If the manufacturer's QMS was found to be critically deficient, a re-inspection and review of any available additional data must take place before the inspection component can be completed.

The team lead of Inspection Services, in collaboration with the WHO inspector, will notify the manufacturer by letter when the inspection is complete.

Successful completion of the inspection of a manufacturing site(s) and re-inspection

When the CAPA plan has been accepted by WHO, the manufacturer will receive a letter advising it of the completion of the inspection process.

The re-inspection period will be determined by WHO using a risk management approach that takes into account various information, including post-market surveillance data, annual reports, and change requests.
 

Criteria for not recommending prequalification 

Based on the consensus view of the inspectors, and following review by WHO, it may be recommended that the product be removed from or not be included in WHO's List of Prequalified In Vitro Diagnostics Products. Criteria/reasons for this decision may include:

• failure to maintain an adequate QMS (deficient QMS)
• falsification of data/submitted evidence or deliberate misrepresentation of facts regarding the manufacturing and/or QMS
• excessive number of nonconformities level 4/5 identified
• failure to implement appropriate action when post market data has identified a pattern of defects
• failure of the product to meet the manufacturer's own specifications
• failure of the manufacturer to respond adequately to requests for submissions relating to nonconformities.

A notice of concern may be issued when an inspection reveals a certain degree of non-compliance. Including when a manufacturer does not manage to resolve the cited significant non-conformity and to apply an appropriate corrective and preventive action plan which in turn may have a negative impact on the safety, efficacy/performance or quality of a prequalified product, and result in failure to comply with relevant standards and norms.

A Notice of Concern will also be issued if the inspection observations relate to misrepresentation of data, falsification or manipulation of data with the intent to deceive.

Further information can be found: https://extranet.who.int/prequal/inspection-services/notice-concern