Performance evaluation options

The manufacturer must choose one of the following two performance evaluation options, and must indicate its choice in the pre-submission form:

• Option 1: Performance evaluation commissioned and coordinated by WHO, and carried out at an evaluating site listed by WHO. The evaluating site will conduct the performance evaluation and share the data from the evaluation directly with WHO.
• Option 2: Performance evaluation commissioned directly by the manufacturer from an independent laboratory, which has been selected from WHO's List of Performance Evaluation Laboratories. If this option is chosen, the manufacturer will be responsible for paying the full cost of the performance evaluation (in addition to paying the applicable prequalification assessment fee) and for coordinating the performance evaluation directly with the evaluating site. The laboratory will conduct the performance evaluation and share the resultant data directly with WHO and with the manufacturer. To ensure the independence of the performance evaluation, data analysis will be performed by the laboratory and reviewed by WHO.

If the product is eligible for prequalification assessment and option 1 is chosen, the manufacturer will receive a letter from WHO outlining the type of assessment that the product will undergo (i.e. either an abridged or a full assessment) and WHO will coordinate the performance evaluation. If the product is eligible for prequalification and option 2 is chosen, the manufacturer will receive a letter from WHO outlining the type of assessment that the product will undergo (i.e. either an abridged or a full assessment), as well as instructions on how the data generated from the performance evaluation will be shared by the evaluating site.

Procedure for performance evaluation

Irrespective of the option chosen, the performance evaluation must be carried out in accordance with a publicly-available WHO protocol developed in collaboration with international experts. The WHO protocols to be used for performance evaluation of IVDs which have been accepted for prequalification assessment can be found below. The evaluations usually include a verification of the clinical performance (sensitivity, specificity), as well as analytical performance (analytical sensitivity, precision, lot to lot variations, etc.). In addition, operational characteristics and ease of use are assessed to inform use in settings with limited infrastructure. Where different specimen types with different performance characteristics (e.g. limit of detection and/or sensitivity and specificity) are claimed, a full performance evaluation will be conducted on one of the specimen types and a limited verification will be conducted on the other specimen type. This limited evaluation will also be conducted if specimen types with different performance characteristics are added as a change request to a prequalified product.  

The manufacturer must send to the evaluating site(s) the requisite quantities and lots of the product (test kits and/or instruments), "free domicile", free-of-charge, and delivered with all customs declarations, customs duties, and transport and other charges paid for by the manufacturer. (The evaluating site must provide the manufacturer with detailed shipping instructions (e.g. number of test kits and/or instruments, number of lots, etc.) in due time.) If necessary, special equipment needed to perform the assay must be made available by the manufacturer at no charge (i.e. customs declaration and payment of customs duties, transport, installation, training, etc., will be made by the manufacturer) to the evaluating site for the duration of the prequalification assessment process. The manufacturer should not send tests to the evaluating site(s) unless explicitly invited to do so.

WHO will have absolute, exclusive, unfettered control over how the prequalification assessment process is carried out (including the performance evaluation and/or the publication of results of the prequalification assessment, irrespective of the outcome). Without prejudice to the foregoing and in agreement with WHO, the manufacturer may decide or be required to visit the specified evaluating site(s) to observe the operator performing the test procedure on the manufacturer's product(s) before starting performance evaluation. After this visit, or if the manufacturer decides not to visit the evaluating laboratory, WHO will not accept any contestation of the adherence to the instructions for use.

Performance evaluation report

• Under option 1 the evaluating site(s) will submit a draft performance evaluation report to WHO. WHO will review the data analysis and report and, if necessary, request clarification from the evaluating site(s). After verification, WHO will send the draft performance evaluation report to the manufacturer.
• Under option 2 the evaluating site(s) will submit a draft performance evaluation report to the manufacturer and to WHO. WHO will review the data analysis and report and, if necessary, request clarification from the evaluation site(s). The manufacturer will be in copy of the exchanges between WHO and the evaluation site(s) on data analysis and report.  

In both cases, after verification, WHO will request the manufacturer to review and comment on the draft report and the results. WHO will reasonably consider any comments by the manufacturer to the draft performance evaluation report, provided that such comments are submitted by the manufacturer in writing to WHO within one month after the manufacturer's receipt of the draft performance evaluation report. To avoid doubt, WHO will maintain full and exclusive control over the data analysis, the reporting of the performance evaluation results and the form and content of any publication thereof. After one month, the performance evaluation report will be considered as final.

A summary of the performance evaluation report will be included in the WHO prequalification public report if the product successfully meets all WHO prequalification requirements. WHO reserves the right to publish the results of the evaluation in a peer-reviewed journal, irrespective of the outcome. In this case, WHO will share the manuscript with the manufacturer for comments at least 30 days before submission, but WHO will have ultimate authority over the version submitted.

If the product fails to meet WHO prequalification acceptance criteria for performance evaluations, the application will be cancelled.