Field evaluations of four SARS-CoV-2 rapid antigen tests during SARS-CoV-2 Delta variant wave in South Africa

doi:10.21203/rs.3.rs-2355146/v1

Download PDF

Research Article

Field evaluations of four SARS-CoV-2 rapid antigen tests during SARS-CoV-2 Delta variant wave in South Africa

https://doi.org/10.21203/rs.3.rs-2355146/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 25 Jul, 2023

Read the published version in Diagnostic and Prognostic Research →

You are reading this latest preprint version

Background: Rapid antigen tests detecting SARS-CoV-2 are being increasingly used across the globe and were shown to be a useful tool in managing the COVID-19 pandemic. The purpose of this prospective study was to characterise the performance of four SARS-CoV-2 rapid antigen tests in a South African setting.

Methods: Rapid antigen test evaluations were performed through drive-through testing centres in Durban, South Africa from July-December 2021. Two evaluation studies were performed: nasal Panbio COVID-19 Ag Rapid Test Device (Abbott) was evaluated in parallel with the nasopharyngeal Espline SARS-CoV-2 Ag test (Fujirebio, nasopharyngeal); followed by the evaluation of nasal RightSign COVID-19 Antigen Rapid test Cassette (Hangzhou Biotest Biotech) in parallel with the nasopharyngeal STANDARD Q COVID-19 Ag test (SD Biosensor). The Abbott RealTime SARS-CoV-2 assay was used as a reference test.

Results: Evaluation of Panbio and Espline Ag tests was performed on 494 samples (31% positivity) while the evaluation of Standard Q and RightTest Ag tests was performed on 539 samples (13.17 % positivity). The overall sensitivity for all four tests ranged between 60-72% with excellent specificity values (>98%). Sensitivity increased to >80% in all tests in samples with Ct value <20. All four tests performed best in samples from patients presenting within the first week of symptom onset.

Conclusions: All four evaluated tests detected a majority of the cases within the first week of symptom onset with high viral load and could be a valuable diagnostic tool for controlling the spread of SARS-CoV-2.

COVID-19

SARS-CoV-2

antigen rapid diagnostic test

performance evaluation.

Diagnostic testing has proven to be imperative for the management of SARS-CoV-2/COVID-19 in the context of reducing transmission and outbreak control [1]. While the gold standard RT-PCR test is highly sensitive and specific there are several disadvantages including cost, complexity and length of the process, and need for specialised equipment and trained personal. The use and availability of rapid antigen tests (RDTs) for the diagnosis of SARS-CoV-2 infection has significantly increased over the last year of the pandemic. RDTs are affordable, fast (10–30 min), simple and do not require specialised laboratory facilities or highly trained personal. Although their sensitivity is lower compared to a laboratory-based RT-PCR, antigen-based RDTs can detect infection early following symptom onset when the viral load is high thereby offering quick screening and detection of SARS-CoV-2/COVID-19 among high-risk groups [2].

The number of available rapid antigen tests is increasing constantly and as of May 2022, South African Health Products Regulatory Authority (SAHPRA) has approved 53 RDT test for use in South Africa [3]. While the World Health Organization (WHO) recommends a minimum of 80% sensitivity and 97% specificity for rapid antigen diagnostics tests to be approved, there is significant variation in RDT performance depending on the study settings [4–6]. Furthermore, there continues to be a limited number of reports on RDT field performance in low- and middle-income country (LMIC) settings. Here, we evaluate the performance of four rapid antigen tests in comparison with the Abbott RT-PCR assay during the B.1.617.2 Delta variant wave in KwaZulu-Natal, South Africa.

Study participant recruitment

Rapid antigen test evaluations were preformed between July and December 2021 through drive-through testing centres in Durban, South Africa. Participants meeting any of the following criteria were included in the study: tested COVID-19 positive in the previous seven days; presence of COVID-19 symptoms in the previous seven days; exposed to COVID-19 5–10 days ago; health care worker; or doctor referral for testing. Two evaluation studies were performed: first the Espline SARS-CoV-2 Ag test [Fujirebio, nasopharyngeal (NP)] was evaluated in parallel with the Panbio COVID-19 Ag Rapid Test Device (Abbott, nasal); this was followed by the evaluation of RightSign COVID-19 Antigen Rapid test Cassette (Hangzhou Biotest Biotech, nasal) in parallel with the STANDARD Q COVID-19 Ag test (SD Biosensor, NP). The Abbott RealTime SARS-CoV-2 assay (target sequences in the SARS-CoV-2 RdRp and N genes of the SARS-CoV-2 genome) was used as a reference test. Study participants were provided with the results of the South African Health Products Regulatory Authority (SAHPRA) approved rapid antigen test on site and Abbott RT-PCR results were reported within 24h of sample collection.

Sample collection and processing.

Study participants provided three swabs: one nasal and two NP. Nasal swab was collected first in order to avoid cross-contamination between sites, followed by a NP swab for second rapid antigen test, followed by the NP swab for SARS-CoV-2 RT-PCR. Swabs were collected and rapid antigen tests were performed by trained medical staff on site. Swabs for RT-PCR were sent to the central laboratory at room temperature without additives. Swab was resuspended in 2ml of viral transport media (VTM) and processed within 3 hours of sample collection. All assays were performed as per manufacturer’s protocol.

Statistical analysis

Statistical analysis was performed using SPSS version 27 and GraphPad Prism version 8.3.1 (GraphPad software, La Jolla, CA). Test performance characteristics were calculated in reference to Abbott RealTime SARS-CoV-2 assay results. Wilson’s score method was used to calculate the 95% confidence intervals to assess the level of uncertainty induced by sample size. The measure of agreement between the assays was evaluated using Cohen’s Kappa coefficient [7]. A t-test was used to assess differences in Ct values between true positive and false negative results.

Study sample characteristics

The evaluation of nasal Panbio and NP Espline Ag tests was done on 494 samples (Table 1) between 21st of July and 19th of August 2021. The median age of study participants was 34 [interquartile range (IQR) 24–47] with 57.29% of participants being female. The SARS-CoV-2 positivity was 31.00% with median Ct value of 9.06 (IQR 5.90–16.90). Majority of the study participants (57.36%) presented for testing during the first week post symptom onset.

Table 1

Study participant/sample characteristics
	Evaluation 1	Evaluation 2
Sample size (N)	494	539
Age, years (median, IQR)	34, 24–47	36, 24–50
Gender (%, n/N Female)	57.29, 283/494	51.76, 279/539
% Positivity (n/N)	31.00, 153/494	13.17 (71/539)
Presence of symptoms
Asymptomatic/presymptomatic (%, n/N)	33.33, 129/387	29.50, 159/539
<7 days PSO (%, n/N)	57.36, 222/387	64.01, 345/539
=>7 days PSO (%, n/N)	9.30, 36/387	6.49, 35/539
HIV positive (%, n/N)^#	0.42, 2/481	0.20, 1/539
Ct value (median, IQR)	9.06, 5.90–16.90	15.08, 11.53–23.86
Oxygen saturation (median, IQR)	96, 95–98	96, 95–97
Missing data: Evaluation 1 (HIV status: 13, Oxygen saturation: 72, Days of symptom onset: 107); Evaluation 2 (Oxygen saturation:26). PSO – post-symptom onset.
^#self-reported

The evaluation of NP Standard Q and nasal RightSign Ag tests was done on 539 samples (Table 1) collected between 15th of September and 8th of December 2021. The median age of study participants was 36 (IQR 24–50) with 51.76% being female. The SARS-CoV-2 positivity was 13.17% with a median Ct value of 15.08 (IQR 11.53–23.86). Majority of the study participants presented for testing during first week post symptom onset (64.01%).

Test Performance Evaluation

The overall performance of nasal Panbio and NP Espline Ag tests is summarized in Sup. Table 1 and Fig. 1. The overall sensitivity and specificity of nasal PanBio Ag test was 67.97% (95% CI 60.22–74.85) and 98.53% (95% CI 96.61–99.37), respectively. The overall performance of NP Espline Ag tests was slightly higher with overall sensitivity and specificity of 72.00% (95% CI 64.33–78.67) and 99.71% (95% CI 98.35–99.95). The sensitivity of both tests increased in samples with lower Ct values: for samples with Ct < 20 sensitivity of nasal PanBio Ag test was 80.16% (95% CI 72.35–86.18) and sensitivity of NP Espline Ag test was 82.11% (95% CI 74.40-87.88). Both tests performed best in samples from individuals presenting during first week of symptom onset, with sensitivities of 74.16% (95% CI 64.20-82.12) for Panbio and 76.14% (95% CI 66.26–83.83) for Espline Ag tests.

The performance of NP Standard Q and nasal RightSign Ag test is summarized in Sup. Table 2 and Fig. 1. The overall sensitivity and specificity of NP Standard Q test was 60.56% (95% CI 48.94–71.11) and 99.79% (95% CI 98.80-99.96), respectively. The overall sensitivity of nasal RightSign Ag test was 63.38% (95% CI 51.76–73.63) with specificity of 100.00% (95% CI 99.19–100.00). The sensitivity of both tests increased in samples with lower Ct values, being 87.23% (95% CI 74.83–94.02) for both Standard Q and RightSign Ag test in samples with Ct < 20.

The kappa coefficient ranged between 0.72 and 0.78 indicating substantial agreement between the tested RDTs and the Abbott RT-PCR assay. As expected, majority of false negative (FN) results for all 4 tests occurred in samples with higher Ct values (Sup. Figure 1). There were 4 invalid test results with Espline Ag test (0.8%), with no invalid results for the remaining three tests.

In this study, we determined the performance characteristics of four RDTs for the detection of SARS-CoV-2 virus in respiratory specimens during the Delta wave of infections in South Africa.

The sensitivity of the four evaluated tests ranged from 60.55%-87.23% with high specificity (ranging from 83.33%-100%). The observed sensitivity of the evaluated tests is comparable with previously published studies from different settings [2, 8]. As reported previously, we observed similar results between the nasal and nasopharyngeal assays performed on equivalent samples [9, 10]. The sensitivity of each of the RDTs increased in samples with lower Ct values, increasing above 80% in samples with Ct <20. As previously reported all four tests performed best in samples from individuals presenting within the first week of symptom onset when the SARS-CoV-2 viral load is highest [11-14]. While the rapid antigen tests have often been criticized for low sensitivity and high rate of false negative results, the majority of the false negative results are observed in samples with higher Ct values/ low viral load that likely have a limited potential for fulling further viral transmission [2, 11, 15, 16]. In fact, antigen-based RDTs were shown to correlate better with replication-competent SARS-CoV-2 compared to RT-PCR [17] further supporting the use of rapid antigen tests in identifying individuals who are at high potential to transmit SARS-CoV-2. Additionally, low cost and scalability represent an important advantage over standard RT-PCR tests, especially in low- and middle-income countries.

There are several limitations to our study. We did not have access to participant vaccination status and presence and timing of previous natural infections, and we therefore could not assess the impact of previous immunity on SARS-CoV-2 viral load and rapid antigen test performance. Even though viral sequencing was not available in our study, the period overlaps with the Delta wave of infections in KwaZulu Natal [18]. While none of the evaluated tests satisfied the WHO requirements for the >80% sensitivity in the overall sample group, they are still a valuable tool in identifying infected individuals within the first week of symptom onset and those with high viral loads and could play an important role in limiting transmission and controlling the COVID-19 pandemic. This study provides valuable information of the performance of rapid antigen tests in drive-through centers in South Africa.

Ethical approval and consent to participate: All study participants provided written informed consent. KwaZulu-Natal Biomedical Research Ethics Committee approved the study (BREC approval No: BREC/00001195/2020 and BREC/00003106/2021).

Consent for publication: Not applicable.

Availability of supporting data: De-identified patient-level data can be accessed by contacting the corresponding author with a detailed description of the research question.

Competing interest:The authors declare that they have no competing interests.

Funding: This project was funded as part of FIND's work as co-convener of the diagnostics pillar of the Access to COVID-19 Tools (ACT) Accelerator, including support from The Swiss Agency for Development and Cooperation [grant number 81066910]. Research presented here was partly funded by the European & Developing Countries Clinical Trials partnership (EDCTP): RIA2020EF-2928 - Mobilisation of research funds for COVID-19 as a Public Health Emergency and the South African Department of Science and Innovation (DSI) Top-up funding to strengthen EDCTP COVID-19 regional networks.

Author contributions:

Designed the study: AS, NS;

Performed the experiments: AS, NS, SN2, TGM, SR, SNM;

Analysed the data: AS, GL, NS, TGM, LL;

Wrote the first draft of the paper: AS, NS, SN1, GL;

Supervised clinical and/or experimental aspects of the study: AS, CC, KN, ABMK, NS, SAK, QAK;

All authors contributed to the editing and finalisation of the manuscript.

Acknowledgments:The authors would like to thank all the study participants for their participation and the District Health Services and frontline healthcare providers for their ongoing support throughout the study. A special thanks to the study staff for the field work, laboratory analysis and for their dedication to these studies.

Baxter C, Abdool Karim Q, Abdool Karim SS. Identifying SARS-CoV-2 infections in South Africa: Balancing public health imperatives with saving lives. Biochem Biophys Res Commun. 2021;538:221–5.
Dinnes J, et al., Rapid, point-of‐care antigen and molecular‐based tests for diagnosis of SARS‐CoV‐2 infection. Cochrane database of systematic reviews, 2021(3).
SAHPRA. Medical Devices and in vitro diagnostics test kits. 2022 [cited 2022 30 October 2022]; Available from: https://www.sahpra.org.za/medical-devices/medical-devices-and-in-vitro-diagnostics-test-kits/.
Organization WH. Advice on the use of point-of-care immunodiagnostic tests for COVID-19: scientific brief, 8 April 2020. World Health Organization; 2020.
Krueger LJ, et al., Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2. medRxiv, 2020.
Denkinger CM, et al. Promise versus reality: optimism bias in package inserts for tuberculosis diagnostics. J Clin Microbiol. 2012;50(7):2455–61.
Landis JR, Koch GG, The measurement of observer agreement for categorical data. biometrics, 1977: p. 159–174.
Akingba OL, et al., Field performance evaluation of the PanBio rapid SARS-CoV-2 antigen assay in an epidemic driven by the B.1.351 variant in the Eastern Cape, South Africa. Journal of Clinical Virology Plus, 2021. 1(1–2).
Lindner AK, et al., Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected nasal swab versus professional-collected nasopharyngeal swab. European Respiratory Journal, 2021. 57(4).
Samsunder N, et al. Clinical Evaluation of Severe Acute Respiratory Syndrome Coronavirus 2 Rapid Antigen Tests During the Omicron Wave in South Africa. J Infect Dis. 2022;226(8):1412–7.
Cevik M, et al. SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis. The lancet microbe. 2021;2(1):e13–22.
Wyllie AL, et al. Saliva or nasopharyngeal swab specimens for detection of SARS-CoV-2. N Engl J Med. 2020;383(13):1283–6.
Zou L, et al. SARS-CoV-2 viral load in upper respiratory specimens of infected patients. N Engl J Med. 2020;382(12):1177–9.
Killingley B, et al. Safety, tolerability and viral kinetics during SARS-CoV-2 human challenge in young adults. Nat Med. 2022;28(5):1031–41.
van Kampen JJ, et al., Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants. MedRxiv, 2020.
Paul G, et al., Klinische Performance eines neuen SARS-CoV-2-Antigen-Tests in der Notaufnahme eines Maximalversorgers. 2020.
Pekosz A, et al. Antigen-Based Testing but Not Real-Time Polymerase Chain Reaction Correlates With Severe Acute Respiratory Syndrome Coronavirus 2 Viral Culture. Clin Infect Dis. 2021;73(9):e2861–6.
Tegally H, et al. Rapid replacement of the Beta variant by the Delta variant in South Africa. MedRxiv; 2021.

Supplementarymaterial.docx

Download PDF

Journal Publication

published 25 Jul, 2023

Read the published version in Diagnostic and Prognostic Research →

Editorial decision: Major revision
09 Mar, 2023
Reviewers agreed at journal
24 Jan, 2023
Reviewers invited by journal
24 Jan, 2023
Editor assigned by journal
12 Dec, 2022
First submitted to journal
12 Dec, 2022

You are reading this latest preprint version

Field evaluations of four SARS-CoV-2 rapid antigen tests during SARS-CoV-2 Delta variant wave in South Africa

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

Study participant recruitment

Statistical analysis

Results

Study sample characteristics

Test Performance Evaluation

Discussion

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1