Evaluation of the Feasibility and Efficacy of Point-of-Care Antibody Tests for Biomarker-Guided Management of Coronavirus Disease 2019 Free

Abstract

Current National Institutes of Health coronavirus disease 2019 (COVID-19) treatment guidelines are partially based on assessments of systemic inflammation [1], but other biomarkers may prove useful for patient management. For example, the Randomized Evaluation of COVID-19 Therapy (RECOVERY) trial found that casirivimab-imdevimab reduced 28-day mortality among those without severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibodies at baseline, but not among those with antibodies [2]. Similarly, the bamlanivimab substudy of the Therapeutics for Inpatients with COVID-19 (TICO) trial found an interaction between treatment group and baseline antibody status for the primary outcome of time to sustained recovery [3]. These studies found that while there may be a benefit from neutralizing monoclonal antibodies for patients who lack antibodies, this is unlikely for patients with antibodies. Moreover, among those in the placebo group of the bamlanivimab substudy of TICO, 96% of those with antibodies at baseline achieved sustained recovery by day 90 compared to 85% of those without antibodies. Thus, regardless of treatment differences, there is evidence that those without baseline antibodies have less favorable outcomes. Awareness of antibody status could provide prognostic data that may alter clinical management. An early review article has discussed possible roles for SARS-CoV-2 antibody tests in patient management [4], but the focus there was on vaccination.

Here we present the results of a field trial and a nested case-control study to assess the feasibility and efficacy of the use of SARS-CoV-2 antibody levels for patient management. While the RECOVERY and TICO trials used antibody tests run at a central laboratory, point-of-care (POC) tests provide a more practical approach to antibody determination. Two POC platforms were investigated: LumiraDx (a fluorescence immunoassay) and RightSign (a lateral flow assay). Both platforms use a capillary fingerstick sample, have a US Food and Drug Administration Emergency Use Authorization and a CE (European Conformity) mark, and test for antibody binding to the receptor-binding domain (RBD) of the SARS-CoV-2 spike protein, although the LumiraDx assay also tests for non-RBD spike protein. A comprehensive review article provides further information on the large number of SARS-CoV-2 antibody tests available [5]. While vaccination and infection history provide information that is likely informative about antibody positivity, incomplete protection and waning immunity potentially limit the usefulness of this information. Nonetheless, vaccination and infection history could serve as a proxy for the presence of antibodies.

This study's primary objective was to compare the POC tests to the centrally conducted antibody assay used in the TICO trial (the GenScript assay). Secondary objectives included the feasibility of the POC tests, comparisons to other assays, and detecting an association between POC test results from stored specimens from TICO and safety and efficacy outcomes of interest in TICO among those in the placebo group from a subset of TICO trials (ie, trials of bamlanivimab [6], sotrovimab, and amubarvimab-romlusevimab [7]).

METHODS

Field Trial Participants and Clinical Data

Eighteen sites in Denmark, Greece, Singapore, and the United States participated. Each site obtained institutional review board approval prior to recruitment and participants provided written informed consent. Eligibility criteria for the field trial were broad and similar to those used for the TICO trial. In brief, participants were adults hospitalized for management of SARS-CoV-2 infection with a positive molecular test for SARS-CoV-2 infection within 3 days and symptoms no more than 12 days prior to consent. Exclusion criteria included recent prior use of a neutralizing monoclonal antibody or other immune-based therapy or severe (eg, organ failure) clinical manifestations potentially attributable to COVID-19; full eligibility criteria and the complete study protocol are provided in the Supplementary Materials.

Clinical sites collected information on demographics, clinical status, medical history, vaccination status, and current medications. There was a single question about race/ethnicity with the options Asian, Black, Hispanic, other, and White, with the potential to select >1 category. If >1 category was selected, a single category was assigned based on the order of race/ethnicity provided in the previous sentence (eg, Black Hispanic was classified as Black); this was how race and ethnicity were handled in the TICO trials. The outcomes of the POC tests and questions for the clinical staff regarding ease of use of the tests were also captured. Up to 2 of each of the POC tests were performed on each participant and a blood draw was conducted for central determination of SARS-CoV-2 antibody levels.

Nested Case-Control Study Participants and TICO Outcomes

The TICO platform trial tested multiple therapeutics for COVID-19 inpatients using a randomized, double-blind, placebo-controlled design. Participants in the TICO trial consented to the use of their specimens for future research on COVID-19. Eligibility criteria were like those listed for the field trial with the date of randomization replacing the date of consent. The primary efficacy outcome in TICO was time to sustained recovery, which was defined as the first time after randomization at which the participant returned to the prehospitalization residence and stayed there for at least 14 days. All-cause mortality through 90 days was a secondary efficacy endpoint. The primary safety endpoint was a composite of death, serious adverse events, grade 3 or 4 adverse events, clinical organ failure, or serious infection (Supplementary Table 1) evaluated at day 5, but this endpoint was also investigated at days 28 and 90 (all are considered here). Cases and controls were selected from participants in the placebo arm of the first 3 trials without any matching. The case definition was death or time to sustained recovery that exceeded 28 days, while controls had time to sustained recovery of <17 days. A power calculation found that 145 participants should provide 80% power for detecting a subhazard ratio of 1.75. Data from 126 participants were used to assess the objectives of the nested case-control study. Thresholds of 17 and 28 days were selected to meet the sample size requirement.

Antibody and Antigen Assays

In addition to the 2 POC tests that were examined, 3 other centrally conducted antibody tests were performed on specimens collected from participants in this study. Levels of neutralizing antibodies directed against the SARS-CoV-2 RBD of the spike protein were determined using the GenScript SARS-CoV-2 surrogate virus neutralization assay (GenScript cPass assay, GenScript, Piscataway, New Jersey). Neutralizing antibodies were expressed as percent binding inhibition; levels >30% were considered positive as recommended by the manufacturer. The level of anti-nucleocapsid (anti-N) antibody was measured using the Bio-Rad Platelia SARS-CoV-2 total antibody assay (Bio-Rad, Hercules, California). A signal-to-noise ratio >1 was considered positive (per manufacturer instruction). The Simoa semi-quantitative SARS-CoV-2 immunoglobulin G (IgG) antibody test (Quanterix, Billerica, Massachusetts) was used to detect anti-spike (anti-S) IgG antibodies (ng/mL). A positive cutoff for this assay was set to 770 ng/mL. Quantitative plasma SARS-CoV-2 nucleocapsid antigen (Ag) levels were measured using a microbead-based immunoassay (Quanterix). All centrally conducted assays were performed blinded to all participant characteristics and the outcomes of the POC tests.

Statistical Methods

Field Trial

Comparisons of test results were conducted using McNemar χ² test restricted to samples that were either positive or negative and supplemented with 95% confidence intervals (CIs) for the difference in the probability of a positive test. The level of agreement between 2 assays was quantified using the proportion of samples for which the assays agreed (also restricted to positive or negative results). Adjusted analyses were conducted using generalized estimating equations (GEEs) with a logistic link that adjusted for site, sex, age, immunocompromised status, and vaccination status (with clustering by participant and an exchangeable correlation structure). Subgroup analyses were conducted using GEEs with tests for interaction for each of the covariates.

Case-Control Study

The Fine-Gray approach to competing risk regression was used to test for an association between the POC test result and time to sustained recovery. The association between POC test results and time to death was investigated using Cox models. The association between baseline POC test results and the primary safety composite at days 5, 28, and 90 was investigated using logistic regression models. All of these analyses controlled for parent substudy and were also conducted while controlling for age, baseline oxygen requirements, and baseline Ag levels on the logarithmic scale. A significance level of .05 was used and analyses were conducted using SAS version 9.0.

RESULTS

Field Trial

For the field trial, 330 participants were consented and provided specimens between 29 April 2022 and 2 September 2022. The median age was 67 years (interquartile range [IQR], 54–77 years) with 150 (46%) participants 70 years or older (Table 1). Most (54%) were men, and the most common race/ethnicity was White (221 [67%]), with 82 (25%) being Black or Hispanic/Latino. Seventy-two percent of participants had at least 2 vaccine doses 14 or more days prior to the onset of symptoms. The median time from final vaccine dose to symptom onset was 236 days (IQR, 171–352 days). Thirty-one (9%) participants had an immune disorder other than human immunodeficiency virus (HIV), and 5 (2%) participants had HIV. Only 97 (29%) were on supplemental oxygen at baseline.

The LumiraDx POC test was not conducted on 12 participants, and among those for whom the test was performed at least once, 196 (62%) were positive, 34 (11%) were negative, and 88 (28%) were invalid on the first test (Supplementary Table 2). The most common reasons for invalid tests were “insufficient sample volume or other problem” (n = 43) and “hematocrit out of range” (n = 35). Sites were instructed to repeat invalid tests 1 time; however, of the 88 invalid test results, 22 (25%) were not redone, 30 (34%) were positive, 3 (3%) were negative, and 33 (38%) were invalid on the second test too. A comparison of the locally conducted test to the test run at a central facility found strong agreement (99% agreement and McNemar P = .56), with most of the results that were invalid locally being classified as positive (Supplementary Table 3). There were no invalid results on the centrally determined test because the central laboratory used plasma samples, thereby ensuring sufficient sample volume and no hematocrit issues. When only using results that were positive or negative after 2 attempts, 86% of the samples were positive.

The RightSign POC test was not conducted on 3 participants, and among those for whom the test was performed at least once, 238 (73%) were positive, 88 (27%) were negative, and 1 (0.3%) was invalid on the first test (Supplementary Table 4). The 1 invalid result was repeated and found to be positive. Almost all (225 [94%]) of the positive results (on first or second test) were positive for IgG only, with 13 (6%) positive for IgG and immunoglobulin M (IgM) and 1 (0.4%) positive for IgM only. The level of agreement between the locally and centrally conducted tests was high (91% agreement and McNemar P = .27) but lower than what was found for the LumiraDx test (P < .001 for the difference in the level of agreement; Supplementary Table 3). When only using results that were positive or negative after 2 attempts, 73% of the samples were positive.

The RightSign assay showed greater agreement with self-reported vaccination status than the LumiraDx assay (95% CI for difference, −3.5% to 6.6%, P = .55 for RightSign; and 8.6%–18.8%, P < .001 for LumiraDx) (Supplementary Table 5).

Comparison of Test Results

The locally conducted POC tests were significantly different from the centrally conducted antibody assays except for the same POC test run centrally (Table 2). This included the GenScript assay (P < .001 for LumiraDx and P = .046 for RightSign, with both comparisons at 89% agreement). The Bio-Rad test, which measures the anti-N antibody response rather than anti-S antibody response, showed the largest difference with the POC tests (31% and 44% agreement with LumiraDx and RightSign, respectively). With GenScript as the gold standard, the sensitivities of the 2 locally conducted assays were 99.5% and 89.5% with specificities of 58.1% and 84.0% for the LumiraDx and RightSign assays, respectively. The 2 locally conducted tests exhibited a statistically significant difference from each other (P < .001; Supplementary Table 6).

Adjusting for covariates and their interactions with test type found that the main effect for test type (ie, LumiraDx vs RightSign) was not a significant predictor of test outcome. However, being immunocompromised was associated with a lower likelihood of a positive test; being vaccinated was associated with a higher likelihood of a positive test; and there was an interaction between test type and vaccination status, with the LumiraDx test more likely to be positive among vaccinated participants (Supplementary Table 7).

Staff found the RightSign assay easier to use. Ninety-two percent of those reporting on ease of use described the RightSign assay as very easy to use, while in contrast 62% described the LumiraDx assay as very easy to use (P < .001 for difference; Supplementary Table 8). The 2 tests took approximately the same amount of time to administer.

Nested Case-Control Study

The nested case-control study used specimens from 126 participants in the TICO trial recruited from 32 sites in Denmark, Poland, Switzerland, and the United States between 12 August 2020 and 26 February 2021 (65 cases and 61 controls). Among these participants, 95 experienced sustained recovery and 21 died. As seen in Table 3, the cases and controls were similar in terms of trial enrolled, country of origin, sex, race/ethnicity, body mass index, and days since symptom onset; however, there were differences in terms of age (P = .002), and cases had greater oxygen requirements at baseline (P < .001). Among cases, 12 (19%) had no requirement for supplemental oxygen compared to 30 (49%) among controls, and 9 (14%) cases were receiving noninvasive ventilation or high-flow nasal canula compared to 0 controls.

Cases were less likely to test positive for SARS-CoV-2 antibody and had higher levels of Ag. As seen in Supplementary Table 9, based on the LumiraDx antibody assay, 33 (51%) cases tested positive for antibody compared to 45 (74%) controls. Similarly, based on the RightSign antibody assay, 29 (45%) cases tested positive compared to 43 (71%) controls. The GenScript assay found that 19 (29%) cases tested positive compared to 34 (56%) controls. Using a cutoff of 1000 ng/L for the Quanterix Ag assay [8], 48 (74%) cases were positive compared to 17 (28%) controls.

All TICO outcomes were found to be associated with LumiraDx results when stratified by parent substudy. Sixty-six (85%) of those who were antibody positive at baseline had a sustained recovery compared to 29 (60%) of those who were antibody negative (Table 4). The recovery rate ratio for the time to sustained recovery was 1.89 (95% CI, 1.24–2.89; P = .003) in favor of those who were antibody positive at baseline. There were 7 (9%) deaths among those who were antibody positive compared to 14 (29%) among those who were antibody negative (95% CI for hazard ratio [HR], .10–.63; P = .003). Finally, the primary safety endpoint differed between those who were antibody positive and antibody negative (95% CI for odds ratio [OR], .18–1.00; P = .049), as did this endpoint evaluated at days 28 and 90.

All TICO outcomes were also found to be associated with RightSign results when stratified by parent substudy. Fifty-nine (82%) of those who were antibody positive at baseline had a sustained recovery compared to 36 (67%) of those who were antibody negative (Table 4). The recovery rate ratio for the time to sustained recovery was 1.65 (95% CI, 1.13–2.41; P = .009) in favor of those who were antibody positive at baseline. There were 7 (10%) deaths among those who were antibody positive compared to 14 (26%) among those who were antibody negative (95% CI for HR, .13–.82; P = .016). Finally, the primary safety endpoint again differed between those who were antibody positive and antibody negative (95% CI for OR, .14–.78; P = .012), as did this endpoint evaluated at days 28 and 90. For both tests, further adjustments largely found the same results with some minor differences when adjustments by age, oxygen requirements, or Ag were performed (Table 4 and Supplementary Table 10).

DISCUSSION

Both POC tests were significantly different from the GenScript assay and showed moderate agreement with this and other centrally conducted tests that interrogated the anti-S SARS-CoV-2 antibody response. Treating the GenScript assay as the gold standard, the LumiraDx assay had high sensitivity and low specificity compared to the RightSign assay. Site staff found the RightSign assay easier to use, in part because it required a smaller volume of blood. There were logistical challenges associated with the use of the LumiraDx assay in the context of a clinical research study. The assay relies on a laboratory instrument that needs to connect to the internet, and this frequently created data security concerns, which led to delays in opening sites. Moreover, the assay frequently failed to provide a result. In contrast, the RightSign assay relied on a simple test cartridge. The nested case-control analysis revealed that both POC tests were predictive of time to sustained recovery, mortality, and safety endpoints in the TICO study.

Due to excess risk of morbidity and mortality from lack of antibody, patient management may benefit from identification of patients without antibody. This requires an antibody test that is highly specific. If one treats the GenScript assay as the gold standard, then the RightSign test is preferable due to its higher specificity. There is ongoing work to develop a quantitative version of the LumiraDx assay, and perhaps this could result in an assay with higher specificity (and likely lower sensitivity). These findings suggest that a role for antibody tests in the management of COVID-19 merits further investigation.

Vaccination and infection history provides another method for assessing if a patient has antibody. Both POC assays provide estimates of the proportion of patients who are antibody positive that are greater than self-reported vaccination status. In the context of a patient population that had been recently vaccinated, a reasonable triage strategy would use vaccination status. This strategy would miss 14% of those without an antibody response based on the RightSign assay (ie, 14% of those vaccinated are antibody negative). Moreover, 41% of those who would be triaged based on vaccination status would in fact have antibody based on the RightSign assay (4% would have been missed, and 60% of those triaged would have antibody using the LumiraDx assay). This amounts to a substantial amount of misallocation of resources for patient triage.

This study had several limitations. There are hundreds of SARS-CoV-2 POC antibody assays commercially available, and 2 were selected based on largely incomplete reports. This likely led to the selection of suboptimal assays; however, the 2 assays selected spanned the range of possibilities. The field trial did not collect data on participant outcomes. While this greatly simplified protocol implementation, it limited our ability to understand outcomes in the context of a patient population that was heavily vaccinated and infected with the Omicron variant. Moreover, the RightSign assay exclusively interrogates the RBD of the SARS-CoV-2 spike protein and so may lack sensitivity in a patient population infected by Omicron sublineages, such as the patient population studied here. Participants in the field trial had low oxygen requirements limiting generalizability. While the nested case-control component of the protocol had rigorously collected outcome data, the patient population was not vaccinated, limiting our ability to assess the joint effect of antibody determination and vaccination status on outcomes. We were nonetheless able to investigate concordance between self-reported vaccination status and POC test results and found that there is a substantial enough divergence between the 2 approaches to risk assessment to conclude that there is a potential role for SARS-CoV-2 POC antibody testing in patient management.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online (http://jid.oxfordjournals.org/). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.

Notes

Disclaimer. The content of this publication does not necessarily reflect the views or policies of the US Department of Health and Human Services or the US government, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government.

Financial support. This work was supported by the National Institutes of Health (NIH) (agreement number 1OT2HL156812-01) and the National Institute of Allergy and Infectious Diseases (NIAID), NIH, in part with federal funds from the NIAID and the National Cancer Institute, NIH (contract number 75N91019D00024; task order numbers 75N91020F00014 and 75N91020F00039).

References

Author notes