Analysis of 2 Reverse Syphilis Testing Algorithms in Diagnosis of Syphilis: A Large-Cohort Prospective Study Free

Abstract

Background

Two serologic syphilis screening algorithms recommended by the US Centers for Disease Control and Prevention (US CDC) and the European Centre for Disease Prevention and Control (ECDC), respectively, are commonly used for syphilis screening; however, which one is optimal remains to be determined.

Methods

We conducted a prospective study of 119891 subjects to analyze the consistency of the US CDC– and ECDC-recommended algorithms. The US CDC–recommended algorithm begins with a treponemal immunoassay, followed by a rapid plasma reagin (RPR) test. RPR-nonreactive samples are confirmed by the Treponema pallidum particle agglutination assay (TPPA). The ECDC-recommended algorithm begins with a treponemal immunoassay, followed by a confirmatory treponemal test. If the confirmatory test is reactive, a quantitative nontreponemal assay is used to assess the disease activity and treatment response. In the present study, a total of 119891 serum samples from a large hospital (sixth largest in China) were included, and each sample was screened with a chemiluminescent immunoassay (CIA). CIA-reactive samples were then simultaneously tested with RPR and TPPA. The consistency of these 2 algorithms was determined by calculating the percentage of agreement and κ coefficient.

Results

The overall percentage of agreement and κ value between these 2 algorithms were 99.996% and 0.999, respectively. The positivity rate for syphilis as determined by the US CDC– and ECDC-recommended algorithms was 1.43% and 1.42%, respectively.

Conclusions

Our results suggest that the US CDC–recommended algorithm and the ECDC-recommended algorithm have comparable performances for syphilis screening in low-prevalence populations.

Syphilis poses a great threat to human health worldwide. The majority of syphilis cases occur in underdeveloped countries, particularly in sub-Saharan Africa and Asia [1]. The syphilis epidemic has also occurred among men who have sex with men in European countries and North America [2]. Moreover, about 10 million new infections appear each year [3]. Thus, it has become increasingly important to apply a better strategy to diagnose, treat, and prevent syphilis.

As the causative agent of syphilis, Treponema pallidum is a bacterium that cannot be detected with simple laboratory staining assays. The diagnosis of syphilis primarily relies on serologic tests, as patients naturally infected with T. pallidum are often characterized by periods without clinical manifestations [4]. Two types of serologic tests are used for the diagnosis of syphilis: treponemal and nontreponemal tests. The former includes chemiluminescent immunoassays (CIA), enzyme immunoassays (EIA), and the T. pallidum particle agglutination assay (TPPA), while the latter includes the rapid plasma reagin (RPR) and Venereal Disease Research Laboratory tests. Treponemal tests are primarily used to identify the presence of syphilis infection, whereas nontreponemal tests are largely used to monitor the infection status and treatment response [5].

Two common algorithms exist for serological diagnosis of syphilis: the traditional and reverse algorithms [6]. In the traditional algorithm, syphilis serologic screening begins with a nontreponemal test, followed by confirmation (if reactive in the nontreponemal test) using 1 of several treponemal tests, such as TPPA [7, 8]. However, this algorithm has several limitations, including lack of specificity, low sensitivity, manual operation, and subjective interpretation of results [9, 10]. To reduce the time and cost required for syphilis screening, more laboratories are adopting a reverse algorithm that begins with a treponemal assay [6, 7, 9–11]. There are 2 implementation schemes for this reverse algorithm. The first one is recommended by the US Centers for Disease Control and Prevention (US CDC), referred to as the “US CDC–recommended algorithm” hereinafter. This scheme involves an initial reactive treponemal screening assay such as CIA, followed by a quantitative nontreponemal assay such as RPR [9, 12]. If the initial and subsequent test results disagree, the specimen is then tested with a second treponemal test (TPPA) [7]. The second scheme is recommended by the European Centre for Disease Prevention and Control (ECDC), and hereinafter referred to as the “ECDC-recommended algorithm.” It involves a reactive treponemal assay, followed by a second, different treponemal assay as a confirmatory test. If the confirmatory test is reactive, a quantitative nontreponemal assay is used to assess the disease activity and treatment response [13]. However, it remains controversial which implementation scheme of the reverse algorithms is better for the diagnosis of syphilis.

This study evaluated the performances of the US CDC– and ECDC-recommended algorithms in the diagnosis of syphilis, as well as explored the necessity of reviewing clinical records and serological follow-up among subjects with CIA-reactive (CIA⁺), RPR-nonreactive (RPR^–), TPPA-nonreactive (TPPA^–) (isolated CIA reactive) serum. Our ultimate goal was to help in the selection of the optimal syphilis serodiagnostic algorithm and improve the diagnostic efficiency for syphilis.

METHODS

Study Design and Ethics Statement

To evaluate the performances of the US CDC– and ECDC-recommended algorithms in the diagnosis of syphilis, we carried out a prospective study in Tongji Hospital of Tongji Medical College of Huazhong University of Science and Technology, Wuhan, China, from January 2015 to January 2016. The subjects of this study included outpatients, inpatients, and populations undergoing routine health examinations. They underwent syphilis testing for screening (if asymptomatic), for diagnosis (if symptomatic), or for monitoring the response to syphilis treatment. Patient groups used in the study are summarized in Figure 1. Subjects were first screened for antibodies against T. pallidum using the Architect syphilis TP assay (a CIA) (Abbott Diagnostics, Abbott Park, Illinois). When determined to be reactive by this assay, the specimens were then tested with TPPA (Fujirebio, Tokyo, Japan) and RPR (KHB, Shanghai, China) (Figure 2). CIA⁺, RPR-reactive (RPR⁺), and TPPA^– specimens were retested simultaneously with the CIA, RPR, and TPPA, as a “rare serological profile.” The consistency of the 2 reverse algorithms for syphilis serodiagnosis was analyzed.

Following the above assays, we collected clinical information via the electronic medical record on patient samples that were CIA⁺ RPR^–, as well as samples that were CIA⁺, RPR⁺, but TPPA^–. For subjects without syphilis-related information, we carried out telephone interview and follow-up syphilis serologic testing. Individuals with duplicate records can be identified via the electronic medical record.

The analytical sensitivity of TPPA was determined by performing serial dilution of treponemal antibody–containing sera coming from syphilis patients. Each of the dilutions was also measured by the CIA test and given a signal-to-cutoff (S/CO) ratio.

Ethical approval covering the study protocol was obtained from the Human Ethics Committee of Tongji Medical College.

Clinical Diagnosis of Syphilis

According to the International Union Against Sexually Transmitted Infection 2014 guidelines [14], a sample was considered syphilis positive when both CIA and TPPA were reactive. However, CIA⁺TPPA⁻ subjects showing prior or subsequent clinical or serological evidence of syphilis were also diagnosed as syphilis positive [14, 15].

Data Analysis

Statistical analyses were performed using SPSS version 19 for Windows (IBM, Chicago, Illinois). The percentage of agreement and κ coefficient were calculated to determine the agreement between the US CDC– and ECDC-recommended algorithms. The agreement of the results (κ value) was categorized as near perfect (0.81–1.0), substantial (0.61–0.8), moderate (0.41–0.6), fair (0.21–0.4), slight (0–0.2), or poor (<0) [16].

RESULTS

A Summary of Syphilis Testing Results

A total of 119891 serum samples were included in this study. Among these samples, 2266 (1.89%) were CIA⁺. A subsequent testing of CIA⁺ samples revealed that 721 (31.81%) were RPR⁺, and 1545 (68.18%) were RPR⁻. Of the 721 CIA⁺RPR⁺ samples, 715 (99.17%), 1 (0.14%), and 5 (0.69%) were determined to be TPPA reactive (TPPA⁺), indeterminate, and TPPA⁻, respectively. Of the 1545 CIA⁺RPR⁻ samples, 993 (64.27%), 82 (5.31%), and 470 (30.42%) were determined to be TPPA⁺, indeterminate, and TPPA⁻, respectively. The positive rate for syphilis determined by the US CDC– and ECDC-recommended algorithms was 1.43% and 1.42%, respectively, indicating a low prevalence of syphilis in this population (Figure 2).

Diagnostic Consistency of the 2 Testing Algorithms

The syphilis diagnostic flow charts using the US CDC– and ECDC-recommended algorithms are illustrated in Figure 2. The overall percentage of agreement and κ value between the 2 algorithms were 99.996% and 0.999, respectively, indicating a high degree of consistency between these 2 algorithms (Table 1). Despite this finding, 5 CIA⁺TPPA⁻RPR⁺ samples showed discordant results between these 2 algorithms. These 5 subjects were >60 years old. They were considered as having past or present syphilis according to the US CDC–recommended algorithm, but no syphilis based on the ECDC-recommended algorithm. Among them, 2 patients (patients 1–2) had a documented history and previous serological evidence of syphilis, and 1 patient (patient 4) had subsequent serological evidence of syphilis. The other 2 patients (patients 3 and 5) had high CIA S/CO ratios (12.44 and 34.34). Patient 5 had a documented history of syphilis, while patient 3 could not provide any syphilis-associated information due to health problem. Patients 1, 3, and 5 may have contracted nonactive syphilis, as they showed no clear syphilis-associated syndromes. Also, the RPR titers were low (≤1:4), and/or the changes in RPR titers in these patients between previous/future RPR results did not exceed 2 folds. Additionally, when analyzing the results of specimens obtained from different time points from patients 1, 2, and 4, we found that the CIA S/CO ratios were lower (but still CIA⁺) in the TPPA^– specimens (5.58, 5.2, and 1.82, respectively) than those in the TPPA⁺ ones (7.78, 6.03, and 1.95, respectively), suggesting a lower sensitivity of the TPPA compared to the CIA.

Analytical Sensitivity of TPPA

Sera from 20 patients with confirmed syphilis were used to determine the analytical sensitivity of TPPA. These sera were serially diluted and tested by the CIA and TPPA. TPPA was nonreactive in 19 of the 20 syphilitic patients when tested on serum dilutions with critical CIA results (S/CO ratios: 1.00–1.10), indicating a lower analytical sensitivity of TPPA compared to CIA. The representative data are shown in Table 2.

Analysis of the CIA⁺TPPA^–RPR^– Results

Out of the 1545 CIA⁺RPR⁻ samples, 470 samples were determined to be TPPA⁻. For these 470 subjects, 78 had records of serological retests. Of them, 9 (11.54% [9/78]) had clinical or serological evidence of previous or subsequent syphilis (Table 3). This included 7 subjects (patients 1–7) who were CIA⁺TPPA⁺RPR^+/− previously but became CIA⁺TPPA^–RPR^– later, suggesting that treponemal antibodies were present in their sera in the past. Two subjects (patients 1–2) were infants of syphilitic patients, and they seroreverted to CIA-nonreactive (CIA^–) serology during the follow-up study. These results suggest that the 2 infants may have passive antibody transfer from their mothers or congenital infection, or show biological false-positive (BFP) findings in their sera. Patients 3–7 had a history of syphilis, with patient 7 later seroreverting to CIA^– serology but seroconverting to TPPA⁺ serology (CIA^–TPPA⁺). The instability of CIA and TPPA seen in this subject likely results from low titer of treponemal antibodies. While patients 1–7 showed previous serological evidence of syphilis, patients 8 and 9 had no records of prior serology tests for syphilis and did not report any history of syphilis. Surprisingly, patients 8 and 9 seroconverted to TPPA⁺ serology during the follow-up study. Overall, a total of 3 CIA⁺TPPA^–RPR^– subjects (patients 7–9) seroconverted to TPPA⁺ but remained RPR^–. These patients did not develop any signs of syphilis, and their serum samples showed low CIA S/CO ratios (<2), suggesting the presence of low titers of treponemal antibodies in their sera.

DISCUSSION

Our results demonstrated a high consistency between the US CDC–recommended algorithm and the ECDC-recommended algorithm in the serodiagnosis of syphilis in a population with a low prevalence of syphilis (1.40%). Interestingly, the high consistency between these 2 algorithms was also seen in a population with a high prevalence (11.40%) of syphilis [15]. In our study, the US CDC–recommended algorithm detected 5 more syphilis patients than the ECDC-recommended algorithm. Moreover, the US CDC–recommended algorithm is more cost-effective than the ECDC-recommended algorithm (Supplementary Table 1). Based on these findings, we propose that the US CDC– and ECDC-recommended algorithms have similar performances, and the US CDC–recommended algorithm has a slight advantage over the ECDC algorithm for syphilis screening in a population with a low prevalence of syphilis. This is different from the study by Tong et al, where the ECDC-recommended algorithm was recommended, although the US CDC–recommended algorithm also detected 18 more syphilis patients in their study. Interestingly, Tong et al did not include a nontreponemal test (eg, RPR) in the ECDC recommended algorithm to assess disease activity when both screening and confirmatory treponemal assays were reactive, which was strongly recommended in the 2014 European guideline on the management of syphilis [14].

Notably, when applying the US CDC–recommended algorithm, the screening treponemal assay (CIA or EIA) should have higher analytical sensitivity than TPPA, which is designated as the sole confirmatory assay [7]. In our study, a CIA (Architect Syphilis TP assay, Abbott Diagnostics) was used as the screening assay, which has been adopted worldwide due to its high sensitivity, full automation, and high throughput capability [17]. Indeed, we found that Architect TP assay had a lower analytical limit than TPPA (Table 2). In contrast to our study, Tong et al used TPPA as the screening assay and a CIA (Boson Biotechnology, Xiamen, China) as the confirmatory assay. This screening approach was different from the standard US CDC–recommended algorithm (ie, CIA and TPPA were performed sequentially). In fact, if the CIA was used as the screening assay and TPPA as the confirmatory assay, 18 syphilis patients with the TPPA⁺RPR⁺CIA^– result would have been missed in the above study.

The US CDC states that CIA⁺RPR⁺ patients are considered to have past or present syphilis [7]. We found most of the CIA⁺RPR⁺ specimens (99.17% [715/721]) were TPPA⁺, except for 5 CIA⁺RPR⁺ specimens determined to be TPPA^–. However, 4 CIA⁺RPR⁺TPPA^– subjects could have nonactive syphilis based on their clinical syndromes, disease histories, and previous or subsequent serological test results. The remaining CIA⁺RPR⁺TPPA^– subject may also have syphilis, as this patient had a definite syphilis history and high RPR titer (1:16). Surprisingly, TPPA seroreversion was observed for this patient even under strict quality control. However, given that the patient was TPPA⁺ 1 and 2 months prior to the current sampling and seroreversion of TPPA is relatively uncommon within such a short period, we do not rule out the possibility of a technical error in this specimen. Nevertheless, our results reinforce the concept that CIA⁺RPR⁺ patients can be considered to have past or present syphilis.

Studies from others and us have shown discordant results among CIA⁺RPR^– samples. For instance, we found 30.42% (470/1545) CIA⁺RPR^– subjects to be TPPA^–. A US CDC laboratory review showed a TPPA negative rate of 31.6% among CIA/EIA⁺RPR^– samples [7]. Hence, how to properly interpret the CIA⁺RPR^–TPPA^– pattern (isolated reactive immunoassay) remains elusive but vital to clinicians [18, 19]. As highlighted by other groups, misclassifying an isolated reactive screening test as positive could lead to false treatment and mental stress, whereas misclassifying an isolated reactive immunoassay as negative could result in a missed diagnosis of early syphilis, infectious syphilis transmission, and the possibility of untreated latent disease [18, 19]. Understanding the reasons behind these discordant results will undoubtedly benefit from population-based laboratory findings. In line with the current US CDC sexually transmitted disease guidelines, 2 US CDC laboratory reviews concluded that syphilis is unlikely if sera are TPPA^– [7, 20]. Other researchers have attempted to assess the significance of cases with an isolated EIA/CIA+ specimen through assessment of individual cases [21, 22]. Woznicová et al [21] studied a Czech sexual health clinic population, and found that a total of 40% (8/20) of cases with an isolated reactive EIA specimen were classified as latent syphilis, 25% (5/20) had passive antibody transfer from their mother, 15% (3/20) had other infections, and the remaining 20% (4/20) showed no overt clinical factors involved. Another study performed in an Australian population [22] also found that 11 of 20 patients (55%) with an isolated reactive CIA specimen had previous or subsequent evidence of syphilis infection. Similarly, we found that 2 of 9 cases with CIA⁺RPR^–TPPA^– may have passive antibody transfer from their mother, and the other 7 cases had previous or subsequent serological evidence of syphilis infection. It is tempting to propose that isolated reactive CIA specimens may represent true T. pallidum infection and may be identified after seroreversion of traditional treponemal assays (such as TPPA), although it remains plausible that this population is at increased risk of BFP results. Our future work is to address if either/both algorithms would miss patients with primary syphilis, by incorporating immunoglobulin M and polymerase chain reaction assays that have been used to diagnose primary syphilis [14].

Once a patient has been treated for syphilis, treponemal antibodies are assumed to be detectable for life [23]. However, recent studies suggest that this may not be always the case. For instance, treponemal antibody seroreversion is documented in up to 24% of syphilis cases diagnosed by traditional treponemal tests [24]. In line with the seroreversion hypothesis, we identified 7 subjects who were CIA⁺TPPA⁺RPR^+/– previously but became CIA⁺TPPA^–RPR^– later, suggesting that they may have seroreverted on TPPA. The median CIA S/CO ratio for these CIA⁺TPPA^–RPR^– specimens (2.32) was lower than that for the CIA⁺TPPA⁺RPR^+/– specimens (12.37). We also observed complete seroreversion (including seroreversion of the syphilis CIA) in 2 infants of syphilis patients. Obviously, the discovery rate of discordant results between screening and confirmatory assays depends on the prevalence of syphilis [25] and the performance of the detection system. Altogether, our results strongly suggest that detailed medical and serological analysis should be performed for subjects with conflicting screening and confirmatory test results.

While our data will be instrumental for the diagnosis of syphilis, there are a few limitations in our study. For instance, some patients may have used false information (name and age) to protect their privacy, making it impossible to determine their syphilis infection and treatment history. Also, some patients either denied or responded that they were unaware of their infection status. In this case, however, we were able to review the clinical records and confirm some syphilis cases through a documented history of syphilis and/or previous or subsequent serological evidence of syphilis infection, making our conclusions more reliable. Another potential variable in our study is that we assessed the algorithms in a population with low prevalence of syphilis, suggesting that the positive predictive values could be relatively low. This also means that the same algorithms may work slightly different in populations with high prevalence of syphilis, although it merits further investigations.

Overall, our data strongly argue for the idea that the US CDC– and ECDC-recommended algorithms have a high consistency in the serodiagnosis of syphilis in low-prevalence populations. Additionally, for cases with an isolated reactive screening test, detailed clinical and serological assessment is warranted to reduce the diagnostic miss rate for syphilis.

Notes

Author contributions. J. P., Y. F. L., H. B. Y., and Z. Y. S. conceived and designed the experiments. Y. F. L., J. P., and T. T. L. performed the experiments. J. P., Y. F. L., H. B. Y., S. J. W., T. T. L., H. J. L., L. Y. D., and Z. Y. S. analyzed the data. Z. Y. S. contributed reagents, materials, and analysis tools. J. P., Y. F. L., and H. B. Y. contributed to the writing of the manuscript.

Acknowledgments. We thank all the subjects involved in this cohort study.

Financial support. This work was supported by the National Mega Project on Major Infectious Disease Prevention (grant number 2017ZX10103005-007).

Potential conflicts of interest. All authors: No reported conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References

Author notes