Two Clinical Prediction Tools to Inform Rapid Tuberculosis Treatment Decision-making in Children

Abstract

Background

In the absence of bacteriologic confirmation to diagnose tuberculosis (TB) in children, it is suggested that treatment should be initiated when sufficient clinical evidence of disease is available. However, it is unclear what clinical evidence is sufficient to make this decision. To identify children who would benefit from rapid initiation of TB treatment, we developed 2 clinical prediction tools.

Methods

We conducted a secondary analysis of a prospective intensified TB patient–finding intervention conducted in Pakistan in 2014–2016. TB disease was determined through either bacteriologic confirmation or a clinical diagnosis. We derived 2 tools: 1 uses classification and regression tree (CART) analysis to develop decision trees, while the second uses multivariable logistic regression to calculate a risk score.

Results

Of the 5162 and 5074 children included in the CART and prediction score, respectively, 1417 (27.5%) and 1365 (26.9%) were eligible for TB treatment. CART identified abnormal chest radiographs and family history of TB as the most important predictors (area under the receiver operating characteristic curve [AUC], 0.949). The final prediction score model included age group (0–4, 5–9, 10–14), weight <5th percentile, cough, fever, weight loss, chest radiograph suggestive of TB disease, and family history of TB; the identified best cutoff score was 9 (AUC, 0.985%).

Conclusions

Use of clinical evidence was sufficient to accurately identify children who would benefit from treatment initiation. Our tools performed well compared with existing algorithms, though these results need to be externally validated before operationalization.

One million children (0–14 years) are estimated to develop tuberculosis (TB) disease annually [1, 2]. Only between 30% and 60% of children age <15 years are diagnosed and reported though; the rest are missed by the health system [3]. About one-quarter of a million children die from TB annually, and 96% of those deaths occur in children who never received treatment [4].

Children often experience delays in diagnosis or misdiagnosis due to nonspecific symptoms [5] and having paucibacillary TB, limiting the sensitivity of conventional tests [6]. Children are also often unable to produce sputum, precluding any potential microbiological confirmation through sputum-based tests [6]. In the absence of a globally accessible, accurate diagnostic test for TB in children, diagnosis remains largely based on a combination of clinical symptoms, medical history, and radiologic evidence. Improving diagnostic strategies for pediatric TB is a priority; identifying and treating children who would benefit from rapid initiation of TB treatment can be life-saving.

In the absence of microbiological confirmation of disease, the World Health Organization (WHO) and the International Union Against Tuberculosis and Lung Disease [7, 8] recommend treating children for whom there is sufficient clinical evidence of TB disease; however, what clinical evidence is sufficient to initiate treatment is unclear. Practical, data-driven treatment decision algorithms could help support more effective and uniform treatment decision-making at health facilities [9], improving treatment outcomes by shortening the time to treatment initiation [10–14].

The WHO recently recommended decentralized models of care and use of integrated treatment decision algorithms for diagnosis of pediatric TB [15]. It presented 1 such algorithm for children in settings where diagnoses are frequently missed [7]. The algorithm first evaluates for signs that may require urgent management and referral to higher care, then stratifies based on risk of TB-associated mortality. Finally, it uses a score-based algorithm to identify features, and combinations of features, from the clinical evaluation and chest radiography to identify children who should be initiated on TB treatment. However, this algorithm has not been validated, and there is a need for evidence of feasibility and implementation research evaluation as several of the signs and symptoms are not routinely collected during programmatic care in places with high burdens of TB, including duration of fever, lethargy, hemoptysis, tachycardia, and tachypnoea. Additionally, often individual chest radiograph features are not recorded as the radiograph is just categorized as suggestive or not suggestive of disease, with well-known challenges associated with chest radiograph quality and interpretation. In programs where these data are not recorded, it would be difficult for a child to obtain a score high enough to triage for treatment initiation.

Thus, in a setting in Pakistan where these data were not collected, we aimed to identify children who would benefit from rapid initiation of TB treatment by deriving 2 clinical prediction tools using only routinely collected data locally.

METHODS

Study Population

Pakistan, with a population of 221 million (2020), has a high TB burden, accounting for 6% of the global TB burden, with a TB incidence of 259 per 100 000 population. In 2020, 48% of TB cases went undetected, and children age 0–14 years made up 14% of reported TB cases [16]. We conducted a secondary analysis of a prospective intensified TB patient–finding intervention conducted in Pakistan in 2014–2016 [17].

TB Screening Program

Between October 2014 and March 2016, an intensified TB patient–finding program was implemented by the Indus Hospital and Health Network in 4 participating public sector health facilities in the rural setting of Jamshoro District, Pakistan [17]. This program involved health workers systematically screening all children presenting to the general, pediatric, and chest outpatient departments at the participating facilities. No inpatient screening was carried out. The systematic screen included health workers asking children and/or their caregivers whether they had a cough for 2 or more weeks, recent contact with someone with TB (last 2 years), glandular swelling, or presence of any of the following symptoms: fever for 2 or more weeks, night sweats, or inappropriate weight loss (failure to thrive). A child who responded yes to having a cough, contact with someone with TB, glandular swelling, and/or had 2 or more other symptoms present was presumed to have TB and referred to a medical officer for further evaluation. Children received a full clinical evaluation, which included a chest radiograph, complete blood count, and erythrocyte sedimentation rate, and, if indicated and if they could produce sputum, GeneXpert MTB/RIF assay (Cepheid, Sunnyvale, CA, USA). Additional testing, including ultrasound, computed tomography, and fine-needle aspiration/biopsy, was also completed if clinically indicated. Chest radiographs were read by the treating physician and recorded as suggestive of TB or not suggestive of TB. A clinician diagnosed children with TB if they had a positive GeneXpert MTB/RIF result or based on a combination of clinical and radiologic evidence. Children diagnosed with TB were referred for appropriate treatment, per local guidelines [18]. A total of 105 338 children age 0–14 years were screened for TB, and, of these, 5880 (5.6%) children screened positive and were presumed to have TB disease [17].

Outcome Variable Definition

The outcome of interest is any diagnosis of TB disease, rendering an individual eligible for TB treatment. As described above, a TB diagnosis may be made based on clinical/radiologic evidence or bacteriologic confirmation of disease.

Exposure Variable Definitions

Characteristics collected from participants included demographics, medical history, and clinical presentation. Demographic information included sex, age, and weight-for-age percentile, which was calculated using the WHO's growth charts for children age ≤2 years and the US Centers for Disease Control and Prevention's growth charts for children age >2 years [19]. We categorized children based on whether their weight-for-age was higher than the 5th percentile or not. Children were also categorized by age group: 0–4, 5–9, and 10–14 years old. Medical history included reporting contact with TB disease in the past 2 years and whether there was a Bacille Calmette-Guerin (BCG) vaccine scar present. Clinical presentation included reporting the presence of cough, cough duration, fever, weight loss, chest radiograph results (suggestive of TB or not), and chest, abdominal, and lymph node examination results (suggestive of TB or not).

Statistical Analysis

We report the frequency and percentage of characteristics of all children who were presumed to have TB and who subsequently completed an evaluation for TB disease. We also report and compare the characteristics of children who were and were not diagnosed with TB using chi-square tests.

We developed 2 clinical prediction tools using different analytic approaches that have complementary strengths and limitations: classification and regression tree (CART) analysis and logistic regression. CART is a nonparametric method that uses recursive partitioning to search through all potential predictors and their cutoff values to identify those whose associated threshold level is most important to predicting the outcome variable [20, 21]. CART presents results in the form of a simple-to-use decision tree that can be easily implemented in clinical settings [20]. In contrast, logistic regression has well-characterized methods for testing model assumptions to ensure validity of the resulting model. However, ascertaining relationships between the outcome and continuous predictor variables is difficult unless meaningful thresholds for categorizing predictors are determined a priori. The development of a prediction score, using regression output, can also be a simple way for clinicians to detect whether an individual is predicted to have the outcome of interest.

All children presumed to have TB disease, based on the symptom screen, and with a full evaluation for TB disease—defined as having a chest radiograph, Xpert MTB/RIF assay on sputum, abdominal ultrasound, or histopathology test performed and being assessed by a TB medical officer—were included in both analyses.

Approach 1. Classification and Regression Tree Analysis: Development of a Decision Tree

All potential predictors described above were considered for inclusion in the final tree. A CART model identified the most important potential predictor after assessing the relevance of each variable in the final model.

We assigned measures of predictive importance using the Gini Gain Index to each potential predictor. We split the data sets into increasingly homogenous subgroups, adding smaller nodes to the tree based on how much each variable improved the prediction of the primary node. Maximal trees were generated and then pruned based on relative misclassification costs, complexity, and parsimony. Ten-fold cross-validation randomly split the whole data set into learning and test data sets. We then used CART analysis to determine the models’ performance and predictive accuracy in these test sets and to produce a composite performance and accuracy measure, including an AUC, for the final tree. To assess the potential for age-specific predictors, we ran CART analysis in all children (0–14) and in subgroups of children age 0–4, 5–9, and 10–14 years.

Approach 2. Logistic Regression: Development of a Prediction Score

We performed multivariable logistic regression. Predictor variables included gender, age group, weight-for-age ≤5th percentile, presence of any cough, fever, or weight loss, chest radiograph suggestive of TB disease, and a family history of TB disease. Only children who had no missing data for these potential predictors were included in the final model. The primary means of selecting the final model were the Akaike Information Criteria (AIC) and the Bayesian Information Criterion (BIC), for which lower values for both indicate better fit [22]. We also used the c-statistic to assess model discrimination. The larger the c-statistic, the better the model discriminates [23]. To assess model calibration, we computed the Hosmer-Lemeshow statistic [24]. We used bootstrap resampling (1000 samples) for internal validation and to obtain a value accounting for model optimism [25, 26].

Log odds values from the final model were normalized by dividing them by their respective standard error (SE) and rounding off to the nearest integer. A cumulative risk score for each subject was calculated by summing these up. To select the optimal cutoff for the prediction score, we used a receiver operating characteristics (ROC) curve analysis using the Youden's index, which maximizes the discriminative properties, including sensitivity and specificity [27]. Using the optimal score cutoff, we assessed the performance of the prediction score to diagnose TB and calculate the sensitivity, specificity, positive predictive value, and negative predictive value. The WHO recommends a community-based triage test to have a minimum sensitivity of >90% and a specificity of >70%, and an optimal sensitivity of >95% and a specificity of >80% [28]. To make the score simple to implement in a clinical setting, we developed an algorithm reflecting the pathways that would indicate a TB diagnosis to inform treatment decision-making.

CART analysis was run using Salford Systems Data Mining and Predictive Analytics Software, version 8.0 (Salford Systems, San Diego, CA, USA). The prediction score analysis was performed using SAS, version 9.4 (SAS Institute, Cary, NC, USA).

Ethics Approval

The Institutional Review Board (IRB) of Interactive Research and Development, Karachi, Pakistan, reviewed and approved the original study protocol (approval number: IRD-IRB-2015–04–002). Verbal informed consent was obtained from all children's guardians as well as from children over the age of 7. The subsequent analysis of de-identifiable data was determined to be nonhuman subjects research by the IRBs of Harvard Medical School (HMS-IRB-20-0479) and Boston University Medical Campus (H-43008).

RESULTS

In this secondary analysis, we examined data for the 5162 children evaluated for pulmonary TB disease. Of these, 1417 (27.5%) children were diagnosed with TB disease; those children had significantly different characteristics than the children who were not diagnosed with TB (Table 1).

Approach 1. Classification and Regression Tree Analysis: Development of a Decision Tree

Of all children age 0–14 years (n = 5162) who were presumed to have TB and who were subsequently evaluated for TB, 1417 (27.5%) were diagnosed with TB and eligible for treatment. The most important predictor identified was having a chest radiograph suggestive of TB, with 99.2% having TB disease, compared with only 6.1% of children with a normal chest radiograph having TB (Figure 1). Of children with a normal chest radiograph, the next best predictor was having a family history of TB, with 35.4% having TB disease, compared with only 3.2% of those without a family history of TB having TB. The overall model AUC was 0.949.

The age-specific analyses included 2192 children with 714 (32.6%) having TB disease (0–4 years), 1825 children with 411 (22.5%) having TB (5–9 years), and 1145 children with 292 (25.5%) having TB (10–14 years). Having a chest radiograph suggestive of TB disease was the most important predictor in all 3 age group models, with 99.5%, 98.1%, and 100% of children in the 0–4, 5–9, and 10–14 age groups, respectively, with this result being diagnosed with TB disease and referred for treatment (Figures 2, 3, 4). In children with a normal chest radiograph, the next most important predictor was having a family history of TB for all models, with 36.7%, 31.8%, and 39.0% of children in the 0–4, 5–9, and 10–14 age groups being referred for TB treatment. Additionally, both the 0–4 and 10–14 age models had a third important predictor; in children with a normal chest radiograph and no family history of TB, having no cough or cough <2 weeks led to 73% and 64%, respectively, being referred for treatment initiation. Important to note is that 84.6% (0–4 model) and 55.5% (10–14 model) of those who fell into that category had EPTB. The overall model AUCs were 0.960, 0.967, and 0.950 for the 0–4, 5–9, and 10–14 age models, respectively.

For the sensitivity analysis, we repeated the overall analysis stratified by type of TB diagnosis. For children with PTB, the most important predictor was chest radiograph suggestive of TB, followed by presence of cough (Supplementary Figure 1). For children diagnosed with EPTB, the most important predictor was family history of TB, followed by absence of cough and weight ≤6.5 kg (Supplementary Figure 2).

Approach 2. Logistic Regression: Development of a Prediction Score

Of the 5162 children presumed to have TB who were subsequently evaluated for disease, 88 did not have potential predictor variables available, so they were excluded from analysis. Of the 5074 children included in the analysis, 1365 (26.9%) had TB disease, of whom only 38 (2.8%) had bacteriologic confirmation of disease. Our final model selected included covariates for age group, weight <5th percentile, cough, fever, weight loss, chest radiograph suggestive of TB disease, and family history. This model had an area under the curve of 0.985. Model optimism was estimated to be 0.06%.

Table 2 summarizes the coefficients for the final logistic regression model and corresponding calculated prediction scores. The highest score was for having a chest radiograph suggestive of TB disease (+17 points), followed by having a family history of TB disease (+9 points) and weight <5th percentile (+8 points). Weight loss resulted in +4 points in the 10–14-year age group; having cough or fever resulted in +2 points each. Being in the 5–9-year age group led to a negative score of −3 points. The median cumulative risk score (interquartile range) was 3 (2–8). Using Youden's J, we identified the optimal cutoff for diagnosis of TB disease to be 9 points. This resulted in a sensitivity of 98.2% (95% CI, 97.3%–98.8%), specificity of 89.2% (95% CI, 88.1%–90.1%), positive predictive value of 76.9% (95% CI, 75.2%–78.5%), and negative predictive value of 99.3% (95% CI, 98.9%–99.5%).

Figure 5 illustrates all clinical pathways to achieving the optimal predictive score threshold of 9.

For the sensitivity analysis, we calculated risk scores separately by type of TB diagnosis. Supplementary Tables 1 and 2 summarize the coefficients for the logistic regression model and corresponding calculated prediction scores for PTB and EPTB diagnosis compared with no TB, respectively. For children with PTB, the highest score was for having a chest radiograph suggestive of TB disease (+20 points). The optimal cutoff for diagnosis of PTB was 12 points, resulting in sensitivity of 96.7% (95% CI, 95.5%–97.6%) and specificity of 99.8% (95% CI, 99.5%–99.9%). For children with EPTB, the highest score was for family history of TB (+12 points). The optimal cutoff for diagnosis of EPTB was 8 points, resulting in sensitivity of 95.8% (95% CI, 92.1%–98.0%) and specificity of 86.8% (95% CI, 85.6%–87.8%).

DISCUSSION

Our results suggest that children presenting to health facilities who have a TB-related symptom or recent contact with an individual with TB can be assessed using a chest radiograph and family history of TB disease to initiate TB treatment. Both derived models identified these 2 characteristics as being most important to identifying children who would benefit from early treatment initiation, with high accuracy.

With limited ability to obtain microbiological confirmation of disease in children and >75% of TB in children being intrathoracic, chest radiograph is an important part of the pediatric TB diagnostic algorithm [5, 6, 29–31]. Interpretation of chest radiographs in children are complicated due to a broad disease spectrum and great variability in radiologic patterns by age, HIV status, and other comorbidities [6, 32–36]. Although chest radiographs are locally classified as “suggestive” or “not suggestive” of TB, which can lead to suboptimal sensitivity and specificity [37, 38], chest radiograph was the most important predictor of benefiting from TB treatment initiation. Incorporation of specific chest radiograph features, if available, may lead to improved discriminatory properties.

Cough duration was identified as an important predictor in the 0–4- and 10–14-year CART models. However, the findings were in the opposite direction, as would be expected. In particular, the subgroup that had a normal chest radiograph, no family history of TB, and no cough or reported cough for <2 weeks had a high proportion requiring TB treatment initiation (73% and 64%). Upon further evaluation, it was identified that the majority of this group were actually diagnosed with extrapulmonary TB disease. This suggests that individuals who report a TB-related symptom or recent contact with an individual with TB, but who lack a TB-suggestive chest radiograph, family history, and cough, still warrant a further clinical/diagnostic workup to assess for TB disease.

The developed prediction tools are unique compared with other existing algorithms. The derived tools are specific to children in Pakistan, where there is a low HIV prevalence and limited to no resources available to routinely induce sputum or use gastric aspirate to obtain a sample for bacteriologic confirmation of TB. This is in contrast to 2 clinical prediction tools developed in South Africa. One only focused on HIV-infected children and had an AUC of 0.870 [39], while the other focused on HIV-uninfected children and had an AUC of 0.750 [40]. This population had resources available to perform gastric aspirate or sputum induction, and 43% of children had bacteriologic confirmation of TB disease, compared with only 3% of children in this cohort from Pakistan [41]. A third algorithm of children in Peru was conducted only in children who were exposed at home to TB and reported lower AUCs ranging from 0.660 to 0.693 [42]. The current tools had far superior accuracy, with AUCs of ≥0.950.

The integrated treatment decision algorithm presented by the WHO [15] is limited to children <10 years old with pulmonary TB. Our tools are for children <15 years old presenting to a health facility with any form of TB. We also focus heavily on clinical examination findings, symptoms, and medical history that are routinely collected during programmatic care. Numerous signs and symptoms included in the WHO algorithm are not systematically collected in the Pakistan clinical setting, including duration of fever, presence of lethargy, hemoptysis, tachycardia, and tachypnea, as well as individual chest radiograph features. The addition of systematic collection and recording of such data to the current program will take time, money, and training of health care personnel; thus the addition of clinical tools using only existing programmatic data can be integrated into care more rapidly. In settings that lack a strong health care infrastructure and have limited resources along with a high burden of TB, such an algorithm can help improve diagnosis of childhood TB. However, this algorithm will need to be validated externally before use.

Our study has some limitations. First, incorporation bias is an issue in this study; because only 3% of children in this population had bacteriologic confirmation of disease, the remainder of children were diagnosed based on clinical and radiological evidence. The same variables that are used to diagnose children with TB are being used to predict the outcome, and additional confirmatory diagnostic tests could not be performed. Thus, of all potential characteristics used to clinically diagnose children with TB, we aim to identify the most important ones that will capture the majority of individuals who would benefit from rapid initiation of treatment. Second, these prediction tools are not developed for all-comers to a health facility, as all children included to derive these tools underwent a systematic verbal screening where they reported either the presence of a TB-related symptom or recent contact with an individual with TB. Third, pediatric TB doctors will need to be able to interpret the TB-associated findings of chest radiographs to use this algorithm. Lastly, before operationalization of the developed algorithm, it is essential to externally validate the tools.

CONCLUSIONS

Optimizing clinical approaches to TB treatment decision-making is important to improve treatment access for children. However, the urgent need to increase TB detection and treatment access must be balanced against the consequences of overdiagnosis and unnecessary treatment. While the development of the WHO algorithm is of great use, as it currently stands, some important features required to apply it are not routinely collected in high–TB burden settings, and there may be a delay in updating programmatic care settings to collect these additional data, which will require balancing additional time, training, effort, and financial resources. These locally tailored clinical prediction tools and accompanying operationalized algorithm, which take into account actual data collected by programs, could be of great use in the interim.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Acknowledgments

Financial support. This project was supported through Stop TB Partnership's TB REACH initiative, the William F. Milton Fund at Harvard University (M.B.B.), the Center for Global Health Delivery at Harvard Medical School (M.B.B. and M.C.B.), the Dubai Harvard Foundation for Medical Research (M.B.B. and M.C.B.), and the National Institutes of Allergy and Infectious Diseases at the National Institutes of Health (K01-AI151083 to M.B.B.). TB REACH is generously supported by Global Affairs Canada.

Role of funder/sponsor. This manuscript's contents are solely the responsibility of the authors and do not necessarily represent the official views of these organizations. The funders/sponsors had no role in the design, conduct, or interpretation of this study.

Author contributions. Dr. Brooks conceptualized and designed the study, conducted the formal analysis, validated study results, interpreted data and results, drafted the initial manuscript, revised the manuscript critically for important intellectual content, and acquired funding to complete this work. Dr. Hussain created data collection forms and contributed to acquisition of data, project implementation, and supervision of project implementation; interpreted data and results; revised the manuscript critically for important intellectual content; and acquired funding to complete this work. Mr. Ahmed, Ms. Siddiqui, and Ms. Jaswal contributed to acquisition of data, project implementation, and revised the manuscript critically for important intellectual content. Dr. Amanullah created data collection forms; contributed to acquisition of data, project implementation, and supervision of project implementation; interpreted data and results; and revised the manuscript critically for important intellectual content. Dr. Becerra conceptualized and designed the study, supervised analysis, interpreted data and results, drafted the initial manuscript, revised the manuscript critically for important intellectual content, and acquired funding to complete this work. Dr. Malik created data collection forms, contributed to acquisition of data, conducted project administration duties, cleaned the data, interpreted data and results, and revised the manuscript critically for important intellectual content.

Data sharing. The de-identified individual participant data can be made available, with reasonable request, by contacting Meredith Brooks ([email protected]) and Amyn Malik ([email protected]).

Patient consent. The Institutional Review Board (IRB) of Interactive Research and Development, Karachi, Pakistan, reviewed and approved the original study protocol (approval number: IRD-IRB-2015-04-002). Verbal informed consent was obtained from all children's guardians as well as from children over the age of 7. The subsequent analysis of de-identifiable data was determined to be nonhuman subjects research by the IRBs of Harvard Medical School (HMS-IRB-20-0479) and Boston University Medical Campus (H-43008).

References

Author notes