Prediction of intradialytic hypotension using pre-dialysis features—a deep learning–based artificial intelligence model

ABSTRACT

Background

Intradialytic hypotension (IDH) is a serious complication of hemodialysis (HD) that is associated with increased risks of cardiovascular morbidity and mortality. However, its accurate prediction remains a clinical challenge. The aim of this study was to develop a deep learning–based artificial intelligence (AI) model to predict IDH using pre-dialysis features.

Methods

Data from 2007 patients with 943 220 HD sessions at seven university hospitals were used. The performance of the deep learning model was compared with three machine learning models (logistic regression, random forest and XGBoost).

Results

IDH occurred in 5.39% of all studied HD sessions. A lower pre-dialysis blood pressure (BP), and a higher ultrafiltration (UF) target rate and interdialytic weight gain in IDH sessions compared with non-IDH sessions, and the occurrence of IDH in previous sessions was more frequent among IDH sessions compared with non-IDH sessions. Matthews correlation coefficient and macro-averaged F1 score were used to evaluate both positive and negative prediction performances. Both values were similar in logistic regression, random forest, XGBoost and deep learning models, developed with data from a single session. When combining data from the previous three sessions, the prediction performance of the deep learning model improved and became superior to that of other models. The common top-ranked features for IDH prediction were mean systolic BP (SBP) during the previous session, UF target rate, pre-dialysis SBP, and IDH experience during the previous session.

Conclusions

Our AI model predicts IDH accurately, suggesting it as a reliable tool for HD treatment.

Graphical Abstract

Open in new tabDownload slide

INTRODUCTION

Intradialytic hypotension (IDH) is one of the most frequent complications of hemodialysis (HD), with a reported incidence of about 10%–12% according to the applied diagnostic criteria [1, 2]. IDH results in inadequate dialysis doses with subsequent volume overload; hence, it is ultimately associated with increased cardiovascular morbidity and mortality rates in patients under maintenance HD [3, 4]. Prior studies have suggested an imbalance between intravascular volume reduction and intracellular refilling; an inadequacy of compensatory mechanisms, such as an increase in heart rate or myocardial contractility; and the constriction of resistance or capacitance vessels as the main cause of IDH [5, 6]. In other words, both patient-related and dialysis-related factors are associated with the hypotension during HD [4, 7, 8]. Due to the complex and multifactorial mechanisms, predicting the occurrence of IDH prior to HD is a challenge for clinicians [9].

Recently, various medical fields have begun to rely on artificial intelligence (AI) due to its ability to integrate and interpret the complexity and the large volume of big data. Recent advancements in deep learning allow feature extraction without the feature selection required for standard machine learning algorithms [10], and interpretation of unstructured forms of data such as images and sounds. In the field of nephrology, studies have applied AI to interpret data derived from dialysis and kidney transplant patients [11, 12], including developments of AI models to predict IDH. The studied IDH prediction models are mostly based on standard machine learning algorithms and were developed using data from a single institution [13–16].

Considering the previous AI studies on IDH, we aimed to develop deep learning algorithms to predict its occurrence using electronic medical records (EMRs) from seven independent general hospitals extracted from the well-established clinical data warehouse (CDW) of the Catholic Medical Center (CMC) [17]. For this, we preprocessed a large dataset of HD patients from each dialysis unit and then developed a deep learning–based prediction model. Use of multi-center data minimizes the clinician bias present in single-institution data, providing a model applicable to all seven hospital dialysis systems. Using the developed AI model, we intended to present a useful screening tool to guide clinicians to identify patients vulnerable to IDH and to help them perform preventive interventions prior to initiation of HD treatment.

MATERIALS AND METHODS

Data sources

All data were extracted from the CDW of the CMC, which provided access to an EMR database of patients from seven affiliated hospitals constituting the CMC, the largest healthcare network in Korea, including the Seoul, Bucheon, Eunpyeong, Incheon, Uijeongbu, and Yeouido St Mary's Hospitals and St Vincent's Hospital. We retrieved data for the following categories consisting of multiple features: demographics, comorbidities, medications, laboratory findings and HD records including HD settings, and vital signs during HD treatment. Since the data were encoded and anonymized, informed consent was not required. The study protocol was reviewed and approved by the Institutional Review Board of Catholic University of Korea (XC20WIDI0025).

Study population

We extracted the medical records of patients who underwent HD at CMC between 1 March 2009 and 31 December 2019. We initially identified 1 359 117 HD sessions among 16 914 patients. We excluded individuals who did not undergo consecutive HD sessions for at least 90 days or were <18 years of age (383 206 sessions in 14 751 patients). After data preprocessing as described later, a total of 943 220 sessions from 2007 patients remained. The selected patients were randomly divided into a training set (70%, 651 380 sessions in 1404 patients) to develop a prediction model, a validation set (15%, 146 808 sessions in 301 patients) to validate the model and a test set (15%, 145 032 sessions in 302 patients) to test the classification performance of the model. The dataset was split in such a way that similar proportions of IDH occurrence remained in each dataset. The training set included 34 616 IDH sessions (≈5.31% of total sessions), the validation set included 7911 IDH sessions (≈5.39% of total sessions) and the test set included 8357 IDH sessions (≈5.76% of total sessions). The flowchart of study population retrieval and splitting is shown in Fig. 1. Even though the number of digits in the session number is very large, the percentage is rounded to only two decimal places. Therefore, there is the discrepancy between the number of sessions calculated by percentage and real counted sessions in database. In order to minimize misunderstanding, ‘≈’ was added in front of the percentage.

Study variables and preprocessing

A total of 36 predictor variables from patient demographics, comorbidities, medications, laboratory findings and HD records was used during prediction model development. The formula for the calculation of body mass index, interdialytic weight change and ultrafiltration (UF) target rate were presented as Supplementary data, Fig. S1.

Missing data were handled according to the characteristics of each feature. All variables with missing data were double-checked using the anonymized CDW database to minimize data loss. For blood pressure (BP) and body weight, outliers were removed from the dataset. If the systolic BP (SBP) was less than or equal to diastolic BP (DBP), then the BP data were considered outliers. Based on the difference between dry weight, pre-dialysis weight and post-dialysis weight, the outliers of body weight were determined. After double-checking missing data and removing outliers, null values of comorbidities, medications and HD records were filled forward under the assumption of continuous treatment. Patients with <50% of missing data of weights and BPs were filled in by rolling the average of a minimum of two and maximum of six measures. Patients who continued to have missing records after missing data and outliers handling were excluded from the study.

IDH was defined according to the following criteria: (i) nadir SBP <100 mmHg when the initial SBP was ≥160 mmHg, (ii) nadir SBP <90 mmHg when the initial SBP was 90 ≤ SBP < 160 mmHg, or (iii) ≥20 mmHg intradialytic SBP decrease when the initial SBP was <90 mmHg. IDH criteria was treated as a binary outcome, with positive occurrence of IDH.

Model development

We developed a predictive deep learning model based on one-dimensional convolutional neural network (1D-CNN) (Fig. 2). The proposed model was built with 1D convolution layers, skip connections and a fully connected layer. The model had five consecutive 1D-CNN blocks and two skip blocks. Each convolution block consisted of three convolution layers, followed by batch normalization and a dropout layer. After convolution computation, a fully connected layer output classification result was obtained using a sigmoid function. Each layer used the rectified linear unit activation function, and the dropout rate was 0.3. Machine learning algorithms of logistic regression, random forest (RF) [18] and XGBoost models [19] were trained and validated to compare their predictive performances with that of the 1D-CNN model.

Model evaluation

The performance of models was evaluated using accuracy, recall, precision, macro-averaged F1 score (macro F1 score), area under the precision-recall curve (AUPRC), area under the receiver operating curve (AUROC), specificity, negative predictive value (NPV) and Matthews correlation coefficient (MCC) [20]. The threshold was set to maximize the F1 score of the positive values. Each of the metric calculations is presented as Supplementary data, Fig. S2, with true negative (TN), true positive (TP), false negative (FN) and false positive (FP) values.

The 95% confidence interval (CI) for each model performance metric was calculated using bootstrapping on the test set, employing 14 503 randomly selected samples with replacement, for a total of 1000 bootstraps.

Feature importance analysis

To examine the effect of each feature on IDH, we analyzed feature importance and visualization on the machine learning algorithms and the developed deep learning models. Traditionally, the machine learning algorithms used in this study have their own methods of extracting feature importance. We also used the Shapley additive explanation (SHaP) tool on all the classification models to calculate the feature contributions to the changes in prediction performance of a model based on game theory approach [21]. SHaP allows users to calculate and visualize feature contributions for deep learning models from tabular data, enabling comparisons between feature importance across all the models used.

Statistical analysis

Numerical variables are expressed as mean ± standard deviation or median (interquartile range), and the categorical variables are presented as total count and percentage. In two-group comparisons, a Mann–Whitney U test was performed for continuous variables. All categorical variables were compared using the chi-square test of independence. The significance level was set as α = 0.05. All statistical analyses were conducted using Python version 3.6.10 (Python Software Foundation, Wilmington, DE, USA) and SciPy library [22].

RESULTS

Baseline characteristics of the HD sessions analyzed

Full baseline demographic features and HD settings are listed in Table 1. IDH occurred in ≈5.39% (50 884 sessions) of all studied HD sessions, and 76.6% of all studied patients experienced IDH at least once in their continuous dialysis treatments. Among all HD sessions, ≈5.38% (50 765 sessions) involved IDH during the previous session. The mean pre-dialysis SBP and DBP values were 142.0 ± 22.9 and 75.5 ± 14.5 mmHg, respectively.

Comparison of the clinical features between IDH and non-IDH HD sessions

We performed comparative group tests on demographic features and HD settings between IDH and non-IDH sessions. IDH sessions showed longer dialysis vintage, lower pre-dialysis SBP and pre-dialysis DBP values, and higher pre-dialysis pulse rate (PR), UF target rates and interdialytic weight gain than the non-IDH sessions (Table 2). The mean SBP and DBP values during previous HD sessions were lower among the IDH sessions and the occurrence of IDH in previous sessions was more frequent among IDH sessions compared with non-IDH sessions (35.6% vs 3.7%, P < .001). There was no significant difference in age, anticoagulation or prescription of calcium-based phosphate binders between two groups. A higher prevalence of comorbidities, such as ischemic heart disease, valvular heart disease, arrhythmia, liver disease, chronic obstructive lung disease and diabetes mellitus, were found among the IDH sessions. Patients in the IDH sessions were prescribed fewer anti-hypertensive drugs than those in the non-IDH sessions.

Performance comparison between prediction models

All classifiers’ prediction performance measures are presented in Table 3, and the predicted results for each threshold value maximizing F1 score are shown in confusion matrices (Supplementary data, Fig. S3). Using the test dataset, the AUROC for predicting IDH was highest in the three-timeframe deep learning model, followed by one-timeframe deep learning model, XGBoost and logistic regression. The AUPRCs for prediction IDH were highest in three-timeframe deep learning model, followed by XGBoost, RF, one-timeframe deep learning model and logistic regression. Since our data were imbalanced, we used MCC and macro F1 score to consider both positive and negative predictions in evaluating predictive performance. MCC was highest in the three-timeframe deep learning model, followed by XGBoost, logistic regression and one-timeframe deep learning model. Similarly, the macro F1 score was highest in the three-timeframe deep learning model.

Feature importance score and ranking in predicting IDH

Feature importance analysis using logistic regression, RF, XGBoost and the two constructed deep learning models was performed to estimate the contribution of each feature in predicting the occurrence of IDH. Figure 3 shows feature ranking plots in decreasing order of Shapley values. The common top-ranked features in machine learning models were: mean SBP, DBP or PR of the previous session; UF target rate; pre-dialysis SBP or DBP; use of calcium channel blockers; IDH experience in the previous session; and sex (male). The deep learning model using current HD session (one timeframe) data showed similar results. The deep learning model using data from the past three sessions (three timeframes) demonstrated that vital signs during the session before the last session also contributed significantly to IDH prediction. The full feature importance scores and plots are shown in Supplementary data, Fig. S4 and Table S1.

DISCUSSION

In this study, we found the important features predicting occurrence of IDH and demonstrated that AI models based on those pre-dialysis features could estimate the risk of IDH occurrence accurately. The feature importance analysis demonstrated that the mean BP during the previous HD session, UF target rate, pre-dialysis BP and IDH occurrence during the previous session contributes most significantly to the prediction of IDH. Although these parameters have been empirically expected and are relatively well known as risk factors of IDH by clinicians in the real world, there is no exact prediction tool with risk stratification. Therefore, the present AI model is suitable to use to calculate the actual risk in real-world HD treatments.

We used enormous medical data resources spanning >10 years directly extracted from the EMR systems of seven independent hospitals to create a dataset to train and validate the AI model. Using our CDW system, it was possible to access high-quality clinical patient information and link to multiple operational data sources [23, 24]. Although the number of current medical records was large in volume for handling—clinical data of 2007 patients who underwent a total of 943 220 sessions—we could refine the HD-related resource based on our well-established CDW system. Also, the seven hospitals included herein employ a uniform EMR system with the same format for entry of HD-related information [25], ensuring credibility and consistency in the data quality. Therefore, our refined data were optimized for AI model development with a relatively small number of data errors or missing levels by validating the consistency of information.

Our next concern was defining significant IDH. There is no consensus on the definition of IDH, so various definitions have been used across studies. For example, IDH was defined in one study based on an SBP decrease during HD with the need for intervention, including saline loading and others [1, 9], while another study defined it as a decrease in either SBP of ≥20 mmHg or mean arterial pressure of ≥10 mmHg as well as symptoms of cramping, headache, lightheadedness, vomiting or chest pain during HD [26]. However, these definitions are insufficient in terms of long-term prognosis prediction. Flythe et al. demonstrated that the nadir-based definitions of IDH best capture the association between IDH and mortality. They studied an association of mortality risk with various definitions of IDH and found that an absolute nadir intradialytic SBP <90 mmHg (Nadir90) and Nadir100 were most potently associated with mortality [1]. Therefore, we used the nadir-based criteria for IDH definition. The frequency of IDH in our cohort based on this definition was 5.39%, which is a lower value than that in previous reports [27]. We assumed this discrepancy exists because clinicians tend to prevent IDH in advance, relying on their empirical risk assessment in real-world practice.

Based on the preprocessed big data, we developed an IDH prediction AI model. To compare prediction performance, we also trained standard machine learning models of logistic regression, RF and XGBoost. Prior studies on AI prediction models for IDH mostly focused on positive prediction [16]. Our deep learning model also demonstrated good AUROC and AUPRC values of 0.87 and 0.35, respectively. However, since the frequency of IDH occurrence is about only 5%, suggesting a very imbalanced dataset, use of both positive and negative predictions is more appropriate to evaluate predictive performance. As precision and recall are in a trade-off relationship, we chose the F1 score in order to maximize both precision and recall. In this regard, we used MCC and macro F1 score, which are employed to effectively evaluate AI model performance in imbalanced data, and these values were 0.37 (95% CI 0.34–0.40) and 0.69 (95% CI 0.67–0.70), respectively, in our final deep learning model. This MCC performance in our study was higher than that in previous reports [28]. Higher values of MCC and macro F1 score mean that the model effectively predicts not only IDH occurrence, but also the absence of IDH. To help clinicians focus on high-risk patients, both positive and negative predictions are important in screening tests.

Even though the performance of this deep learning model was superior to previously reported results, the model showed similar prediction performance to those of other standard machine learning models in this study when using only one-timeframe data. In this regard, we expanded medical resources into previous sessions to implement the best predictive performance. Among diverse deep learning algorithms, we applied 1D-CNN to construct our prediction model. This type of neural network is efficient at extracting features from neighboring information [29], which is appropriate to explain the complex relationship between the predictors to estimate IDH occurrence. In addition, longer interdialytic period sessions during thrice-weekly dialysis are associated with greater interdialytic weight gain and higher IDH risk [30]. Therefore, it is reasonable to refer the most recent three sessions, which involve all types of interdialytic intervals during 1 week, for better prediction performance. With this idea, we combined clinical data from three past sessions and created a deep learning model to achieve the best predictive performance, with MCC value of 0.44 from 0.37, and macro F1 score of 0.72 from 0.69.

Previous AI models found that the mean SBP and DBP values during previous sessions were most highly correlated with SBP [15], and a previous IDH history imparts a high feature ranking score [16]. Our feature importance analysis also showed that the mean SBP during the previous session and recent IDH history are highly correlated with IDH risk. In the developed IDH prediction models, our feature importance analysis showed that UF target rate, which closely relates to interdialytic weight gain, and pre-dialysis SBP are common important features, as expected [31]. Moreover, our results suggest that use of calcium channel blocker is an important risk factor for IDH. Theoretically, anti-hypertensive drugs, which are known to improve left ventricular hypertrophy and impaired left ventricular relaxation, might be expected to reduce IDH, while drugs with vasodilatory effects might increase the frequency of IDH. However, there is a lack of evidence supporting the use of any particular anti-hypertensive mediation to influence IDH risk [32–34]. Future research is needed to uncover any existing association between certain anti-hypertensive drug classes and IDH risk.

In order to effectively prevent IDH, the integration of our deep learning model with an AI-biofeedback system is crucial. To this end, we propose a “preemptive IDH intervention bundle” as part of this biofeedback system, which includes pre-lowering the patient's target UF, more frequent blood pressure monitoring, reducing the dialysate temperature and extending the dialysis session time for the day, among other measures. If this intervention bundle can be seamlessly integrated with our deep learning model, we believe that we can achieve proficient prevention of IDH in clinical settings. Therefore, we are planning a prospective study to demonstrate the efficacy of our AI-biofeedback systems, which will compare the incidence rate of IDH between two groups. One group will receive preemptive IDH intervention bundle guided by our AI model, while the other group receive preventive interventions without our AI model.

Our study had several limitations. First, unidentified features in this study, such as cardiac function, dialysate sodium concentration and dialysate temperature, may provide additional information to help discern the risk of IDH and further improve the model performance [31]. To overcome this limitation, we trained AI models with data on comorbidities, including ischemic heart disease, valvular heart disease and arrhythmia, using International Classification of Diseases, 10th revision codes. Second, dialysis records were filled in manually, resulting in many typos or outliers and missing data. We preprocessed the data for the longest duration of time to date during this study, double-checking the database and correcting typos. However, this problem could be solved by linking dialysis records automatically to EMR systems as performed in recent practice. The third limitation is the lack of intra-dialytic data, which makes it difficult to accurately predict when IDH will occur during a dialysis session. Furthermore, the major important features of our AI model are the risk factors that are already known in experienced dialysis units. However, we are confident that our AI model will provide significant assistance to less experienced HD staff to prevent IDH.

In conclusion, we generated a deep learning–based AI model with information of three past HD sessions, and it showed accurate predictive performance for both the occurrence and absence of IDH. We expect that this deep learning model would be a reliable and easy screening tool for IDH. Hence, it could allow clinicians to identify vulnerable patients and adjust HD settings before HD treatment.

ACKNOWLEDGEMENTS

We especially thank PYEONGHWA IS Co. for their help in performing this research.

FUNDING

This study was supported by a grant from the Korean Nephrology Research Foundation (K-NRF-FMC research fund 2021) and by research fund of Seoul St Mary's hospital, The Catholic University of Korea.

AUTHORS’ CONTRIBUTIONS

H.L., E.S.K. and B.H.C. designed the study. H.L., J.W.M., H.S.P., H.E.Y., Y.S.K., H.W.K., C.W.Y., S.C. and B.H.C carried out data acquisition. H.L., S.J.M., S.W.K., E.S.K. and B.H.C. analyzed and interpreted the data. H.L. drafted the manuscript. E.S.K. and B.H.C. supervised the study. All authors approved the final manuscript and agreed to submit this work to this journal.

DATA AVAILABILITY STATEMENT

The data underlying this article will be shared on reasonable request to the corresponding author.

CONFLICT OF INTEREST STATEMENT

All the authors have declared no competing interest.

REFERENCES

Author notes