Identification of recurrent atrial fibrillation using natural language processing applied to electronic health records

Abstract

Aims

This study aimed to develop and apply natural language processing (NLP) algorithms to identify recurrent atrial fibrillation (AF) episodes following rhythm control therapy initiation using electronic health records (EHRs).

Methods and results

We included adults with new-onset AF who initiated rhythm control therapies (ablation, cardioversion, or antiarrhythmic medication) within two US integrated healthcare delivery systems. A code-based algorithm identified potential AF recurrence using diagnosis and procedure codes. An automated NLP algorithm was developed and validated to capture AF recurrence from electrocardiograms, cardiac monitor reports, and clinical notes. Compared with the reference standard cases confirmed by physicians’ adjudication, the F-scores, sensitivity, and specificity were all above 0.90 for the NLP algorithms at both sites. We applied the NLP and code-based algorithms to patients with incident AF (n = 22 970) during the 12 months after initiating rhythm control therapy. Applying the NLP algorithms, the percentages of patients with AF recurrence for sites 1 and 2 were 60.7% and 69.9% (ablation), 64.5% and 73.7% (cardioversion), and 49.6% and 55.5% (antiarrhythmic medication), respectively. In comparison, the percentages of patients with code-identified AF recurrence for sites 1 and 2 were 20.2% and 23.7% for ablation, 25.6% and 28.4% for cardioversion, and 20.0% and 27.5% for antiarrhythmic medication, respectively.

Conclusion

When compared with a code-based approach alone, this study's high-performing automated NLP method identified significantly more patients with recurrent AF. The NLP algorithms could enable efficient evaluation of treatment effectiveness of AF therapies in large populations and help develop tailored interventions.

Graphical Abstract

Development and application of NLP algorithms to identify AF recurrences in EHRs.

Open in new tabDownload slide

Introduction

Atrial fibrillation (AF) is the most common clinically significant arrhythmia and is associated with excess morbidity and mortality.^1–3 Standard AF rhythm control treatment options include antiarrhythmic medications and procedures such as cardioversion and catheter ablation.^1–3 However, these therapies have shown varied success rates in randomized controlled trials, with rates of documented recurrent AF within 12 months remaining high.^4,5 Furthermore, there is a need to efficiently compare the effectiveness of AF rhythm control treatment options in large real-world populations.^6–8

Identifying recurrent AF episodes after starting rhythm control therapies using electronic health records (EHRs) has been challenging. For instance, most prospective cohort studies used protocol-driven electrocardiograms (ECGs) and Holter monitors to assess for recurrent AF at predefined intervals (e.g. at 3, 6, and 12 months).^4,9,10 To obtain similar information from retrospective studies using real-world data would traditionally require manual chart reviews of a large number of cases, which is not usually feasible. Instead of manual chart reviews, other commonly used methods to identify recurrent AF cases using structured data from EHRs include using a combination of medication, diagnosis, and procedure codes; however, the sensitivity and specificity of such code-based methods are limited by the lack of specific codes for recurrent AF.^11,12

Natural language processing (NLP), a subfield of artificial intelligence, has been increasingly used to ascertain information from unstructured data in the EHR without resource-intensive chart reviews. Recently, Shah et al. used NLP to identify patients with a history of AF by extracting clinical text data.¹³ The NLP algorithm showed a limited performance with an F-score of 0.8, a sensitivity of 0.97, and a specificity of 0.63. However, the algorithm was limited to determining whether or not individuals have any evidence of AF.

The goal of the current study was to derive and validate an NLP algorithm to identify episodes of recurrent AF or atrial flutter within 12 months after the initiation of a rhythm control strategy. We developed and applied code-based, NLP-based, and combined (both code-based and NLP-based) algorithms and compared the frequency of AF recurrence using these different algorithms.

Methods

Setting

This retrospective cohort study was conducted at Kaiser Permanente Southern California (KPSC, site 1) and Northern California (KPNC, site 2), two integrated healthcare delivery systems that provide comprehensive care to >9 million racially, ethnically, and socioeconomically diverse members at their 36 hospitals and 493 medical offices throughout California. The prepaid health plans provide strong incentives for members to use services at Kaiser Permanente facilities. Kaiser Permanente HealthConnect^®, a comprehensive EHR system based on Epic^®, stores nearly all aspects of the care provided to members, including medical encounters, diagnoses, procedures, laboratory tests, pharmacy utilization, membership history, and billing claims. This study used structured data and free-text clinical notes from the EHR.

The KPNC and KPSC institutional review boards approved this study. Given the observational nature of the study, patient informed consent was waived.

Study population

We identified members aged 21 years or older with newly diagnosed AF between 1 January 2010 and 31 December 2017, and who initiated rhythm control therapy between 1 January 2010 and 31 December 2018. AF diagnoses were based on the International Classification of Diseases, Ninth and Tenth Edition (ICD-9 and ICD-10) codes (ICD-9427.31, and ICD-10 I48.0, I48.1, I48.2, and I48.91 in any position of the primary diagnosis) from inpatient stays, emergency department encounters, and outpatient visits. To identify incident AF, we excluded patients with a diagnosis code of AF up to 5 years before enrollment into the study. This method was validated in both sites and was found to have a positive predictive value of 96%.¹⁴ The index date is the date of the first AF diagnosis.

For the present analyses, we included patients receiving antiarrhythmic drugs (AADs) or rhythm control procedures after the first AF diagnosis. We identified patients who filled prescriptions for AADs (i.e. amiodarone, sotalol, disopyramide, flecainide, propafenone, dronedarone, dofetilide, quinidine, and ibutilide) or patients who underwent targeted AF procedures (i.e. catheter or surgical ablation, cardioversion, and pacemaker implantation + nodal ablation for rhythm control) using ICD-9/10, Current Procedural Terminology (CPT) codes, or health plan registry data. We selected patients who received an AF procedure (with/without AADs) or had AADs without an AF procedure. Patients were followed for 12 months following the first AF treatment (a procedure or medication dispensing) date. Detailed exclusion criteria are available in Supplementary material online, Table S1.

Demographic and clinical variables

We obtained demographic information including age, self-reported sex, self-reported race, and self-reported ethnicity. Socioeconomic status (educational attainment and annual household income) was obtained by linking the patient's residence and geocoded Census data. The most proximal body mass index, laboratory, and vital signs before the patient's presumed incident AF index date were collected from ambulatory encounter data. Baseline medical history was ascertained using 5 years of diagnosis and procedure data before each patient's index AF diagnosis date. Baseline medication use was ascertained using 12 months of pharmacy dispensing data before the index AF diagnosis date. We used procedure codes, diagnosis codes, and registry data to define code-based recurrent AF.

Free text data sources for NLP development and validation

We identified and extracted three types of free text data that were likely to contain AF recurrence information. First, 12-lead ECG reports were extracted from the GE Healthcare MUSE Cardiology Information System at site 1. In site 2, we searched ECG-related keywords in EHR data sources to retrieve presumptive ECG reports (see Supplementary material online, Text S1). Second, we collected cardiac monitoring reports from implanted and wearable cardiac rhythm monitors. These included reports from Holter, patch recorder, cardiac event monitor, pacemaker, loop recorder, implantable cardioverter defibrillator, and cardiac telemetry. These cardiac rhythm monitor reports were not consistently collected and stored in the EHR. In site 1, cardiac monitor reports were stored as procedure reports, or as telephone encounter notes. In site 2, cardiac monitor reports were stored with other general clinical notes (see Supplementary material online, Text S1). Lastly, we included clinical notes from inpatient and outpatient encounters based on the visiting department (e.g. cardiology, family practice), authorizing provider (e.g. physician, physician assistant), and note type (e.g. progress note, discharge summary) that were likely to include assessments of the patient's AF status (details are available in Supplementary material online, Table S2).

We defined AF recurrence as a newly detected AF event documented in an ECG or cardiac monitor report, or a current or recently occurred AF event as documented in the clinical note. In clinical notes, the occurrence of an AF event was based on the objective findings documented in the Physical Exam section (e.g. irregular rhythm) or based on the assessment of contextual information such as temporal references surrounding the AF documentation.

Overall natural language processing algorithm development process

NLP algorithms were first built using training data from site 1 (Figure 1). We developed separate algorithms for the three types of clinical documents (ECG reports, cardiac monitoring reports, and clinical notes). Each document was categorized by the NLP algorithms as having a positive or negative AF recurrence. After building the initial algorithms, the algorithms were tested using development data. We refined the algorithms until we reached prespecified performance levels for positive predictive value (PPV) and negative predictive value (NPV) >90%. Once algorithm performance targets were reached in site 1, the algorithms were tested using development data from site 2. The algorithms were further refined to meet the prespecified performance thresholds at both sites. The final algorithms were validated at both sites using the validation datasets.

Training, development, and validation datasets

For site 1, all the ECGs, cardiac monitoring reports, and clinical notes during the 12 months following the first AF treatment date were included as the training, development, and validation datasets. We first created validation datasets (n = 100) at each site using random stratified sampling, dividing them equally between those who started antiarrhythmic drugs without an AF procedure and those who underwent an AF procedure (Figure 1). From the remaining cases, we created the development datasets (n = 1000) at both sites, which were reserved to test and refine the algorithms. The remaining cases from site 1 (n = 9368) served as the training dataset to build the initial NLP algorithms at site 1.

NLP algorithm development

We developed the initial NLP algorithms at site 1.^15–20 The steps for pre-processing text and developing terminology were described in the Supplementary material online, Text S2. We created the rule-based NLP algorithms with the Linguamatics I2E software (Linguamatics, an IQVIA company, Cambridge, United Kingdom), which allows for the rapid extraction of unstructured data via knowledge integration and parallel indexing and searching via rules-based search algorithms. The NLP algorithms were developed to search each indexed note at different levels: section (e.g. ‘Physical Exam’, ‘Assessment/Plan’), intra-sentence, and cross-sentence. A distance-based relationship algorithm was applied to identify related terms based on the number of words or sentences between them. The relationship search identified the words or phrases (e.g. negated, uncertain, and hypothetical statements) that modify the ascertainment of interested concepts (e.g. AF, irregular heart rhythm). These terms were based on the ‘Context Terms’ ontology provided by I2E software. Examples of the relationship search are provided in the Supplementary material online, Text S2. The temporal relationship algorithm identified past AF recurrence events (e.g. ‘3 months ago’, ‘history of’, and ‘after 12/19/14 admission for atrial fibrillation’). We tested and updated the algorithms based on the adjudicated medical records performed by four physicians on our study team during the algorithm development process.

NLP algorithm validation

The same physicians who manually adjudicated the initial NLP results during the algorithm development also independently reviewed the free-text data in the validation datasets. Two cardiologists (M.S.L., C.C.) independently reviewed 451 reports from site 1 to determine true recurrent AF cases. The results of the two cardiologists’ reviews were compared, and conflicts were resolved through discussions and consensus. In site 2, two internists (B.L., N.B.) reviewed 826 reports using the same way as in site 1. All adjudicators were blinded to the NLP results. The adjudicated results served as the reference standard against the NLP algorithms. We evaluated the performance of the final NLP algorithms against the reference standard cases (n = 100 for each site) created by physician reviewers.

Definition of recurrent AF for the patient-level analysis

We applied the final NLP-based algorithms along with code-based algorithms to the entire cohort of patients with incident AF initiating rhythm control therapy and identified definite and probable AF recurrence during the subsequent 12 months. In the code-based algorithm, a definite AF recurrence was defined by CPT codes or health plan registry data for ablation or cardioversion, which is considered a repeated procedure after their index procedure. A probable AF recurrence in the code-based algorithm was defined by hospitalization with a primary diagnosis of AF or receipt of pacemaker implantation in combination with nodal ablation. In the NLP-based algorithm, a definite AF recurrence was defined as an AF event identified from ECG or cardiac monitor reports. A probable AF recurrence was defined as an AF event identified only from physician clinical notes.

Statistical analysis

We used descriptive statistics for patient demographic and clinical characteristics using means (SD) for continuous variables, and frequencies (percentages) for categorical variables.

To calculate the NLP algorithm performance measures, we first determined the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) identified by each NLP algorithm per type of clinical document and site. Then, we calculated the sensitivity, specificity, PPV, NPV, and F-score²¹ (also called the F1 score when |$b{\rm{\ is\ }}1$|⁠, which is the harmonic mean of sensitivity and PPV) and 95% confidence intervals. The F-score ranges from 0 to 1, with higher values indicating better predictions and a value of 1 indicating perfect classification.

We reported the numbers (percentages) of patients with visit reports and AF recurrence by applying code-based, NLP-based, and combined (both code-based and NLP-based) algorithms for each rhythm control treatment type (ablation, antiarrhythmic medication, cardioversion, and pacemaker) and per site. The primary analysis evaluated AF recurrence during the first 12-month post-treatment period. In a sensitivity analysis, we evaluated AF recurrence during the first 3 months, and months 4–12.²² SAS version 9.4 (SAS Institute, Cary, NC, USA) was used for data analysis.

Results

Study population

We identified 22 970 eligible patients with incident AF. The mean ± SD age of the cohort was 70.9 ± 11.0 years, 42.2% were women, 80.4% were White, 6.1% were Black, 8.8% were Asian or Pacific Islander, and 11.9% were of Hispanic ethnicity (Table 1). A sizable proportion of patients had comorbidities of dyslipidemia (80.7%), hypertension (78.2%), diabetes mellitus (33.5%), chronic heart failure (21.5%), and ischemic stroke or transient ischemic attack (5.1%). Both sites had similar patient demographic and clinical characteristics (see Table 1 and Supplementary material online, Table S3).

NLP algorithm validation

Compared with the physician-adjudicated reference cases of recurrent AF, F-scores were 0.995 and 0.953 (sites 1 and 2) for the NLP algorithms applied to ECG reports, 0.928 and 0.870 for the algorithms applied to cardiac monitoring reports, and 0.928 and 0.911 for the algorithms applied to clinical notes (Table 2). The sensitivity, specificity, PPVs, and NPVs were all above 0.90 for the NLP algorithms applied to the ECG reports and clinical notes at both sites. The only exception was the algorithm applied to the cardiac monitoring reports for site 2 with a sensitivity of 0.833. The specificity was higher than the sensitivity on most of the validation datasets. The algorithm was found to consistently perform well in both female and male groups.

Identification of recurrent AF

The final cohort included 987 patients who underwent ablation, 19 841 patients treated with AADs, 7828 patients who underwent cardioversion, and 76 patients with combined pacemaker implantation and nodal ablation. The percentages of patients who had at least one clinical document potentially addressing recurrent AF during the first 12 months are shown in Table 3 for each type of clinical data source (ECG, cardiac monitor, and clinical notes). Over 90% had at least one eligible clinical data source during the first 12 months post-treatment regardless of the type of rhythm control therapy. Clinical documents were more prevalent in patients undergoing cardiac procedures than in those taking AADs. The variation was even greater for ECG or cardiac monitor reports. For example, although 94.5% and 98.9% of patients with ablation had ECG or cardiac monitor reports, only 79.2% and 79.3% of AAD patients had ECG or cardiac monitor reports. In addition, not all selected clinical free-text data commented specifically on the presence or absence of AF (see Supplementary material online, Table S4).

The numbers and percentages of patients who experienced AF recurrence in the first 12 months after AF therapies are shown in Table 4 (see Supplementary material online, Table S5). For sites 1 and 2, the percentages of definite or probable AF recurrence identified by the NLP-based algorithms between the first 12 months were 60.7 and 69.9% (ablation), 64.5 and 73.7% (cardioversion), and 49.6 and 55.5% (AAD), respectively. The numbers of patients with AF recurrence found by NLP are two to three times higher than those found by codes (Table 4). The percentages of patients with code-identified AF recurrence for sites 1 and 2 were 20.2 and 23.7% for ablation, 25.6 and 28.4% for cardioversion, and 20.0 and 27.5% for antiarrhythmic medication, respectively. The great majority of recurring cases found by code or NLP algorithm were identified by the NLP algorithm. The findings were consistent when we examined the recurrence in 1–3 months and 4–12 months (see Supplementary material online, Tables S5–10). The percentages of AF recurrence were comparable between the two study sites (see Figure 2 and Supplementary material online, Figure S1).

Discussion

We developed and validated NLP algorithms to identify valid recurrent AF episodes using various clinical data sources from EHRs. Compared with the physician-adjudicated reference standard, the NLP algorithms showed high accuracy, and consistently performed well at two large integrated healthcare delivery systems across different clinical data sources. Compared with code-based algorithms commonly used in retrospective observational studies, the NLP algorithms identified far more recurrent AF cases. These results not only verify the NLP method but also suggest that code-based algorithms may significantly underestimate rates of recurrent AF.

Previous studies identified AF recurrence using ECG and cardiac monitors, which were used in this study to detect NLP-definite AF recurrence. The percentages of patients with NLP-definite AF recurrences were comparable to other studies. The proportions of ablation patients with 1-year NLP-definite AF recurrence (47.0% site 1, 54.6% site 2) were similar to the percentage (45.9%) reported from a prospective multicenter real-world study (n = 3679) based on the German national ablation registry.⁵ Patients in that study were evaluated every three months at each ablation clinic, which included a personal phone interview, ECG, and 24-hour Holter monitor. The percentages of patients with NLP-definite AF recurrence during the first three months following ablation (27.3% site 1, 17.6% site 2) in our study were also consistent with the percentage (24.1%) reported in a study of 3120 Korean patients who had their first ablation.²³ Patients in that cohort study had ECG during their 2 weeks’ outpatient visits following discharge and were then recommended to use portable cardiac monitors if they developed symptoms suggestive of recurrence. Following the 3-month blanking period, the percentages of ablation patients with NLP-definite AF recurrence during the first year (33.3% site 1, 48.6% site 2) were similar to the percentage (36.4%) reported in the CABANA study (n = 611).²⁴ In that multicenter randomized controlled trial, AF recurrences were captured using symptom-activated recording and a monthly 24-hour cardiac monitor.

In real-world settings, many patients do not get repeated ECG or cardiac monitoring as patients enrolled in studies using protocol-driven cardiac monitoring at specific follow-up times.^5,10,23,24 Our study makes a novel contribution by detecting AF recurrence using EHR-based outpatient and inpatient clinical notes, which could improve the identification of AF recurrence in real-world clinical settings or pragmatic studies. For example, after incorporating clinical notes, the NLP-detected recurrence rates for the ablation group increased from 33.3 and 48.6% (definite AF) to 45.1 and 58.6% (definite or probable AF) for the two sites between months 4 and 12. The differences were more notable for the other treatment groups. The growing number of probable AF recurrences by therapy types could be attributed to patient characteristics and care practices. For instance, patients who underwent AF procedures are more likely to have more follow-up clinic visits and receive greater surveillance monitoring than patients who were only treated with AADs.

Even though the two sites shared many similarities, we discovered variations in how the EHR systems collected and stored data. The growing availability of cardiac monitoring technologies allows more sensitive detection of AF recurrence. However, various types of monitoring devices present a barrier to obtaining their results from complex EHR systems. Care providers, for example, could obtain results from many different sources, such as telemetry, or be notified by monitor device manufacturers. These results were routed and recorded in various locations in the EHR systems depending on data sources, institutions, hospitals, departments, and providers. With the growing adoption of new consumer-based wearable devices with irregular rhythm notification capabilities, integrating wearable data into EHRs requires careful planning to minimize data silos, ensure clinical accuracy, and enhance meaningful use.²⁵

The evolution of EHRs has also made it easier to create lengthy and bloated notes.^26–28 Cardiology is one of the specialties with the longest notes and the largest note redundancy (73% are ‘the amount of text identical to the patient's last note’), with 37 and 40% of contents being copied or created from templates, respectively.²⁸ In this study, we observed that when managing AF patients, cardiologists frequently copied and pasted the historical AF visit information, which is likely used to support recall and clinical reasoning for disease management. These note texts may lack the detailed temporal information necessary for an NLP algorithm to exclude previous AF episodes. Previous NLP efforts on identifying possible copied note text used sequence alignment to find identical sequences of text across notes.^28,29 In this study, we used methods such as excluding sections and statements that may have been copied forward as historical information. The most effective approach for detecting copied text may be EHRs that employ markup tags to explicitly annotate copied content, which can be evaluated in future studies.

One of the primary goals of AF rhythm control therapy is to reduce AF recurrence and any associated symptoms. While identifying AF recurrence is clinically important, frequent ECG monitoring employed in large prospective research studies is not usually feasible due to cost and logistical reasons. Furthermore, reliance on administrative diagnosis code-based identification using EHRs may significantly underestimate or overestimate AF recurrence. Moreover, due to the time and resources required, population-based retrospective studies to evaluate AF recurrence by manual chart review would be extremely difficult to conduct.

Using NLP algorithms, we could gain a more complete understanding of AF recurrence, facilitating clinical research and improving system-level patient care strategies. In research studies, our NLP algorithms can aid in assessing the efficacy of various rhythm control therapies, identifying risk factors for AF recurrence, evaluating long-term outcomes, and directing future research.^3,8 The NLP programme, for example, could identify the timing and frequency of AF occurrences before and after AF treatments, which can evaluate effectiveness within and between AF treatments. Automatic detection of AF recurrence using NLP could benefit not only retrospective studies but also prospective studies and clinical trials including patient recruitment.³⁰ The NLP algorithms could also help clinicians monitor disease progression, optimize patient management, and predict future outcomes in clinical practice.⁹ For instance, early AF recurrence was found to be a predictor of later AF recurrence after ablation,²³ which can lead to future treatment strategies to prevent early AF recurrence and improve long-term treatment success.

Study strengths and limitations

This study was performed within two large, well-characterized, diverse community-based populations within two large integrated healthcare delivery systems with comprehensive EHRs. The included health plans provide strong incentives for members to use services at our owned facilities, so clinical documentation is likely to be more comprehensive than in EHRs representing patients whose care is more fragmented. We developed separate NLP algorithms to identify AF recurrence from multiple free-text data sources within EHRs, including clinical notes, which provide a wealth of information but also have significant variability in terms of structure, content, and accuracy. The algorithms were highly accurate, as independently validated through physician adjudication of relevant EHR information. Our rule-based NLP programmes require far fewer computational resources than deep neural network (DNN)-based programmes. Unlike DNN-based NLP, which requires large amounts of annotated data for training, rule-based NLP can be implemented with relatively small amounts of data. In addition, DNN models can be difficult to interpret due to their black-box nature. Rule-based systems, on the other hand, are intended to be more transparent and explainable. Having these advantages makes rule-based systems more practical for clinical use.

This study has several limitations. We did not include more patients in the validation data due to time and resource constraints, which resulted in wide CIs for some NLP performance measurements. Due to the small validation data sample size, we were unable to evaluate the algorithm performances by race and ethnicity. We found minor performance degradation when the algorithms were evaluated on data from site 2. Performance variances may be related to differences in the training data or data sources. Additionally, reporting language and style can differ between institutions and physicians. Our NLP method may perform differently in other test datasets. Another limitation was that AF recurrence detection can be affected by the frequency of AF-related clinical encounters and testing, which varies among healthcare systems, clinicians, and patients.³¹ For instance, site 2 had a much larger proportion of patients identified by cardiac monitor data than site 1 (39.9% vs. 27.3%). The intensity of visits and monitoring may be also influenced by patients’ symptoms or type of therapy. For instance, patients with symptomatic AF are more likely to receive more frequent consultations, monitoring, and rhythm control therapies.⁷ As a result, patients in this study may have more intense monitoring than individuals receiving just rate control therapy. Compared with studies where participants were monitored by cardiac monitors at preset intervals (e.g. every 3 or 6 months),¹⁰ our study may detect more recurrences with temporal dispersion of the testing.³¹ On the other hand, our method may identify fewer recurrences for patients who had fewer clinical encounters. Moreover, AF recurrence from EHR data relied on clinicians’ interpretation, which may not always adhere to a standardized criterion (e.g. documented 30 s of AF) for establishing a recurrent AF episode.^24,32,33 Our study was also unable to assess the AF burden because all patients were not consistently monitored for a sufficient amount of time with relevant technologies.³³ Finally, the accuracy of the code-based methods was not validated in this work. Although our results show that the code-based algorithms appear to have low sensitivity, we were unable to analyse their specificity and PPV.

Conclusions

We developed and validated a robust automated NLP method for identifying AF recurrence. Compared with the code-based methods, the NLP algorithms identified and classified significantly more patients with recurrent AF. We demonstrated how a computational tool might be used to support population-based care or a study using real-world EHR data. The automated identification of AF recurrence may aid future studies on the effectiveness of AF treatment and help develop tailored interventions.

Funding

This work was supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health (R01HL142834).

Conflicts of Interests: D.E.S. has received research support from Bristol Myers Squibb and served as a consultant to Bristol Myers Squibb, Fitbit (now Google), Medtronic, and Pfizer. A.S.G. has received research funding through his institution from iRhythm Technologies, the Bristol Myers Squibb/Pfizer alliance, and Janssen Research and Development. The remaining authors have no conflicts of interest to report.

Data availability

The data underlying this article cannot be shared publicly due to HIPPA and institutional restrictions. The requests for NLP algorithms should be made to the corresponding author of this paper. They will be forwarded and considered on an individual basis by the Kaiser Permanente research and legal departments to verify whether the request is subject to any intellectual property or confidentiality obligations.

References