Abstract

Conventional randomized controlled trials (RCTs) can be expensive, time intensive, and complex to conduct. Trial recruitment, participation, and data collection can burden participants and research personnel. In the past two decades, there have been rapid technological advances and an exponential growth in digitized healthcare data. Embedding RCTs, including cardiovascular outcome trials, into electronic health record systems or registries may streamline screening, consent, randomization, follow-up visits, and outcome adjudication. Moreover, wearable sensors (i.e. health and fitness trackers) provide an opportunity to collect data on cardiovascular health and risk factors in unprecedented detail and scale, while growing internet connectivity supports the collection of patient-reported outcomes. There is a pressing need to develop robust mechanisms that facilitate data capture from diverse databases and guidance to standardize data definitions. Importantly, the data collection infrastructure should be reusable to support multiple cardiovascular RCTs over time. Systems, processes, and policies will need to have sufficient flexibility to allow interoperability between different sources of data acquisition. Clinical research guidelines, ethics oversight, and regulatory requirements also need to evolve. This review highlights recent progress towards the use of routinely generated data to conduct RCTs and discusses potential solutions for ongoing barriers. There is a particular focus on methods to utilize routinely generated data for trials while complying with regional data protection laws. The discussion is supported with examples of cardiovascular outcome trials that have successfully leveraged the electronic health record, web-enabled devices or administrative databases to conduct randomized trials.

A visual representation of how leveraging routinely collected data and technology can streamline all aspects of randomized controlled trials. EHR, electronic health record.
Graphical Abstract

A visual representation of how leveraging routinely collected data and technology can streamline all aspects of randomized controlled trials. EHR, electronic health record.

Introduction

While randomized controlled trials (RCTs) have played a central role in advancing cardiovascular medicine, contemporary RCTs have become highly complex and resource-intensive.1,2 Most cardiovascular RCTs are designed to maximize estimated treatment effect and minimize adverse events (i.e. explanatory design), rather than demonstrate effectiveness in everyday clinical patients and settings (i.e. pragmatic design).3 Explanatory trial designs involve the use of narrow inclusion criteria that select patients most likely to derive benefit from the intervention. These inclusion criteria are often enriched for the primary outcome to ensure an adequate event rate to minimize sample size and shorten the time required to detect a treatment effect.4 The need to increase pragmatism in RCTs is being increasingly recognized. The Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) tool allows investigators to assess the pragmatism of a trial design in each of nine domains: eligibility, recruitment, setting, organization, flexibility in delivery, flexibility in adherence, follow-up, primary outcome, and primary analysis.5 The move towards pragmatism in cardiovascular outcome trials (CVOTs) could potentially benefit from leveraging real-world data and technology—which have the potential to reduce workload for investigators, increase the speed of evidence generation, and ameliorate costs, all while yielding more generalizable findings.

The advent and implementation of electronic health records (EHRs) have revolutionized healthcare delivery by allowing better organization and access to patient data, improved adherence to clinical guidelines, and reduced prescription errors, among other advantages.6–9 The use of EHRs has been adopted in virtually all hospitals in the USA and Europe, as well as many other regions of the world where RCTs are led.10 Many jurisdictions, in some cases entire countries, have registries that systematically capture consecutive patient data from a range of healthcare settings. Technological devices such as smartwatches, smartphones, and fitness trackers can record heart rhythm, pulse, and other biometric measures, enabling cardiovascular health to be tracked with unprecedented detail and scale.11 Biomedical and healthcare data volumes are increasing faster than most other industries,12 with data being generated from multiple sources as highlighted in Figure 1.

Sources of digital data in the healthcare industry.
Figure 1

Sources of digital data in the healthcare industry.

Integrating RCTs with existing data-generating systems (EHRs, registries, and wearable sensors) can potentially automate and streamline the identification of eligible patients, patient recruitment, informed consent, randomization, follow-up monitoring, outcome ascertainment, and outcome adjudication (Graphical Abstract). This, in turn, can reduce the trial burden on site personnel and participants, and decrease costs.13 Access to data routinely collected in clinical care for the purpose of trial recruitment can potentially broaden access to RCT participation. Moreover, the decreased organization and costs associated with using routinely generated data can allow the conduct of trials that compare healthcare policies or treatment strategies (different drug combinations or sequencing), i.e. trials which do not seek a regulatory label and are thus less likely to be industry funded. The conceptual workflow involved in a trial conducted using technology and routinely generated healthcare data is summarized in Figure 2. Despite its potential, the implementation of data systems and technology into the conduct of RCTs has lagged. In this review, we discuss how integration of access to real-world datasets and technology can optimize the conduct of CVOTs, the current barriers to integration, and potential solutions.14

Workflow involved in an automated technology-driven randomized clinical trial. EHR, electronic health record; EKG, electrocardiogram; SpO2, oxygen saturation.
Figure 2

Workflow involved in an automated technology-driven randomized clinical trial. EHR, electronic health record; EKG, electrocardiogram; SpO2, oxygen saturation.

Patient identification, informed consent, and recruitment

Challenges in conventional cardiovascular trials

Inadequate recruitment is the most important reason for premature termination of CVOTs; up to 80% of conventional RCTs do not reach their planned enrollment targets.15 To ensure adequate recruitment, enrollment duration and number of participating sites are often increased, each of which increases costs.16 In most trials, accrual rates are slow at the start, followed by a steep mid-section, and then a plateau which represents trial fatigue.17 In longer trials, trial fatigue can lead to site discontinuation, patient dropout, and increased cross-over across treatment arms. To deal with these consequences, protocol amendments and changes in the trial endpoint are often required. A study of 836 clinical trials showed that almost half of them (45%) had at least one substantial protocol amendment.18 The median number of amendments was 2.3 in a Phase 3 clinical trial. Factors associated with amendments included larger sample size, longer patient recruitment durations, and slow accrual rates.18

A common reason for recruitment difficulties in conventional trials is the use of narrow eligibility criteria. While trials often select patients at increased risk for the outcomes of interest, they also commonly exclude the very highest risk patients, such as older adults and those with extensive and advanced comorbidities, due to the competing risks of death and elevated safety/tolerability concerns. For example, more than 25% of heart failure (HF) trials have excluded patients by an arbitrary upper age limit, and over 80% exclude patients with one or more serious or advanced comorbidity.19 Given that elderly patients with multiple comorbidities make up a substantial proportion of patients with cardiovascular disease, this compromises the generalizability of the results to the broader population of afflicted patients.

Other common reasons for poor recruitment include limited availability of routinely collected data on potentially eligible trial participants to inform study design and recruitment strategies, and a lack of clarity in understanding the trial objectives and inclusion criteria among site investigators.20 Moreover, current recruitment practices often result in eligible patients being missed, due to study coordinators not being notified that there is a potential patient eligible for the study.

Role of routinely generated data and technology

The integration of routinely collected data into standard RCT operations provides a potential solution to streamline the process of pre-screening and obtaining informed consent, while facilitating enrollment of a more diverse and representative patient population. The EHRs or registries can be prospectively queried to identify sites with high volumes of trial-eligible patients.20,21 In addition to identifying the volume of trial-eligible patients, EHRs and administrative databases can be queried to estimate event rates for outcomes of interest. For example, Rapsomaniki et al.22 developed a model using an EHR cohort of >100 000 patients with stable coronary artery disease which could reliably predict all-cause mortality (c-index: 0.81) and the composite of non-fatal myocardial infarction or coronary death (c-index: 0.78). This has an important implication for clinical trials—the estimated event rates from EHR can assist with the calculation of power.

During the recruitment phase, systems can be put into place to automatically pre-screen patients for eligibility for trials. This process is relatively straightforward if patient characteristics of interest are available in a structured and standardized form within the EHR. In the absence of structured data, clinical narratives (i.e. free text) in the EHR can be analysed using natural language processing to identify potentially eligible patients. Natural language processing is a subfield of artificial intelligence that enables the computer to analyse free text using a collection of processing algorithms based on statistics, syntactic, and/or semantic rules. It has well-established utility for pre-screening patients and is increasingly being used to pre-screen patients for cardiovascular disease trials.23

Alerts within EHRs can be used to systematically inform providers of patients potentially eligible for a trial during routine clinical encounters. At the same time, the provider can be provided with a short description of the trial. In 2011, a pilot RCT was conducted to compare weight-based insulin dosage with a sliding scale insulin dosing regimen in hospitalized patients for the control of hyperglycaemia.24 For this RCT, the Veterans Affair Healthcare System was optimized to automatically identify eligible patients (those for whom insulin was being ordered) and notify treating providers of potential recruitment for the trial. If a potentially eligible patient was identified by the system, the treating provider would be presented with a short summary of the trial and then given the option to enrol the patient. With agreement of the treating provider and consent of the patient, the research team would randomize patients. Strategies such as these can considerably reduce the need for additional research personnel and trial-specific training for site investigators to encourage recruitment. They can also ensure that potentially eligible patients are not missed—thereby improving the inclusiveness of trials and speed of enrollment.

Sometimes, clinical trials seek to recruit participants with a specific severity of symptoms. For example, almost all CVOTs use the New York Heart Association (NYHA) classification as part of the screening criteria. Unfortunately, the NYHA class is often not recorded in charts and even if it were, the known interphysician variability in its assessment can minimize its validity.25–28 The growing availability to the internet through smart devices can enable the collection of validated patient-reported health status, which has greater reproducibility, sensitivity, and prognostic value than clinician-assigned assessments. This can help identify sufficiently symptomatic patients to meet the goals of the RCT without the requirements and costs of a trial-specific face-to-face visit. Additionally, if self-reported health status suggests that the patient requires clinical care, the smartphone application can notify both the patient and the treating physician regarding this.29,30

Once eligible patients are identified from EHRs or registries, consent may be obtained remotely. Patients can be informed about risks, benefits, and financial disclosures via videos that use patient-centred language and minimal medico-legal jargon. This was exemplified in the Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) trial, which used a US Food and Drug Administration (FDA)-compliant electronic informed consent method.31,32 Potentially eligible patients (identified through EHRs) were provided with a link to the trial web portal, along with a personalized code to access it. Within the portal, participants were presented with an informational video about the trial and a consent form. The consent form was adapted locally at each site and approved by the institutional review board of the site. An on-site, in-person consent process was available for those who were uncomfortable with technology or had no internet access. The FDA requires verification of the identity of the person signing the consent form, but does not mandate a particular method for verification. Possible verification methods include the use of information from official identification such as birth certificate or driver’s license, or the use of security questions.33

From an investigator perspective, remote e-consent for research allows contact with a much larger number of patients compared with the traditional practice of approaching patients encountered in routine clinical practice. From the patient perspective, bringing the trial to the patient, rather than the patient to the trial, is more convenient and can facilitate engagement with relatives to help make an informed decision on whether to participate. The potential downside of video-based consent and recruitment is that it may be challenging for certain patients. For example, non-native English speakers may not be fully ‘informed’ prior to making a decision, unless translation is provided. Similarly, elderly patients who are unfamiliar with new technology may find this process difficult.

Data collection

Challenges in conventional cardiovascular trials

Conventional RCTs involve extensive data collection, with multiple protocolized trial-specific visits, diagnostic tests, and surveys routinely required to gather data. More than a fifth of these procedures, although complementary to the trial integrity, are not directly related to the core objectives of the RCT.34,35 For industry-sponsored RCTs being conducted for product registration, partial or complete source data verification is conducted—a practice wherein the collected data are checked by study monitors to ensure that they are complete and consistent with study participant source records. Source data verification is expensive and can consume a large proportion of the trial budget.34,35 Moreover, source data verification is responsible for a major chunk of the overall carbon footprint of clinical trials.34,36 However, it remains unclear if source data verification yields any benefit.34,35 The FDA does not require extensive source data verification and recommends limiting verification to essential data points or when anomalies are suspected.37 Despite its unproven utility, it is commonly used due to a conservative interpretation of regulations and the notion that it preserves integrity of trial results.34

Role of routinely generated data and technology

The use of routinely collected clinical data can reduce the burden of trial-specific data collection. Patients’ demographics, clinical course, diagnostic test results, treatment regimens, and clinical events of interest are typically updated in the EHRs or registries with every in-person or telehealth visit with the provider. In many countries, data from hospital records are systematically coded and transferred to federal administrative datasets. With the appropriate technological infrastructure in place, this data collection may be standardized and captured for use in clinical trials. For investigators and stakeholders, automated data standardization and capture from EHR, registries, and digital devices can reduce the need for personnel and sites to support data collection and reduce the risk of transcribing errors. Estimates show that automated data capture to populate RCT case report forms can result in a >5% reduction in the cost of cardiovascular clinical trials.34,35 Automated data collection may also allow much longer and detailed follow-up than conventional trials, increasing sensitivity to capture important rare or delayed adverse effects. Importantly, to make routinely generated data optimally ‘research-accessible’, technological infrastructures, processes, and codification of terms need to be broadly generalizable and ideally harmonized, such that any cardiovascular RCTs could capitalize on such access.

Data derived from wearable devices, smartphones, and other technologies in everyday use can be used to remotely track specific parameters of cardiovascular health of enrolled participants. Large-scale e-registries that gather cardiovascular data already exist. A pertinent example is the MyHeart Counts study, which obtains data on physical activity, fitness, and sleep via a smartphone application.38 Almost 50 000 participants across the USA voluntarily consented and were enrolled into this study within 8 months since the launch of the application. Similarly, the Health eHeart study—which enrolled over 200 000 participants—also collects cardiovascular health and fitness data remotely through a smartphone application (https://www.pcori.org/research-results/2015/health-eheart-alliance). In addition, a subset of participants is given gadgets, such as wearable sensors and Bluetooth-linked blood pressure measuring devices, to gather more in-depth data. The Apple Heart Study is an example of a technology-based, pragmatic study design where a smartwatch was used to detect irregular rhythm in >25 000 trial participants, and linkage to administrative data is being used to assess the clinical outcome benefits of being exposed to the device.39 The notifications for irregular pulse are transmitted through an application to study investigators following which participants were sent an electrocardiogram patch to characterize the irregular rhythm, determine if atrial fibrillation was present, and among those with atrial fibrillation, classify it as paroxysmal or continuous. The Apple Heart Study, MyHeart Counts, and Health eHeart highlight the potential of technology to enhance the process of obtaining consent, collecting data, and recruiting large sample populations. Moreover, data gathered from commonly used devices and wearables in the community may provide detailed insights that may not be gleaned from clinical evaluation. This was demonstrated by the results from the Nitrate’s Effect on Activity Tolerance in HF With Preserved Ejection Fraction (NEAT-HFpEF) trial,40 which studied the effect of isosorbide dinitrate on activity levels in patients with HF and preserved ejection fraction. Two concurrent outcomes were used to evaluate physical activity—the conventional 6-min walk distance and daily physical activity level measured by wearable activity monitors with high-sensitivity accelerometers. Isosorbide dinitrate (all doses vs. placebo) was found to have no effect on physical activity when measured using the 6-min walk distance, but daily activity was significantly reduced by isosorbide dinitrate when measured using the wearable sensors. This discrepancy highlights that conclusions from RCTs are often based on limited data that are gathered during office visits and in restricted/selected patient cohorts, and the use of technology may increase sensitivity by obtaining more complete and continuous information, without placing additional burden on the participants. Observational studies conducted using wearable technologies would exclude participants who cannot afford or do not have access to smartphones and wearable devices—which may create further inequities in healthcare research. However, this is less of a concern in RCTs, where the required devices are often provided to enrolled participants.

Once data have been captured from EHRs or registries, algorithms may be applied to identify suspicious data patterns in clinical trials. The algorithms can potentially identify anomalous values at least at two levels: at the participant level and site level. Potentially erroneous data at the participant level can then be verified by manually cross-checking the case report forms with the source data. If one or more sites have data that deviate considerably from the overall pattern, the sites can be assessed for systematic errors in data collection or entry, which can then be rapidly addressed. This risk-based monitoring approach to clinical trial surveillance can allow the early detection of site failure, fraud, and data inconsistencies or incompleteness—while simultaneously reducing the costs associated with source data verification.41

Recently, it has been shown that linking genetic biorepositories with EHR can facilitate the conduct of genome-wide association studies (the identification of various genetic variants associated with a phenotype) and phenome-wide association studies (the identification of various phenotypes associated with a genetic variant).42,43 Moreover, clinical outcomes and treatment allocation may be improved if the identified gene-phenotype associations are returned to the EHR and used to support clinical decisions (e.g. by incorporating genetic information into prognostic risk scores).44,45 This has important implications for the conduct of embedded clinical trials. If EHRs linked to biorepositories are used to conduct RCTs, the data can be used to identify patients with genotypes/phenotypes, which are most likely to respond positively to the intervention. This can potentially allow precise treatment allocation, which maximizes efficacy and safety, and minimizes futile use of the intervention.46

Barriers to the implementation of routinely generated data, possible solutions, and current progress

Despite the immense potential of using existing information systems for clinical trials, several barriers need to be overcome before this process can become a standard (Figure 3).

Current barriers and potential solutions to the implementation of technology-driven randomized controlled trials. *Electronic health record system vendors that control the degree of data sharing with other electronic health record systems. EHR, electronic health record.
Figure 3

Current barriers and potential solutions to the implementation of technology-driven randomized controlled trials. *Electronic health record system vendors that control the degree of data sharing with other electronic health record systems. EHR, electronic health record.

Inaccurate and incomplete codification of routinely generated data

Incomplete codification of data in EHRs and registries represents one of the biggest barriers for EHR-embedded research. Provider notes within the EHR are extensively and almost exclusively in the form of free text.47 Formal diagnosis lists, and problem lists that map onto objective clinical codes, are often insufficiently curated in the EHR, greatly compromising their utility for screening and follow-up of patients for research. In addition to being incomplete, the codified data are often inaccurate. The International Classification of Disease (ICD) diagnosis and procedure codes, a comprehensive system established by the World Health Organization to harmonize and codify clinical and procedural annotations, has been adopted by healthcare systems across the globe. However, there is no common imperative to document these codes (at least in the USA), and the codes have questionable sensitivity and specificity, as highlighted by Fawcett et al.48 in their study of diagnosis codes for infective endocarditis. Only 44% of total cases assigned with endocarditis codes actually had endocarditis, while 24% of patients with possible or definite endocarditis were not assigned their respective ICD codes.48

Potential solutions and current progress

To allow appropriate use of structured/coded EHR data for research, completeness and accuracy of structured data need to improve. To overcome incomplete availability of structured data, EHRs should be designed to curate data that are more complete for use in research. Optimization of these systems for research would require guidelines as well as incentives for manufacturers. The inaccuracy of codified data can be addressed by continued evolution in training and practice for clinicians to accurately document diagnoses in problem lists and history domains. In the absence of accurately codified EHR data, evaluation of clinician notes (free text) may be a more reliable method to determine diagnoses. However, in RCTs, clinician notes are traditionally reviewed by personnel—which adds to the workload and costs. Artificial intelligence tools, such as natural language processing, have the potential to parse unstructured text (including but not limited to clinician notes) in the EHR and convert it to structured data. Clinicians may use different terms to refer to the same thing (such as ‘hypertension’ and ‘elevated blood pressure’)—natural language processing can accumulate similar terms under the same code. The structured data may then be used for research purposes.49 Although the accuracy of natural language processing–derived data has increased greatly over the past decade, there remains room for improvement, and models should be internally validated before use for research. Research studies utilizing structured EHR data should follow best practices, such as those laid out in the recently published CODE-EHR framework.50

Patient privacy and legal concerns

Conducting embedded RCTs requires sharing of patient data across healthcare systems and with investigators, inevitably placing patient privacy at risk. This raises the critical question about how patient data can be shared for the successful conduct of RCTs, while maintaining patient confidentiality and data security. The use of interoperable EHRs for clinical care is already underway, especially in Europe. For example, in the UK, almost 75% of health systems have some means in place to electronically access patient data from other healthcare systems. The eHealth Digital Service Infrastructure (eHDSI) is an infrastructure that further allows exchange of data across countries in the European Union (EU). By 2025, patient summaries and prescription information should be exchangeable across 25 countries in the EU. Both the General Data Protection Regulation (GDPR) in the EU and the Health Insurance Portability and Accountability Act of 1996 (HIPAA) in the USA require measures to ensure security of health information. These measures include limiting access to authorized individuals, encryption of stored data, and an ‘audit trail’ feature (which records who accessed a patient’s information, what changes were made, and when).51,52 While these measures are sufficient for clinical care, policies on data sharing for research are less simple.

The GDPR in the EU and the Common Rule (a policy by the Department of Health and Human Services) in the USA regulate the use of personal patient data for research. Both allow the use of personal patient data for research if the data are appropriately de-identified or if the patients have provided informed consent for research. However, the definitions of ‘appropriate de-identification’ and ‘consent’ should be noted. The GDPR is more stringent than the Common Rule. For example, in order to waive the requirement for consent, GDPR requires data to be anonymized—the highest form of de-identification. Anonymization involves the removal of all personal data, including direct identifiers (e.g. name, telephone number, and social security number) as well as indirect identifiers (e.g. date of birth, gender, ethnicity, and location). Moreover, GDPR does not consider data as anonymized if personal data are coded and can be reversed to uncover the identity of patients. Additionally, GDPR requires that consent is obtained from patients for the specific research project for which their personal data will be used, rather than ‘broad consent’. The GDPR does recognize that it may not always be possible to fully identify the purpose for which the data will be utilized, and in these cases allows the purpose to be described at a more ‘general level’ while obtaining consent—however, it does not specify the level of generality that is acceptable.53

In the USA, HIPAA outlines a set of 18 specific identifiers that must be removed or coded in order for data to be considered de-identified.54 Moreover, the Common Rule and HIPAA do not consider coded information as ‘identifiable private information’ as long as steps are taken to prevent the investigator from obtaining means to link the code to the patient’s identity. Furthermore, the Common Rule allows for ‘broad consent’ from individuals for the storage, maintenance, and secondary research use of their identifiable private information.54 Both the GDPR and the Common Rule do not consider silence, pre-ticked boxes, and inactivity as consent. For example, the consent statement should read ‘please tick here if you consent to sharing your personal data for this study’, rather than ‘please tick here if you refuse to share your personal data for this study’.53

These differences have important implications for the conduct of embedded trials. For example, in the USA, investigators may be able to use coded patient data to pre-screen patients and invite them to participate in a trial while maintaining their anonymity. Once a patient consents to participation in a trial and processing of their personal data, the data can be uncoded to identify the participants. Pre-screening would be more difficult in the EU, where reversibly coded patient data are protected by the GDPR.

Potential solutions and progress

Sharing and utilizing patient data for research while complying with legal requirements are complex, but possible. In the EU, the Electronic Health Records for Clinical Research (EHR4CR) project exemplifies this. The EHR4CR is a sustainable research platform that links the EHR systems of 34 academic and pharmaceutical partners in Europe. The software ‘InSite’ was developed to allow researchers to query the EHRs of all hospitals in the EHR4CR system in order to estimate the number of eligible patients for a clinical trial at each site, assess the feasibility of the trial protocol, and fine-tune the protocol if necessary. The researchers query completely anonymized data, making this solution compliant with GDPR. The EHR to electronic data capture (EHR2EDC) builds on the success of EHR4CR. Once a patient consents to participation in a trial, the EHR2EDC platform exports relevant data from the patient’s EHR to the trial investigator’s system. This can minimize or eliminate the need for manual data collection—reducing costs, preventing errors, and saving time.

While the EHR4CR is a useful tool for protocol development, the UK has also established ‘Secure Data Environments’ that can allow the conduct of EHR-enabled RCTs while remaining GDPR compliant. The use of such a system was successfully employed in the High-Sensitivity Troponin in the Evaluation of Patients with Suspected Acute Coronary Syndrome (High-STEACS) trial.55 High-STEACS was a stepped-wedge cluster-randomized trial in which 10 hospitals in Scotland were randomly assigned either early or late implementation of a high-sensitivity cardiac troponin I (hs-cTnI) assay to diagnose myocardial infarction. The Community Health Index is a population healthcare register that includes all individuals living in Scotland. For this trial, each patient’s Community Health Index number was used to link multiple data sources—including EHRs, databases containing investigation results and treatments, and national registries containing hospitalization and mortality records. The linked data were held securely in the ‘NHS safe haven’ of each health system and used for screening, enrollment, and outcome adjudication of each patient. These data were then anonymized (thus making them shareable under GDPR regulations) and transferred to a national analytical platform for analysis. The High-STEACS trial showed that implementation of hs-cTnI assays did not reduce the risk of re-infarction or death at 1 year. This trial was unique in that patient written informed consent for research was not required throughout, as determined by Scotland, a Research Ethics Committee, the Public Benefit and Privacy Panel for Health and Social Care, and by each NHS Health Board.55 No informed consent for research was required for enrollment as the intervention was implemented at the hospital level, and no informed consent for research was required for data sharing as data were anonymized before it left the health system. The High-Sensitivity Cardiac Troponin on Presentation to Rule Out Myocardial Infarction (HiSTORIC) trial, which evaluated the use of hs-cTnI to rule out myocardial infarction early, was designed in a similar manner to avoid the need for patient informed consent for research.56

In the USA, the Health Information Technology for Economic and Clinical Health (HITECH) Act was passed in 2009, which led to the widespread adoption of EHRs across US healthcare systems through financial incentives and penalties for non-compliance.57 The HITECH Act also pushes for health information exchange between healthcare systems. The National Institutes of Health (NIH) Health Care Systems Research Collaboratory was founded in 2012, with the aim to provide a platform for designing and implementing EHR-embedded RCTs. In 2014, the Patient-Centered Clinical Research Network (PCORnet) was launched.58 This is a program that links the EHR systems of nine large clinical research networks. The ADAPTABLE was the first trial to employ the PCORnet tool. The trial applied computational phenotype modelling, an efficient method of converting study inclusion criteria into a query, to EHRs across multiple participating sites to identify potentially eligible participants, provide trial invitation from a central centre, negotiate informed consent virtually, and collect all trial data from EHR documentation.31

Thus, EHR4CR and PCORnet demonstrate the feasibility of developing reusable digital platforms that can optimize the conduct of trials while complying with regional laws. The ‘TriNetX’ platform applies these principles on an international level. TriNetX is a solution that allows researchers to query the EHRs of several healthcare organizations across several countries, including the USA, Japan, Singapore, Malaysia, Israel, and India. This is made possible due to its federated nature—i.e. data are retained within institutional boundaries, and researchers can access only anonymized patient data. In 2019, TriNetX acquired the InSite platform, thus expanding its network to include European healthcare organizations that are part of the EHR4CR system.

Gatekeepers and costs

Even within the same country, health systems may have different EHRs, most often with limited capacity for integrated data handling and analyses.59 In a survey in the USA, one-quarter of health information exchange leaders believed that health systems force hospitals and individual providers to adopt particular EHR systems instead of creating an interoperable interface between existing EHRs,60 motivated by financial interests and market dominance. Half of the respondents also reported EHR vendors intentionally making the interface not amenable to information exchange and charging high fees for data sharing. Thus, these EHR vendors act as ‘gatekeepers’ to data sharing.61 Moreover—although EHR-embedded trials have been touted as economically advantageous—the initial investment required for setting up appropriate EHR infrastructure with seamless interoperability between different healthcare systems to facilitate CVOTs conduct is big. Additional costs may be incurred due to the requirement of personnel with deep systems knowledge and big data analytic skills—which is crucial for the operation and analysis of EHR-embedded trials.

Potential solutions and progress

To allow the development of interoperable EHR systems, gatekeepers must be brought to the table for discussions and swayed towards the idea of integration. National level policies, along with incentives for gatekeepers, may assist in overcoming this barrier.61 In addition to the EHR4CR and PCORnet systems discussed earlier, another example of efficient data sharing is the SWEDEHEART registry, which records all patients admitted to a coronary care unit with a suspected or definite acute coronary syndrome in Sweden and Iceland.62,63 This registry was leveraged to conduct the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial, as well as the Bivalirudin versus Heparin in ST-Segment and Non–ST-Segment Elevation Myocardial Infarction in Patients on Modern Antiplatelet Therapy (VALIDATE) trial. These trials used the registry for all aspects of the trial, including screening, enrollment, capture of baseline characteristics, follow-up data, and outcome acquisition. EuroHeart, an initiative of the European Society of Cardiology (ESC), is a continuous registry being developed across ∼20 countries in Europe.64 National registry programmes will be set up in each country using a common data set of standardized variables, and the local infrastructure will belong to the respective participating country. The aim of the project is to provide infrastructure to support observational and randomized research, as well as safety surveillance of new cardiovascular drugs and devices. This project is a step up from ESC EuroObservational Research Programme (EORP), which aimed to promote and monitor the implementation of new treatments and adherence to ESC guidelines—but was criticized for its substandard data quality and coverage of common cardiovascular diseases.64

Moreover, fees to utilize services of these interoperable systems can be steep, with exact costs depending on the services rendered. For example, EHR4CR services can range from identification of potentially eligible patients to complete infrastructure support of a trial, with corresponding estimated costs ranging from 50 000 to 500 000 Euros (as of 2012).65 However, ultimately, the efficiency gains achieved by clinical trial sponsors (i.e. the person-time saved) can substantially reduce the cost of conducting clinical trials. As per their website, TriNetX (which now includes the EHR4CR network) has allowed researchers to test over 39 000 trial protocols and present over 10 000 trial opportunities to healthcare organizations.66

Varying data formats across electronic health record systems

The EHRs are built using a wide variety of data models, with the coding format varying from one EHR system to another, i.e. different EHR systems often use different codes for the same data point. Thus, most analyses focus on data from a single database, using a single analysis method customized to the underlying data model and its terminologies. Thus, even after data sharing between EHRs, the heterogeneity in coding across systems poses a major challenge for the use of the data for research purposes. However, this challenge can be overcome, as discussed below.

Potential solutions and current progress

An analysis across multiple disparate databases must either tailor the analysis to accommodate each of the underlying data models and terminologies or code the data into a common format. For example, in the USA, the PCORnet community developed the Common Data Model, which standardizes codes for data points across multiple EHR systems into a common format. This allows researchers to use a single query to obtain a particular data point from the EHRs of millions of patients. International consortia such as the Observational Medical Outcomes Partnership (OMOP) and Clinical Data Interchange Standards Consortium (CTAC) have helped make progress towards standardization of the format of routinely generated data for research and regulatory submissions.67,68 The Academic Research Consortia, which are collaborations between clinical experts, various academic research organizations, and the US FDA, aim to develop consensus definitions and standard nomenclature for use in cardiovascular clinical trials.69 Expansion of such efforts to create standardized data definitions for EHRs can play an important role in the advancement of embedded trials.

Exposure fatigue due to electronic health record–based alerts

The EHR-based alerts regarding patients potentially eligible for an RCT—although an attractive option to increase and simplify recruitment—can lead to exposure fatigue and lack of response from providers. Results from a cross-sectional study evaluating the utility of EHR alerts for potential trial eligibility demonstrated that providers became less likely to respond to alerts after several weeks of exposure to EHR-based alerts. Although the decline was statistically significant, it was slow, and response rates remained reasonably high (30%–40%) even after 36 weeks of exposure.70

Potential solutions and current progress

Exposure fatigue to EHR alerts can be reduced by increasing the specificity of alerts, wherein physicians are alerted when the likelihood of the patient’s eligibility for a trial is relatively high. A systematic approach to assessment and reduction of alert burden has been proposed—and has been successfully carried out by large healthcare systems such as Geisinger Health and Penn Medicine.71 Moreover, human factor principles can be applied when designing alerts, with care given to the format, content, legibility, and the colour of alerts. Additionally, alerts should be designed to be customizable by the physician, giving them the flexibility of turning off alerts for some or all clinical trials or reducing the intensity of alerts. Indeed, it has been shown that user-configurable alerts have greater acceptance among users.72

Absence of health-related quality of life data in routinely collected data

Assessment of health-related quality of life (HRQoL) data, a patient-relevant outcome, is a crucial component of pragmatism in RCTs. Over the past three decades, the science of validly collecting patients’ experiences of their disease has evolved, and the FDA has even qualified some questionnaires as clinical outcomes assessments, meaning that drugs and devices can be approved if significant improvements in patients’ health status can be demonstrated, even if there is no difference in clinical events. However, HRQoL is typically not assessed in a clinical encounter and therefore is not accurately represented in EHRs, registries, and administrative datasets.

Potential solutions and current progress

Growing internet connectivity, combined with widespread smartphone use, can allow collection of patient-reported outcomes through ‘apps’—a rich, important, and low-cost option for outcomes collection. The recently completed Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in HF (CHIEF-HF) trial used the Kansas City Cardiomyopathy Questionnaire as the primary outcome, and the entire trial was conducted completely during the Covid-19 pandemic without a single face-to-face visit.73

Challenges with outcome adjudication

Traditionally, central committees have been established to adjudicate outcome events using clinical annotations and trial-specific definitions pre-specified by charter for each trial. While event adjudication is widely considered to be the best practice, there is conflicting evidence on whether adjudication reduces misclassification of events.74,75 The use of routinely collected data to ascertain outcomes has the advantage of being automated, independent, and highly cost effective. However, the accuracy of automated outcome adjudication using the non-standardized free text in EHR and claims data has been called into question.

Potential solutions and current progress

The most appropriate method to overcome the challenges with automated outcome adjudication would be to make EHR data available in a standardized format. Countries in the EU have started to unify EHRs and employ standardized diagnostic coding, and accuracy of standardized coding is improving as these data are increasingly used for healthcare quality improvement and reimbursement. Furthermore, guidelines have helped to standardize definitions and the use of diagnostic coding in the delivery of RCTs is gaining popularity. Results from systematic reviews suggest that effect size estimates are unaltered when analysing site investigator–reported events or routinely collected data.74 Studies comparing results ascertained using events identified by claims data vs. those confirmed by trial event adjudication committees show that both methods yield similar findings of all-cause death. However, findings from claims data often differ from adjudicated data for non-fatal and non-procedural cardiovascular outcomes—likely due to the absence of these data in structured format or due to non-standardized definitions for these outcomes.76,77 In the absence of structured data, nuanced trial outcomes may be determined using machine learning algorithms which analyse free text in EHRs. Results from few recent studies have shown that natural language processing is able to ascertain clinical outcomes with ∼90% accuracy.78–80 Once further refined, tested, and adopted, techniques such as this can potentially eliminate the need for adjudication of outcomes by central clinical trial committees.

Contemporary guidelines and regulations

In recent years, both international and national guidelines and regulatory bodies have become more amenable to the idea of RCTs layered onto routine clinical practice. In March 2015, international reporting guidelines for studies conducted using routinely collected health data were published that encourage use of the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) checklist.81 In April 2021, the International Council of Harmonization released an early draft of the updated version of the 1996 Good Clinical Practice guidelines, which advocates for use of electronic consent and calls for incorporation of technology in existing healthcare infrastructure to enable use of a greater variety of data sources.82,83 In May 2021, an extension of the Consolidated Standards of Reporting Trials (CONSORT) guidelines was published in order to standardize the elements that should be recorded and reported in RCTs conducted using routinely collected data.84

In Europe, in June 2021, the European Medicines Agency (EMA) released a draft version of guidelines for the use of computerized systems and electronic data in clinical trials.85 This draft is currently undergoing revisions based on public consultation, and an implementation date will soon be announced. In December 2021, the Medicines & Healthcare products Regulatory Agency (MHRA) published a guidance on the use of real-world data in clinical studies to support regulatory decisions.86 Moreover, in August 2022, the ESC and the BigData@Heart consortium together published the CODE-EHR best practice framework for the use of structured EHRs in clinical research.50

In the USA, in September 2021, the FDA issued requisites that trialists must adhere to when utilizing EHR and claims data for generation of evidence for new medical therapies, in order to support regulatory decision-making.87 Moreover, in December 2021, the FDA issued a draft guidance for the use of digital technologies (such as wearable sensors and smartphones) for remote data acquisition in clinical investigations.88

Examples of ongoing and completed trials that leverage routinely generated data and technology

Examples of ongoing and completed trials that leverage routinely generated data and technology are discussed in Supplementary material online and summarized in Tables 1 and 2.

Table 1

Examples of completed cardiovascular RCTs that leveraged routinely generated data and technology

TrialData source(s)Location(s)Total N (enrolled)InterventionControlPrimary outcomeTechnology-based interventions
The Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term (ADAPTABLE)32PCORnet—in which 40 centres and one health plan were participatingUSA15 076Aspirin 325 mgAspirin 81 mgPrimary effectiveness outcome: all-cause death, MI, and stroke at 12 months
Primary safety outcome: major bleeding requiring blood transfusion at 12 months
• Identification of patients with established ASCVD through EHR across all sites
• Electronic informed consent
• Baseline clinical characteristics queried through Patient-Centered Outcomes Research PCORnet tool
• Video/telephone-based follow-up encounters
• Patient-reported outcome measures collected via electronic patient portal
Thrombus Aspiration during ST-Segment Elevation Myocardial Infarction (TASTE)63Swedish Coronary Angiography and Angioplasty Registry (SCAAR)Sweden and Iceland7244Thrombus aspiration followed by routine balloon angioplastyBalloon angioplasty only30-day all-cause mortality• Identification of patients with STEMI through EHR-based registry (SCAAR)
• Online randomization model within SCAAR database
• Trial endpoint data directly obtained analysed from database
Patient-Centered Care Transitions in HF—A Pragmatic Cluster Randomized Trial (PACT-HF)89Canadian Institute for Health Information DatabaseCanada2494Discharge planning based on PACT-HF modelStandard discharge planningTime to composite all-cause re-admissions/emergency visits/deaths at 30 days and 3 months• Identification of patients through Canadian Institute for Health Information Database
• Clinical data and patient-reported outcomes following discharge collected through telecommunications and administrative database linkage
A Randomized Clinical Trial of an Automated mHealth Intervention for Physical Activity Promotion90Ambulatory outpatient clinic in Baltimore, MarylandUSA200Blinded digital activity tracker with smartphone messagingUnblinded digital activity tracker/no smartphone messagingMean daily step count• Physical activity tracking performed using a wearable, display-free, triaxial accelerometer that pairs with compatible smartphones
• Smartphone application (Fitbug) allowed participants to view physical activity metrics
• Fitbug linked with smart texting system with content generated by providers for positive reinforcement of patient’s behaviours
Study of Access Site for Enhancement of PCI for Women (SAFE-PCI)91CathPCI RegistryUSA1787Trans-radial approach to PCITrans-femoral approach to PCIBleeding and vascular complications, procedural failure• National Cardiovascular Research Infrastructure (NCRI)-based data collection—data stream of PCI registry-participating sites accessed electronically to auto-populate clinical trial database for consented participants• Randomization performed via online module incorporated into NCRI database
Coffee and Real-time Atrial and Ventricular Ectopy (CRAVE)92Continuously recording heart monitor, glucose monitor, and Fitbit for each patientUSA108Coffee consumption for 14 daysNo coffee consumptionChange in cardiac ectopy burden• Patients equipped with devices for EKG recording, continuous glucose monitoring, and physical activity tracking (Fitbit)
• Devices integrated with smartphone application to record ectopic rhythm, physical activity, and sleep time
• Daily reminders provided through smartphone application for protocol adherence
Individualized Studies of Triggers of Paroxysmal Atrial Fibrillation (I-STOP AF)93Smartphone application—used by patients to report health-related quality of lifeUSA446Exposure to known AF triggersNo exposure to AF triggersRemote health-related quality of life assessed using the Atrial Fibrillation Effect on Quality of Life (AFEQT) survey• Participants recruited through email invitation, social media, word of mouth, and healthcare providers
• Participants equipped with wearable EKG recording device linked to smartphone application
• Participants filled AFEQT survey regularly through smartphone application
Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in HF (CHIEF-HF)73Smartphone application—used by patients to report health-related quality of lifeUSA476CanagliflozinPlaceboChange from baseline in Kansas City Cardiomyopathy Questionnaire-Total Symptom Score (KCCQ-TSS)• Smartphone application designed to ensure completion of clinical assessment tools and medication compliance
• Patients provided smartwatch to record physical activity and cardiovascular parameters for secondary endpoint ascertainment
The Investigation of Palpitations in the ED Study (IPED)94Intervention: smartphone-based event recorder (AliveCor)
Control: standard care
UK243Smartphone-based EKG recorder providedNo interventionNumber of participants with symptomatic rhythm detection up to 90 days• Smartphone-based EKG recorder provided to intervention arm participants for 90-day symptomatic rhythm detection following initial encounter for pre-syncope
mHealth Screening to Prevent Strokes (mSToPS)95Aetna claims dataUSA2659Wearable EKG monitoring device and wristband for pulse monitoringUsual care for 4 months, followed by crossover to intervention group for 4 monthsIncidence of newly diagnosed AF• Email-based invitation to eligible patients with link provided to informational website
• Electronic informed consent
• Continuous EKG monitoring via continuous ambulatory skin adhesive patch
Telemedical Interventional Management in HF II (TIM-HF2)96Data collected by investigatorsGermany1571Remote patient management including remote blood pressure, EKG and weight monitoring integrated with patient-communication platformGuideline based care for HFPercentage of days lost due to unplanned cardiovascular hospitalization or all-cause mortality• Telemonitoring system installed in patient’s home consisting of digital table that captures data from an EKG device, blood pressure measuring device, and weighing scale
• HF and telemonitoring system education provided through telephone
Telemonitoring in the Management of HF (TEMA-HF1)97Data collected by investigatorsBelgium160Telemonitoring with use of wearable blood pressure device and weighing scale integrated with cell phonesUsual careAll-cause mortality• Weighing scale and blood pressure monitoring device connected to cell phone via Bluetooth
• Pre-specified alert limits identified; provider and HF alerted if weight, heart rate, or blood pressure parameters fell out of range through automatically generated email
• Online HF database created to allow general practitioner and HF specialist to record changes in patient’s regimen and for interprovider communication
• Central computer alerted automatically if no readings detected from patients for 2 or more consecutive days
Bivalirudin Versus Heparin Monotherapy in Myocardial Infarction (VALIDATE)62SCAAR registrySweden6006BivalirudinHeparin6-month all-cause mortality, myocardial infarction, and major bleeding• SCAAR registry-based recruitment, randomization, and clinical data collection
• Telecommunication-based follow-up and endpoint ascertainment
The Determination of the Role of Oxygen in Suspected Acute Myocardial Infarction (DETO2X-AMI)98SWEDEHEART registrySweden6629Administration of 6 L/min oxygenUsual care (no oxygen)1-year all-cause mortality• Trial embedded in SWEDEHEART registry
• Registry-based patient recruitment, randomization, data collection, and endpoint ascertainment
The E-Coach technology-assisted care transition system: a pragmatic randomized trial99Intervention: interactive voice response system which called patients after discharge and recorded data reported by patients and presented it to nurses on a computer system
Control: data collected by investigators
USA478E-coach intervention following discharge among HF and chronic obstructive pulmonary disease (COPD patients) including interactive voice response (IVR) systemUsual discharge care30-day re-hospitalization• Intervention based on integration of proactive IVR system following discharge of HF and COPD patients for daily assessment of participant’s health status
• Web-based dashboard developed for care transition nurses to review patient data recorded in computer system
High-Sensitivity Troponin in the Evaluation of Patients with Acute Coronary Syndrome (HIGH-STEACS)55Data collected by investigatorsScotland48 282High-sensitivity cardiac troponinCardiac troponin1-year myocardial infarction or death• Patient identification through EHR and national registries
• Follow-up data capture through EHR and national registries
• Policy randomization at the level of sites; thus, consent from participants was not required
Torsemide Comparison with Furosemide for Management of HF (TRANSFORM-HF)100EHRUSA2859TorsemideFurosemideAll-cause mortality, as measured by national death index and centralized call center• Traditional in-person study-specific follow-up visits replaced with phone interviews
• No lab testing/procedures beyond usual care
• Randomization at any point during hospitalization
• No in-person follow-up study visits, data entry, or event reporting
TrialData source(s)Location(s)Total N (enrolled)InterventionControlPrimary outcomeTechnology-based interventions
The Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term (ADAPTABLE)32PCORnet—in which 40 centres and one health plan were participatingUSA15 076Aspirin 325 mgAspirin 81 mgPrimary effectiveness outcome: all-cause death, MI, and stroke at 12 months
Primary safety outcome: major bleeding requiring blood transfusion at 12 months
• Identification of patients with established ASCVD through EHR across all sites
• Electronic informed consent
• Baseline clinical characteristics queried through Patient-Centered Outcomes Research PCORnet tool
• Video/telephone-based follow-up encounters
• Patient-reported outcome measures collected via electronic patient portal
Thrombus Aspiration during ST-Segment Elevation Myocardial Infarction (TASTE)63Swedish Coronary Angiography and Angioplasty Registry (SCAAR)Sweden and Iceland7244Thrombus aspiration followed by routine balloon angioplastyBalloon angioplasty only30-day all-cause mortality• Identification of patients with STEMI through EHR-based registry (SCAAR)
• Online randomization model within SCAAR database
• Trial endpoint data directly obtained analysed from database
Patient-Centered Care Transitions in HF—A Pragmatic Cluster Randomized Trial (PACT-HF)89Canadian Institute for Health Information DatabaseCanada2494Discharge planning based on PACT-HF modelStandard discharge planningTime to composite all-cause re-admissions/emergency visits/deaths at 30 days and 3 months• Identification of patients through Canadian Institute for Health Information Database
• Clinical data and patient-reported outcomes following discharge collected through telecommunications and administrative database linkage
A Randomized Clinical Trial of an Automated mHealth Intervention for Physical Activity Promotion90Ambulatory outpatient clinic in Baltimore, MarylandUSA200Blinded digital activity tracker with smartphone messagingUnblinded digital activity tracker/no smartphone messagingMean daily step count• Physical activity tracking performed using a wearable, display-free, triaxial accelerometer that pairs with compatible smartphones
• Smartphone application (Fitbug) allowed participants to view physical activity metrics
• Fitbug linked with smart texting system with content generated by providers for positive reinforcement of patient’s behaviours
Study of Access Site for Enhancement of PCI for Women (SAFE-PCI)91CathPCI RegistryUSA1787Trans-radial approach to PCITrans-femoral approach to PCIBleeding and vascular complications, procedural failure• National Cardiovascular Research Infrastructure (NCRI)-based data collection—data stream of PCI registry-participating sites accessed electronically to auto-populate clinical trial database for consented participants• Randomization performed via online module incorporated into NCRI database
Coffee and Real-time Atrial and Ventricular Ectopy (CRAVE)92Continuously recording heart monitor, glucose monitor, and Fitbit for each patientUSA108Coffee consumption for 14 daysNo coffee consumptionChange in cardiac ectopy burden• Patients equipped with devices for EKG recording, continuous glucose monitoring, and physical activity tracking (Fitbit)
• Devices integrated with smartphone application to record ectopic rhythm, physical activity, and sleep time
• Daily reminders provided through smartphone application for protocol adherence
Individualized Studies of Triggers of Paroxysmal Atrial Fibrillation (I-STOP AF)93Smartphone application—used by patients to report health-related quality of lifeUSA446Exposure to known AF triggersNo exposure to AF triggersRemote health-related quality of life assessed using the Atrial Fibrillation Effect on Quality of Life (AFEQT) survey• Participants recruited through email invitation, social media, word of mouth, and healthcare providers
• Participants equipped with wearable EKG recording device linked to smartphone application
• Participants filled AFEQT survey regularly through smartphone application
Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in HF (CHIEF-HF)73Smartphone application—used by patients to report health-related quality of lifeUSA476CanagliflozinPlaceboChange from baseline in Kansas City Cardiomyopathy Questionnaire-Total Symptom Score (KCCQ-TSS)• Smartphone application designed to ensure completion of clinical assessment tools and medication compliance
• Patients provided smartwatch to record physical activity and cardiovascular parameters for secondary endpoint ascertainment
The Investigation of Palpitations in the ED Study (IPED)94Intervention: smartphone-based event recorder (AliveCor)
Control: standard care
UK243Smartphone-based EKG recorder providedNo interventionNumber of participants with symptomatic rhythm detection up to 90 days• Smartphone-based EKG recorder provided to intervention arm participants for 90-day symptomatic rhythm detection following initial encounter for pre-syncope
mHealth Screening to Prevent Strokes (mSToPS)95Aetna claims dataUSA2659Wearable EKG monitoring device and wristband for pulse monitoringUsual care for 4 months, followed by crossover to intervention group for 4 monthsIncidence of newly diagnosed AF• Email-based invitation to eligible patients with link provided to informational website
• Electronic informed consent
• Continuous EKG monitoring via continuous ambulatory skin adhesive patch
Telemedical Interventional Management in HF II (TIM-HF2)96Data collected by investigatorsGermany1571Remote patient management including remote blood pressure, EKG and weight monitoring integrated with patient-communication platformGuideline based care for HFPercentage of days lost due to unplanned cardiovascular hospitalization or all-cause mortality• Telemonitoring system installed in patient’s home consisting of digital table that captures data from an EKG device, blood pressure measuring device, and weighing scale
• HF and telemonitoring system education provided through telephone
Telemonitoring in the Management of HF (TEMA-HF1)97Data collected by investigatorsBelgium160Telemonitoring with use of wearable blood pressure device and weighing scale integrated with cell phonesUsual careAll-cause mortality• Weighing scale and blood pressure monitoring device connected to cell phone via Bluetooth
• Pre-specified alert limits identified; provider and HF alerted if weight, heart rate, or blood pressure parameters fell out of range through automatically generated email
• Online HF database created to allow general practitioner and HF specialist to record changes in patient’s regimen and for interprovider communication
• Central computer alerted automatically if no readings detected from patients for 2 or more consecutive days
Bivalirudin Versus Heparin Monotherapy in Myocardial Infarction (VALIDATE)62SCAAR registrySweden6006BivalirudinHeparin6-month all-cause mortality, myocardial infarction, and major bleeding• SCAAR registry-based recruitment, randomization, and clinical data collection
• Telecommunication-based follow-up and endpoint ascertainment
The Determination of the Role of Oxygen in Suspected Acute Myocardial Infarction (DETO2X-AMI)98SWEDEHEART registrySweden6629Administration of 6 L/min oxygenUsual care (no oxygen)1-year all-cause mortality• Trial embedded in SWEDEHEART registry
• Registry-based patient recruitment, randomization, data collection, and endpoint ascertainment
The E-Coach technology-assisted care transition system: a pragmatic randomized trial99Intervention: interactive voice response system which called patients after discharge and recorded data reported by patients and presented it to nurses on a computer system
Control: data collected by investigators
USA478E-coach intervention following discharge among HF and chronic obstructive pulmonary disease (COPD patients) including interactive voice response (IVR) systemUsual discharge care30-day re-hospitalization• Intervention based on integration of proactive IVR system following discharge of HF and COPD patients for daily assessment of participant’s health status
• Web-based dashboard developed for care transition nurses to review patient data recorded in computer system
High-Sensitivity Troponin in the Evaluation of Patients with Acute Coronary Syndrome (HIGH-STEACS)55Data collected by investigatorsScotland48 282High-sensitivity cardiac troponinCardiac troponin1-year myocardial infarction or death• Patient identification through EHR and national registries
• Follow-up data capture through EHR and national registries
• Policy randomization at the level of sites; thus, consent from participants was not required
Torsemide Comparison with Furosemide for Management of HF (TRANSFORM-HF)100EHRUSA2859TorsemideFurosemideAll-cause mortality, as measured by national death index and centralized call center• Traditional in-person study-specific follow-up visits replaced with phone interviews
• No lab testing/procedures beyond usual care
• Randomization at any point during hospitalization
• No in-person follow-up study visits, data entry, or event reporting

ASCVD, atherosclerotic cardiovascular disease; STEMI, ST-elevation myocardial infarction; SWEDEHEART, The Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies.

Table 1

Examples of completed cardiovascular RCTs that leveraged routinely generated data and technology

TrialData source(s)Location(s)Total N (enrolled)InterventionControlPrimary outcomeTechnology-based interventions
The Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term (ADAPTABLE)32PCORnet—in which 40 centres and one health plan were participatingUSA15 076Aspirin 325 mgAspirin 81 mgPrimary effectiveness outcome: all-cause death, MI, and stroke at 12 months
Primary safety outcome: major bleeding requiring blood transfusion at 12 months
• Identification of patients with established ASCVD through EHR across all sites
• Electronic informed consent
• Baseline clinical characteristics queried through Patient-Centered Outcomes Research PCORnet tool
• Video/telephone-based follow-up encounters
• Patient-reported outcome measures collected via electronic patient portal
Thrombus Aspiration during ST-Segment Elevation Myocardial Infarction (TASTE)63Swedish Coronary Angiography and Angioplasty Registry (SCAAR)Sweden and Iceland7244Thrombus aspiration followed by routine balloon angioplastyBalloon angioplasty only30-day all-cause mortality• Identification of patients with STEMI through EHR-based registry (SCAAR)
• Online randomization model within SCAAR database
• Trial endpoint data directly obtained analysed from database
Patient-Centered Care Transitions in HF—A Pragmatic Cluster Randomized Trial (PACT-HF)89Canadian Institute for Health Information DatabaseCanada2494Discharge planning based on PACT-HF modelStandard discharge planningTime to composite all-cause re-admissions/emergency visits/deaths at 30 days and 3 months• Identification of patients through Canadian Institute for Health Information Database
• Clinical data and patient-reported outcomes following discharge collected through telecommunications and administrative database linkage
A Randomized Clinical Trial of an Automated mHealth Intervention for Physical Activity Promotion90Ambulatory outpatient clinic in Baltimore, MarylandUSA200Blinded digital activity tracker with smartphone messagingUnblinded digital activity tracker/no smartphone messagingMean daily step count• Physical activity tracking performed using a wearable, display-free, triaxial accelerometer that pairs with compatible smartphones
• Smartphone application (Fitbug) allowed participants to view physical activity metrics
• Fitbug linked with smart texting system with content generated by providers for positive reinforcement of patient’s behaviours
Study of Access Site for Enhancement of PCI for Women (SAFE-PCI)91CathPCI RegistryUSA1787Trans-radial approach to PCITrans-femoral approach to PCIBleeding and vascular complications, procedural failure• National Cardiovascular Research Infrastructure (NCRI)-based data collection—data stream of PCI registry-participating sites accessed electronically to auto-populate clinical trial database for consented participants• Randomization performed via online module incorporated into NCRI database
Coffee and Real-time Atrial and Ventricular Ectopy (CRAVE)92Continuously recording heart monitor, glucose monitor, and Fitbit for each patientUSA108Coffee consumption for 14 daysNo coffee consumptionChange in cardiac ectopy burden• Patients equipped with devices for EKG recording, continuous glucose monitoring, and physical activity tracking (Fitbit)
• Devices integrated with smartphone application to record ectopic rhythm, physical activity, and sleep time
• Daily reminders provided through smartphone application for protocol adherence
Individualized Studies of Triggers of Paroxysmal Atrial Fibrillation (I-STOP AF)93Smartphone application—used by patients to report health-related quality of lifeUSA446Exposure to known AF triggersNo exposure to AF triggersRemote health-related quality of life assessed using the Atrial Fibrillation Effect on Quality of Life (AFEQT) survey• Participants recruited through email invitation, social media, word of mouth, and healthcare providers
• Participants equipped with wearable EKG recording device linked to smartphone application
• Participants filled AFEQT survey regularly through smartphone application
Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in HF (CHIEF-HF)73Smartphone application—used by patients to report health-related quality of lifeUSA476CanagliflozinPlaceboChange from baseline in Kansas City Cardiomyopathy Questionnaire-Total Symptom Score (KCCQ-TSS)• Smartphone application designed to ensure completion of clinical assessment tools and medication compliance
• Patients provided smartwatch to record physical activity and cardiovascular parameters for secondary endpoint ascertainment
The Investigation of Palpitations in the ED Study (IPED)94Intervention: smartphone-based event recorder (AliveCor)
Control: standard care
UK243Smartphone-based EKG recorder providedNo interventionNumber of participants with symptomatic rhythm detection up to 90 days• Smartphone-based EKG recorder provided to intervention arm participants for 90-day symptomatic rhythm detection following initial encounter for pre-syncope
mHealth Screening to Prevent Strokes (mSToPS)95Aetna claims dataUSA2659Wearable EKG monitoring device and wristband for pulse monitoringUsual care for 4 months, followed by crossover to intervention group for 4 monthsIncidence of newly diagnosed AF• Email-based invitation to eligible patients with link provided to informational website
• Electronic informed consent
• Continuous EKG monitoring via continuous ambulatory skin adhesive patch
Telemedical Interventional Management in HF II (TIM-HF2)96Data collected by investigatorsGermany1571Remote patient management including remote blood pressure, EKG and weight monitoring integrated with patient-communication platformGuideline based care for HFPercentage of days lost due to unplanned cardiovascular hospitalization or all-cause mortality• Telemonitoring system installed in patient’s home consisting of digital table that captures data from an EKG device, blood pressure measuring device, and weighing scale
• HF and telemonitoring system education provided through telephone
Telemonitoring in the Management of HF (TEMA-HF1)97Data collected by investigatorsBelgium160Telemonitoring with use of wearable blood pressure device and weighing scale integrated with cell phonesUsual careAll-cause mortality• Weighing scale and blood pressure monitoring device connected to cell phone via Bluetooth
• Pre-specified alert limits identified; provider and HF alerted if weight, heart rate, or blood pressure parameters fell out of range through automatically generated email
• Online HF database created to allow general practitioner and HF specialist to record changes in patient’s regimen and for interprovider communication
• Central computer alerted automatically if no readings detected from patients for 2 or more consecutive days
Bivalirudin Versus Heparin Monotherapy in Myocardial Infarction (VALIDATE)62SCAAR registrySweden6006BivalirudinHeparin6-month all-cause mortality, myocardial infarction, and major bleeding• SCAAR registry-based recruitment, randomization, and clinical data collection
• Telecommunication-based follow-up and endpoint ascertainment
The Determination of the Role of Oxygen in Suspected Acute Myocardial Infarction (DETO2X-AMI)98SWEDEHEART registrySweden6629Administration of 6 L/min oxygenUsual care (no oxygen)1-year all-cause mortality• Trial embedded in SWEDEHEART registry
• Registry-based patient recruitment, randomization, data collection, and endpoint ascertainment
The E-Coach technology-assisted care transition system: a pragmatic randomized trial99Intervention: interactive voice response system which called patients after discharge and recorded data reported by patients and presented it to nurses on a computer system
Control: data collected by investigators
USA478E-coach intervention following discharge among HF and chronic obstructive pulmonary disease (COPD patients) including interactive voice response (IVR) systemUsual discharge care30-day re-hospitalization• Intervention based on integration of proactive IVR system following discharge of HF and COPD patients for daily assessment of participant’s health status
• Web-based dashboard developed for care transition nurses to review patient data recorded in computer system
High-Sensitivity Troponin in the Evaluation of Patients with Acute Coronary Syndrome (HIGH-STEACS)55Data collected by investigatorsScotland48 282High-sensitivity cardiac troponinCardiac troponin1-year myocardial infarction or death• Patient identification through EHR and national registries
• Follow-up data capture through EHR and national registries
• Policy randomization at the level of sites; thus, consent from participants was not required
Torsemide Comparison with Furosemide for Management of HF (TRANSFORM-HF)100EHRUSA2859TorsemideFurosemideAll-cause mortality, as measured by national death index and centralized call center• Traditional in-person study-specific follow-up visits replaced with phone interviews
• No lab testing/procedures beyond usual care
• Randomization at any point during hospitalization
• No in-person follow-up study visits, data entry, or event reporting
TrialData source(s)Location(s)Total N (enrolled)InterventionControlPrimary outcomeTechnology-based interventions
The Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term (ADAPTABLE)32PCORnet—in which 40 centres and one health plan were participatingUSA15 076Aspirin 325 mgAspirin 81 mgPrimary effectiveness outcome: all-cause death, MI, and stroke at 12 months
Primary safety outcome: major bleeding requiring blood transfusion at 12 months
• Identification of patients with established ASCVD through EHR across all sites
• Electronic informed consent
• Baseline clinical characteristics queried through Patient-Centered Outcomes Research PCORnet tool
• Video/telephone-based follow-up encounters
• Patient-reported outcome measures collected via electronic patient portal
Thrombus Aspiration during ST-Segment Elevation Myocardial Infarction (TASTE)63Swedish Coronary Angiography and Angioplasty Registry (SCAAR)Sweden and Iceland7244Thrombus aspiration followed by routine balloon angioplastyBalloon angioplasty only30-day all-cause mortality• Identification of patients with STEMI through EHR-based registry (SCAAR)
• Online randomization model within SCAAR database
• Trial endpoint data directly obtained analysed from database
Patient-Centered Care Transitions in HF—A Pragmatic Cluster Randomized Trial (PACT-HF)89Canadian Institute for Health Information DatabaseCanada2494Discharge planning based on PACT-HF modelStandard discharge planningTime to composite all-cause re-admissions/emergency visits/deaths at 30 days and 3 months• Identification of patients through Canadian Institute for Health Information Database
• Clinical data and patient-reported outcomes following discharge collected through telecommunications and administrative database linkage
A Randomized Clinical Trial of an Automated mHealth Intervention for Physical Activity Promotion90Ambulatory outpatient clinic in Baltimore, MarylandUSA200Blinded digital activity tracker with smartphone messagingUnblinded digital activity tracker/no smartphone messagingMean daily step count• Physical activity tracking performed using a wearable, display-free, triaxial accelerometer that pairs with compatible smartphones
• Smartphone application (Fitbug) allowed participants to view physical activity metrics
• Fitbug linked with smart texting system with content generated by providers for positive reinforcement of patient’s behaviours
Study of Access Site for Enhancement of PCI for Women (SAFE-PCI)91CathPCI RegistryUSA1787Trans-radial approach to PCITrans-femoral approach to PCIBleeding and vascular complications, procedural failure• National Cardiovascular Research Infrastructure (NCRI)-based data collection—data stream of PCI registry-participating sites accessed electronically to auto-populate clinical trial database for consented participants• Randomization performed via online module incorporated into NCRI database
Coffee and Real-time Atrial and Ventricular Ectopy (CRAVE)92Continuously recording heart monitor, glucose monitor, and Fitbit for each patientUSA108Coffee consumption for 14 daysNo coffee consumptionChange in cardiac ectopy burden• Patients equipped with devices for EKG recording, continuous glucose monitoring, and physical activity tracking (Fitbit)
• Devices integrated with smartphone application to record ectopic rhythm, physical activity, and sleep time
• Daily reminders provided through smartphone application for protocol adherence
Individualized Studies of Triggers of Paroxysmal Atrial Fibrillation (I-STOP AF)93Smartphone application—used by patients to report health-related quality of lifeUSA446Exposure to known AF triggersNo exposure to AF triggersRemote health-related quality of life assessed using the Atrial Fibrillation Effect on Quality of Life (AFEQT) survey• Participants recruited through email invitation, social media, word of mouth, and healthcare providers
• Participants equipped with wearable EKG recording device linked to smartphone application
• Participants filled AFEQT survey regularly through smartphone application
Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in HF (CHIEF-HF)73Smartphone application—used by patients to report health-related quality of lifeUSA476CanagliflozinPlaceboChange from baseline in Kansas City Cardiomyopathy Questionnaire-Total Symptom Score (KCCQ-TSS)• Smartphone application designed to ensure completion of clinical assessment tools and medication compliance
• Patients provided smartwatch to record physical activity and cardiovascular parameters for secondary endpoint ascertainment
The Investigation of Palpitations in the ED Study (IPED)94Intervention: smartphone-based event recorder (AliveCor)
Control: standard care
UK243Smartphone-based EKG recorder providedNo interventionNumber of participants with symptomatic rhythm detection up to 90 days• Smartphone-based EKG recorder provided to intervention arm participants for 90-day symptomatic rhythm detection following initial encounter for pre-syncope
mHealth Screening to Prevent Strokes (mSToPS)95Aetna claims dataUSA2659Wearable EKG monitoring device and wristband for pulse monitoringUsual care for 4 months, followed by crossover to intervention group for 4 monthsIncidence of newly diagnosed AF• Email-based invitation to eligible patients with link provided to informational website
• Electronic informed consent
• Continuous EKG monitoring via continuous ambulatory skin adhesive patch
Telemedical Interventional Management in HF II (TIM-HF2)96Data collected by investigatorsGermany1571Remote patient management including remote blood pressure, EKG and weight monitoring integrated with patient-communication platformGuideline based care for HFPercentage of days lost due to unplanned cardiovascular hospitalization or all-cause mortality• Telemonitoring system installed in patient’s home consisting of digital table that captures data from an EKG device, blood pressure measuring device, and weighing scale
• HF and telemonitoring system education provided through telephone
Telemonitoring in the Management of HF (TEMA-HF1)97Data collected by investigatorsBelgium160Telemonitoring with use of wearable blood pressure device and weighing scale integrated with cell phonesUsual careAll-cause mortality• Weighing scale and blood pressure monitoring device connected to cell phone via Bluetooth
• Pre-specified alert limits identified; provider and HF alerted if weight, heart rate, or blood pressure parameters fell out of range through automatically generated email
• Online HF database created to allow general practitioner and HF specialist to record changes in patient’s regimen and for interprovider communication
• Central computer alerted automatically if no readings detected from patients for 2 or more consecutive days
Bivalirudin Versus Heparin Monotherapy in Myocardial Infarction (VALIDATE)62SCAAR registrySweden6006BivalirudinHeparin6-month all-cause mortality, myocardial infarction, and major bleeding• SCAAR registry-based recruitment, randomization, and clinical data collection
• Telecommunication-based follow-up and endpoint ascertainment
The Determination of the Role of Oxygen in Suspected Acute Myocardial Infarction (DETO2X-AMI)98SWEDEHEART registrySweden6629Administration of 6 L/min oxygenUsual care (no oxygen)1-year all-cause mortality• Trial embedded in SWEDEHEART registry
• Registry-based patient recruitment, randomization, data collection, and endpoint ascertainment
The E-Coach technology-assisted care transition system: a pragmatic randomized trial99Intervention: interactive voice response system which called patients after discharge and recorded data reported by patients and presented it to nurses on a computer system
Control: data collected by investigators
USA478E-coach intervention following discharge among HF and chronic obstructive pulmonary disease (COPD patients) including interactive voice response (IVR) systemUsual discharge care30-day re-hospitalization• Intervention based on integration of proactive IVR system following discharge of HF and COPD patients for daily assessment of participant’s health status
• Web-based dashboard developed for care transition nurses to review patient data recorded in computer system
High-Sensitivity Troponin in the Evaluation of Patients with Acute Coronary Syndrome (HIGH-STEACS)55Data collected by investigatorsScotland48 282High-sensitivity cardiac troponinCardiac troponin1-year myocardial infarction or death• Patient identification through EHR and national registries
• Follow-up data capture through EHR and national registries
• Policy randomization at the level of sites; thus, consent from participants was not required
Torsemide Comparison with Furosemide for Management of HF (TRANSFORM-HF)100EHRUSA2859TorsemideFurosemideAll-cause mortality, as measured by national death index and centralized call center• Traditional in-person study-specific follow-up visits replaced with phone interviews
• No lab testing/procedures beyond usual care
• Randomization at any point during hospitalization
• No in-person follow-up study visits, data entry, or event reporting

ASCVD, atherosclerotic cardiovascular disease; STEMI, ST-elevation myocardial infarction; SWEDEHEART, The Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies.

Table 2

Summary of ongoing cardiovascular RCTs leveraging routinely generated data and technology

TrialData source(s)Location(s)Total N (planned)InterventionControl/other interventionPrimary outcomeTechnology-based interventions
Pragmatic Evaluation of Events and Benefits of Lipid-lowering in Older Adults (PREVENTABLE)
NCT04262206
EHR, Medicare, and National Death IndexUSA20 000AtorvastatinPlaceboDiagnosis of dementia and persistent disability• Identification of patients across 100 US sites through integrated EHR
• Cardiovascular event ascertainment through data curation from EHR, Medicare, and National Death Index
Pragmatic Randomized Trial of Icosapent Ethyl for High Cardiovascular Risk Adults (MITIGATE)
NCT04505098
Virtual follow-ups and EHRUSA16 500Pre-treatment with icosapent ethyl (IPE)Usual carePercentage of patients with viral upper respiratory tract infection (URI); worsening of clinical status of patients with confirmed viral URI• Trial embedded within KPNC’s learning healthcare delivery system; EHR-based recruitment from 21 hospitals, ∼255 outpatient clinics
• Pre-randomization of study subjects (prior to obtaining consent)
• Electronic informed consent
• No in-person visits; only EHR-based follow-up with monthly telephone reminders to ensure medication compliance
• Endpoint assessment via EHR-based validated approaches using ICD, CPT codes, and structured data elements
Pragmatic Trial of Messaging to Providers About Treatment of Hyperlipidemia (PROMPT-LIPID)
NCT04394715
EHRUSA100Electronic alert indicating patient’s ASCVD* riskUsual care with no alertsProportion of patients with intensification of lipid-lowering therapy• In-built electronic alerts in Epic to alert providers of patient’s ASCVD risk to assess appropriate intensification of lipid-lowering therapy
Early Dronedarone Versus Usual Care to Improve Outcomes in Persons with Newly Diagnosed Atrial Fibrillation (CHANGE-AFIB)
NCT05130268
GWTG atrial fibrillation registryUSA3000Dronedarone along with usual careUsual care (no anti-arrhythmic)Cardiovascular hospitalization or death• Trial embedded in GWTG registry
• Registry-based recruitment
• Clinical characteristics and follow-up data ascertainment curated from registry
Risk Evaluation and its Impact on Clinical Decision-making and Outcomes in Heart Failure (REVEAL-HF)
NCT03845660
EHRUSA4000EHR-based alerts on updated patient prognosis when provider accesses patient chartsUsual care1-year all-cause mortality, 30-day hospital re-admission• Patients automatically identified through EHR if they meet inclusion criteria
• Patients randomized electronically to informational risk alert vs. usual care
• Alert displayed to provider at order entry
Dapagliflozin Effects on Cardiovascular Events in Patients with an Acute Heart Attack (DAPA-MI)
NCT04564742
SWEDEHEART and MINAP registries
Data collected by investigators
Sweden and UK6400DapagliflozinPlaceboFirst HF hospitalization or cardiovascular death• Patients identified via registries
• Follow-up data captured from registries or via mobile phone application
• Smart prescription bottles used to assess treatment adherence in real time
TrialData source(s)Location(s)Total N (planned)InterventionControl/other interventionPrimary outcomeTechnology-based interventions
Pragmatic Evaluation of Events and Benefits of Lipid-lowering in Older Adults (PREVENTABLE)
NCT04262206
EHR, Medicare, and National Death IndexUSA20 000AtorvastatinPlaceboDiagnosis of dementia and persistent disability• Identification of patients across 100 US sites through integrated EHR
• Cardiovascular event ascertainment through data curation from EHR, Medicare, and National Death Index
Pragmatic Randomized Trial of Icosapent Ethyl for High Cardiovascular Risk Adults (MITIGATE)
NCT04505098
Virtual follow-ups and EHRUSA16 500Pre-treatment with icosapent ethyl (IPE)Usual carePercentage of patients with viral upper respiratory tract infection (URI); worsening of clinical status of patients with confirmed viral URI• Trial embedded within KPNC’s learning healthcare delivery system; EHR-based recruitment from 21 hospitals, ∼255 outpatient clinics
• Pre-randomization of study subjects (prior to obtaining consent)
• Electronic informed consent
• No in-person visits; only EHR-based follow-up with monthly telephone reminders to ensure medication compliance
• Endpoint assessment via EHR-based validated approaches using ICD, CPT codes, and structured data elements
Pragmatic Trial of Messaging to Providers About Treatment of Hyperlipidemia (PROMPT-LIPID)
NCT04394715
EHRUSA100Electronic alert indicating patient’s ASCVD* riskUsual care with no alertsProportion of patients with intensification of lipid-lowering therapy• In-built electronic alerts in Epic to alert providers of patient’s ASCVD risk to assess appropriate intensification of lipid-lowering therapy
Early Dronedarone Versus Usual Care to Improve Outcomes in Persons with Newly Diagnosed Atrial Fibrillation (CHANGE-AFIB)
NCT05130268
GWTG atrial fibrillation registryUSA3000Dronedarone along with usual careUsual care (no anti-arrhythmic)Cardiovascular hospitalization or death• Trial embedded in GWTG registry
• Registry-based recruitment
• Clinical characteristics and follow-up data ascertainment curated from registry
Risk Evaluation and its Impact on Clinical Decision-making and Outcomes in Heart Failure (REVEAL-HF)
NCT03845660
EHRUSA4000EHR-based alerts on updated patient prognosis when provider accesses patient chartsUsual care1-year all-cause mortality, 30-day hospital re-admission• Patients automatically identified through EHR if they meet inclusion criteria
• Patients randomized electronically to informational risk alert vs. usual care
• Alert displayed to provider at order entry
Dapagliflozin Effects on Cardiovascular Events in Patients with an Acute Heart Attack (DAPA-MI)
NCT04564742
SWEDEHEART and MINAP registries
Data collected by investigators
Sweden and UK6400DapagliflozinPlaceboFirst HF hospitalization or cardiovascular death• Patients identified via registries
• Follow-up data captured from registries or via mobile phone application
• Smart prescription bottles used to assess treatment adherence in real time

ASCVD, atherosclerotic cardiovascular disease; GWTG, Get with The Guidelines; PCI, percutaneous coronary intervention; AF, atrial fibrillation; HF, heart failure; MI, myocardial infarction; SWEDEHEART, The Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies; MINAP, The Myocardial Ischaemia National Audit Project.

Table 2

Summary of ongoing cardiovascular RCTs leveraging routinely generated data and technology

TrialData source(s)Location(s)Total N (planned)InterventionControl/other interventionPrimary outcomeTechnology-based interventions
Pragmatic Evaluation of Events and Benefits of Lipid-lowering in Older Adults (PREVENTABLE)
NCT04262206
EHR, Medicare, and National Death IndexUSA20 000AtorvastatinPlaceboDiagnosis of dementia and persistent disability• Identification of patients across 100 US sites through integrated EHR
• Cardiovascular event ascertainment through data curation from EHR, Medicare, and National Death Index
Pragmatic Randomized Trial of Icosapent Ethyl for High Cardiovascular Risk Adults (MITIGATE)
NCT04505098
Virtual follow-ups and EHRUSA16 500Pre-treatment with icosapent ethyl (IPE)Usual carePercentage of patients with viral upper respiratory tract infection (URI); worsening of clinical status of patients with confirmed viral URI• Trial embedded within KPNC’s learning healthcare delivery system; EHR-based recruitment from 21 hospitals, ∼255 outpatient clinics
• Pre-randomization of study subjects (prior to obtaining consent)
• Electronic informed consent
• No in-person visits; only EHR-based follow-up with monthly telephone reminders to ensure medication compliance
• Endpoint assessment via EHR-based validated approaches using ICD, CPT codes, and structured data elements
Pragmatic Trial of Messaging to Providers About Treatment of Hyperlipidemia (PROMPT-LIPID)
NCT04394715
EHRUSA100Electronic alert indicating patient’s ASCVD* riskUsual care with no alertsProportion of patients with intensification of lipid-lowering therapy• In-built electronic alerts in Epic to alert providers of patient’s ASCVD risk to assess appropriate intensification of lipid-lowering therapy
Early Dronedarone Versus Usual Care to Improve Outcomes in Persons with Newly Diagnosed Atrial Fibrillation (CHANGE-AFIB)
NCT05130268
GWTG atrial fibrillation registryUSA3000Dronedarone along with usual careUsual care (no anti-arrhythmic)Cardiovascular hospitalization or death• Trial embedded in GWTG registry
• Registry-based recruitment
• Clinical characteristics and follow-up data ascertainment curated from registry
Risk Evaluation and its Impact on Clinical Decision-making and Outcomes in Heart Failure (REVEAL-HF)
NCT03845660
EHRUSA4000EHR-based alerts on updated patient prognosis when provider accesses patient chartsUsual care1-year all-cause mortality, 30-day hospital re-admission• Patients automatically identified through EHR if they meet inclusion criteria
• Patients randomized electronically to informational risk alert vs. usual care
• Alert displayed to provider at order entry
Dapagliflozin Effects on Cardiovascular Events in Patients with an Acute Heart Attack (DAPA-MI)
NCT04564742
SWEDEHEART and MINAP registries
Data collected by investigators
Sweden and UK6400DapagliflozinPlaceboFirst HF hospitalization or cardiovascular death• Patients identified via registries
• Follow-up data captured from registries or via mobile phone application
• Smart prescription bottles used to assess treatment adherence in real time
TrialData source(s)Location(s)Total N (planned)InterventionControl/other interventionPrimary outcomeTechnology-based interventions
Pragmatic Evaluation of Events and Benefits of Lipid-lowering in Older Adults (PREVENTABLE)
NCT04262206
EHR, Medicare, and National Death IndexUSA20 000AtorvastatinPlaceboDiagnosis of dementia and persistent disability• Identification of patients across 100 US sites through integrated EHR
• Cardiovascular event ascertainment through data curation from EHR, Medicare, and National Death Index
Pragmatic Randomized Trial of Icosapent Ethyl for High Cardiovascular Risk Adults (MITIGATE)
NCT04505098
Virtual follow-ups and EHRUSA16 500Pre-treatment with icosapent ethyl (IPE)Usual carePercentage of patients with viral upper respiratory tract infection (URI); worsening of clinical status of patients with confirmed viral URI• Trial embedded within KPNC’s learning healthcare delivery system; EHR-based recruitment from 21 hospitals, ∼255 outpatient clinics
• Pre-randomization of study subjects (prior to obtaining consent)
• Electronic informed consent
• No in-person visits; only EHR-based follow-up with monthly telephone reminders to ensure medication compliance
• Endpoint assessment via EHR-based validated approaches using ICD, CPT codes, and structured data elements
Pragmatic Trial of Messaging to Providers About Treatment of Hyperlipidemia (PROMPT-LIPID)
NCT04394715
EHRUSA100Electronic alert indicating patient’s ASCVD* riskUsual care with no alertsProportion of patients with intensification of lipid-lowering therapy• In-built electronic alerts in Epic to alert providers of patient’s ASCVD risk to assess appropriate intensification of lipid-lowering therapy
Early Dronedarone Versus Usual Care to Improve Outcomes in Persons with Newly Diagnosed Atrial Fibrillation (CHANGE-AFIB)
NCT05130268
GWTG atrial fibrillation registryUSA3000Dronedarone along with usual careUsual care (no anti-arrhythmic)Cardiovascular hospitalization or death• Trial embedded in GWTG registry
• Registry-based recruitment
• Clinical characteristics and follow-up data ascertainment curated from registry
Risk Evaluation and its Impact on Clinical Decision-making and Outcomes in Heart Failure (REVEAL-HF)
NCT03845660
EHRUSA4000EHR-based alerts on updated patient prognosis when provider accesses patient chartsUsual care1-year all-cause mortality, 30-day hospital re-admission• Patients automatically identified through EHR if they meet inclusion criteria
• Patients randomized electronically to informational risk alert vs. usual care
• Alert displayed to provider at order entry
Dapagliflozin Effects on Cardiovascular Events in Patients with an Acute Heart Attack (DAPA-MI)
NCT04564742
SWEDEHEART and MINAP registries
Data collected by investigators
Sweden and UK6400DapagliflozinPlaceboFirst HF hospitalization or cardiovascular death• Patients identified via registries
• Follow-up data captured from registries or via mobile phone application
• Smart prescription bottles used to assess treatment adherence in real time

ASCVD, atherosclerotic cardiovascular disease; GWTG, Get with The Guidelines; PCI, percutaneous coronary intervention; AF, atrial fibrillation; HF, heart failure; MI, myocardial infarction; SWEDEHEART, The Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies; MINAP, The Myocardial Ischaemia National Audit Project.

A call to action

The Clinical Trials Transformation Initiative (CTTI) has laid out five goals for the conduct of clinical trials, to be met by 2030. Figure 4 summarizes how leveraging routinely generated data can help cardiovascular trials achieve these goals. Similar goals and solutions have been proposed by expert panels in the UK and USA, which represents convergent thinking among academics across continents and across disciplines.101,102

Overview of how leveraging routinely generated data can help cardiovascular trials meet the Clinical Trials Transformation Initiative goals for clinical trials in 2030. EHR, electronic health record; CTTI, Clinical Trials Transformation Initiative.
Figure 4

Overview of how leveraging routinely generated data can help cardiovascular trials meet the Clinical Trials Transformation Initiative goals for clinical trials in 2030. EHR, electronic health record; CTTI, Clinical Trials Transformation Initiative.

Conclusion

Conducting cardiovascular RCTs using routinely generated healthcare data from EHRs, registries, and administrative databases has the potential to streamline all aspects of trial conduct, including the identification of eligible patients, recruitment, consent, randomization, follow-up, and outcome ascertainment. The EHR-embedded trials can reduce costs, thus allowing the enrollment of larger and more heterogeneous participant populations and longer follow-ups. Using routinely collected data to enhance enrollment of patients with broader eligibility criteria can also allow a shift towards pragmatism by yielding findings that are more generalizable to routine clinical practice. However, to successfully utilize routinely generated data to streamline cardiovascular RCTs, there is a need for technological innovations which allow capture of accurate data from diverse databases and standardize these data such that they can be used for analysis. Importantly, technological infrastructures which make routinely generated data ‘research-worthy’ need to be transferable such that multiple cardiovascular RCTs can be embedded into healthcare system through them over time. Moreover, these systems should align with patient privacy and data protection standards. Despite these barriers, certain cardiovascular trials have successfully utilized EHR systems to conduct large-scale trials. Given the high burden of cardiovascular disease and the fact that cardiovascular medicine has one of the highest outputs of RCTs annually, it is in a unique position to lead the adoption of EHR-embedded trials.

Key Definitions

Embedded RCTs: an embedded randomized clinical trial is one that is integrated within healthcare delivery settings, most often designed to optimize ease and breadth of recruitment and participation for both participants and investigators and to optimize generalizability of trial results.

Routinely generated data: refers to data from EHRs, registries, and administrative claims. Embedded trials often use these databases as the primary source of data for the study. Data from wearable sensors (i.e. fitness trackers) can also serve as a valuable source of routinely generated data.

Technology enablement: refers to transferable technological infrastructure to streamline trial conduct and connect disparate real-world data sources to operationalize the trial. It also refers to the ability to contact patients through smart devices to systematically and efficiently collect patient-reported outcome questionnaires.

Pragmatism: refers to trial designs with a focus on simplicity and generalizability. The goal of pragmatic trials is to assess effectiveness in everyday clinical settings rather than efficacy in highly controlled settings and to minimize trial-specific organization and data collection. Pragmatism is a broad term encompassing multiple design domains; however, the use of routinely generated data and technology enablement are two methods to enhance pragmatism.

Supplementary data

Supplementary data is available at European Heart Journal online.

Data availability

No new data were generated or analysed in support of this review article.

Funding

All authors declare no funding for this contribution.

Author contributions

Muhammad Shahzeb Khan (Conceptualization [lead], Project administration [lead], Supervision [equal]), Stefan K. James (Methodology [equal], Supervision [equal], Writing—original draft [supporting]), Stefan D. Anker (Investigation [equal], Writing—original draft [equal], Writing—review & editing [equal]), Matthew T. Roe (Investigation [equal], Supervision [equal], Writing—review & editing [equal]), John A. Spertus (Investigation [equal], Supervision [equal]), Sunil Rao (Investigation [equal], Supervision [equal], Writing—review & editing [equal]), Gregg C. Fonarow (Supervision [equal], Writing—original draft [equal], Writing—review & editing [equal]), Robert J. Mentz (Investigation [equal], Supervision [equal], Writing—original draft [equal], Writing—review & editing [equal]), Ziad A. Ali (Supervision [equal], Writing—review & editing [equal]), Nicholas L. Mills (Supervision [equal], Writing—original draft [supporting], Writing—review & editing [supporting]), Sadiya S. Khan (Conceptualization [supporting], Supervision [equal], Writing—original draft [Supporting]), Muthiah Vaduganathan (Investigation [supporting], Supervision [equal], Writing—original draft [supporting]), Stephen J. Greene (Supervision [equal], Writing—original draft [supporting], Writing—review & editing [equal]), Harriette G.C. Van Spall (Supervision: Supporting; Writing—original draft: Supporting; Writing—review & editing: Equal), Talha Khuwaja (Investigation [equal], Writing—original draft [equal], Writing—review & editing [equal]), Muhammad Shariq Usman (Conceptualization [equal], Investigation [equal], Methodology [supporting], Writing—original draft [lead]), Javed Butler (Investigation [equal], Supervision [lead], Writing—review & editing [lead]), and Darren K. McGuire (Investigation [equal], Supervision [lead], Writing—review & editing [lead]).

References

1

Lopaschuk Gary
D
,
Verma
S
.
Mechanisms of cardiovascular benefits of sodium glucose co-transporter 2 (SGLT2) inhibitors
.
JACC Basic Transl Sci
2020
;
5
:
632
644
. https://doi.org/10.1016/j.jacbts.2020.02.004

2

Winter
M-P
,
Bartko
P
,
Hofer
F
,
Zbiral
M
,
Burger
A
,
Ghanim
B
, et al.
Evolution of outcome and complications in TAVR: a meta-analysis of observational and randomized studies
.
Sci Rep
2020
;
10
:
15568
. https://doi.org/10.1038/s41598-020-72453-1

3

Rodriguez
F
,
Califf
RM
,
Harrington
RA
.
Consequences of slow progress toward pragmatism in randomized clinical trials: it is time to get practical
.
JAMA Cardiol
2019
;
4
:
1129
1130
. https://doi.org/10.1001/jamacardio.2019.3922

4

Ford
I
,
Norrie
J
.
Pragmatic trials
.
N Engl J Med
2016
;
375
:
454
463
. https://doi.org/10.1056/NEJMra1510059

5

Loudon
K
,
Treweek
S
,
Sullivan
F
,
Donnan
P
,
Thorpe
KE
,
Zwarenstein
M
.
The PRECIS-2 tool: designing trials that are fit for purpose
.
BMJ
2015
;
350
:
h2147
. https://doi.org/10.1136/bmj.h2147

6

Enriquez
JR
,
de Lemos
JA
,
Parikh
SV
,
Simon
DN
,
Thomas
LE
,
Wang
TY
, et al.
Modest associations between electronic health record use and acute myocardial infarction quality of care and outcomes
.
Circ Cardiovasc Qual Outcomes
2015
;
8
:
576
585
.

7

Manca
DP
.
Do electronic medical records improve quality of care? Yes
.
Can Fam Physician
2015
;
61
:
846
851
.

8

Menachemi
N
,
Collum
TH
.
Benefits and drawbacks of electronic health record systems
.
Risk Manag Healthc Policy
2011
;
4
:
47
55
. https://doi.org/10.2147/RMHP.S12985

9

Payne
TH
.
The electronic health record as a catalyst for quality improvement in patient care
.
Heart
2016
;
102
:
1782
1787
. https://doi.org/10.1136/heartjnl-2015-308724

10

Adler-Milstein
J
,
Holmgren
AJ
,
Kralovec
P
,
Worzala
C
,
Searcy
T
,
Patel
V
.
Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide
.
J Am Med Inform Assoc
2017
;
24
:
1142
1148
. https://doi.org/10.1093/jamia/ocx080

11

Bayoumy
K
,
Gaber
M
,
Elshafeey
A
,
Mhaimeed
O
,
Dineen
EH
,
Marvel
FA
, et al.
Smart wearable devices in cardiovascular care: where we are and how to move forward
.
Nat Rev Cardiol
2021
;
18
:
581
599
. https://doi.org/10.1038/s41569-021-00522-7

12

Dash
S
,
Shakyawar
SK
,
Sharma
M
,
Kaushik
S
.
Big data in healthcare: management, analysis and future prospects
.
J Big Data
2019
;
6
:
54
. https://doi.org/10.1186/s40537-019-0217-0

13

Kotecha
D
,
DeVore
AD
,
Asselbergs
FW
.
Fit for the future: empowering clinical trials with digital technology
.
Eur Heart J
2023
;
44
:
64
67
. https://doi.org/10.1093/eurheartj/ehac650

14

Hemingway
H
,
Asselbergs
FW
,
Danesh
J
,
Dobson
R
,
Maniadakis
N
,
Maggioni
A
, et al.
Big data from electronic health records for early and late translational cardiovascular research: challenges and potential
.
Eur Heart J
2018
;
39
:
1481
1495
. https://doi.org/10.1093/eurheartj/ehx487

15

Baldi
I
,
Lanera
C
,
Berchialla
P
,
Gregori
D
.
Early termination of cardiovascular trials as a consequence of poor accrual: analysis of ClinicalTrials.gov 2006-2015
.
BMJ Open
2017
;
7
:
e013482-e
. https://doi.org/10.1136/bmjopen-2016-013482

16

Huang
GD
,
Bull
J
,
Johnston McKee
K
,
Mahon
E
,
Harper
B
,
Roberts
JN
.
Clinical trials recruitment planning: a proposed framework from the clinical trials transformation initiative
.
Contemp Clin Trials
2018
;
66
:
74
79
. https://doi.org/10.1016/j.cct.2018.01.003

17

Winkler
G
,
Levin
EA
,
Whalen
EY
,
Larus
J
.
Overcoming “trial fatigue”: a strategy for optimizing patient accrual speed and resource utilization
.
Drug Info J
1996
;
30
:
35
40
. https://doi.org/10.1177/009286159603000105

18

Getz
KA
,
Stergiopoulos
S
,
Short
M
,
Surgeon
L
,
Krauss
R
,
Pretorius
S
, et al.
The impact of protocol amendments on clinical trial performance and cost
.
Ther Innov Regul Sci
2016
;
50
:
436
441
.

19

Tahhan
AS
,
Vaduganathan
M
,
Greene
SJ
,
Fonarow
GC
,
Fiuzat
M
,
Jessup
M
, et al.
Enrollment of older patients, women, and racial and ethnic minorities in contemporary heart failure clinical trials: a systematic review
.
JAMA Cardiol
2018
;
3
:
1011
1019
. https://doi.org/10.1001/jamacardio.2018.2559

20

Howard
L
,
de Salis
I
,
Tomlin
Z
,
Thornicroft
G
,
Donovan
J
.
Why is recruitment to trials difficult? An investigation into recruitment difficulties in an RCT of supported employment in patients with severe mental illness
.
Contemp Clin Trials
2009
;
30
:
40
46
. https://doi.org/10.1016/j.cct.2008.07.007

21

Marquis-Gravel
G
,
Roe
MT
,
Turakhia
MP
,
Boden
W
,
Temple
R
,
Sharma
A
, et al.
Technology-enabled clinical trials
.
Circulation
2019
;
140
:
1426
1436
. https://doi.org/10.1161/CIRCULATIONAHA.119.040798

22

Rapsomaniki
E
,
Shah
A
,
Perel
P
,
Denaxas
S
,
George
J
,
Nicholas
O
, et al.
Prognostic models for stable coronary artery disease based on electronic health record cohort of 102 023 patients
.
Eur Heart J
2014
;
35
:
844
852
. https://doi.org/10.1093/eurheartj/eht533

23

Reading Turchioe
M
,
Volodarskiy
A
,
Pathak
J
,
Wright
DN
,
Tcheng
JE
,
Slotwiner
D
.
Systematic review of current natural language processing methods and applications in cardiology
.
Heart
2022
;
108
:
909
916
. https://doi.org/10.1136/heartjnl-2021-319769

24

Fiore
LD
,
Brophy
M
,
Ferguson
RE
,
D’Avolio
L
,
Hermos
JA
,
Lew
RA
, et al.
A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen
.
Clin Trials
2011
;
8
:
183
195
. https://doi.org/10.1177/1740774511398368

25

Chan
PS
,
Oetgen
WJ
,
Buchanan
D
,
Mitchell
K
,
Fiocchi
FF
,
Tang
F
, et al.
Cardiac performance measure compliance in outpatients: the American College of Cardiology and National Cardiovascular Data Registry’s PINNACLE (practice innovation and clinical excellence) program
.
J Am Coll Cardiol
2010
;
56
:
8
14
. https://doi.org/10.1016/j.jacc.2010.03.043

26

Tran
AT
,
Chan
PS
,
Jones
PG
,
Spertus
JA
.
Comparison of patient self-reported health status with clinician-assigned New York Heart Association classification
.
JAMA Network Open
2020
;
3
:
e2014319
. https://doi.org/10.1001/jamanetworkopen.2020.14319

27

Bennett
JA
,
Riegel
B
,
Bittner
V
,
Nichols
J
.
Validity and reliability of the NYHA classes for measuring research outcomes in patients with cardiac disease
.
Heart Lung
2002
;
31
:
262
270
. https://doi.org/10.1067/mhl.2002.124554

28

Raphael
C
,
Briscoe
C
,
Davies
J
,
Ian Whinnett
Z
,
Manisty
C
,
Sutton
R
, et al.
Limitations of the New York Heart Association functional classification system and self-reported walking distances in chronic heart failure
.
Heart
2007
;
93
:
476
482
. https://doi.org/10.1136/hrt.2006.089656

29

Greene
SJ
,
Butler
J
,
Spertus
JA
,
Hellkamp
AS
,
Vaduganathan
M
,
DeVore
AD
, et al.
Comparison of New York Heart Association class and patient-reported outcomes for heart failure with reduced ejection fraction
.
JAMA Cardiol
2021
;
6
:
522
531
. https://doi.org/10.1001/jamacardio.2021.0372

30

Spertus
JA
,
Birmingham
MC
,
Butler
J
,
Lingvay
I
,
Lanfear
DE
,
Abbate
A
, et al.
Novel trial design: CHIEF-HF
.
Circ Heart Fail
2021
;
14
:
e007767
.

31

Marquis-Gravel
G
,
Roe
MT
,
Robertson
HR
,
Harrington
RA
,
Pencina
MJ
,
Berdan
LG
, et al.
Rationale and design of the aspirin dosing—a patient-centric trial assessing benefits and long-term effectiveness (ADAPTABLE) trial
.
JAMA Cardiol
2020
;
5
:
598
607
. https://doi.org/10.1001/jamacardio.2020.0116

32

Jones
WS
,
Mulder
H
,
Wruck
LM
,
Pencina MJ, Kripalani S, Muñoz D
, et al.
Comparative effectiveness of aspirin dosing in cardiovascular disease
.
N Engl J Med
2021
;
384
:
1981
1990
. https://doi.org/10.1056/NEJMoa2102137

33

US Food and Drug Adminstration
.
Use of Electronic Informed Consent
. https://www.fda.gov/media/116850/download. Last Accessed: 21st Nov 2022.

34

Sertkaya
A
,
Birkenbach
A
,
Berlind
A
,
Eyraud
J
;
Eastern Research Group
.
Examination of Clinical Trial Costs and Barriers for Drug Development 2014.
https://aspe.hhs.gov/reports/examination-clinical-trial-costs-barriers-drug-development-0

35

Examination of Clinical Trial Costs and Barriers for Drug Development
.

36

Group Sustainable Trials Study
.
Towards sustainable clinical trials
.
BMJ
.
2007
;
334
:
671
673
. https://doi.org/10.1136/bmj.39140.623137.BE

37

FDA
.
Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring.
https://www.fda.gov/media/116754/download

38

McConnell
MV
,
Shcherbina
A
,
Pavlovic
A
,
Homburger JR, Goldfeder RL, Waggot D
, et al.
Feasibility of obtaining measures of lifestyle from a smartphone app: the MyHeart counts cardiovascular health study
.
JAMA Cardiol
2017
;
2
:
67
76
. https://doi.org/10.1001/jamacardio.2016.4395

39

Perez
MV
,
Mahaffey
KW
,
Hedlin
H
,
Rumsfeld JS, Garcia A, Ferris T
, et al.
Large-scale assessment of a smartwatch to identify atrial fibrillation
.
N Engl J Med
2019
;
381
:
1909
1917
. https://doi.org/10.1056/NEJMoa1901183

40

Redfield
MM
,
Anstrom
KJ
,
Levine
JA
,
Koepp GA, Borlaug BA, Chen HH
, et al.
Isosorbide mononitrate in heart failure with preserved ejection fraction
.
N Engl J Med
2015
;
373
:
2314
2324
. https://doi.org/10.1056/NEJMoa1510774

41

Weissler
EH
,
Naumann
T
,
Andersson
T
,
Ranganath
R
,
Elemento
O
,
Luo
Y
, et al.
The role of machine learning in clinical research: transforming the future of evidence generation
.
Trials
2021
;
22
:
537
. https://doi.org/10.1186/s13063-021-05489-x

42

Moore
CB
,
Verma
A
,
Pendergrass
S
,
Verma
SS
,
Johnson
DH
,
Daar
ES
, et al.
Phenome-wide association study relating pretreatment laboratory parameters with human genetic variants in AIDS Clinical Trials Group protocols
.
Open Forum Infect Dis
2015
;
2
:
ofu113
. https://doi.org/10.1093/ofid/ofu113

43

Kullo
IJ
,
Fan
J
,
Pathak
J
,
Savova
GK
,
Ali
Z
,
Chute
CG
.
Leveraging informatics for genetic studies: use of the electronic medical record to enable a genome-wide association study of peripheral arterial disease
.
J Am Med Inform Assoc
2010
;
17
:
568
574
. https://doi.org/10.1136/jamia.2010.004366

44

Kullo
IJ
,
Jouni
H
,
Austin
EE
,
Brown
SA
,
Kruisselbrink
TM
,
Isseh
IN
, et al.
Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES Clinical Trial)
.
Circulation
2016
;
133
:
1181
1188
. https://doi.org/10.1161/CIRCULATIONAHA.115.020109

45

Herr
TM
,
Peterson
JF
,
Rasmussen
LV
,
Caraballo
PJ
,
Peissig
PL
,
Starren
JB
.
Pharmacogenomic clinical decision support design and multi-site process outcomes analysis in the eMERGE Network
.
J Am Med Inform Assoc
2019
;
26
:
143
148
. https://doi.org/10.1093/jamia/ocy156

46

Listgarten
J
,
Stegle
O
,
Morris
Q
,
Brenner
SE
,
Parts
L
.
Personalized medicine: from genotypes and molecular phenotypes towards therapy—session introduction
.
Pac Symp Biocomput
2014
;
19
:
224
228
.

47

Raman
SR
,
Curtis
LH
,
Temple
R
,
Andersson
T
,
Ezekowitz
J
,
Ford
I
, et al.
Leveraging electronic health records for clinical research
.
Am Heart J
2018
;
202
:
13
19
. https://doi.org/10.1016/j.ahj.2018.04.015

48

Fawcett
N
,
Young
B
,
Peto
L
,
Quan
TP
,
Gillott
R
,
Wu
J
, et al.
‘Caveat emptor’: the cautionary tale of endocarditis and the potential pitfalls of clinical coding data—an electronic health records study
.
BMC Med
2019
;
17
:
169
. https://doi.org/10.1186/s12916-019-1390-x

49

Edgcomb
JB
,
Zima
B
.
Machine learning, natural language processing, and the electronic health record: innovations in mental health services research
.
Psychiatr Serv
.
2019
;
70
:
346
349
. https://doi.org/10.1176/appi.ps.201800401

50

Kotecha
D
,
Asselbergs
FW
,
Achenbach
S
,
Anker
SD
,
Atar
D
,
Baigent
C
, et al.
CODE-EHR best practice framework for the use of structured electronic healthcare records in clinical research
.
Eur Heart J
2022
;
43
:
3578
3588
. https://doi.org/10.1093/eurheartj/ehac426

51

Gonçalves-Ferreira
D
,
Sousa
M
,
Bacelar-Silva
GM
,
Frade
S
,
Antunes
LF
,
Beale
T
, et al.
OpenEHR and general data protection regulation: evaluation of principles and requirements
.
JMIR Med Inform
2019
;
7
:
e9845
. https://doi.org/10.2196/medinform.9845

52

Farhadi
M
,
Haddad
HM
,
Shahriar
H
.
Compliance of Open Source EHR Applications with HIPAA and ONC Security and Privacy Requirements 2019
.

55

Shah
ASV
,
Anand
A
,
Strachan
FE
,
Ferry
AV
,
Lee
KK
,
Chapman
AR
, et al.
High-sensitivity troponin in the evaluation of patients with suspected acute coronary syndrome: a stepped-wedge, cluster-randomised controlled trial
.
Lancet
2018
;
392
:
919
928
. https://doi.org/10.1016/S0140-6736(18)31923-8

56

Anand
A
,
Lee
KK
,
Chapman
AR
,
Ferry
AV
,
Adamson
PD
,
Strachan
FE
, et al.
High-sensitivity cardiac troponin on presentation to rule out myocardial infarction: a stepped-wedge cluster randomized controlled trial
.
Circulation
2021
;
143
:
2214
2224
. https://doi.org/10.1161/CIRCULATIONAHA.120.052380

57

Adler-Milstein
J
,
Jha
AK
.
HITECH Act drove large gains in hospital electronic health record adoption
.
Health Aff
2017
;
36
:
1416
1422
. https://doi.org/10.1377/hlthaff.2016.1651

58

Forrest
CB
,
McTigue
KM
,
Hernandez
AF
,
Cohen
LW
,
Cruz
H
,
Haynes
K
, et al.
PCORnet® 2020: current state, accomplishments, and future directions
.
J Clin Epidemiol
2021
;
129
:
60
67
. https://doi.org/10.1016/j.jclinepi.2020.09.036

59

Reisman
M
.
EHRs: the challenge of making electronic data usable and interoperable
.
P T
2017
;
42
:
572
575
.

60

Adler-Milstein
J
,
Pfeifer
E
.
Information blocking: is it occurring and what policy strategies can address it?
Milbank Q
2017
;
95
:
117
135
. https://doi.org/10.1111/1468-0009.12247

61

Whicher
DM
,
Miller
JE
,
Dunham
KM
,
Joffe
S
.
Gatekeepers for pragmatic clinical trials
.
Clin Trials
2015
;
12
:
442
448
. https://doi.org/10.1177/1740774515597699

62

Erlinge
D
,
Omerovic
E
,
Fröbert
O
,
Linder
R
,
Danielewicz
M
,
Hamid
M
, et al.
Bivalirudin versus heparin monotherapy in myocardial infarction
.
N Engl J Med
2017
;
377
:
1132
1142
. https://doi.org/10.1056/NEJMoa1706443

63

Fröbert
O
,
Lagerqvist
B
,
Olivecrona
GK
,
Omerovic
E
,
Gudnason
T
,
Maeng
M
, et al.
Thrombus aspiration during ST-segment elevation myocardial infarction
.
N Engl J Med
2013
;
369
:
1587
1597
. https://doi.org/10.1056/NEJMoa1308789

64

Wallentin
L
,
Gale
CP
,
Maggioni
A
,
Bardinet
I
,
Casadei
B
.
Euroheart: European Unified Registries On Heart care Evaluation and Randomized Trials: an ESC project to develop a new IT registry system which will encompass multiple features of cardiovascular medicine
.
Eur Heart J
2019
;
40
:
2745
2749
. https://doi.org/10.1093/eurheartj/ehz599

65

Beresniak
A
,
Schmidt
A
,
Proeve
J
,
Bolanos
E
,
Patel
N
,
Ammour
N
, et al.
Cost-benefit assessment of using electronic health records data for clinical research versus current practices: Contribution of the Electronic Health Records for Clinical Research (EHR4CR) European Project
.
Contemp Clin Trials
2016
;
46
:
85
91
. https://doi.org/10.1016/j.cct.2015.11.011

66

The global health research network
. https://trinetx.com/about-trinetx/community/ (21 November 2022, date last accessed).

67

FitzHenry
F
,
Resnic
FS
,
Robbins
SL
,
Denton
J
,
Nookala
L
,
Meeker
D
, et al.
Creating a common data model for comparative effectiveness with the observational medical outcomes partnership
.
Appl Clin Inform
2015
;
06
:
536
547
. https://doi.org/10.4338/ACI-2014-12-CR-0121

68

Wood
FE
,
Fitzsimmons
MJ
.
Clinical data interchange standards consortium (CDISC) standards and their implementation in a clinical data management system
.
Drug Inform J
2001
;
35
:
853
862
. https://doi.org/10.1177/009286150103500323

69

Garcia-Garcia
HM
,
McFadden
EP
,
Farb
A
,
Mehran
R
,
Stone
GW
,
Spertus
J
, et al.
Standardized end point definitions for coronary intervention trials: the academic research consortium-2 consensus document
.
Circulation
2018
;
137
:
2635
2650
. https://doi.org/10.1161/CIRCULATIONAHA.117.029289

70

Embi
PJ
,
Leonard
AC
.
Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study
.
J Am Med Inform Assoc
.
2012
;
19
(
e1
):
e145-8
. https://doi.org/10.1136/amiajnl-2011-000743

71

McGreevey
JD
III,
Mallozzi
CP
,
Perkins
RM
,
Shelov
E
,
Schreiber
R
.
Reducing alert burden in electronic health records: state of the art recommendations from four health systems
.
Appl Clin Inform
2020
;
11
:
1
12
.

72

Hussain
MI
,
Reynolds
TL
,
Zheng
K
.
Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review
.
J Am Med Inform Assoc
2019
;
26
:
1141
1149
. https://doi.org/10.1093/jamia/ocz095

73

Spertus
JA
,
Birmingham
MC
,
Nassif
M
,
Damaraju CV, Abbate A, Butler J
, et al.
The SGLT2 inhibitor canagliflozin in heart failure: the CHIEF-HF remote, patient-centered randomized trial
.
Nat Med
2022
;
28
:
809
813
. https://doi.org/10.1038/s41591-022-01703-8

74

Meah
MN
,
Denvir
MA
,
Mills
NL
,
Norrie
J
,
Newby
DE
.
Clinical endpoint adjudication
.
Lancet
2020
;
395
:
1878
1882
. https://doi.org/10.1016/S0140-6736(20)30635-8

75

Farrant
M
,
Easton
JD
,
Adelman
EE
,
Cucchiara
BL
,
Barsan
WG
,
Tillman
HJ
, et al.
Assessment of the end point adjudication process on the results of the platelet-oriented inhibition in new TIA and minor ischemic stroke (POINT) trial: a secondary analysis
.
JAMA Network Open
2019
;
2
:
e1910769
. https://doi.org/10.1001/jamanetworkopen.2019.10769

76

Strom
JB
,
Faridi
KF
,
Butala
NM
,
Zhao
Y
,
Tamez
H
,
Valsdottir
LR
, et al.
Use of administrative claims to assess outcomes and treatment effect in randomized clinical trials for transcatheter aortic valve replacement
.
Circulation
2020
;
142
:
203
213
. https://doi.org/10.1161/CIRCULATIONAHA.120.046159

77

Butala Neel
M
,
Strom Jordan
B
,
Faridi Kamil
F
,
Kazi Dhruv
S
,
Zhao
Y
,
Brennan
JM
, et al.
Validation of administrative claims to ascertain outcomes in pivotal trials of transcatheter aortic valve replacement
.
JACC Cardiovasc Interv
2020
;
13
:
1777
1785
. https://doi.org/10.1016/j.jcin.2020.03.049

78

Ambrosy
AP
,
Parikh
RV
,
Sung
SH
,
Narayanan
A
,
Masson
R
,
Lam
PQ
, et al.
A natural language processing-based approach for identifying hospitalizations for worsening heart failure within an integrated health care delivery system
.
JAMA Netw Open
2021
;
4
:
e2135152
. https://doi.org/10.1001/jamanetworkopen.2021.35152

79

Kehl
KL
,
Elmarakeby
H
,
Nishino
M
,
Van Allen
EM
,
Lepisto
EM
,
Hassett
MJ
, et al.
Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports
.
JAMA Oncol
2019
;
5
:
1421
1429
. https://doi.org/10.1001/jamaoncol.2019.1800

80

Kehl
KL
,
Xu
W
,
Lepisto
E
,
Elmarakeby
H
,
Hassett
MJ
,
Van Allen
EM
, et al.
Natural language processing to ascertain cancer outcomes from medical oncologist notes
.
JCO Clin Cancer Inform
2020
;
4
:
680
690
. https://doi.org/10.1200/CCI.20.00020

81

Benchimol
EI
,
Smeeth
L
,
Guttmann
A
,
Harron
K
,
Moher
D
,
Petersen
I
, et al.
The Reporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement
.
PLoS Med
2015
;
12
:
e1001885
. https://doi.org/10.1371/journal.pmed.1001885

82

Vijayananthan
A
,
Nawawi
O
.
The importance of good clinical practice guidelines and its role in clinical trials
.
Biomed Imaging Interv J
2008
;
4
:
e5
. https://doi.org/10.2349/biij.4.1.e5

83

Mentz
RJ
,
Hernandez
AF
,
Berdan
LG
,
Rorick
T
,
O’Brien
EC
,
Ibarra
JC
, et al.
Good clinical practice guidance and pragmatic clinical trials: balancing the best of both worlds
.
Circulation
2016
;
133
:
872
880
. https://doi.org/10.1161/CIRCULATIONAHA.115.019902

84

Kwakkenbos
L
,
Imran
M
,
McCall
SJ
,
McCord
KA
,
Fröbert
O
,
Hemkens
LG
, et al.
CONSORT extension for the reporting of randomised controlled trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE): checklist with explanation and elaboration
.
BMJ
2021
;
373
:
n857
.

86

MHRA guidance on the use of real-world data in clinical studies to support regulatory decisions.
https://www.gov.uk/government/publications/mhra-guidance-on-the-use-of-real-world-data-in-clinical-studies-to-support-regulatory-decisions (30 January 2023, date last accessed).

87

Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products 2021.
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory

88

US FDA.
2022
. Digital Health Technologies for Remote Data Acquisition in Clinical Investigations. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/digital-health-technologies-remote-data-acquisition-clinical-investigations (4 March 2022, date last accessed).

89

Van Spall HGC, Lee SF, Xie F, Oz UE, Perez R, Mitoff PR
, et al.
Effect of patient-centered transitional care services on clinical outcomes in patients hospitalized for heart failure: the PACT-HF randomized clinical trial
.
JAMA
2019
;
321
:
753
761
. https://doi.org/10.1001/jama.2019.0710

90

Martin SS, Feldman DI, Blumenthal RS, Jones SR, Post WS, McKibben RA
, et al.
mActive: a randomized clinical trial of an automated mHealth intervention for physical activity promotion
.
J Am Heart Assoc
2015
;
4
:
e002239
. https://doi.org/10.1161/JAHA.115.002239

91

Rao SV, Hess CN, Barham B, Aberle LH, Anstrom KJ, Patel TB
, et al.
A registry-based randomized trial comparing radial and femoral approaches in women undergoing percutaneous coronary intervention: the SAFE-PCI for Women (Study of Access Site for Enhancement of PCI for Women) trial
.
JACC Cardiovasc Interv
2014
;
7
:
857
867
. https://doi.org/10.1016/j.jcin.2014.04.007

92

Marcus GM, Rosenthal DG, Nah G, Vittinghoff E, Fang C, Ogomori K
, et al.
Acute effects of coffee consumption on health among ambulatory adults
.
N Engl J Med
2023
;
388
:
1092
1100
. https://doi.org/10.1056/NEJMoa2204737

93

Marcus GM, Modrow MF, Schmid CH, Sigona K, Nah G, Yang J
, et al.
Individualized studies of triggers of paroxysmal atrial fibrillation: the I-STOP-AFib randomized clinical trial
.
JAMA Cardiol
2022
;
7
:
167
174
. https://doi.org/10.1001/jamacardio.2021.5010

94

Reed MJ, Grubb NR, Lang CC, O'Brien R, Simpson K, Padarenga M
, et al.
Multi-centre randomised controlled trial of a smartphone-based event recorder alongside standard care versus standard care for patients presenting to the emergency department with palpitations and pre-syncope: the IPED (Investigation of Palpitations in the ED) study
.
EClinicalMedicine
2019
;
8
:
37
46
. https://doi.org/10.1016/j.eclinm.2019.02.005

95

Steinhubl SR, Waalen J, Edwards AM, Ariniello LM, Mehta RR, Ebner GS
, et al.
Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS randomized clinical trial
.
JAMA
2018
;
320
:
146
155
. https://doi.org/10.1001/jama.2018.8102

96

Koehler F, Koehler K, Deckwart O, Prescher S, Wegscheider K, Winkler S
, et al.
Telemedical Interventional Management in Heart Failure II (TIM-HF2), a randomised, controlled trial investigating the impact of telemedicine on unplanned cardiovascular hospitalisations and mortality in heart failure patients: study design and description of the intervention
.
Eur J Heart Fail
2018
;
20
:
1485
1493
. https://doi.org/10.1002/ejhf.1300

97

Dendale P, De Keulenaer G, Troisfontaines P, Weytjens C, Mullens W, Elegeert I
, et al.
Effect of a telemonitoring-facilitated collaboration between general practitioner and heart failure clinic on mortality and rehospitalization rates in severe heart failure: the TEMA-HF 1 (TElemonitoring in the MAnagement of Heart Failure) study
.
Eur J Heart Fail
2012
;
14
:
333
340
. https://doi.org/10.1093/eurjhf/hfr144

98

Andell P, James S, Östlund O, Yndigegn T, Sparv D, Pernow J
, et al.
Oxygen therapy in suspected acute myocardial infarction and concurrent normoxemic chronic obstructive pulmonary disease: a prespecified subgroup analysis from the DETO2X-AMI trial
.
Eur Heart J Acute Cardiovasc Care
2020
;
9
:
984
992
. https://doi.org/10.1177/2048872619848978

99

Ritchie CS, Houston TK, Richman JS, Sobko HJ, Berner ES, Taylor BB
, et al.
The E-Coach technology-assisted care transition system: a pragmatic randomized trial
.
Transl Behav Med
2016
;
6
:
428
437
. https://doi.org/10.1007/s13142-016-0422-8

100

Mentz RJ, Anstrom KJ, Eisenstein EL, Sapp S, Greene SJ, Morgan S
, et al.
Effect of torsemide vs furosemide after discharge on all-cause mortality in patients hospitalized with heart failure: the TRANSFORM-HF randomized clinical trial
.
JAMA
2023
;
329
:
214
223
. https://doi.org/10.1001/jama.2022.23924

101

Sydes
MR
,
Barbachano
Y
,
Bowman
L
,
Denwood
T
,
Farmer
A
,
Garfield-Birkbeck
S
, et al.
Realising the full potential of data-enabled trials in the UK: a call for action
.
BMJ Open
2021
;
11
:
e043906
. https://doi.org/10.1136/bmjopen-2020-043906

102

Warner
JJ
,
Crook
HL
,
Whelan
KM
,
Bleser
WK
,
Roiland
RA
,
Hamilton Lopez
M
, et al.
Improving cardiovascular drug and device development and evidence through patient-centered research and clinical trials: a call to action from the value in healthcare initiative’s partnering with regulators learning collaborative
.
Circ Cardiovasc Qual Outcomes
2020
;
13
:
e006606
. https://doi.org/10.1161/CIRCOUTCOMES.120.006606

Author notes

Muhammad Shahzeb Khan and Muhammad Shariq Usman co-first authors.

Conflict of interest H.G.C.V.S. leads trials with routinely available data, with funding from the Canadian Institutes of Health Research and the Heart and Stroke Foundation of Canada. S.J.G. has received research support from the Duke University Department of Medicine Chair’s Research Award, American Heart Association, Amgen, AstraZeneca, Bristol Myers Squibb, Cytokinetics, Merck, Novartis, Pfizer, and Sanofi; has served on advisory boards for Amgen, AstraZeneca, Bristol Myers Squibb, Cytokinetics, and Sanofi; and serves as a consultant for Amgen, Bayer, Bristol Myers Squibb, Merck, and Vifor. M.V. has received research grant support, served on advisory boards, or had speaker engagements with American Regent, Amgen, AstraZeneca, Bayer AG, Baxter Healthcare, Boehringer Ingelheim, Chiesi, Cytokinetics, Lexicon Pharmaceuticals, Merck, Novartis, Novo Nordisk, Pharmacosmos, Relypsa, Roche Diagnostics, Sanofi, and Tricog Health, and participates on clinical trial committees for studies sponsored by AstraZeneca, Galmed, Novartis, Bayer AG, Occlutech, and Impulse Dynamics. S.S.K. reported grants from the American Heart Association and the National Institutes of Health outside the submitted work. N.L.M. reports research grants awarded to the University of Edinburgh from Abbott Diagnostics and Siemens Healthineers outside the submitted work and honoraria from Abbott Diagnostics, Siemens Healthineers, Roche Diagnostics, and LumiraDx. Z.A.A. reports institutional research grants to Columbia University—Abbott, Cardiovascular Systems, Inc and serves as a consultant at Abbott, Abiomed, AstraZeneca, and Shockwave. R.J.M. reported receiving personal fees from Novartis International AG, Amgen Inc, Boehringer Ingelheim, Bayer AG, and Merck & Co, Inc and receiving research support and honoraria from Abbott Laboratories, American Regent, Inc, Amgen Inc, AstraZeneca, Bayer AG, Boehringer Ingelheim/Eli Lilly & Company, Boston Scientific Corporation, Cytokinetics, Inc, FAST BioMedical, Gilead Sciences, Inc, Innolife, Medtronic PLC, Merck & Co, Inc, Novartis International AG, Relypsa, Inc, Respicardia, Windtree Therapeutics, Inc, and ZOLL Medical Corporation outside the submitted work. G.C.F. reports research support from the National Institutes of Health and consulting for Abbott, Amgen, AstraZeneca, Bayer, Cytokinetics, Eli Lilly, Janssen, Medtronic, Merck, Novartis, and Pfizer. S.V.R. reported receiving institutional research funding from Bayer and the National Heart, Lung, and Blood Institute outside the submitted work. J.A.S. is a consultant to Bayer, Janssen, Merck, Bristol Meyers Squibb, Kineksa, Novartis, Terumo, and United Healthcare. He receives grant support from Janssen, Abbott Vascular, and Bristol Meyers Squibb. He holds the copyright to the Peripheral Artery Questionnaire, Kansas City Cardiomyopathy Questionnaires, and the Seattle Angina Questionnaire. He serves on the Board of Blue Cross/Blue Shield of Kansas City. M.T.R. reports grants from the American College of Cardiology, American Heart Association, Bayer Pharmaceuticals, Familial Hypercholesterolemia Foundation, Ferring Pharmaceuticals, Myokardia, and Patient Centered Outcomes Research Institute; grants and personal fees from Amgen, AstraZeneca, and Sanofi Aventis; personal fees from Janssen Pharmaceuticals, Elsevier Publishers, Regeneron, Roche-Genetech, Eli Lilly, Novo Nordisk, Pfizer, and Signal Path; and is an employee of Verana Health. S.D.A. declares grants or personal fees from Vifor Int, Bayer, Boehringer Ingelheim, Servier, Abbott Vascular, Cardiac Dimensions, Actimed, Astra Zeneca, Amgen, Bioventrix, Janssen, Respicardia, V-Wave, Brahms, Cordio, and Occlutech, outside the submitted work. S.K.J. has received institutional research/grant support from AstraZeneca, Bayer, Janssen, and Novartis. J.B. has served as a consultant to Abbott, Adrenomed, Arena Pharma, Array, Amgen, Applied Therapeutics, Astra Zeneca, Bayer, Boehringer Ingelheim, Cardior, CVRx, Eli Lilly, G3 Pharma, Imbria, Impulse Dynamics, Innolife, Janssen, LivaNova, Luitpold, Medtronic, Merck, Novartis, Novo Nordisk, Sequana Medical, V-Wave Limited, and Vifor. D.K.M. reports honoraria for clinical trial leadership from Ventyx, Boehringer Ingelheim, Sanofi, Merck & Co, Pfizer, Akebia, AstraZeneca, Novo Nordisk, Esperion, AbbVie, Lilly USA, Eidos, Arena, Dynavax, Lexicon, Otsuka, and CSL Behring, and honoraria for consultancy from Lilly USA, Boehringer Ingelheim, GlaxoSmithKline, Pfizer, Altimmune, Intercept, Merck & Co, Novo Nordisk, Applied Therapeutics, Metavant, Sanofi, Afimmune, CSL Behring, and Bayer. All other authors report no conflicts.

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://dbpia.nl.go.kr/pages/standard-publication-reuse-rights)

Supplementary data