Table 2. Open in new tab Systematic...

Validation domain	Description	What has been done
Cross-EHR source concordance	To what extent is the phenotype concordant across EHR sources?	The proportion of HF cases recorded in primary care and hospital care EHR was 27%³¹	The proportion of nonfatal AMI defined across primary care, hospital care, and disease registry was 32%²⁹	The proportion of bleeding events recorded in primary care and hospital care was 12%, with 47% of bleeding events recorded only in primary care and 12% only in hospital care
Case-note review	What is the PPV and the NPV when comparing the algorithm with clinician review of case notes or “gold standard” source of information?		Compared with AMI defined in the disease registry, the PPV of AMI recorded in primary care was 92.2% (95% CI, 91.6%-92.8%) and in hospital admissions was 91.5% (95% CI, 90.8%-92.1%)²⁹	Compared through independent review by 2 clinicians, the PPV of bleeding events identified through the phenotyping algorithm was 0.88
Etiology	Are the prospective associations with risk actors consistent with previous evidence?	Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴^,^AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ sex⁹²	Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴ AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ influenza infection,⁹³ ischemic presentations,⁹⁴ sex⁹²	At 5 y, 29.1% (95% CI, 28.2%-29.9%) of atrial fibrillation patients, 21.9% (95% CI, 21.2%-22.5%) of myocardial infarction patients, 25.3% (95% CI, 24.2%-26.3%) of unstable angina patients and 23.4% (95% CI, 23.0%-23.8%) of stable angina had bleeding of any kind
Prognosis	Are the risks of subsequent events plausible?	Corrected for age and sex, HF was strongly associated with mortality, with HRs for all‐cause mortality ranging from 7.01 (95% CI, 6.83-7.20) to 7.23 (95% CI, 7.03-7.43), and up to 15.38 (95% CI, 15.02-15.83) for patients in primary care with acute HF hospitalization, primary care only, and patients hospitalized but no primary care record³¹	Patients with myocardial infarction identified in the disease registry had lower crude 30-d mortality (10.8%; 95% CI, 10.2%-11.4%) than did those identified in hospital care (13.9%; 95% CI, 13.3%-14.4%) or in primary care (14.9%; 95% CI, 14.4%-15.5%) (Figure 3⇓). At 1 year, however, mortality was similar in all 3 groups, at around 20%²⁹ Of the 24 479 patients with AMI, 5775 (23.6%) developed HF during a median follow-up of 3.7 years (incidence rate per 1000 person-years, 63.8; 95% CI, 62.2-65.5)⁹⁵	The HR for all-cause mortality was 1.98 (95% CI, 1.86-2.11) for primary care bleeding with markers of severity, and 1.99 (95% CI, 1.92-2.05) for hospitalized bleeding without markers of severity, compared with patients with no bleeding
Genetic associations	Are the observed genetic associations plausible and concordance with previous evidence?		Consistent direction and magnitude of associations were replicated in 67 (97%) of previously reported genetic variants⁴
External populations	Has the algorithm been tested (in any of the previous validation domains) in different countries?		We observed high 3-y crude cumulative risks of all-cause death (from 19.6% [England] to 30.2% [United States]); the composite of AMI, stroke, or death (from 26.0% [France] to 36.2% [United States]); and hospitalized bleeding (from 3.1% [France] to 5.3% [United States])⁶⁴

Validation domain

Description

What has been done

AMI

Bleeding

Cross-EHR source concordance

To what extent is the phenotype concordant across EHR sources?

The proportion of HF cases recorded in primary care and hospital care EHR was 27%³¹

The proportion of nonfatal AMI defined across primary care, hospital care, and disease registry was 32%²⁹

The proportion of bleeding events recorded in primary care and hospital care was 12%, with 47% of bleeding events recorded only in primary care and 12% only in hospital care

Case-note review

What is the PPV and the NPV when comparing the algorithm with clinician review of case notes or “gold standard” source of information?

Compared with AMI defined in the disease registry, the PPV of AMI recorded in primary care was 92.2% (95% CI, 91.6%-92.8%) and in hospital admissions was 91.5% (95% CI, 90.8%-92.1%)²⁹

Compared through independent review by 2 clinicians, the PPV of bleeding events identified through the phenotyping algorithm was 0.88

Etiology

Are the prospective associations with risk actors consistent with previous evidence?

Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴^,^AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ sex⁹²

Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴ AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ influenza infection,⁹³ ischemic presentations,⁹⁴ sex⁹²

At 5 y, 29.1% (95% CI, 28.2%-29.9%) of atrial fibrillation patients, 21.9% (95% CI, 21.2%-22.5%) of myocardial infarction patients, 25.3% (95% CI, 24.2%-26.3%) of unstable angina patients and 23.4% (95% CI, 23.0%-23.8%) of stable angina had bleeding of any kind

Prognosis

Are the risks of subsequent events plausible?

Corrected for age and sex, HF was strongly associated with mortality, with HRs for all‐cause mortality ranging from 7.01 (95% CI, 6.83-7.20) to 7.23 (95% CI, 7.03-7.43), and up to 15.38 (95% CI, 15.02-15.83) for patients in primary care with acute HF hospitalization, primary care only, and patients hospitalized but no primary care record³¹

Patients with myocardial infarction identified in the disease registry had lower crude 30-d mortality (10.8%; 95% CI, 10.2%-11.4%) than did those identified in hospital care (13.9%; 95% CI, 13.3%-14.4%) or in primary care (14.9%; 95% CI, 14.4%-15.5%) (Figure 3⇓). At 1 year, however, mortality was similar in all 3 groups, at around 20%²⁹
Of the 24 479 patients with AMI, 5775 (23.6%) developed HF during a median follow-up of 3.7 years (incidence rate per 1000 person-years, 63.8; 95% CI, 62.2-65.5)⁹⁵

The HR for all-cause mortality was 1.98 (95% CI, 1.86-2.11) for primary care bleeding with markers of severity, and 1.99 (95% CI, 1.92-2.05) for hospitalized bleeding without markers of severity, compared with patients with no bleeding

Genetic associations

Are the observed genetic associations plausible and concordance with previous evidence?

Consistent direction and magnitude of associations were replicated in 67 (97%) of previously reported genetic variants⁴

External populations

Has the algorithm been tested (in any of the previous validation domains) in different countries?

We observed high 3-y crude cumulative risks of all-cause death (from 19.6% [England] to 30.2% [United States]); the composite of AMI, stroke, or death (from 26.0% [France] to 36.2% [United States]); and hospitalized bleeding (from 3.1% [France] to 5.3% [United States])⁶⁴

Validation domain	Description	What has been done
Cross-EHR source concordance	To what extent is the phenotype concordant across EHR sources?	The proportion of HF cases recorded in primary care and hospital care EHR was 27%³¹	The proportion of nonfatal AMI defined across primary care, hospital care, and disease registry was 32%²⁹	The proportion of bleeding events recorded in primary care and hospital care was 12%, with 47% of bleeding events recorded only in primary care and 12% only in hospital care
Case-note review	What is the PPV and the NPV when comparing the algorithm with clinician review of case notes or “gold standard” source of information?		Compared with AMI defined in the disease registry, the PPV of AMI recorded in primary care was 92.2% (95% CI, 91.6%-92.8%) and in hospital admissions was 91.5% (95% CI, 90.8%-92.1%)²⁹	Compared through independent review by 2 clinicians, the PPV of bleeding events identified through the phenotyping algorithm was 0.88
Etiology	Are the prospective associations with risk actors consistent with previous evidence?	Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴^,^AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ sex⁹²	Type 2 diabetes,⁸⁴ systolic/diastolic blood pressure,³² heart rate,⁸⁵ socioeconomic deprivation,⁸⁶ alcohol consumption,⁸⁷ smoking,⁸⁸ ethnicity,⁴⁴ AMI,²⁹ depression,⁸⁹ neutrophil counts,⁹⁰ eosinophil/lymphocyte counts,⁹¹ atrial fibrillation,³⁰ influenza infection,⁹³ ischemic presentations,⁹⁴ sex⁹²	At 5 y, 29.1% (95% CI, 28.2%-29.9%) of atrial fibrillation patients, 21.9% (95% CI, 21.2%-22.5%) of myocardial infarction patients, 25.3% (95% CI, 24.2%-26.3%) of unstable angina patients and 23.4% (95% CI, 23.0%-23.8%) of stable angina had bleeding of any kind
Prognosis	Are the risks of subsequent events plausible?	Corrected for age and sex, HF was strongly associated with mortality, with HRs for all‐cause mortality ranging from 7.01 (95% CI, 6.83-7.20) to 7.23 (95% CI, 7.03-7.43), and up to 15.38 (95% CI, 15.02-15.83) for patients in primary care with acute HF hospitalization, primary care only, and patients hospitalized but no primary care record³¹	Patients with myocardial infarction identified in the disease registry had lower crude 30-d mortality (10.8%; 95% CI, 10.2%-11.4%) than did those identified in hospital care (13.9%; 95% CI, 13.3%-14.4%) or in primary care (14.9%; 95% CI, 14.4%-15.5%) (Figure 3⇓). At 1 year, however, mortality was similar in all 3 groups, at around 20%²⁹ Of the 24 479 patients with AMI, 5775 (23.6%) developed HF during a median follow-up of 3.7 years (incidence rate per 1000 person-years, 63.8; 95% CI, 62.2-65.5)⁹⁵	The HR for all-cause mortality was 1.98 (95% CI, 1.86-2.11) for primary care bleeding with markers of severity, and 1.99 (95% CI, 1.92-2.05) for hospitalized bleeding without markers of severity, compared with patients with no bleeding
Genetic associations	Are the observed genetic associations plausible and concordance with previous evidence?		Consistent direction and magnitude of associations were replicated in 67 (97%) of previously reported genetic variants⁴
External populations	Has the algorithm been tested (in any of the previous validation domains) in different countries?		We observed high 3-y crude cumulative risks of all-cause death (from 19.6% [England] to 30.2% [United States]); the composite of AMI, stroke, or death (from 26.0% [France] to 36.2% [United States]); and hospitalized bleeding (from 3.1% [France] to 5.3% [United States])⁶⁴