https://orcid.org/0000-0002-1627-5047
Abstract
The performance of breast cancer risk models for women with a family history but negative BRCA1 and/or BRCA2 mutation test results is uncertain. We calculated the cumulative 10-year invasive breast cancer risk at cohort entry for 14 657 unaffected women (96.1% had an affected relative) not known to carry BRCA1 or BRCA2 mutations at baseline using three pedigree-based models (Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm, BRCAPRO, and International Breast Cancer Intervention Study). During follow-up, 482 women were diagnosed with invasive breast cancer. Mutation testing was conducted independent of incident cancers. All models underpredicted risk by 26.3%–56.7% for women who tested negative but whose relatives had not been tested (n = 1363; 63 breast cancers). Although replication studies with larger sample sizes are needed, until these models are recalibrated for women who test negative and have no relatives tested, caution should be used when considering changing the breast cancer risk management intensity of such women based on risk estimates from these models.
Breast cancer risk models are used to guide decisions about screening, risk-reducing mastectomy, and chemoprevention. Three commonly used models, Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA), BRCAPRO, and International Breast Cancer Intervention Study (IBIS), incorporate BRCA1 and BRCA2 mutation test results into risk estimation (1–3). As mutation testing becomes more affordable, this information will be available for a greater proportion of the population. Although these risk models have been externally validated (4), including from the Prospective Family Cohort Study (ProF-SC) (5), it is uncertain how well they perform specifically for women with negative mutation test results.
ProF-SC comprises baseline and follow-up data from the Breast Cancer Family Registry Cohort, formed by a collaboration between six centers in the United States, Canada, and Australia and the Kathleen Cuningham Foundation Consortium for Research into Familial Breast Cancer (kConFab) (6). The vast majority of women in the Prospective Family Study Cohort have a family history of breast cancer. All participants provided written informed consent before enrollment, and the study protocols were approved by institutional review boards (6–9).
We collected breast cancer risk factor data using identical baseline questionnaires (6), which captured demographic characteristics, height and weight, history of benign breast disease, breast surgeries, reproductive history, lifestyle factors, and family history of breast and other cancers including age at diagnosis and vital status with date or age of death where applicable across multiple generations. We verified information on 81% of self- and relative-reported incident breast cancers through pathology reports, cancer registries, medical records, death certificates, or pathologist review of tissue samples (6–8). Screening for germline mutations in BRCA1 and BRCA2 was conducted independently of incident cancer diagnoses and is described in the Supplementary Methods (available online) (7,10,11).
We applied the following mutually exclusive exclusions to the 18 856 women who were unaffected by breast cancer at baseline: history of bilateral prophylactic mastectomy (n = 113), less than 2 months of follow-up (n = 517), personal history of ovarian cancer (n = 316), age younger than 20 years or older than 70 years at baseline (n = 2082), insufficient family history information available (n = 96), or were known BRCA1- or BRCA2-mutation carriers (n = 1075), leaving 14 657 women. We limited eligibility to age 70 years or younger at baseline because BOADICEA does not calculate risk beyond age 80 years.
Each participant’s 10-year risk of invasive breast cancer was calculated with BOADICEA v3 (https://pluto.srl.cam.ac.uk/cgi-bin/bd3/v3/bd.cgi) (1), BRCAPRO v2.1–3 (https://projects.iq.harvard.edu/bayesmendel/brcapro) (2), and IBIS v8b (http://www.ems-trials.org/riskevaluator/) (3) using baseline pedigree and risk factor information. Country- and age-specific incidences (Australia, Canada, and US white [Hispanics and non-Hispanics combined]) were used for BOADICEA unless otherwise specified, whereas US and UK incidences were used for the BRCAPRO and IBIS models, respectively. The default sensitivities of testing for BRCA1 and BRCA2 mutations were used unless otherwise specified.
Time at risk of breast cancer started at 2 months after the baseline questionnaire and continued to the first of 10 years and 2 months later, date last known to be undiagnosed with breast cancer, date of diagnosis of invasive or in situ breast cancer, date of bilateral mastectomy, or date of 80th birthday. Deaths from nonbreast cancer causes were considered as competing risks (not censored), but this assumption did not materially affect risk estimates see (5).
We evaluated model calibration by comparing the expected number of incident invasive breast cancers (E) with the observed number (O). If follow-up time was less than 10 years, we used the Amir method to adjust the predicted risk to the woman’s follow-up time (12). The 95% confidence interval (CI) for the ratio of expected to observed numbers was (E/O)*exp[±1.96*(1/O)1/2]. We evaluated discrimination using the C-statistic, accounting for incomplete follow-up (13).
During a median follow-up of 10 years (interquartile range = 6.1–10.0 years) of 14 657 women (96.1% had an affected relative) who were not known to carry BRCA1 and BRCA2 mutations (119 439 total person-years), 482 women developed an incident invasive breast cancer (4.0 per 1000 women per year; see Supplementary Table 1 [available online] for additional details).
All models underpredicted risk by 26.3–56.7% for women who had been tested and found to be negative but whose relatives had not been tested (n = 1363; 63 breast cancers) (Table 1; Figure 1). BOADICEA and IBIS overpredicted risk by up to 4.03-fold (95% CI = 2.29- to 7.09-fold) for women who were tested for the family-specific BRCA1 or BRCA2 mutation but found not to carry that mutation, even after setting the mutation sensitivities to 1.0 for BOADICEA (Supplementary Table 2, available online). BOADICEA and IBIS were well-calibrated for untested women (Table 1).
Calibration and discrimination of Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA), BRCAPRO, and International Breast Cancer Intervention Study (IBIS) models by BRCA1- and BRCA2-mutation carrier status. *Tested for mutations in BRCA1 and/or BRCA2. †First- or second-degree relatives of the woman being tested.
Calibration and discrimination of BOADICEA, BRCAPRO, and IBIS models by BRCA1- and BRCA2-mutation carrier status*
| Model | Expected number of breast cancer cases | Observed number of breast cancer cases | Ratio of expected to observed cases (95% CI) | C-statistic (95% CI) |
|---|---|---|---|---|
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 47.9 | 12 | 4.03 (2.29 to 7.09) | 0.74 (0.63 to 0.85) |
| BRCAPRO | 17.2 | 12 | 1.46 (0.83 to 2.57) | 0.74 (0.59 to 0.88) |
| IBIS | 23.8 | 12 | 2.02 (1.15 to 3.55) | 0.72 (0.59 to 0.86) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 59.5 | 72 | 0.82 (0.65 to 1.04) | 0.65 (0.60 to 0.71) |
| BRCAPRO | 30.6 | 72 | 0.42 (0.34 to 0.53) | 0.63 (0.57 to 0.69) |
| IBIS | 60.9 | 72 | 0.84 (0.67 to 1.06) | 0.69 (0.64 to 0.75) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 46.4 | 63 | 0.73 (0.57 to 0.94) | 0.53 (0.46 to 0.59) |
| BRCAPRO | 27.4 | 63 | 0.43 (0.34 to 0.55) | 0.53 (0.46 to 0.60) |
| IBIS | 46.6 | 63 | 0.74 (0.58 to 0.94) | 0.53 (0.46 to 0.60) |
| Woman not tested, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 13.2 | 13 | 1.01 (0.59 to 1.74) | 0.58 (0.43 to 0.72) |
| BRCAPRO | 9.0 | 13 | 0.69 (0.40 to 1.20) | 0.57 (0.41 to 0.73) |
| IBIS | 13.9 | 13 | 1.07 (0.62 to 1.84) | 0.56 (0.40 to 0.73) |
| Woman not tested, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 190.0 | 189 | 1.01 (0.87 to 1.16) | 0.67 (0.64 to 0.71) |
| BRCAPRO | 91.8 | 189 | 0.49 (0.42 to 0.56) | 0.63 (0.59 to 0.67) |
| IBIS | 192.3 | 189 | 1.02 (0.88 to 1.17) | 0.68 (0.65 to 0.72) |
| Woman not tested, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 135.3 | 133 | 1.02 (0.86 to 1.21) | 0.67 (0.63 to 0.71) |
| BRCAPRO | 81.5 | 133 | 0.61 (0.52 to 0.73) | 0.62 (0.57 to 0.66) |
| IBIS | 146.8 | 133 | 1.10 (0.93 to 1.31) | 0.65 (0.60 to 0.69) |
| Model | Expected number of breast cancer cases | Observed number of breast cancer cases | Ratio of expected to observed cases (95% CI) | C-statistic (95% CI) |
|---|---|---|---|---|
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 47.9 | 12 | 4.03 (2.29 to 7.09) | 0.74 (0.63 to 0.85) |
| BRCAPRO | 17.2 | 12 | 1.46 (0.83 to 2.57) | 0.74 (0.59 to 0.88) |
| IBIS | 23.8 | 12 | 2.02 (1.15 to 3.55) | 0.72 (0.59 to 0.86) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 59.5 | 72 | 0.82 (0.65 to 1.04) | 0.65 (0.60 to 0.71) |
| BRCAPRO | 30.6 | 72 | 0.42 (0.34 to 0.53) | 0.63 (0.57 to 0.69) |
| IBIS | 60.9 | 72 | 0.84 (0.67 to 1.06) | 0.69 (0.64 to 0.75) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 46.4 | 63 | 0.73 (0.57 to 0.94) | 0.53 (0.46 to 0.59) |
| BRCAPRO | 27.4 | 63 | 0.43 (0.34 to 0.55) | 0.53 (0.46 to 0.60) |
| IBIS | 46.6 | 63 | 0.74 (0.58 to 0.94) | 0.53 (0.46 to 0.60) |
| Woman not tested, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 13.2 | 13 | 1.01 (0.59 to 1.74) | 0.58 (0.43 to 0.72) |
| BRCAPRO | 9.0 | 13 | 0.69 (0.40 to 1.20) | 0.57 (0.41 to 0.73) |
| IBIS | 13.9 | 13 | 1.07 (0.62 to 1.84) | 0.56 (0.40 to 0.73) |
| Woman not tested, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 190.0 | 189 | 1.01 (0.87 to 1.16) | 0.67 (0.64 to 0.71) |
| BRCAPRO | 91.8 | 189 | 0.49 (0.42 to 0.56) | 0.63 (0.59 to 0.67) |
| IBIS | 192.3 | 189 | 1.02 (0.88 to 1.17) | 0.68 (0.65 to 0.72) |
| Woman not tested, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 135.3 | 133 | 1.02 (0.86 to 1.21) | 0.67 (0.63 to 0.71) |
| BRCAPRO | 81.5 | 133 | 0.61 (0.52 to 0.73) | 0.62 (0.57 to 0.66) |
| IBIS | 146.8 | 133 | 1.10 (0.93 to 1.31) | 0.65 (0.60 to 0.69) |
BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model; CI = confidence interval; IBIS = International Breast Cancer Intervention Study model.
Calibration and discrimination of BOADICEA, BRCAPRO, and IBIS models by BRCA1- and BRCA2-mutation carrier status*
| Model | Expected number of breast cancer cases | Observed number of breast cancer cases | Ratio of expected to observed cases (95% CI) | C-statistic (95% CI) |
|---|---|---|---|---|
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 47.9 | 12 | 4.03 (2.29 to 7.09) | 0.74 (0.63 to 0.85) |
| BRCAPRO | 17.2 | 12 | 1.46 (0.83 to 2.57) | 0.74 (0.59 to 0.88) |
| IBIS | 23.8 | 12 | 2.02 (1.15 to 3.55) | 0.72 (0.59 to 0.86) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 59.5 | 72 | 0.82 (0.65 to 1.04) | 0.65 (0.60 to 0.71) |
| BRCAPRO | 30.6 | 72 | 0.42 (0.34 to 0.53) | 0.63 (0.57 to 0.69) |
| IBIS | 60.9 | 72 | 0.84 (0.67 to 1.06) | 0.69 (0.64 to 0.75) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 46.4 | 63 | 0.73 (0.57 to 0.94) | 0.53 (0.46 to 0.59) |
| BRCAPRO | 27.4 | 63 | 0.43 (0.34 to 0.55) | 0.53 (0.46 to 0.60) |
| IBIS | 46.6 | 63 | 0.74 (0.58 to 0.94) | 0.53 (0.46 to 0.60) |
| Woman not tested, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 13.2 | 13 | 1.01 (0.59 to 1.74) | 0.58 (0.43 to 0.72) |
| BRCAPRO | 9.0 | 13 | 0.69 (0.40 to 1.20) | 0.57 (0.41 to 0.73) |
| IBIS | 13.9 | 13 | 1.07 (0.62 to 1.84) | 0.56 (0.40 to 0.73) |
| Woman not tested, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 190.0 | 189 | 1.01 (0.87 to 1.16) | 0.67 (0.64 to 0.71) |
| BRCAPRO | 91.8 | 189 | 0.49 (0.42 to 0.56) | 0.63 (0.59 to 0.67) |
| IBIS | 192.3 | 189 | 1.02 (0.88 to 1.17) | 0.68 (0.65 to 0.72) |
| Woman not tested, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 135.3 | 133 | 1.02 (0.86 to 1.21) | 0.67 (0.63 to 0.71) |
| BRCAPRO | 81.5 | 133 | 0.61 (0.52 to 0.73) | 0.62 (0.57 to 0.66) |
| IBIS | 146.8 | 133 | 1.10 (0.93 to 1.31) | 0.65 (0.60 to 0.69) |
| Model | Expected number of breast cancer cases | Observed number of breast cancer cases | Ratio of expected to observed cases (95% CI) | C-statistic (95% CI) |
|---|---|---|---|---|
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 47.9 | 12 | 4.03 (2.29 to 7.09) | 0.74 (0.63 to 0.85) |
| BRCAPRO | 17.2 | 12 | 1.46 (0.83 to 2.57) | 0.74 (0.59 to 0.88) |
| IBIS | 23.8 | 12 | 2.02 (1.15 to 3.55) | 0.72 (0.59 to 0.86) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 59.5 | 72 | 0.82 (0.65 to 1.04) | 0.65 (0.60 to 0.71) |
| BRCAPRO | 30.6 | 72 | 0.42 (0.34 to 0.53) | 0.63 (0.57 to 0.69) |
| IBIS | 60.9 | 72 | 0.84 (0.67 to 1.06) | 0.69 (0.64 to 0.75) |
| Woman tested negative for mutations in BRCA1 and/or BRCA2, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 46.4 | 63 | 0.73 (0.57 to 0.94) | 0.53 (0.46 to 0.59) |
| BRCAPRO | 27.4 | 63 | 0.43 (0.34 to 0.55) | 0.53 (0.46 to 0.60) |
| IBIS | 46.6 | 63 | 0.74 (0.58 to 0.94) | 0.53 (0.46 to 0.60) |
| Woman not tested, ≥1 first- or second-degree relatives tested positive for mutations in BRCA1 or BRCA2 | ||||
| BOADICEA | 13.2 | 13 | 1.01 (0.59 to 1.74) | 0.58 (0.43 to 0.72) |
| BRCAPRO | 9.0 | 13 | 0.69 (0.40 to 1.20) | 0.57 (0.41 to 0.73) |
| IBIS | 13.9 | 13 | 1.07 (0.62 to 1.84) | 0.56 (0.40 to 0.73) |
| Woman not tested, ≥1 first- or second-degree relatives tested negative for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 190.0 | 189 | 1.01 (0.87 to 1.16) | 0.67 (0.64 to 0.71) |
| BRCAPRO | 91.8 | 189 | 0.49 (0.42 to 0.56) | 0.63 (0.59 to 0.67) |
| IBIS | 192.3 | 189 | 1.02 (0.88 to 1.17) | 0.68 (0.65 to 0.72) |
| Woman not tested, no first- or second-degree relatives tested for mutations in BRCA1 and BRCA2 | ||||
| BOADICEA | 135.3 | 133 | 1.02 (0.86 to 1.21) | 0.67 (0.63 to 0.71) |
| BRCAPRO | 81.5 | 133 | 0.61 (0.52 to 0.73) | 0.62 (0.57 to 0.66) |
| IBIS | 146.8 | 133 | 1.10 (0.93 to 1.31) | 0.65 (0.60 to 0.69) |
BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model; CI = confidence interval; IBIS = International Breast Cancer Intervention Study model.
The ability of the models to distinguish between breast cancer cases and noncases at the individual level was very low for women who tested negative and whose relatives were untested (C-statistics were 0.53 [95% CI = 0.46 to 0.59] for BOADICEA, 0.53 [95% CI = 0.46 to 0.60] for BRCAPRO and IBIS [Table 1]). Overall results were similar when using UK incidence rates for BOADICEA (Supplementary Table 2, available online), women with at least one affected first-degree relative (Supplementary Table 3, available online), or women who tested negative for mutations both in BRCA1 and BRCA2 (Supplementary Table 4, available online).
Therefore, although these models may be well calibrated overall (5), they generally performed inadequately for women who tested negative for BRCA1 and/or BRCA2 mutations. Possible reasons are that all models account for familial aggregation only in terms of genetic factors, BRCAPRO assumes these are only BRCA1 and BRCA2, and IBIS uses an optimistic mutation sensitivity of 1.0.
A key strength of validating risk models using a cohort enriched for familial risk is the increased power to assess model performance across the upper range of absolute risk. Our results, though, are not necessarily generalizable to women without a family history, a subpopulation currently rarely tested for mutations (14). Other limitations include the small number of events observed for certain subgroups, incomplete testing information, as well as the lack of information on other risk factors now included in some models such as mammographic density (5), though another study found little change to calibration from including mammographic density data when available (15). In a clinical setting complete data will not always be available; therefore, well-calibrated models should be used when the data are limited.
Although replication studies with larger sample sizes are needed, until these models are recalibrated for women who test negative and have no relatives tested, caution should be used when considering changing the breast cancer risk management intensity of such women based on estimates from these models.
Funding
This work was supported by the US National Institute of Health (grant number RO1CA159868). The Australian Breast Cancer Family Registry (ABCFR) was supported in Australia by the National Health and Medical Research Council, the New South Wales Cancer Council, the Victorian Health Promotion Foundation, the Victorian Breast Cancer Research Consortium, Cancer Australia, and the National Breast Cancer Foundation. The six sites of the BCFR were supported by grant UM1 CA164920 from the US National Cancer Institute. The content of this manuscript does not necessarily reflect the views or policies of the National Cancer Institute or any of the collaborating centers in the BCFR, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government or the BCFR.
This work was supported by grants to kConFab and the kConFab Follow-Up Study from Cancer Australia (grant numbers 809195, 1100868), the Australian National Breast Cancer Foundation (grant number IF 17), the National Health and Medical Research Council (grant numbers 454508, 288704, 145684), the US National Institutes of Health (grant number RO1CA159868), the Queensland Cancer Fund, the Cancer Councils of New South Wales, Victoria, Tasmania and South Australia, and the Cancer Foundation of Western Australia (grant numbers not applicable). KAP is a National Breast Cancer Foundation (Australia) Practitioner Fellow (grant number PRAC-17–004). JLH and MCS are National Health and Medical Research Council Senior Principal and Senior Research Fellows, respectively.
Notes
The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Dr Dite reports grants from Genetic Technologies Pty Ltd outside the submitted work. Dr Friedlander reports personal fees from AstraZeneca, grants from AstraZeneca, and personal fees from MSD outside the submitted work. Dr Phillips has a patent System and Process of Cancer Risk Estimation (Australian Innovation Patent) issued. The other authors declared no conflicts of interest during the conduct of this study outside the grant funding listed in the Funding section.
We thank all the participants in this study, the entire team of BCFR past and current investigators, as well as the kConFab investigators and all the BCFR and kConFab coordinators, research nurses, interviewers and data management staff, and the heads and staff of the participating Family Cancer Clinics. RJM and MBT had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
