Abstract

Context

The incidence of pancreatic neuroendocrine tumors (PNETs) is increasing. Current staging systems include nodal positivity, but the association of lymph node status and worse survival is controversial.

Objective

The study aim was to determine the prognostic significance of lymph node ratio (LNR) and compare it with nodal positivity for PNET.

Design, Setting, Participants, and Intervention

A retrospective analysis of the Surveillance, Epidemiology, and End Results database between 2004 and 2011 was performed in patients who underwent a pancreatectomy with lymphadenectomy. The primary outcome was disease-specific survival (DSS).

Results

Of the 896 patients analyzed, T stage, N stage, distant metastasis, grade, extent of resection, sex, and age ≥57 years were significantly associated with worse DSS on univariate analysis. On multivariate analysis, age ≥57 [hazard ratio (HR) 1.75, 95% confidence interval (CI), 1.12 to 2.74, P = 0.015], male sex (HR 1.58; 95% CI, 1.01 to 2.48; P = 0.046), grade (poorly differentiated/undifferentiated: HR 7.59; 95% CI, 4.71 to 12.23; P < 0.001), distant metastases (HR 2.45; 95% CI, 1.58 to 3.79; P < 0.001), and partial pancreatectomy (HR 2.55; 95% CI, 1.2 to 5.4; P = 0.015) were associated with worse DSS. Comparison between staging models constructed based on LNR cutoffs and the American Joint Committee on Cancer (AJCC) eighth edition staging system revealed that a model based on LNR ≥0.5 was superior.

Conclusions

LNR ≥0.5 is independently associated with worse DSS. A staging system with LNR ≥0.5 was superior to the current AJCC eighth edition staging system.

Pancreatic neuroendocrine tumors (PNETs) are rare tumors, with an incidence of 5.25 per 100,000 (1), that account for ~3% of all pancreatic neoplasms (2). The prognosis is superior to that of pancreatic adenocarcinomas, with a reported 5-year survival of 59.3% and 10-year survival of 37.7% (3). The natural history of PNETs is highly variable, with some tumors showing indolent behavior but others displaying an aggressive course, with local invasion and distant metastasis (2, 4). Higher tumor grade, distant metastasis, and older age have been consistently found to be the most important prognostic factors after PNET resection (3).

Several staging systems have been developed to better stratify prognosis in patients with PNET. The 2010 World Health Organization staging system incorporates mitotic count and Ki-67 index to separate patients into three categories (5). The eighth edition of the American Joint Committee on Cancer (AJCC) tumor–node–metastasis staging system includes primary tumor size, presence of lymph node metastasis on pathologic examination, and presence of distant metastases (6). Lymph node status is particularly important in PNET staging systems, because the presence of lymph node metastasis would render a patient stage III according to both the AJCC and European Neuroendocrine Tumor Society staging systems (6, 7). Optimal risk stratification is important because of the ever-expanding surgical and nonsurgical treatment options for patients with PNET, including somatostatin analogs (8), targeted therapies (everolimus, sunitinib) (9, 10), chemotherapy (streptozocin, 5-fluorouracil, temozolomide, capecitabine) (11), and peptide receptor radionuclide therapy (12).

The prognostic significance of lymph node metastasis in PNET is controversial. Several investigators have reported a significant association between lymph node metastasis and survival (13–15), whereas others have not found a significant association (3, 16). Lymph node ratio (LNR) is calculated based on the number of positive lymph nodes compared with the number of lymph nodes examined and has emerged as a prognostic marker possibly superior to simply assessing the presence or absence of lymph node metastasis. LNR has been evaluated in several malignancies, including pancreatic adenocarcinoma, where large studies have found it to be the most important prognostic factor among all variables considered (17, 18). Two retrospective small studies have assessed the prognostic significance of LNR in patients with PNET by using different cutoffs, with disease-free survival as the primary outcome measure (19, 20). The significance of LNR and disease-specific survival (DSS) has not been reported in patients with PNET. Furthermore, an optimal LNR cutoff in a large cohort of patients with PNET has not been identified. Optimal staging of patients with PNET is necessary to best determine which management strategies should be used both in the adjuvant setting and in those without surgically resectable disease. The purpose of this study was to determine the prognostic significance of LNR in patients with PNET and compare it with nodal positivity for PNET.

Methods

The National Cancer Institute’s Surveillance, Epidemiology, and End Results (SEER) database was used in this study. Individual cases were retrieved with the SEER*Stat software (version 8.1.5, 31 March 2014; Cancer Statistics Branch, NCI, Bethesda, MD). Because of the SEER database’s inclusion of unidentifiable patient information, this study was exempted for approval by the Office of Human Subjects Research of the National Institutes of Health.

Inclusion and exclusion criteria

Patients who received a diagnosis and underwent surgery with resection of lymph nodes for PNET between 2004 and 2011 were identified. Cases with unavailable information about the number of positive lymph nodes on examination or DSS were excluded. The following histologic subtypes were included in the analysis, based on their International Classification of Diseases for Oncology, Third Edition codes: islet cell carcinoma (8150), malignant insulinoma (8151), malignant glucagonoma (8152), malignant gastrinoma (8153), VIPoma (8155), malignant somatostatinoma (8156), malignant carcinoid (8240), goblet cell carcinoid (8243), and neuroendocrine carcinoma (8246). The following variables were included in the analysis: age at diagnosis, race, sex, year of diagnosis, primary tumor location, tumor grade, tumor functionality, AJCC T stage, AJCC N stage, number of positive lymph nodes, number of lymph nodes examined, presence of distant metastasis, type of surgery performed, and DSS. AJCC T stage was coded based on the eighth edition of the AJCC staging system based on the variables collaborative stage tumor size (2004+) and collaborative stage extension (2004+) (6). The T component of the European Neuroendocrine Tumor Society staging system was not tested separately, because it is identical to that of the eighth edition of the AJCC staging system (6, 7).

Statistical analysis

Survival analysis was performed with DSS as the primary outcome measure. To identify differences in survival between groups, the Kaplan-Meier method and the log-rank test were used for univariate analysis. A stepwise algorithm was used to identify optimal cutoffs for LNR, adequate lymphadenectomy, and lymph node positivity. For multivariate analysis, the backward selection method of Cox proportional regression analysis was used to identify independent prognostic factors, including only the variables that were significant on univariate analysis (P < 0.05). Cases with missing data were excluded from multivariate analysis.

The cutoffs that were independent predictors of worse survival were then used to construct staging models for comparison. Each model consisted of all the factors used in PNET staging, namely tumor grade, AJCC T stage, AJCC M stage, and either LNR or AJCC N stage for the nodal component of the staging system. Comparison of these models was subsequently performed with the following statistical methods that are used for staging model comparison: Akaike information criterion [AIC = −2 log maximum likelihood + 2 × degrees of freedom (df)], (21), Bayesian information criterion (BIC = −2 log maximum likelihood + log(sample size) × df) (22), corrected Akaike information criterion (AICc = −2 log maximum likelihood + [2 * n * (df + 1)/(n − df − 2)]) (23), and Harrell’s C-statistic (24). Models with lower AIC, BIC, and AICc values are considered superior, whereas models with higher C-statistics are considered superior.

Statistical tests used two-tailed P values, and P values of <0.05 were considered significant. All statistical analyses were performed in SPSS (version 21; IBM, Armonk, NY).

Results

Prognostic factors

Overall, 896 patients were identified, of whom 459 (51.2%) were men and 437 (48.8%) were women. Median age at diagnosis was 57 years. Most tumors were in the pancreatic tail (321, 35.8%), followed by the pancreatic head (288, 32.1%). Most tumors were well differentiated (WD) on histologic examination (55.7%). Most tumors were also nonfunctioning (96.4%) and nonmetastatic (75%). The median number of lymph nodes examined was 9, and the average number of lymph nodes examined was 11.4 ± 9.8. Detailed clinical, demographic, and pathologic features of the study cohort are summarized in Table 1.

Table 1.

Demographic, Clinical, and Pathologic Characteristics of Study Cohort

VariableNumber of Patients (n = 896), n (%)
Age, median, y57 (range 17–87)
Sex
 Male459 (51.2%)
 Female437 (48.8%)
Race
 White722 (80.6%)
 African American90 (10%)
 Other79 (8.8%)
Tumor location
 Head288 (32.1%)
 Body97 (10.8%)
 Tail321(35.8%)
 Other190 (21.2%)
Extent of resection
 Local excision771 (86%)
 Partial pancreatectomy47 (5.2%)
 Total pancreatectomy78 (8.7%)
Grade
 WD499 (55.7%)
 MD102 (11.4%)
 PD or UD75 (8.4%)
Functionality
 Functioning32 (3.6%)
 Nonfunctioning864 (96.4%)
AJCC T stagea
 T1152 (17%)
 T2184 (20.5%)
 T3382 (42.6%)
 T4155 (17.3%)
AJCC N stage
 N0459 (51.2%)
 N1437 (48.8%)
AJCC M stage
 M0672 (75%)
 M1218 (24.3%)
VariableNumber of Patients (n = 896), n (%)
Age, median, y57 (range 17–87)
Sex
 Male459 (51.2%)
 Female437 (48.8%)
Race
 White722 (80.6%)
 African American90 (10%)
 Other79 (8.8%)
Tumor location
 Head288 (32.1%)
 Body97 (10.8%)
 Tail321(35.8%)
 Other190 (21.2%)
Extent of resection
 Local excision771 (86%)
 Partial pancreatectomy47 (5.2%)
 Total pancreatectomy78 (8.7%)
Grade
 WD499 (55.7%)
 MD102 (11.4%)
 PD or UD75 (8.4%)
Functionality
 Functioning32 (3.6%)
 Nonfunctioning864 (96.4%)
AJCC T stagea
 T1152 (17%)
 T2184 (20.5%)
 T3382 (42.6%)
 T4155 (17.3%)
AJCC N stage
 N0459 (51.2%)
 N1437 (48.8%)
AJCC M stage
 M0672 (75%)
 M1218 (24.3%)
a

Based on the eighth edition of the AJCC staging system for PNETs.

Table 1.

Demographic, Clinical, and Pathologic Characteristics of Study Cohort

VariableNumber of Patients (n = 896), n (%)
Age, median, y57 (range 17–87)
Sex
 Male459 (51.2%)
 Female437 (48.8%)
Race
 White722 (80.6%)
 African American90 (10%)
 Other79 (8.8%)
Tumor location
 Head288 (32.1%)
 Body97 (10.8%)
 Tail321(35.8%)
 Other190 (21.2%)
Extent of resection
 Local excision771 (86%)
 Partial pancreatectomy47 (5.2%)
 Total pancreatectomy78 (8.7%)
Grade
 WD499 (55.7%)
 MD102 (11.4%)
 PD or UD75 (8.4%)
Functionality
 Functioning32 (3.6%)
 Nonfunctioning864 (96.4%)
AJCC T stagea
 T1152 (17%)
 T2184 (20.5%)
 T3382 (42.6%)
 T4155 (17.3%)
AJCC N stage
 N0459 (51.2%)
 N1437 (48.8%)
AJCC M stage
 M0672 (75%)
 M1218 (24.3%)
VariableNumber of Patients (n = 896), n (%)
Age, median, y57 (range 17–87)
Sex
 Male459 (51.2%)
 Female437 (48.8%)
Race
 White722 (80.6%)
 African American90 (10%)
 Other79 (8.8%)
Tumor location
 Head288 (32.1%)
 Body97 (10.8%)
 Tail321(35.8%)
 Other190 (21.2%)
Extent of resection
 Local excision771 (86%)
 Partial pancreatectomy47 (5.2%)
 Total pancreatectomy78 (8.7%)
Grade
 WD499 (55.7%)
 MD102 (11.4%)
 PD or UD75 (8.4%)
Functionality
 Functioning32 (3.6%)
 Nonfunctioning864 (96.4%)
AJCC T stagea
 T1152 (17%)
 T2184 (20.5%)
 T3382 (42.6%)
 T4155 (17.3%)
AJCC N stage
 N0459 (51.2%)
 N1437 (48.8%)
AJCC M stage
 M0672 (75%)
 M1218 (24.3%)
a

Based on the eighth edition of the AJCC staging system for PNETs.

Univariate survival analysis identified age at diagnosis >57 years (75.9 ± 1.9 months vs 81.5 ± 1.7 months, P = 0.02), male sex (76.2 ± 1.9 months vs 81.3 ± 1.7 months, P = 0.035), greater extent of resection (local excision 79.6 ± 1.4 months, partial pancreatectomy 57.7 ± 5.4 months, total pancreatectomy 81.6 ± 3.7 months, P = 0.001), higher tumor grade [WD 84.8 ± 1.7 months, moderately differentiated (MD) 76.4 ± 3.6 months, poorly differentiated (PD) or undifferentiated (UD) 40.3 ± 5.1 months, P < 0.001], higher T stage (T1 86 ± 2.9 months, T2 85.1 ± 2.2 months, T3 77.8 ± 2 months, T4 63.9 ± 3.1 months, P < 0.001), nodal positivity (73.6 ± 2 months vs 84.3 ± 1.6 months, P < 0.001), and presence of distant metastatic disease (64.3 ± 2.9 months vs 84.5 ± 1.3 months, P < 0.001) to be significantly associated with worse DSS (Fig. 1). In contrast, race, tumor location, and tumor functionality were not associated with worse DSS. On multivariate analysis, age at diagnosis >57 years [hazard ratio (HR) 1.75; 95% confidence interval (CI), 1.12 to 2.74; P = 0.015], male sex (HR 1.58; 95% CI, 1.01 to 2.48; P = 0.046), lesser extent of resection (partial pancreatectomy vs local excision: HR 2.55; 95% CI, 1.2 to 5.4; P = 0.015; total pancreatectomy vs local excision: HR 0.72; 95% CI, 0.35 to 1.46; P = 0.36), tumor grade (MD: HR 1.52; 95% CI, 0.81 to 2.84; P = 0.19; PD or UD: HR 7.59; 95% CI, 4.71 to 12.23; P < 0.001), and presence of distant metastasis (HR 2.45; 95% CI, 1.58 to 3.79; P < 0.001) were independently associated with worse DSS (Table 2).

Disease-specific survival curves for (a) age, (b) sex, (c) extent of resection, (d) tumor grade, (e) T stage, (f) N stage, (g) M stage, and (h) LNR ≥0.5.
Figure 1.

Disease-specific survival curves for (a) age, (b) sex, (c) extent of resection, (d) tumor grade, (e) T stage, (f) N stage, (g) M stage, and (h) LNR ≥0.5.

Table 2.

Univariate and Multivariate Analysis for DSS

P on Univariate AnalysisHR (95% CI)P on Multivariate Analysis
Age0.020.015
 <57 y1.00
 ≥57 y1.75 (1.12–2.74)
Sex0.0350.046
 Female1.00
 Male1.58 (1.01–2.48)
Race0.657
Tumor location0.698
Extent of resection0.0010.026
 Local excision1.00
 Partial pancreatectomy2.55 (1.2–5.4)0.015
 Total pancreatectomy0.72 (0.35–1.46)0.36
Grade<0.001<0.001
 WD1.00
 MD1.52 (0.81–2.84)0.19
 PD or UD7.59 (4.71–12.23)<0.001
Functionality0.345
AJCC T stage<0.0010.516
 T11.00
 T21.74 (0.56–5.42)0.338
 T32.06 (0.72–5.86)0.178
 T42.23 (0.74–6.71)0.154
AJCC N stage<0.0010.053
 N01.00
 N11.59 (0.99–2.54)
AJCC M stage<0.001<0.001
 M01.00
 M12.45 (1.58–3.79)
P on Univariate AnalysisHR (95% CI)P on Multivariate Analysis
Age0.020.015
 <57 y1.00
 ≥57 y1.75 (1.12–2.74)
Sex0.0350.046
 Female1.00
 Male1.58 (1.01–2.48)
Race0.657
Tumor location0.698
Extent of resection0.0010.026
 Local excision1.00
 Partial pancreatectomy2.55 (1.2–5.4)0.015
 Total pancreatectomy0.72 (0.35–1.46)0.36
Grade<0.001<0.001
 WD1.00
 MD1.52 (0.81–2.84)0.19
 PD or UD7.59 (4.71–12.23)<0.001
Functionality0.345
AJCC T stage<0.0010.516
 T11.00
 T21.74 (0.56–5.42)0.338
 T32.06 (0.72–5.86)0.178
 T42.23 (0.74–6.71)0.154
AJCC N stage<0.0010.053
 N01.00
 N11.59 (0.99–2.54)
AJCC M stage<0.001<0.001
 M01.00
 M12.45 (1.58–3.79)
Table 2.

Univariate and Multivariate Analysis for DSS

P on Univariate AnalysisHR (95% CI)P on Multivariate Analysis
Age0.020.015
 <57 y1.00
 ≥57 y1.75 (1.12–2.74)
Sex0.0350.046
 Female1.00
 Male1.58 (1.01–2.48)
Race0.657
Tumor location0.698
Extent of resection0.0010.026
 Local excision1.00
 Partial pancreatectomy2.55 (1.2–5.4)0.015
 Total pancreatectomy0.72 (0.35–1.46)0.36
Grade<0.001<0.001
 WD1.00
 MD1.52 (0.81–2.84)0.19
 PD or UD7.59 (4.71–12.23)<0.001
Functionality0.345
AJCC T stage<0.0010.516
 T11.00
 T21.74 (0.56–5.42)0.338
 T32.06 (0.72–5.86)0.178
 T42.23 (0.74–6.71)0.154
AJCC N stage<0.0010.053
 N01.00
 N11.59 (0.99–2.54)
AJCC M stage<0.001<0.001
 M01.00
 M12.45 (1.58–3.79)
P on Univariate AnalysisHR (95% CI)P on Multivariate Analysis
Age0.020.015
 <57 y1.00
 ≥57 y1.75 (1.12–2.74)
Sex0.0350.046
 Female1.00
 Male1.58 (1.01–2.48)
Race0.657
Tumor location0.698
Extent of resection0.0010.026
 Local excision1.00
 Partial pancreatectomy2.55 (1.2–5.4)0.015
 Total pancreatectomy0.72 (0.35–1.46)0.36
Grade<0.001<0.001
 WD1.00
 MD1.52 (0.81–2.84)0.19
 PD or UD7.59 (4.71–12.23)<0.001
Functionality0.345
AJCC T stage<0.0010.516
 T11.00
 T21.74 (0.56–5.42)0.338
 T32.06 (0.72–5.86)0.178
 T42.23 (0.74–6.71)0.154
AJCC N stage<0.0010.053
 N01.00
 N11.59 (0.99–2.54)
AJCC M stage<0.001<0.001
 M01.00
 M12.45 (1.58–3.79)

A stepwise algorithm was used to identify whether a minimum number of lymph nodes examined was associated with improved DSS, but no cutoff was found to be significant on univariate analysis (Table 3). The same stepwise method was then used in subanalyses based on tumor location (head, body, and tail tumors) and extent of resection (local excision and partial or total pancreatectomy) to identify whether a cutoff for minimum number of lymph nodes examined could be identified for these patients, but no significant cutoff was established (Table 4). The identification of potential cutoffs for positive lymph nodes on pathologic examination was performed in a stepwise analysis. A pathological finding of either a single, two, three, four, five, or eight positive nodes was significantly associated with worse DSS. Similarly, a stepwise analysis was performed to identify an optimal cutoff for LNR, which revealed various cutoffs significantly associated with worse DSS (Table 3).

Table 3.

DSS by Lymph Nodes Examined, Number of Positive Lymph Nodes, and LNR, With Different Cutoffs for Each Variable

Number of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Lymph nodes examined
 ≥2826 (92.2%)0.521
 ≥3756 (84.4%)0.924
 ≥4697 (77.8%)0.712
 ≥5605 (67.5%)0.861
 ≥6654 (73%)0.957
 ≥7605 (67.5%)0.861
 ≥8517 (57.7%)0.501
 ≥9475 (53%)0.221
 ≥10439 (49%)0.375
 ≥15264 (29.5%)0.55
Positive lymph nodes
 ≥2297 (33.1%)0.010.551.15 (0.74–1.78)
 ≥3214 (23.9%)<0.0010.1961.34 (0.86–2.1)
 ≥4158 (17.6%)<0.0010.0691.58 (0.97–2.59)
 ≥5115 (12.8%)0.0040.2011.42 (0.83–2.42)
 ≥680 (8.9%)0.06
 ≥762 (6.9%)0.058
 ≥848 (5.4%)0.0030.2951.44 (0.73–2.84)
LNR
 ≥0.05423 (47.2%)<0.0010.0681.53 (0.97–2.43)
 ≥0.07395 (44.1%)<0.0010.0451.58 (1.01–2.48)
 ≥0.1350 (39.1%)<0.0010.0621.52 (0.98–2.36)
 ≥0.2262 (29.2%)<0.0010.0321.62 (1.04–2.5)
 ≥0.3205 (22.9%)<0.0010.0251.68 (1.07–2.63)
 ≥0.4158 (17.6%)<0.0010.1141.48 (0.91–2.42)
 ≥0.5131 (14.6%)<0.0010.0391.71 (1.03–2.84)
 ≥0.694 (10.5%)0.0210.2781.41 (0.76–2.64)
 ≥0.772 (8%)0.0010.1251.66 (0.87–3.16)
 ≥0.863 (7%)0.0190.0781.83 (0.93–3.6)
Number of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Lymph nodes examined
 ≥2826 (92.2%)0.521
 ≥3756 (84.4%)0.924
 ≥4697 (77.8%)0.712
 ≥5605 (67.5%)0.861
 ≥6654 (73%)0.957
 ≥7605 (67.5%)0.861
 ≥8517 (57.7%)0.501
 ≥9475 (53%)0.221
 ≥10439 (49%)0.375
 ≥15264 (29.5%)0.55
Positive lymph nodes
 ≥2297 (33.1%)0.010.551.15 (0.74–1.78)
 ≥3214 (23.9%)<0.0010.1961.34 (0.86–2.1)
 ≥4158 (17.6%)<0.0010.0691.58 (0.97–2.59)
 ≥5115 (12.8%)0.0040.2011.42 (0.83–2.42)
 ≥680 (8.9%)0.06
 ≥762 (6.9%)0.058
 ≥848 (5.4%)0.0030.2951.44 (0.73–2.84)
LNR
 ≥0.05423 (47.2%)<0.0010.0681.53 (0.97–2.43)
 ≥0.07395 (44.1%)<0.0010.0451.58 (1.01–2.48)
 ≥0.1350 (39.1%)<0.0010.0621.52 (0.98–2.36)
 ≥0.2262 (29.2%)<0.0010.0321.62 (1.04–2.5)
 ≥0.3205 (22.9%)<0.0010.0251.68 (1.07–2.63)
 ≥0.4158 (17.6%)<0.0010.1141.48 (0.91–2.42)
 ≥0.5131 (14.6%)<0.0010.0391.71 (1.03–2.84)
 ≥0.694 (10.5%)0.0210.2781.41 (0.76–2.64)
 ≥0.772 (8%)0.0010.1251.66 (0.87–3.16)
 ≥0.863 (7%)0.0190.0781.83 (0.93–3.6)
Table 3.

DSS by Lymph Nodes Examined, Number of Positive Lymph Nodes, and LNR, With Different Cutoffs for Each Variable

Number of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Lymph nodes examined
 ≥2826 (92.2%)0.521
 ≥3756 (84.4%)0.924
 ≥4697 (77.8%)0.712
 ≥5605 (67.5%)0.861
 ≥6654 (73%)0.957
 ≥7605 (67.5%)0.861
 ≥8517 (57.7%)0.501
 ≥9475 (53%)0.221
 ≥10439 (49%)0.375
 ≥15264 (29.5%)0.55
Positive lymph nodes
 ≥2297 (33.1%)0.010.551.15 (0.74–1.78)
 ≥3214 (23.9%)<0.0010.1961.34 (0.86–2.1)
 ≥4158 (17.6%)<0.0010.0691.58 (0.97–2.59)
 ≥5115 (12.8%)0.0040.2011.42 (0.83–2.42)
 ≥680 (8.9%)0.06
 ≥762 (6.9%)0.058
 ≥848 (5.4%)0.0030.2951.44 (0.73–2.84)
LNR
 ≥0.05423 (47.2%)<0.0010.0681.53 (0.97–2.43)
 ≥0.07395 (44.1%)<0.0010.0451.58 (1.01–2.48)
 ≥0.1350 (39.1%)<0.0010.0621.52 (0.98–2.36)
 ≥0.2262 (29.2%)<0.0010.0321.62 (1.04–2.5)
 ≥0.3205 (22.9%)<0.0010.0251.68 (1.07–2.63)
 ≥0.4158 (17.6%)<0.0010.1141.48 (0.91–2.42)
 ≥0.5131 (14.6%)<0.0010.0391.71 (1.03–2.84)
 ≥0.694 (10.5%)0.0210.2781.41 (0.76–2.64)
 ≥0.772 (8%)0.0010.1251.66 (0.87–3.16)
 ≥0.863 (7%)0.0190.0781.83 (0.93–3.6)
Number of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Lymph nodes examined
 ≥2826 (92.2%)0.521
 ≥3756 (84.4%)0.924
 ≥4697 (77.8%)0.712
 ≥5605 (67.5%)0.861
 ≥6654 (73%)0.957
 ≥7605 (67.5%)0.861
 ≥8517 (57.7%)0.501
 ≥9475 (53%)0.221
 ≥10439 (49%)0.375
 ≥15264 (29.5%)0.55
Positive lymph nodes
 ≥2297 (33.1%)0.010.551.15 (0.74–1.78)
 ≥3214 (23.9%)<0.0010.1961.34 (0.86–2.1)
 ≥4158 (17.6%)<0.0010.0691.58 (0.97–2.59)
 ≥5115 (12.8%)0.0040.2011.42 (0.83–2.42)
 ≥680 (8.9%)0.06
 ≥762 (6.9%)0.058
 ≥848 (5.4%)0.0030.2951.44 (0.73–2.84)
LNR
 ≥0.05423 (47.2%)<0.0010.0681.53 (0.97–2.43)
 ≥0.07395 (44.1%)<0.0010.0451.58 (1.01–2.48)
 ≥0.1350 (39.1%)<0.0010.0621.52 (0.98–2.36)
 ≥0.2262 (29.2%)<0.0010.0321.62 (1.04–2.5)
 ≥0.3205 (22.9%)<0.0010.0251.68 (1.07–2.63)
 ≥0.4158 (17.6%)<0.0010.1141.48 (0.91–2.42)
 ≥0.5131 (14.6%)<0.0010.0391.71 (1.03–2.84)
 ≥0.694 (10.5%)0.0210.2781.41 (0.76–2.64)
 ≥0.772 (8%)0.0010.1251.66 (0.87–3.16)
 ≥0.863 (7%)0.0190.0781.83 (0.93–3.6)
Table 4.

DSS by Lymph Nodes Examined for Different Subgroups

Lymph Nodes ExaminedNumber of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Tumor location
 Head (n = 288)
  ≥2270 (93.8%)0.38
  ≥3257 (89.2%)0.771
  ≥4247 (85.8%)0.408
  ≥5240 (83.3%)0.307
  ≥6230 (79.9%)0.461
  ≥7221 (76.7%)0.661
  ≥8211 (73.3%)0.63
  ≥9197 (68.4%)0.842
  ≥10190 (66%)0.728
  ≥15124 (43.1%)0.113
 Body (n = 97)
  ≥287 (89.7%)0.909
  ≥380 (82.5%)0.284
  ≥473 (75.3%)0.359
  ≥568 (70.1%)0.171
  ≥660 (61.9%)0.175
  ≥754 (55.7%)0.249
  ≥851 (52.6%)0.138
  ≥944 (45.4%)0.0410.2192.7 (0.55–13.16)
  ≥1040 (41.2%)0.245
  ≥1530 (30.9%)0.09
 Tail (n = 321)
  ≥2296 (92.2%)0.726
  ≥3264 (82.2%)0.949
  ≥4240 (74.8%)0.874
  ≥5217 (67.6%)0.922
  ≥6199 (62%)0.598
  ≥7180 (56.1%)0.874
  ≥8152 (47.4%)0.323
  ≥9139 (43.3%)0.281
  ≥10122 (38%)0.254
  ≥1556 (17.4%)0.094
Extent of resection
 Local excision (n = 771)
  ≥2713 (92.5%)0.894
  ≥3648 (84%)0.681
  ≥4596 (77.3%)0.88
  ≥5556 (72.1%)0.568
  ≥6513 (66.5%)0.575
  ≥7477 (61.9%)0.558
  ≥8429 (55.6%)0.342
  ≥9394 (51.1%)0.207
  ≥10361 (46.8%)0.248
  ≥15212 (27.5%)0.478
 Partial or total pancreatectomy (n = 125)
  ≥2113 (90.4%)0.216
  ≥3108 (86.4%)0.312
  ≥4101 (80.8%)0.136
  ≥598 (78.4%)0.144
  ≥692 (73.6%)0.285
  ≥789 (71.2%)0.286
  ≥888 (70.4%)0.367
  ≥981 (64.8%)0.846
  ≥1078 (62.4%)0.443
  ≥1552 (41.6%)0.615
Lymph Nodes ExaminedNumber of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Tumor location
 Head (n = 288)
  ≥2270 (93.8%)0.38
  ≥3257 (89.2%)0.771
  ≥4247 (85.8%)0.408
  ≥5240 (83.3%)0.307
  ≥6230 (79.9%)0.461
  ≥7221 (76.7%)0.661
  ≥8211 (73.3%)0.63
  ≥9197 (68.4%)0.842
  ≥10190 (66%)0.728
  ≥15124 (43.1%)0.113
 Body (n = 97)
  ≥287 (89.7%)0.909
  ≥380 (82.5%)0.284
  ≥473 (75.3%)0.359
  ≥568 (70.1%)0.171
  ≥660 (61.9%)0.175
  ≥754 (55.7%)0.249
  ≥851 (52.6%)0.138
  ≥944 (45.4%)0.0410.2192.7 (0.55–13.16)
  ≥1040 (41.2%)0.245
  ≥1530 (30.9%)0.09
 Tail (n = 321)
  ≥2296 (92.2%)0.726
  ≥3264 (82.2%)0.949
  ≥4240 (74.8%)0.874
  ≥5217 (67.6%)0.922
  ≥6199 (62%)0.598
  ≥7180 (56.1%)0.874
  ≥8152 (47.4%)0.323
  ≥9139 (43.3%)0.281
  ≥10122 (38%)0.254
  ≥1556 (17.4%)0.094
Extent of resection
 Local excision (n = 771)
  ≥2713 (92.5%)0.894
  ≥3648 (84%)0.681
  ≥4596 (77.3%)0.88
  ≥5556 (72.1%)0.568
  ≥6513 (66.5%)0.575
  ≥7477 (61.9%)0.558
  ≥8429 (55.6%)0.342
  ≥9394 (51.1%)0.207
  ≥10361 (46.8%)0.248
  ≥15212 (27.5%)0.478
 Partial or total pancreatectomy (n = 125)
  ≥2113 (90.4%)0.216
  ≥3108 (86.4%)0.312
  ≥4101 (80.8%)0.136
  ≥598 (78.4%)0.144
  ≥692 (73.6%)0.285
  ≥789 (71.2%)0.286
  ≥888 (70.4%)0.367
  ≥981 (64.8%)0.846
  ≥1078 (62.4%)0.443
  ≥1552 (41.6%)0.615
Table 4.

DSS by Lymph Nodes Examined for Different Subgroups

Lymph Nodes ExaminedNumber of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Tumor location
 Head (n = 288)
  ≥2270 (93.8%)0.38
  ≥3257 (89.2%)0.771
  ≥4247 (85.8%)0.408
  ≥5240 (83.3%)0.307
  ≥6230 (79.9%)0.461
  ≥7221 (76.7%)0.661
  ≥8211 (73.3%)0.63
  ≥9197 (68.4%)0.842
  ≥10190 (66%)0.728
  ≥15124 (43.1%)0.113
 Body (n = 97)
  ≥287 (89.7%)0.909
  ≥380 (82.5%)0.284
  ≥473 (75.3%)0.359
  ≥568 (70.1%)0.171
  ≥660 (61.9%)0.175
  ≥754 (55.7%)0.249
  ≥851 (52.6%)0.138
  ≥944 (45.4%)0.0410.2192.7 (0.55–13.16)
  ≥1040 (41.2%)0.245
  ≥1530 (30.9%)0.09
 Tail (n = 321)
  ≥2296 (92.2%)0.726
  ≥3264 (82.2%)0.949
  ≥4240 (74.8%)0.874
  ≥5217 (67.6%)0.922
  ≥6199 (62%)0.598
  ≥7180 (56.1%)0.874
  ≥8152 (47.4%)0.323
  ≥9139 (43.3%)0.281
  ≥10122 (38%)0.254
  ≥1556 (17.4%)0.094
Extent of resection
 Local excision (n = 771)
  ≥2713 (92.5%)0.894
  ≥3648 (84%)0.681
  ≥4596 (77.3%)0.88
  ≥5556 (72.1%)0.568
  ≥6513 (66.5%)0.575
  ≥7477 (61.9%)0.558
  ≥8429 (55.6%)0.342
  ≥9394 (51.1%)0.207
  ≥10361 (46.8%)0.248
  ≥15212 (27.5%)0.478
 Partial or total pancreatectomy (n = 125)
  ≥2113 (90.4%)0.216
  ≥3108 (86.4%)0.312
  ≥4101 (80.8%)0.136
  ≥598 (78.4%)0.144
  ≥692 (73.6%)0.285
  ≥789 (71.2%)0.286
  ≥888 (70.4%)0.367
  ≥981 (64.8%)0.846
  ≥1078 (62.4%)0.443
  ≥1552 (41.6%)0.615
Lymph Nodes ExaminedNumber of Cases Above Each Cutoff (%)P on Univariate AnalysisP on Multivariate AnalysisHR (95% CI)
Tumor location
 Head (n = 288)
  ≥2270 (93.8%)0.38
  ≥3257 (89.2%)0.771
  ≥4247 (85.8%)0.408
  ≥5240 (83.3%)0.307
  ≥6230 (79.9%)0.461
  ≥7221 (76.7%)0.661
  ≥8211 (73.3%)0.63
  ≥9197 (68.4%)0.842
  ≥10190 (66%)0.728
  ≥15124 (43.1%)0.113
 Body (n = 97)
  ≥287 (89.7%)0.909
  ≥380 (82.5%)0.284
  ≥473 (75.3%)0.359
  ≥568 (70.1%)0.171
  ≥660 (61.9%)0.175
  ≥754 (55.7%)0.249
  ≥851 (52.6%)0.138
  ≥944 (45.4%)0.0410.2192.7 (0.55–13.16)
  ≥1040 (41.2%)0.245
  ≥1530 (30.9%)0.09
 Tail (n = 321)
  ≥2296 (92.2%)0.726
  ≥3264 (82.2%)0.949
  ≥4240 (74.8%)0.874
  ≥5217 (67.6%)0.922
  ≥6199 (62%)0.598
  ≥7180 (56.1%)0.874
  ≥8152 (47.4%)0.323
  ≥9139 (43.3%)0.281
  ≥10122 (38%)0.254
  ≥1556 (17.4%)0.094
Extent of resection
 Local excision (n = 771)
  ≥2713 (92.5%)0.894
  ≥3648 (84%)0.681
  ≥4596 (77.3%)0.88
  ≥5556 (72.1%)0.568
  ≥6513 (66.5%)0.575
  ≥7477 (61.9%)0.558
  ≥8429 (55.6%)0.342
  ≥9394 (51.1%)0.207
  ≥10361 (46.8%)0.248
  ≥15212 (27.5%)0.478
 Partial or total pancreatectomy (n = 125)
  ≥2113 (90.4%)0.216
  ≥3108 (86.4%)0.312
  ≥4101 (80.8%)0.136
  ≥598 (78.4%)0.144
  ≥692 (73.6%)0.285
  ≥789 (71.2%)0.286
  ≥888 (70.4%)0.367
  ≥981 (64.8%)0.846
  ≥1078 (62.4%)0.443
  ≥1552 (41.6%)0.615

Multivariate analyses were performed for each cutoff of both positive lymph nodes and LNR that were significant on univariate analysis. Significant univariate variables included age >57 years, sex, extent of resection, tumor grade, T stage, and M stage. No cutoff for positive lymph nodes was found to be significant on multivariate analysis. On the contrary, various LNR cutoffs were independently associated with worse DSS (Table 3).

Comparison of different staging models to determine the optimal LNR

Each of the staging models based on LNR cutoffs that were independent prognostic factors of worse DSS were then selected for comparison. We performed this comparison by creating multivariate models containing each parameter of the standard staging process (i.e., tumor grade, AJCC T stage, and AJCC M stage), as well as LNR with direct comparison based on AIC, BIC, AICc, and C-statistic values. AIC, BIC, AICc, and C-statistic values have been used to identify optimal staging systems (25–29). Lower values indicate superior prognostic value for AIC, BIC, and AICc, whereas higher values indicate superior comparative prognostic value for the C-statistic. Among these cutoffs and models, the model using an LNR ≥0.5 had the lowest AIC (920.79), BIC (916.61), and AICc (923.01), and it also had the highest C-statistic (0.7792). In comparison, the eighth edition of the AJCC staging system (i.e., tumor grade, AJCC T stage, AJCC N stage, and AJCC M stage) had a higher AIC (923.38), BIC (919.2), and AICc (925.6), and it also had a lower C-statistic (0.7591). Notably, this was the least prognostic staging system compared with various LNR cutoffs in terms of AIC, BIC, AICc, and C-statistic (Table 5).

Table 5.

Comparison of Staging Models Based on LNR and AJCC Staging Systems

Staging ModelAICBICAICcC-Statistic
LNR 0.07922.58918.40924.800.7625
LNR 0.2921.25917.07923.470.7748
LNR 0.3921.44917.26923.660.7747
LNR 0.5920.79916.61923.010.7792
AJCC eighth edition923.38919.20925.600.7591
N stage and LNR 0.5922.50917.32924.780.7719
Staging ModelAICBICAICcC-Statistic
LNR 0.07922.58918.40924.800.7625
LNR 0.2921.25917.07923.470.7748
LNR 0.3921.44917.26923.660.7747
LNR 0.5920.79916.61923.010.7792
AJCC eighth edition923.38919.20925.600.7591
N stage and LNR 0.5922.50917.32924.780.7719
Table 5.

Comparison of Staging Models Based on LNR and AJCC Staging Systems

Staging ModelAICBICAICcC-Statistic
LNR 0.07922.58918.40924.800.7625
LNR 0.2921.25917.07923.470.7748
LNR 0.3921.44917.26923.660.7747
LNR 0.5920.79916.61923.010.7792
AJCC eighth edition923.38919.20925.600.7591
N stage and LNR 0.5922.50917.32924.780.7719
Staging ModelAICBICAICcC-Statistic
LNR 0.07922.58918.40924.800.7625
LNR 0.2921.25917.07923.470.7748
LNR 0.3921.44917.26923.660.7747
LNR 0.5920.79916.61923.010.7792
AJCC eighth edition923.38919.20925.600.7591
N stage and LNR 0.5922.50917.32924.780.7719

Discussion

Accurate assessment of a patient’s prognosis after surgery is vital to determine the best therapeutic treatment. Recently, the eighth edition of the AJCC was released to guide clinicians and patients with PNETs in assessing prognosis. In this study, the prognostic values of nodal positivity and the LNR were examined. Various LNR cutoffs were found to be significantly associated with worse DSS by multivariate analysis. A comparison of staging systems incorporating these LNR cutoffs and the eighth edition of AJCC staging system indicated that the optimal staging system incorporated T stage, M stage, LNR ≥0.5, and tumor grade.

The contribution of lymph node metastasis to prognosis is controversial in patients with PNET. Some studies suggest that lymph node metastasis is independently associated with worse disease-free survival (4, 30, 31) and overall survival (13, 14) in patients with PNET, whereas others did not find an association with overall survival (3, 16). Krampitz et al. (15) suggested that nodal positivity is associated with a shorter time to developing liver metastases and worse disease-related survival after an average of 8 years of follow up. Curran et al. (31) used the SEER database to examine the prognostic utility of nodal positivity and identified it as an independent predictor of DSS when patients were matched for grade and tumor size. In their analysis, high grade and increasing tumor size were also found to predict nodal metastasis. Nodal positivity was not found to be associated with DSS by multivariate analysis in this study. In contrast, LNR was found to be associated with DSS and should be incorporated into the PNET staging system.

Discrepancies in the results regarding the prognostic significance of lymph node metastasis may be explained by the low lymph node sampling rates for PNET, which ranged from 20.5% for enucleations to 77.2% for major anatomic resections (32). Low lymph node sampling rates result in part from the inconsistent practice of routine prophylactic lymphadenectomy in patients with PNET, with different criteria advocated by surgeons (30, 31, 33). For instance, Tsutsumi et al. (33) suggested that lymphadenectomy should be performed for all gastrinomas and nongastrinomas ≥15 mm, whereas Jiang et al. (30) proposed that routine lymphadenectomy in nonfunctional PNETs ≤25 mm is not necessary. Curran et al. (31) observed that no lymph node metastases were found in low-grade tumors that were <1 cm in patients who underwent lymphadenectomy, and thus they deemed lymphadenectomy unnecessary for these patients. However, assessing grade preoperatively by endoscopy can be difficult and not accurate. Hasegawa et al. (34) noted that the concordance rate of Ki-67 based on endoscopic fine needle aspiration and resected specimens is only 77.8%. LNR takes into account the adequacy of a lymphadenectomy, and therefore its association with DSS may be a more accurate determinant of patient prognosis with adequate lymph node resection. Nodal status as assessed by LNR is important for prognosis and should be completed in all patients with PNET.

The results of this study also demonstrate that higher nodal burden does not discriminate between low- and high-risk patients, because no cutoff number of positive lymph nodes was found to be significant on multivariate analysis. This ability of LNR to better stratify low- and high-risk patients is explained by consideration of the number of lymph nodes examined. Based on LNR, a patient with three positive lymph nodes out of four examined would be expected to have worse prognosis than a patient with three positive lymph nodes out of eight examined, whereas they would be expected to have the same prognosis if the number of positive lymph nodes were taken into consideration. If more lymph nodes were sampled, considering the low sampling rates during PNET surgery, the former patient would have a higher likelihood of having even more positive lymph nodes than the latter patient, indicating more aggressive disease behavior overall. In this way, an LNR-based staging system eliminates the variability in lymph node sampling during PNET surgery and is better able to stratify low- and high-risk patients than higher nodal burden.

The prognostic significance of LNR in PNET has been explored in two retrospective studies that used recurrence-free survival (RFS) as the primary endpoint. Ricci et al. (19) found that tumor size, Ki-67, and LNR >0.07 were independently associated with lower RFS, whereas Boninsegna et al. (20) identified LNR >0.2 and Ki-67 >5% as independent prognostic factors associated with lower RFS. These studies provide a foundation for determining whether LNR is associated with DSS, which is the most important outcome measure for staging and planning future treatment strategies. In the current study, LNRs ≥0.07 and ≥0.2 were assessed and found to be independently associated with DSS. However, we selected LNR ≥0.5 as the optimal cutoff because of a higher HR compared with LNRs ≥0.07 and ≥0.2 and improved prognostic staging when we compared the various LNR cutoffs.

The current study has several limitations, including the retrospective design and exclusion of cases based on the availability of data. Large cancer registries contain patients from different institutions and time periods and are subject to potential miscoding and missing information. In addition, there is no information on whether additional treatment was given. However, large cancer registries are the best current data available. The SEER database also does not collect information on benign tumors but instead includes tumors that have local invasion or metastatic disease. In this way, data on localized, less aggressive tumors may not be included in the current analysis. This limitation may explain why our cutoff was higher than the ones reported by other studies (19, 20). Furthermore, although an analysis to identify the minimum number of lymph nodes examined was performed, which did not reveal a significant cutoff, the results of this study are potentially limited by stage migration due to the insufficient number of lymph nodes examined. For this purpose, future prospective studies with a predetermined minimum number of lymph nodes examined could more appropriately answer this question.

In conclusion, LNR ≥0.5 is independently associated with worse DSS. A staging system with LNR ≥0.5 is superior to the current AJCC eighth edition staging system, which uses nodal positivity, and therefore LNR ≥0.5 should replace nodal positivity in future pathological staging systems.

Abbreviations:

     
  • AIC

    Akaike information criterion

  •  
  • AICc

    corrected Akaike information criterion

  •  
  • AJCC

    American Joint Committee on Cancer

  •  
  • BIC

    Bayesian information criterion

  •  
  • CI

    confidence interval

  •  
  • df

    degrees of freedom

  •  
  • DSS

    disease-specific survival

  •  
  • HR

    hazard ratio

  •  
  • LNR

    lymph node ratio

  •  
  • MD

    moderately differentiated

  •  
  • PD

    poorly differentiated

  •  
  • PNET

    pancreatic neuroendocrine tumor

  •  
  • RFS

    recurrence-free survival

  •  
  • SEER

    Surveillance, Epidemiology, and End Results

  •  
  • UD

    undifferentiated

  •  
  • WD

    well differentiated.

Acknowledgments

We thank Dr. Seth Steinberg and Dr. David Venzon for assistance in statistical analysis and interpretation.

Financial Support: This research was supported by the intramural research program of the Center for Cancer Research, National Cancer Institute, National Institutes of Health.

Disclosure Summary: The authors have nothing to disclose.

References

1.

Yao
JC
,
Hassan
M
,
Phan
A
,
Dagohoy
C
,
Leary
C
,
Mares
JE
,
Abdalla
EK
,
Fleming
JB
,
Vauthey
JN
,
Rashid
A
,
Evans
DB
.
One hundred years after “carcinoid”: epidemiology of and prognostic factors for neuroendocrine tumors in 35,825 cases in the United States
.
J Clin Oncol
.
2008
;
26
(
18
):
3063
3072
.

2.

Fesinmeyer
MD
,
Austin
MA
,
Li
CI
,
De Roos
AJ
,
Bowen
DJ
.
Differences in survival by histologic type of pancreatic cancer
.
Cancer Epidemiol Biomarkers Prev
.
2005
;
14
(
7
):
1766
1773
.

3.

Bilimoria
KY
,
Talamonti
MS
,
Tomlinson
JS
,
Stewart
AK
,
Winchester
DP
,
Ko
CY
,
Bentrem
DJ
.
Prognostic score predicting survival after resection of pancreatic neuroendocrine tumors: analysis of 3851 patients
.
Ann Surg
.
2008
;
247
(
3
):
490
500
.

4.

Hashim
YM
,
Trinkaus
KM
,
Linehan
DC
,
Strasberg
SS
,
Fields
RC
,
Cao
D
,
Hawkins
WG
.
Regional lymphadenectomy is indicated in the surgical treatment of pancreatic neuroendocrine tumors (PNETs)
.
Ann Surg
.
2014
;
259
(
2
):
197
203
.

5.

Bosman
FT
,
Carneiro
F
,
Hruban
RH
,
Theise
ND
.
WHO Classification of Tumours of the Digestive System
. 4th ed.
Geneva, Switzerland
:
World Health Organization
;
2010
.

6.

Amin MB, Edge SB, American Joint Committee on Cancer. AJCC Cancer Staging Manual. http://www.springer.com/us/book/9783319406176. Accessed 5 May 2017.

7.

Rindi
G
,
Klöppel
G
,
Alhman
H
,
Caplin
M
,
Couvelard
A
,
de Herder
WW
,
Erikssson
B
,
Falchetti
A
,
Falconi
M
,
Komminoth
P
,
Körner
M
,
Lopes
JM
,
McNicol
AM
,
Nilsson
O
,
Perren
A
,
Scarpa
A
,
Scoazec
JY
,
Wiedenmann
B
;
all other Frascati Consensus Conference participants
;
European Neuroendocrine Tumor Society (ENETS)
.
TNM staging of foregut (neuro)endocrine tumors: a consensus proposal including a grading system
.
Virchows Arch
.
2006
;
449
(
4
):
395
401
.

8.

Caplin
ME
,
Pavel
M
,
Ćwikła
JB
,
Phan
AT
,
Raderer
M
,
Sedláčková
E
,
Cadiot
G
,
Wolin
EM
,
Capdevila
J
,
Wall
L
,
Rindi
G
,
Langley
A
,
Martinez
S
,
Blumberg
J
,
Ruszniewski
P
;
CLARINET Investigators
.
Lanreotide in metastatic enteropancreatic neuroendocrine tumors
.
N Engl J Med
.
2014
;
371
(
3
):
224
233
.

9.

Yao
JC
,
Shah
MH
,
Ito
T
,
Bohas
CL
,
Wolin
EM
,
Van Cutsem
E
,
Hobday
TJ
,
Okusaka
T
,
Capdevila
J
,
de Vries
EG
,
Tomassetti
P
,
Pavel
ME
,
Hoosen
S
,
Haas
T
,
Lincy
J
,
Lebwohl
D
,
Öberg
K
;
RAD001 in Advanced Neuroendocrine Tumors, Third Trial (RADIANT-3) Study Group
.
Everolimus for advanced pancreatic neuroendocrine tumors
.
N Engl J Med
.
2011
;
364
(
6
):
514
523
.

10.

Raymond
E
,
Dahan
L
,
Raoul
J-L
,
Bang
YJ
,
Borbath
I
,
Lombard-Bohas
C
,
Valle
J
,
Metrakos
P
,
Smith
D
,
Vinik
A
,
Chen
JS
,
Hörsch
D
,
Hammel
P
,
Wiedenmann
B
,
Van Cutsem
E
,
Patyna
S
,
Lu
DR
,
Blanckmeister
C
,
Chao
R
,
Ruszniewski
P
.
Sunitinib malate for the treatment of pancreatic neuroendocrine tumors
.
N Engl J Med
.
2011
;
364
(
6
):
501
513
.

11.

Young
K
,
Iyer
R
,
Morganstein
D
,
Chau
I
,
Cunningham
D
,
Starling
N
.
Pancreatic neuroendocrine tumors: a review
.
Future Oncol
.
2015
;
11
(
5
):
853
864
.

12.

Strosberg
J
,
El-Haddad
G
,
Wolin
E
,
Hendifar
A
,
Yao
J
,
Chasen
B
,
Mittra
E
,
Kunz
PL
,
Kulke
MH
,
Jacene
H
,
Bushnell
D
,
O’Dorisio
TM
,
Baum
RP
,
Kulkarni
HR
,
Caplin
M
,
Lebtahi
R
,
Hobday
T
,
Delpassand
E
,
Van Cutsem
E
,
Benson
A
,
Srirajaskanthan
R
,
Pavel
M
,
Mora
J
,
Berlin
J
,
Grande
E
,
Reed
N
,
Seregni
E
,
Öberg
K
,
Lopera Sierra
M
,
Santoro
P
,
Thevenet
T
,
Erion
JL
,
Ruszniewski
P
,
Kwekkeboom
D
,
Krenning
E
;
NETTER-1 Trial Investigators
.
Phase 3 trial of 177Lu-Dotatate for midgut neuroendocrine tumors
.
N Engl J Med
.
2017
;
376
(
2
):
125
135
.

13.

Tomassetti
P
,
Campana
D
,
Piscitelli
L
,
Casadei
R
,
Santini
D
,
Nori
F
,
Morselli-Labate
AM
,
Pezzilli
R
,
Corinaldesi
R
.
Endocrine pancreatic tumors: factors correlated with survival
.
Ann Oncol
.
2005
;
16
(
11
):
1806
1810
.

14.

Bettini
R
,
Boninsegna
L
,
Mantovani
W
,
Capelli
P
,
Bassi
C
,
Pederzoli
P
,
Delle Fave
GF
,
Panzuto
F
,
Scarpa
A
,
Falconi
M
.
Prognostic factors at diagnosis and value of WHO classification in a mono-institutional series of 180 non-functioning pancreatic endocrine tumours
.
Ann Oncol
.
2008
;
19
(
5
):
903
908
.

15.

Krampitz
GW
,
Norton
JA
,
Poultsides
GA
,
Visser
BC
,
Sun
L
,
Jensen
RT
.
Lymph nodes and survival in pancreatic neuroendocrine tumors
.
Arch Surg
.
2012
;
147
(
9
):
820
827
.

16.

Kazanjian
KK
,
Reber
HA
,
Hines
OJ
.
Resection of pancreatic neuroendocrine tumors: results of 70 cases
.
Arch Surg
.
2006
;
141
(
8
):
765
769, discussion 769–770
.

17.

Pawlik
TM
,
Gleisner
AL
,
Cameron
JL
,
Winter
JM
,
Assumpcao
L
,
Lillemoe
KD
,
Wolfgang
C
,
Hruban
RH
,
Schulick
RD
,
Yeo
CJ
,
Choti
MA
.
Prognostic relevance of lymph node ratio following pancreaticoduodenectomy for pancreatic cancer
.
Surgery
.
2007
;
141
(
5
):
610
618
.

18.

Riediger
H
,
Keck
T
,
Wellner
U
,
zur Hausen
A
,
Adam
U
,
Hopt
UT
,
Makowiec
F
.
The lymph node ratio is the strongest prognostic factor after resection of pancreatic cancer
.
J Gastrointest Surg
.
2009
;
13
(
7
):
1337
1344
.

19.

Ricci
C
,
Casadei
R
,
Taffurelli
G
,
Buscemi
S
,
D’Ambra
M
,
Monari
F
,
Santini
D
,
Campana
D
,
Tomassetti
P
,
Minni
F
.
The role of lymph node ratio in recurrence after curative surgery for pancreatic endocrine tumours
.
Pancreatology
.
2013
;
13
(
6
):
589
593
.

20.

Boninsegna
L
,
Panzuto
F
,
Partelli
S
,
Capelli
P
,
Delle Fave
G
,
Bettini
R
,
Pederzoli
P
,
Scarpa
A
,
Falconi
M
.
Malignant pancreatic neuroendocrine tumour: lymph node ratio and Ki67 are predictors of recurrence after curative resections
.
Eur J Cancer
.
2012
;
48
(
11
):
1608
1615
.

21.

Akaike
H
.
A new look at the statistical model identification
.
IEEE Trans Automat Contr
.
1974
;
19
(
6
):
716
723
.

22.

Schwarz
G
.
Estimating the dimension of a model
.
Ann Stat
.
1978
;
6
(
2
):
461
464
.

23.

Hurvich
CM
,
Simonoff
JS
,
Tsai
C-L
.
Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion
.
J R Stat Soc Ser B. Stat Methodol
.
1998
;
60
(
2
):
271
293
.

24.

Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med. 1984;3(2):143–152. http://www.ncbi.nlm.nih.gov/pubmed/6463451. Accessed 31 July 2017.

25.

Deng
J
,
Zhang
R
,
Wu
L
,
Zhang
L
,
Wang
X
,
Liu
Y
,
Hao
X
,
Liang
H
.
Superiority of the ratio between negative and positive lymph nodes for predicting the prognosis for patients with gastric cancer
.
Ann Surg Oncol
.
2015
;
22
(
4
):
1258
1266
.

26.

Hari
DM
,
Leung
AM
,
Lee
J-H
,
Sim
MS
,
Vuong
B
,
Chiu
CG
,
Bilchik
AJ
.
AJCC Cancer Staging Manual 7th Edition criteria for colon cancer: do the complex modifications improve prognostic assessment
?
J Am Coll Surg
.
2013
;
217
(
2
):
181
190
.

27.

Malm
I-J
,
Fan
CJ
,
Yin
LX
,
Li
DX
,
Koch
WM
,
Gourin
CG
,
Pitman
KT
,
Richmon
JD
,
Westra
WH
,
Kang
H
,
Quon
H
,
Eisele
DW
,
Fakhry
C
.
Evaluation of proposed staging systems for human papillomavirus–related oropharyngeal squamous cell carcinoma
.
Cancer
.
2017
;
123
(
10
):
1768
1777
.

28.

Ning
Z
,
Wang
Z
,
Chen
J
,
Li
XD
,
Chen
LJ
,
Xu
B
,
Gu
WD
,
Shao
YJ
,
Xu
Y
,
Huang
J
,
Pei
HL
,
Jiang
JT
.
Proposed modification of nodal staging as an alternative to the seventh edition of the American Joint Committee on Cancer Tumor–Node–Metastasis Staging System improves the prognostic prediction in the resected esophageal squamous-cell carcinoma
.
J Thorac Oncol
.
2015
;
10
(
7
):
1091
1098
.

29.

Zhuge
Y
,
Zhang
F
,
Qiu
Y
,
Li
Z
,
Zhang
J
.
Prognostic accuracy of staging systems in patients with primary liver cancer undergoing transarterial chemoembolization
.
Hepatogastroenterology
.
2013
;
60
(
123
):
481
488
.

30.

Jiang
Y
,
Jin
J-B
,
Zhan
Q
,
Deng
X-X
,
Shen
B-Y
.
Impact and clinical predictors of lymph node metastases in nonfunctional pancreatic neuroendocrine tumors
.
Chin Med J (Engl)
.
2015
;
128
(
24
):
3335
3344
.

31.

Curran
T
,
Pockaj
BA
,
Gray
RJ
,
Halfdanarson
TR
,
Wasif
N
.
Importance of lymph node involvement in pancreatic neuroendocrine tumors: impact on survival and implications for surgical resection
.
J Gastrointest Surg
.
2015
;
19
(
1
):
152
160, discussion 160
.

32.

Parekh
JR
,
Wang
SC
,
Bergsland
EK
,
Venook
AP
,
Warren
RS
,
Kim
GE
,
Nakakura
EK
.
Lymph node sampling rates and predictors of nodal metastasis in pancreatic neuroendocrine tumor resections: the UCSF experience with 149 patients
.
Pancreas
.
2012
;
41
(
6
):
840
844
.

33.

Tsutsumi
K
,
Ohtsuka
T
,
Mori
Y
,
Fujino
M
,
Yasui
T
,
Aishima
S
,
Takahata
S
,
Nakamura
M
,
Ito
T
,
Tanaka
M
.
Analysis of lymph node metastasis in pancreatic neuroendocrine tumors (PNETs) based on the tumor size and hormonal production
.
J Gastroenterol
.
2012
;
47
(
6
):
678
685
.

34.

Hasegawa
T
,
Yamao
K
,
Hijioka
S
,
Bhatia
V
,
Mizuno
N
,
Hara
K
,
Imaoka
H
,
Niwa
Y
,
Tajika
M
,
Kondo
S
,
Tanaka
T
,
Shimizu
Y
,
Kinoshita
T
,
Kohsaki
T
,
Nishimori
I
,
Iwasaki
S
,
Saibara
T
,
Hosoda
W
,
Yatabe
Y
.
Evaluation of Ki-67 index in EUS-FNA specimens for the assessment of malignancy risk in pancreatic neuroendocrine tumors
.
Endoscopy
.
2014
;
46
(
1
):
32
38
.