Pre-dialysis serum sodium and mortality in a national incident hemodialysis cohort

Abstract

Background

A consistent association between low serum sodium measured at a single-point-in-time (baseline sodium) and higher mortality has been observed in hemodialysis patients. We hypothesized that both low and high time-varying sodium levels (sodium levels updated at quarterly intervals as a proxy of short-term exposure) are independently associated with higher death risk in hemodialysis patients.

Methods

We examined the association of baseline and time-varying pre-dialysis serum sodium levels with all-cause mortality among adult incident hemodialysis patients receiving care from a large national dialysis organization during January 2007–December 2011. Hazard ratios were estimated using multivariable Cox models accounting for case-mix+laboratory covariates and incrementally adjusted for inter-dialytic weight gain, blood urea nitrogen and glucose.

Results

Among 27 180 patients, a total of 7562 deaths were observed during 46 194 patient-years of follow-up. Median (IQR) at-risk time was 1.4 (0.6, 2.5) years. In baseline analyses adjusted for case-mix+laboratory results, sodium levels <138 mEq/L were associated with incrementally higher mortality risk, while the association of sodium levels ≥140 mEq/L with lower mortality reached statistical significance only for the highest level of pre-dialysis sodium (reference: 138–<140 mEq/L). In time-varying analyses, we observed a U-shaped association between sodium and mortality such that sodium levels <138 and ≥144 mEq/L were associated with higher mortality risk. Similar patterns were observed in models incrementally adjusted for inter-dialytic weight gain, blood urea nitrogen and glucose.

Conclusions

We observed a U-shaped association of time-varying pre-dialysis serum sodium and all-cause mortality in hemodialysis patients, suggesting that both hypo- and hypernatremia carry short-term risk in this population.

INTRODUCTION

Dysnatremias are the most common electrolyte disorder encountered in clinical practice, and are disproportionately observed across a variety of medical conditions, including chronic kidney disease (CKD) [1–3]. For example, epidemiologic data have shown that CKD patients have a 2-fold higher prevalence of hyponatremia compared with their non-CKD counterparts [3–5]. In CKD patients undergoing maintenance hemodialysis, a consistent association between low serum sodium and higher mortality has been observed across a number of studies [3, 6–11]. While high serum sodium has also been associated with higher death risk in non-CKD patients [12, 13], the association of hypernatremia with all-cause mortality has not previously been reported in studies of hemodialysis patients. This may have been due to exclusion of patients with hypernatremia [9, 10], non-granular examination of observed sodium values and analysis of sodium measured at a single-point-in-time [7, 9, 10] (i.e. baseline sodium, which captures long-term exposure and does not account for changes in sodium over time) in past studies. However, in a recent study examining time-varying sodium levels (i.e. sodium levels updated at quarterly intervals, as a proxy of short-term exposure) in CKD patients not undergoing maintenance dialysis, both hypo- and hypernatremia were associated with higher mortality risk [3, 14].

In addition, it remains uncertain as to whether the aforementioned hyponatremia–mortality associations in hemodialysis patients are causal, or due to confounding by underlying conditions that predispose to dysnatremia [4, 15]. For example, in past studies there has been variable consideration of risk factors for dysnatremia (e.g. large inter-dialytic weight gain (IDWG) [16–20], poor nutritional status and low solute intake [21–23], comorbidities predisposing to excess thirst [11, 24]) that are in and of themselves associated with death risk. Interpretation of prior data are also made difficult due to focus upon prevalent hemodialysis patients whose characteristics may be confounded by survivor bias [6–8, 10, 11], and examination of select study populations (i.e. randomized controlled trial participants) who may not be representative of the dialysis population at large [11]. Thus to better inform the field, we sought to examine the association of baseline and repeated measures of pre-dialysis serum sodium level with all-cause mortality among incident hemodialysis patients from a large dialysis organization in the USA with comprehensive availability of comorbidity, dialysis treatment and laboratory data. We hypothesized that both hypo- and hypernatremia were independently associated with higher death risk in this nationally representative hemodialysis population.

MATERIALS AND METHODS

Source cohort description

We conducted an observational study using data from a large dialysis organization in the USA with detailed patient-level information on sociodemographics, comorbidities, laboratory tests, dialysis treatment characteristics, clinical events and vital status [25]. The original source population was a cohort of 208 820 adult incident dialysis patients receiving care in one of the facilities operated by the dialysis provider over a 5-year period (1 January 2007–31 December 2011). Patients were included provided that at study entry (defined as the time of baseline sodium measurement) they had a dialysis vintage of at least 60 days, were undergoing thrice-weekly in-center hemodialysis throughout the entire study period and had one or more sodium measurement(s) during their baseline quarter (first 91 days) of dialysis. Patients were excluded if at study entry they underwent treatment with peritoneal dialysis or home hemodialysis at any time during follow-up, had missing censor/death date information or had an outlier sodium value (<125 mEq/L; Supplementary data, Figure S1). The study was approved by the Institutional Review Committees of the Los Angeles Biomedical Research Institute at Harbor-UCLA, University of California Irvine Medical Center and University of Washington.

Exposure ascertainment

In our primary analyses, we sought to granularly examine the association between pre-dialysis serum sodium level and all-cause mortality. Serum samples for laboratory testing were drawn pre-dialysis within the outpatient dialysis clinics of the large dialysis organization using uniform techniques, and were transported to a single central laboratory in Deland, FL, USA, typically within 24 h for sodium measurement. All laboratory testing was conducted using automated and standardized methods. The exposure was defined as serum sodium level divided into nine categories: <130, 130–<132, 132–<134, 134–<136, 136–<138, 138–<140 (reference group), 140–<142, 142–<144 and ≥144 mEq/L. In sensitivity analyses, in order to flexibly examine serum sodium as a continuous predictor of mortality, we conducted a restricted cubic spline analysis with knots defined at the 25th, 50th and 75th percentiles of observed sodium values (sodium levels 136, 138 and 140 mEq/L, respectively). Sodium levels were ascertained as the mean of all values over successive 91-day periods from the date of first dialysis.

Given that serum sodium levels in hemodialysis patients may be influenced by IDWG, which is an independent predictor of mortality [11, 16–20], we examined effect-modification of the association of serum sodium with mortality by IDWG in two ways. We first compared 15 combinations of sodium level (divided into five categories: <133, 133–<136, 136–<139, 139–<142 and ≥142 mEq/L) and IDWG (divided into three categories of <1.5, 1.5–<2.5 and ≥2.5 kg defined as Low, Moderate and High IDWG, respectively) with a single reference category (136–<139 mEq/L + Low IDWG). We then examined the association between sodium level (divided into the same five aforementioned categories) and mortality across IDWG strata (divided into the same three aforementioned categories), in which the sodium reference group was 136–<139 mEq/L for each IDWG strata (i.e. three reference groups across the three IDWG strata).

Outcome ascertainment

The primary outcome of interest was all-cause mortality. At-risk time began the day after the baseline quarter of sodium measurement. Patients were censored for kidney transplantation, transfer to a dialysis facility operated by another provider or at the end of the study (31 December 2011).

Statistical analyses

The association between sodium level and mortality was examined using two approaches: (i) baseline sodium–mortality associations were assessed in which sodium and covariates were determined at baseline and their association with subsequent mortality were estimated (in order to ascertain long-term exposure–mortality associations) and (ii) time-varying sodium–mortality associations were examined in which sodium and time-varying covariates were time-updated (in order to ascertain short-term exposure–mortality associations) [26]. The median (IQR) frequency of quarterly-averaged sodium measurements per patient was 4 (2–9) for the time-varying analyses.

We estimated the association between sodium level and mortality using Cox proportional hazard models with up to six levels of covariate adjustment. In time-varying analyses, vascular access, as well as laboratory covariates and IDWG, were examined as time-varying variables summarized over 91-day periods (i.e. mean or median values over the quarter for each patient):

• ‘unadjusted’: adjusted for patients' calendar quarter of entry into the cohort;

• ‘case-mix adjusted’: adjusted for covariates in the unadjusted model, as well as age; sex; race/ethnicity; primary insurance; vascular access; comorbidities including diabetes, alcohol dependence, congestive heart failure, atherosclerotic heart disease, other cardiovascular disease, chronic obstructive pulmonary disease, cerebrovascular disease, human immunodeficiency virus, malignancy and hypertension;

• ‘case-mix+IDWG adjusted model’: adjusted for covariates in the case-mix adjusted model and IDWG;

• ‘case-mix+laboratory adjusted’: adjusted for covariates in the case-mix adjusted model, as well as dialysis adequacy (spKt/V), residual urea clearance (mL/min), body mass index, serum albumin, creatinine, total iron binding capacity, ferritin, iron saturation, bicarbonate, calcium, phosphorus, parathyroid hormone, blood hemoglobin, white blood cell (WBC) count and normalized protein catabolic rate;

• ‘case-mix+laboratory+IDWG adjusted’: adjusted for covariates in the case-mix+laboratory adjusted model and IDWG;

• ‘case-mix+extended laboratory+IDWG adjusted’: adjusted for covariates in the case-mix+laboratory+IDWG adjusted model, as well as serum blood urea nitrogen and glucose levels.

We also conducted subgroup analyses across clinically relevant categories of sociodemographics, comorbidity status, body anthropometry and laboratory measures. Missing data were handled using imputation by means or medians. Most covariates except for serum creatinine, residual urea clearance and glucose had <1% missing values. Proportional hazards assumptions were checked by graphical and formal testing. Analyses and figures were conducted using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA), Stata version 13.1 (Stata Corporation, College Station, TX, USA) and SigmaPlot Version 12.5 (Systat Software, San Jose, CA, USA).

RESULTS

Study population description

The overall cohort consisted of 27 180 patients who met eligibility criteria (Supplementary data, Figure S1), among whom 8% had hyponatremia (sodium <134 mEq/L) and 2% had hypernatremia (sodium ≥144 mEq/L). Among these patients, the mean ± SD, median (IQR) and minimum–maximum of observed baseline pre-dialysis sodium levels were 138 ± 3, 138 (136, 140) and 125–150 mEq/L, respectively. We first compared baseline characteristics among included versus excluded patients in the study, and found that the included patients were more likely to: have a central venous catheter as their vascular access; have diabetes, hyperlipidemia, congestive heart failure or other cardiovascular disease; had lower serum ferritin levels; and had a higher proportion of deaths (Supplementary data, Table S1). We then examined baseline characteristics that were associated with a greater likelihood of having a baseline sodium measurement among patients who met the initial eligibility criteria (i.e. vintage >60 days, receiving in-center thrice-weekly hemodialysis and no missing censor date; Supplementary data, Table S2). We observed that patients who were older and Caucasian; had hyperlipidemia or other cardiovascular disease; and had higher spKt/V, residual urea clearance, serum albumin, phosphorus, parathyroid hormone, hemoglobin, ferritin, iron saturation, blood urea nitrogen and WBC counts were more likely to have undergone serum sodium measurement. In contrast, those who were African-American; had diabetes, chronic obstructive pulmonary disease or hypertension; and who had lower glucose levels were less likely to have undergone sodium measurement.

Comparison of baseline characteristics among patients according to baseline sodium category is shown in Table 1. Compared with patients in the highest sodium category (≥144 mEq/L), those in the lowest category (<130 mEq/L) were more likely to be female, Caucasian or Hispanic and less likely to be African-American; were more likely to have a central venous catheter and less likely to have an arteriovenous fistula; were more likely to have congestive heart failure and less likely to have hyperlipidemia or hypertension; were more likely to have higher IDWG and lower residual urea clearance; had lower body mass index, serum albumin, serum creatinine and parathyroid hormone levels; and had higher alkaline phosphatase and glucose levels.

Sodium and mortality

Patients contributed a total of 46 194 years of follow-up during which time 7562 deaths were observed. Median (IQR) at-risk time was 1.4 (0.6, 2.5) years. In analyses of baseline sodium divided into nine categories, our case-mix+laboratory adjusted model showed that sodium levels <138 mEq/L were associated with incrementally higher mortality risk (reference: 138–140 mEq/L): adjusted HRs (aHRs) (95% CI) 1.13 (1.06–1.20), 1.23 (1.14–1.33), 1.33 (1.20–1.47), 1.46 (1.26–1.70) and 1.93 (1.59–2.35) for sodium categories 136–<138, 134–<136, 132–<134, 130–<132 and <130 mEq/L, respectively (Figure 1 and Supplementary data, Table S3). Conversely, the association of sodium levels ≥140 mEq/L with lower mortality reached statistical significance only for the highest level of pre-dialysis sodium (reference: 138–140 mEq/L): aHRs (95% CI) 0.95 (0.89, 1.02), 0.92 (0.84–1.01) and 0.83 (0.71–0.98) for sodium categories 140–<142, 142–<144 and ≥144 mEq/L, respectively. A similar association between lower sodium level and higher mortality risk was observed in models incrementally adjusted for (i) IDWG and (ii) blood urea nitrogen and glucose. In analyses examining continuous sodium as a restricted cubic spline adjusted for case-mix+laboratory test results, we observed a quadratic association between baseline sodium level and all-cause mortality, whereas sodium levels <137 mEq/L were associated with progressively higher mortality risk, and sodium levels ∼137–145 mEq/L were associated with lower mortality risk (Supplementary data, Figure S2). The low frequency of patients with sodium levels >145 mEq/L likely rendered estimates unstable above this threshold.

In analyses of time-varying sodium divided into nine categories, analyses adjusted for case-mix+laboratory test results demonstrated a U-shaped association between sodium and mortality such that sodium levels <138 and ≥144 mEq/L were associated with higher mortality risk (reference: 138–140 mEq/L): aHRs (95% CI) 2.25 (1.95–2.59), 1.70 (1.50–1.93), 1.50 (1.36–1.65), 1.31 (1.21–1.41), 1.12 (1.04–1.19), 0.97 (0.90–1.04), 1.10 (0.99–1.21) and 1.49 (1.28–1.74) for sodium categories <130, 130–<132, 132–<134, 134–<136, 136–<138, 140–<142, 142–<144 and ≥144 mEq/L, respectively (Figure 1 and Supplementary data, Table S3). A similar U-shaped association between sodium level and mortality risk was observed in models incrementally adjusted for (i) IDWG and (ii) blood urea nitrogen and glucose.

Sodium, IDWG and mortality

We sought to examine the interaction between serum sodium level, IDWG and mortality in two ways. We first compared 15 combinations of sodium levels + IDWG (reference: 136–<139 mEq/L + Low IDWG). In baseline analyses, sodium–IDWG categories in which sodium levels were <133 and 133–<136 mEq/L were consistently associated with higher mortality risk across all models, with incrementally stronger associations observed with higher IDWG categories (Figure 2 and Supplementary data, Table S4). In contrast, sodium–IDWG categories in which sodium levels were ≥142 mEq/L were consistently associated with lower mortality risk across all models, with incrementally stronger associations observed with lower IDWG categories. In the case-mix+laboratory adjusted model, the <133 mEq/L + High IDWG category and the ≥142 mEq/L + Low IDWG category were associated with the highest and lowest mortality risks, respectively.

In time-varying analyses, we similarly observed that sodium–IDWG categories in which sodium was <133 mEq/L were consistently associated with higher mortality risk across all models, although a graded association was not observed with higher IDWG categories (Figure 3 and Supplementary data, Table S4). In contrast to the baseline analyses, we observed that sodium–IDWG categories in which sodium was 139–<142 mEq/L were consistently associated with lower mortality risk, whereas those in which sodium was ≥142 mEq/L had variable associations with mortality.

We then examined the association between sodium level and mortality across IDWG strata, in which the sodium reference group was 136–<139 mEq/L for each IDWG strata. In baseline analyses adjusted for case-mix, there was an inverse linear association between lower sodium level and higher mortality risk across all IDWG strata (Supplementary data, Figure S3 and Table S5). However, with additional adjustment for laboratory parameters, this inverse sodium–mortality association was only observed in the Moderate and High IDWG categories. In time-varying analyses adjusted for case-mix or case-mix+laboratory parameters, there was a reverse J-shaped association between sodium level and mortality across all IDWG strata, such that sodium levels <136 mEq/L were associated with incrementally higher mortality risk, whereas sodium levels ≥142 mEq/L had a null association with mortality (Supplementary data, Figure S4 and Table S5).

Clinically relevant subgroup analyses

We then examined the association between sodium level dichotomized as <140 versus ≥140 mEq/L (reference: ≥140 mEq/L) and mortality across clinically relevant subgroups. In baseline analyses adjusted for case-mix, we observed that lower sodium was associated with higher mortality across all subgroups except for those with liver disease, body mass index ≥40 kg/m² and serum phosphorus <3.5 mg/dL (Figure 4A and B and Supplementary data, Table S5). In time-varying analyses adjusted for case-mix, we observed that lower sodium was associated with higher mortality across all subgroups except those of other race/ethnicity and phosphorus <3.5 mg/dL (Figure 4C and D and Supplementary data, Table S5).

DISCUSSION

In this study of a nationally representative incident hemodialysis cohort that has been the largest primary analysis of sodium and mortality conducted to date in dialysis patients, we observed that a sizeable proportion of patients had dysnatremia at baseline (>10%). In baseline analyses, we observed a graded association between lower sodium levels <138 mEq/L and higher mortality risk, and a decreased risk for death with sodium ≥144 mEq/L. In time-varying analyses, we observed that both lower and higher sodium levels (sodium <138 and ≥144 mEq/L, respectively) were associated with higher mortality risk (i.e. U-shaped association).

Prior studies of dysnatremia in hemodialysis patients have shown that there is an inverse association between serum sodium level and mortality risk [6–11]. In a study of 11 555 prevalent hemodialysis patients from the DOPPS I and III cohorts, lower serum sodium (derived from the mean of patients' first three sodium measurements) was associated with higher mortality risk, whereas higher levels were associated with lower mortality risk [6]. Similarly, in a study of 6127 incident patients from the ArMORR cohort, each 3 mEq/L decrement in sodium level was associated with 14% higher risk of mortality [9]. While these collective data suggest that higher sodium levels are protective, data limitations precluded consideration of longitudinal changes in serum sodium over time and the latter study examined sodium levels over a narrower range, which excluded patients with sodium levels >145 mEq/L. Yet in a rigorous study of 1549 prevalent hemodialysis patients from the HEMO trial that examined pre-dialysis sodium as a baseline and time-varying predictor, higher sodium levels were also associated with incrementally lower mortality risk [11]. However, it should be noted that these analyses did not account for IDWG as a confounder, and was conducted among a select cohort of randomized controlled trial participants who may not be representative of the general hemodialysis population.

To our knowledge, ours is the first study that has shown a U-shaped association between time-varying sodium level and mortality in hemodialysis patients. In a study that evaluated a spectrum of sodium concentrations in 655 493 US veterans with CKD not undergoing maintenance dialysis, both time-varying hypo- and hypernatremia (sodium <136 and >145 mEq/L, respectively) were associated with higher mortality risk (with a sodium level of 140 mEq/L corresponding to the lowest death risk) [3]. Given that time-varying analyses provide insight into the short-term exposure–mortality associations, our data extend upon these observations by suggesting that both hypo- and hypernatremia carry short-term death risk in hemodialysis patients. While our time-varying analyses of serum sodium and mortality across strata of IDWG did not detect an association between higher sodium level (≥142 mEq/L) and higher mortality risk, it is possible that this may have been due to (i) consolidation of the sodium categories 142–<144 and ≥144 mEq/L, which may have diluted the association of higher sodium levels ≥144 mEq/L with mortality risk, or (ii) limited power and sample size within the sodium–IDWG strata to detect a statistically significant association.

There are several mechanisms by which dysnatremia may directly predispose to mortality among hemodialysis patients, a unique population in whom serum sodium levels are influenced by dialysate and dietary sodium and fluid intake, as opposed to elevated vasopressin secretion resulting from underlying clinical conditions (e.g. congestive heart failure, cirrhosis) [11, 19]. First, both hypo- and hypernatremic patients may experience recurring alterations in their serum sodium and osmolarity induced by higher/lower dialysate sodium concentrations that may be directly harmful [11]. Second, hypo- and hypernatremia may lead to disequilibrium, gait disturbances, falls, fractures and subsequent death risk [4, 27–30]. Hyponatremia may further predispose to fracture risk by reducing bone mineral density, volume and metabolism [31, 32]. Indeed, in animal studies hyponatremia has been shown to stimulate osteoclast activity (increasing calcium and suppressing parathyroid hormone), and it has been also been suggested that it may directly activate osteoblasts (increasing alkaline phosphatase) [9, 32, 33]. Similar to prior observations, our data also showed that lower sodium levels were associated with lower parathyroid hormone and higher alkaline phosphatase levels [9]. Dysnatremia may also be directly toxic to other end organs including the brain (i.e. acute hyponatremia and hypernatremia if not treated promptly may lead to cerebral edema/seizures and coma, respectively) [34–36], heart (i.e. hyponatremia may inhibit calcium-channel circuits and perturb cardiac function) [37] and musculoskeletal system (i.e. rhabdomyolysis) [38]. Emerging data also suggest that hyponatremia may increase risk of infection, presumably due to (i) breakdown of the microbial barrier function resultant from cellular edema of the mucosal membrane and (ii) impaired function of IL-17 producing helper T cells that play an important role in host immunity [39–41].

It is also possible that hyponatremia may be a marker of underlying conditions that are in and of themselves associated with death such as (i) protein-energy wasting and poor solute intake [6, 11]; (ii) underlying disease states that lead to higher angiotensin II levels and subsequent thirst and polydipsia [4, 11]; (iii) non-adherence with prescribed fluid intake leading to chronic volume overload and maladaptive cardiac changes (e.g. left ventricular hypertrophy and fibrosis)[17, 19, 20]; and (iv) higher ultrafiltration volumes/rates leading to intra-dialytic hypotension, myocardial stunning and cardiac remodeling [16, 19, 42]. However, our observed dysnatremia–mortality association persisted in multivariable-adjusted analyses that accounted for many of these confounders, suggesting that altered serum sodium may be directly harmful in hemodialysis patients.

Our study's strengths include its examination of a large, nationally representative cohort of incident hemodialysis patients, availability of repeated measures of pre-dialysis sodium that were measured in the ambulatory setting and analyzed in a single laboratory, granular data on comorbidities, dialysis treatment characteristics and laboratory tests, and a relatively long follow-up period over which to observe outcomes. However, several limitations of our study bear mention. First, included patients were required to have at least one serum sodium value, and while the indications for which sodium measurement within the study population cannot be ascertained, this was likely at the discretion of medical providers. While Supplementary data, Table S1 shows that included patients had a higher baseline comorbidity burden compared with excluded patients, which may impact generalizability, Supplementary data, Table S2 suggests that likelihood of sodium measurement was associated with various favorable baseline characteristics (e.g. higher residual urea clearance and serum albumin levels). While these differences among included versus excluded patients and those who did versus did not have sodium measurement may limit the study's generalizability (external validity), given that the criteria for sodium measurement applied equally to patients across all sodium level categories, they are unlikely to impair the study's internal validity (i.e. generate differential bias). Second, while we were able to adjust for a large number of confounders of the sodium–mortality association, we were unable to account for dietary factors (sodium and fluid intake), dialysate sodium and potassium concentrations, and membrane flux due to a high proportion of missingness or lack of data, which may have resulted in residual confounding. Third, we did not have information on cause-specific death (e.g. cardiovascular, infection). Lastly, given the observational cohort study design our findings do not confirm a causal association between dysnatremia and mortality risk in hemodialysis patients, and further studies are needed to determine if hypo- and hypernatremia are markers of mediators of mortality in this population. In summary, our study shows that both hypo- and hypernatremia are associated with higher death risk in incident hemodialysis patients. Further studies are needed to determine the mechanistic pathways by which altered serum sodium is associated with end-organ damage and mortality in hemodialysis patients. At this time, studies examining the impact of dietary (i.e. fluid and solute intake) and dialytic interventions (i.e. dialysate sodium concentration) upon the dysnatremia–mortality association in hemodialysis patients are urgently needed.

CONFLICT OF INTEREST STATEMENT

J.C.A., A.N.A. and K.K.-Z. have received honoraria from Otsuka. J.J.S. has received research funding from Otsuka. None of the other authors declare any relevant conflicts of interest.

ACKNOWLEDGMENTS

Portions of these data have been presented as an abstract at the National Kidney Foundation Spring Clinical Meeting, May 26–29, 2015, Dallas, TX, USA.

The authors are supported by the research grants from the NIH/NIDDK including K23-DK102903 (C.M.R.), K24-DK091419 (K.K.-Z.), R01-DK09568 (R.M. and K.K.-Z.), R01-DK078106 (K.K.-Z.), R01-DK096920 (C.P.K. and K.K.-Z.), U01-DK102163 (K.K.-Z. and C.P.K.), UL1 TR000153 (D.V.N.), and philanthropist grants from Mr Harold Simmons and Mr Louis Chang.

REFERENCES