Multidisciplinary perspectives on cumulative impact assessment for vulnerable communities: expert elicitation using a Delphi method

Abstract

Prompted by a series of executive orders, the U.S. Environmental Protection Agency (USEPA) is promoting cumulative impact assessment (CIA) to integrate numerous factors that have the potential to impact community health, which include nonchemical stressors such as socioeconomic conditions, pre-existing health conditions, and many others that historically have not been addressed by USEPA’s chemical risk assessment paradigm. Understanding the cumulative impact of all stressors on responses to environmental exposures requires multidisciplinary input from social scientists, economists, and others not traditionally involved in chemical risk assessments. To gather input from these disciplines, a group of 13 independent experts with perspectives on CIA as a social scientist, economist, public health expert, or decision analyst participated in a virtual workshop to obtain their perspectives regarding key aspects of CIA. The independent experts, who have decades of experience studying cumulative impacts in vulnerable population groups and environmental justice (EJ), responded anonymously to charge questions specific to their expertise and then were asked to review and comment on other’s responses within and outside their discipline. The questions and responses were organized by the authors into general topics (e.g., screening tools and indexes, role of nonchemical stressors in cumulative impacts, uncertainties), and discussions across and within the four disciplines were summarized by the authors. The expert’s opinions were used to frame a set of future research objectives to advance the development of CIA and improve its use in the EJ context. Specifically, the experts’ recommendations addressed the need for regulatory impact analysis, the appropriate use of screening tool information and indexes, the role and measurement of nonchemical stressors, relevance of a risk modifier approach to CIA, inclusion of uncertainty and causality, metrics to assess effectiveness of interventions, and methods for community communication.

Background

The Biden Administration’s Executive Orders on environmental justice (EJ; Executive Orders in 2021 [13985, 14091, 14008] and 2023 [14096]) have ushered in a more comprehensive approach to evaluating cumulative impacts. Cumulative impact assessment (CIA) is the framework for assessing health impacts that “requires a systematic approach to characterize the combined effects from exposures to both chemical and non-chemical stressors over time across the affected population group or community. It evaluates how stressors from the built, natural, and social environments affect groups of people in both positive and negative ways” (USEPA, 2022). The U.S. Environmental Protection Agency (USEPA) has developed the framework depicted in Figure 1 to identify the key factors related to CIA and is pursuing research objectives to provide a scientific foundation for CIA to support health risk-based environmental regulatory decisions in the future (USEPA, 2022).

It is important to note that USEPA also uses the term “cumulative risk assessment” (CRA), which is defined as “an analysis, characterization, and possible quantification of the combined risks to health and/or the environment from multiple agents and/or stressors” (USEPA, 2023). In their definitions, USEPA has not clearly distinguished CRA from CIA. However, CRA focuses on quantifying risks and focuses on combined chemical exposures.

Cumulative impact assessment ultimately requires integrating many factors that have the potential to impact community health, including chemical stressors and nonchemical stressors, such as socioeconomic and psychosocial stressors, systems biology and pre-existing health conditions, lifestyle and cultural influences, food and health care access, ecosystem services, and climate change influences. This has resulted in the need for disciplines that are not typically represented in planning and execution of human health risk assessments to provide important perspectives when attempting to address the nexus of risk domains (biological, behavioral, social, and chemical) potentially contributing to cumulative environmental impacts.

Purpose and objectives

With an objective of beginning to develop multidisciplinary scientific perspectives regarding key aspects of CIA, a panel of independent experts was formed from the disciplines of social science, economics, public health, and decision analysis/communication to participate in a structured expert elicitation process. These are not the only disciplines that may have relevance to CIA, but they were selected based on their relevance to some of the first-order issues faced by the current early stage of CIA development, such as how to use CIA in decision-making, how to address socioeconomic and other nonchemical stressors, how to combine chemical and nonchemical stressors into a characterization of cumulative impact, balancing net impacts, defining disparity in impacts, and effectively including a community in the CIA process. The selected experts have decades of documented experience in the approaches, limitations, and uncertainties related to assessing cumulative environmental impacts, including impacts to vulnerable population groups. The elicitation focused on understanding how approaches and techniques from those fields of study, as well as future research in each discipline, could benefit the development of CIA. Because this is an initial effort, the panel does not include all of the many disciplines that may be relevant to assessing cumulative impacts. It is limited to certain primary disciplines with perspectives relevant and important to CIA but not usually included in standard environmental health risk assessments. This panel did not include community representatives who provide important expert perspectives based on lived experience, which are critical to improving CIA. Input from community members and other stakeholders would be a useful endeavor for another study. The results of this study are part of larger efforts to integrate multidisciplinary information and are not intended to be the only evaluation related to these disciplines or cumulative impact assessment in general.

Expert elicitation process

An expert panel was recruited for a virtual workshop and engaged using the methods described in Kirman et al. (2019) with specific roles defined for the review sponsor, review manager, and independent expert panel members. Multiple design elements were included in this review to minimize potential sources of bias and groupthink and to improve the review’s transparency. These elements include: (1) a hybrid-blinding process, which includes components of both single- and double-blinded design, was adopted for panel recruitment and engagement to minimize potential participation bias; (2) the identities of experts were masked (e.g., labeled as Expert 1, Expert 2, etc.) during all online deliberations; (3) a multiround format was adopted to collect both independent and deliberative input from the topic experts in an effort to minimize potential groupthink; (4) panelists were specifically asked if there were any issues in the review material that warranted attention and discussion by the panel in an attempt to minimize potential scope bias associated with charge questions that are too narrowly focused; (5) individual responses and comments from the panelists were recorded to provide precise measurements for the degree of consensus, ensure transparency, and minimize potential reporting bias; and (6) although individual responses are provided, they are attributed to each panelist’s anonymous display name (e.g., to Expert 1, Expert 2, etc.) rather than to specific panelist identities in an effort to provide psychological safety (i.e., scientists should feel free to express their scientific opinions without fear of negative repercussions). This type of review process proved to be so robust that it was used to support a cancer weight of evidence decision by USEPA for 1,3-dichloropropene (USEPA, 1990). The formal elicitation process used is depicted in Figure 2 and summarized below.

Panel recruitment

The objective of identifying panel experts was to obtain perspectives on certain key issues from a relatively small group in four disciplines not typically involved in human health risk assessments: (1) environmental economics, (2) social science, (3) public health policy, and (4) decision analysis/communication. Importantly, the preferred experts should have a depth of knowledge and experience in areas germane to CIA for vulnerable communities with potential EJ issues.

The expert panel was recruited by casting as wide a net as feasible to obtain a pool of potential candidates. Potential candidates were identified as having relevant experience by using a variety of sources, including (1) searches of an internal database of experts managed by SciPinion, (2) searches for authors of relevant publications between 1994 and 2024 on the topics of interest in online databases (e.g., PubMed, Google Scholar), (3) searches of profiles on social media databases (e.g., LinkedIn), and (4) referrals (e.g., all invitations also included a specific request to please forward the invitation to anyone with relevant expertise). An email invitation was sent to potential candidates, requesting interested candidates to volunteer and to upload a copy of their curriculum vitae (CV) and provide a qualification statement for the advertised assignment.

Over 300 people responded to the email invitation. Twenty applicants were excluded for failing to upload their CV. A single-blinded approach was used to select the panel based on a summary table of the remaining 311 applicants, which included the applicant’s name, demographic data (e.g., country, sector of employment), expertise metrics (degree, years of experience, number of publications), application statement in which the applicants indicated their experience in one or more of the four discipline areas in the context of assessing cumulative health impacts in vulnerable communities, and full CV. The application statement in particular and their CV provided information on their relevant experience. To prevent possible selection bias, no preview of their opinions was requested or reviewed as part of the selection process.

Those with backgrounds that appeared to be general in nature and not directly related to CIA and vulnerable communities were removed as candidates. Examples include researchers focused on stress on plants, general environmental science, communicable disease prevention, clinical drug therapy, and broad environmental risk assessment. Those with no apparent experience with impact/risk assessment or EJ relevant to the United States were also removed as candidates. After this process, 32 candidate experts remained. These were subjected to the following selection criteria:

• Include at least two representatives from each of the four disciplines.

• Exclude candidates from regulatory agencies, as they are likely already invested in tools and approaches to CIA.

• Exclude candidates from regulated industry, to avoid bias influenced by regulatory burden.

• Include experts with depth of direct knowledge and experience in assessing environmental impacts on public health, approaches and issues of susceptibility and impact disparity, assessments in support of regulatory decisions, or characterizing and communicating impacts to decision-makers and the public.

• Prefer candidates with more years of experience.

• Prefer candidates with more relevant subject matter of publications in recognized peer-reviewed journals, books, or formal reports.

• Prefer expertise reflected in committee appointments, professional society positions, or awards.

Based on this information and consideration of project scope, expert availability, and resource limitations, 13 experts were selected by the authors (the project sponsor was not involved in selection) to cover four disciplines relevant to CIA: two environmental economists, three social scientists, five public health experts, and three decision analysis/communication experts. Coincidentally, four of the 13 experts also were medical doctors. This expert panel represented more than 400 person-years of experience (average = 31 years), with an average number of 157 articles published per expert.

Panel engagement

Each expert was provided with a set of review materials that included key documents (provided in online supplementary Table S1) and charge questions (provided in online supplementary Table S2) based on key issues and uncertainties identified by a comprehensive evaluation of literature on frameworks, methods, and metrics for CIA of vulnerable communities (Rish et al., 2024). Charge questions were developed for each discipline (i.e., economics, social science, public health, and decision analysis/communication) and were related to the main topic areas shown in Figure 3. The expert elicitation process occurred from August to October 2023.

Panelists were also given the opportunity to request access to additional publications/reports to support their review as needed, and many panelists provided their own references as part of their discussion. Some of the charge questions were repeated for more than one of the four disciplines. Panelists were assigned generic identifiers (Expert 1, Expert 2, etc.) to maintain anonymity throughout the elicitation process. The experts came from the following disciplines:

• Experts 1 and 2 were environmental economists,

• Experts 3 to 5 were social scientists,

• Experts 6, 7, and 9 were from the field of public health,

• Experts 11 and 13 were from the fields of decision analysis and risk communication, and

• Experts 8, 10, and 12 had expertise in both public health and risk communication.

The panel engagement was designed using a modified Delphi format consisting of three rounds, as follows.

Round 1—Each panel member was tasked with independently reviewing materials and answering charge questions specific to their discipline. Panelists were also invited to respond to charge questions in other sections at their own discretion; therefore, the number of respondents for each section varied. All 13 experts submitted their responses to their assigned charge questions.

Round 2—Using their generic identifiers, the panel members were tasked with anonymously commenting on and debating each other’s answers to Round 1 charge questions across all disciplines. A total of 105 comments were received, with all 13 experts participating.

Round 3—The panel members were asked to revisit their responses and prepare a summary of recommendations for future research based on a second set of questions.

Discussion of expert panel responses

A table of the charge questions by discipline and topic area are provided in online supplementary Table S2, as are the complete responses from the panelists (see online supplementary Appendix A). The discussion below represents a summary of responses across disciplines organized by common issues discussed by the panel members in answering the charge questions. In some cases, direct quotes are provided, and in other cases the comments and recommendations are generally summarized. These responses represent the opinions expressed by the expert panel, independent of the authors of this paper.

1. Communication and community engagement

A fundamental feature of CIA is to engage the community throughout the CIA process. We asked the experts for examples of best practices for including the public in the CIA process. Each of the experts provided similar references, but Expert 10 ultimately concluded after secondary review that there was no real disagreement across commenters. However, citing the example of 5G transmission towers, Expert 10 acknowledged that many processes do not allow for relevant public comments or objects to really be incorporated into the process.

As provided by Expert 12, Payne-Sturges et al. (2018) reviewed seven USEPA-funded projects on nonchemical stressors and community engagements and suggested the following guidelines:

1. Provide for flexibility in the mode of community engagement.

2. Address conflict between research timing and engagement needs.

3. Develop approaches for communicating about the uniquely sensitive issues of nonchemical stressors and social risks.

4. Encourage the evaluation of community engagement efforts.

Expert 12 also provided an example framework from Health Canada.

1. Strategic risk communications are integral to integrated risk management.

2. Stakeholders are the focal point.

3. Decisions are evidence-based, tapping both social and natural sciences.

4. Risk management and risk communications processes are transparent.

5. The Strategic Risk Communications Process requires continuous improvement through evaluation.

Expert 10 also recommended systematic reviews of risk communication by Zipkin et al. (2014) and Frank et al. (2012). Another expert recommended a review article by Balog-Way et al. (2020).

Expert 13 and other experts identified USEPA’s Community Advisory Group process as a potential framework for developing a working relationship with the community (USEPA, 2024). Expert 10 recommended guidance published by a United Kingdom charity called “Involve” (Involve, 2024) and that an international reference is the OECD’s Guidelines for Citizen Participation (OECD, 2022). Expert 12 emphasized the importance of understanding the community concerns and addressing them with understanding and engagement.

In general, recommendations from the panel for communicating relative risk were as follows:

• Clear and concise language free of jargon

• Clear visuals, such as charts, graphs, and maps

• Message tailored to the audience

• Use of multiple communication channels

• Transparency and openness

• Acknowledgment of scientific limitations

Another recommendation is to identify trusted messengers who have ties to the community to provide some direct information through a video or podcast. Local representatives and/or community organizers can be used to vet outreach materials to verify that they convey the intended message.

Expert 10 recommended reports on the specific burden of illness attributable to air pollution analyzed by the Institute for Health Metrics and Evaluation (IHME) for every country (data availability permitting) every few years as an example of relevant communication as well as a model of performing and communicating such analysis (https://www.healthdata.org/research-analysis/health-risks-issues/air-pollution). Expert 12 identified IHME as a reference for analysis methodology as well.

Another example provided by Expert 12 comes from the COVID-19 vaccination campaign in Mississippi. Minority population civic groups were invited to a historically Black college and university’s (HBCU’s) urban campus. They were asked to interact with health care professionals from the Centers for Disease Control and Prevention (CDC) and state health departments. They were updated and educated on the progress of such efforts, particularly addressing the trust issues and improvements over the years to manage ethical violations, and the legal safeguards in place. This approach resulted in African Americans having higher vaccination coverage in Mississippi than other races, except Asians, with vaccination rates at 77%, 52%, 46%, 42%, and 25% for Asians, African Americans, Whites, Hispanics, and American Indians/Alaska Natives, respectively (Ghosh et al., 2022).

Communities inherently have community-specific knowledge that can guide the CIA process. With the advent of less expensive sampling and analytical methods, the opportunities for communities to generate their own data also have increased. The expert panel provided some thoughts regarding community-based research.

• Results should be communicated in line with values and concerns gathered during the listening stage and agency mandates/missions.

• Cultural aspects (not political) of the local audience should be incorporated. Wutzke et al. (2018) is a useful resource on this topic in the health sector, although aimed at improving the utilization of research specifically for chronic disease prevention.

Expert 13 recommended that USEPA develop a website on CIA that has reference links to current CIAs, archived CIAs, and guidance for stakeholder involvement, modeling, causality, samplings, etc.

2. Cumulative impact assessment frameworks

Federal and state agencies have begun to identify frameworks and metrics, in some cases through the development of screening tools, that can inform and guide CIAs. As noted by USEPA in EJScreen, screening tools are not intended to be a CIA. The social scientists and public health experts were asked about the factors that should be considered in CIA and how to manage a process with such breadth.

In general, the experts deem the range of topics considered in various frameworks to be sufficiently broad but recognize that the process of CIA will reduce those topics to only those relevant to the people and place for the CIA, and there must be flexibility to add topics based on community input. The CIA framework should not be confined to the information provided by screening or mapping tools, which are more limited and are not based on specific community concerns. Different communities will place more importance on some factors than others (e.g., rural and urban communities), which does not allow for a priori selection or weighting.

Several experts agreed that the current science and evidence for general weighting of social/demographic or other factors is very limited and does not allow assignment of valid weights to all the components of composite indexes of potential vulnerability present in the environmental justice screening tools (Experts 3, 4, 6, and 8). Assigning specific weights to any factor in a composite index is not recommended by the experts. In fact, assuming all factors are equally weighted is also unsupported but may be the only possible approach that does not introduce an arbitrary factor. Expert 10 suggested that weighting factors should be developed with the community to reflect their priorities, but that weighting factors cannot generally reflect a diversity of circumstances. According to Expert 8, this makes moving forward in a measurable way very difficult. Expert 9 states that “they would be surprised (no, shocked) if professional bioethicist would think a score should be employed,” although Expert 3 suggests using AI approaches to develop “best measurement combinations.”

Another approach identified by Expert 7 is to categorize factors into broad categories and then score within the categories, as seen in the Ontario Marginalization Indices (Public Health Ontario, 2023). In that case, the four factors are households and dwellings, material resources, age and labor force, and racialized and newcomer populations. Expert 1 stated that, if a score must be developed, perhaps a Shapley value-based attribution of total effects to the components that jointly produce them would be useful (Algaba et al., 2019). Shapley values are a part of game theory used to evaluate the utility of participating in a game rather than simply the outcome of the game itself.

Expert 7 was concerned about allowing USEPA to consider so many nonchemical stressors that stray into social sciences. They thought USEPA should continue to focus on chemical exposures, with concurrent consideration of vulnerable populations. Expert 7 also indicated that there needs to be flexibility to add factors such as violence, green space, and food quality/availability for specific communities. Expert 6 indicated that the cumulative impact assessment measurement and output (from the current EJ mapping tools) did not appear to cover important factors such as health care, social support, loneliness, agriculture and food production/security, stress, safety, spiritual/religious factors, lifestyle, and work environment, among others. There are metrics for many potentially relevant stress factors developed by other disciplines for other purposes that could be considered. For example, the experts identified that CDC has developed an approach to measure adverse childhood experiences (ACEs; CDC, 2023).

3. Role of nonchemical stressors in interventions

Cumulative impact assessment introduces nonchemical stressors into the assessment of potential health impacts related to chemical stressors. The USEPA and other entities have developed frameworks for performing CIAs that provide a wide variety of ways to characterize the intersections of chemical and nonchemical stressors and health. The panel of experts across all disciplines was asked about the role of nonchemical stressors, particularly with respect to interventions identified by a CIA.

There was agreement across the expert panel that both chemical and nonchemical stressors should be considered in environmental regulatory decisions. There was a general consensus that there would be situational variability as to which factors should be considered and which may be most important.

The social scientists stated that if addressing nonchemical stressors offers more significant health or well-being outcomes, these interventions should be prioritized. The social scientists also emphasized the importance of community engagement in the decisions regarding interventions.

Expert 6 indicated that, on average (across time and location worldwide), 10% to 20% of the health of a community can be determined by environmental factors (not strictly chemical exposures). Expert 7 opined that “my gut reaction is that the non-chemical stressors are the most important.” Nonchemical stressors are directly attributable to health-seeking and health-promoting behavior.

Expert 3 summarized the sentiment of the social scientists stating that the proper balance between interventions aimed at nonchemical versus chemical stressors should be determined by the evidence for their independent and joint impacts, not on which government agency is involved. Expert 10 suggested that public-sector authorities should pay more attention to the socioeconomic of health, simply because they are so powerful, ubiquitous, and “protean” (i.e., affect so many different kinds of ill health).

Expert 8 was concerned that “relative expressions of risk-reduction benefits can lead to interventions that are very expensive with little absolute gain. For example, there might be a 100% relative decrease in cancer risk, but the absolute difference is 1 in 100,000.” Actions based on relative expressions of risk can result in high costs with little or no measurable benefit, particularly compared with alternative choices.

Expert 1 indicated that there are nonscientific issues, such as jurisdiction, enabling legislation, and legitimate scope of regulatory decision-making, that would play an important role in how interventions for nonchemical stressors could be implemented. If interventions for nonchemical stressors are considered, the environmental economists identified that an important priority going forward would be to consider the regulatory authority of USEPA and advocate for reform to expand its authority to nonchemical stressors, in light of evolved and improved scientific evidence. They view this as a medium- to long-term strategy that will not solve the decision-making challenges encountered in the present but will still be a necessary step for legitimate and durable decision-making based on a solid regulatory authorization. They pointed out that if USEPA or similar agencies do not have the mandate for such holistic assessments of these combined burdens of illness, interagency task forces may be required to tackle these issues effectively.

4. Using modifiers to account for nonchemical stress

Because the quantitative relationship between chemical and nonchemical stressors is currently largely uncertain, some researchers have suggested that chemical dose-response modifiers, such as dividing a reference dose by some factor or using a lower risk goal, may be appropriate in the near term to provide an extra level of protection for vulnerable communities to account for additional susceptibility related to nonchemical stressors. The economists, social scientists, and public health experts were asked whether there is a basis for supporting the modifiers.

Experts 1 and 2 did not favor the response or risk goal modifier approach but rather advocated that nonchemical stressors should be analyzed as separate independent variables because they are intrinsic to the more obvious effects of chemical and physical exposures. The social scientists recognized that advancement along all assessment stages for nonchemical stressors is imperative because to incorporate these stressors, we need a deeper grasp of how social stress influences disease and dose-response. Also, forcing multiple nonchemical stressors into a single response modifier may downplay the role and importance of the individual nonchemical stressors relative to the chemical stressors for guiding regulatory decisions. However, some experts considered using response modifiers a reasonable, immediate solution to a complex problem.

It was recognized that using a response modifier forces nonchemical stressors to fit into USEPA’s statutory mandate and reduces the risk of legal or political challenges to regulatory decisions. Rather than a one-size-fits-all approach of using a single response modifier, one social scientist indicated that only those nonchemical stressors causally related to the specific health outcomes anticipated should be considered.

Public health experts differed in their response to this question. Expert 9 agreed with using a response-modifier approach, whereas Expert 10 indicated “there is no a priori generalization which is appropriate,” and empirical data are necessary to support any role of nonchemical stressors on health (e.g., modifier, mediator, or independent causal driver). Termed “mediation/moderation analysis” (VanderWeele, 2016), empirical methods have been developed and could sort out whether two or more exposure/risk factors influence the risk of a given health outcome (1) independently, (2) by interacting with each other, or (3) something in between.

5. Uncertainty and causality

Cumulative impact assessments include assumptions that are subject to uncertainty. In typical chemical risk assessments, uncertainties are addressed by using toxicological uncertainty factors, reasonable maximum estimates of exposure, and other precautions so as not to underestimate risks. Cumulative impact assessments introduce a new set of uncertainties to the assessment process when accounting for interactions between chemical stressors, nonchemical stressors, and other cumulative factors. The social scientists and decision analysis/communication experts were asked to discuss how uncertainties in CIA should be addressed.

The social scientists recognized that significant uncertainty is related to the associations assumed between stressors and health impact. Expert 4 indicated that “relying solely on statistical associations may not always be in the best interest of society, especially if the interventions derived from them prove ineffective.” Conversely, Expert 4 points out that not building on these associations could lead to misallocated resources, unforeseen repercussions, and public disapproval. Their consensus opinion is that decision-making should always reflect the potential severity of impacts where causality is not fully established. If causality is uncertain and potential or worst-case impacts are mild to moderate, these impacts should not carry the same weight in decision-making as stressors whose causality is in question but the potential impacts could be severe. In addition, Expert 5 noted that “robust, multi-disciplinary evidence” should remain at the center of decision-making to avoid solely ideological debates. The social scientists suggested that to determine the extent to which uncertainties can be tolerated, decision-makers should evaluate consequences; consult with stakeholders; learn from past decisions; evaluate the appetite for risk; develop contingency plans for the unexpected; and consider ethical, financial, and legal implications.

The panel experts mentioned several methods for understanding uncertainties in causality and correlation/covariance among stressors. They recommended:

• Transparent, qualitative description of uncertainties; weight-of-evidence approaches may be a best first step

• Monte Carlo analysis

• Data collection based on stressor combinations

• Including EJ variables as class variables or covariates in a regression model

• Epidemiological methods for modeling causation in the presence of more than one relevant exposure/risk factor, termed “mediation/moderation analysis” (VanderWeele, 2016)

Experts in the risk decision analysis/communication disciplines recommended the following approaches to characterize uncertainty and evaluate its implication on decisions.

• A seven-step expert elicitation process is recommended in Knol et al. (2010).

• Directional acyclic graphs (DAGs) can be used to inform causal structures, confounders, effect-measure modifiers, and other causes of exposures and effects.

• Uncertainty in data can be characterized or quantified by a probability distribution, such as a probability density function, which is assigned based on information, such as expert opinion or empirical evidence about the likelihood of the actual value (Lin et al., 2014). Probabilistic modeling (e.g., two-stage Monte Carlo) can be used to tease out the contributions of the variable vs. uncertain chemical and nonchemical stressors.

• Value of information (VOI) analysis may be useful for addressing uncertainties (Tuffaha, 2021). Value of information compares the expected monetary benefits and costs of research studies.

• Causal pie modeling approach that attributes diseases to multiple pathways (similar to an exposome approach) may be used (Chen & Lee, 2018).

• The biostatistical approach to this issue is to use inference testing and confidence intervals around each of the quantified outputs of risk analysis used, whether they represent local populations’ precise excess burdens of illness attributable to particular exposures/risk factors or the most cost-effective interventions for remediation.

Some were concerned that characterization of impacts might conflate association and causation, and Experts 1, 2, 10, and 13 acknowledged the utility of Hill’s (1965) criteria for evaluating causation. Expert 10 opined:

“The challenge in environmental health studies of the typical epidemiological designs (cohort and case-control) is that they often lack accurate individual-level measures of environmental chemical or physical exposures, relying instead on single area-level estimates (e.g. of air pollutant concentrations) for all subjects in entire local subpopulations. This has known weaknesses, especially exposure misclassification effects, typically diluting observed associations’ RRs away from the null. More methodological innovation is needed to overcome these challenges, including the use of new biomarkers for ‘absorbed dose’ of the toxicant.”

However, Expert 1 indicated that some recent commentators have considered the Bradford Hill considerations to be underpowered for addressing interventional causation (Ioannidis, 2016; Shimonovich et al., 2021). Expert 10 concurred with Expert 1’s opinion but indicated that “modern epidemiologists have overcome these limitations using more recently developed methods such as mediation/moderation analysis.”

6. Health, well-being, and quality of life measures

The USEPA’s current definition of cumulative impacts references “effects on health, well-being, and quality of life outcomes” (USEPA, 2022). In theory, interventions to address cumulative impacts should be measured as improvements to these three metrics, but these metrics are not simple to measure. The economists, social scientists, and public health experts were asked to provide definitions and useful measures for these terms.

Expert 5 cautioned against using broad definitions, preferring that specific quantitative, reproducible metrics be developed for any measures of effectiveness or current conditions. They believe that generalized, vague definitions increase uncertainties and reduce predictability of regulatory decisions.

A set of criteria for all three metrics was provided by the public health experts with concurrence in some cases from the economists and in general by the social science experts in subsequent discussions. Expert 10 commented that these metrics are easily understood by community representatives and nontechnical decision-makers, and the work of the Institute of Health Metrics and Evaluation (Institute for Health Metrics and Evaluation, 2024) has gained widespread acceptance of these types of metrics. However, as Expert 7 points out, some of the metrics (e.g., mortality) would not allow for timely measurements of benefits.

• Health—Expert 4 provided three definitions of health: lack of disease or physical/mental defect, capacity of an individual to meet the demands of everyday life without disease or impairment, and an equilibrium within the individual and between the individual and their social and physical surroundings. Expert 6 provided the following measures of health: years of life lost (YLLs), quality-adjusted life-years (QALYs), and disability-adjusted life-years (DALYs). Expert 1 recommended using health-state-dependent utility functions for health risks and income and social utility functions and social preference analyses, including attitudes toward health and wealth inequalities (Attema et al., 2023; Finkelstein et al., 2009; Viscusi, 2019).

• Well-being—Expert 3 described three dimensions to well-being—physical, mental, and social—and identified a possible measure for practical use in the NHANES 18-item general well-being scale (National Libraries of Medicine, 2023). Public health experts described well-being as the state of happiness, contentment, and fulfillment, which could be measured by a subjective well-being (SWB) happiness index and mental health indicators.

• Quality of life—Expert 5 described quality of life as the “individuals’ perception of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards, and concerns.” The CDC has a measure for health-related quality of life (HRQOL), which is a 14-item healthy days measure that assesses perceived health status and activity limitation (CDC, 2000). The CDC’s measure appears to focus on the health aspect of quality of life. Expert 6 suggested using a human development index, environmental quality index, and social indicators for quality of life.

As described by the social scientists, the metrics for these three criteria are somewhat interrelated. For CIAs in environmental contexts, the focus on health aspects for well-being and quality of life is consistent with the purpose of a CIA. Whereas YLLs, QALYs, and DALYs are objective measures, the measures for well-being and quality of life are subjective and individual, which could be much more difficult to administer for a specific community because of privacy considerations, participation levels, and temporal considerations.

Expert 2 discussed whether the goal of the CIA as estimating the average treatment effect (ATE), conditional average treatment effect (CATE), distributions of heterogeneous treatment effects (HTEs), weighted average treatment effects, or something else (Huntington-Klein, 2022). Expert 1 believed that HTE measures of how an intervention would change the distribution of individual-level risks could be a good benchmark to use. Those metrics based on average effects may miss the unique impact on various groups that HTE approaches are intended to identify. In addition, there is research on fairness metrics, if those were to be added (Xinying Chen & Hooker, 2023). This expert also recommended social welfare functions (SWF) and social utility functions (SUF; Adler et al., 2021; Attema et al., 2023), which could be used to measure overall benefits.

7. Assessing benefits and equity

From an economic and regulatory perspective, it is typical to assess the benefits of a regulatory action relative to the costs. However, for CIA this becomes complicated when benefits may be hard to measure and distributed more broadly than the costs (e.g., a city having a recycling center is good for the entire city, but any associated burden is borne by a local population).

Expert 1 suggested the use of social welfare function and social utility function models that include equity concerns could be extended to address this question (Khmelnitskaya, 2002). Exposure models for aggregate (e.g., spatially averaged) exposure concentrations have been validated, including cross-validation analyses of exposure models. The difference between average exposures and individual exposures is typically ignored (average exposures are used as proxies for individual exposures without using errors-in-variables techniques to address the differences between them).

Expert 1 also indicated that it is an untested assumption that proportional hazards (PH) regression coefficients and hazard ratios can be used to predict effects of interventions. This unvalidated assumption is actually contrary to much methodological work in mathematical epidemiology and statistics on “collapsibility” for PH models, which has established that PH model coefficients and hazard ratios do not have a causal interpretation in general.

The social scientists considered that the goal should be equitable impacts for all places, even when those places comply with the set criteria and standards designed for human health protection. Expert 4 stated that “simply meeting a benchmark does not guarantee that every individual or community is protected equally,” although it should be noted that health benchmarks are designed to be protective of sensitive groups. When communities thrive in healthier environments, the broader society enjoys reduced health care expenses and elevated productivity—for example, a reduction in disease rates, directly correlating to fewer illnesses or health conditions linked to environmental exposures. This leads to tangible economic gains, as communities save on reduced health care costs and benefit from increased productivity. Moreover, there is a potential for an uptick in property values in areas that were previously affected by pollutants. However, increased property values are a double-edged sword because they can result in gentrification, which displaces the original community. Another measurable benefit is the decrease in mortality rates attributed to specific environmental hazards. If successful, a decrease in mortality rates is further underlined by reduced frequency of health care visits, with fewer hospitalizations and medical consultations tied to conditions induced by pollution.

Expert 3 agreed that it is appropriate to strive for equitable impacts for all places. Even if this goal remains elusive for a long time to come, they believe we must work toward it, and should not accept that it is somehow permissible for some communities to experience greater harm or worse stressors than others. Equity can be a broader term that considers whether those disproportionately affected made a conscious choice to accept employment in and move to a specific location prone to greater pollution impacts, are compensated adequately for the increased risk, and benefit from targeted efforts by government agencies to limit or minimize such impacts, even if they cannot be altogether avoided.

Expert 4 opined that actively prioritizing community benefits in regulatory decisions can cultivate and reinforce trust between the community and regulatory authorities. By directly involving the community in the decision-making process, not only do interventions become more tailored and effective, but a sense of collective ownership and trust is fostered.

8. Shifts in the burden of environmental impacts

Environmental conditions and exposed population groups can shift with time and also geographically. The USEPA identifies a need to “include consideration of trade-offs or shifts in the burden of pollution across time and space” (USEPA, 2022). The environmental economists were asked how to consider shifts in burdens.

Expert 2 opined that balancing and prioritizing resources is important when comparing chemical factors (e.g., air pollution, water pollution) and other conditions related to health (e.g., physical insecurity, food insecurity). They raised the idea of an EJ bond market (like an emission trading system) to distribute the costs of maximizing EJ, although such a market would require a local component where local costs and benefits would be considered. Similarly, redistribution schemes, such as those used to recycle revenues from carbon pricing, could be used to target the benefits of activities with economic benefits to the communities disproportionately affected by the impacts of those activities. This is the approach used in California’s AB617 (California Air Resources Board, 2024).

Expert 2 said that from an economic perspective, there are specialized methods for traffic/transportation models, network analysis, land, rent, and land economics theory that could support a cradle-to-grave assessment of human health and environmental consequences. Partial and general equilibrium analyses can be used to model temporal changes in supply and demand and the effect of economic and production constraints. Social utility function models could also address shifts in burden (Attema et al., 2023). However, Expert 1 points out that idealizations made in SUF analyses are often far removed from realities in which people may care less about whether everyone is made better off by a change than about whether the differences between their own benefits and those of others increase or decrease (Loewenstein et al., 1989).

9. Measures for economic impacts of stressors and health burden

In their research recommendations, USEPA has identified a need to “develop and apply economic methodologies, such as non-market valuation, to measure economic impacts of stressors and health burdens and to assess the distribution of these damages” (USEPA, 2022). The environmental economists were asked about methods and metrics that are available or could be developed to address this need.

The following six nonmarket methods were recommended by Expert 2 with concurrence from Expert 1 (see detailed discussion from Expert 2 in the online supplementary material).

1. Contingent valuation method (CVM) elicits answers from respondents through questionnaires.

2. Contingent ranking method (CRM) is the direct elicitation of attitudes determined through individual responses to pictures or photographs of the natural resource being assessed.

3. Travel cost method (TCM) consists of determining, through surveys administered to visitors or potential visitors, the cost of travel from several zones near the areas of interest to those visitors. The TCM is used to estimate the parameters of the individual demand function for the tradeoffs made by those visiting a park or other area.

4. Hedonic price method (HPM) attempts to place a monetary value on public goods and services through the price of other observed goods.

5. Production function method (PFM) measures the amount of goods and services produced by the environment and then places a value on that output.

6. Damage functions using mechanisms and data from computer simulations of epidemiological cohort studies assess exposures.

Expert 1 offers a caveat on these options because “behavioral economics has emphasized that the hypothetical nature of the choices and responses to many of these methods, the sensitivity of responses to ‘supposedly irrelevant’ factors, details of framing and elicitation methods, and the frequent insensitivity of responses to normatively relevant factors, make straightforward interpretation and use challenging.” Expert 2 agreed with Expert 1’s comments.

10. Need for regulatory impact assessment

The USEPA and many state agencies have invested a great deal of resources into formulating approaches to CIA and integrating CIA into regulatory programs. However, to date, a regulatory impact assessment (RIA) has not been performed to assess the costs and benefits of using or requiring a CIA as a basis for regulatory decisions. The authors asked the two environmental economists whether and how an RIA should be performed.

Collectively, the environmental economists agreed that an RIA, or something like it, should be used to explicitly evaluate the net benefits of using CIAs as a basis for USEPA’s decisions. The critical assumption that underlies Expert 2’s response to this question is that “there cannot be environmental benefits without at least some costs…, and both costs and benefits—tangible intangible, direct, and indirect—should normatively be identified, qualified, quantified to the extent possible, and fully evaluated.” They posit that not all USEPA decisions are equivalent, and there may be a threshold above which the costs of requiring a CIA outweigh the benefits. There was consensus among the economists that these questions should be addressed before adopting comprehensive CIA as a regulatory requirement for all decisions.

The economic experts indicated that an RIA would include baseline analysis, regulatory alternatives, consequences, quantification and monetization of costs and benefits, discounting, evaluation of nonquantified and nonmonetized benefits and costs, and the characterization of uncertainties. The effort and cost of developing CIAs may be expensive, and an RIA would identify when the effort and cost is balanced with the subsequent benefits. Cost-effectiveness (CE) analysis and cost-benefit analysis (CBA) are integral to the RIA effort, but there is an asymmetry between incurring costs and recognizing benefits because costs are paid before socioeconomic benefits can be achieved. As described by Expert 2:

“…costs, for instance, compliance with regulatory laws, precede the generation of benefits; moreover, they are tangible, relatively well known, and—once allocated and spent—detract from the overall societal budget. But there are other costs: capital scarcity due to increasing competition for money from large-scale (EJ-related) government expenditures may increase interest rates, pressures will occur in the labor market with the cost of labor (the wage rate) increasing, and so on. To omit those from the analysis phase of the RIA is logically and theoretically incorrect but, perhaps more importantly, it may also affect the distribution, magnitude, and equity of already existing social benefits.”

Heterogeneous treatment effect models (Velentgas et al., 2013), Qini curves (Yadlowsky et al., 2021), and coefficients from marketing science were also suggested to help understand disparities in environmentally related health impacts and the various values of interventions. Heterogeneity of treatment effect (HTE) is the nonrandom, explanatory variability in the direction and magnitude of treatment effects for individuals within a population. The main goals of HTE analysis are to estimate treatment effects in clinically relevant groups and to predict whether an individual might benefit from a treatment. Qini curves are also known as cost curves and were originally developed to measure the success of targeted marketing actions.

These tools are typically used to identify whether a treatment, change, etc., produces an effect across a variable population by accounting for the variability. This is comparable to the concept of cumulative impact assessment of identifying disparate adverse effects within a population. Several experts acknowledged the difficulties in measuring disparity but pointed out that interventions should focus on cases where the greatest relative disparities are identified.

One expert recommended the application of computational general equilibrium (CGE) modeling; CGEs currently are used by governmental organizations and academic institutions to analyze the economy-wide effects of events such as climate change, tax policies, and immigration (Burfisher, 2011).

In response to Expert 2, Expert 1 commented that:

“The kernel of most cost-benefit assessments is causation: the exposure-damage functions for different consequences. At present the EPA uses estimates of impact functions based mainly on associational (e.g., regression) models that do not address interventional causation in a technically sound, correct way (Cox, 2023). This does a disservice to those who look to the EPA for normative guidance and sound analysis in EJ and other matters. Important steps that could be taken now to improve the estimation of exposure-damage functions immediately include recent econometric methods such as Heterogeneous Treatment Effects (e.g., causal forest and causal survival forest) and interventional causal analysis.”

Conclusions and general recommendations

The expert panel’s responses to the topics, charge questions, and their virtual discussions with each other were combined to develop 12 top-line general recommendations to improve CIA methods and assessments (Table 1). Other recommendations and more detailed discussions are included in the experts’ responses to the elicitation (see online supplementary material).

The transition from traditional chemical risk assessment to cumulative impact assessment in support of environmental regulatory decisions will necessarily involve blending the knowledge and approaches from multiple disciplines beyond the four included herein. The virtual workshop approach is a structured method that was used to collect perspectives from disciplines not typically involved in regulatory risk assessment to date. It is hoped that these perspectives will begin to improve the multidisciplinary approach needed for CIA. The panel was limited in disciplines, and future efforts may need to include other perspectives such as community members, legal experts, and regulators. The charge questions were broad in nature, and future efforts may need to focus on issues relevant to a specific topic area, population group, place, or decision context being supported by a CIA. Even a limited effort to blend multiple perspectives into the design and problem formulation of a CIA will improve the relevance and usefulness of the assessment. Also, the level of effort needs to be appropriate to the decision context, resources available, and time constraints. Assessing cumulative impacts needs to be a collaborative approach among multiple disciplines and stakeholders, particularly the communities experiencing the impacts.

There currently is no scientifically supportable method or model for combining the multiple influence factors from various domains and disciplines to create a simple measure of cumulative impact or to identify the most effective combination of interventions to improve community health. In the short term, both quantitative analysis and qualitative information will need to be reviewed and evaluated together. Prioritizing consistent metrics to quantify impacts to health, well-being, and quality of life could help identify the interventions that have the greatest potential to make a difference (i.e., change the metrics). Without focus, there could be too many factors to possibly consider. One step to move the science forward would be to develop case examples evaluating, for example, health-state-dependent utility functions for health risks, the utility of DALYs, QALYs, and YLL as potential measures for CI, and SUF models to address shifts in environmental burden. Evidence of the strength of association, independently and jointly, of chemical and nonchemical stressors on cumulative impacts should be developed to identify effective and coordinated ways to improve community health that are not predicated solely upon the authority of the lead regulatory agency.

Supplementary material

Supplementary material is available online at Integrated Environmental Assessment and Management.

Data availability

Data, associated metadata, and calculation tools are available through supplemental files.

Author contributions

Ann Verwiel (Investigation, Project administration, Writing—original draft, Writing—review & editing) and William Rish (Conceptualization, Funding acquisition, Methodology, Writing—review & editing)

Funding

This work was supported by the Foundation for Chemistry Research and Initiatives.

Conflicts of interest

None declared.

Disclaimer

The workshop was structured to request the participants’ scientific opinions and written discussion based on evidence from their research and knowledge and did not inquire about personal information.

Acknowledgments

We’d like to thank the members of the independent expert advisory panel whose responses informed this research.

Hafiz Ahmad, PhD

Michael Mehling, JD, PhD

Howard Bauchner, MD

Paolo Ricci, JD, PhD

Carol Burns, PhD

David Rueda, MPH, MD, PhD

Tony Cox, PhD

Linda Teuschler, MS

John Franke, MD

Douglas Weed, MPH, MD, PhD

Michael Hendryx, PhD

Adam Zwickle, PhD

Dean Kyne, PhD

We would also like to thank Chris Kirman and Sean Hays of SciPinion for administering the expert elicitation process.

References