A scoping review of outer context constructs in dissemination and implementation science theories, models, and frameworks

Abstract

Many studies have explored organizational factors that facilitate implementation. However, there is still a limited understanding of determinants external to the implementing organization and their effects on evidence-based intervention (EBI) adoption, implementation, and outcomes. The purpose of this scoping review was to assess definitions of context and identify salient determinants of outer context found in dissemination and implementation theories, models, and frameworks. We employed a compilation of dissemination and implementation frameworks from two reviews as the data source. We abstracted the following information: type of article, outcomes of the framework, presence of a context definition, presence of any outer setting definition and the definition, number and domains of outer setting mentioned, definitions of outer context constructs, and any quantitative measures of outer setting. We identified 19 definitions of outer context. Forty-seven (49%) frameworks reported one or more specific constructs of the outer setting. While the outer context domains described in the frameworks varied, the most common domains were policy (n = 24), community (n = 20), partnerships (n = 13), and communications (n = 12). Based on our review of the frameworks, more conceptualization and measurement development for outer context domains are needed. Few measures were found and definitions of domains varied across frameworks. Expanding outer context construct definitions would advance measure development for important factors external to the organizations related to EBI implementation.

INTRODUCTION

Many public health theories, models, and frameworks include context, defined as “a set of characteristics and circumstances that surround the implementation effort” [1] and are related to the adoption and implementation of evidence-based interventions (EBIs). Some contextual factors are internal to the organizations that are delivering or disseminating EBIs, while others are factors in the broader environment in which the organization functions [2, 3]. Researchers have proposed a number of factors in the fiscal and policy context that may influence uptake of EBIs [2]. The Consolidated Framework for Implementation Research (CFIR) labels external environmental factors as “outer setting”; these factors include patient needs and resources, cosmopolitanism, peer pressure, and policy and incentives [2]. Greenhalgh et al. also identified several contextual factors that play a role in disseminating and sustaining innovations in health service delivery and organizations through an extensive literature review; the outer context in their framework included social political climate, incentives and mandates, interorganizational norms and networks, and environmental stability [3].

Context plays a key role in intervention implementation and study outcomes, with study results varying based on contextual variables [4–6]. Many studies have explored factors related to the organization that lead to implementation [4]. Despite this, there is still a limited understanding of determinants external to the implementing organization and their effects on EBI adoption, implementation, and outcomes. For example, one study found that policy and financial investment in research were related to the presence of an evidence-based state policy [7]. A review of effective quality improvement (QI) efforts found that competition (or external pressures) was associated with QI success [8]. Systematic reviews have assessed measures of contextual factors influencing implementation [9, 10], with few measures of outer setting constructs found. A recent review found a limited number of measures across outer context domains [11] and speaks to the need for better understanding these constructs and the importance of operationalization for use in implementation research.

No previous study has systematically investigated context within dissemination and implementation science theories, models, and frameworks to assess how they define or describe context and the array of external contextual determinants embedded in these frameworks. The purpose of this scoping review was to collect definitions of context and identify salient determinants of outer context found in dissemination and implementation theories, models, and frameworks. Key research questions were: (a) what are definitions of outer context in the dissemination and implementation literature? (b) what are common outer context constructs in dissemination and implementation theories, models, and frameworks? and (c) what outer context measures exist?

METHODS

To address the study aims, we conducted a scoping review from May to October 2020 of theories, models, and frameworks (subsequently referred to as “frameworks”) identified in the most comprehensive published studies of dissemination and implementation frameworks to date [12, 13]. The team developed the research questions, methods, and analyses following steps for a scoping review recommended by Arksey and O’Malley: identifying the research question, identifying relevant studies, selecting studies, charting the data, and summarizing and reporting on findings [14].

Search strategy

We use the frameworks found in two reviews to guide the theories, models, and frameworks included in our search [12, 13]. Our team reviewed all frameworks included in the cited article(s) or books from both systematic reviews, the latest of which was published in 2020. Some frameworks were based on a single article, book chapter or book; however, some frameworks had several sources. We used the original or first published article for the data abstraction but added relevant information from other articles. There were seven duplicates in the two reviews and two documents that were not abstracted due to access of the materials.

Data abstraction

Each article or document (e.g., book chapter) was reviewed by two of our four reviewers (CE, SS, LM, and CA). We conducted data abstraction in a systematic process into an Excel spreadsheet. For each framework, we abstracted the following information: type of article (i.e., conceptual, cross-sectional, intervention, other), outcomes of the framework, presence of a context definition, qualitative definition of context, presence of any outer setting definition, the qualitative definition of outer setting, number and domains of outer setting mentioned in the framework, definition of outer context setting constructs, and any measures of outer setting both qualitative and quantitative. In this case, outer context refers to factors external to the implementing organization related to implementation.

For article or document type, “conceptual” ones were those that described the framework in general terms and simply explained relationships between constructs. Otherwise, they were classified by their study design, “Cross-sectional,” “intervention,” or “other.” Other includes qualitative study or case study. For frameworks that reported any outer setting constructs, we assessed if definitions were explicitly stated, if so we abstracted definitions and constructs described verbatim. We noted if the framework article had any measures for outer setting constructs and described the measure. The first abstractor read the article and abstracted the data into the Excel database. The second abstractor read the article and made additional comments and/or changes in another color from the first review. Consensus was reached by both abstractors.

For constructs related to determinants external to the organization, we examined the list of framework constructs across the studies, we then mapped or grouped the constructs into similar categories (i.e., policy, patient/community, partnerships, social, etc.) and mapped the constructs found in each study. Table 1 presents a working definition of each of the common outer setting definitions from the review. We then counted the numbers across the studies that reported outer context constructs. In addition, we noted if there was a measure of outer context and documented any qualitative or quantitative items in a separate table. Those were compiled for that description of measures.

Data synthesis

All of the information from the abstraction was combined into a final database in Excel. All studies that provided ­sufficient data regarding definitions and constructs related to outer setting were included in the analysis. Data summaries were prepared for definitions and descriptive statistics were run for number of definitions (e.g., context, outer setting, outer setting constructs).

RESULTS

Overview of frameworks

The two articles that formed the foundation of our review presented 61 implementation frameworks in Tabak and 26 of the 35 frameworks in Esmail, for a total of 87 frameworks [12, 13]. We abstracted 108 references across the frameworks. Among the 108 total references, 81 were conceptual articles or documents (describing a framework in general), 13 were cross-sectional studies, and 10 were intervention studies and 4 were other types (e.g., qualitative, empirical, case study).

Definitions of outer context

Of the 87 frameworks, 46 (52.8%) included a definition of context and 19 (21.8%) included a definition of outer setting. Supplementary Table 1 provides the outer setting definitions in the 19 papers. Common words used to describe outer context included, “context,” “environment,” “community,” “implementation process,” and “research setting.” For example, in A Conceptual Model for the Diffusion of Innovations in Service Organizations, we abstracted the following definition, “the outer (interorganizational) context, including the impact of environmental variables, policy incentives and mandates, and interorganizational norms and networking” ([3], p. 585). As another example, the CFIR describes outer setting as “the economic, political, and social context within which an organization resides” ([2] p. 4). The complete definitions for the 19 frameworks that included a definition of outer setting is described in Supplementary Table 1.

Outer context constructs

Although only 19 frameworks included a definition of outer setting, 47 (54%) frameworks contained one or more specific constructs of the outer setting. Table 2 provides descriptions of the 11 outer setting domains we found in the 47 frameworks. The range of frequency of outer context domains was 6 frameworks citing the funding and incentives domain to 24 frameworks citing the policy domain. The most common factors external to the organization were policy (n = 24, 51.1%) [2, 3, 15–38], community (n = 20, 42.6%) [2, 16, 17, 19, 22, 23, 30–32, 35–47], partnerships (n = 13, 27.7%) [3, 15, 19, 23–25, 34–36, 38, 39, 48, 49], communications (n = 12, 25.5%) [15, 34–37, 40, 42, 44–46, 50–52], and social environment (n = 11, 23.4%) [16–18, 21, 22, 30, 31, 33, 42, 51, 53–55]. Other factors included legal/regulations (n = 9, 19.1%) [2, 3, 16, 17, 20, 22, 26, 30, 31, 34, 42, 56, 57], economic environment (n = 9, 19.1%) [16, 17, 20, 21, 27, 32, 42, 50, 53, 56], profession (n = 8, 17.0%) [2, 22, 31, 34, 37, 38, 47, 56, 58], cultural environment (n = 8, 17.0%) [16–18, 24, 27, 34–36, 53], peer pressure/norms (n = 7, 14.9%) [2, 3, 27, 42, 49, 58, 59], and funding and incentives (n = 6, 12.8%) [3, 24–27, 56, 57]. Seventeen (36.1%) included other domains that did not fall in any of the previously mentioned categories. Examples of domains in the other category include technological factors [16], acquiring prestige [42], design teams [43], research interests [44], ideology [27], or a general concept of environmental factors [60, 61]. The most comprehensive framework was Damschroder’s Consolidated Framework for Implementation Research (CFIR) with named outer context dimensions and definitions [2]. Table 2 provides a detailed list of which of these outer context domains were included in which framework in addition to whether each framework is more focused on dissemination (D), implementation (I), just one or the other, one more than the other or equally both D and I. Twelve frameworks were more focused on dissemination than implementation, another 13 placed equal weight on both, another 10 focused on dissemination only. Nine frameworks focused only on implementation and three weighed implementation over dissemination.

Measures of outer context

Seven frameworks contained outer context measures: three frameworks presented qualitative measures (i.e., open-ended questions) (Supplementary Table 2) [2, 23, 48]. Three ­frameworks also presented quantitative measures [33, 57, 63], and one framework used a mixed methods data collection instrument [64]. Among the qualitative items, the domains of outer context they aimed to measure varied. Damschroder et al. reported on the highest number of constructs, consisting of patient needs and resources; cosmopolitanism, peer pressure, and external policies and incentives [2]. In addition, their CIFRguide.org website provides full interview guides. Robinson et al. focused on the broader context and policies (e.g., “In what ways have the larger national and provincial contexts influenced the design, implementation and outcomes of the project?”; “To what extent was the shape of the project influenced by provincial health reform or changes made to the health care system?”) [48]. Dreisinger et al. measured partnerships (e.g., “who are some of the key collaborators to your program” and other external challenges (i.e., “what are some of the external challenges and issues that your program has encountered”) [23]. The concept of partnerships or collaboration and external policies were most commonly integrated constructs among frameworks with qualitative measures.

The frameworks with only quantitative measures, items covered a variety of outer context constructs. Kitson et al. measured collaborative partnership (i.e., “I value collaborative partnership working”) and open communication (i.e., “I value communication and dialogue”) in their measures. Gholami’s items focused on external funding and incentives for the program and Szulanski’s had two items focused on communication and seeking assistance or sharing with those external to the organization. The quantitative measures were assessed using 5-point Likert scale response options from unfavorable to favorable or from definitely yes to definitely no. In these articles with measures, none offered information about the psychometric properties of these scales (e.g., reliability, validity).

DISCUSSION

The goal of this scoping review was to comprehensively identify definitions of the outer context, common constructs, and measures across the dissemination and implementation science literature. Overall, just over half of the examined frameworks included a definition of context and few included an explicit definition of outer setting. Furthermore, outer context is not described consistently or comprehensively in these frameworks. Additionally, we identified 11 unique domains of the outer setting, with the most common being policy. Lastly, while there were many frameworks that include outer context constructs, very few (n = 7) reported measures (both qualitative and quantitative) related to outer context.

We found a large variation in the number of outer context determinants (i.e., categories) described in the frameworks. The most commonly identified domains of outer setting included policy (51.1%), community (42.6%), partnerships (27.7%), and communications (5.5%). For the purposes of this review, policy included regulations, legislation, mandates, and/or directives that influence implementation in organizations. There are opportunities to research policy impact on EBIs adoption, implementation, and maintenance since this construct appears in the most number of frameworks. Some have found that policies provide an important implementation context for public health interventions and can facilitate adoption of EBIs [10]. Recently, political mandates for use of EBI or practices have facilitated their uptake through funding mechanisms such as through the Center for Disease Control and Prevention (CDC) cancer programs. Approaches such as “policy, systems, and environmental change” have been developed to support the development of broader, more systemic changes. For example, advocacy efforts to include funding for healthy options for after school programs [65]. These changes may accelerate the adoption of effective interventions by integrating approaches into existing infrastructures.

The domains of community (n = 20, 42.6%), the preference, attitudes, and knowledge, needs, and resources among individuals in a community, and partnerships (n = 13, 28%; e.g., interorganizational processes) can further bolster implementation efforts. There are a number of promising opportunities to enhance our understanding of these outer context constructs to either develop interventions or support the implementation of existing interventions in addressing community members or patients’ needs. For example, network interventions can capitalize on outer context features such as community perspectives and interorganizational relationships through identifying individuals within a community that to help spread interventions, segment the community or partners to strategically target a group of people, incite new relationships between partners to link/activate opportunities, or design interventions that alter the community or partnerships to better support health [66]. In addition, the economic environment is important since external funds lead to program adoption [10], can provide resources for implementation, or support for program activities [3, 67]. External incentives also have been demonstrated to be important to EB/I implementation [2]. The environment is also linked to program sustainability [68].

The use of these outer context determinants may vary with implementation outcome such as adoption or implementation or implementation phases (e.g., planning, implementation, postimplementation). Some research have found relevant domains including patient needs and external policies and incentive for adoption of medications for alcohol use disorder [69] and patient needs and resources, and cosmopolitanism for delivery of violence screening [70]. Many researchers may use CFIR postimplementation to understand what factors contribute to uptake and implementation or what constitute facilitators or barriers to implementation. Fewer studies have explored factors related to sustainability of evidence-based practices. Conceptualization of outer context domains may advance implementation research for various implementation outcomes.

We found several research gaps related to outer context domains and implementation science. First, only a small number of studies (n = 7) identified measures aligned with outer setting constructs. These measures varied with outer context domains assessed and in number of items used to assess each domain. There was no comprehensive measure or gold standard for outer context found in this review. Prior research has found that outer setting factors are less frequently assessed [9, 11]. Our findings also align with a recent review that found that only four of 366 published implementation science measures included the outer setting through the Society for Implementation Research Collaboration Instrument Review Project [71]. Nilsen et al. conducted a scoping review of context in implementation science frameworks and identified 17 unique frameworks contextual determinants [72]. They found two dimensions reported related to outer context were patients (e.g., their preferences, knowledge, needs and resources) found in 10 frameworks and wider environment (e.g., exogenous influences including policies, legislation, manages, organizational networks, etc.) found in 11 frameworks. However, the wider environment had a wider range of factors related to implementation. In terms of measurement of outer context, a more recent review of outer setting measures found four outer setting measures (four measures of the general outer setting domain, seven of cosmopolitanism, four of external policy and incentives, four of patient needs and resources and one measure of peer) [11]. These results matched our similar findings of community needs, partnerships and external polices being more frequently found in our D&I frameworks review.

Second, there is a critical need to expand outer context construct definition within these D&I frameworks. The large majority of frameworks were conceptual articles versus the very few that were theoretically tested with empirical findings. Most frameworks simply listed external factors in their conceptual models without comprehensive definitions. This study presents outer context dimensions that could be employed for further rich discussion by implementation scientists for construct definitions or scale development for many of the important factors found. Other researchers have noted that context and outer context have been not consistently been defined [1, 72]. Implementation science researchers may have to select from priori relevant outer context domains and seek items from the review for measures found in these frameworks or other assessment to apply in their research [11].

We offer the following recommendations to advance our understanding of outer context constructs in implementation science:

1. Better operationalization of dimensions of outer context is necessary to develop valid and reliable measurement instruments needed to advance the field of implementation science. Ours and other studies have found few valid and reliable measures of outer context domains. Researchers could start with our domains and definitions to create new items or select items from the literature for scale development.

2. D&I frameworks should depict causal pathways through which these constructs are theorized to influence a specific outcome of organizational behavior (i.e., adoption, implementation) and/or other implementation outcomes. For example, Roger’s Diffusion of Innovations [46] and Dobbin’s Framework for Research Dissemination and Utilization [34] have pathways from communications or research dissemination leading to evidence-based decision making and adoption of the intervention, which could result in organizational performance indicators or patient outcomes. Identification of important outer context factors could help practitioners and researchers map relevant implementation strategies that could achieve population, health service and implementation outcomes as recommended by the Implementation Research Logic Model [73].

3. Studies should examine or test the effects of these outer context factors on implementation outcomes such as adoption, quality implementation, and/or sustainability. For example, Wiltsey Stirman et al. found that funding, policies, and partnerships/collaboration influenced ­program sustainability [68]. These data could contribute to our understanding of critical external factors related to the outcomes and how to support public health and clinical efforts to enhance these factors.

4. Research can explore further the concept of cultural environment as an important factor. We found that only 17% of frameworks addressed this area. This construct should further be conceptualized in recognition of the growing support for addressing social determinants of health and promoting health equity through D&I science [74, 75].

This study is not without limitations. As a scoping study, the primary goal was to provide an overview of the existing literature related to outer context. We focused on providing preliminary information about outer context and identifying gaps and opportunities in this area of research. Thus, further research about the quality of constructs and measures is warranted. Additionally, our data abstraction was based on explicit definitions and elements found in the published articles of the framework mentioned in the reviews; this may not have all relevant definitions or discussion of these elements. We also are delimited to frameworks in these two articles and not others that been published in other reviews. Additional reviews of implementation science theories have been published [76, 77]; however, there were overlaps with the frameworks reported here. In addition, D&I theories may have been updated since the original article(s) included in this scoping study. For example, CFIR recently has been updated in changing some constructs (patient needs and resources) and adding others such as local attitudes and local conditions that affect implementation [78].

Our review provides an overview of outer context research and demonstrates the importance of these constructs, as well as gaps in research and measurement of outer context in the field of D&I science. Our categorization of outer context domains may be helpful to implementation researchers. We found some commonalities in frequently identified dimensions in D&I theories such as policies, community needs and resources, and partnerships. We also identified challenges, including the need for further refinement of outer setting definition in the specific dissemination and implementation science frameworks, as well as opportunities to conceptualize and use common constructs to develop and test new measures or scales. The continuation of this research will elucidate what are critical factors external to organizations that are associated with quality implementation and positive outcomes for EBIs.

Funding

This research was supported by the National Cancer Institute Award Numbers R01CA218389 and P30CA138292 (Escoffery) and K00CA253576 (Allen) and by the Centers for Disease Control and Prevention grant number U48DP006377. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the CDC.

Compliance with Ethical Standards

Conflict of Interest: All authors declare that they have no conflicts of interest.

Ethical Approval: This article does not contain any studies with human participants performed by any of the authors.

Informed Consent: This study does not involve human participants and informed consent was therefore not required.

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.

Transparency Statements:

1. This study was not formally registered.

2. The analysis plan was not formally preregistered.

3. Deidentified data from this study are not available in a public archive. Deidentified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.

4. There is not analytic code associated with this study.

5. Materials used to conduct the study are not publicly available.

REFERENCES