https://orcid.org/0000-0001-6410-0068
Abstract
Implementation strategies are activities to support integration of evidence-based programs (EBPs) into routine care. Comprised of 170+ facilities, the Veterans Affairs Healthcare System is conducive to evaluating feasibility and scalability of implementation strategies on a national level. In previous work evaluating implementation of three EBPs for older Veterans (hospital-based walking, caregiver skills training, group physical therapy), we found facilities varied in their need for implementation support, with some needing minimal guidance and others requiring intensive support. Committed to national scalability, our team developed an implementation intensification model consisting of foundational (low-touch) and enhanced (high-touch) implementation support. This Forum article describes our multilevel and multistep process to develop and evaluate implementation intensification. Steps included (a) review completed trial data; (b) conduct listening sessions; (c) review literature; (d) draft foundational and enhanced implementation support packages; (e) iteratively refine packages; and (7) devise an evaluation plan. Our model of implementation intensification may be relevant to other health care systems seeking strategies that can adapt to diverse delivery settings, optimize resources, help build capacity, and ultimately enhance implementation outcomes. As more health care systems focus on spread of EBPs into routine care, identifying scalable and effective implementation strategies will be critical.
The Department of Veterans Affairs (VA) Healthcare System offers a natural laboratory to study widespread implementation of evidence-based programs for older adults (Sullivan & Hughes, 2021). As the largest integrated health care system in the United States, the VA is organized into 18 integrated regions (Veteran Integrated Service Networks, VISNs). Together, these VISNs include just over 170 medical centers and an additional 1,100 community-based outpatient clinics (U.S. Department of Veterans Affairs, 2022a). The size of the system lends itself to natural diversity across sites and patient populations. As a learning health care system, research and quality improvement play a central role in VA clinical practice. This infrastructure, along with a strong implementation culture and climate, present an ideal setting for innovation and integration of effective programs into routine care. Central to the VA’s focus on implementation is the Quality Enhancement Research Initiative (QUERI). Established in 1998, QUERI’s mission is to “Improve Veteran health by accelerating effective programs and practices into routine care” (U.S. Department of Veterans Affairs, 2022b).
National Implementation of Programs to Promote Function and Independence in Older Veterans
Veterans aged 65 and older account for approximately one-half of VA Healthcare enrollees (Wang et al., 2021). Funded by QUERI, the Optimizing Function and Independence in Older Veterans program (“Function QUERI”) is housed at the Durham VA Healthcare System and focuses on widespread implementation of three evidence-based programs (EBPs) to promote function and independence of older Veterans and their caregivers. In brief, these three programs include a supervised walking program for hospitalized older adults (STRIDE), skills-based caregiver training (Caregivers FIRST), and group-based physical therapy (Group PT; see Table 1 for additional details of each program).
Description of EBPs Belonging to Function QUERI
| STRIDE | Caregivers FIRST | Group physical therapy | |
|---|---|---|---|
| Program objective | Reduce preventable harm of hospital-associated disability through a supervised walking program | Improve caregiver and Veteran functioning through caregiver skill training and support | Improve access to PT to Veterans with knee osteoarthritis by delivering PT in a group setting |
| Setting | Hospital (general medicine ward) | Outpatient setting | Outpatient setting |
| Primary VA partner | Geriatrics and Extended Care | VA Caregiver Support Program | Physical Medicine and Rehabilitation |
| Target population | Older Veterans | Caregivers | Veterans with knee osteoarthritis |
| Focus of initial implementation phase (2016–2021) | 8 sites | 8 sites | QI evaluation of program at a single VA Medical Center |
| Focus of current implementation phase (2021–2025)a | 32 sites | 24 sites who did not adopt program during mandated period | 16 sites |
| EBP description | Initial gait assessment followed by 1:1 supervised walk | Four group-based sessions delivering caregiver skills training | Six group-based PT sessions |
| Adoption metrics | Five or more patients with a STRIDE consult and walk documented in the medical record in months 5–6 of the implementation period | One or more rounds of Caregivers FIRST delivered in the first 6 months, class attendance of four or more caregivers | Ten or more patients initiate the program, at least one-half of these patients enrolled during months 5–6 of the implementation period |
| Options for enhanced implementation support | Sites randomized to receive enhanced support; only non-adopters participate | Sites randomized to receive enhanced support | Sites randomized to receive enhanced support; only non-adopters participate |
| Special considerations for current study | VA Gold Standard Practice; VA Diffusion of Innovation partner | Mandated practice as of October 2021 | Expand initial QI evaluation for wider implementation |
| STRIDE | Caregivers FIRST | Group physical therapy | |
|---|---|---|---|
| Program objective | Reduce preventable harm of hospital-associated disability through a supervised walking program | Improve caregiver and Veteran functioning through caregiver skill training and support | Improve access to PT to Veterans with knee osteoarthritis by delivering PT in a group setting |
| Setting | Hospital (general medicine ward) | Outpatient setting | Outpatient setting |
| Primary VA partner | Geriatrics and Extended Care | VA Caregiver Support Program | Physical Medicine and Rehabilitation |
| Target population | Older Veterans | Caregivers | Veterans with knee osteoarthritis |
| Focus of initial implementation phase (2016–2021) | 8 sites | 8 sites | QI evaluation of program at a single VA Medical Center |
| Focus of current implementation phase (2021–2025)a | 32 sites | 24 sites who did not adopt program during mandated period | 16 sites |
| EBP description | Initial gait assessment followed by 1:1 supervised walk | Four group-based sessions delivering caregiver skills training | Six group-based PT sessions |
| Adoption metrics | Five or more patients with a STRIDE consult and walk documented in the medical record in months 5–6 of the implementation period | One or more rounds of Caregivers FIRST delivered in the first 6 months, class attendance of four or more caregivers | Ten or more patients initiate the program, at least one-half of these patients enrolled during months 5–6 of the implementation period |
| Options for enhanced implementation support | Sites randomized to receive enhanced support; only non-adopters participate | Sites randomized to receive enhanced support | Sites randomized to receive enhanced support; only non-adopters participate |
| Special considerations for current study | VA Gold Standard Practice; VA Diffusion of Innovation partner | Mandated practice as of October 2021 | Expand initial QI evaluation for wider implementation |
Notes: EBP = Evidence-based program; QUERI = Quality Enhancement Research Initiative; PT = physical therapy; VA = Veterans Affairs; QI = quality improvement.
aSee Figure 1 for study design overview.
Description of EBPs Belonging to Function QUERI
| STRIDE | Caregivers FIRST | Group physical therapy | |
|---|---|---|---|
| Program objective | Reduce preventable harm of hospital-associated disability through a supervised walking program | Improve caregiver and Veteran functioning through caregiver skill training and support | Improve access to PT to Veterans with knee osteoarthritis by delivering PT in a group setting |
| Setting | Hospital (general medicine ward) | Outpatient setting | Outpatient setting |
| Primary VA partner | Geriatrics and Extended Care | VA Caregiver Support Program | Physical Medicine and Rehabilitation |
| Target population | Older Veterans | Caregivers | Veterans with knee osteoarthritis |
| Focus of initial implementation phase (2016–2021) | 8 sites | 8 sites | QI evaluation of program at a single VA Medical Center |
| Focus of current implementation phase (2021–2025)a | 32 sites | 24 sites who did not adopt program during mandated period | 16 sites |
| EBP description | Initial gait assessment followed by 1:1 supervised walk | Four group-based sessions delivering caregiver skills training | Six group-based PT sessions |
| Adoption metrics | Five or more patients with a STRIDE consult and walk documented in the medical record in months 5–6 of the implementation period | One or more rounds of Caregivers FIRST delivered in the first 6 months, class attendance of four or more caregivers | Ten or more patients initiate the program, at least one-half of these patients enrolled during months 5–6 of the implementation period |
| Options for enhanced implementation support | Sites randomized to receive enhanced support; only non-adopters participate | Sites randomized to receive enhanced support | Sites randomized to receive enhanced support; only non-adopters participate |
| Special considerations for current study | VA Gold Standard Practice; VA Diffusion of Innovation partner | Mandated practice as of October 2021 | Expand initial QI evaluation for wider implementation |
| STRIDE | Caregivers FIRST | Group physical therapy | |
|---|---|---|---|
| Program objective | Reduce preventable harm of hospital-associated disability through a supervised walking program | Improve caregiver and Veteran functioning through caregiver skill training and support | Improve access to PT to Veterans with knee osteoarthritis by delivering PT in a group setting |
| Setting | Hospital (general medicine ward) | Outpatient setting | Outpatient setting |
| Primary VA partner | Geriatrics and Extended Care | VA Caregiver Support Program | Physical Medicine and Rehabilitation |
| Target population | Older Veterans | Caregivers | Veterans with knee osteoarthritis |
| Focus of initial implementation phase (2016–2021) | 8 sites | 8 sites | QI evaluation of program at a single VA Medical Center |
| Focus of current implementation phase (2021–2025)a | 32 sites | 24 sites who did not adopt program during mandated period | 16 sites |
| EBP description | Initial gait assessment followed by 1:1 supervised walk | Four group-based sessions delivering caregiver skills training | Six group-based PT sessions |
| Adoption metrics | Five or more patients with a STRIDE consult and walk documented in the medical record in months 5–6 of the implementation period | One or more rounds of Caregivers FIRST delivered in the first 6 months, class attendance of four or more caregivers | Ten or more patients initiate the program, at least one-half of these patients enrolled during months 5–6 of the implementation period |
| Options for enhanced implementation support | Sites randomized to receive enhanced support; only non-adopters participate | Sites randomized to receive enhanced support | Sites randomized to receive enhanced support; only non-adopters participate |
| Special considerations for current study | VA Gold Standard Practice; VA Diffusion of Innovation partner | Mandated practice as of October 2021 | Expand initial QI evaluation for wider implementation |
Notes: EBP = Evidence-based program; QUERI = Quality Enhancement Research Initiative; PT = physical therapy; VA = Veterans Affairs; QI = quality improvement.
aSee Figure 1 for study design overview.
The Function QUERI program began with an initial five years of funding from 2016 to 2021 (referred as “initial implementation”). This initial period largely focused on developing implementation support tools to effectively integrate these new innovations into practice across a small number of facilities across the Healthcare System. Details of these studies have been reported elsewhere (Allen et al., 2020; Boucher et al., 2021; Hastings et al., 2020). Now in its second phase of funding (referred to as “current implementation,” 2021–2025), Function QUERI is focused on widespread integration of the three EBPs into VA Medical Centers (VAMCs) nationwide. Based in Durham, NC, the Function QUERI team comprises three independent project teams, each led by an individual Principal Investigator and staffed by a research coordinator, implementation specialist, and research assistant. The three projects are also supported by a central Implementation Core that provides guidance around the use of common conceptual frameworks and measurement approaches across the EBPs. A central focus of our current implementation phase is on developing and evaluating scalable yet effective implementation support strategies to support widespread scaling of EBPs for older Veterans. The objective of this paper is to present a rationale for considering scalability of implementation strategies in national efforts such as those included within Function QUERI. Specifically, this article presents one theory-informed approach to the development and evaluation of a model of implementation intensification currently being tested within the national VA Healthcare System.
Introduction to Implementation Strategies
Implementation strategies are the tools and activities used to promote successful integration of EBPs into routine practice (Bauer & Kirchner, 2020; Kirchner et al., 2020; Powell et al., 2015). The Expert Recommendations for Implementing Change (ERIC) compilation includes over 70 discrete implementation strategies which can be loosely grouped into different clusters: use evaluative and iterative strategies, provide interactive assistance, train and educate stakeholders, support clinicians, engage consumers, utilize financial strategies, and change infrastructure (Powell et al., 2015; Waltz et al., 2015). Implementation can involve one discrete implementation strategy or can be multifaceted, consisting of multiple strategies (Kirchner et al., 2020). Since its initial program development and implementation, Function QUERI has been guided by the Replicating Effective Programs (REP) Framework (Kilbourne et al., 2007). In its application to Function QUERI, REP is operationalized as a bundle of implementation activities taking place across all phases of implementation. Some of these initial implementation activities included preliminary interviews and trainings with program staff (preconditions and preimplementation), ongoing technical assistance and facilitation (implementation), and postimplementation access to a network of sites who had successfully implemented the EBP. Our current implementation phase expands upon REP to also include principles and practices from two leading implementation frameworks. The QUERI Implementation Roadmap (Goodrich et al., 2020; Kilbourne et al., 2019) outlines a series of evidence-informed action steps within each phase of the implementation process (preimplementation, implementation, sustainment). Given the large, integrated nature of the VA Healthcare System, the QUERI Roadmap provides activities to help design for long-term sustainability, understand multilevel implementation contexts, and select and adapt implementation strategies to accommodate diverse settings. The Dynamic Sustainability Framework (Chambers et al., 2013) further emphasizes the importance of acknowledging contextual characteristics throughout the implementation process as critical to enhancing “fit” between a program and its implementation setting and increasing the potential for sustainability of the EBP over time. Together, these frameworks suggest implementation is a dynamic and iterative process and that improved sustainment of innovations over time may be achieved through regular attention to context, ongoing engagement with key stakeholders, and careful adaptation of the EBP itself and/or its related implementation strategies.
Designing for Scalability: Implementation Intensification
As described above, our team’s initial implementation phase involved a select number of VA facilities across the country. This small number of sites afforded our team the opportunity to apply time- and resource-intensive implementation support activities. However, a central priority of the current implementation phase is to develop and evaluate scalable support strategies. More specifically, we aim to explore how to best match the level of implementation support to sites’ needs and capacity and, second, to develop and evaluate an implementation support approach that relies less heavily on a central research team, and, instead emphasizes comprehensive, self-guided tools to be used by teams working across a range of different VA care settings, including those with fewer resources and/or less experience with quality improvement, innovation, or implementation science. Our current implementation phase includes a research study design that will allow us to pursue research questions around the effects of foundational (low touch) and enhanced (high touch) implementation support on implementation outcomes (see Figure 1).
Function QUERI study design overview. EBP = evidence-based program; EHR = electronic health record; QUERI = Quality Enhancement Research Initiative; R = randomization. All caregivers FIRST sites retain access to foundational support materials for the duration of the study period.
To develop foundational and enhanced implementation support packages, we applied principles from several commonly used methods including implementation mapping (Fernandez et al., 2019) and user-centered design (Dopp et al., 2019). Together, these methods utilize theory-based and evidence-informed steps in a collaborative approach to select and refine implementation strategies best aligned to the needs and capacity of the implementation setting and its team. Our process is described below (see Figure 2 for overview).
Implementation intensification development process.
Step One: Review Data From Initial Implementation Phase
Function QUERI’s initial implementation phase focused on exploring the implementation climate and context and delivering strategies, resources, and supports needed to successfully integrate the EBPs into routine care. Although our team’s prior demonstration projects and clinical trials had demonstrated efficacy of hospital-based walking (STRIDE; Hastings et al., 2014) and caregiver support (HI-FIVES, now called Caregivers FIRST) programs (Van Houtven et al., 2019), respectively, there had been less focus on the implementation strategies that could support sites in integrating the EBP into care. During the initial implementation phase, the Function QUERI team worked closely with EBP stakeholders during the preimplementation period to create and finalize standardized implementation support materials to be used by program staff and patients. Primary implementation activities included prelaunch clinical program training followed by in-person site visits and monthly facilitation sessions (via telephone or videoconference). Monthly sessions were guided by principles of facilitation, a collaborative problem-solving process used to explore implementation barriers and collectively identify potential steps forward (Powell et al., 2015). A portion of sites were also randomized to receive CONNECT, a set of interactive activities designed to “promote team function, communication, and readiness for implementation of EBPs” (Anderson et al., 2012; Colón-Emeric et al., 2014).
Multiple data sources were used to track implementation activities and evaluate implementation outcomes and experiences. In our initial implementation phase, we were most interested in adoption of the EBP, defined as integration of the program into routine care, and sustained delivery beyond the initial implementation phase. Specific to the purposes of developing an implementation intensification plan, we focused on review of qualitative memos generated to summarize implementation process notes and postimplementation qualitative interviews. Qualitative interviews were conducted with multilevel members of each facility’s implementation team during the postimplementation period and explored major domains including team structure and communication, barriers and facilitators to implementation, and overall implementation experience.
Together, these data sources highlight the variation in implementation outcomes under one consistent level of implementation support. In short, some sites successfully implemented an EBP independently, without significant external support from the Function QUERI team. In contrast, other sites were unable to successfully implement an EBP independently and required more extensive external support from the Function QUERI team. Some of the most common challenges experienced during implementation, or adoption, included lack of dedicated staff support full time equivalent (FTE) to carry out EBP, poor leadership support, difficulty in assembling interdisciplinary implementation teams, and/or integrating program documentation templates into the electronic medical record system.
Step Two: Conduct Listening Sessions
Two structured listening sessions were held with members of the central Function QUERI team (e.g., implementation specialists, research assistants) who facilitated all implementation support activities in the initial implementation phase. Structured discussion guides explored the following: (a) perceived successes and challenges experienced with one-on-one facilitation sessions during the initial implementation period; (b) common content covered during facilitation sessions; and (c) ideas on how to integrate principles of team building activities (i.e., CONNECT) into current implementation phase. All project staff involved in delivering in-person or virtual implementation support during the initial implementation period were invited to attend. Listening sessions reveal several common themes across our three projects:
Facilitated interactions felt more valuable if sites had reviewed materials in advance.
High-dose facilitation was perceived to be less sustainable over time.
Some sites were hesitant to engage in implementation support activities until all staff and resources were in place.
Sites expressed a desire to connect with other VA facilities actively implementing the same EBP.
A secondary goal of these listening sessions was to identify opportunities and challenges in delivering external facilitation remotely. Unlike our initial implementation period that included in-person site visits, all current activities are designed for virtual delivery. In our listening sessions, implementation specialists revealed there is often no replacement for “breaking bread” with implementation teams and that virtual meetings can make it difficult to observe the physical space, identify which individuals might be critical to yet missing from facilitation sessions, and gauge team dynamics. To address some of these challenges, we will incorporate best practices from other VA teams delivering virtual facilitation. These include planning in advance, establishing relationships with the local team, use of engaging visuals, and ensuring that participants leave the session with an action plan in which new skills can be applied in the real-world environment (Hartmann et al., 2021).
Step Three: Review Literature on Strategies to Support Adoption and Sustainment of Evidence-Based Programs, Including Use of External Facilitation
Although our bundle of implementation strategies was effective in our initial implementation phase, we recognize the implementation literature continues to grow. Across our three EBPs, we are interested in both the adoption and sustainment of the innovation. Adoption is defined as the uptake, or integration, of the innovation into practice (Proctor et al., 2011), while sustainment is defined as the continued delivery of the EBP over time (Moore et al., 2017). In support of these goals, we reviewed the literature for implementation strategies associated with both successful adoption and sustainment of innovations within health care settings. We were also interested in identifying strategies that could be applied across our three projects. Implementation strategies frequently reported and/or associated with positive adoption outcomes included needs assessments, goal-setting, progress monitoring, education and training, and distribution of tools and resources (Lau et al., 2015; Moussa et al., 2019).
Evidence continues to grow around implementation strategies that are effective in promoting sustainability of evidence-based interventions (Shelton et al., 2018; Wiltsey Stirman et al., 2012). Implementation strategies associated with sustainment included availability of resources, engaged staff and leadership, ongoing monitoring and evaluation, and alignment with clinical or organizational needs and priorities (Bergmark et al., 2019; Hailemariam et al., 2019).
Step Four: Identify and Operationalize Core Components of Foundational and Enhanced Implementation Support Packages
The process of drafting implementation support packages was both collaborative and iterative. The team’s Implementation Core met biweekly to review existing data sources and collectively identify and operationalize the core components of both foundational and enhanced implementation support. Draft materials were then shared biweekly with the larger Function QUERI research team, consisting of external facilitators, project coordinators, and project staff, to present the most recent support package, obtain feedback, and discuss potential refinements. Congruent with other movements within implementation science that recognize similar strategies can take on different forms (Perez Jolles et al., 2019), the support packages were designed to be flexible, allowing for project-specific adaptations and refinements, as appropriate. Foundational and enhanced implementation support packages are described below (see Table 2 for additional details).
Core Components of Foundational and Enhanced Implementation Support
| Function QUERI implementation component | Implementation strategya | Implementation component description | Foundational support (low touch) | Enhanced
support (high touch) |
|---|---|---|---|---|
| Self-guided tools | ||||
| Introductory webinars | Education | Self-guided webinars designed to orient implementation teams to program objectives and initial steps for implementation | ● | ● |
| Program toolkit | Education | Specific to each EBP. Includes guidance on core and modifiable components of EBP, staffing model(s), and options for adaptation for local site | ● | ● |
| Data dashboards | Monitor and evaluate | Electronic dashboards to help sites track progress toward key program metrics | ● | ● |
| Interactive tools | ||||
| Office hours | Technical assistance | Held monthly during the implementation period for each project. Includes a round robin update on implementation progress from each site plus unstructured time for sites to pose questions and share best practices with one another | ● | ● |
| Diffusion network callsb | Technical assistance Learning collaborative | Held quarterly during the implementation period for each project. Designed to capture and share local knowledge, as well as create a collaborative environment for peer-to-peer sharing of experiences and best practices to support implementation. Open to all sites who have implemented the EBP to date | ● | ● |
| Microsoft teams channel | Network weaving | Electronic channels established for communication within EBP project cohorts (Foundational Support). Site-specific channels established for direct communication between implementation team and facilitators (Enhanced Support) | ● | ● |
| Enhanced program support calls | External facilitation | Sites participate in a series of three to four scheduled calls with an external facilitator from the Function QUERI research team. Facilitation calls serve as an opportunity for collaborative problem-solving between the site and facilitator with the goal of identifying specific implementation strategies matched to the challenges and barriers identified by the site. Sites will also discuss strategies for ongoing capacity building, including tips for sustaining the EBP over time | ● | |
| Peer matching | Learning collaborative | Expand upon learning collaboratives used in foundational support by connecting enhanced support sites to peer sites that have successfully adopted and/or sustained EBP, including those that may have experienced similar implementation challenges | ● |
| Function QUERI implementation component | Implementation strategya | Implementation component description | Foundational support (low touch) | Enhanced
support (high touch) |
|---|---|---|---|---|
| Self-guided tools | ||||
| Introductory webinars | Education | Self-guided webinars designed to orient implementation teams to program objectives and initial steps for implementation | ● | ● |
| Program toolkit | Education | Specific to each EBP. Includes guidance on core and modifiable components of EBP, staffing model(s), and options for adaptation for local site | ● | ● |
| Data dashboards | Monitor and evaluate | Electronic dashboards to help sites track progress toward key program metrics | ● | ● |
| Interactive tools | ||||
| Office hours | Technical assistance | Held monthly during the implementation period for each project. Includes a round robin update on implementation progress from each site plus unstructured time for sites to pose questions and share best practices with one another | ● | ● |
| Diffusion network callsb | Technical assistance Learning collaborative | Held quarterly during the implementation period for each project. Designed to capture and share local knowledge, as well as create a collaborative environment for peer-to-peer sharing of experiences and best practices to support implementation. Open to all sites who have implemented the EBP to date | ● | ● |
| Microsoft teams channel | Network weaving | Electronic channels established for communication within EBP project cohorts (Foundational Support). Site-specific channels established for direct communication between implementation team and facilitators (Enhanced Support) | ● | ● |
| Enhanced program support calls | External facilitation | Sites participate in a series of three to four scheduled calls with an external facilitator from the Function QUERI research team. Facilitation calls serve as an opportunity for collaborative problem-solving between the site and facilitator with the goal of identifying specific implementation strategies matched to the challenges and barriers identified by the site. Sites will also discuss strategies for ongoing capacity building, including tips for sustaining the EBP over time | ● | |
| Peer matching | Learning collaborative | Expand upon learning collaboratives used in foundational support by connecting enhanced support sites to peer sites that have successfully adopted and/or sustained EBP, including those that may have experienced similar implementation challenges | ● |
Notes: QUERI = Quality Enhancement Research Initiative; EBP = evidence-based program; ERIC = Expert Recommendations for Implementing Change.
aImplementation strategies informed by ERIC taxonomy.
bDiffusion Network Calls not included in the Group Physical Therapy protocol (Office Hours only).
Core Components of Foundational and Enhanced Implementation Support
| Function QUERI implementation component | Implementation strategya | Implementation component description | Foundational support (low touch) | Enhanced
support (high touch) |
|---|---|---|---|---|
| Self-guided tools | ||||
| Introductory webinars | Education | Self-guided webinars designed to orient implementation teams to program objectives and initial steps for implementation | ● | ● |
| Program toolkit | Education | Specific to each EBP. Includes guidance on core and modifiable components of EBP, staffing model(s), and options for adaptation for local site | ● | ● |
| Data dashboards | Monitor and evaluate | Electronic dashboards to help sites track progress toward key program metrics | ● | ● |
| Interactive tools | ||||
| Office hours | Technical assistance | Held monthly during the implementation period for each project. Includes a round robin update on implementation progress from each site plus unstructured time for sites to pose questions and share best practices with one another | ● | ● |
| Diffusion network callsb | Technical assistance Learning collaborative | Held quarterly during the implementation period for each project. Designed to capture and share local knowledge, as well as create a collaborative environment for peer-to-peer sharing of experiences and best practices to support implementation. Open to all sites who have implemented the EBP to date | ● | ● |
| Microsoft teams channel | Network weaving | Electronic channels established for communication within EBP project cohorts (Foundational Support). Site-specific channels established for direct communication between implementation team and facilitators (Enhanced Support) | ● | ● |
| Enhanced program support calls | External facilitation | Sites participate in a series of three to four scheduled calls with an external facilitator from the Function QUERI research team. Facilitation calls serve as an opportunity for collaborative problem-solving between the site and facilitator with the goal of identifying specific implementation strategies matched to the challenges and barriers identified by the site. Sites will also discuss strategies for ongoing capacity building, including tips for sustaining the EBP over time | ● | |
| Peer matching | Learning collaborative | Expand upon learning collaboratives used in foundational support by connecting enhanced support sites to peer sites that have successfully adopted and/or sustained EBP, including those that may have experienced similar implementation challenges | ● |
| Function QUERI implementation component | Implementation strategya | Implementation component description | Foundational support (low touch) | Enhanced
support (high touch) |
|---|---|---|---|---|
| Self-guided tools | ||||
| Introductory webinars | Education | Self-guided webinars designed to orient implementation teams to program objectives and initial steps for implementation | ● | ● |
| Program toolkit | Education | Specific to each EBP. Includes guidance on core and modifiable components of EBP, staffing model(s), and options for adaptation for local site | ● | ● |
| Data dashboards | Monitor and evaluate | Electronic dashboards to help sites track progress toward key program metrics | ● | ● |
| Interactive tools | ||||
| Office hours | Technical assistance | Held monthly during the implementation period for each project. Includes a round robin update on implementation progress from each site plus unstructured time for sites to pose questions and share best practices with one another | ● | ● |
| Diffusion network callsb | Technical assistance Learning collaborative | Held quarterly during the implementation period for each project. Designed to capture and share local knowledge, as well as create a collaborative environment for peer-to-peer sharing of experiences and best practices to support implementation. Open to all sites who have implemented the EBP to date | ● | ● |
| Microsoft teams channel | Network weaving | Electronic channels established for communication within EBP project cohorts (Foundational Support). Site-specific channels established for direct communication between implementation team and facilitators (Enhanced Support) | ● | ● |
| Enhanced program support calls | External facilitation | Sites participate in a series of three to four scheduled calls with an external facilitator from the Function QUERI research team. Facilitation calls serve as an opportunity for collaborative problem-solving between the site and facilitator with the goal of identifying specific implementation strategies matched to the challenges and barriers identified by the site. Sites will also discuss strategies for ongoing capacity building, including tips for sustaining the EBP over time | ● | |
| Peer matching | Learning collaborative | Expand upon learning collaboratives used in foundational support by connecting enhanced support sites to peer sites that have successfully adopted and/or sustained EBP, including those that may have experienced similar implementation challenges | ● |
Notes: QUERI = Quality Enhancement Research Initiative; EBP = evidence-based program; ERIC = Expert Recommendations for Implementing Change.
aImplementation strategies informed by ERIC taxonomy.
bDiffusion Network Calls not included in the Group Physical Therapy protocol (Office Hours only).
Foundational support includes a mixture of asynchronous, self-guided materials (introductory webinars, program toolkit, data dashboard) combined with regular opportunities for sites to obtain interactive technical assistance (Microsoft Teams channel, monthly office hours, quarterly learning collaborative). Foundational support tools aim to educate implementation teams about the clinical problem addressed by the EBP, provide an orientation to core and adaptable elements of the EBP, offer suggestions for engaging service line leaders and other stakeholders, and present a series of steps to support preimplementation and implementation of the program. Self-guided materials are designed to be reviewed and digested at a site’s own pace and used in combination with more interactive technical assistance tools. These more interactive tools consist of monthly office hours attended by all sites currently launching the EBP plus larger quarterly Diffusion Network calls open to all sites who have previously implemented and continue to offer the EBP. Additionally, all sites can access online chat forums via Microsoft Teams.
Enhanced implementation provides tailored, site-specific guidance via one-on-one and/or group-based coaching. Sites randomized sites to high touch implementation support and who do not meet prespecified adoption benchmarks will receive enhanced implementation support (see Figure 1). Enhanced implementation support consists of a minimum of three to four sessions over a 2- to 4-month period. Individual support calls are scheduled roughly two to three weeks apart, allowing sites the opportunity to apply recommended implementation strategies between each support call. External facilitation, defined as collaborative problem-solving, serves as the core component of the enhanced implementation support package. Function QUERI utilizes an external facilitation approach in which an implementation facilitator (“implementation specialist”) is based with the central research team conducts one-on-one virtual facilitation sessions with one or more members of an individual site’s implementation team.
By definition, external facilitation is flexible and intended to be tailored to the receiving site. Facilitation is a core component of implementation in leading implementation science frameworks, including PARIHS and the QUERI Implementation Roadmap, both of which emphasize the importance of tailoring facilitation activities, including recommendation of specific implementation strategies, to match the needs and contexts of each implementation site (Goodrich et al., 2020; Rycroft-Malone et al., 2002). However, in an effort to develop a more intensive yet scalable implementation support package, all sites participating in enhanced implementation will follow a similar call schedule and structure. All calls will include an opportunity for the site to provide an update on the implementation process, including any challenges and/or successes to date, discuss and prioritize current barriers, and collaboratively select implementation strategies to address barriers. Call one will mimic an abbreviated needs assessment with a particular focus on implementation progress to date, identification of barriers and facilitators in the implementation process, and access to implementation resources and supports. A central goal of the initial call is for the implementation facilitator to work with the site to collaboratively craft an implementation plan and identify potential steps forward. Call two will include general implementation strategies to address implementation barriers and challenges (e.g., poor leadership support, challenges recruiting patients, cross-service communication and collaboration, staffing shortages). Call three will focus on more advanced strategies to build and sustain site capacity. The final call will focus on strategies for sustainment (e.g., monitoring progress over time, securing funding, maintaining stakeholder and leadership engagement, planning for adaptations). Unlike some implementation initiatives that employ external facilitation throughout the course of implementation, we believe that introducing tailored, high touch support after sites have embarked on the implementation process may be both more efficient and effective.
Step Five: Iteratively Refine Support Packages for Each EBP
Although all of Function QUERI’s EBPs target older Veterans, each program addresses unique clinical needs, and each is at a different stage within the process of national implementation (see Table 1). Given this natural variation, our development process included opportunities to review implementation support packages in collaboration with each project team (see Figure 2). These collaborative meetings focused on identifying opportunities to adapt and/or refine individual implementation support components to best meet the needs, capacity, and resources of each individual EBP.
Adaptation is typically encouraged within the process of implementation as doing so can enhance the fit between an intervention or its implementation approach and the target setting (Chambers et al., 2013). Specifically, adaptation to implementation strategies may take place in response to specific contextual characteristics, including differing levels of capacity, readiness, or resilience within different organizations (Goodrich et al., 2020). Adaptations will be made to align with the scope and feasibility of each project. For example, a quicker paced clinical program like Group PT may choose to focus technical assistance efforts via monthly Office Hours rather than quarterly Diffusion Networks calls. In contrast, a nationally mandated program like Caregivers FIRST may consider how to deliver external facilitation in a group-based format given the large number of facilities undergoing implementation at the same time.
Over the course of the 4-year project period, the research team will monitor and track implementation support at regular intervals, as described in the section below. Additional refinements to implementation support components will be made as appropriate and tracked accordingly (Miller et al., 2021).
Step Six: Develop an Evaluation Plan
A primary goal of our current implementation research study is to evaluate the effect of enhanced, or high touch, support on implementation outcomes and to explore sites’ perceptions and experiences of this intensification. A secondary goal is to describe the structure and process of enhanced implementation support. Specifically, this includes identifying the types and frequencies of implementation strategies used and how such strategies are matched to barriers identified by each site. Together, this information will allow our team to explore the mechanism through which such strategies address barriers and impact overall implementation outcomes. To achieve these goals, we will collect data from a range of different data sources that will be utilized to conduct rigorous mixed methods analyses:
Quantitative surveys: Validated instruments examining organizational readiness to change, implementation climate, organizational resilience, and program sustainability.
Adaptation reports: Sites will document planned and unplanned EBP adaptations at regular intervals.
Implementation support notes: Structured note templates will be used to capture content presented on all Office Hours, Diffusion Network, and facilitation calls as well as sites’ attendance, engagement, and questions. Notes from enhanced implementation support calls will be used to record commonly reported implementation barriers, identify recommended implementation strategies, and explore how such strategies were matched to implementation barriers identified by each site.
Qualitative interviews: Semistructured interviews will be conducted with select key members of the implementation team at both pre- and postimplementation. Sites who participate in enhanced implementation support will participate in a second postimplementation interview at the conclusion of the enhanced support period.
Implementation reflections and debriefs: At minimum, the research team will meet quarterly with each project team’s implementation facilitators to review the use of foundational and enhanced implementation support to date and identify any areas for improvement.
Anticipated Challenges and Potential Solutions
Our team’s pursuit of foundational and enhanced implementation support stems from our interest in developing and testing a scalable yet effective model of implementation support that can be deployed across a large, diverse setting like the VA Healthcare System. However, this same diversity may present several challenges to be considered. First, the diversity in implementation settings includes differences in organizational characteristics such as clinic size, staffing, and available resources. These differences may inherently lend themselves to variations in implementation climate, culture, readiness, and capacity. We recognize a central challenge in working toward national implementation rests in achieving a balance between standardized yet flexible implementation support packages.
Second, our current design evaluates sites randomized to receive enhanced implementation support, rather than evaluating this more intensive support in response to performance metrics or key timepoints. Sites are randomized to an implementation support arm early in the study period. For STRIDE and Group PT, only sites that do not meet prespecified adoption benchmarks then receive enhanced implementation support beginning at six months. For Caregivers FIRST, study enrollment is limited to a selection of sites that have not implemented during the mandated program period; only half will be randomized to receive enhanced implementation support beginning at two months. We recognize other study designs exist, including beginning with higher touch implementation support before scaling back over time or intensifying implementation support at prespecified intervals (Kilbourne et al., 2014). We also recognize that there are numerous ways to operationalize and offer external facilitation, including multitiered facilitation (Penney et al., 2021) and blended facilitation that combines both internal and external facilitation (Pimentel et al., 2019). Finally, although all enhanced support sites will follow a similar call structure and content, we recognize that implementation facilitation, by definition, is meant to be individualized. Thus, we may find greater variation from this structure than anticipated or that some sites would benefit from additional support, or facilitation, not permitted in our protocol. However, our team believes such findings can make significant contributions to the literature.
Third, the time and resources needed to develop these implementation support packages and train external implementation specialists to deliver facilitation must be taken into account (Ritchie et al., 2021). For example, the Function QUERI team may choose to obtain feedback on the usability of our implementation support packages from members of our Technical Advisory Panel or from implementation researchers and practitioners based at other QUERI programs or facilities. Other project teams interested in developing a similar implementation intensification plan are encouraged to factor in the time, cost, and expertise needed for both package development and staff training.
Finally, working within a large learning health care system often involves natural shifts in system-level priorities and initiatives that can affect implementation research studies. In the case of Function QUERI, there has been significant implementation momentum around two of our EBPs, STRIDE and Caregivers FIRST. These two programs have been declared a gold standard practice and a mandated practice, respectively. The research team has embraced the ongoing expansion of these two programs as a natural opportunity to actively pursue questions around scalability, adaptation, and spread.
Conclusion
As the population of older adults continues to grow, the need and demand for EBPs to promote and sustain function and independence will increase. Much focus has been placed on scalability of evidence-based interventions for widespread implementation. However, less focus has been placed on scalability of implementation strategies to support this widespread implementation. This article describes a collaborative and evidence-informed process used to develop foundational and enhanced implementation support packages for use across three unique EBPs to promote function and independence in older Veterans. To our knowledge, the Function QUERI program is one of the first studies to compare the effectiveness of foundational support and enhanced support on implementation outcomes across three unique EBPs being actively implemented throughout a nationwide health care system.
Acknowledgments
The authors thank the members of the Function QUERI research team: Josh D’Adolf, Caitlin B. Kappler, Teri L. Malo, Cassie Meyer, Caitlin Sullivan, Matthew Tucker, and Sara Webb as well as the VA project offices and Veterans who have agreed to participate in our projects. This work is dedicated to the late Elizabeth P. Mahanna, MPH, whose unmatched dedication for improving the lives of Veterans, honest enthusiasm for qualitative research, and curiosity for advancing the methods of implementation science helped to build the foundation of our implementation intensification program. This study is currently underway at time of publication, thus data and materials are not available. This study is registered at clinicaltrial.gov (NCT04868656; NCT05319535; NCT05282927). The views expressed here are of the authors and do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.
Funding
This work is supported by the United States Department of Veterans Affairs Quality Enhancement Research Initiative (QUE-16-170); Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410); Research Career Scientist Program, Office of Research and Development, United States Department of Veterans Affairs (CVH: RCS 19-332, KDA: RCS 21-137). Activities to support program implementation at participating sites were considered to be non-research operations activities, as defined in VHA Handbook 1058.05.
Conflict of Interest
None declared.
