https://orcid.org/0000-0001-8365-8681
Abstract
The coronavirus disease (COVID-19) pandemic disrupted healthcare and clinical research, including a suite of 11 pragmatic clinical trials (PCTs), across clinics within the Department of Veterans Affairs (VA) and the Department of Defense (DOD). These PCTs were designed to evaluate an array of nonpharmacological treatments and models of care for treatment of patients with pain and co-occurring conditions.
The aims of the study are to (a) describe modifications to PCTs and interventions to address the evolving pandemic and (b) describe the application of implementation science methods for evaluation of those PCT modifications.
The project used a two-phase, sequential, mixed-methods design. In Phase I, we captured PCT disruptions and modifications via a Research Electronic Data Capture questionnaire, using Periodic Reflections methods as a guide. In Phase II, we utilized the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) taxonomy to develop a focus group interview guide and checklist that would provide more in-depth data than Phase I. Data were analyzed using directed content analysis.
Phase I revealed that all PCTs made between two and six trial modifications. Phase II, FRAME-guided analyses showed that the key goals for modifying interventions were increasing treatment feasibility and decreasing patient exposure to COVID-19, while preserving intervention core elements. Context (format) modifications led eight PCTs to modify parts of the interventions for virtual delivery. Content modifications added elements to enhance patient safety; tailored interventions for virtual delivery (counseling, exercise, mindfulness); and modified interventions involving manual therapies.
Implementation science methods identified near-real-time disruptions and modifications to PCTs focused on pain management in veteran and military healthcare settings.
Lay Summary
Active-duty personnel and veterans often report pain and seek treatment in military and veteran healthcare settings. Nondrug treatments, such as self-care, counseling, exercise, and manual therapy, are recommended for most patients with chronic pain. The COVID-19 pandemic has affected clinical trials of these nondrug treatments in military and veteran populations. In this study, we explored how 11 research teams adapted study trials on pain to address COVID-19. Team members completed online questions, brief checklists, and a one-time focus group about how they modified their trials. Each of the 11 trials made 2 to 6 changes to their studies. Most paused or delayed recruitment efforts. Many shifted parts of the study to a virtual format. Goals for adapting treatments included improved feasibility and decreased patient exposure to COVID-19. Context or format changes increased virtual delivery of study treatments. Content changes focused on patient safety, tailoring treatments for virtual delivery, and offering varied manual therapies. Provider concerns about technology and patient willingness to seek in-person care during the pandemic also were factors driving changes. These findings may support the increased use of virtual care for pain management in military and veteran health settings.
Practice: The coronavirus disease (COVID-19) pandemic has emphasized the importance for healthcare systems to adapt evidence-based practices for a given clinical setting, particularly for complex nonpharmacological treatment (NPT) approaches for pain.
Policy: Policymakers who want to optimize access to NPT approaches for pain should consider incentivizing adaptable interventions that meet the local context while being patient centered.
Research: Rapid and flexible implementation science methods are critical for measuring modifications and adaptations to understand how best to get evidence-based practices in place.
INTRODUCTION
Nonpharmacological treatments (NPTs) are identified by the Department of Veterans Affairs (VA) and Department of Defense (DOD) as preferred approaches for individuals experiencing pain [1]. Both U.S. federal agencies provide healthcare to millions of veterans and military service members and their dependents; they are also leaders in clinical research. Studies across VA and DOD have demonstrated the benefits of NPTs in reducing pain intensity and improving functioning [2–5]. In 2017, the National Institutes of Health (NIH) established the NIH-DOD-VA Pain Management Collaboratory (PMC) to support 11 pragmatic clinical trials (PCTs) that are testing innovative NPTs and integrative models of pain management in veteran and military healthcare settings [6]. These PCTS were designed as phased studies involving a pilot phase, followed by an implementation/study enactment phase.
Starting in February 2020, this suite of 11 PCTs was impacted by the COVID-19 pandemic, which caused unprecedented disruption to healthcare operations across the USA and globally [7, 8]. In-person delivery of NPTs was limited in favor of the need for social distancing to reduce person-to-person transmission of COVID-19. As a result, healthcare systems pivoted to a reliance on virtual modalities to deliver patient care [9–11]. The pandemic also caused major shifts in healthcare research, including study delays and the need to identify alternative modes of treatment delivery (e.g., virtual) and modify protocols [12–14]. The profound disruption of COVID-19 also brought a critical opportunity for scientists—to understand how rapid modifications are made in PCT protocols and the delivery of clinical care for those receiving NPTs for pain management across multiple agencies, sites, and geographic locations.
Investigators often make modifications, or adaptations, to the design or procedures of a clinical trial, which may be instituted prior to, concurrently with, or following trial implementation, and that require proper documentation and reporting to support trial replication or adoption into clinical settings. During the uncertainty of the early pandemic (February to May 2020), the PMC Coordinating Center encouraged investigators to consider proactive trial modifications to evolving COVID-related disruptions, while capturing relevant data to evaluate the potential impact of the pandemic on patient health, trial implementation, and study results. The pandemic also provided an opportunity to catalog disruptors to clinical research in VA and DOD healthcare environments and identify common and unique modifications across PCTs. Reports by Coleman et al. [15] and Fritz et al. [16] describe the initial impacts of COVID-19 on the PCTs, which included potential pauses in treatment delivery, shifts to telehealth and other virtual modalities, updates to outcome assessments, changes in consent processes and participant recruitment strategies, and considerations on sample sizes and data analysis plans.
The present evaluation adds to this previous PMC work by addressing two aims. The first aim was to characterize the types of modifications each PCT team made to their trials in response to the evolving COVID-19 pandemic, including identifying modifications that could have implications for future clinical research. The second aim was to describe the application of implementation science methods to the evaluation of PCT modifications, including how these methods can be applied in future evaluation of the implementation of clinical practices or programs. Recently, Taylor et al. [17] cautioned that, during COVID-19, “Traditional approaches to data collection in implementation science may not match the timeline of this crisis.” As such, we used rapid data collection and analytic methods based in scientific rigor [18], including Periodic Reflections [19] and the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) taxonomy [20], to guide data collection and analysis. The Periodic Reflections method was originally designed to consistently document key activities, implementation challenges, PCT modifications, and other phenomena in near-real-time during the implementation process of complex healthcare interventions within organizational settings [19]. The FRAME is a highly useful framework for identifying modifications that are generally not evaluated or understood in the field of implementation science or PCTs [20]. The term modifications is used purposefully throughout this paper, as the changes to PCTs and interventions were reactive to the COVID-19 pandemic and not planned changes (i.e., adaptations) [20]. Reliance on these complementary approaches was well suited for assessing the dynamic processes unfolding across the 11 PCTs, as each project team grappled with time-dependent decisions to modify or maintain trial protocols within the context of the pandemic.
METHODS
Study design
In this two-phase study, a sequential, mixed-methods design was used to explore COVID-related clinical trial modifications across 11 PCTs. The first phase was intended to identify the breadth of modifications whereas the second phase was intended to gain more in-depth knowledge on the modifications identified. The second qualitative phase was informed by rapid analysis of the first (core) qualitative phase (see Fig. 1) [21]. Only 10 PCT teams participated in Phase II of this study.
Sequential mixed methods design of current study.
In Phase I, a structured, web-based Research Electronic Data Capture (REDCap) questionnaire [22] was used to capture emerging COVID-related impacts on PCTs during the first 7 months of the pandemic across three time points (May, July, and September 2020), using the Periodic Reflections method as a guide [19]. Preliminary analyses of Phase I data shaped the development of Phase II, focus-group interview guide, which was informed by FRAME. During this phase, three focus groups based on a PCT setting (i.e., two VA groups, and one DOD group) convened via VA-approved videoconferencing software (January to February 2021). Participants in focus groups provided verbal consent for audio recording. We elicited additional details from PCT investigators on types of modifications made to their PCTs using a brief FRAME checklist, which was distributed after focus group sessions. This project received a nonhuman-subjects research waiver from the Institutional Review Board (IRB) of record for the lead of the Implementation Science Work Group (ISWG). All study procedures were done in accordance with quality improvement standards (see Supplementary Appendix 1—Standards for Quality Improvement Reporting Excellence).
Study setting and participants
The PMC PCTs are conducted in over 60 sites located within DOD and VA healthcare facilities in 32 states [23]. Supplementary Appendix 2 provides an overview of the 11 PMC PCT teams involved in this study. Seven PCTs are in VA settings, three are in DOD facilities, and one takes place in both settings. Each project’s principal investigator (PI) or a designated team member completed Phase I REDCap questionnaires. Among the Phase II focus group participants, 10 were PIs or co-investigators with expertise in psychology, medicine, or implementation science and six were staff (e.g., program managers, program coordinators, research assistants). There were seven men and nine women represented.
Phase I: periodic reflections data collection
REDCap questionnaire development
We tailored the Periodic Reflections method [19] for use in a structured, web-based, REDCap questionnaire to gather essential data. The primary advantages of this approach were the ability for project teams to complete the questionnaires at times convenient to their schedules, to consult meeting minutes and reference sources to assure the accuracy of submitted data and to allow varied team personnel to asynchronously contribute information with the unique survey link. From March to April 2020, ISWG members and PMC Coordinating Center personnel iteratively refined questionnaire drafts, which developed around four key areas: status updates, changes/adaptations to interventions and implementation plans, environment/context, and planning/reflections (Supplementary Appendix 3). Open-ended text boxes were included to allow teams to elaborate on their responses.
Data collection
The REDCap questionnaire was launched in May 2020. Teams were asked via e-mail invitations sent every 6 weeks to complete a structured set of questions on REDCap. A 100% response rate from the 11 PCT teams was achieved across all three time points. The average time between baseline data collection and the first follow-up was 48 days (July 2020), and between the first and second follow-ups was 50 days (September 2020). By the third reflection (September 2020), teams provided much less data, either due to the option of answering “no changes” from the prior period, or because PCT modifications had slowed significantly. The study team (A.M.M., S.A.S., L.K., S.J.J.) reviewed the responses independently and then met to discuss the methods to use in Phase II, including focus group versus individual interviews and which study personnel should participate in the interviews. The study team determined that focus group interviews would likely provide a depth of information related to modifications in the context of limited study personnel resources.
Phase II: qualitative focus groups guided by FRAME
In Phase II, we conducted focus groups and analyzed data using directed content analysis guided by the FRAME taxonomy, a framework designed to classify modifications of evidence-based practices during implementation [20]. Specifically, the FRAME facilitates identification of modifications based on different domains, including context (format) and content ones, as well as reasons for them. Modifications are any changes made that are deliberate or proactive, but they are typically in response to unanticipated events in a trial [20]. All qualitative focus group procedures are in accordance with Standards for Reporting Qualitative Research (see Supplementary Appendix 4) [24].
Interview guide development
The study team generated iterative drafts of the semi-structured interview guide based on the FRAME (Supplementary Appendix 5). A post-interview checklist from the FRAME was developed to identify persons or entities involved in the decision to modify, the content and context of the modification, the level of delivery at which the modification was made, and the intended goals of the changes made (Supplementary Appendix 6).
Data collection
To minimize participant burden and to facilitate sharing, we conducted three focus group sessions via Microsoft Teams, a video-based conference platform that allowed for recording the interviews and automated transcription. Focus groups lasted approximately 60 min and were completed between January and February 2021. Four project teams attended the DOD-focused session (n = 9 participants); six VA-based teams attended one of two sessions (n = 5 and n = 4 participants, respectively); and one team declined participation because they did not have an active intervention during the time of evaluation.
One PI from each team was required to attend the focus group; most teams invited another PI or project manager familiar with daily PCT operations. The ISWG lead (A.M.M.) moderated all focus groups, two trained qualitative researchers (S.A.S. and S.J.J.) took fieldnotes and asked follow-up questions, and the ISWG project manager (L.K.) handled recording/transcription procedures and managed technical details.
Analytic approach: directed content analysis
Directed content analysis was the primary analytic approach for both Phase I and II [25]. The use of this approach allowed us to identify relationships among key concepts in the FRAME and to identify implementation factors in the modifications of interventions in response to COVID-19. In Phase I, directed content analysis [25] of response matrices (A.M.M. and S.A.S.) revealed that most respondents reported modifications to PCT protocols and/or clinical interventions, but details were missing as to what modifications were made and why. These missing details appeared to be a result of using a web-based form (vs. qualitative interviews) and motivated the second phase of qualitative focus groups to better understand and characterize the types of modifications made. In Phase II, two team members (A.M.M., S.J.J.) independently coded focus group notes, transcripts, and interview checklists using FRAME concepts, meeting biweekly over 3 months to discuss the application and interpretation of FRAME codes. The coders also met with the larger team to resolve discrepancies in data interpretation. Data synthesis identified emergent themes specific to our investigation (Table 1).
Emergent domains/themes from qualitative focus groups
| FRAME category | No. of trials | Emergent domain/theme |
|---|---|---|
| Contextual modification | ||
| Contextual modification to format of intervention | 8 | Adapted part or all of the intervention to be delivered virtually |
| Content modification | ||
| Tailoring/refining | 3 | Adjusted protocols to facilitate delivery of intervention |
| Adding elements | 4 | Elements added to enhance safety/security of recipient during COVID-19 |
| Removing or skipping elements | 3 | Hands-on pain treatment modalities (e.g., chiropractic care, physical therapy) availability reduced |
| Training and evaluation | 3 | Trainings conducted remotely rather than in-person |
| Modifications to implementation strategies | 9 | Recruitment suspended or delayed |
| Relationship to fidelity | ||
| Core elements of intervention preserved | 8 | Intervention delivered as intended |
| Core elements of intervention changed | 2 | Core elements of “usual care” conditions may have been compromised |
| Reasons for adapting/modifying | ||
| Sociopolitical | 5 | Existing mandates, regulations, and political climate resulting from COVID-19 affected trial |
| Organization/setting | 5 | Competing demands, variability of COVID-19 social distancing guidelines, organizational transition to virtual care delivery affected trial |
| Provider | 8 | Provider concerns with safety, inability to deliver usual care, comfort with telehealth affected trial |
| Recipient | 7 | Patient concerns with safety |
| FRAME category | No. of trials | Emergent domain/theme |
|---|---|---|
| Contextual modification | ||
| Contextual modification to format of intervention | 8 | Adapted part or all of the intervention to be delivered virtually |
| Content modification | ||
| Tailoring/refining | 3 | Adjusted protocols to facilitate delivery of intervention |
| Adding elements | 4 | Elements added to enhance safety/security of recipient during COVID-19 |
| Removing or skipping elements | 3 | Hands-on pain treatment modalities (e.g., chiropractic care, physical therapy) availability reduced |
| Training and evaluation | 3 | Trainings conducted remotely rather than in-person |
| Modifications to implementation strategies | 9 | Recruitment suspended or delayed |
| Relationship to fidelity | ||
| Core elements of intervention preserved | 8 | Intervention delivered as intended |
| Core elements of intervention changed | 2 | Core elements of “usual care” conditions may have been compromised |
| Reasons for adapting/modifying | ||
| Sociopolitical | 5 | Existing mandates, regulations, and political climate resulting from COVID-19 affected trial |
| Organization/setting | 5 | Competing demands, variability of COVID-19 social distancing guidelines, organizational transition to virtual care delivery affected trial |
| Provider | 8 | Provider concerns with safety, inability to deliver usual care, comfort with telehealth affected trial |
| Recipient | 7 | Patient concerns with safety |
Emergent domains/themes from qualitative focus groups
| FRAME category | No. of trials | Emergent domain/theme |
|---|---|---|
| Contextual modification | ||
| Contextual modification to format of intervention | 8 | Adapted part or all of the intervention to be delivered virtually |
| Content modification | ||
| Tailoring/refining | 3 | Adjusted protocols to facilitate delivery of intervention |
| Adding elements | 4 | Elements added to enhance safety/security of recipient during COVID-19 |
| Removing or skipping elements | 3 | Hands-on pain treatment modalities (e.g., chiropractic care, physical therapy) availability reduced |
| Training and evaluation | 3 | Trainings conducted remotely rather than in-person |
| Modifications to implementation strategies | 9 | Recruitment suspended or delayed |
| Relationship to fidelity | ||
| Core elements of intervention preserved | 8 | Intervention delivered as intended |
| Core elements of intervention changed | 2 | Core elements of “usual care” conditions may have been compromised |
| Reasons for adapting/modifying | ||
| Sociopolitical | 5 | Existing mandates, regulations, and political climate resulting from COVID-19 affected trial |
| Organization/setting | 5 | Competing demands, variability of COVID-19 social distancing guidelines, organizational transition to virtual care delivery affected trial |
| Provider | 8 | Provider concerns with safety, inability to deliver usual care, comfort with telehealth affected trial |
| Recipient | 7 | Patient concerns with safety |
| FRAME category | No. of trials | Emergent domain/theme |
|---|---|---|
| Contextual modification | ||
| Contextual modification to format of intervention | 8 | Adapted part or all of the intervention to be delivered virtually |
| Content modification | ||
| Tailoring/refining | 3 | Adjusted protocols to facilitate delivery of intervention |
| Adding elements | 4 | Elements added to enhance safety/security of recipient during COVID-19 |
| Removing or skipping elements | 3 | Hands-on pain treatment modalities (e.g., chiropractic care, physical therapy) availability reduced |
| Training and evaluation | 3 | Trainings conducted remotely rather than in-person |
| Modifications to implementation strategies | 9 | Recruitment suspended or delayed |
| Relationship to fidelity | ||
| Core elements of intervention preserved | 8 | Intervention delivered as intended |
| Core elements of intervention changed | 2 | Core elements of “usual care” conditions may have been compromised |
| Reasons for adapting/modifying | ||
| Sociopolitical | 5 | Existing mandates, regulations, and political climate resulting from COVID-19 affected trial |
| Organization/setting | 5 | Competing demands, variability of COVID-19 social distancing guidelines, organizational transition to virtual care delivery affected trial |
| Provider | 8 | Provider concerns with safety, inability to deliver usual care, comfort with telehealth affected trial |
| Recipient | 7 | Patient concerns with safety |
We used several forms of triangulation in this study to validate our results: methodological triangulation with quantitative REDCap data, investigator triangulation (i.e., two researchers took part in analysis), and data triangulation (via peer debriefing meetings and member-checking with interviewees, where PCT PIs were able to review study results and clarify or correct findings) [26, 27].
RESULTS
Phase I: periodic reflection results
Respondents from all 11 PCTs completed the Periodic Reflections phase. Table 2 provides an overview of study modifications reported by the PCT PIs or their designees. If the designee completed the survey, then the PI reviewed the responses to confirm. The same person from each PCT team completed the survey across all time points. Teams reported between 2 and 6 modifications within the first 5 months of the pandemic. In general, modifications were related to whether a trial was in the pilot/demonstration or implementation/study enactment phase. For example, PCT teams either in or transitioning to the implementation phase reported clinical trial delays in the areas of site/clinic onboarding, staff training, and/or patient recruitment. Three PCT teams were required by institutional policies to pause intervention delivery after the initiation of patient recruitment. Intervention protocols also were modified—five teams shifted from in-person to remote site visits for training, and six teams either added or expanded virtual delivery of the study intervention.
Reported trial delays and protocol changes in pragmatic clinical trials of nonpharmacological approaches to pain management
Reported trial delays and protocol changes in pragmatic clinical trials of nonpharmacological approaches to pain management
Eight PCT teams modified their data collection measures or procedures, with many adding instruments developed through the PMC to assess the impact of the COVID-19 pandemic on study participants [28]. Five PCT teams also re-evaluated or modified their statistical analysis plans. Team members also noted variations in the telehealth software used by various clinical sites, and that preferred platforms (WebEx, Microsoft Teams, Zoom, VA Video Connect) changed over time or varied by geographic location. The PCT teams that were testing manual therapies, such as “hands-on” physical therapy or chiropractic care, experienced extensive restrictions to clinical care (including clinic closures) and the need to adhere to new safety and infection control protocols. The PCT teams that were still able to recruit participants experienced decreased inquiries as fewer patients seeking healthcare services. Since most teams reported modifications to the interventions, and no other PMC efforts were systematically evaluating them, we examined them more closely through semi-structured interview guides based on the FRAME.
Phase II: focus group results grounded in FRAME
Sixteen representatives from 10 PCT teams joined one of the three focus groups. Each team that participated in a focus group completed the post-interview checklist. Table 1 presents focus group emergent domains and themes mapped to concepts from the FRAME. Emergent themes and illustrative quotes are further described in this section.
FRAME concept: the goal of making modification
Participants reported modifying PCTs to (i) improve the feasibility of the intervention (n = 5), (ii) decrease patient exposure or risk related to COVID-19 (n = 4), (iii) increase reach or engagement with patients (n = 3), (iv) increase retention of patients (n = 3), and (v) decrease provider exposure or risk related to COVID-19 (n = 3). One PCT team reported that they wanted to adapt the intervention to improve the effectiveness of the intervention and to improve patient satisfaction.
FRAME concept: who was involved in making the decision to modify?
Participants reported that the primary parties involved in making decisions about modifications were the study team (n = 6), followed by nonclinical staff (n = 4), funders (n = 4), and clinical teams at sites (n = 3). One PCT’s designated respondent reported that site leadership was involved in making decisions about modifications.
FRAME concept: context modifications (format)
Interventions were modified to be delivered virtually.
Eight PCT teams reported deliberate modifications of part or all the intervention for virtual delivery in response to COVID-19. One respondent said, “We changed the delivery of one arm of the intervention, so that the in-person group sessions will be delivered online for the entire trial.” Investigators in manual therapy trials considered how “hands-on” interventions might be delivered virtually or supplemented by other interventions: “We have changed our protocol to include both in-person and virtual visits. … All visits previously had to be in person.”
FRAME concept: content modifications
Three types of content modifications were reported across PCTs: adding/avoiding COVID-19 elements to enhance the safety or security of intervention recipients; tailoring/modifying/refining trial protocols to facilitate virtual intervention delivery; and removing hands-on pain treatment modalities.
Elements added to enhance safety.
Four PCT teams added elements to enhance patient safety, such as brief check-ins about the patient’s situation or feelings about COVID-19 at the start of intervention sessions to the creation of tracking tools. One PCT team added a qualitative evaluation to understand how the pandemic affected participants’ health, social determinants of health, and access to healthcare, with the intent of altering their counseling approach to be responsive to these stressors. The PCT respondent stated, “We’re doing a qualitative analysis of [COVID-related] discussions… to look at pandemic influences on the nature of the counseling.”
Three PCT teams reported avoiding changing study protocol during COVID-19 to avoid influencing intervention outcomes. Two of these PCT teams described that “all guidelines are set by local clinic leadership” or “they are following clinic guidelines.” A third PCT, which focuses on adherence to clinical practice guidelines for physical therapy, did not change the study protocol since physical therapy is often delivered in-person. They stated they “do not want to influence outcomes by adding modifications to increase patient engagement or motivation.”
Protocol changes to facilitate intervention delivery.
Three PCT teams reported tailoring, tweaking, or refining trial protocols to facilitate the delivery of their intervention. In one case, the intervention was switched from an in-person, “usual care” condition, in which counseling was delivered by a trained counselor in-person, to either a live video conference or phone session. In the second PCT, investigators added a “group zero” condition to train potential participants on how to use Zoom and to work as a group in a virtual care delivery setting. In a third PCT, which focused on engaging patients in pain care services following a brief intervention, investigators added a booster session during weeks 12–32 to provide more opportunities to enhance patient engagement with virtual and/or community services given difficulties accessing them during the pandemic. One respondent stated:
We loosened up a little bit and allowed our counselors to guide people towards other non-VA interventions... So non-VA apps would be things like stretching exercise videos or like yoga and then [PI] also mentioned knowing about substance use treatment available in the community through online forums... I know the counselors had to get educated about that.
Modifying hands-on treatment modalities.
The virulence of COVID-19 and strict rules for social distancing led three PCT teams to delay or skip hands-on pain treatment modalities, even when these treatments were initially considered integral to their intervention. In one PCT, recruitment for hands-on chiropractic care was delayed with virtual alternatives to in-person modalities (e.g., education, therapeutic exercise, and self-care) becoming options for participants. Similarly, two other PCT teams said that the availability of hands-on modalities (e.g., chiropractic care, physical therapy, acupuncture) became greatly reduced or unavailable due to COVID-19. One respondent said,
Well, I would say the [interventions became] initially less available or unavailable. And then ultimately became virtual where things like chiropractic care, PT, even acupuncture, which was often delivered by some chiropractic groups and so it wasn’t the full treatment available, but there were aspects of care that became available as people were adjusting to the pandemic. Even the mental health services or addiction services that we might you know, refer people to those became virtual for almost the majority of the time. And then I would say our counselors became much more aware of apps and self-help approaches.
FRAME concept: training and/or evaluation
Training had to be conducted remotely.
Three PCT teams adapted in-person staff training for virtual delivery. One PCT added a train/retrain chiropractic model because of moving from in-person to virtual visits. In a second PCT, all training had to be conducted remotely and as a result, one- to two-day site visit training became spread out into several virtual sessions. A third PCT team reported that they transitioned to telehealth training due to changes to standards of care practices.
FRAME concept: implementation and scale-up activities
Recruitment suspended or delayed.
A key modification to implementation across most PCTs (n = 10) was that intervention recruitment was either suspended or delayed due to COVID-19. The VA/DOD mandates paused clinical research activities to address COVID-related priorities. Recruitment was suspended or delayed either at the start of the pre-implementation (or intervention) phase (i.e., before the pilot phase) across three PCTs, and at the start of the implementation (or intervention) phase across seven PCTs. Reasons for recruitment delays varied but included stakeholder influence (n = 1); logistical delays (n = 2; e.g., contract completion delayed; IRB approval pending); intervention providers assigned to COVID-19 clinical duties (n = 1); and unspecified (n = 6).
FRAME concept: core elements
Core intervention elements preserved.
While most PCT teams (n = 8) reported that core elements of the intervention were preserved, there was some concern that the effect of the intervention on pain outcomes would be reduced because of COVID-19. For instance, a respondent said:
I don’t think the intervention core elements have changed really at all, it’s just harder for them to be effective in a service system that has less options available to Veterans…. We think the effect size of the intervention might be diminished.
Some PCT teams chose not to modify interventions that were not originally designed for virtual delivery. For instance, one respondent said, “We’re trying to study an intervention that isn’t virtual and probably isn’t going to be virtual after this… We wanted to keep the integrity of the intervention as it was.” Another respondent said, “We’re looking at acute pain after surgery…. We can’t do the surgery without anesthesia and really, our intervention is just part of that process and for that reason, we haven’t had to make any modifications on what we do.”
Core intervention elements modified.
Two PCT teams of “hands-on” care that included physical therapy and/or spinal manipulation reported that core elements were modified to allow virtual delivery. However, as few studies have tested telehealth for manual therapy, investigators had concerns about PCT results. For example, one investigator said, “Spinal manipulation for the virtual visits will be a missing component.... If spinal manipulation is in fact the magic ingredient, we will definitely potentially see a diminished effect size.” Similarly, another respondent described the knowledge gap between guideline-concordant physical therapy and telehealth:
Our project is on guideline-based physical therapy for management of back pain, and the guidelines are pretty clear on active approaches for physical therapy. …we have had to make decisions about the use of telehealth to apply these interventions. However, it’s not consistent with guideline-based care to some extent, and what can be delivered through telehealth.
FRAME concept: reasons for modifications
Organization/setting response to COVID-19.
Five PCT teams modified their interventions to address organizational responses to COVID-19, such as competing demands (n = 2), social distancing variances by clinic (n = 1), and rapid transitions to virtual care delivery (n = 2). One respondent noted, “[Our organization] never prioritized virtual interventions because we haven’t had to, so we just don’t… All of a sudden, COVID hit and it was like you can use whatever you want.” Another respondent described that the influx of patients with COVID-19 symptoms made chronic pain a lower priority across their organizations, which reduced intervention recruitment, “Chronic pain tends to take a back seat, so… just a lot less people are coming into primary care… they were prioritizing… other types of more serious illnesses.”
Provider preferences, competency, and perceptions.
Intervention fidelity was affected both by provider comfort with telehealth technology and the ability of providers to deliver care “as usual” during the pandemic (n = 8). Provider comfort with technology was high in three PCT teams, such as one where providers transitioned seamlessly to delivering care via telehealth and Microsoft Teams. In a second PCT, providers shifted from completing monthly treatment monitoring checklists pre-COVID to a virtual platform. In the third PCT, a respondent described how providers expanded patient engagement with virtual modalities:
Our counselors became much more aware of apps and self-help approaches. I think initially we were more conservative about that because we wanted to make sure they were VA approved, and afterwards because we wanted to provide Veterans with some assistance.
In contrast, some PCT teams reported that modifications were not so seamless due to concerns that care “as usual” conditions were compromised. For instance, in one PCT, providers referred every potential participant, yet fewer patients were coming into the clinic due to COVID-19. In a second PCT, some providers exhibited COVID-19 symptoms (before testing was widespread) and could not work at the clinic in-person, which drastically reduced patient volume.
Recipient demographics, motivation, and readiness.
Patient, or “recipient” factors, such as motivation, demographics, and technology barriers, influenced intervention modifications (n = 7) in the short-term, with the longer-term effect on PCT results uncertain. In one PCT, the transition from in-person to virtual care enhanced patients’ motivation and readiness, “Some people there were definitely more interested in [participating in the intervention] because of Zoom. Because of distance to the [clinic] or because of COVID concerns.” Yet, most PCT teams (n = 6) reported that concerns related to COVID transmission led them to delay in-person treatment. For instance, one respondent said, “There are plenty of patients that are like, ‘Okay, yeah, I think I’ll wait until I can come in,’ because they perceive that hands-on is an important part of therapy.”
Clinic volume was greatly reduced in some cases, which affected interventions. For instance, prior to COVID-19, patients in one PCT would come into the clinic to receive treatment twice per week, but investigators reported that during COVID-19, in-person visits increased to once every 16 days. A separate PCT team discussed that patients with more medically complex pain presentations required more intervention sessions than less complex patients. Yet, the former group is more concerned about COVID-19 and less likely to come into the clinic to receive treatment.
Barriers to technology also reduced patient engagement across some PCTs. In one PCT, investigators reported that one of their five recruitment sites was comprised of an older population, which struggled with technology and telehealth more than the younger population. In another PCT, investigators reported that patient engagement via videoconferencing was reduced, with several participants turning off their video and audio during individual intervention sessions.
Additional themes
Contributions to conduct of PCTs.
One contribution involved knowledge related to the virtual conduct of pain interventions such as mindfulness and chiropractic care. Respondents from a trial of a counseling intervention for veterans with musculoskeletal disorders described broader clinical implications of the COVID-19 pandemic:
We need to address pandemic-related issues in the counseling intervention because they are affecting participants’ access to VA and community care options. In addition, effects on their work, psychosocial well-being, and functioning may influence their experience of pain and risk of problematic substance use. We also need to provide alternative pain care and substance use support options for people when traditional in-person services are unavailable, greatly reduced, or only partially available via virtual platforms.
DISCUSSION
This sequential, mixed methods study revealed that multiple types of modifications were made across 11 PCTs evaluating an array of NPTs for pain. All PCTs made some modifications, with a majority altering their data collection approach to collect data on the impact of COVID-19 and approximately half of PCTs altering their statistical analytic plans. The delivery of the intervention was paused for a few PCTs, with most PCTs modifying the delivery to a virtual format for part or all of the intervention. PCT teams made significant modifications that could not be captured with the Periodic Reflections methods alone, thus necessitating the need for a new method for data capture that provided a structure for categorization of modifications—the FRAME [20]. Focus groups revealed that all PCTs had the format (context) of the intervention adapted, with several PCTs making a range of content modifications, including refining or tailoring the original intervention.
Many sites were able to pivot to telehealth-delivered NPT modalities in a relatively rapid manner, which has significant implications for expansion of care to hard-to-reach populations (e.g., those living far from a healthcare setting). At the same time, it is important to consider whether telehealth interventions in randomized clinical trials (RCTs) are as safe and effective as in-person interventions in improving patient outcomes. Findings thus far are mixed. A recent systematic rapid review of eight RCTs concluded that NPTs delivered through synchronous telehealth (telephone and/or videoconferencing) were as effective and safe as in-person interventions in reducing pain and other health outcomes [29]. However, a recent study on patients’ acceptance of telemedicine for chronic pain treatment [30], conducted during the first 4 months of COVID-19, found that higher pain levels and anxiety were associated with lower acceptance of virtual care. This points to the need for further research to better understand the effectiveness of different delivery modalities of interventions among different patient subpopulations (e.g., socioeconomic status, rural vs. urban residency, or access to the internet and/or mobile devices). Another key finding is that encouragement of self-care/self-management approaches (e.g., mobile applications) increased when the delivery of care was paused or modified. This highlights the importance of continuing some type of evidence-based care and leveraging patients’ feelings of resilience and personal growth to motivate the use of adaptive coping skills and engagement in self-management [31].
Our study findings have implications for the delivery of clinical care for patients with chronic pain both within and outside of DOD/VA contexts, particularly when a disruption in a healthcare context occurs [17]. Many of our study sites were able to pivot to telehealth-delivered NPT modalities in a relatively rapid manner. For instance, most CIH clinical treatments were able to be delivered via telehealth [32]. The ability of these modalities to be delivered in a virtual setting may reduce access barriers for patients who lack transportation or who are immunocompromised and would not be able to travel to an in-person appointment safely during a COVID-19 surge.
Our methods can inform rigorous conduct of clinical studies conducted in busy healthcare settings. We found that the flexibility of our data collection as well as the sequential phases allowed for data that had both breadth and depth. We also focused on minimizing participant burden through structured web-based forms and focus groups [19, 33, 34]. We found that the group format was particularly useful in improving the quality of information. For example, other’s reports would serve as memory cues for participants on the call, while the semi-structured nature allowed the interview leader to circle back to those participants who had previously responded to fill in gaps.
The sequential phases of the study were also advantageous. The first phase provided key domains of interest when examining modifications while highlighting areas the evaluation team could probe further with the second phase [21]. To our knowledge, this is the first study using a two-phase qualitative sequence relying on the Periodic Reflections method and the FRAME. These implementation science methods were key to identifying domains of modifications in the first phase, followed by elucidation of greater details in the second phase, relying on focus groups. The use of this FRAME in the context of multi-modal medical interventions was at times challenging for the evaluation team, given the use the FRAME was originally designed for use in measuring modifications to evidence-based psychotherapies [20]. The challenges in the present study highlight the potential for a FRAME that is adjusted for multi-modal medical interventions and perhaps a more rapid FRAME coding template could be developed for instances such as this one when more rapid evaluation methods are needed to understand modifications occurring within an evolving context.
At the organizational level, our study indicates that clinician and organizational leaders should anticipate potential disruptions to care delivery and plan accordingly. For instance, staff involved in several of these PCTs had to be trained remotely during COVID-19. While virtual training may reduce some of the benefits of in-person training (e.g., building rapport), more staff may be able to access virtual training, thus building capacity for delivering these services during unanticipated disruptions.
Lastly, this sequential, mixed methods study not only provided important findings for modifications and highlighted the value in Periodic Reflections and the FRAME in a phased manner, but it provided short-term benefits to the PMC, including the sponsors. The evaluation team provided them with rapid feedback regarding modifications to the 11 PCTS. The Phase II focus groups provided a forum for PCT PIs and project staff to share best practices and challenges they had faced, allowing PIs to learn from each other on what has worked/not worked for them.
Limitations
Although there are key strengths to this study, there are some limitations to note. The need for dynamic evaluation amid a constantly evolving context (i.e., COVID-19) meant an extensive revision of the study design or materials was not possible. Thus, it is possible other methods would have yielded different results, such as more in-depth qualitative methods and one-on-one interviews. Relatedly, we chose to use the Periodic Reflections method in a structured survey format instead of a semi-structured interview format. It is possible this difference could impact our findings.
CONCLUSIONS
The COVID-19 global pandemic highlights the need to maintain flexibility and speaks to the potential need for a shift in pragmatic trials that expedites the collection of relevant data [35], including data that informs a subsequent implementation trial. Although not a main focus of the trials, it would be remiss to not mention that the switch to virtual modalities and the evaluation of these modalities, could have implications for certain vulnerable populations, including those who have had challenges staying connected to care (e.g., unstably housed individuals). We know that the COVID-19 pandemic has exacerbated pre-existing disparities in certain vulnerable populations [36, 37], and there is an urgent need to apply findings from the burgeoning number of trials examining virtual modalities, such as the ones in this NIH-DOD-VA PMC.
Acknowledgments
The authors want to acknowledge all teams from the NIH-VA-DOD Pain Management Collaboratory who participated in this project. The study was made possible by Grant Number U24 AT009769 (Pain Management Collaboratory Coordinating Center) from the National Center for Complementary and Integrative Health (NCCIH), and the Office of Behavioral and Social Sciences Research (OBSSR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NCCIH, OBSSR, and the National Institutes of Health, and VA’s HSR&D (FOP 21-140). This manuscript is a product of the NIH-DOD-VA Pain Management Collaboratory. For more information about the Collaboratory, visit https://painmanagementcollaboratory.org/.
Compliance With Ethical Standards
Conflict of Interest: All authors declare they have no conflicts of interest.
Authors' Contributions: AMM led the project and paper writing, with substantial contributions by SJJ, SAS, and RDK in the conception and design of the work. AMM, SJJ, SAS, and LK conducted the analysis of data for the work. All authors contributed to acquisition and interpretation of data, as well as writing and revising the paper for important intellectual content. All authors approved the final version to be published.
Ethical Approval: Stanford University Institutional Review Board granted a non-human-subjects research waiver for the project. This study does not involve human participants and informed consent was therefore not required. Participants provided verbal consent to allow audio recording of the focus groups. This article does not contain any studies with animals performed by any of the authors.
Transparency Statements: This study was not formally registered. The analysis plan was not formally pre-registered. De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author. There is no analytic code associated with this study. Some materials used to conduct the study are provided in Appendices or are not publicly available. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the National Institutes of Health, the National Center for Complementary and Integrative Health, the Department of Defense, the Department of Veterans Affairs or the United States government, or the university affiliates of each author.
