Coulombe et al. Respond to “Baby Steps to a Learning Mental Health–Care System”

We enjoyed reading the commentary by Shiner and Watts (1) and thank them for their feedback on our work (2). In their commentary, Shiner and Watts suggest a path forward for learning health-care systems (LHS) (3) focusing on improving mental health care. When estimates of intervention effects exist only from observational studies, such as retrospective cohort studies with data collected from clinical care, Shiner and Watts propose key factors to consider before clinical implementation of the better intervention (strength of the design, robustness to unmeasured confounders, large effects or large population affected, and that there exists no other evidence). We second their call to the scientific community to develop and apply new methods for answering important questions related to mental health care in observational studies, particularly when high-quality evidence from randomized trials is unlikely to be forthcoming due to the high resource needs for conducing such studies.

As Shiner and Watts discuss, electronic health records represent a wealth of information for learning about mental health care, and some pitfalls of randomized trials (like smaller sample sizes) are alleviated in observational studies. Improving future care using data captured from routine clinical care is a pillar of LHS and is essential for moving mental health care forward. Coauthor Shortreed is the Methods Core lead for a research network made up of the research institutes associated with several of learning mental health–care systems, the Mental Health Research Network, whose mission is to synthetize information on research, practice, and policy to improve mental health care (4, 5). A growing synergy between statisticians, epidemiologists, and clinicians, enabled by initiatives funded by LHS themselves as well as traditional funding bodies, allows for such resources to focus on practical questions, like the head-to-head comparison of antidepressant drugs. For example, pragmatic trials, often conducted in partnership between LHS and researchers, could be designed to address scientific questions faster than traditional research, using randomized trial designs. While some long-standing scientific questions can be answered only by randomized trials, we support the assertion by Shiner and Watts that not all questions can or will be tackled experimentally. We need alternative study designs to address critical research questions.

To further evidence-based knowledge using observational data, additional statistical methods must be translated to the special setting of individualized treatment strategies, and sensitivity analyses must be developed to assess the robustness of findings. Methods for power calculations for adaptive treatment strategies already exist for sequential multiple assignment randomized trials (SMARTs) (6, 7), but special attention is needed for observational situations where inference is affected by confounding and irregular visit patterns. Sensitivity analyses for unmeasured confounders are being developed and will certainly be used, but a better synergy between statisticians, epidemiologists, and clinicians could improve data availability for research (for instance, better algorithms for reading free text from electronic databases) and, ultimately, clinical practice. These important topics are currently under active investigation by Moodie, Shortreed, and collaborators.

We thank Shiner and Watts for raising the question of when results from an observational study deserve consideration for being incorporated into clinical care. We agree wholeheartedly with their recommendations, applaud current progress and collaborations, and encourage continued advancement. Beyond their suggestions, we note that using all available observational data from across several different reliable health systems or data sources could make inferential methods for individualized treatment strategies more robust to potential drawbacks of observational data. Although more LHS are regularly collecting patient-reported outcomes, electronic health records often do not contain information on symptom severity or other important confounders of antidepressants’ causal effects.

We share the thought Shiner and Watts offer on confirmatory studies being welcomed, but often unlikely to materialize, and their encouragement to strengthen scientific collaborations to continuously improve mental health care.

ACKNOWLEDGMENTS

Author affiliations: Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Québec, Canada (Janie Coulombe, Erica E. M. Moodie, Christel Renoux); Biostatistics Unit, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, United States (Susan M. Shortreed); Biostatistics Department, University of Washington, Seattle, Washington, United States (Susan M. Shortreed); Centre for Clinical Epidemiology, Lady Davis Institute for Medical Research, Jewish General Hospital, Montréal, Québec, Canada (Christel Renoux); and Department of Neurology and Neurosurgery, McGill University, Montréal, Québec, Canada (Christel Renoux).

This work was funded by an Innovative Ideas Award from Healthy Brains for Healthy Lives (#HBHL 1c-II-11). E.E.M.M. acknowledges salary support from a senior chercheur-boursier salary award from the Fonds de recherche du Québec–Santé. S.M.S. acknowledges that research reported in this publication was supported by the National Institute of Mental Health (award R01 MH114873). C.R. acknowledges a chercheur-boursier salary award from the Fonds de recherche du Québec–Santé.

S.M.S. has worked on grants awarded to Kaiser Permanente Washington Health Research Institute by Bristol Meyers Squibb and by Pfizer. She was also a co-investigator on grants awarded to Kaiser Permanente Washington Health Research Institute from Syneos Health, who represented a consortium of pharmaceutical companies carrying out Food and Drug Administration–mandated studies regarding the safety of extended-release opioids. The other authors report no conflicts.

REFERENCES