10 best resources on … mixed methods research in health systems

Abstract

Introduction

In the last two decades, mixed methods research has become increasingly popular as a third approach along with qualitative and quantitative methods. Indeed, its application has grown in the health sector, and more resources have become available. To provide researchers and programme managers with a guide to mixed methods research in understanding and evaluating health systems, we review the best resources in this area with a specific focus on application in low- and middle-income countries (LMICs).

The definition of mixed methods research has yet to be agreed upon among early developers (Leech 2010). At least 19 different versions have been provided by highly published researchers (Johnson et al. 2007), mainly because of varying degrees of importance placed on the philosophy, methods and research designs. In essence, mixed methods studies intentionally integrate or combine quantitative and qualitative data to maximize the strengths of each, to answer questions that are inadequately answered by one approach. Mixed methods researchers use diverse philosophical positions (e.g. post-positivist and social constructivist worldviews, pragmatic perspectives and transformative perspectives) and often draw upon one or more theoretical frameworks from the social, behavioural or biological sciences to inform the study (Green 2007).

Mixed methods research is important in health systems because it allows researchers to view problems from multiple perspectives, contextualize information, develop a more complete understanding of a problem, triangulate results, quantify hard-to-measure constructs, provide illustrations of context for trends, examine processes/experiences along with outcomes and capture a macro picture of a system (Creswell and Plano Clark 2011). A number of common characteristics between health systems (Hoffman et al. 2012) and mixed methods research (Creswell and Plano Clark 2011) ground inevitable links between the two (Gilson et al. 2011). Health systems researchers may develop survey instruments, interventions or programmes informed by qualitative findings. Alternatively, mixed methods research may be used to identify participants with which to follow up or explain mechanisms behind the quantitative results (Bryman 2008). While the quantitative components allow us to know the extent of the situation and understand how representative the findings are, qualitative studies can enhance the depth of our understandings by presenting various stakeholder perspectives and offer rationale for health systems performance.

We present a selection of resources that could be helpful for readers who may be new to mixed methods research or may not have kept up to date with the literature. To select 10 best resources to highlight in this article, we first identified categories below (overview, design, examples and toolkit) to guide our selection, decided inclusion criteria and carried out searches in electronic databases. The criteria included importance in the mixed methods research field (based on number of citations), comprehensiveness of content, usefulness to readers as well as relevance to public health and health systems research especially in LMIC contexts. We tried to incorporate different resources (eight articles and two textbooks) with examples from various countries (China, Vietnam, Kenya, Tanzania, Zambia and India) across a variety of health topics [tuberculosis (TB), malaria, HIV testing and healthcare costs]. While this is by no means a comprehensive resource, it is our hope that this serves as a guide for those seeking to learn more about mixed methods research in health systems.

Overview of mixed methods

As a starting guide, ‘Designing and conducting mixed methods research’ by Creswell and Plano Clark (2011) targets individuals across many social and human science fields learning about mixed methods research for the first time. It follows the process of conducting a study, from deciding whether or not to use mixed methods, understanding its historic and philosophical underpinnings, on to collecting, analysing and interpreting data in mixed methods research. The book presents six major mixed methods designs with examples in appendices. These designs are: (1) convergent parallel, (2) explanatory sequential, (3) exploratory sequential, (4) embedded, (5) transformative and (6) multiphase (see Figure 1). The authors note that determining the level of interaction, priority, timing and where and how to mix the quantitative and qualitative strands would inform the choice of mixed methods design (Creswell 2003). The book presents a wide-ranging view of mixed methods research, with tables citing numerous authors’ contributions and areas of remaining controversies in the field.

Figure 1

Prototypes of research designs. Source: Informed by Creswell and Plano Clark (2011). Designing and conducting mixed methods research.

Open in new tabDownload slide

For individuals looking for a brief orientation to mixed methods research, ‘Mixed methods: a review of literature and the future of the new research paradigm’ by Migiro and Magangi (2011) provide a basic review. The paper targets a social science researcher novice to mixed methods design, by laying out the strengths and weaknesses of the approach. It discusses the literature behind the philosophical foundation, rationale and steps for conducting and evaluating mixed methods research. In particular, the paper addresses sampling approaches and data analysis stages. The authors endorse a pragmatist paradigm as a basis to encourage researchers to formally recognize mixed methods research as a distinct discipline.

Mixed methods designs

Before the inception of the ‘Sage handbook of mixed methods in social and behavior research’, edited by Tashakkori and Teddlie (2010), mixed methodology was largely ‘self-taught’ (Leech 2010). This 912-page handbook, contributed by diverse authors both within and across disciplines, has been carefully and systematically prepared to integrate experience and opinions of early mixed methods adopters. Readers can learn about how conceptual orientations can affect the conduct and interpretation of mixed studies; what guiding principles and frameworks are available for design, sampling, data analysis and inferences as well as be informed of recent developments in mixed methods data analysis and presentation.

Researchers who are more focused on qualitative approaches may find useful the chapter on computer-assisted integration of mixed methods data sources and analyses (Bazeley 2010) or that on some advanced use of quantitative techniques to aid in the interpretation of qualitative findings (Newman and Ramlo 2010). Researchers with a more quantitative background may enjoy reading the chapter about using mixed methods in monitoring and evaluation (Bamberger et al. 2010) or the use of mixed methods research in systematic reviews (Harden and Thomas 2010).

On sampling issues, the chapter by Collins et al. provides a review on some challenges, presents practical guidelines on minimum sample sizes by research design (Collins et al. 2007) and demonstrates their experience on applying robust mixed methodology to assess the prevalence of sampling designs through a systematic search (Collins 2010). A related but more conceptual article on sampling design is available by Teddlie and Yu (2007) entitled ‘Mixed methods sampling: a typology with examples’. The authors discuss the differences between probability and purposive sampling techniques and put mixed methods sampling in the middle of this continuum. The article provides guidelines for mixed methods sampling, which may be useful to researchers developing a sampling procedure for a mixed methods study. The next four articles provide examples of different mixed methods research designs applied in health systems contexts.

Mixed methods application in health systems in LMICs

Health systems researchers may struggle in explaining or contextualizing quantitative information if a qualitative component is not considered during the project inception. Long et al. (2008) overcome this in a study on ‘Barriers to accessing TB diagnosis for rural-to-urban migrants with chronic cough in Chongqing, China: a mixed methods study’, which provides a good example of an explanatory sequential design. The study started with a prospective cohort of adult TB suspect migrants and permanent residents. Information on health care seeking experiences was collected using a questionnaire and quantitatively analysed to identify TB cases. Qualitative focus group discussions and interviews were then held with stakeholders to obtain a more in-depth insight on the issue. With careful design, the authors were able to capture both ‘the extent and a holistic understanding’ of the factors affecting delay in TB diagnosis, which would not have been possible with only the prospective cohort design alone.

In some occasions, health systems researchers work with complex issues such as socio-cultural factors that have no clear frameworks or measurements. To explore Vietnam’s success of its national malaria control programme, Morrow et al. (2009) in ‘Pathways to malaria persistence in remote central Vietnam: a mixed-method study of health care and the community’ conducted a study using an exploratory sequential design. The study starts with the ‘formative stage’ that applied a number of qualitative techniques such as observations, focus group discussions and semi-structured interviews. These were then used to guide the development of tools in the ‘assessment stage’ that applied quantitative approaches. One could see the triangulated findings presented in various themes, where in-depth qualitative information is supported by quantitative figures. Use of mixed methodology also facilitated collaborations between malaria experts and social scientists, which allowed the team to propose non-biological pathways to malaria persistence.

At times logistical or financial hurdles may only allow for one phase of data collection. An example of a convergent parallel design is presented by Njeru et al. (2011) in the paper ‘Practicing provider-initiated HIV testing in high prevalence settings: consent concerns and missed preventive opportunities’. To examine the use of provider-initiated HIV testing services in Kenya, Tanzania and Zambia, the authors carried out a population-based survey along with focus group discussions and in-depth interviews. While the quantitative approaches examined the proportion of people utilizing HIV testing services, the qualitative approaches explored informants’ experiences and perceptions towards HIV testing services. Analysis from both approaches was interpreted and discussed concurrently, where authors relate the quantitative findings on exposure to HIV testing with quotes from qualitative analysis.

Some health systems studies may require multiple phases to contextualize the study, quantify the scale of the phenomenon and then follow up with in-depth studies. Ranson et al. (2012) carried out a multiphase design study in ‘Strategies for coping with the costs of inpatient care: a mixed methods study of urban and rural poor in Vadodara District, Gujarat, India’. The study took a three-step methodology, which involved focus groups to understand the situation and develop options for the survey, followed by exit survey interviews at public and private health facilities to document costs. The authors then carried out in-depth interviews among poor hospital users from the survey to explore their coping mechanisms. The research phases build on one another, and the authors interpret findings combining results from all phases (Gilson 2012).

Mixed methods toolkits

Some toolkits in mixed methods research may be useful for researchers and programme managers in analysis and evaluation. In the article ‘Using quantitative and qualitative data in health services research: what happens when mixed method findings conflict?’, Moffatt et al. (2006) present an approach for managing apparent discrepancies that may arise between qualitative and quantitative data. In fact, conflicting findings may help us improve the problem analysis, adapt hypotheses, describe different dimensions of interventions and contexts and better understand the causal web underlying complex interventions. The authors suggest six ways of further exploring the data: (1) treating the methods as fundamentally distinct, (2) exploring the methodological rigour of each component, (3) exploring dataset comparability, (4) collecting additional data for further comparisons, (5) exploring whether the intervention worked as expected and (6) exploring whether the outcomes of the two components match.

Finally, O’Cathain et al. (2008) in ‘The quality of mixed methods studies in health services research’ suggest guidelines for evaluating the quality of mixed methods research in ‘Good Reporting of A Mixed Methods Study’ (GRAMMS). The authors assessed proposals and/or final reports of mixed methods studies and propose guiding principles that can help assess the quality of mixed methods studies. These are to describe: (1) the justification for using mixed methods to the research question; (2) the design in terms of the purpose, priority and sequence of methods; (3) each method in terms of sampling, data collection and analysis; (4) where and how integration has occurred; (5) any limitations and (6) insights gained from mixing methods. For those looking for further guidance to develop and evaluate mixed methods research grant applications, ‘best practices for mixed methods in the health sciences’ was prepared for the US National Institutes of Health (Creswell et al. 2011). Some components such as research planning and review criteria for research applications may be relevant for researchers working in LMIC contexts.

Funding

KP received partial funding support from the Ratchadapiseksompotch Fund, Faculty of Medicine, Chulalongkorn University, grant number GDNS 55-077-30-009RER.

Conflict of interest

None declared.

10 Best Resources

References