Evidence on cataract in low- and middle-income countries: an updated review of reviews using the evidence gap maps approach

Abstract

In 2014, Sightsavers developed the first evidence gap map (EGM) to assess the extent and quality of review-level evidence on cataract relevant to low-and middle-income countries. The EGM identified 52 studies across five broad themes. This paper reports the update of the EGM conducted in 2021 and changes to the extent and quality of the evidence base. We updated the EGM using the exact process conducted to develop the original. Searches were run to 14 September 2021, and two independent reviewers selected eligible studies, critically appraised them and extracted data using the Supporting the Use of Research Evidence checklist. A summary quality assessment was shared with the authors for comments. Forty-six new reviews were identified, and the EGM now includes 98 reviews. The new reviews predominantly focus on treatment and risk factors. The overall methodological quality was found to be improved, with 13/46 reporting high confidence in findings. EGMs remain a useful tool for policy-makers to make informed decisions and periodic updates are important to assess changes and to refine the focus for future research. The EGM highlights significant disparity in the topics addressed by reviews, with health system interventions particularly neglected.

Introduction

Cataract is the leading cause of blindness and the second leading cause of moderate and severe visual impairment globally, affecting >100 million people, 17 million of whom are blind.¹ Over 90% of people with visual impairment due to cataract live in low- and middle-income countries (LMICs), where access to relatively simple and cost-effective treatment, cataract surgery, is often restricted by poor awareness of services, a limited number of eye care facilities and high user fees and costs of transport.^2–5 A review of national cataract surgical rates (CSRs), an indicator, which measures the number of cataract operations per million population per year, published by Wang et al. in 2016, showed significant regional variations in access to cataract surgery, with national CSRs ranging from 10 000 per million in some of the wealthiest countries of Europe and North America to <500 per million in the poorest parts of sub-Saharan Africa.⁶

Because prevalence of cataract increases sharply with age, the combination of a growing and ageing population in the next few decades will substantially increase the number of people with cataract and subsequent demand for cataract services.^7–10 In May 2021, the World Health Assembly (WHA) endorsed effective cataract surgical coverage as a global indicator for monitoring progress towards universal health coverage (UHC) and recommended to countries a target of a 30% increase in coverage by 2030.¹¹ The WHA also stressed the importance of equity of cataract surgical coverage for all relevant population subgroups. These demographic pressures and global policy commitments call for a stronger focus on evidence-based policies and practice at both national and subnational levels.^12–14 An evidence-based approach to decision-making uses critical thinking and the best available evidence and helps policy-makers and practitioners make better decisions, achieve higher impact and gain greater value for money spent.¹⁵

Evidence gap maps (EGMs) are a tool for presenting the state of evidence in a particular topic with the aims of providing easy access to the best available evidence and highlighting knowledge gaps.¹⁶ The approach was developed by the International Initiative for Impact Evaluation (3ie) and has been applied in different sectors, including health, education, water and sanitation, food and nutrition, agriculture and many others.¹⁷ EGM methods draw on the principles of systematic reviews. The process involves a systematic search of relevant electronic databases and sifting sources identified against inclusion and exclusion criteria. Evidence included in an EGM can be drawn from any source, but many EGMs focus on evidence from systematic reviews. As the EGM approach was originally developed to make evidence more accessible to decision-makers in LMICs, evidence sources are usually assessed on their relevance to low- and middle-income contexts. Resultant EGMs are presented as visual matrices describing the areas of evidence available and evidence strength. Each cell is linked to a summary page and shows a confidence rating of the review using the traffic light system. The rating is assigned using a standardised methodology used for critical appraisals of systematic reviews; it aims to help EGM users judge how much they can rely on the evidence presented.

In 2014, using the 3ie methodology, Sightsavers developed the first eye care EGM with a focus on cataract (Figure 1). The process of the EGM development and the evidence base included in the map was published in 2016.¹⁸ The map included 52 systematic reviews identified through a search of Cochrane Library, MEDLINE and EMBASE from inception to the end of 2013. The reviews were mapped across 5 broad themes and 14 subthemes (Figure 1). These were developed in consultation with practising clinicians and researchers from a number of LMICs. Reviews identified provided evidence in 7 out of 14 subthemes, with the majority of evidence available on the quality of clinical care (20/52) and types of treatment (18/50). The other subthemes were risk factors and prevention (9/52), access, uptake and equity (3/52), cost-effectiveness/cost-benefit (3/52), burden of disease/cost of illness (3/52) and productivity and quality of life (2/52). With regards to quality, 16 reviews were rated as high confidence, 13 were medium confidence and 29 were low confidence. Overall, the EGM showed that the evidence on cataract available in systematic reviews published up to the end of 2013 was primarily on types of treatment and other aspects of clinical care. Five reviews examined the impact and cost-effectiveness of cataract services and clear gaps were identified on broader aspects of cataract-related health systems.¹⁸ The EGM also drew attention to the need to better present and describe the methodology of published systematic reviews to ensure that their search strategies, selection criteria and data syntheses are transparent and clear to the reader.

In this paper we describe an update to our original cataract EGM completed in 2021 by reviewing and appraising systematic reviews published during 2014–2021. We report the updated state of evidence on cataract and specifically examine whether the extent and quality of the newly published reviews has changed since the original EGM. We also highlight persistent evidence gaps for researchers to concentrate on.

Materials and Methods

The update of the EGM followed the same methodology as the original.¹⁸ In summary, Cochrane Library, MEDLINE and EMBASE databases were systematically searched for all systematic and literature reviews published from 1 January 2014 to 1 January 2019 using the same search terms as previously reported (Appendix A).¹⁸ A further search was conducted on 14 September 2021, once again with the same terms as previously reported.

Sources were included if they (1) presented systematic or literature reviews, (2) described their methods for collecting and synthesising data, (3) contained evidence relevant to LMICs and (4) were published from 1 January 2014 to 14 September 2021. Reviews that restricted their searches to evidence solely from high-income settings were excluded. References in full-text articles were screened for relevance and added if they met the inclusion criteria.

We applied no restrictions on the language of publication. The relevance to LMICs was assessed jointly by an ophthalmologist with substantial clinical experience in LMICs and a Sightsavers researcher. Therefore, if a review question was assessed as relevant to LMICs (e.g. relevant clinical procedures/techniques) by the ophthalmologist and a Sightsavers researcher, and the search strategy was open to evidence from LMIC settings, it would be included in the cataract EGM regardless of where the studies identified and included actually came from.

Primary studies, purely narrative reviews, reviews where the search methodology was restricted to high-income settings and reviews that restricted their search strategies to high-income settings only, were excluded.

Data from the reviews that met the inclusion criteria were independently extracted by two reviewers, who also categorised the strength of evidence of effect (strong, inconclusive or weak) and appraised the methodological quality of the review (high, medium or low). All data extraction and critical appraisal were in the original language of the publication.

Strength of evidence of effect summarises the extent to which the authors of each review were able to reach a conclusive answer to their research question, using the evidence they reviewed. Strength of evidence of effect was categorised as strong if the review found consistent strong evidence in response to the research question or outcome, inconclusive if the review reported mixed results and weak if the review found weak or no evidence in response to the research question or outcome.

The appraisal of methodological quality used an adapted version of the Supporting the Use for Research Evidence (SURE) tool developed by the SURE collaboration (Appendix B).¹⁹ This checklist has been used for all EGMs supported by 3ie.¹⁷ The tool gives reviews an overall rating of high, medium or low confidence based on the assessment of three components: (1) methods used to identify, include and critically appraise studies; (2) methods used to analyse the findings; and (3) reliability of the review.¹⁸ The validity of the SURE checklist was tested using another critical appraisal tool (the Scottish Intercollegiate Guidelines Network checklist) in our original publication¹⁸; the test showed a high level of agreement between the two tests (kappa statistics 0.79). A summary of the methodological quality assessment was produced for each review and a confidence rating was assigned as follows: high confidence, if all review methods were systematic and appropriate, as described in the appraisal checklist, and the risk of bias was minimised; medium confidence, if all or some of the review methods were not systematic but the methodological limitations were acknowledged; and low confidence if most review methods were not systematic or appropriate, and the methodological limitations were not acknowledged or taken into account in the conclusions of the review. All appraisals were conducted by two reviewers independently. Any disagreements were resolved by discussion or the involvement of a third reviewer.

A short summary describing the methodology and key findings of each review was developed for sharing publicly on the EGM (e.g. https://research.sightsavers.org/gap-map/surgical-interventions-age-related-cataract/). Authors of all included reviews were contacted and provided with the summary and quality appraisal for verification. If the authors were able to provide additional information to support a change in the methodological quality assessment, this was sent to the original reviewers, who conducted another independent evaluation. However, there were no cases of this happening.

The reviews were displayed across the same themes and subthemes as in the original map, although the format of the map was slightly revised in 2020 following a user-testing study described elsewhere.²⁰

Results

Description of included reviews

Out of 948 unique records identified using our search strategy, 46 reviews met the inclusion criteria, resulting in a total of 98 reviews included in the updated EGM. The sifting process is described in Figure 2. Based on full-text screening, reviews were excluded, as the majority were genetic-focused studies or the techniques used were deemed irrelevant to LMICs, followed by a lack of reported methodological details, while a few identified were already reported in the original EGM (data not available).

Six reviews in the original map provided evidence in two subthemes each and thus were shown in the map twice, resulting in 104 entries on the map in total (Figure 3). All of the included reviews published since 2014 were in English. The majority of the reviews in the updated EGM (92/98) focused on age-related cataract and six reviews provide evidence on paediatric cataract.

Thematic focus

The majority of newly identified reviews were on types of cataract treatment (15/46) and risk factors and prevention (12/46). The remaining reviews provided evidence on quality of clinical care (10/46), access, uptake and equity of care (5/46), epidemiology/burden of disease (2/46), quality of life (1/46) and cost-effectiveness (1/46). There were no new reviews on the impact of cataract treatment on economic productivity or on cataract-related health systems.

Cumulatively, all reviews included in the updated map (a total of 104 due to 6 reviews reporting on multiple themes) provided evidence on cataract treatment (35/104), quality of clinical care (30/104), cataract risk factors and prevention (20/104), access, uptake and equity (8/104), epidemiology/burden of disease (5/104), cost-effectiveness/cost-benefit (3/104) and productivity/quality of life (3/104) (Figure 4).

Geographic representation within reviews

Figure 5 shows the number of reviews including at least one primary study from each of the six WHO geographic regions, and the themes of those studies. Thirty of the 98 studies do not report on the geographical location of the primary studies they include. Eleven reviews include primary studies from only one geographic region; 30 include studies from two regions; 12 include studies from three regions; 16 from four regions; 10 from five regions; and only 3 reviews include primary studies from all six regions. The Africa and Eastern Mediterranean regions have the fewest number of primary studies included in reviews (15 and 16 reviews, respectively), while Europe (52), Western Pacific and America (50 each) and South-East Asia regions (37) have the most.

Strength of evidence of effect

The distribution of the reviews based on the strength of evidence of effect has not changed much over time, with the majority of the reviews providing strong evidence in response to the research question (28/46 for the newly identified reviews and 55/98 for all reviews in the updated EGM). The remaining reviews were equally split between those where evidence in response to the research question was weak (7/46 and 19/98) and where it was mixed and inconclusive (11/46 and 24/98) (Figure 6).

Methodological quality of reviews

The majority of newly identified reviews (17/46) were rated as low confidence in findings and conclusions. Sixteen reviews were rated as medium confidence and 13 reviews were high confidence (Figure 7). Although the number of reviews rated as low confidence continued to be high (43/98), the number of reviews rated as high confidence has nearly doubled (27/98), and while in 2014 more than half of the reviews available were of low confidence (29/52), in 2021 nearly 60% of the reviews were either high or medium confidence (55/98).

Among 55 reviews that provided strong evidence in response to the research question, 11 were rated as high confidence, 19 were medium confidence and 25 were low confidence (Figure 8). Among 19 reviews that showed weak evidence in response to the research question, 10 were high confidence, 3 were medium confidence and 6 were low confidence. Among 24 reviews where evidence was inconclusive, 6 were high confidence, 6 were medium confidence and 12 were low confidence.

The updated cataract EGM is available publicly and can be accessed at https://research.sightsavers.org/evidence-gap-maps/cataract-gap-map/.

Discussion

Our EGM shows that cataract continues to be a high priority topic for systematic and literature reviews, with 46 new reviews published in 7 y since our original EGM. This is not surprising, as cataract continues to be the main contributor to the burden of blindness in low- and middle-income settings and has been a primary focus of the global efforts to eliminate avoidable visual impairment in the past 3 decades.^1,7 However, thematic distribution of the evidence base available continues to be skewed towards different aspects of clinical care and treatment of cataract. For example, the largest increase in the number of reviews in the past 7 y was observed in the area of cataract treatment, with many new reviews focusing on the efficacy and safety of clinical procedures or the medicines used during or after cataract surgery.

One interesting development in the available evidence base is the significant increase in the number of reviews of risk factors associated with cataract, including dietary factors such as consumption of vegetables²¹ or vitamin C,²² use of prescribed medicines, for example, antidepressants,²³ as well as a review of association of cataract with mortality.²⁴ This evidence opens up new opportunities for building links between cataract care and broader health systems and for integrating health promotion messages in primary and community care systems, as a measure to influence patient behaviour and reduce the fast-growing burden of cataract.

Another positive trend is the increased number of systematic reviews on barriers to cataract services,²⁵ patient education²⁶ and gender inequities.^27,28 We commend authors for undertaking these important reviews. Synthesis of evidence on access to services, health behaviour and health inequalities is extremely difficult, as primary studies producing such evidence tend to be heterogeneous and context specific. In addition, many of these studies are observational in design and often have methodological caveats or gaps in their presentation of findings. However, the availability of such evidence across settings is critical for addressing eye health inequities and progress towards UHC and we call for both more primary research and systematic reviews on these topics.

The overall methodological quality of the reviews included has also improved, with many new reviews being rated as high confidence. Based on the analysis of methodological quality of reviews in our original EGM, we used various opportunities to encourage researchers to follow more standardised approaches and guidelines for systematic reviews and to provide details required in the critical appraisal checklists. The apparent trend toward more transparent presentation of systematic reviews and resultant higher levels of confidence in the conclusions is very promising. However, the methodological quality of reviews concluding strong evidence of effect were much more likely to be of low or medium quality (44/55) compared with those concluding weak evidence of effect, which were more likely to be of high methodological quality (10/18). This indicates the need for continued advocacy on using standardised and transparent approaches, and for the users of reviews to remain alert to study limitations and the implications of the conclusions they draw.

On the less positive side, in the 7 y since the original EGM, there was only one additional review that examined links between cataract surgical rate and socioeconomic development and only one additional review on the impact of cataract surgery on quality of life. There also continues to be a dearth of evidence on cataract services and health systems, which is at stark odds with evidence needs. Recent publications have identified global disparities in eye health access and outcomes, and highlighted evidence of inequality within countries, with women, poor people and those living in rural areas particularly disadvantaged. Calls for equity-focused health systems cannot be answered without high-quality evidence of how this can be achieved in different settings.

Close inspection of the location of primary studies included in the reviews is hindered by lack of reporting in nearly one-third of reviews (30/98). Although crude, the map included (Figure 5) highlights a disparity in the geographic provenance of primary studies, with evidence from the WHO Africa and Eastern Mediterranean regions significantly underrepresented in reviews. It is worth noting that disparity also exists within regions, for example, the majority of included primary studies within Western Pacific Region were conducted in just two countries. Furthermore, some reviews may include multiple studies from one region, and just one from another, and they would be represented equally on the map.

Very few of the reviews report outcomes with a focus on equity, for example, only 5% of the reviews report outcomes-related gender equity. Because access to cataract services remains a major issue in low-income settings, particularly among certain population groups, the lack of evidence in these areas is of particular concern.

EGMs are highly dependent on the availability of primary studies that are available to be synthesised in systematic reviews. Thus, the lack of reviews on these topics is likely to be due to two factors, the lack of primary research published and the heterogeneity of the few studies that are available. This is in line with a finding reported by the Cochrane Eyes and Vision Group, who note that issues of equity are rarely considered within primary studies, which filters through to associated systematic reviews conducted by their group.²⁹ Our findings therefore call for more geographically diverse primary studies that report outcomes by different characteristics of equity and examine interventions around the health system blocks, including governance, financing, procurement, workforce and information systems. We also call for more standardised approaches to assessing and measuring the performance of eye health systems, so that the evidence can be synthesised across settings. Tools such as the PRISMA-Equity 2012 extension reporting guidelines can help researchers to ensure that their reviews adequately reflect the state of the evidence with regard to inequities.³⁰

Eye health is now globally recognised as an integral component of UHC and must be factored into national planning, resourcing and delivery of healthcare.^1,11 While evidence is clearly only one factor considered by policy-makers in their decision-making processes, it is important that the evidence they have access to is of high quality and relevant to the questions they face. The EGMs attempt to make clear the availability of quality evidence in a way that can be easily accessed by policy-makers. Although we have conducted some exploration into the usability of the EGMs,²⁰ the extent to which they are used by this audience, and the impact they have on decision-making, is unknown and warrants further investigation.

Conclusion

Our work on the updated cataract EGM has shown once again that EGMs are a useful tool, not only for providing a one-stop shop for the best available evidence, but also for monitoring trends in the development of the evidence base and closing evidence gaps. They are also useful in encouraging a discourse on the methodological quality of evidence available and thus encouraging more rigorous and transparent approaches to evidence generation. We have developed similar EGMs for other areas of eye care, including refractive error, glaucoma, diabetic retinopathy and trachoma. All these EGMs are available in the public domain and we would encourage policy-makers, programme managers and clinicians to use them in their day-to-day decision-making (https://research.sightsavers.org/evidence-gap-maps/). We would like also to encourage other researchers to use the EGM approach to collate and present the state of evidence in other areas. The approach may be particularly useful in complex areas, such as disability inclusion.

With regard to cataract, the evidence base relevant for decision-making in LMICs is substantial, but more primary studies and systematic reviews are required on the impact of cataract interventions, their cost-effectiveness, equity of services and broader aspects of cataract-related health systems, particularly in low-income settings.

Authors’ contributions

EJ, BV and ES conceived the study; BV, EJ and ES designed the study protocol; BV conducted searches and initial searching; BV, VP, MB and EM conducted full text review, critical appraisal, data extraction and drafted summaries. EJ and ES drafted the manuscript. BV, VP, MB and EM critically revised the manuscript for intellectual content. All authors read and approved the final manuscript. EJ and ES are guarantors of the paper.

Acknowledgements

The authors gratefully acknowledge the following people for their support in conducting the study: Iris Gordon for her support in conducting the searches and sifting based on titles/abstracts and Dr Baixiang Xiao for support in reviewing manuscripts written in the Chinese language.

Funding

None.

Competing interests

The authors declare no competing interests.

Ethical approval

This was a secondary review of previously published reviews and ethical approval was not required.

Data Availability

Data is availble at https://research.sightsavers.org/evidence-gap-maps/ or on reasonable request to the authors

References

Appendix A. Appendix A. Literature search strategy

The following databases were searched:

• Cochrane Library

• Medline

• Embase

• CINAHL

• WHO library (Index Medicus for the Eastern Mediterranean Region [IMEMR], Index Medicus for South-East Asia Region [IMSEAR],Western Pacific Region Index Medicus [WPIRM], Global Health and Africa-wide)

• LILACS (Latino Americana e do Caribe em Ciencias da Saude).

Appendix B. Appendix B. Checklist for making judgements about how much confidence to place in a systematic review of effects (adapted version of SURE checklist)ⁱ