Methodology manual for European Association for Cardio-Thoracic Surgery (EACTS) clinical guidelines

Abstract

The goal of all clinical guidelines is to assist patients and practitioners in making healthcare decisions. However, clinical guidelines have been questioned about their quality, transparency and independence. Based on the revision of manuals by other scientific cardiothoracic organizations, this document provides instructions for the development of European Association for Cardio-Thoracic Surgery (EACTS) clinical guidelines and other types of evidence-based documents. Four key areas have been addressed: (i) selection of taskforce members and transparency of relations with the industry, (ii) methods for critical appraisal of medical evidence, (iii) rules for writing recommendations and (iv) review process. It is hoped that, by adopting this methodology, clinical guidelines produced by the EACTS will be well balanced, objective and, importantly, trusted by physicians and patients who benefit from their implementation.

INTRODUCTION

Clinical practice guidelines are ‘statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’ [ 1 ]. When treating patients, physicians and other healthcare providers are often faced with difficult decisions and considerable uncertainty. As well as their knowledge, they rely on the available scientific literature, personal experience and patient preferences to inform their decisions. As an aid for decision-making, clinical practice guidelines can be instrumental, because they include clear and concise recommendations intended to optimize patient care. Such recommendations reflect a balanced assessment of the benefits and harms of the alternative care options, and may either be evidence-based (if any evidence exists) or formed from a consensus of expert opinion [ 2 ]. Guidelines are only recommendations and, ultimately, the final decision about how best to treat a patient will be down to the physician's judgement and the patient's preference, with treatment tailored to the individual and their specific circumstances.

The objective of clinical guidelines produced by the European Association for Cardio-Thoracic Surgery (EACTS) is to help physicians involved in the diagnosis and treatment of patients with cardiopulmonary diseases to harmonize practice and improve the standard of patient care. The EACTS has produced several clinical guidelines to date, employing the best evidence topic methodology, which uses a systematic review protocol to construct each recommendation [ 3 ].

The EACTS consists of four domains: acquired cardiac disease, congenital heart disease, thoracic disease and vascular disease. The role of the EACTS Guideline Committee, which operates under the umbrella of the quality improvement programme initiative, is to define the rules for making EACTS guidelines; supervise their production; select the topics that should be included from those the EACTS domains and experts propose; establish relationships with other societies to enable collaboration and development of joint guidelines and outline how best to implement new guidelines.

In recent years, clinical guidelines have come under increased scrutiny, being accused of a lack of transparency, bias arising from panel members' relationships with industry (RWI) and associated conflicts of interest (COI), and confusion caused by different sets of recommendations being produced in answer to the same clinical question [ 4–7 ]. Similarly, the approach adopted by specialty societies to develop their clinical guidelines has been questioned, if not doubted, because they do not use a multidisciplinary panel approach to reduce the risk of bias when formulating recommendations [ 8 ]. As a result of these critiques, there have been calls for radical reform in the way in which guidelines are produced to allow for the return of trust [ 9 ]. Several issues deserve consideration, which include, but are not limited to: selection criteria for task force members, transparency of financial and intellectual RWI, the process of accumulating and summarizing evidence and the methods applied for grading the strength of evidence.

In this context, the EACTS Guideline Committee has decided to reassess its rules for the production of clinical guidelines and create a single reference tool, in the form of a methodology manual for EACTS clinical guidelines, which outlines the process that all future guideline development should follow. The ultimate goal is to increase physician and patient trust in clinical recommendations that adopt transparent and standardized methods.

METHODOLOGY

Numerous documents have been established on the process of developing clinical guidelines [ 10 ]. Based on a review of existing guideline manuals, the Guideline Committee proposes a methodology manual for EACTS with the objective of providing a standardized framework for guideline development [ 11–14 ]. In view of historical and ongoing collaborations between EACTS and other cardiovascular and thoracic societies, the methodology manual sets out, where possible, common rules with those adopted by these scientific societies.

Besides clinical guidelines, the EACTS publishes expert consensus statements, technology reviews and clinical statements. These publications provide expert opinion and evidence summaries on important focused topics too narrow or immature to deserve a full clinical guideline, but still influential in guiding patient management. The development of these documents follows a simplified in-house process, whereby the authors, assisted by an EACTS-appointed clinical methodologist/research analyst, perform the evidence review and analyses.

All types of document can be endorsed purely by the EACTS or, preferably, circumstances allowing, may be written and produced in collaboration with other scientific societies, depending on the topic. The same applies to clinical guidelines. The decision to include other disciplines or organizations belongs to the EACTS Council. Despite an increased organizational burden, joint publications with other disciplines increase their impact across continents other than Europe and with specialties other than cardiovascular or thoracic surgery.

The main characteristics of each of the four types of document the EACTS Guideline Committee commissions are summarized in Table 1 , and are discussed in more detail below.

Clinical practice guidelines

Clinical practice guidelines are ‘statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’ [ 1 ]. This report describes the methodology used to develop clinical guidelines by the EACTS. The process of guideline development is summarized in Fig. 1 and outlined in more detail later in the report.

Crucial for the development of well-balanced and objective clinical guidelines is that evidence analyses and review be performed jointly by a Task Force of experts (selected by the EACTS Guideline Committee) and an independent Evidence Analysis Organization (EAO) or clinical methodologist, which can also be assisted by a dedicated research fellow. Sufficient data from randomized studies should be available; if however, data are primarily from non-randomized studies, the Task Force will ensure the adequacy of the evidence provided; for example, it may be that the design of a particular study is not well suited to the question at hand or the type of analysis techniques employed are not appropriate.

Before they can be published, clinical guidelines undergo rigorous review by relevant organizations and practitioners under the supervision of the EACTS Guideline Committee in collaboration with the European Journal of Cardio-Thoracic Surgery (EJCTS) Editor-in-Chief. Following the review process, and successful approval by the EACTS Council, the final clinical guideline document is sent for publication.

Expert consensus statements

Expert consensus statements describe an expert position on an issue where there is controversy/uncertainty and where high-level evidence is not available to guide clinical recommendations. After defining the main clinical question and sub-questions, the expert Task Force performs a full systematic literature review and meta-analysis (if required). An expert consensus statement is composed of an introduction (including background information); methodology (used for the literature search); a summary of the evidence and the advocated statement with the respective level of evidence, strengths and limitations provided for each question and a conclusion. The expert consensus statement undergoes peer-review and requires approval by the EACTS Guideline Committee prior to submission for publication.

Technology reviews

Technology reviews are statements that are directed at an intervention and technology standardization.

Clinical statements

A clinical statement is a comprehensive report on an important topic written by a Task Force of experts (selected by the EACTS Guideline Committee), providing a historical overview, a summary of the present situation, a vision for the future and subsequent recommendations. The objective is to share the EACTS position on a clinical issue of interest.

RECOMMENDATIONS

The objective of the methodology manual for EACTS clinical guidelines is to provide a standardized framework for guideline development. There are four key areas that deserve special consideration:

• Organization of the Task Force, namely the selection of members and management of RWI and COI.

• Methods for critical appraisal of medical evidence.

• Rules for writing clinical guidelines.

• The review process.

The following sections of this report discuss each of these key areas in turn.

Task force organization

Selection of topics and chair

Topics are areas where there is a need for guidance to assist physicians in diagnosis, prevention and/or clinical management, and are chosen on the basis of the burden of disease, the existence of variation in practice and the potential to improve patient outcome. EACTS domains, in association with their respective working teams, select the topic and critical clinical questions and propose a chairperson to the EACTS Guideline Committee and the Secretary General. Individual EACTS members can propose topics, which require approval by the Domain Chair and the Guideline Committee. The Guideline Committee, in collaboration with the domains, determine the type of document and propose physicians (based on their expertise) to be invited as Task Force Guideline Chair and members; once selected, final approval from the EACTS Council is required.

Proposals and applications for clinical guidelines, expert consensus statements, technology reviews or clinical statements must include the scope, objectives, description of methodology and timelines.

The role of the Guideline Chair is to act as a facilitator who is willing to commit time and make themselves available to staff and committee members during the production process. A Co-Chair can be appointed to share responsibilities with the Chair. The responsibilities of the Guideline Chair, with support from the EACTS staff and Guideline Committee, include: selecting Task Force members in collaboration with the Guideline Committee, allocating writing assignments, managing RWI and COI [ 15 ], organizing and chairing Task Force meetings, coordinating drafting of the document, reviewing and editing the draft document, managing the review process and coordinating responses from and to reviewers and coordinating the production of executive summary/pocket guidelines (if required).

The Task Force Guideline Chair defines the clinical guideline work plan, sets out the timeline for completion and provides the Guideline Committee with regular progress updates.

Determining guideline scope and objectives

A preliminary literature search is made to identify relevant existing clinical guidelines, and establish the extent and quality of studies to determine the feasibility of a clinical guideline.

The Domain Chair, or any other domain member, together with the Guideline Chair outlines the scope and objectives of the clinical guideline by addressing the following basic questions:

• What is the guideline's targeted health condition, diagnostic test or procedure?

• What is the purpose of the guideline?

• What is within and outside the scope of the guideline?

• What is the target patient population?

• What is the literature inclusion date range?

• What are the important clinical objectives?

• What sub-topics must be included?

This is summarized by the Guideline Chair and Task Force members in the form of PICOT (Population, Intervention, Comparison, Outcome and Time) questions [ 16 ]:

• Population : The type of person (patient) involved. The population usually consists of a group of people with a disease of interest. It is important to be very specific in defining the patient population.

• Intervention : The intervention defines the treatment or diagnostic procedure being considered. The question almost always asks whether this intervention or the diagnostic test should be performed. For a diagnostic test, the alternative to the ‘intervention’—a positive test result—is a negative test result.

• Comparison : The comparison is the alternative to the intervention of interest, usually the standard of care, or control. For therapeutic questions, the comparison could be no treatment (or placebo) or an alternative treatment. For a population-screening question, the alternative is not to screen.

• Outcome : The outcomes to be assessed should include all outcomes clinically relevant to the patient. Indirect (or surrogate) outcome measures, such as laboratory or radiological results, should be avoided. For therapeutic questions, the relevant outcomes of interest are the effectiveness, safety and tolerability of the treatment. In diagnostic or prognostic questions, outcome relates to improving the physician's ability to predict the presence of the disease or the disease prognosis.

• Time : The time it takes to demonstrate an outcome, e.g. the time it takes for the intervention to achieve an outcome or how long participants are observed.

Selection of task force members and management of relationships with industry and conflicts of interest

The selection of the Task Force members depends on expertise, scientific contributions and capacity to deliver assigned tasks on time, and is made by the Domain Chair in collaboration with the appointed Guideline Chair and the Guideline Committee.

The Task Force writing committees are multidisciplinary and include, in addition to specialists from the involved domain, representatives from other domains or related specialties and their respective societies when relevant. An even geographical distribution is sought to avoid members originating from a limited number of countries. Inclusion of Task Force members with diverging/opposing views (pro and con opinions) on the clinical question at hand is sought to provide a balanced view on the topic.

The responsibilities of Task Force members include performing a full literature search of their respective topic, reaching consensus on the scope of the document, committing to actively participate in meetings/conference calls, respecting timelines, adhering to the RWI/COI policy, contributing to the writing and production of the document, addressing reviewers' comments and reviewing the final document before publication.

‘A COI exists when an individual's personal interests, e.g. direct and indirect financial or intellectual, have the potential to compete with or influence behaviour related to the individual's professional interests or obligations, i.e. evaluating the evidence and drafting recommendations for clinical practice guidelines’ [ 17 ]. RWI and other entities involved in the production, marketing, distribution or reselling of healthcare goods, services, advice or information consumed by patients, investors and/or physicians must be fully disclosed [ 11 ]. The EACTS has adopted a standard form for disclosure of COI/RWI [ 11 ].

Management of RWI/COI varies widely among scientific organizations, but reporting Task Force members' financial conflicts has become standard practice. Members with COI represent the minority of the Task Force and Chair(s) are free of COI [ 1 , 9 ]. Additional measures used to manage COI include the multidisciplinary structure of Guideline Committees and the involvement of a clinical methodology expert or an independent EAO in the evidence synopsis and guidelines integration.

The EACTS has adopted the recommendations of the Institute of Medicine standards for developing trustworthy clinical practice guidelines and the American Thoracic Society/European Respiratory Society workshop report on guideline funding and COI (Fig. 2 ) [ 1 , 17 ].

Confidentiality agreement

All Task Force members must sign a document biding them not to reveal any information relating to the content and development of the guideline or statement until publication. If any information is provided to third parties despite this agreement, the Task Force member risks suspension from participation in all current and future EACTS Guideline Committees.

Non-clinical staff support

It is advised to include a clinical methodologist/research analyst to assist physicians in performing systematic reviews and data analyses [ 11 , 13 , 14 , 16 ]. Expert consensus statements, technology reviews and clinical statements use this methodology. For EACTS clinical guidelines, evidence gathering/analysis can be assisted by a dedicated research fellow and is outsourced to either an independent EAO or to a commissioned clinical methodologist [ 18 ]. This process does not dispense each Task Force member from making a full literature search of their respective topic, but merely engages a permanent interaction between Task Force members and the evidence gatherer.

A document manager is also assigned to assist writers in the process of developing any type of document.

The EACTS office staff work alongside the Guideline Committee and Task Force, offering logistics support, organizing meetings and conference calls and providing regular progress reports.

Budget

The Guideline Committee calculates the budget with support from the EACTS office staff, based on the expected number of meetings during the allocated time. Expenses cover meeting costs, travel, accommodation, food and meeting facilities. As a general rule, face-to-face meetings are held at EACTS House in Windsor and during Annual Meetings, and conference calls are used as much as possible. In addition to meeting expenses, the potential cost of an EAO or charges related to a commissioned clinical methodologist or dedicated research fellow must be added.

Identification and critical appraisal of medical evidence

Physicians may not have the appropriate expertise for the critical appraisal of medical evidence, since study designs are getting more complex and often involve sophisticated statistical analyses [ 19 ]. Therefore, for physicians who refer to scientific evidence to guide their clinical decisions, it may be difficult to assess the quality of the research, whether the data reported are reliable and trustworthy, and whether the conclusions drawn are in line with the presented data. Furthermore, physicians are susceptible to unconscious bias in selecting and grading the evidence when they are, directly or indirectly, involved in the diagnostic or therapeutic process being evaluated.

Systematic literature review

A systematic literature review includes searching the evidence, rating the risk of bias, deciding when to perform a meta-analysis, grading the evidence and formulating evidence-based conclusions. This process is reserved for clinical guidelines covering topics that have been the subject of substantial scientific publications, including randomized controlled trials. The literature search focuses on the best available evidence to address each key question and follows a predefined approach [ 20 ]. Based on the jointly developed PICOTs, the EAO/clinical methodologist develops a standardized protocol, which forms the basis of the overall evidence report, describing the inclusion and exclusion criteria and the databases used for the search. All processes, including the search strategy, inclusion and exclusion criteria, and the covered period, are documented and recorded. Following the evidence search, abstracts are filtered for relevance, and data are extracted from papers and summarized into tables.

Critical appraisal of medical evidence

The methodological quality assessment of individual studies begins with the search for bias based on existing checklists ( www.sign.ac.uk ) [ 20 ]. Steps that are followed include: (i) assessing the consistency of studies (statistical heterogeneity); (ii) evaluating whether the study results are relevant to a European population (external validity); (iii) determining the reliability of estimates of effect size in terms of relative and absolute risk (precision) and (iv) assessing the likelihood of publication bias [ 20 ].

The available evidence is then summarized into tables (including patient characteristics, results and study limitations). The Task Force reviews this information together with the EAO/clinical methodologist at each Task Force meeting. In addition, contributing authors conduct their own literature review, enabling them to be directly involved with evidence synopsis and apply their informed judgement when deriving recommendations. A final report, containing an executive summary, methods, results with evidence tables, recommendations and a reference list, is produced by the joint collaboration of the Task Force and EAO/clinical methodologist.

For expert consensus statements and technology reviews, the EACTS may appoint a clinical methodologist/research analyst to assist the Task Force in selecting the PICOT questions, performing the literature search, conducting a meta-analysis if required and grading the evidence.

Rules for writing guidelines

How to write a recommendation

Recommendations are the ‘core’ of guidelines, with the text providing context, clarification and support. Unambiguous language and clearly defined terms are used to describe the patient population, the specific indication and to whom the recommendation applies. The wording, depending on the ‘class of recommendation’, must use the correct verb (Table 2 ).

Assigning classes of recommendation and levels of evidence

Definitions for ‘classes of recommendation’ and ‘levels of evidence’ are provided for guidance (Tables 2 and 3 ) [ 12 ].

Consensus achievement

Task Force discussions and consensus development are ongoing at all stages of clinical guideline production through conference calls, email exchanges and face-to-face Task Force meetings.

The manner in which an agreement or consensus is reached regarding recommendations varies [ 1 , 9 , 11 , 16 ]. Consensus can be reached through expert panel discussion and common sense, although quantification of expert opinion may be problematic and agreement will not always be reached in an expert panel discussion. Reaching consensus may follow a (modified) Delphi process, in which Task Force members vote for recommendations and receive an anonymous summary of the voting, after which voting rounds continue until a consensus is reached [ 21 ]. Lack of consensus, or the need to exceed a certain number of predefined voting rounds, is an indication to classify a recommendation as weak. EACTS clinical guidelines adopt a stringent transparent model whereby Task Force members with RWI and/or COI abstain from voting for recommendations where there may exist a potential COI. Lack of consensus is presented in the commentaries accompanying the recommendations to highlight the strength of the supporting evidence.

Document format

Evidence is summarized into tables that provide additional information, allowing comparison of study designs and results. Outcome measures are provided both in actual rates (e.g. percentages) and in the form of quantitative statistics (risk reduction, and odds or hazard ratios, with confidence intervals and P -values), to recognize the need for different measures to appropriately compare opposing treatment strategies.

Clinical guideline recommendations are provided in recommendation tables, with the class of recommendation and the level of evidence.

It is encouraged to incorporate visual summaries, for example, algorithms and/or diagrams, to help convey essential messages to readers.

Essential messages, summary cards and pocket guidelines that include recommendations, take-home messages and gaps in evidence are useful additional tools to improve clinical guideline dissemination and implementation.

Review process

At the start of the clinical guidelines production process, the EACTS Guideline Committee in collaboration with the EJCTS Editor-in-Chief, proposes five active members from the EJCTS (Associate Editors, Assistant Editors and/or Editorial Board members), who are experts on the topic and function as peer-reviewers. The reviewers' contribution is acknowledged by publication of their names in the final document. Information on RWI and confidentiality agreements is required from all peer-review members. The Guideline Chair and section writers respond to comments received by the reviewers and members, and the document is revised accordingly. All Task Force members must approve the revised version. The EACTS Guideline Committee then gives final approval for publication and the document is submitted to the EJCTS , and in the case of joint clinical guidelines, other flagship journals of participating societies.

Updating published guidelines

As medical practice evolves and new treatments or diagnostic options become available, it may be necessary to periodically review and update existing clinical guidelines or any other type of document. Ideally, documents should be updated as and when new evidence becomes available, which can result in either a partial/selective or a complete update. Partial updating of documents on a regular basis is often preferred over complete rewriting. The need for a Task Force and extent of consultation will depend on the nature of the changes required.

FUTURE PERSPECTIVES

The EACTS Guideline Committee recognizes four main areas for future work to increase the impact of guidance documents:

• Identify ‘gaps in knowledge’ and areas in which evidence is lacking. In these cases, guidance is more appropriately provided through expert consensus statements, based on expert opinion. Alternatively, evidence can be gathered through prospective multicentre registries or a Cardiothoracic Surgical Trials Network.

• Adopt a multidisciplinary approach for development of clinical guidelines by inclusion of patient organizations, clinical methodologists and consumer representatives.

• Evaluate implementation of recommendations, identify main barriers/obstacles for clinical guideline implementation and identify strategies to increase understanding and implementation of guidance documents.

• Evaluate the impact of clinical guidelines by determining their influence on patient health and other outcomes.

CONCLUSIONS

Clinical guidelines have been criticized for lack of transparency, bias, being specialty-oriented and subject to COI, leading to a lack of trust. Methodology for developing clinical guidelines must therefore be clear and should follow a set of predefined rules. We have reviewed the rules adopted by cardiovascular, thoracic and other societies to produce clinical guideline documents, and have set out a methodology manual for EACTS clinical guideline development, the key points of which are as follows: (1) Task Force Chairs and members are selected based on their expertise and have no or limited RWI/COI; (2) clinical guidelines are reserved for selected topics that have wide implications for patient care and for which there is extensive published evidence; in areas where there is little (or low-quality) evidence, an expert consensus statement is more appropriate; (3) for clinical guidelines the systematic search of the literature and subsequently the selection, appraisal and synopsis of relevant evidence can be assisted by a dedicated research fellow and outsourced to an EAO or clinical methodologist that complies with evidence processing standards.

It is hoped that by adopting this methodology, clinical guidelines produced by the EACTS will be well balanced, objective and, importantly, trusted by the physicians who reference them when making important decisions regarding the diagnosis and treatment of patients.

ACKNOWLEDGEMENTS

We thank Rianne Kalkman and Claire Hempson for editorial assistance.

Conflict of interest: Manuel Castella: Consultant for Atricure, Inc. and Medtronic, Inc. Philippe Kolh: Advisory Board member for AstraZeneca; honorarium (lecturing) from B. Braun.

REFERENCES