The development of a geriatric postgraduate education assessment instrument using a modified Delphi procedure

Abstract

There is currently wide variation in the structure and content of higher medical training in geriatric medicine across Europe and no common framework within which existing efforts can be compared. We set out to develop an audit tool to compare training between countries. An initial review of indexed and grey literature was used to develop an audit tool which was used as the basis of an Internet-based modified Delphi process incorporating the views of 14 expert geriatricians from across Europe. Items in the audit tool were included or excluded when supported by ≥75% or <50% of respondents, respectively. Items supported by 50–74% of respondents were carried forward with additional suggestions and modifications included following Round 1. Thirteen experts representing 12 countries responded to both rounds. 40/45 items were supported at Round 1. Five items were carried forward. A further 13 elements were introduced for consideration at Round 2. Consensus was gained after the second round. The final tool describes 52 items across four domains: general considerations, topics referring to knowledge in patient care, different roles that should be considered in medical training and topics regarding assessment. The resulting tool can be used as a basis for comparing higher medical training programmes in geriatric medicine between countries. Individual countries can use this to audit current practice. At an European Union level, the insights gained through such audit will form the basis of future work to develop an agreed postgraduate curriculum in the specialty.

Introduction

Global population ageing has been identified by the World Health Organisation as ‘the next global health challenge’ [1]. Traditional training for healthcare professionals has resulted in a skills deficit that will render health services unable to meet this challenge. Medical training, in particular, has historically focused on a reductionist medical diagnostic paradigm which is inadequate when faced with multi-morbidity and frailty [2]. Evidence-based care for older people with frailty, in contrast, requires a multi-domain, multi-disciplinary approach, feeding into iterative and case-managed care [3].

Recent consensus building across 29 European countries, led by the Union of European Medical Societies (UEMS), established a recommended undergraduate curriculum in ageing and geriatric medicine [4]. While this established the core concepts of ageing which all doctors must grasp, there is a need to agree higher level medical competencies for doctors routinely required to provide more specialist care to patients with frailty.

We are some way from such consensus. The format of geriatric medicine varies substantially between European countries, and some countries have not yet established postgraduate training in the discipline [5, 6]. A recent survey on postgraduate specialist training in Europe found 24 of the 31 countries had a recognised curriculum [7]. However, these differed considerably in terms of duration of training, specified core competences and mechanisms for assessment and certification [8]. Some countries reported structured and accredited programmes while others had informal mechanisms for training. In countries where training was less formal, it was possible that core competencies might be covered, but fail to be recognised as ‘in scope’ and hence not reported.

To better facilitate a shared understanding of postgraduate training in Geriatric Medicine across Europe, we set out to develop an audit tool to enable mapping of training programmes.

Methods

Development of the initial version of the instrument

An initial version of the template was developed by two experts in geriatric medicine and medical education (K.S., R.R.W.) using the information gained from a literature review referring to postgraduate education in geriatric medicine since 2005 and consideration of six European postgraduate curricula identified during our previous survey work (from the Czech Republic, Ireland, Romania, the Netherlands, Switzerland and the UK).

The initial version of the template described four main domains and related objectives specified to enable comparison between different national curricula. The four domains were as follows: general considerations (Domain 1); knowledge in patient care (Domain 2); roles that should be considered in medical training (Domain 3) and assessment in postgraduate education (Domain 4). The initial version of the template was piloted in a small group of experts in geriatric medicine comprising participants and members of the European Academy for the Medicine of Aging (EAMA), who were consultant geriatricians from a number of member states with English as a both first and second language, to resolve comprehension problems. Wording was changed according to the feedback.

Expert panel and modified Delphi process

To further refine the template, an international expert panel was invited to participate in a modified Delphi process [9, 10]. Of the 15 experts invited by mail, 14 agreed to participate. Thirteen were consultants in geriatric medicine and one in palliative care medicine. All had an additional degree or special interest in medical education, indicated for example through membership of the Special Interest Group of the European Union Geriatric Medicine Society, or did research in postgraduate medical education. Names and affiliations of the panel are described in Supplementary data, Table S1, available in Age and Ageing online. An initial version of the template was sent as an Internet-based questionnaire to the panel in March 2014. This included 12 elements in Domain 1, 22 in Domain 2, 7 in Domain 3 and 5 in Domain 4. Panel members were asked to select items they felt to be important when comparing curricula for postgraduate training in geriatric medicine. For all domains, there was the possibility to suggest changes or missing content. Responses were counted, and feedback of the panel was evaluated. Items with <50% and >75% acceptance were excluded from or included in the template, respectively. Items with an acceptance rate between 50 and 75% were re-evaluated in a second round. Comments and suggestions from the panel were evaluated, condensed and integrated in the domains of the second version of the template and sent out again for voting. In the second round, the panel was asked to decide on 20 elements in all four domains. Elements accepted by ≥75% were included in the template at this round, with all other items excluded.

Results

Of the 14 experts who agreed to take part, 13 participated in both decision rounds, representing 12 European countries. During the first decision round, 40/45 elements were accepted by ≥75% of the panel members. The remaining five items had 50–74% acceptance and therefore entered the second decision round. After evaluation of the expert comments, another 13 elements were identified as missing from the first draft and included in the second round. Expert comments mainly focused upon the degree of specificity of existing elements and adding additional aspects to Domain 2. Exact numbers of accepted elements in the first and second decision round as well as the number of added items following panel review are summarised in Table 1. The final draft of the template after the second round is shown in Table 2.

Discussion

This paper describes part of a continued programme of research and consensus building about what represents core teaching in geriatric medicine and ageing across Europe. Its main output is an audit tool for higher level specialist training in geriatric medicine (Table 2). We present this to describe the ongoing research journey but also to provide colleagues across Europe with a tool to enable audit and reflection upon current educational practice. The next research step will be to use this as a common template to gather more detailed descriptors of postgraduate programmes across Europe. Core requirements of learning outcomes are that they should be achievable and realistic [11]. We believe that a detailed understanding of the variation in current practice is necessary to arrive at a realistic starting point ahead of a final consensus process on a curriculum. These next steps, by virtue of their complexity, may take some time, and this is our rationale for presenting this intermediate phase in a paper.

The first domain of the audit tool considers components recognised as important constituents of a formal curriculum [12]. The second looks for curricular objectives regarding geriatric knowledge, attitudes and skills. They illustrate the broad remit of geriatric medicine including prevention, diagnosis, treatment and social aspects of illness in older people [13]. The third domain focuses on additional skills and attitudes that reflect the complex roles geriatricians have to fulfil [14, 15]. These include the ability to train other healthcare professionals to be involved in the care of older patients and to act as health advocates for this patient group [16]. The fourth domain describes assessment, which has been recognised to play an important role in curricular development. Beside its role to ‘drive’ the learning process it serves as a method to review the learning goals [17].

The main strength of this paper is that it describes a multinational consensus process, conducted in an objective and systematic fashion using a modification of the accepted Delphi methodology. The proposals put to the consensus group were derived in a robust fashion through previous survey work and review of indexed and grey literature. A high level of consensus was achieved suggesting that the audit tool has content validity. The main weakness is that only 14 European experts from 12 countries were consulted. We were therefore unable to canvas the broader range of opinions seen in recent UEMS undergraduate work. However, this audit tool was designed to compare current practice with those countries where higher medical training is most formalised and clearly specified. We wanted to be able to audit practice against what ought to be taught, rather than what can be taught within individual legislations. Indeed, as already described, establishing what can be taught by comparison with the gold standard is a core objective of our next phase of research. Thus, we canvassed opinions from experts in those countries where our previous survey had shown higher specialty training to be well established.

We encourage colleagues, now, to use this tool to describe their current practice. The data generated through such work can be used to support innovation and change or, where practice is particularly good, to highlight exemplary training in the specialty.

Conflicts of interest

None declared. The study/modified Delphi process does not have any funding source.

Acknowledgements

We thank the following colleagues for being available as expert raters (listed in alphabetical order): Cherubini Antonio, Cruz-Jentoft Alfonso, Ekdahl Anne, Franco Alain, Ellen Holm, Knight Paul, Lüttje Dieter, Münzer Thomas, Mulpeter Ken, Pulford Claire, Schiesl Christine, Strandberg Timo, Topinkova Eva and van der Cammen Tisha.

References