https://orcid.org/0000-0002-9781-2085
Abstract
Antimicrobial resistance (AMR) is a significant global public health threat and key priority for the public health, especially health protection, workforce to lead by example. There is a paucity of learning resources on this for public health professionals (PHPs) in the UK. This project aimed to develop and disseminate a tailored interactive learning resource and evaluate impact on self-reported intention to change behaviour.
Learning objectives were agreed, content developed by the multi-disciplinary team and piloted by PHPs in 2022 alongside a matched pre- and post-implementation evaluation survey. Questions were mapped to the capability–opportunity–motivation–behaviour change model. Before and after responses were calculated to compare change in self-reported knowledge, understanding and behaviour. Significance of change in binary responses was estimated.
The resource was delivered using an interactive, user-friendly and cost-free internal platform. Thirty-one PHPs completed the pilot e-learning and survey. Perceived and actual knowledge increased in parallel. Actual knowledge on AMR burden increased from 6.45 to 35.48% (P = 0.004). Self-reported confidence to explain AMR to others improved by 0.71 (95% CI; 0.38–1.04: P = 0.0001) Likert points on a five-point scale. Motivation to advocate for antimicrobial stewardship (AMS) in day-to-day work, improved by 0.71 (95% CI; 0.34–1.08: P < 0.00001) Likert points. Case scenarios were well-received as an effective way to apply theory to practice.
Ensuring a well-informed and confident public health workforce is vital for reducing the AMR threat and advocating for AMS with the public and partner organizations. This targeted e-learning module is an effective additional learning medium in contributing to PHPs knowledge, understanding and self-reported intention to change behaviours.
Introduction
Antimicrobial resistance—a public health priority
Antimicrobials comprise of antibiotics, antivirals, antifungals and antiparasitics. Antimicrobials are used in clinical and veterinary medicine, agriculture and farming. They are one of the most important tools in clinical medicine and public health in reducing morbidity and mortality from infectious diseases. Antimicrobial resistance (AMR) to those antimicrobials currently developed poses a significant global public health threat.1,2 It is estimated that by 2050 10 million lives will be at risk annually because of AMR.2 The UK’s national action plan highlights that tackling AMR is a key public health priority for the UK Health Security Agency (UKHSA).3
The public health workforce is responsible for protecting the public from the impact of health threats including AMR. As a significant health threat AMR demands engagement and education of broader public health professionals (PHPs) in tackling AMR, and PHPs should be leading by example tackling AMR and advocating for antimicrobial stewardship (AMS).4 In the UK, PHPs, including specialists in health protection—health protection practitioners (HPPs), work in several settings including national public health institutes, local authorities and health and social care settings.
Public health AMR educational resources and learning needs
Teaching resources, including e-learning modules on AMR and AMS are available for many healthcare professionals in primary and secondary care;5–7 however, a review of resources available in the UK and a 2019 learning needs assessment amongst PHPs in Southeast England of a national public health institute identified a gap in resources, specific to the public health workforce.8 This was also identified as a national learning need following advertisement of the resource through a national health protection meeting in March 2022 for all PHPs in England’s national public health institute (UKHSA) and from other external UK based public health organizations including the national public health institutes of the devolved administrations of the UK.9 Learning needs on AMR and AMS for the public health workforce in UKHSA were identified at different levels of cognitive learning identified in Blooms taxonomy framework.10
Bloom’s taxonomy is a hierarchical ordering of cognitive skills in levels, used to help those delivering educational materials to target training to individual learning needs. These levels of cognitive learning describe the learning behaviours that result from the learning opportunity. These include gaining knowledge (remembering), comprehension (understanding), application of the learning into practice and then higher levels or analysis, synthesis and evaluation. Learning objectives from standalone e-learning modules should address gaps identified, public health priorities and aim to achieve the first three levels of learning (knowledge, comprehension and application).11
Assessing the impact of educational resources
There are competing priorities on time and educational needs for PHPs and organizations must prioritize this range of learning needs and the time available for professional development. The utility of a resource to individuals choosing to use it for professional development is further impacted by its acceptability and relevance. To invest time in undertaking e-learning, professionals and organizations require assurance that a learning resource is impactful. The capability, opportunity, motivation and behaviour (COM-B) framework has successfully been used to develop and analyze the impact of educational materials on knowledge, understanding and self-reported intention to change behaviour.12,13 Studies using self-matched before and after surveys can be used to assess such behaviour change.14
Objective
Here, we describe the role of a multi-disciplinary team (MDT) in designing and piloting an e-learning resource targeted to professional groups in the public health workforce to successfully deliver on organizational priorities and learning objectives at the right cognitive level. We describe how the utility of the resource was optimized and estimate effectiveness of the resource in improving knowledge, confidence and self-reported intention to change behaviours including advocating AMS.
Methods
Developing educational content
The MDT comprised of medical and non-medical PHPs as opposed to the primary or secondary healthcare workforce, including epidemiologists, public health consultants and registrars, pharmacists and a nurse consultant with specialist knowledge and skills in health protection, AMR and AMS, inclusion health, adult learning and e-learning platforms. The MDT worked together to agree the learning objectives required, for an educational resource targeting this professional group.
Learning objectives were identified based on a learning needs assessment undertaken in a Southeast regional Health Protection team, specialist knowledge and from organizational priorities identified by the national public health institute.4 Learning objectives were mapped to cognitive learning levels in Bloom’s taxonomy. Interactive content, reading materials and methods of assessment were developed to deliver the material at the right cognitive level. Once development of the content, relevance to different professional roles and geographical regions, and appropriate length for completion in the working day, was agreed by the MDT, the content was uploaded to Xerte. Consideration was given to different learning styles and methods of delivery required to meet different levels of cognitive learning. Xerte is a UKHSA hosted, free and interactive shareable content object reference model (SCORM) platform.15
Piloting the resource and the matched pre- and post-survey
As a platform, internal only to UKHSA, the module was piloted by a convenience sample of volunteers and individuals nominated through regional AMR leads. Using a quota sample with the same target as previous studies of 0.1%,13 we sought to identify a minimum of 20 UKHSA volunteers amongst health protection professionals. The pilot, which was open August and September 2022, ran alongside a matched pre- and post-implementation survey (Supplement 1). The survey was completed anonymously, and matching was based on a unique identifier which the respondents created themselves. They could choose to create their own identifier or as a prompt we suggested the last three digits of their phone number and first three of a memorable month. Questions were designed to determine the utility of the e-learning package to users through an assessment of relevance, acceptability and engagement with the resource and satisfaction with the resource in meeting learning objectives. Survey questions were mapped to the capability–opportunity–motivation–behaviour change model to assess changes in knowledge, understanding, motivation to use learning and self-reported intention to change behaviour (Supplement 2).
A self-matched pre- and post-study design was used in-order to assess actual or self-perceived changes at the individual level before and after the intervention. It is the recommended study design for assessing digital (including e-learning) interventoins,14 as it can tell us about the effect of the learning resource, does not require randomization and can run alongside the use of the learning, ensuring all volunteers can access and benefit from the learning. Response bias can occur in self-reported surveys were respondents feel that the survey is directing them to acknowledge material they may have missed. We therefore created some non-existent resources, that respondents could select as true resources, as part of the survey as a method of testing knowledge whilst reducing the impact of response bias.
Results of the pre- and post-surveys
Results of the pre- and post-surveys were exported into a statistical software programme called STATA 17. Pre- and post-survey results were then linked on self-selected unique identifiers.16 Results of the surveys are described. Where comments and feedback were provided in a response to the survey, these are shown in the results as illustrative quotes. Where a before and after assessment was made, these are described with proportions from both surveys. Significance of change in binary responses was estimated using McNamar’s exact test, and for 5-point Likert scale responses (depending upon distribution) the paired t-test or Wilcoxon Signed Rank test. All questions were mandatory, therefore there was no missing data.
Results
Learning objectives
From previous learning needs assessment, organizational priorities and specialist knowledge, the MDT identified four key AMR specific learning outcomes for the module. Utilizing the specialist skills and knowledge of the MDT, the organization’s priorities for health protection and to vulnerable and inclusion health populations were met.17 Using educational theory, a variety of methods to deliver and assess learning were used. This included case studies which allow learners to practice using their new skills and knowledge to practice how they could use it in day-to-day Public Health practice. These methods and the content spanned the know, understand and apply levels of learning in Bloom’s taxonomy (table 1). The MDT identified 12 key areas in which PHPs can act and these are highlighted in the promotional material for the module (figure 1).
Promotional material for the AMR-learning module and 12 areas in which the public health workforce can act on a day-to-day basis to tackle the AMR threat and advocate for AMS.
Mapping learning objectives to cognitive levels in the Bloom’s taxonomy framework, and methods of delivery and assessment in the e-learning module
| Learning objective | Bloom’s taxonomy – cognitive level for learning objective | Delivery methods | Assessment method |
|---|---|---|---|
| To ‘describe’ what antimicrobial resistance is, the scale of the burden of antimicrobial resistance, reasons for its spread and the impact of AMR in the context of health inequalities | ‘Know’: Recall the facts and basic concepts |
| Summative quiz assessing
|
| To ‘know’ what advice and guidance exists and ‘understand’ which resources to signpost colleagues, other healthcare professionals and organizations to | ‘Know and Understand’: Explain ideas or concepts |
|
|
| To ‘demonstrate’ through interactive scenarios what actions you can take in your routine health protection practice to optimize antimicrobial stewardship | ‘Apply’: Use information in new situations | Case scenarios on:
| Response to scenarios with feedback |
| To ‘understand’ your role and the actions you can take | ‘Understand’: Explain ideas or concepts | Framework demonstrating health protection practice actions. Application of framework to case scenarios | Assessment of motivation to use learning in day-to-day role and to advocate for AMS |
| Learning objective | Bloom’s taxonomy – cognitive level for learning objective | Delivery methods | Assessment method |
|---|---|---|---|
| To ‘describe’ what antimicrobial resistance is, the scale of the burden of antimicrobial resistance, reasons for its spread and the impact of AMR in the context of health inequalities | ‘Know’: Recall the facts and basic concepts |
| Summative quiz assessing
|
| To ‘know’ what advice and guidance exists and ‘understand’ which resources to signpost colleagues, other healthcare professionals and organizations to | ‘Know and Understand’: Explain ideas or concepts |
|
|
| To ‘demonstrate’ through interactive scenarios what actions you can take in your routine health protection practice to optimize antimicrobial stewardship | ‘Apply’: Use information in new situations | Case scenarios on:
| Response to scenarios with feedback |
| To ‘understand’ your role and the actions you can take | ‘Understand’: Explain ideas or concepts | Framework demonstrating health protection practice actions. Application of framework to case scenarios | Assessment of motivation to use learning in day-to-day role and to advocate for AMS |
Mapping learning objectives to cognitive levels in the Bloom’s taxonomy framework, and methods of delivery and assessment in the e-learning module
| Learning objective | Bloom’s taxonomy – cognitive level for learning objective | Delivery methods | Assessment method |
|---|---|---|---|
| To ‘describe’ what antimicrobial resistance is, the scale of the burden of antimicrobial resistance, reasons for its spread and the impact of AMR in the context of health inequalities | ‘Know’: Recall the facts and basic concepts |
| Summative quiz assessing
|
| To ‘know’ what advice and guidance exists and ‘understand’ which resources to signpost colleagues, other healthcare professionals and organizations to | ‘Know and Understand’: Explain ideas or concepts |
|
|
| To ‘demonstrate’ through interactive scenarios what actions you can take in your routine health protection practice to optimize antimicrobial stewardship | ‘Apply’: Use information in new situations | Case scenarios on:
| Response to scenarios with feedback |
| To ‘understand’ your role and the actions you can take | ‘Understand’: Explain ideas or concepts | Framework demonstrating health protection practice actions. Application of framework to case scenarios | Assessment of motivation to use learning in day-to-day role and to advocate for AMS |
| Learning objective | Bloom’s taxonomy – cognitive level for learning objective | Delivery methods | Assessment method |
|---|---|---|---|
| To ‘describe’ what antimicrobial resistance is, the scale of the burden of antimicrobial resistance, reasons for its spread and the impact of AMR in the context of health inequalities | ‘Know’: Recall the facts and basic concepts |
| Summative quiz assessing
|
| To ‘know’ what advice and guidance exists and ‘understand’ which resources to signpost colleagues, other healthcare professionals and organizations to | ‘Know and Understand’: Explain ideas or concepts |
|
|
| To ‘demonstrate’ through interactive scenarios what actions you can take in your routine health protection practice to optimize antimicrobial stewardship | ‘Apply’: Use information in new situations | Case scenarios on:
| Response to scenarios with feedback |
| To ‘understand’ your role and the actions you can take | ‘Understand’: Explain ideas or concepts | Framework demonstrating health protection practice actions. Application of framework to case scenarios | Assessment of motivation to use learning in day-to-day role and to advocate for AMS |
Pilot findings
The pilot received 89 views over the two months it was open. Thirty-one participants from eight of nine UKHSA regions and two national UKHSA teams completed the pre- and post-pilot. Respondents included HPPs (n = 14), public health registrars and consultants (n = 11), scientists, information officers and programme managers (n = 6). Thirty (97%) reported the module relevant and useful to their work. Content imparting knowledge on what AMR is, why it is a problem, how it spreads, and available sources of guidance and advice was useful to 30 respondents, including all 14 (100%) HPPs. Understanding how to apply knowledge to day-to-day health protection work and how to stop spread was useful to 30 respondents including all 14 (100%) HPPs. Applying the knowledge and skills learnt earlier in the module to case scenarios was useful to 26 respondents, many of those who did not find it useful were more generalist PHPs and not in health protection roles. Of the 14 HPPs, 13 (93%) found it useful and one, who was new to the role was neutral (figure 2a and b).
Utility of content of the e-learning module to all respondents (n = 31) and health protection practitioners (n = 14), UKHSA, September 2022.
Case scenarios were the most commented on element of training, with respondents noting they ‘really liked the inclusion of scenarios—it really made you think’. Many respondents highlighted that the scenarios were the ‘most useful part’ and that they ‘enabled learners to put into practice what was previously covered’, ‘linking theory to practice’. Likewise, it was noted that scenarios, ‘showcase the work HPPs do’ and ‘the scenarios help HPPs appreciate that what they do does count towards AMS’.
Survey findings
Survey responses mapped to the COM-B model showed an increase in physical capability (actual knowledge after completion of the module) and a statistically significant increase in knowledge on the burden of AMR, from 6.45 to 35.48% (P = 0.004) (table 2). This was paralleled by an increase in psychological capability (perceived knowledge) with a statistically significant increase in perceived knowledge on causes of AMR from 67.7 to 96.8% (P = 0.01). Respondents’ knowledge of which existing resources they should signpost colleagues and stakeholders to also improved—significantly for Start SMART (a toolkit for English Hospitals) and TARGET (a toolkit for English GPs).6,7 To test response bias, we created non-existent resources as part of the survey. ‘Finish the Course’ was one of these non-existent resources, for which a non-significant increase in the number of respondents who would have referred to it was observed. For ‘Stop prescribing’ (another non-existent resource), the number who correctly would not signpost to it improved (P = 0.02).
Estimation of significance in change in response before and after completion of e-learning module, UKHSA, September 2022 (n = 31)
| Topic | Correct response (%) | Odds ratio (95% CI) of having more knowledgea | P-valueb | |
|---|---|---|---|---|
| Before | After | |||
| Capability—what is AMR? | 64.52 | 83.87 | 7.00 (0.90–315.48) | 0.07 |
| Capability—burden of AMR | 6.45 | 35.48 | Not estimable | <0.01 |
| Capability—AMR processes | 74.19 | 61.29e | 0.33 (0.03–1.86) | 0.29 |
| Capability—start SMART | 54.84 | 90.32 | Not estimable | <0.01 |
| Capability—TARGET | 58.06 | 93.55 | Not estimable | <0.01 |
| Capability—NICE | 90.32 | 93.55 | Not estimable | 1.00 |
| Capability—stop prescribing | 25.81 | 51.61 | 0.11 (0.00–0.80) | 0.02 |
| Capability—finish the course | 70.97 | 64.52e | 1.67 (0.32–10.73) | 0.73 |
| Confident in response (%) | Odds ratio (95% CI) of having more belief in knowledge | |||
| Perception—what is AMR? | 67.74 | 96.77 | 10.00 (1.42–433.98) | 0.01 |
| Perception—burden of AMR | 67.74 | 87.10 | 7.00 (0.90–315.48) | 0.07 |
| Perception—AMR processes | 90.32 | 93.55 | 2.00 (0.10–117.99) | 1.00 |
| Agree or strongly agree (%) | Mean increase in Likert points (95% CI) on 1–5 scale | |||
| Confidence—explain AMR to others | 74.20 | 100.00 | 0.71 (0.38–1.04) | <0.01c |
| Confidence—signpost guidance | 64.52 | 83.87 | 0.68 (0.29–1.06) | <0.01d |
| Understand—provide guidance | 67.74 | 87.10 | 0.71 (0.39–1.03) | <0.01d |
| Understand—use learning day-to-day | 70.97 | 83.87 | 0.10 (−0.36 to 0.55) | 0.71d |
| Motivation—advocate for AMS | 67.74 | 90.32 | 0.71 (0.34–1.08) | <0.01c |
| Agree or strongly agree (%) | Odds ratio (95% CI) of using learning | |||
| Motivation—will use learning | 41.94 | 80.65 | 5.00 (1.41–26.94) | 0.01 |
| Topic | Correct response (%) | Odds ratio (95% CI) of having more knowledgea | P-valueb | |
|---|---|---|---|---|
| Before | After | |||
| Capability—what is AMR? | 64.52 | 83.87 | 7.00 (0.90–315.48) | 0.07 |
| Capability—burden of AMR | 6.45 | 35.48 | Not estimable | <0.01 |
| Capability—AMR processes | 74.19 | 61.29e | 0.33 (0.03–1.86) | 0.29 |
| Capability—start SMART | 54.84 | 90.32 | Not estimable | <0.01 |
| Capability—TARGET | 58.06 | 93.55 | Not estimable | <0.01 |
| Capability—NICE | 90.32 | 93.55 | Not estimable | 1.00 |
| Capability—stop prescribing | 25.81 | 51.61 | 0.11 (0.00–0.80) | 0.02 |
| Capability—finish the course | 70.97 | 64.52e | 1.67 (0.32–10.73) | 0.73 |
| Confident in response (%) | Odds ratio (95% CI) of having more belief in knowledge | |||
| Perception—what is AMR? | 67.74 | 96.77 | 10.00 (1.42–433.98) | 0.01 |
| Perception—burden of AMR | 67.74 | 87.10 | 7.00 (0.90–315.48) | 0.07 |
| Perception—AMR processes | 90.32 | 93.55 | 2.00 (0.10–117.99) | 1.00 |
| Agree or strongly agree (%) | Mean increase in Likert points (95% CI) on 1–5 scale | |||
| Confidence—explain AMR to others | 74.20 | 100.00 | 0.71 (0.38–1.04) | <0.01c |
| Confidence—signpost guidance | 64.52 | 83.87 | 0.68 (0.29–1.06) | <0.01d |
| Understand—provide guidance | 67.74 | 87.10 | 0.71 (0.39–1.03) | <0.01d |
| Understand—use learning day-to-day | 70.97 | 83.87 | 0.10 (−0.36 to 0.55) | 0.71d |
| Motivation—advocate for AMS | 67.74 | 90.32 | 0.71 (0.34–1.08) | <0.01c |
| Agree or strongly agree (%) | Odds ratio (95% CI) of using learning | |||
| Motivation—will use learning | 41.94 | 80.65 | 5.00 (1.41–26.94) | 0.01 |
Not estimable is noted where the OR is not calculatable as no pre–post matched change in one group.
McNamar’s exact test unless otherwise denoted.
Paired t-test for 5-point Likert scale responses.
Wilcoxon Signed Rank test for 5-point Likert scale responses.
This is a negative change.
Estimation of significance in change in response before and after completion of e-learning module, UKHSA, September 2022 (n = 31)
| Topic | Correct response (%) | Odds ratio (95% CI) of having more knowledgea | P-valueb | |
|---|---|---|---|---|
| Before | After | |||
| Capability—what is AMR? | 64.52 | 83.87 | 7.00 (0.90–315.48) | 0.07 |
| Capability—burden of AMR | 6.45 | 35.48 | Not estimable | <0.01 |
| Capability—AMR processes | 74.19 | 61.29e | 0.33 (0.03–1.86) | 0.29 |
| Capability—start SMART | 54.84 | 90.32 | Not estimable | <0.01 |
| Capability—TARGET | 58.06 | 93.55 | Not estimable | <0.01 |
| Capability—NICE | 90.32 | 93.55 | Not estimable | 1.00 |
| Capability—stop prescribing | 25.81 | 51.61 | 0.11 (0.00–0.80) | 0.02 |
| Capability—finish the course | 70.97 | 64.52e | 1.67 (0.32–10.73) | 0.73 |
| Confident in response (%) | Odds ratio (95% CI) of having more belief in knowledge | |||
| Perception—what is AMR? | 67.74 | 96.77 | 10.00 (1.42–433.98) | 0.01 |
| Perception—burden of AMR | 67.74 | 87.10 | 7.00 (0.90–315.48) | 0.07 |
| Perception—AMR processes | 90.32 | 93.55 | 2.00 (0.10–117.99) | 1.00 |
| Agree or strongly agree (%) | Mean increase in Likert points (95% CI) on 1–5 scale | |||
| Confidence—explain AMR to others | 74.20 | 100.00 | 0.71 (0.38–1.04) | <0.01c |
| Confidence—signpost guidance | 64.52 | 83.87 | 0.68 (0.29–1.06) | <0.01d |
| Understand—provide guidance | 67.74 | 87.10 | 0.71 (0.39–1.03) | <0.01d |
| Understand—use learning day-to-day | 70.97 | 83.87 | 0.10 (−0.36 to 0.55) | 0.71d |
| Motivation—advocate for AMS | 67.74 | 90.32 | 0.71 (0.34–1.08) | <0.01c |
| Agree or strongly agree (%) | Odds ratio (95% CI) of using learning | |||
| Motivation—will use learning | 41.94 | 80.65 | 5.00 (1.41–26.94) | 0.01 |
| Topic | Correct response (%) | Odds ratio (95% CI) of having more knowledgea | P-valueb | |
|---|---|---|---|---|
| Before | After | |||
| Capability—what is AMR? | 64.52 | 83.87 | 7.00 (0.90–315.48) | 0.07 |
| Capability—burden of AMR | 6.45 | 35.48 | Not estimable | <0.01 |
| Capability—AMR processes | 74.19 | 61.29e | 0.33 (0.03–1.86) | 0.29 |
| Capability—start SMART | 54.84 | 90.32 | Not estimable | <0.01 |
| Capability—TARGET | 58.06 | 93.55 | Not estimable | <0.01 |
| Capability—NICE | 90.32 | 93.55 | Not estimable | 1.00 |
| Capability—stop prescribing | 25.81 | 51.61 | 0.11 (0.00–0.80) | 0.02 |
| Capability—finish the course | 70.97 | 64.52e | 1.67 (0.32–10.73) | 0.73 |
| Confident in response (%) | Odds ratio (95% CI) of having more belief in knowledge | |||
| Perception—what is AMR? | 67.74 | 96.77 | 10.00 (1.42–433.98) | 0.01 |
| Perception—burden of AMR | 67.74 | 87.10 | 7.00 (0.90–315.48) | 0.07 |
| Perception—AMR processes | 90.32 | 93.55 | 2.00 (0.10–117.99) | 1.00 |
| Agree or strongly agree (%) | Mean increase in Likert points (95% CI) on 1–5 scale | |||
| Confidence—explain AMR to others | 74.20 | 100.00 | 0.71 (0.38–1.04) | <0.01c |
| Confidence—signpost guidance | 64.52 | 83.87 | 0.68 (0.29–1.06) | <0.01d |
| Understand—provide guidance | 67.74 | 87.10 | 0.71 (0.39–1.03) | <0.01d |
| Understand—use learning day-to-day | 70.97 | 83.87 | 0.10 (−0.36 to 0.55) | 0.71d |
| Motivation—advocate for AMS | 67.74 | 90.32 | 0.71 (0.34–1.08) | <0.01c |
| Agree or strongly agree (%) | Odds ratio (95% CI) of using learning | |||
| Motivation—will use learning | 41.94 | 80.65 | 5.00 (1.41–26.94) | 0.01 |
Not estimable is noted where the OR is not calculatable as no pre–post matched change in one group.
McNamar’s exact test unless otherwise denoted.
Paired t-test for 5-point Likert scale responses.
Wilcoxon Signed Rank test for 5-point Likert scale responses.
This is a negative change.
Social opportunity to access the resource was assessed based on the acceptability of the module in terms of platform and time to complete the module. Respondents reported the module taking a median of 30 min (interquartile range 25–35 min, range 15–60 min) to complete. Thirty (97%) respondents reported that this was an acceptable amount of time during the working day and only one (3%) reported finding the platform hard to use. Physical opportunity to confidently use skills to explain AMR to others, signpost others and knowing when to provide guidance significantly improved (P < 0.05) in these three areas of the four assessing physical opportunity (Supplement 2). Confidence in explaining AMR to others was described by all post-survey respondents, with an increase from 74.2 to 100% (P = 0.0001) and motivation to use AMR skills day-to-day was self-reported to have increased from 41.9 to 80.6% (P = 0.01).
The number of respondents who self-reported they would advocate for AMS in day-to-day work increased from 67.7 to 90.3% (P < 0.0005). This shows a statistically significant increase in the self-reported intention to change behaviours in applying AMS skills and knowledge.
Changes to the resource
Feedback from the pilot identified 18 recommendations for change for the final module. Eleven changes were recommended to the presentation of pages, four were related to the case scenarios to make them more relevant to all regions and roles, two to change the amount of information on a page and the last recommendation to change the way a survey question was worded. These recommendations were used to improve and update content. The final version launched in World Antibiotic Awareness Week (18–24 November 2022)18–20 received 120 views in the first 10 days of its launch.
Discussion
Respondents working in England’s national public health institute from a wide geographical spread and a range of public health roles found the module useful, relevant and appropriate to AMR and AMS in their day-to day roles, thereby successfully meeting their learning needs. The content met the learning objectives set and organizational priorities and the methods of delivery and assessment, including the use of case studies, enabled successful self-reported knowledge transfer, a good level of understanding and gave participants the confidence to apply learning in the 12 areas identified that PHPs can act in. For HPPs case scenarios were a well-received way to show how learning could be applied to day-to-day work. There were some suggested areas for improvement; these were incorporated into the final resource. The survey also showed that capability opportunity and motivation to change self-reported behaviours related to AMR were successfully improved after completion of the learning.
Results show that for this specific target audience undertaking the learning module resulted in self-reported improved knowledge and understanding of AMR, AMS and related resources. Previous studies have shown that in the right setting e-learning can be an important pedagogical teaching tool focusing knowledge and encouraging application21 and in workplaces can be an effective way of delivering training flexibly to suit the needs of different employment groups.22 The materials designed to impart knowledge and test understanding were found to be useful by most respondents, and by all HPPs. The module was accessible for respondents from all roles and regions; this is reflected in the finding that both physical and psychological capabilities were improved in parallel. Gains in self-perception of knowledge comparable with actual knowledge change have been shown in other effective healthcare teaching modules.23 Confidence to apply this new knowledge is vital in realizing behaviour change and ensuring individuals and organizations can meet AMR and AMS public health priorities in routine work.
Previous studies have identified that 40 minutes is an ideal duration of e-learning for medical and nursing professionals, minimizing the impacts of display screen on physical health whilst allowing learners to remain focused and absorb a sufficient amount of content to meet learning needs.24 A duration of 30 min as the median time taken by respondents is appropriate without overwhelming the learner. The findings also suggested self-reported impactful outcomes on behaviour which makes this resource effective use of organizational time to meet a key organizational priority.
The use of case scenarios and interactive learning tools were the elements which satisfied the key target audience the most. They allowed users to see how learning in earlier phases of the resource was relevant to routine health protection work, how they already do contribute, and could expand their contribution to AMR and AMS.10,11 Use of progressive interactive case studies in e-learning modules has also been shown to improve performance in biomedical settings.25 Respondents to the survey reported that interactive elements and the case studies improved their engagement with the e-learning resource. Improved engagement with and enjoyment of the learning experience improves peer-to-peer advertisement of the resource and improves reach. E-learning has been shown to be a viable solution to augment self-directed learning as well as a way of promoting and encouraging continuous professional development on this topic to others in a professional learning community.26 Combining increased reach with a resource that is relevant, accessible and can be completed in work-time improves the chances that the target audience will be able to complete the learning.21 The results of the COM-B study show that participants self-report that they intend to use opportunities to apply their learning.
The design and development of the module to meet organizational priorities, deliver on key learning objectives and be evaluated to a level to show assurance of effectiveness would not have been possible without a multi-regional and MDT. The importance of mapping e-learning to key organizational priorities using digital technologies to do this, has been shown as a successful method of enhancing organizational learning.27 Whilst it was originally designed to meet one specific role in health protection, the breadth of experience of the MDT and results of the pilot study have shown that this resource effectively enhances learning on AMR and AMS for a range of PHPs, experiences and regions. Interest in this resource, driven by the gap in learning resources for this workforce, has been expressed by external organizations. The gap in provision for wider PHPs outside the national public health institute this resource was piloted in is therefore still present.
One of the main limitations of this project was in limiting content to ensure it did not complete with other educational priorities and could be completed in the working day. This meant we prioritized content that was the most relevant to the learning objectives. The resource is also currently hosted on an internal virtual manager platform, thereby limiting reach to the PHPs working in organizations external to UKHSA.15 Improvements have been made to the resource, including changing the wording of questions aimed to identify falsely named resources (the only area of deterioration seen). It is important that the resource is appropriately assured and updated to keep up to date with new findings and kept refreshed to ensure continued and relevant engagement—especially with case scenarios. This type of study can also only assess self-reported intention to change behaviour, an observational, possibly ethnographic, study of behaviours would be needed to see if the intention results in action.
Conclusions
We successfully designed, developed and launched a pilot e-learning resource on AMR targeted specifically to address a learning need of the public health workforce. The resource effectively contributed to improving knowledge and understanding of AMR and AMS and increased self-reported confidence and intended behaviour change in using skills and knowledge to meet AMR public health priorities in routine work. Launch alongside a pre- and post-survey not only enabled us to collect information on utility and impact, but also feedback to improve the resource for a future launch.
Use of case scenarios and other interactive elements relevant to routine health protection work were reported by respondents to be well-received and improve their engagement and enjoyment of learning. Use of case studies in particular allowed the MDT to successfully deliver content to achieve the application of knowledge levels of cognitive learning, the highest level of learning achieved by participants. Design and development of content and case scenarios by our multi-regional and MDT ensured content was relevant to our target audience, addressed the learning gaps identified and met all relevant organizational priorities.
Recommendations
We would encourage Public Health educators in countries with a public health workforce to develop professional group targeted AMR learning resources as clinical resources for frontline staff are less relevant for PHPs whose impact and influence is unique. We recommend those who are developing resources to use interactive content and case scenarios and to have a multi-regional and MDT approach to optimize relevance across different regions and organization and to reflect any slight differences in organizational public health priorities. We have found a matched pre- and post- behaviour change study completed alongside the launch of pilot educational resources informative to enable quantification and assurance of the self-reported impact and effectiveness of the resource on staff knowledge and behaviour. In the UK, we recommend that this resource continue to be used and promoted to existing members of UKHSA staff. The e-learning resource should be included in induction training programmes for new staff. Its relevance and effectiveness should continue to be monitored and content updated. We will also look to export to an external platform to enable positive AMS behaviours are adopted by PHPs in health protection and AMR roles in the UK for those internal and external to UKHSA.
Supplementary data
Supplementary data are available at EURPUB online.
Acknowledgements
We would like to acknowledge the work of Dalia Youssef and Nileema Patel in developing the concepts for the first iterations of a learning resource. We would like to thank Alicia Barrasa from UK FETP for her statistical advice and support. We would also like to thank Paul Cleary for his technical expertize and support with the Xerte platform.
Funding
This project received no funding.
Conflicts of interest: None declared.
Ethics
All who participated in the e-learning and pre- and post-surveys did so strictly in their professional capacity after attending a session on the pilot. Volunteers indicated interest and consent in participating by providing their e-mails. The survey was anonymous with participants creating their own identifier to link the surveys. Results were held securely in the UKHSA internal network in line with GDPR.
Author contribution
All authors were integral to the development of the case scenarios and e-learning resource. All authors contributed to the development of the survey questions. H.M.T. wrote the analysis protocol and undertook the analysis of results. All authors contributed to writing and reviewing this article.
Data availability
The source data used in the matched pre- and post-implementation survey analysis are available at Supplement 3.
Targeted AMR learning resources should be developed for public health professionals as a global public health priority, particularly those in health protection as this will empower them to use this knowledge and influence as impactful antimicrobial stewards.
AMR learning resources benefit from multi-disciplinary input and should include interactive elements and role-based relevant case studies, such as risk assessing contacts for chemoprophylaxis, offering infection prevention control advice and advocating for antimicrobial stewardship (AMS) when working with the public and other stakeholders.
This study has shown e-learning can be impactful and local use should be evaluated and kept up to date to ensure it best supports improvements in knowledge, understanding and motivation.
Comments