Article Text
PDF
Abstract
Objective Guideline and protocol adherence in prehospital and in-hospital emergency departments (EDs) is suboptimal. Therefore, the objective of this systematic review was to identify effective strategies for improving guideline and protocol adherence in prehospital and ED settings.
Design Systematic review.
Data sources PubMed (including MEDLINE), CINAHL, EMBASE and Cochrane.
Methods We selected (quasi) experimental studies published between 2004 and 2018 that used strategies to increase guideline and protocol adherence in prehospital and in-hospital emergency care. Pairs of two independent reviewers performed the selection process, quality assessment and data extraction.
Results Eleven studies were included, nine of which were performed in the ED setting and two studies were performed in a combined prehospital and ED setting. For the ED setting, the studies indicated that educational strategies as sole intervention, and educational strategies in combination with audit and feedback, are probably effective in improving guideline adherence. Sole use of reminders in the ED setting also showed positive effects. The two studies in the combined prehospital and ED setting showed similar results for the sole use of educational interventions.
Conclusions Our review does not allow firm conclusion on how to promote guideline and protocol adherence in prehospital emergency care, or the combination of prehospital and ED care. For ED settings, the sole use of reminders or educational interventions and the use of multifaceted strategies of education combined with audit and feedback are all likely to be effective in improving guideline adherence.
- emergency medical technicians
- emergency medical services
- guideline adherence
- implementation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A strength of this systematic review is the extensive and systematic search of the literature and the inclusion of controlled study designs.
A second strength is the overview of strategies used, and their effectiveness in improving adherence to guideline and protocols in prehospital and emergency department settings.
A third strength is the use of a systematic and careful selection process, quality assessment and data extraction, all performed by pairs of independent reviewers.
A limitation of this study is the absence of a meta-analysis, which was impossible due to heterogeneity of designs, guidelines, methods and outcomes used in the studies.
A second limitation is that most of the identified studies came with a high risk of performance and detection bias.
Background
Clinical practice guidelines and protocols are developed to improve quality of care, to reduce variation of practice and to ensure that evidence is actually used when appropriate.1 A guideline consists of systematically developed recommendations to assist practitioners and patients in decisions about appropriate healthcare for specific clinical circumstances.2 A guideline recommendation is defined as any statement that promotes or advocates a particular course of action in clinical care.3 To assist implementation of guidelines, a protocol can be developed which yields a specification of a guideline and exactly formulates how to act and what steps to follow.4
Similar to other settings, guidelines and protocols have become an important aspect of prehospital ambulance care provided by physicians, paramedics and ambulance nurses (from now on referred to as prehospital), and in hospital-delivered emergency care (from now on referred to as ED) provided by emergency physicians and emergency nurses.5 6 Although patient-specific circumstances might lead to guideline deviations, in most cases, improved guideline adherence improves patient outcomes.7–9 Yet a relatively recent (2013) systematic review on guideline adherence in the prehospital and ED settings showed adherence rates to be highly variable and ranging from 7.8% to 95% for guideline adherence rates in the prehospital setting and 0%–98% for guideline adherence rates in the ED setting.10 Studies published after this systematic review show similar ranges.11–14 These results indicate that patients might be exposed to undertreatment or overtreatment and related risks.
To improve guideline adherence, it is important to identify implementation strategies which have been shown effective in improving adherence to guidelines and protocols. A commonly used classification for strategies is the EPOC Taxonomy (see online supplementary appendix 1 for selected examples of strategies and their definitions from the EPOC Taxonomy).15
Supplementary file 1
General literature on healthcare delivery improvement shows that opinion leaders, educational meetings, printed educational materials, educational outreach visits, and audit and feedback show positive effects in improving professional practice, although effects are small and do not always occur.16–20 To successfully select or develop implementation strategies for improved healthcare delivery and to optimise chances of effectiveness, it is generally recommended to tailor strategies to identified barriers and facilitators.21 Although combining and tailoring of interventions seems logical, literature is not totally decisive on the effectiveness of combining strategies as compared with the sole use of strategies.22 23 Tailored interventions can change professional practice with small to moderate effects,24 which might be due to a mismatch between the identified barriers and the designed or selected intervention.25 In addition, evidence is not compelling that a multifaceted intervention is more effective than a single-component intervention.26
Though the implementation literature at large offers guidance on the effectiveness of implementation strategies, for prehospital and ED settings it is unknown which strategies are effective to improve guideline and protocol adherence.27 As guideline adherence and implementation in prehospital and ED settings are influenced by multiple factors of which many are context specific,28–30 and as there are time constraints that might influence guideline adherence and the effectiveness of implementation strategies, a review for prehospital and ED settings is necessary. Therefore, the objective of this systematic review is to provide an overview of implementation strategies used, and to assess the effectiveness of these strategies in the improvement of guideline and protocol adherence in emergency care settings (including prehospital emergency care and in-hospital ED settings). More specifically, this review looks at the content and the effectiveness of single and/or multifaceted implementation strategies in relation to healthcare professionals’ adherence to guidelines/protocols in prehospital and emergency care settings.
Methods
A systematic review of the literature was performed according to the steps of the Cochrane Handbook for Systematic Reviews of Interventions.31 This review is reported in concordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement.32
Literature search
First, the Cochrane database for systematic reviews and the DARE database were checked for a similar review. Second, systematic searches were performed in PubMed (including MEDLINE), CINAHL and EMBASE in April 2018. Search strategies were structured to represent ‘terms for professionals’ OR ‘terms for settings’ AND ‘terms for strategies’ AND ‘terms for adherence’ AND ‘terms for guidelines/protocols’. Full search strategies per database are given in online supplementary appendix 2. Searches were restricted by year of publication (≥2004) to avoid outdated studies from the perspective of progress in prehospital and emergency care treatment and care. In addition to the electronic searches, we used included articles as a source to identify relevant studies by searching the reference lists.
Supplementary file 2
Study inclusion criteria
We included study reports published in any peer-reviewed scientific journal between 1 January 2004 and 10 April 2018. As for designs, (cluster-) randomised controlled trials and quasi-experimental designs (no random allocation) that described the use of one or more strategies to improve adherence to guidelines or protocols in the prehospital ambulance care and ED settings were considered for inclusion. The sole inclusion of (quasi) experimental designs is recommended for implementation effectiveness studies.33 Conference abstracts, editorials, personal communications or unpublished studies were excluded. Studies using simulation, training manikins or self-report methods were excluded since they insufficiently represent actual clinical practice and incorporate a risk of overestimation.34 The guideline or protocol described in the studies had to be a national or international evidence-based emergency care guideline or protocol. We included studies concerning all types of medical conditions/procedures. Local guidelines/protocols were excluded as their evidence base is often less explicit and/or not transparently reported.
Study selection
Pairs of two reviewers (RHAE, FS, URM, LCMV, TvA) independently screened the search results on title and abstract. Differences were then discussed and resolved, and in case of doubt, a third reviewer was asked. Articles were included if the title or abstract described attempts at improving adherence to a guideline or protocol in a prehospital or emergency care setting. The remaining articles were screened full text by pairs of two reviewers (RHAE, FS, URM, LCMV, TvA) independently. In addition, reference lists of included articles were screened (RHAE, FS) and potentially relevant publications were screened in a similar way.
Outcome measures and data extraction
For data extraction, we were guided by the Cochrane Effective Practice and Organization of Care Review Group (EPOC) Data Collection Checklist. From the EPOC perspective, this review focuses on assessment of change at the process level of care provided to patients. Therefore, one of the outcome measures of the study had to include professionals’ adherence to guidelines or protocols measured by observations or registration, and for which self-reporting was excluded. For transparent and uniform reporting on implementation strategies, we mapped the implementation strategies described in the studies to the EPOC taxonomy (2015 version). Though this enabled our overview and interpretation of results, some of the EPOC’s use of terminology and definitions can be debated. EPOC for instance defines audit and feedback as “A summary of health workers’ performance over a specified period of time, given to them in a written, electronic or verbal format. The summary may include recommendations for clinical action”. Given this definition, we did not classify instant feedback as generated through resuscitation devices (eg, instructing the professional to adjust the frequency or depth of heart massage) under this label, as the definition excludes immediate feedback. Instead, we decided this was closest to the EPOC definition of reminders, being “Manual or computerized interventions that prompt health workers to perform an action during a consultation with a patient, for example computer decision support systems”. However, we recognise that instant feedback, as generated and delivered through resuscitation devices, would fit the definition of feedback as found in other, more theoretically informed classifications such as the Taxonomy of Behavior Change Methods,35 which defines feedback as “Giving information to individuals and environmental agents regarding the extent to which they are accomplishing learning or performance, or the extent to which performance is having an impact”.
We extracted the following study characteristics: setting, design, methods, involved organisations, professionals targeted, units of measurement, the guideline and the type of intervention/improvement strategy used. Data were extracted by two independent researchers (RHAE, FS, LCMV). If these data extractors disagreed, the difference was discussed and they tried to reach consensus. If consensus was not reached, a third reviewer (LCMV and TvA) decided. The EPOC taxonomy15 was used to classify the strategies in well-defined categories, which was again done by a pair of two independent reviewers with the same procedure to reach a final decision (RHAE, FS, URM, TvA).
Intervention reporting
Studies describing usage of all types of implementation interventions to improve adherence to guidelines and protocols were included; no interventions were excluded. Interventions were analysed and reported to conform to the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations,36 meaning a detailed description of the intervention, clarification of assumed change process, access to intervention manuals/protocols and a detailed description of active control conditions. All interventions were assessed according to the WIDER recommendations by a pair of two independent reviewers (RHAE, FS, URM, LCMV). In the first round, there was an agreement on an average of 15.7 out of 20 items. After discussion, we reached total agreement. Furthermore, for each intervention, we described if the intervention was tailored at identified influencing factors (barriers and incentives identified before the intervention was developed and applied) and if the intervention was multifaceted or single-component.
Quality assessment of included studies
To assess the risk of bias, we used the ‘risk of bias assessment tool’.31 This tool is a domain-based evaluation to assess selection bias, performance bias, attrition bias, detection bias and reporting bias. For non-randomised studies, the Cochrane collaboration recommends to add additional domains. Therefore, we added the domain randomisation (yes/no). The quality assessment was performed by two researchers (RHAE, FS, URM, TvA) independently. After the first round of quality assessment, there was agreement on an average of 6.7 out of 8 items. After discussion, in the second round of quality assessment, we reached agreement on an average of 7.9 out of 8 items.
Data synthesis and presentation
Due to heterogeneity of the studies with regard to clinical topics, patient populations and outcome assessment, a meta-analysis was not possible. Instead, we extensively analysed and synthesised the studies by describing strategies used and their effects in detail. As most studies used multiple primary outcomes, we only concluded positive effects were found if significant and positive results were reported for at least two-thirds of these outcomes.
Patient and public involvement
There were no patients involved in this study.
Results
Review statistics
The initial electronic searches resulted in 1829 unique hits and successive selection rounds identified eight articles meeting the selection criteria. In addition, three articles were identified after searching the reference lists (figure 1), bringing the total number of articles to 11. During the full-text selection process, 126 articles were screened and 115 were excluded. Reasons for exclusion were no prospective intervention design (n=77), locally developed guidelines or protocols (n=13), adherence is not an outcome (n=11), another setting than prehospital or ED (n=8), having a guideline is the only intervention (n=5) and the full text article could not be retrieved (n=1).
Literature search flow diagram.
Study characteristics
The characteristics of the included studies (n=11) are displayed per setting in table 1. Nine studies were conducted in the ED setting and two were conducted in the combined setting of prehospital and ED. The studies were conducted in North America (n=4), Europe (n=3), Australia (n=3) and Asia (n=1). Professionals included in these studies were ED physicians, residents and medical officers; ED nurses; paramedics; and physician assistants. Four of the studies were monocentric, meaning they included one unit, either one ED or one prehospital Emergency Medical Service, while the rest were multicentric. Of the 11 included studies, nine were (cluster) randomised controlled trials and two were quasi-experimental studies (QESs).
Characteristics of included studies (n=11)
Prehospital setting
No studies in the prehospital setting were identified.
Combined prehospital emergency care and ED setting (n=2)
Two studies described adherence improvement initiatives in the prehospital and ED setting, one being a cluster-randomised controlled trial37 of relatively good quality, while the other had a quasi-experimental design38 with a higher risk of bias (online supplementary appendix 3). These studies focused on the implementation of guidelines on acute treatment of ischaemic stroke and the management of myocardial infarctions, respectively.
Supplementary file 3
As for the WIDER assessment (online supplementary appendix 4), the first study37 met all criteria for detailed description of the intervention, clarification of assumed change process and design principles, and detailed description of active control conditions. The other study38 only met six out of eight criteria for detailed description of the intervention; criteria for the other WIDER recommendations were not met. Both studies used sole educational implementation strategies and showed improvements in adherence rates (table 2).
Supplementary file 4
Results (n=11)
Emergency department setting (n=9)
Nine studies described adherence improvement initiatives in the ED setting. Three were cluster-randomised trials,39–41 five were randomised controlled trials42–46 and one was a quasi-experimental study.47 The quality assessment of the studies in the ED setting indicated overall higher risk for performance and detection bias, and lower risk for selection, attrition and reporting bias (online appendix 3). The guidelines covered in the ED studies related to the ABCD approach,39 asthma,40 42 47 pulmonary embolism,41 44 triage,45 acute gastroenteritis43 and foot/ankle complaints.46
None of the studies conducted in the ED setting fulfilled all four WIDER recommendations (online appendix 4). Only one study met all criteria for recommendation 1 (detailed description of the intervention), and the majority of the studies reported the characteristics of the recipients, the setting, the mode of delivery, the intensity and a detailed description of the intervention content provided for the intervention group. Two studies met all criteria for clarification of assumed change process and design principles, with the rest of the studies reporting either the development of the intervention or the change techniques used while failing to provide details on the causal processes targeted by these techniques. Three articles offered access to intervention manuals or protocols, either within the article itself or through a reference to an earlier published article. Out of all WIDER recommendations, criteria for the detailed description of active control conditions were met the least. Only one study reported on all criteria for the detailed description of active control conditions.
In the ED setting, eight out of nine studies showed positive effects on guideline adherence.39–41 43–47 Six studies used educational components in their implementation strategy.39 40 42 43 45 47 Of these, three combined education with audit and feedback, and all showed improved guideline adherence.40 45 47 Two other studies combined education with reminders,42 43 one of which showed positive effects on guideline adherence.43 One study solely used educational implementation strategies and showed improvement in adherence.39 Two studies used reminders as a sole intervention and showed improvements in adherence.41 46 Finally, one study used audit and feedback as a sole intervention and showed positive effects on guideline adherence.44
Tailoring in relation to outcome
In our review, only two studies used strategies tailored to preidentified determinants for adherence.45 47 Both studies showed significant improvements in adherence.
Discussion
This systematic review attempted to give an overview of strategies used in improving adherence to guideline and protocols in emergency care and their effectiveness. In total, 11 studies were identified.
The studies performed in the ED setting indicated that educational strategies such as educational meetings or distribution of educational materials as sole intervention or in combination with audit and feedback are probably effective in improving guideline adherence. We found similar results in the two studies of good to moderate quality, performed in the combined settings of prehospital and ED care; here, sole use of education also improved guideline adherence. Whether studies used educational meetings and/or distributed educational materials did not seem to make a difference to these results. Our findings on the sole use of education seem in contrast with systematic reviews indicating that educational meetings alone are not likely to be effective for changing complex behaviours17 or that when used alone, printed educational materials may have only small beneficial effects on professional practice outcomes.18 Perhaps this is due to limited awareness or knowledge of existing guidelines and protocols among emergency care professionals,28 awareness that can be enhanced by the sole use of education.
Another finding from our review is that sole use of reminders to improve guideline and protocol adherence in the ED was effective. This finding can be understood from general findings on using reminders to improve healthcare delivery indicating that the effect sizes of reminders are larger than those of most other strategies.48 Also, reminders included in the projects on ED care were usually automated and very instant and specific reminders, which could have positively contributed to their effectiveness.
The identified studies performed in the ED setting also indicated that combining implementation strategies is often effective in increasing adherence to guidelines and protocols. This finding is in line with conclusions of several other reviews which already concluded that multifaceted implementation strategies are likely to improve practice49 and that addressing combinations of different determinants shows good results.50 The latter of these two reviews addressed adherence to hand hygiene prescriptions in hospital nurses and indicated that effect sizes went up with each additional determinant of adherence being addressed by the implementation approach. Whereas Huis et al found that using strategies addressing up to five determinants of practice was effective, we found that fewer strategies per implementation approach could also be effective. In our review, studies reported on one or two different strategies in their approach. Also, the variation in strategies was relatively limited (n=5 unique strategies: educational meetings, educational material, educational outreach visits/academic detailing, reminders, and audit and feedback) in relation to the range of options as indicated in taxonomies of implementation strategies (eg, over 100 types of strategies in EPOC subcategories).
Our review also showed that most of the studies identified through our systematic search were conducted in the ED setting, and fewer in the combined settings of prehospital and ED care. Although guidelines and protocols are an important aspect of prehospital care and guideline adherence can be suboptimal,10 we did not identify rigorous research on effective implementation strategies. Yet, the need for well-designed experimental studies on the effectiveness of such strategies is clear from previous studies describing that there is a time gap between publication of a guideline and actual usage and adherence.51–53
A final finding from our review is that only two studies tailored implementation strategies to previously identified factors influencing adherence. This is despite the general recommendations and evidence for the need to tailor strategies,21 24 and also supports the relevance of conducting rigorous research on the identification of influencing factors for guideline adherence in emergency care settings.29
Limitations of included studies
A first limitation is that most of the included studies came with high risk of performance and detection bias, although risk for selection, attrition and reporting bias was lower. Also, no studies addressed strategies to improve guideline adherence in the prehospital setting, and only two studies addressed the mixed prehospital and ED settings. The need to assess effectiveness in these (combined) settings is urged by growing attention for implementation in (combined) emergency care settings.54–56 In addition, we considered reporting on guideline and protocol adherence after discontinuation of implementation strategies to assess sustainability of effects. However, this was not possible given the limitations of the included studies. Several studies had no follow-up other than directly after strategy delivery, several studies mentioned a follow-up but not its results, and studies that reported on follow-up used highly variable follow-up periods and formats (reporting on a single point in time vs reporting on adherence over the period of a whole year).
With regard to the WIDER recommendations, overall reporting on the nature of the implementation strategies used in interventions group was suboptimal, and descriptions of what was done in control groups was poor. Regarding the detailed description of the intervention, most of the studies described the characteristics of recipients, the setting, the mode of delivery, the intensity, the duration and a detailed description of the content (criteria 2–6, 8). The characteristics of those delivering the intervention and adherence to delivery protocols were less well described. Only two studies gave information relevant to all criteria for the clarification of assumed change processes and design principles, with item B (the change techniques used) being described the most. Only three studies provided access to intervention manuals/protocols. As for description of the control conditions, three studies described eight or nine criteria; the other studies described less or no criteria. This poor reporting of change strategies is reported in literature,57 58 and limits the replicability and generalisability of used strategies.
Study strengths and limitations
We conducted our review in concordance with the EPOC taxonomy, used the PRISMA statement for transparent reporting and applied the WIDER recommendations to report the interventions. Our procedures were up to date and in complete concordance with these tools. Also, we had a high inter-rater reliability during the assessment of quality of the studies included in this systematic review, with an average agreement of 6.9 for the eight items. Usage of the EPOC taxonomy (2015 version) implies that our results and conclusions should be seen in the light of the EPOC taxonomy as a reference for use of terminology. Although the EPOC taxonomy is widely applied, other taxonomies of implementation strategies and behaviour change techniques are developed and described.59–62 A limitation of the EPOC relates to the absence of clear links to relevant theory or determinants of practice for many of its strategies. For this, some of the alternative taxonomies could provide more guidance.
As for limitations, we were unable to do a meta-analysis due to a high heterogeneity of guidelines and methods used in the studies. In relation to this, we looked for significant effects in study outcomes and did not so much focus on effect sizes, which was also hindered by incomparability of outcomes and the way they were reported. This implies that we were able to arrive at general conclusions, but perhaps not to cover all relevant details on effects. Furthermore, the cut-off point of at least two-thirds of the outcomes showing significantly positive improvements was used to conclude on positive results, but the exact cut-off point is necessarily arbitrary. Another limitation concerns the fact that we were not able to retrieve the relationship between length of time following guideline implementation and outcomes.
Finally, excluding studies on the implementation of local guidelines and protocols from our review might have influenced our results. We excluded these studies as (1) from perspective of evidence-based practice only guidelines and protocols with a solid evidence base should be implemented, (2) the generalisability of local guidelines and protocols might be less clear or restricted to the specific context, and (3) as findings from these studies might not be as informative to professionals outside this specific context. We can only speculate as to how this could have influenced our results.
Conclusions and further recommendations
There is a lack of rigorous experimental studies to inform the promotion of guideline adherence in the prehospital care setting, while there is limited evidence for the ED setting. For ED settings, sole use of educational interventions, sole use of reminders, and using education combined with audit and feedback are all likely to improve adherence to guidelines. As reporting on change strategies is poor, replicability of implementation strategies is limited. Finally, factors influencing guideline adherence in emergency care settings are little considered in care quality improvement studies, thus hindering tailoring the implementation strategies.
Based on these conclusions, we offer several suggestions for research and practice. First, we have found that implementation strategies targeted at guidelines adherence in the prehospital settings and in the combined chain of care are limited. Thus, in the future, more studies targeted at prehospital care and across emergency care settings are needed. As for study quality, performing more rigorous RCTs or QES studies should be encouraged for all settings in order to reduce the risk of bias. In addition, change strategies should be reported using the WIDER recommendations. Second, our results indicate that further research on effectiveness of tailored implementation strategies in prehospital and emergency settings should be encouraged. Third, reminders seem to be an important change strategy when it comes to increasing adherence to the guidelines and protocol at the ED, given their usually larger effect size. Integrating this strategy more frequently may in fact improve results. Finally, a rather limited range of strategies for improving healthcare practice seems to be used in emergency care. Some creativity in the selection of implementation strategies in the sense of using more and different combinations of implementation strategies from the EPOC taxonomy can possibly increase the level of adherence and improve the quality of care further, although tailoring strategies to context should always be taken into account.
References
Footnotes
Contributors Study design: RHAE, URM, LCMV, TvA. Data collection and analysis: RHAE, FS, URM, LCMV, TvA. Quality assessment and WIDER assessment: RHAE, FS, URM, LCMV, TvA. Manuscript preparation: RHAE, FS, URM, LCMV, TvA. All authors read and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Files from the study selection process are available.