You are here

Article Text

Article menu
Endoscopy
Original article
Training and competence assessment in GI endoscopy: a systematic review
  1. Vivian E Ekkelenkamp,
  2. Arjun D Koch,
  3. Robert A de Man,
  4. Ernst J Kuipers
  1. Erasmus MC—University Medical Center, Rotterdam, The Netherlands
  1. Correspondence to Vivian E Ekkelenkamp, Erasmus MC University Medical Center Rotterdam, The Netherlands; v.ekkelenkamp{at}erasmusmc.nl

Abstract

Introduction Training procedural skills in GI endoscopy once focused on threshold numbers. As threshold numbers poorly reflect individual competence, the focus gradually shifts towards a more individual approach. Tools to assess and document individual learning progress are being developed and incorporated in dedicated training curricula. However, there is a lack of consensus and training guidelines differ worldwide, which reflects uncertainties on optimal set-up of a training programme.

Aims The primary aim of this systematic review was to evaluate the currently available literature for the use of training and assessment methods in GI endoscopy. Second, we aimed to identify the role of simulator-based training as well as the value of continuous competence assessment in patient-based training. Third, we aimed to propose a structured training curriculum based on the presented evidence.

Methods A literature search was carried out in the available medical and educational literature databases. The results were systematically reviewed and studies were included using a predefined protocol with independent assessment by two reviewers and a final consensus round.

Results The literature search yielded 5846 studies. Ninety-four relevant studies on simulators, assessment methods, learning curves and training programmes for GI endoscopy met the inclusion criteria. Twenty-seven studies on simulator validation were included. Good validity was demonstrated for four simulators. Twenty-three studies reported on simulator training and learning curves, including 17 randomised control trials. Increased performance on a virtual reality (VR) simulator was shown in all studies. Improved performance in patient-based assessment was demonstrated in 14 studies. Four studies reported on the use of simulators for assessment of competence levels. Current simulators lack the discriminative power to determine competence levels in patient-based endoscopy. Eight out of 14 studies on colonoscopy, endoscopic retrograde cholangiopancreatography and endosonography reported on learning curves in patient-based endoscopy and proved the value of this approach for measuring performance. Ten studies explored the numbers needed to gain competence, but the proposed thresholds varied widely between them. Five out of nine studies describing the development and evaluation of assessment tools for GI endoscopy provided insight into the performance of endoscopists. Five out of seven studies proved that intense training programmes result in good performance.

Conclusions The use of validated VR simulators in the early training setting accelerates the learning of practical skills. Learning curves are valuable for the continuous assessment of performance and are more relevant than threshold numbers. Future research will strengthen these conclusions by evaluating simulation-based as well as patient-based training in GI endoscopy. A complete curriculum with the assessment of competence throughout training needs to be developed for all GI endoscopy procedures.

  • ENDOSCOPY
  • GASTROINTESINAL ENDOSCOPY
  • ENDOSCOPIC PROCEDURES

https://doi.org/10.1136/gutjnl-2014-307173

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Significance of this study

    What is already known on this subject?

    • The use and validation of simulators in endoscopy training is extensively studied.

    • Moreover, the classic master–apprentice model for teaching endoscopy is nowadays not accepted anymore.

    • Threshold numbers for the assessment of competence in GI endoscopy are widely used, but might be outdated.

    What are the new findings?

    • There is sufficient evidence for the use of validated simulators in endoscopy training curricula. However, the extent to which simulator training should be carried out is a matter of debate.

    • Continuous assessment of a trainee's performance with a validated assessment tool is valuable and provides insight into the individual and group learning curve.

    • This is definitely more thorough than defining competence based on threshold numbers alone.

    How might it impact on clinical practice in the foreseeable future?

    • We would advocate a prepatient training curriculum on a validated simulator for flexible endoscopy. Moreover, the use of learning curves in patient-based training is a sound method for the assessment of competence. However, more research is needed in order to evaluate a complete endoscopy training programme: from novice to competence to excellence.

    Introduction

    The focus on training in procedural skills in GI endoscopy is shifting from threshold numbers towards an individual approach. This illustrates the awareness that the classic master–apprentice model may not reflect all necessary aspects of training. Moreover, the old adage “see one, do one” seems no longer appropriate for educating health professionals to perform complex technical procedures, such as flexible endoscopy.1 Virtual reality (VR) simulators may be of benefit in the education of gastroenterology trainees. However, a substantial part of training still has to be patient-based. The assessment of a trainee's competence is not clearly defined and competence benchmarks for trainees are sparse. The use of threshold numbers is nowadays considered a poor surrogate marker for competence. Keeping track of one's performance by measuring skill development seems preferable. However, training guidelines differ worldwide and there is no consensus on the skills a trainee has to possess at the end of education. On top of that, for most procedural skills in flexible endoscopy, the proper assessment tools to measure these skills are lacking.

    The aim of this systematic review was therefore to evaluate the available literature on different training and assessment methods in GI endoscopy. Second, we aimed to identify the role of simulator training and competence development in patient-based training, specifically for procedures that normally will be learnt during residency. Third, we aimed to propose a structured training curriculum based on the presented evidence.

    Methods

    Literature search strategy

    A systematic literature search was carried out in July 2013 in seven different medical and educational literature databases: Embase, Medline OvidSP, Web of Science, Cochrane central, Google Scholar, Research and Development Resource Base and Education Recourse Information Center. There was no restriction regarding the time of publication or language. The search strategy for Medline OvidSP is shown in supplementary table S1.

    Inclusion and exclusion criteria

    All studies pertaining to training and assessment in GI endoscopy (colonoscopy, esophagogastroduodenoscopy [EGD], endoscopic retrograde cholangiopancreatography [ERCP] and endosonography [EUS]) were included in this review. The studies were to report outcome measures with respect to learning curves, assessment methods or tools and training programmes including simulators. Two reviewers independently examined all retrieved studies. When disagreement existed over studies to be included or excluded, these were discussed until consensus was reached. Reviews, systematic reviews, meta-analyses and abstracts were excluded, as well as studies on tools to improve completion of colonoscopy. However, reference lists of potential relevant systematic reviews and meta-analyses were checked for any missed papers.

    Data extraction and analysis

    For each study, the methods, way of assessment and endpoints were recorded according to a predefined protocol. Two reviewers extracted all data. The quality of the studies was appraised and the reviewers assigned a level of evidence to each study using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria.2 The quality of the studies was distributed in four categories: very low (⊕∘∘∘); low (⊕⊕∘∘); moderate (⊕⊕⊕∘); and high (⊕⊕⊕⊕). Based on the presented evidence, we aimed to provide recommendations regarding the different subjects. The validation method and type of each simulator study was designated according to the consensus guidelines for the validation of VR simulators as described by Carter et al.3 The validation of simulators is in most cases performed by demonstrating different types of validity. Validity in itself is defined as the extent to which an assessment tool, in this case a simulator, measures what it is supposed to measure. One of the simplest forms of validity is face validity. This is demonstrated by questioning a defined group of subjects, to judge the simulator on realism between the simulator and the real activity. Usually, a group of experts is questioned. This is why the term expert validity is also used. Construct validity describes the extent to which the simulator can distinguish between different levels of expertise. The most used method of establishing construct validity is that the simulator can distinguish beginners from more experienced endoscopists and experts by the simulators performance parameters. The reliability of the simulator relates to the power of the simulator to provide consistent results. The most commonly used test is the test–retest reproducibility. It predicts to what extent a subject can ‘beat the test’ by repeated assessment. The most powerful evidence of validity is concurrent validity. This refers to the level of which performance on the simulator correlate to the real activity, in this case patient-based endoscopy.

    Since we aimed to provide a complete overview of the available literature on training and assessment in GI endoscopy, the included studies were fairly heterogeneous in clinical outcome. Therefore, it was judged that the statistical pooling of the data was not suitable.

    Results

    Inclusion

    Figure 1 shows the flowchart of the selection process of the included studies. Ninety-four studies investigating simulators, assessment methods, learning curves and training programmes for GI endoscopy were included in this review. In order to provide a systematic overview of these studies, they were divided into different categories (simulator training, learning curves, numbers needed to gain competence, assessment of performance and evaluation of [patient-based] training models). Table 1 provides an overview of the included studies per category. In a more detailed discussion, we will focus on studies of moderate to high quality.

    View this table:
    Table 1

    Overview of included studies per subject

    Figure 1
    Figure 1

    Flow diagram of the studies included.

    Simulator validation studies

    Twenty-seven simulator validation studies were retrieved.4–29 The validation studies comprised the evaluation of the devices alone. We included 11 studies on colonoscopy, two on flexible sigmoidoscopy, three studies on basic flexible endoscopy in general, one study on EGD, six studies on ERCP, two on EUS and one study on dexterity exercises in forward viewing endoscopy. All studies are shown in online supplementary table S2. Besides the ERCP and EUS studies, all five other categories of studies focused on conventional, forward viewing flexible endoscopy with a large overlap in outcome parameters. Procedures like ERCP and EUS show profound differences compared with basic forward viewing flexible endoscopy, because of combination with radiological or ultrasonographical imaging and also of a complete different perception by the endoscopist in side viewing endoscopy. We have therefore analysed them separately from the larger group that we refer to as forward viewing flexible endoscopy procedures. Eight validation studies on flexible endoscopy tasks were performed using the Simbionix GI Mentor VR computer simulator.8–10 ,12 ,14 ,15 ,22 ,25 Two studies reported on face validity. The largest study included 35 experts and demonstrated good face validity for colonoscopy.15 A smaller study reported the low level of realism as judged by six experts on all modules of the simulator.14 All studies reported consistent results and good construct for performance metrics on procedure times of the GI Mentor. These procedural times varied from time to caecal intubation, time spent with clear view and time spent with endoscope loops. Although these types of parameters, measuring a time aspect, are usually considered surrogate markers for competence, it seems to be the most consistent and therefore the most reliable parameter to distinguish between competence levels. There is a fairly large clinical heterogeneity on other outcome parameters. Five studies reported on the AccuTouch Immersion Medical computer simulator.4 ,19–21 ,27 Two studies reported on face validity with conflicting results. Again, as for the GI Mentor, realism was judged as valid by experts for the colonoscopy module but not for the complete set of modules on the simulator as a whole. The AccuTouch simulator seemed to have the same construct validity profile as the GI Mentor. That is, construct validity was consistently reported as good for performance measures related to procedural times in all published studies. Three validation studies reported on the Olympus Endo TS-1 VR computer simulator for colonoscopy.13 ,16 ,28 Face validity was rated as good by two studies and all three demonstrated good construct validity on all studied procedures. One study reported good construct the validity of the Kyoto Kagaku Colonoscope Training Model. Face validity was not studied.24 The last study demonstrated good face, construct and concurrent validity in a bovine explant colon model.26 Performance on this model correlated well with the level of expertise of the subjects and their performance in caecal intubation in patient-based endoscopy.

    Six validation studies were performed on ERCP. Two studies were feasibility studies and no formal validation was done. These two were both in mechanical models.11 ,23 Only one validation study was performed using a VR computer-based simulator. This study demonstrated both face and construct validity for the ERCP modules in the Simbionix GI Mentor II simulator.7 A similar study was done for the X-Vision ERCP Training System, a mechanical simulator, showing both face and construct validity.29 Two studies on the same mechanical ERCP training simulator were performed by the same research group.17 ,18 The ERCP Mechanical Simulator demonstrated a good construct validity and excellent face validity. In a direct comparison to an ex vivo porcine stomach model, the mechanical simulator was rated more realistic and useful. Another study compared live porcine models versus the Erlangen Endo-Trainer versus the Simbionix GI Mentor VR simulator for ERCP.30 The Erlangen model scored highest on realism and educational value. The GI Mentor scored lowest. However, it was felt that the GI Mentor was more easily incorporated in a training programme. Although the validation studies for ERCP simulation comprised a fairly heterogeneous group of simulators, the strongest evidence was provided for the mechanical simulators. For EUS, only two studies by the same author reported on feasibility to perform EUS and fine needle aspiration (FNA) in a porcine model.5 ,6 No attempt at validation has been published to date.

    Simulator training and learning curve studies

    Twenty-three studies reported on simulator training and learning curves.31–53 All but one of these studies focused on the diagnostic aspects of forward viewing flexible endoscopy, for example, intubation skills. Twenty studies reported on forward viewing flexible endoscopy (three EGD, three sigmoidoscopy and 14 colonoscopy), one study reported on EUS, one on ERCP and one on training haemostasis in upper GI bleeds. The studies are shown in online supplementary table S3. Eleven studies were performed using the AccuTouch Immersion Medical VR computer simulator for training.31 ,33 ,34 ,40 ,44 ,45 ,47 ,49–52 All studies on flexible sigmoidoscopy and colonoscopy had a randomised design and compared simulator-based training groups versus controls. Acquired competence was evaluated using the same simulator and in six studies also during patient-based assessment. The most consistent outcome parameters demonstrating improved performance were on procedural times, caecal intubation rates (CIRs), and times in red-out, meaning that luminal view was lost. Patient comfort scores were measured in two studies.40 ,50 One study favoured simulator training versus no simulator training prior to starting patient endoscopies, the second study showed no difference between groups.

    Six studies were carried out using the Simbionix GI Mentor VR simulator for training and learning curves.32 ,34 ,36–38 ,41 Four studies were on colonoscopy tasks, two on EGD. All studies demonstrated that simulator training improved the performance of novices. There were no learning effects for experienced endoscopists. Due to the methodological heterogeneity of these studies, improved performance could not be expressed in terms of exact numbers. Performance was assessed by means of the simulator construct in three studies. Two studies used patient-based assessment for the evaluation of the simulator-based learning effect. The competence parameters that consistently improved significantly were: (I) procedure time, (II) CIR: a direct comparison of simulator-training versus controls showed a 4.5-fold increased CIR in the simulator-training group in the early learning curve,34 (III) time with clear view, (IV) time of endoscope looping, and (V) objective performance scores, as judged by expert supervisors during patient-based endoscopy assessment. Improved performance in the simulator-trained groups versus controls was observed in up to 60 patient-based assessed EGDs and 80 procedures in colonoscopy training. Only one study used the Olympus Endo TS-1 colonoscopy simulator for training.42 This multicentre, randomised study compared simulator-based training versus patient-based training. Blinded experts assessed performance during patient-based endoscopy. Both groups (simulator trained and patient trained) showed equal performance during patient-based colonoscopy. One multicentre, randomised study was performed using all kinds of simulators.53 The study showed that patient-based training with complementary simulator training was superior to patient or simulator-based training alone. One study was done on ERCP.46 This study had a multicentre, randomised design. It demonstrated significantly higher cannulation success rates in less time in the study group after training on the ERCP Mechanical Simulator. One study was performed evaluating the CompactEASIE simulator, a mechanical simulator with an ex vivo porcine stomach.48 Significant improvement in skills in endoscopic haemostatic therapy was demonstrated with a sufficient level of evidence. No previous formal validation of the model was carried out. Only one study was performed on the subject of learning diagnostic and therapeutic EUS.35 Only a description of improved performance on live porcine models before and after a hands-on training course was provided. No formal statistical calculation was carried out. The model had not been previously validated.

    Simulator competence assessment studies

    Four studies reported on the use of simulators for the assessment of competence.54–57 Two studies focussed on colonoscopy, one on sigmoidoscopy and one on both EGD and sigmoidoscopy; all evaluated the diagnostic aspects of endoscopy. The studies are summarised in online supplementary table S4. In two studies of moderate quality, performance parameters derived from the simulators did not correlate to performance scores given by blinded experts.55 ,57 It seems that current simulators lack the discriminative power to assess performance and determine competence levels in patient-based endoscopy.

    Learning curves

    Fifteen studies reported on learning curves for colonoscopy (n=8), ERCP (n=5) and EUS (n=2).58–72 These are shown in online supplementary table S5 (A,B,C, respectively).

    For colonoscopy, four studies reached a sufficient quality level.59 ,61 ,62 ,65 These studies had a prospective design and evaluated 8–41 trainees with procedure numbers varying from 2887 to 4351. All of these studies evaluated patient-based performance, focussing on the intubation of the caecum. However, outcome measures and use of competence standards were fairly heterogeneous. The studies reported on CIR or completion rate, time to caecum or a combination of those outcomes. One group described the learning curve by means of scoring different aspects of the procedure on a newly developed assessment tool (Mayo Colonoscopy Skills Assessment Tool, MCSAT), but also described learning curves for outcomes such as CIR.65 The Rotterdam Assessment Form for Colonoscopy (RAF-C) was used by another group as an easy-to-use assessment form to document the colonoscopy learning curve but also provide a platform for repetitive assessment and feedback to improve performance.61 The number of colonoscopies that trainees needed to perform in order to achieve a CIR of >85–90% varied from 150 to 280 procedures. The two studies of the highest quality reported 275 and 280 procedures needed to achieve a 90% CIR.61 ,65 These consistent numbers probably provide the best evidence currently available regarding threshold numbers for colonoscopy training in intubating the caecum.

    From the five studies focusing on ERCP, only two had a reasonable quality level.60 ,71 These described a prospective evaluation of, respectively, 17 and 20 trainees, with the following outcome measures: subjective score regarding performance (overall and per part of the procedure) on a six-point scale where a score of 1, 2, or 3 was considered competent, and the success of selective cannulation of the common bile duct (CBD) or pancreatic duct (PD). One study concluded that an overall sufficient score was reached after 137 (probability of success=0.8) or 185 ERCPs (probability of success=0.9).60 A different group reported that an 85% selective cannulation rate was reached after 70 procedures for the PD and after >100 ERCPs for the CBD.71

    The two studies on EUS described the performance per anatomic station of the procedure.70 ,72 There was a large variability in achieving overall competence, with acceptable performance after a range of 255–>400 EUS procedures.70 One study did not report on overall competence, but stated that 78 procedures were necessary for competence in duodenal examination.72

    Threshold numbers needed to gain competence

    Nine studies reported numbers needed to gain competence in different procedures in GI endoscopy.72–81 These studies are shown in online supplementary table S5 as well. Two studies handled both EGD and colonoscopy,73 ,80 whereas most of the studies pertained to colonoscopy alone.74 ,75 ,77–79 There were two single studies on sigmoidoscopy and ERCP.76 ,81 The quality of the studies was moderate for most of them due to the designs and numbers of procedures evaluated. Only three groups performed studies (regarding EGD and colonoscopy) with a prospective design and a considerable amount of trainees evaluated.73 ,74 ,80 These will be discussed in further detail. These studies evaluated patient-based performance and focussed on intubation skills, for both EGD and colonoscopy.

    For EGD, competence was measured in two ways: intubation of the oesophagus and reaching a sufficient score on the Global Assessment of Gastrointestinal Endoscopic Skills (GAGES). One group demonstrated an 80% success rate of oesophageal intubation after 100 procedures, whereas another study concluded a plateau in the GAGES score after 50 procedures.73 ,80 Concerning colonoscopy, competence was measured through CIR and scores on the GAGES form as well. Two studies concluded that 100 colonoscopies was insufficient for reaching a >90% CIR,73 ,74 whereas the GAGES score displayed a plateau score at n=75 procedures.80 All studies confirmed that the performance of trainees increased with experience. The best evidence for a threshold number to reach a steady 90% CIR was already mentioned in the previous paragraph in two high quality studies with a threshold number of 275–280 colonoscopies during training.61 ,65

    Assessment and grading of performance

    Nine studies described the development and evaluation of assessment tools for colonoscopy (n=6), sigmoidoscopy (n=1), both (n=1) and both colonoscopy and EGD (n=1).82–90 These are shown in online supplementary table S6. All studies of moderate to high quality focused on colonoscopy, flexible sigmoidoscopy or both, had a prospective design and reported on 18–162 participants.82 ,86–89 The British Direct Observation of Procedural Skills (DOPS) appears effective for the evaluation of competence for already registered endoscopists.82 This form addressed both intubational and withdrawal and therapeutic skills. The MCSAT was more effective in discriminating different experience levels, and therefore applicable in training settings.87 The RAF-C was already mentioned in the paragraph on learning curves. This assessment form documents objective parameters such as caecal intubation, procedural times and polyp detection and combines this evaluation of performance, self-reflection and an improvement plan.61 Two studies reported on some sort of video assessment of endoscopic skills.88 ,89 The tri-split video recording assessment tool proved to be valid, but reliability was lacking.88 The other study on video assessment described the development of an assessment tool for sigmoidoscopy withdrawals in a series of five experiments.89 They concluded that the sequential assessment of five withdrawals led to the highest agreement. However, all procedures included in this video study were performed by experienced endoscopists. Some assessment tools were applicable in training situations, while others were only evaluated in a setting with experienced endoscopists. This difference makes it therefore difficult to compare the assessment tools.

    Training models

    Finally, seven studies reported on different kinds of training models for colonoscopy (n=4), sigmoidoscopy (n=1) and EGD (n=2).91–97 Online supplementary table S7 provides an overview of these studies. Two groups described the evaluation of the accelerated colonoscopy training course as it is carried out in the UK.96 ,97 This training model comprised theory, simulator sessions, hands-on training and live case assessment. Both studies concluded that performance in knowledge, colonoscopy performance and DOPS scores improved significantly after the training week. Thomas-Gibson et al added an evaluation at a median follow-up of 9 months. There were, however, no differences between post-training assessment and follow-up. A different training model was the ‘gastroenterological education—training endoscopy’ model.91 The model focussed on knowledge and simulator training; there were no patient-based endoscopies involved. This training model showed improvement in post-test results and simulator performance. A German group tried to identify predictors for performance in a 1-week training course by psychological and psychomotor tests.94 The training week resulted in improved performance, but only one specific (double labyrinth) test was identified as a predictor for improvement in performance.

    One randomised control trial (RCT) evaluated the impact of systematic feedback on patient-based colonoscopy performance.92 Although only four trainees were evaluated, there was a significant improvement in CIR performance in the feedback group, while the control group showed no improvement.

    Discussion

    Forward viewing flexible endoscopy procedures

    GI endoscopic procedures are fairly complex. The sole use of the classic master–apprentice model for teaching endoscopy is nowadays less accepted. The use of simulators in the early training phase is gaining acceptance and several VR endoscopy simulators have been validated (see online supplementary table S2). The GI Mentor, AccuTouch and Endo TS-1 were shown to have good validity.4 ,8–10 ,12–16 ,19–22 ,25 ,27 ,28 These can thus be considered as realistic devices that have discriminative abilities for distinguishing dexterity and competence levels in flexible endoscopy. Since these simulators proved to have good validity, we recommend to use one of these devices in early training. ⊕⊕⊕∘

    Following validation, the impact of simulator training on learning curves needs to be assessed. A VR simulator with good validity, but not improving performance after repeated exercise, especially in patient-based endoscopy, is not suitable for implementing in a training programme. Three studies provided high quality evidence for the positive effect of simulator training in novices in flexible endoscopy, measured in terms of both VR as well as live endoscopy.37 ,42 ,53 Two of these were well-designed randomised multicentre trials comparing the combination of simulator- and bedside-training versus bedside training alone for the colonoscopy training of novices. These studies demonstrated that simulator training is effective.37 ,42 The first RCT demonstrated significantly higher objective competency rates during the first 80 patient-based colonoscopies after 10 h of unsupervised simulator training. There was, however, no difference in the number of procedures to reach a 90% competency level. The second study demonstrated similar performance of novices during patient-based endoscopy, as judged by blinded experts, after either 16 h of supervised simulator training or 16 h patient-based training. There was no follow-up of participants to procedural competency in this study. Several studies on simulator learning curves for EGD, sigmoidoscopy and colonoscopy had a moderate level of quality.31 ,32 ,34 ,36 ,38–41 ,43–45 ,47 ,49–52 Nonetheless, some studies only measured performance during VR endoscopy, which is obviously inferior to measuring performance during patient-based endoscopy, since the ultimate goal is improvement of patient-based performance.33 ,35 ,36 ,38 ,44 ,45 ,47 ,51 Based on this evidence, one can conclude that simulator training is complementary to patient-based learning and is useful in the early training phase in speeding up the early learning curve and reducing patient burden. To reach procedural competency in patient-based endoscopy, the same numbers of patient-based procedures seem to be necessary. We do nonetheless recommend the use of simulators in the early training phase. ⊕⊕⊕∘

    The four studies that reported on the use of a simulator as a competence assessment tool showed diverging results.54–57 Therefore, we would suggest that at this point, the use of simulators as an assessment tool is not sensible. ⊕∘∘∘

    Elaborating further on the learning curve, the next step is (continuous) assessment of a trainee's performance during patient-based training. The currently available recommendations and guidelines focus mainly on minimum numbers as a threshold for competence.73–80 However, outcomes and proposed minimum numbers for flexible endoscopic procedures vary widely. Nowadays there is a tendency to define more objective criteria for competence. Two large prospective single-centre studies of high quality provided evidence for the use of an assessment form as a measure of competence, respectively, the MCSAT and the RAF-C.61 ,65 The learning curves obtained in these studies were similar. Both studies focus on the description of the gradual process of acquisition of competence rather than setting a prefixed threshold number. The strength of these curves is that improved performance over time is documented rather than the assessment of an ‘incidental lucky procedure’ or a total number of procedures performed without any qualitative content. We recommend to implement the use of one of these forms in assessment of trainees’ performance and learning curves. On top of that, we recommend to use the DOPS for assessment of ‘end-stage’ competence. ⊕⊕⊕⊕82

    Overall, some high-quality studies have been performed for each individual step in training, providing valuable information on the effect of simulator training, learning curves and assessment methods. The most and best evidence for all these stages regarding basic flexible endoscopy is available for colonoscopy. However, one can imagine that some results can be extrapolated to other basic GI endoscopy procedures as well, since the techniques are comparable.

    Endoscopic retrograde cholangiopancreatography

    One of the most challenging procedures with high complication rates in GI endoscopy is ERCP. It takes a great deal of training and a large number of procedures to reach competence. However, little is known about the learning curve for trainees in ERCP. A number of questions remain unanswered when it comes to the shape of the learning curve, the number of procedures needed to gain competence, and the definition of competence itself. The six studies on learning curves in ERCP varied widely in design, the number of trainees and procedures included, as well as outcome, resulting in a large heterogeneity among them. Successful cannulation in >85% of the patients was seen after a number of 100–185 ERCPs. Due to the large clinical heterogeneity between the studies, we cannot make any recommendations regarding ERCP learning curves. ⊕∘∘∘ There would be a great benefit if part of the learning curve for endoscopists could be accomplished by training on simulator models. In reality the number of available simulators for training in ERCP is limited. Of in total six devices, the GI Mentor is the only validated VR simulator for ERCP.7 ,30 The face and construct validity was demonstrated in these two studies and although it received lower scores than the ex vivo or live porcine model in a head-to-head comparison, it was considered the easiest of all ERCP simulator models to incorporate in a training curriculum.7 ,30 The live porcine model was validated only once in comparison to the ex vivo model and the GI Mentor in the same study.30 The ex vivo simulators and purely mechanical simulators are highly comparable among each other and achieve similar results. All of these models require a real endoscope to be introduced to reach a papilla which is either a synthetic or an ex vivo papilla located in a mechanical tube representing the duodenum or an ex vivo duodenum. Overall these ERCP simulator models receive the highest scores on realism. The use of one of the validated ERCP simulators before patient-based training is recommendable, since these have fairly good validity.7 ,30 ⊕⊕⊕∘ We cannot provide a recommendation regarding the use of simulators in order to speed up the learning curve, since there was only one study performed.46 There were no studies found on validated competence assessment tools to objectify performance in ERCP. The most common performance parameter is cannulation success rate. This only partly reflects the extent, therapeutic intent and diversity of a therapeutic procedure like an ERCP.

    Endosonography

    EUS is widely practiced with an increasing number of therapeutic possibilities since the first reports of transgastric drainage of pseudocysts by Grimm et al.98 This makes EUS more complex. Especially the therapeutic procedures have a marked overlap with ERCP and demand a great deal of experience. There are only a few reports on simulator-based training in EUS.5 ,6 Training diagnostic and interventional EUS seems logical and feasible in a live porcine model but no formal attempt at validation has been made. No grade of recommendation can be given based on these studies. A learning effect by repeated exercise and improvement of performance during EUS procedures in the live porcine model itself was documented in one study.35 There is a lack of scientific evidence of transfer of competence to a patient-based setting. There is an even greater scarcity of evidence on learning curves and numbers to reach competence in EUS. Two studies were performed that both included five trainees. The first study included only radial EUS.72 They reported no additional effect of observing large numbers of procedures; the largest benefit was achieved during hands-on training. There is only one study of moderate quality performed.70 The learning curves differed considerably among the five trainees. These studies demonstrated the substantial need for much more training than the 150 procedures recommended by the American Society for Gastrointestinal Endoscopy in order to reach proficiency. ⊕⊕∘∘

    Limitations

    The clinical heterogeneity of the studies regarding forward viewing endoscopy limits the conclusions that can be drawn. This systematic review covers a broad range of studies regarding training and assessment in GI endoscopy. This broad approach automatically results in a large variety of methodology, devices used and endpoints measured. This hampers head-to-head comparison of individual studies. Another limitation concerns the fact that all studies focused on specific aspects of the endoscopic procedure, instead of on overall performance, which is both overall competence assessment from novice to experienced, certified endoscopy, as well as expert levels for specific procedures.

    The evidence in the literature on learning curves and competence measures for ERCP is highly heterogeneous. This makes it impossible to provide a level of recommendation. Also, cannulation success rates do relate to improved performance but do not entirely reflect the diversity of a complex procedure like ERCP. No solid data are currently available on other aspects of therapeutic interventions related to learning curves and benchmarks in ERCP. As of yet, no validated competence assessment tools have been developed for ERCP. This should be a prerequisite before attempts to define learning goals and benchmarks are made.

    Future research

    Future research, based on the presented evidence in this review, should therefore include a complete training programme. We propose a prepatient curriculum using simulator training. The transfer of simulation skills to patient-based procedures needs to be further explored. Simulation training needs to be followed by the continuous assessment of patient-based endoscopies to provide individual and group learning curves and after a period of time, (repeated) overall assessments of performance by an expert. Therefore, the development of validated assessment tools is necessary and the effect of expert assessments on daily practice needs to be measured.

    With respect to ERCP, there is a rationale to start training using simulators. There is however no evidence yet as to what extent or performance level simulator-based training has to be carried out. The next step would be to investigate the transfer of skills to patient-based training. These research objectives seem to be clear goals for future research. There is a need for the development of validated objective assessment tools in ERCP to document progress in training and finally proficiency. Benchmarks can be set using the same assessment tools in ERCP performed by experts.

    The evidence on training and competence assessment in EUS is extremely scarce. Although training in a live porcine model seems logical, in the current era of evidence-based medicine, validation studies should be carried out to establish the degree of realism and training potential. Current threshold numbers for training appear to be inadequate, but the available data are sparse. We seem to be far away from establishing benchmarks for competence in EUS and validated assessment tools are lacking.

    General conclusions and recommendations

    Based on the presented evidence, we propose the implementation of simulator training in GI endoscopic training curricula. Regarding basic flexible endoscopy (EGD, sigmoidoscopy and colonoscopy), simulator-based training has proven its value and it is justifiable to start a prepatient training course using a validated simulator. This will result in speeding up the early learning curve and reducing patient burden. However, to reach procedural competency in patient-based endoscopy, the same numbers of patient-based procedures seem to be necessary. The extent to which simulator-based training should be carried out is still a matter of debate. A structured and supervised 2 days or 16 h training course seems to be of added value. Furthermore, objective outcome parameters should be measured continuously in patient-based training. This provides insight in the learning curve in a qualitative fashion and is preferable to threshold numbers. The MCSAT, RAF-C and DOPS assessment forms seem to be the best forms to document progress or proficiency levels. Regarding ERCP training, we would recommend a prepatient training curriculum using a validated simulator as well. Evidence for evaluation of learning curves and continuous assessment in ERCP is scarce. This makes competency-based training difficult. The available data support prolonged training, at least to a larger extent than current upheld threshold numbers in most countries. The results so far may hopefully stimulate further research. The evidence on endosonography training and competence is yet the least investigated. A prepatient training curriculum is logical and attractive. However, the evidence is too scarce to give recommendations at this moment.

    References

    1. ↵
      1. Luursema JM,
      2. Buzink SN,
      3. Verwey WB, et al
      . Visuo-spatial ability in colonoscopy simulator training. Adv Health Sci Educ 2010;15:685–94. doi:10.1007/s10459-010-9230-y
      OpenUrlCrossRef
    2. ↵
      1. Guyatt GH,
      2. Oxman AD,
      3. Vist GE, et al
      . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924–6. doi:10.1136/bmj.39489.470347.AD
      OpenUrlFREE Full Text
    3. ↵
      1. Carter FJ,
      2. Schijven MP,
      3. Aggarwal R, et al
      . Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 2005;19:1523–32. doi:10.1007/s00464-005-0384-2
      OpenUrlCrossRefPubMed
    4. ↵
      1. Datta V,
      2. Mandalia M,
      3. Mackay S, et al
      . The PreOp flexible sigmoidoscopy trainer: validation and early evaluation of a virtual reality based system. Surg Endosc 2002;16:1459–63. doi:10.1007/s00464-002-9014-4
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Bhutani MS,
      2. Aveyard M,
      3. Stills HF Jr.
      . Improved model for teaching interventional EUS. Gastrointest Endosc 2000;52:400–3. doi:10.1067/mge.2000.108408
      OpenUrlCrossRefPubMed
    6. ↵
      1. Bhutani MS,
      2. Hoffman BJ,
      3. Hawes RH
      . A swine model for teaching endoscopic ultrasound (EUS) imaging and intervention under EUS guidance. Endoscopy 1998;30:605–9. doi:10.1055/s-2007-1001364
      OpenUrlCrossRefPubMed
    7. ↵
      1. Bittner Iv JG,
      2. Mellinger JD,
      3. Imam T Jr., et al
      . Face and construct validity of a computer-based virtual reality simulator for ERCP. Gastrointest Endosc 2010;71:357–64. doi:10.1016/j.gie.2009.08.033
      OpenUrlCrossRefPubMed
    8. ↵
      1. Fayez R,
      2. Feldman LS,
      3. Kaneva P, et al
      . Testing the construct validity of the Simbionix GI Mentor II virtual reality colonoscopy simulator metrics: module matters. Surg Endosc 2010;24:1060–5. doi:10.1007/s00464-009-0726-6
      OpenUrlCrossRefPubMed
    9. ↵
      1. Felsher JJ,
      2. Olesevich M,
      3. Farres H, et al
      . Validation of a flexible endoscopy simulator. Am J Surg 2005;189:497–500. doi:10.1016/j.amjsurg.2005.01.008
      OpenUrlCrossRefPubMed
    10. ↵
      1. Ferlitsch A,
      2. Glauninger P,
      3. Gupper A, et al
      . Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy 2002;34:698–702. doi:10.1055/s-2002-33456
      OpenUrlCrossRefPubMed
    11. ↵
      1. Frimberger E,
      2. von Delius S,
      3. Rosch T, et al
      . A novel and practicable ERCP training system with simulated fluoroscopy. Endoscopy 2008;40:517–20. doi:10.1055/s-2007-995456
      OpenUrlCrossRefPubMed
    12. ↵
      1. Grantcharov TP,
      2. Carstensen L,
      3. Schulze S
      . Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS 2005;9:130–3.
      OpenUrlPubMed
    13. ↵
      1. Haycock AV,
      2. Bassett P,
      3. Bladen J, et al
      . Validation of the second-generation Olympus colonoscopy simulator for skills assessment. Endoscopy 2009;41:952–8. doi:10.1055/s-0029-1215193
      OpenUrlCrossRefPubMed
    14. ↵
      1. Kim S,
      2. Spencer G,
      3. Makar GA, et al
      . Lack of a discriminatory function for endoscopy skills on a computer-based simulator. Surg Endosc Interv Tech 2010;24:3008–15. doi:10.1007/s00464-010-1077-z
      OpenUrlCrossRef
    15. ↵
      1. Koch AD,
      2. Buzink SN,
      3. Heemskerk J, et al
      . Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy. Surg Endosc 2008;22:158–62. doi:10.1007/s00464-007-9394-6
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Koch AD,
      2. Haringsma J,
      3. Schoon EJ, et al
      . A second-generation virtual reality simulator for colonoscopy: validation and initial experience. Endoscopy 2008;40:735–8. doi:10.1055/s-2008-1077508
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Leung JW,
      2. Lee JG,
      3. Rojany M, et al
      . Development of a novel ERCP mechanical simulator. Gastrointest Endosc 2007;65:1056–62. doi:10.1016/j.gie.2006.11.018
      OpenUrlCrossRefPubMed
    18. ↵
      1. Leung JW,
      2. Wang D,
      3. Hu B, et al
      . A head-to-head hands-on comparison of ERCP mechanical simulator (EMS) and ex-vivo porcine stomach model (PSM). J Intervent Gastroenterol 2011;1:108–13.
      OpenUrl
    19. ↵
      1. MacDonald J,
      2. Ketchum J,
      3. Williams RG, et al
      . A lay person versus a trained endoscopist: can the preop endoscopy simulator detect a difference? Surg Endosc 2003;17:896–8. doi:10.1007/s00464-002-8559-6
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Mahmood T,
      2. Darzi A
      . A study to validate the colonoscopy simulator: it is usefully discriminatory for more than one measurable outcome. Surg Endosc 2003;17:1583–9. doi:10.1007/s00464-002-9222-y
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. McConnell RA,
      2. Kim S,
      3. Ahmad NA, et al
      . Poor discriminatory function for endoscopic skills on a computer-based simulator. Gastrointest Endosc 2012;76:993–1002. doi:10.1016/j.gie.2012.07.024
      OpenUrlCrossRefPubMed
    22. ↵
      1. Moorthy K,
      2. Munz Y,
      3. Jiwanji M, et al
      . Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surg Endosc 2004;18:328–33. doi:10.1007/s00464-003-8513-2
      OpenUrlCrossRefPubMed
    23. ↵
      1. Neumann M,
      2. Mayer G,
      3. Ell C, et al
      . The Erlangen endo-trainer: life-like simulation for diagnostic and interventional endoscopic retrograde cholangiography. Endoscopy 2000;32:906–10. doi:10.1055/s-2000-8090
      OpenUrlCrossRefPubMed
    24. ↵
      1. Plooy AM,
      2. Hill A,
      3. Horswill MS, et al
      . Construct validation of a physical model colonoscopy simulator. Gastrointest Endosc 2012;76:144–50. doi:10.1016/j.gie.2012.03.246
      OpenUrlCrossRefPubMed
    25. ↵
      1. Ritter EM,
      2. McClusky Iii DA,
      3. Lederman AB, et al
      . Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg 2003;7:871–8. doi:10.1007/s11605-003-0032-x
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Sedlack RE,
      2. Baron TH,
      3. Downing SM, et al
      . Validation of a colonoscopy simulation model for skills assessment. Am J Gastroenterol 2007;102:64–74. doi:10.1111/j.1572-0241.2006.00942.x
      OpenUrlCrossRefPubMed
    27. ↵
      1. Sedlack RE,
      2. Kolars JC
      . Validation of a computer-based colonoscopy simulator. Gastrointest Endosc 2003;57:214–18. doi:10.1067/mge.2003.81
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Sugden C,
      2. Aggarwal R,
      3. Banerjee A, et al
      . The development of a virtual reality training curriculum for colonoscopy. Ann Surg 2012;256:188–92. doi:10.1097/SLA.0b013e31825b6e9c
      OpenUrlCrossRefPubMed
    29. ↵
      1. von Delius S,
      2. Thies P,
      3. Meining A, et al
      . Validation of the X-vision ERCP training system and technical challenges during early training of sphincterotomy. Clin Gastroenterol Hepatol 2009;7:389–96. doi:10.1016/j.cgh.2008.11.004
      OpenUrlCrossRefPubMed
    30. ↵
      1. Sedlack R,
      2. Petersen B,
      3. Binmoeller K, et al
      . A direct comparison of ERCP teaching models. Gastrointest Endosc 2003;57:886–90. doi:10.1016/S0016-5107(03)70025-X
      OpenUrlCrossRefPubMed
    31. ↵
      1. Tuggy ML
      . Virtual reality flexible sigmoidoscopy simulator training: impact on resident performance. J Am Board Fam Pract 1998;11:426–33. doi:10.3122/jabfm.11.6.426
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Van Sickle KR,
      2. Buck L,
      3. Willis R, et al
      . A multicenter, simulation-based skills training collaborative using shared GI mentor II systems: results from the Texas association of surgical skills laboratories (TASSL) flexible endoscopy curriculum. Surg Endosc Interv Tech 2011;25:2980–6. doi:10.1007/s00464-011-1656-7
      OpenUrlCrossRef
    33. ↵
      1. Ahad S,
      2. Boehler M,
      3. Schwind CJ, et al
      . The effect of model fidelity on colonoscopic skills acquisition. A randomized controlled study. J Surg Educ 2013;70:522–7. doi:10.1016/j.jsurg.2013.02.010
      OpenUrlCrossRefPubMed
    34. ↵
      1. Ahlberg G,
      2. Hultcrantz R,
      3. Jaramillo E, et al
      . Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy 2005;37:1198–204. doi:10.1055/s-2005-921049
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Barthet M,
      2. Gasmil M,
      3. Boustiere C, et al
      . EUS training in a live pig model: does it improve echo endoscope hands-on and trainee competence? Endoscopy 2007;39:535–9. doi:10.1055/s-2007-966336
      OpenUrlCrossRefPubMed
    36. ↵
      1. Buzink SN,
      2. Koch AD,
      3. Heemskerk J, et al
      . Acquiring basic endoscopy skills by training on the GI Mentor II. Surg Endosc Interv Tech 2007;21:1996–2003. doi:10.1007/s00464-007-9297-6
      OpenUrlCrossRef
    37. ↵
      1. Cohen J,
      2. Cohen SA,
      3. Vora KC, et al
      . Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 2006;64:361–8. doi:10.1016/j.gie.2005.11.062
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Eversbusch A,
      2. Grantcharov TP
      . Learning curves and impact of psychomotor training on performance in simulated colonoscopy: a randomized trial using a virtual reality endoscopy trainer. Surg Endosc 2004;18:1514–18. doi:10.1007/s00464-003-9264-9
      OpenUrlCrossRefPubMed
    39. ↵
      1. Ferlitsch A,
      2. Schoefl R,
      3. Puespoek A, et al
      . Effect of virtual endoscopy simulator training on performance of upper gastrointestinal endoscopy in patients: a randomized controlled trial. Endoscopy 2010;42:1049–56. doi:10.1055/s-0030-1255818
      OpenUrlCrossRefPubMed
    40. ↵
      1. Gerson LB,
      2. Van Dam J
      . A prospective randomized trial comparing a virtual reality simulator to bedside teaching for training in sigmoidoscopy. Endoscopy 2003;35:569–75. doi:10.1055/s-2003-40243
      OpenUrlCrossRefPubMed
    41. ↵
      1. Giulio ED,
      2. Fregonese D,
      3. Casetti T, et al
      . Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: a randomized controlled trial. Gastrointest Endosc 2004;60:196–200. doi:10.1016/S0016-5107(04)01566-4
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Haycock A,
      2. Koch AD,
      3. Familiari P, et al
      . Training and transfer of colonoscopy skills: a multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest Endosc 2010;71:298–307. doi:10.1016/j.gie.2009.07.017
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Kaltenbach T,
      2. Leung C,
      3. Wu K, et al
      . Use of the colonoscope training model with the colonoscope 3d imaging probe improved trainee colonoscopy performance: a pilot study. Dig Dis Sci 2011;56:1496–502. doi:10.1007/s10620-011-1614-1
      OpenUrlCrossRefPubMed
    44. ↵
      1. Kruglikova I,
      2. Grantcharov TP,
      3. Drewes AM, et al
      . The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity virtual-reality simulation: a randomised controlled trial. Gut 2010;59:181–5. doi:10.1136/gut.2009.191825
      OpenUrlAbstract/FREE Full Text
    45. ↵
      1. Kruglikova I,
      2. Grantcharov TP,
      3. Drewes AM, et al
      . Assessment of early learning curves among nurses and physicians using a high-fidelity virtual-reality colonoscopy simulator. Surg Endosc Interv Tech 2010;24:366–70. doi:10.1007/s00464-009-0555-7
      OpenUrlCrossRef
    46. ↵
      1. Lim BS,
      2. Leung JW,
      3. Lee J, et al
      . Effect of ERCP mechanical simulator (EMS) practice on trainees ERCP performance in the early learning period: US multicenter randomized controlled trial. Am J Gastroenterol 2011;106:300–6. doi:10.1038/ajg.2010.411
      OpenUrlCrossRefPubMed
    47. ↵
      1. Mahmood T,
      2. Darzi A
      . The learning curve for a colonoscopy simulator in the absence of any feedback: no feedback, no learning. Surg Endosc 2004;18:1224–30. doi:10.1007/s00464-003-9143-4
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Maiss J,
      2. Millermann L,
      3. Heinemann K, et al
      . The compactEASIE (registered trademark) is a feasible training model for endoscopic novices: a prospective randomised trial. Dig Liver Dis 2007;39:70–8. doi:10.1016/j.dld.2006.07.007
      OpenUrlCrossRefPubMed
    49. ↵
      1. Sedlack RE,
      2. Kolars JC
      . Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 2004;99:33–7. doi:10.1111/j.1572-0241.2004.04007.x
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      1. Sedlack RE,
      2. Kolars JC,
      3. Alexander JA
      . Computer simulation training enhances patient comfort during endoscopy. Clin Gastroenterol Hepatol 2004;2:348–52. doi:10.1016/S1542-3565(04)00067-9
      OpenUrlCrossRefPubMed
    51. ↵
      1. Snyder CW,
      2. Vandromme MJ,
      3. Tyra SL, et al
      . Retention of colonoscopy skills after virtual reality simulator training by independent and proctored methods. Am Surg 2010;76:743–6.
      OpenUrlPubMedWeb of Science
    52. ↵
      1. Park J,
      2. MacRae H,
      3. Musselman LJ, et al
      . Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 2007;194:205–11. doi:10.1016/j.amjsurg.2006.11.032
      OpenUrlCrossRefPubMedWeb of Science
    53. ↵
      1. Ende A,
      2. Zopf Y,
      3. Konturek P, et al
      . Strategies for training in diagnostic upper endoscopy: a prospective, randomized trial. Gastrointest Endosc 2012;75:254–60. doi:10.1016/j.gie.2011.07.063
      OpenUrlCrossRefPubMed
    54. ↵
      1. Elvevi A,
      2. Cantu P,
      3. Maconi G, et al
      . Evaluation of hands-on training in colonoscopy: is a computer-based simulator useful? Dig Liver Dis 2012;44:580–4. doi:10.1016/j.dld.2012.03.014
      OpenUrlCrossRefPubMed
    55. ↵
      1. Moorthy K,
      2. Munz Y,
      3. Orchard TR, et al
      . An innovative method for the assessment of skills in lower gastrointestinal endoscopy. Surg Endosc 2004;18:1613–19. doi:10.1007/s00464-004-9002-y
      OpenUrlCrossRefPubMed
    56. ↵
      1. Phitayakorn R,
      2. Marks JM,
      3. Reynolds HL, et al
      . Expert benchmark for the GI Mentor IITM. Surg Endosc Interv Tech 2009;23:611–14. doi:10.1007/s00464-008-0166-8
      OpenUrlCrossRef
    57. ↵
      1. Sarker SK,
      2. Albrani T,
      3. Zaman A, et al
      . Procedural performance in gastrointestinal endoscopy: live and simulated. World J Surg 2010;34:1764–70. doi:10.1007/s00268-010-0579-0
      OpenUrlCrossRefPubMed
    58. ↵
      1. Biau DJ,
      2. Williams SM,
      3. Schlup MM, et al
      . Quantitative and individualized assessment of the learning curve using LC-CUSUM Review. Br J Surg 2008;95:925–9. doi:10.1002/bjs.6056
      OpenUrlCrossRefPubMedWeb of Science
    59. ↵
      1. Chung JI,
      2. Kim N,
      3. Um MS, et al
      . Learning curves for colonoscopy: a prospective evaluation of gastroenterology fellows at a single center. Gut Liver 2010;4:31–5. doi:10.5009/gnl.2010.4.1.31
      OpenUrlCrossRefPubMedWeb of Science
    60. ↵
      1. Jowell PS,
      2. Baillie J,
      3. Branch MS, et al
      . Quantitative assessment of procedural competence: a prospective study of training in endoscopic retrograde cholangiopancreatography. Ann Intern Med 1996;125:983–9. doi:10.7326/0003-4819-125-12-199612150-00009
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      1. Koch AD,
      2. Haringsma J,
      3. Schoon EJ, et al
      . Competence measurement during colonoscopy training: the use of self-assessment of performance measures. Am J Gastroenterol 2012;107:971–5. doi:10.1038/ajg.2011.481
      OpenUrlCrossRefPubMed
    62. ↵
      1. Lee SH,
      2. Chung IK,
      3. Kim SJ, et al
      . An adequate level of training for technical competence in screening and diagnostic colonoscopy: a prospective multicenter evaluation of the learning curve. Gastrointest Endosc 2008;67:683–9. doi:10.1016/j.gie.2007.10.018
      OpenUrlCrossRefPubMedWeb of Science
    63. ↵
      1. Marshall JB
      . Technical proficiency of trainees performing colonoscopy: a learning curve. Gastrointest Endosc 1995;42:287–91. doi:10.1016/S0016-5107(95)70123-0
      OpenUrlCrossRefPubMedWeb of Science
    64. ↵
      1. Parry BR,
      2. Williams SM
      . Competency and the colonoscopist: a learning curve. Aust New Zealand J Surg 1991;61:419–22. doi:10.1111/j.1445-2197.1991.tb00254.x
      OpenUrlCrossRef
    65. ↵
      1. Sedlack RE
      . Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 2011;74:355–66+366.e351–366.e352. doi:10.1016/j.gie.2011.02.019
      OpenUrlCrossRefPubMed
    66. ↵
      1. Selvasekar CR,
      2. Holubar SD,
      3. Pendlimari R, et al
      . Assessment of screening colonoscopy competency in colon and rectal surgery fellows: a single institution experience. J Surg Res 2012;174:e17–23. doi:10.1016/j.jss.2011.09.048
      OpenUrlCrossRefPubMed
    67. ↵
      1. Tassios PS,
      2. Ladas SD,
      3. Grammenos I, et al
      . Acquisition of competence in colonoscopy: the learning curve of trainees. Endoscopy 1999;31:702–6. doi:10.1055/s-1999-146
      OpenUrlCrossRefPubMedWeb of Science
    68. ↵
      1. Verma D,
      2. Gostout CJ,
      3. Petersen BT, et al
      . Establishing a true assessment of endoscopic competence in ERCP during training and beyond: a single-operator learning curve for deep biliary cannulation in patients with native papillary anatomy. Gastrointest Endosc 2007;65:394–400. doi:10.1016/j.gie.2006.03.933
      OpenUrlCrossRefPubMed
    69. ↵
      1. Waller HM,
      2. Connor SJ
      . Cumulative sum (Cusum) analysis provides an objective measure of competency during training in endoscopic retrograde cholangio-pancreatography (ERCP). Hpb 2009;11:565–9. doi:10.1111/j.1477-2574.2009.00091.x
      OpenUrlCrossRefPubMed
    70. ↵
      1. Wani S,
      2. Cote GA,
      3. Keswani R, et al
      . Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training. Gastrointest Endosc 2013;77:558–65. doi:10.1016/j.gie.2012.10.012
      OpenUrlCrossRefPubMed
    71. ↵
      1. Watkins JL,
      2. Etzkorn KP,
      3. Wiley TE, et al
      . Assessment of technical competence during ERCP training. Gastrointest Endosc 1996;44:411–15. doi:10.1016/S0016-5107(96)70090-1
      OpenUrlCrossRefPubMedWeb of Science
    72. ↵
      1. Meenan J,
      2. Anderson S,
      3. Tsang S, et al
      . Training in radial EUS: what is the best approach and is there a role for the nurse endoscopist? Endoscopy 2003;35:1020–3. doi:10.1055/s-2003-44587
      OpenUrlCrossRefPubMed
    73. ↵
      1. Cass OW,
      2. Freeman ML,
      3. Peine CJ, et al
      . Objective evaluation of endoscopy skills during training. Ann Intern Med 1993;118:40–4. doi:10.7326/0003-4819-118-1-199301010-00008
      OpenUrlCrossRefPubMedWeb of Science
    74. ↵
      1. Chak A,
      2. Cooper GS,
      3. Blades EW, et al
      . Prospective assessment of colonoscopic intubation skills in trainees. Gastrointest Endosc 1996;44:54–7. doi:10.1016/S0016-5107(96)70229-8
      OpenUrlCrossRefPubMed
    75. ↵
      1. Church J,
      2. Oakley J,
      3. Milsom J, et al
      . Colonoscopy training: the need for patience (patients). ANZ J Surg 2002;72:89–91. doi:10.1046/j.1445-2197.2002.02312.x
      OpenUrlCrossRefPubMed
    76. ↵
      1. Hawes R,
      2. Lehman GA,
      3. Hast J
      . Training resident physicians in fiberoptic sigmoidoscopy. How many supervised examinations are required to achieve competence? Am J Med 1986;80:465–70. doi:10.1016/0002-9343(86)90721-7
      OpenUrlCrossRefPubMedWeb of Science
    77. ↵
      1. Leyden JE,
      2. Doherty GA,
      3. Hanley A, et al
      . Quality of colonoscopy performance among gastroenterology and surgical trainees: a need for common training standards for all trainees? Endoscopy 2011;43:935–40. doi:10.1055/s-0030-1256633
      OpenUrlCrossRefPubMed
    78. ↵
      1. Spier BJ,
      2. Benson M,
      3. Pfau PR, et al
      . Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 2010;71:319–24. doi:10.1016/j.gie.2009.05.012
      OpenUrlCrossRefPubMedWeb of Science
    79. ↵
      1. Spier BJ,
      2. Durkin ET,
      3. Walker AJ, et al
      . Surgical resident's training in colonoscopy: numbers, competency, and perceptions. Surg Endosc Interv Tech 2010;24:2556–61. doi:10.1007/s00464-010-1002-5
      OpenUrlCrossRef
    80. ↵
      1. Vassiliou MC,
      2. Kaneva PA,
      3. Poulose BK, et al
      . How should we establish the clinical case numbers required to achieve proficiency in flexible endoscopy? Am J Surg 2010;199:121–5. doi:10.1016/j.amjsurg.2009.10.004
      OpenUrlCrossRefPubMed
    81. ↵
      1. Vitale GC,
      2. Zavaleta CM,
      3. Vitale DS, et al
      . Training surgeons in endoscopic retrograde cholangiopancreatography. Surg Endosc Interv Tech 2006;20:149–52. doi:10.1007/s00464-005-0308-1
      OpenUrlCrossRef
    82. ↵
      1. Barton JR,
      2. Corbett S,
      3. Van Der Vleuten CP
      . The validity and reliability of a Direct Observation of Procedural Skills assessment tool: assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc 2012;75:591–7. doi:10.1016/j.gie.2011.09.053
      OpenUrlCrossRefPubMed
    83. ↵
      1. Boyle E,
      2. Al-Akash M,
      3. Patchett S, et al
      . Towards continuous improvement of endoscopy standards: validation of a colonoscopy assessment form. Colorectal Dis 2012;14:1126–31. doi:10.1111/j.1463-1318.2011.02883.x
      OpenUrlCrossRefPubMed
    84. ↵
      1. Hope WW,
      2. Hooks Iii WB,
      3. Kilbourne SN, et al
      . Assessing resident performance and training of colonoscopy in a general surgery training program. Surg Endosc Interv Tech 2013;27:1706–10. doi:10.1007/s00464-012-2660-2
      OpenUrlCrossRef
    85. ↵
      1. Mohamed R,
      2. Shaheen AA,
      3. Raman M
      . Evaluation of colonoscopy skills—How well are we doing? Can J Gastroenterol 2011;25:198–200.
      OpenUrlPubMed
    86. ↵
      1. Sarker SK,
      2. Albrani T,
      3. Zaman A, et al
      . Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal tool. Am J Surg 2008;196:450–5. doi:10.1016/j.amjsurg.2007.10.024
      OpenUrlCrossRefPubMed
    87. ↵
      1. Sedlack RE
      . The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 2010;72:1125–33.e1123. doi:10.1016/j.gie.2010.09.001
      OpenUrlCrossRefPubMed
    88. ↵
      1. Shah SG,
      2. Thomas-Gibson S,
      3. Brooker JC, et al
      . Use of video and magnetic endoscope imaging for rating competence at colonoscopy: validation of a measurement tool. Gastrointest Endosc 2002;56:568–73. doi:10.1016/S0016-5107(02)70449-5
      OpenUrlCrossRefPubMed
    89. ↵
      1. Thomas-Gibson S,
      2. Rogers PA,
      3. Suzuki N, et al
      . Development of a video assessment scoring method to determine the accuracy of endoscopist performance at screening flexible sigmoidoscopy. Endoscopy 2006;38:218–25. doi:10.1055/s-2005-870445
      OpenUrlCrossRefPubMed
    90. ↵
      1. Vassiliou MC,
      2. Kaneva PA,
      3. Poulose BK, et al
      . Global assessment of gastrointestinal endoscopic skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc Interv Tech 2010;24:1834–41. doi:10.1007/s00464-010-0882-8
      OpenUrlCrossRef
    91. ↵
      1. Gotzberger M,
      2. Rosch T,
      3. Geisenhof S, et al
      . Effectiveness of a novel endoscopy training concept. Endoscopy 2011;43:802–7. doi:10.1055/s-0030-1256372
      OpenUrlCrossRefPubMed
    92. ↵
      1. Harewood GC,
      2. Murray F,
      3. Winder S, et al
      . Evaluation of formal feedback on endoscopic competence among trainees: the EFFECT trial. Ir J Med Sci 2008;177:253–6. doi:10.1007/s11845-008-0161-z
      OpenUrlCrossRefPubMedWeb of Science
    93. ↵
      1. Neumann M,
      2. Hahn C,
      3. Horbach T, et al
      . Score card endoscopy: a multicenter study to evaluate learning curves in 1-week courses using the Elangen Endo-Trainer. Endoscopy 2003;35:515–20. doi:10.1055/s-2003-39670
      OpenUrlCrossRefPubMed
    94. ↵
      1. Neumann M,
      2. Meining A,
      3. Buerschaper C, et al
      . Training in GI endoscopy: can we objectively predict the performance of trainees? A prospective correlation study. Z Gastroenterol 2005;43:445–50. doi:10.1055/s-2005-857930
      OpenUrlCrossRefPubMed
    95. ↵
      1. Proctor DD,
      2. Price J,
      3. Dunn KA, et al
      . Prospective evaluation of a teaching model to determine competency in performing flexible sigmoidoscopies. Am J Gastroenterol 1998;93:1217–21. doi:10.1111/j.1572-0241.1998.00398.x
      OpenUrlCrossRefPubMedWeb of Science
    96. ↵
      1. Suzuki N,
      2. Thomas-Gibson S,
      3. Vance M, et al
      . Efficacy of an accelerated colonoscopy training week: audit from one national colonoscopy training center in the UK. Dig Endosc 2006;18:288–93. doi:10.1111/j.1443-1661.2006.00663.x
      OpenUrlCrossRef
    97. ↵
      1. Thomas-Gibson S,
      2. Bassett P,
      3. Suzuki N, et al
      . Intensive training over 5 days improves colonoscopy skills long-term. Endoscopy 2007;39:818–24. doi:10.1055/s-2007-966763
      OpenUrlCrossRefPubMed
    98. ↵
      1. Grimm H,
      2. Binmoeller KF,
      3. Soehendra N
      . Endosonography-guided drainage of a pancreatic pseudocyst. Gastrointest Endosc 1992;38:170–1. doi:10.1016/S0016-5107(92)70384-8
      OpenUrlCrossRefPubMed

    Supplementary materials

    Footnotes

    • VEE and ADK contributed equally to this work.

    • Contributors VEE, ADK: substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; drafting the work or revising it critically for important intellectual content; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. RDM, EJK: drafting the work or revising it critically for important intellectual content; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.