Introduction

Colonoscopy is a technically demanding procedure, requiring proficiency with both technical and cognitive skills. At least 100 cases are needed to attain a completion rate of 75%.1 2 The traditional colonoscopy training is based on supervised ‘learning by doing’ on patients and is associated with prolonged procedure time. This leads to a significant increase of expense and decreased capacity of endoscopy centres.3 Furthermore, some studies showed negative association between colonoscopy experience and complication rates.4 5 A recent development in computer technologies has provided VR trainers that allow acquisition of professional skills in a virtual environment prior to procedures on a real patient. Although a virtual endoscopy training can not totally replace traditional bedside teaching,6 it has the potential to shorten the learning curves in the endoscopy suite and hereby lead to patient and economic benefits.7–10 The process of developing and validating standardised endoscopy training curricula has not been completely fulfilled. To date there is very limited knowledge on the relevance of external feedback in acquisition of basic endoscopy skills.11 The present study aimed to assess the impact of a concurrent external feedback on learning curves and skills retention using a virtual-reality (VR) colonoscopy simulator.

Materials and methods

The study was carried out in a Department of surgical gastroenterology at a teaching hospital.

The Accu Touch Endoscopy Simulator (Immersion Medical Corporation, Gaithersburg, Maryland, USA) was used in the study. This system and the nature of the tasks have previously been described.9 12 13 Thirty subjects were included in the study: 22 trainees, without colonoscopy experience and eight experts (performed ≥500 colonoscopies). The participants were randomised, using sealed envelopes, into two groups: feedback (five females, median age 29 (range 26–37) years, median years since graduation 0.75 (range 0.5–4)) and non-feedback (four females, median age 30 (range 28–34) years, median years since graduation 2 (range 0.5–5)). The subjects in both groups were equally distributed with respect to experience with computer games. No participants had previous experience with endoscopic simulation. All subjects received identical pre-trial instruction on the simulator and in basic colonoscopy technique, using videos from the simulator's didactic content. Expert colonoscopists were asked to perform three repetitions of task 3 from the introduction module in order to provide the reference expert level of performance. Task 3 is of median complexity and involves an alpha loop. Perforation may occur if the loop is not reduced with clockwise rotation of the scope. Untrained doctors were asked to perform 15 repetitions of task 3, divided in four sessions within 1 month with a maximum two sessions per week and three to five repetitions per session. Trainees in both groups were allowed to use feedback from the ‘virtual attending physician’, to view the position of the scope and the appearance of the virtual colon. All had access to the computer generated case report after each procedure. Trainees were not allowed to provide sedation to the virtual patient. Trainees in the feedback group were, in addition, provided with structured constructive verbal feedback by the same instructor during and after every procedure, including a debriefing session with evaluation of the simulator generated results. In the case of perforation trainees were asked to perform an extra procedure and their procedure time before the perforation was added to the following procedure's time as a penalty score. Four to six weeks after the last attempt trainees from both groups performed delayed retention or delayed transfer tests, consisting of two virtual colonoscopies: task 3 and the new task 5 from the introduction colonoscopy module, without external feedback. Task 5 involves reverse alpha loop which needs to be reduced when applying counter clockwise rotation of the scope to avoid a perforation; moreover there are stool debris in the left colon and the splenic flexure has a slightly more acute bend then in task 3. The following computer generated metrics were selected to evaluate the participant's performance: total procedure time, time to advance the scope, per cent of time without discomfort, per cent of mucosa visualised and amount of the insufflated air. One participant from the feedback group had only performed ten virtual colonoscopies during the training period because of technical problems with the simulator; one participant from the non-feedback group was not able to attend the retention/transfer tests because of scheduling difficulties (figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flow of participants through the trial. VR, virtual reality.

Statistical analysis

Data was analysed using StataMP 10 software package. The Mann–Whitney U non-parametric test was used to examine the differences between the groups of participants. Multiple comparisons were made for the first repetition; the fifth, the tenth and the fifteenth repetitions were compared in both groups of residents and afterwards to the third repetition in expert's group. Wilcoxon matched-pairs signed-rank test was used to evaluate the retention or transfer of the skills within the group. A p-value <0.05 was considered statistically significant. Values are given as median (range) unless stated otherwise.

Results

All subjects were able to complete the colonoscopy on the simulator. The virtual patient did not experience extreme discomfort in any of the groups. No perforations occurred in the feedback and the experts groups, compared with seven perforations in the control group (median 0, range 0–3), all within the first five repetitions. Retention and transfer tests were performed by all the groups without complications. Retroflexion in rectum was not performed in 20 procedures in the non-feedback group (median 1 (0–4), and in two retention and four transfer tests as compared to one (training period) and two (transfer tests, task 5) in feedback group. Experts did not demonstrate significant learning curves in any of the simulator metrics from the second to the third repetition (Wilcoxon sign-rank test, p≥0.2). Both groups of residents improved their procedure time and reduced amount of insufflated air (figure 2a,b,d). Experts and the subjects in the feedback group visualised significantly higher percent of mucosa during the first, the fifth and the tenth attempts (feedback group) then non-feedback group (figure 2c). The experts performed significantly faster than both groups of residents (total time) during the first, the fifth and the tenth attempts, while the feedback group over-performed the non-feedback group during the fifth and the tenth repetitions (figure 2a). Experts and the feedback group reached the caecum significantly faster than the non-feedback group during the first, fifth and the tenth repetitions, experts over-performed the feedback group only during the first repetition (figure 2b). There were no significant differences between the two groups of residents regarding patient discomfort level; however, the virtual patient experienced significantly more discomfort in the expert group (figure 2e). No significant differences in the amount of the insufflated air were observed between the three groups. Performance differences between the fifteenth repetition and retention (non-feedback group) or transfer (feedback group) testing (task 3), were compatible for both groups (p≥0.6), none of the residents groups demonstrated significant degradation in performance, except in total procedure time for feedback group (p=0.02) though the median values for procedure time and procedure time degradation were lower for feedback group as compared to non-feedback. In retention/transfer tests subjects in feedback group faster and visualised more mucosa. However, significance values were achieved only for advance time (task 3) and visualised mucosa (task 5) (figures 3 and 4).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Learning curves for feedback group (white circles), non-feedback group (black circles) and experts (black triangles). (a) total procedure time, (b) time to reach the caecum, (c) percentage of the mucosa visualised, (d) amount of insufflated air, (e) percentage of the procedure time without the virtual patient experiencing discomfort.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Retention tests on the simulator, case 3 for feedback group (white circles) and non-feedback group (black circles). Horizontal bars represent the median. (a) Total procedure time, (b) time to reach the caecum, (c) percentage of the mucosa visualised, (d) amount of insufflated air, (e) percentage of the procedure time without the virtual patient experiencing discomfort.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Transfer tests on the simulator, case 5 for feedback group (white circles) and non-feedback group (black circles). Horizontal bars represent the median. (a) Total procedure time, (b) time to reach the caecum, (c) percentage of the mucosa visualised, (d) amount of insufflated air, (e) percentage of the procedure time without the virtual patient experiencing discomfort.

Discussion

The present study provides evidence on the impact of external feedback on the quality of skill acquisition in gastrointestinal endoscopy. The study showed that constructive feedback has a beneficial effect on learning compared to independent training. The trainees in the feedback group performed better than the residents in the non-feedback group in most of the outcome parameters: procedure time, time to reach the caecum and visualised mucosa up to the tenth repetition. The trainees who received feedback reached proficiency level on the simulator significantly faster. Furthermore, the residents who received feedback performed significantly better than the controls during the delayed retention/transfer tests for the parameters advance time (task 3) and visualised mucosa (task 5).

Previous studies examining the impact of external feedback for motor skills acquisition have produced contradictory results. Schmidt and Wulf14 in a randomised study showed that continuous concurrent feedback enhanced performance during practice when feedback is operating but it did not contribute to learning and may even degrade it as measured on retention/transfer tests. These tests are considered as the best predictors for long-term learning.15 Stefanidis et al16 showed that limited external feedback and video tutorials optimised learning in laparoscopic suturing as compared to video tutorials or intense or limited feedback alone. In this study data from two different randomised trials were used; however, retention or transfer tests were not performed. Porte et al17 and Rogers et al18 demonstrated a positive effect of external feedback on knot-tying learning in randomised controlled trials. However, in the latter, long-term retention or transfer of acquired skills were not tested.

A general limitation of the above-mentioned studies is that they use simplistic models and their results might not be applicable for more complex tasks and procedures.19 No previous studies have assessed the impact of external feedback on acquisition of basic skills in gastrointestinal endoscopy.

Mahmood and Darzi11 demonstrated the absence of learning curve in colonoscopy simulator without any kind of feedback and concluded that there is no learning curve for the machine. Therefore we have allowed trainees in both groups to use feedback from the ‘virtual attending physician’, to view the position of the scope tip and appearance of virtual patient's colon and to become acquainted with computer-generated case reports after each procedure; these features were available on Accu Touch colonoscopy training device during the independent training. A study group with no constructive expert feedback is obsolete in bedside teaching but seems very relevant when speaking about endoscopy training in a virtual environment. The present study shows that constructive expert feedback is superior to the simulator feedback alone. None of the residents included in this study found the computer generated concurrent feedback ‘Virtual attending physician’ helpful. The only message providing by the computer in different situations was: “You are stretching the patient's colon, withdraw the scope in few cm”, ignoring the particular condition. Future software developments should aim at providing improved constructive, real-time feedback thus eliminating the need for a supervising expert at each teaching session.20

The virtual patient experienced more discomfort during the experienced endoscopists' performance compared with novices. This can be explained by expert's acceptance of a slightly higher discomfort level as compared to novices and by the fact that the computer-generated tactile and verbal feedback are not sufficiently realistic.

The limitations of the present study are that it is not blinded (though this is not possible), the outcome measures are simulator based and there is no primary outcome measure. However, our results allow us to conclude that constructed concurrent feedback ensures safe acquisition of cognitive and psychomotor skills relevant for gastrointestinal endoscopy. This can potentially contribute to reduction of errors and improved patient safety. Further studies are needed to evaluate the optimal training curriculum for gastrointestinal endoscopy incorporating all evidence-based teaching tools. Simulator provided feedback needs improvement in order to make independent training on the simulator more effective.