Abstract
Background
The use of simulation for competency assessment requires validation of the simulator’s performance metrics. This study evaluated whether the Simbionix GI Mentor II virtual reality simulator metrics differentiate gastrointestinal endoscopists with varying clinical experience (known-groups construct validity).
Methods
For this study, 20 subjects (medical and surgical) were classified into two groups based on self-reported clinical experience with colonoscopy: a novice group (<5 scope experiences, n = 12) and an experienced group (>50 scope experiences, n = 8). Three virtual colonoscopy simulation modules of increasing difficulty were used (modules I-1, II-2, and I-7). The data reported by the simulator after each module were compared using the Wilcoxon–Mann–Whitney test. Data are expressed as median and interquartile range (IQR). A p value less than 0.05 was considered statistically significant.
Results
With module 1, only the time taken to reach the cecum was different between the groups: experienced group (1.6 min; IQR, 1.2–1.9 min) versus novice group (3.2 min; IQR, 2.4–4 min) (p < 0.01). With module 2, the two groups differed only in the time needed to reach the cecum (experienced group: 2.3 min; IQR, 1.6–2.3 min vs novice group: 3.3 min; IQR, 2.3–4.2 min; p = 0.03) and overall efficiency (experienced group: 94%; IQR, 94–96% vs novice group: 88%, IQR, 69–92%) (p < 0.01). In contrast, with the module 3 (the most difficult), performance differed between the groups for most of the parameters. The experienced group reached the cecum faster (5.7 min; IQR, 3.6–6.6 min vs. 14 min; IQR, 9–16 min; p < 0.01) and had fewer occasions of lost view (0.5; IQR, 0–1 vs. 2; IQR, 2–3; p < 0.01), fewer episodes of excessive pressure (2; IQR, 1–2 vs. 4.5; IQR, 2.5–6; p < 0.01), and greater overall efficiency (87%; IQR, 82–89% vs. 29%; IQR, 23–55%; p < 0.01). There were no differences in the percentage of time the patient was in pain or in the total time the colon was looped. The experienced group saw slightly less of the mucosa (91%; IQR, 89–92% vs 94%; IQR, 93–95%; p = 0.01).
Conclusion
The GI Mentor II metrics differentiated novice colonoscopists from those with more clinical experience, but primarily when used to evaluate the more complex scenarios. In setting performance benchmarks, the case scenario must be taken into account.
Similar content being viewed by others
References
Bini EJ, Firoozi B, Choung RJ, Ali EM, Osman M, Weinshel EH (2003) Systematic evaluation of complications related to endoscopy in a training setting (SECRETS): a prospective 30-day outcomes study. Gastrointest Endosc 57:8–16
Fried GM, Feldman LS (2008) Objective assessment of technical performance. World J Surg 32:156–160
Greenwald D, Cohen J (2006) Evolution of endoscopy simulators and their application. Gastrointest Endosc Clin North Am 16:389–406
Sturm LP, Windsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ (2008) A systematic review of skills transfer after surgical simulation training. Ann Surg 248:166–179
Felsher JJ, Olesevich M, Farres H, Rosen M, Fanning A, Dunkin BJ, Marks JM (2005) Validation of a flexible endoscopy simulator. Am J Surg 189:497–500
Grantcharov TP, Carstensen L, Schulze S (2005) Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS 9:130–133
Koch AD, Buzink SN, Heemskerk J, Botden SMBI, Veenendaal R, Jakimowicz JJ, Schoon EJ (2007) Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy. Surg Endosc 22:158–162
Bar-Meir S (2006) Simbionix simulator. Gastrointest Endosc Clin North Am 16:471–478
Gerson LB (2006) Evidence-based assessment of endoscopic simulators for training. Gastrointest Endosc Clin North Am 16:489–509
Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, Bini EJ, Bodenheimer H, Cerulli M, Gerdes H et al (2006) Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 64:361–368
Phitayakorn R, Marks JM, Reynolds HL, Delaney CP (2009) Expert benchmark for the GI mentor II. Surg Endosc 23:611–614
ACGME memorandum (2006) Changes in minimum requirements for laparoscopy and endoscopy. Retrieved 9 March 2009 at www.acgme.org/acWebsite/RRC_440/440_minReqLaparoscopy.asp
Disclosures
Raad Fayez, Liane S. Feldman, Pepa Kaneva, and Gerald M. Fried have no conflicts of interest of financial ties to disclose.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fayez, R., Feldman, L.S., Kaneva, P. et al. Testing the construct validity of the Simbionix GI Mentor II virtual reality colonoscopy simulator metrics: module matters. Surg Endosc 24, 1060–1065 (2010). https://doi.org/10.1007/s00464-009-0726-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-009-0726-6