Development of a Video Assessment Scoring Method to Determine the Accuracy of Endoscopist Performance at Screening Flexible Sigmoidoscopy

 

 Buy Article Permissions and Reprints

Background and Aims: Variation in the adenoma detection rate (ADR) at flexible sigmoidoscopy screening has been shown to be due to variation in endoscopist performance. There are no objective methods for scoring an endoscopist’s performance reliably, and the aim of this study was to develop a valid and reliable objective scoring method using video footage of screening flexible sigmoidoscopies.
Methods: In a series of five experiments, experienced endoscopists (the scorers) independently scored a sample (n = 43) of the 40 000 flexible sigmoidoscopy extubations recorded as part of the United Kingdom Flexible Sigmoidoscopy Screening Trial (UK FSST). The scoring system, the parameters scored, and their definitions evolved over the course of the five experiments. The initial visual analogue score (range 0 - 100) used in the first two experiments evolved into a five-point score that ranged from 1 (E, poor) to 5 (A, excellent) in the last three experiments. The final parameters scored were: time spent viewing the mucosa, re-examination of poorly viewed areas, suctioning of fluid pools, distension of the lumen, lower rectal examination, and overall quality of the examination. The first four experiments scored one individual case per endoscopist; in experiment 5, an overall score was awarded for five cases performed by each endoscopist being assessed.
Results: Scoring five cases examined by an individual endoscopist using the A - E grading system was the most reliable method (interclass correlation coefficient 0.89). Cluster analysis demonstrated that the endoscopists in the high-scoring ADR group (ADR 14.7 - 15.9 %) could be differentiated from those in the intermediate- and low-scoring ADR groups (ADR 8.6 - 12.6 %).
Conclusions: An objective scoring system for assessing the accuracy of performance at screening flexible sigmoidoscopy, based on video footage, is described. Endoscopists who might benefit from further training can be identified using this method.

References

• 1 Atkin W S, Cuzick J, Northover J M, Whynes D K. Prevention of colorectal cancer by once-only sigmoidoscopy.  Lancet. 1993;  341 736-740
  CrossrefPubMedGoogle Scholar
• 2 UK Flexible Sigmoidoscopy Screening Trial Investigators . Single flexible sigmoidoscopy screening to prevent colorectal cancer: baseline findings of a UK multicentre randomised trial.  Lancet. 2002;  359 1291-1300
  CrossrefPubMedGoogle Scholar
• 3 Atkin W, Rogers P, Cardwell C. et al . Wide variation in adenoma detection rates at screening flexible sigmoidoscopy.  Gastroenterology. 2004;  126 1247-1256
  CrossrefPubMedGoogle Scholar
• 4 Bretthauer M, Skovlund E, Grotmol T. et al . Inter-endoscopist variation in polyp and neoplasia pick-up rates in flexible sigmoidoscopy screening for colorectal cancer.  Scand J Gastroenterol. 2003;  38 1268-1274
  CrossrefPubMedGoogle Scholar
• 5 Winawer S J, Zauber A G, Ho M N. et al . Prevention of colorectal cancer by colonoscopic polypectomy.  The National Polyp Study Workgroup N Engl J Med. 1993;  329 1977-1981
  CrossrefPubMedGoogle Scholar
• 6 Rex D K, Bond J H, Winawer S. et al . Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer.  Am J Gastroenterol. 2002;  97 1296-1308
  CrossrefPubMedGoogle Scholar
• 7 Bowles C J, Leicester R, Romaya C. et al . A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow?.  Gut. 2004;  53 277-283
  CrossrefPubMedGoogle Scholar
• 8 Joint Advisory Group on Gastrointestinal Endoscopy .Guidelines for the training, appraisal and assessment of trainees in gastrointestinal endoscopy. 2004 [accessed 2005 Jul 5]. Available from: URL: http//www.thejag.org.uk/JAG_2004.pdf. 
  Google Scholar
• 9 Rex D K. Colonoscopic withdrawal technique is associated with adenoma miss rates.  Gastrointest Endosc. 2000;  51 33-36
  CrossrefPubMedGoogle Scholar
• 10 Ramakrishnan K, Scheid D C. Predictors of incomplete flexible sigmoidoscopy.  J Am Board Fam Pract. 2003;  16 478-484
  PubMedGoogle Scholar
• 11 Walter L C, de Garmo P, Covinsky K E. Association of older age and female sex with inadequate reach of screening flexible sigmoidoscopy.  Am J Med. 2004;  116 174-178
  CrossrefPubMedGoogle Scholar
• 12 Eloubeidi M A, Wallace M B, Desmond R, Farraye F A. Female gender and other factors predictive of a limited screening flexible sigmoidoscopy examination for colorectal cancer.  Am J Gastroenterol. 2003;  98 1634-1639
  CrossrefPubMedGoogle Scholar
• 13 Armitage P, Berry G, Matthews J NS. Statistical methods in medical research. 4th edn. Oxford; Blackwell Science 2002
  Google Scholar
• 14 Hawes R, Lehman G A, Hast J. et al . Training resident physicians in fiberoptic sigmoidoscopy: how many supervised examinations are required to achieve competence?.  Am J Med. 1986;  80 465-470
  CrossrefPubMedGoogle Scholar
• 15 Cass O W. Training to competence in gastrointestinal endoscopy: a plea for continuous measuring of objective end points.  Endoscopy. 1999;  31 751-754
  Article in Thieme ConnectPubMedGoogle Scholar
• 16 Lal S K, Barrison A, Heeren T, Schroy P C III. A national survey of flexible sigmoidoscopy training in primary care graduate and postgraduate education programs.  Am J Gastroenterol. 2004;  99 830-836
  CrossrefPubMedGoogle Scholar
• 17 Weissman G S, Winawer S J, Baldwin M P. et al . Multicenter evaluation of training of non-endoscopists in 30-cm flexible sigmoidoscopy.  CA Cancer J Clin. 1987;  37 26-30
  PubMedGoogle Scholar
• 18 Health and Public Policy Committee, American College of Physicians . Clinical competence in colonoscopy.  Ann Intern Med. 1987;  107 772-774
  PubMedGoogle Scholar
• 19 Saad J A, Pirie P, Sprafka J M. Relationship between flexible sigmoidoscopy training during residency and subsequent sigmoidoscopy performance in practice.  Fam Med. 1994;  26 250-253
  PubMedGoogle Scholar
• 20 ASGE . Principles of training in gastrointestinal endoscopy. From the ASGE. American Society for Gastrointestinal Endoscopy.  Gastrointest Endosc. 1999;  49 845-853
  PubMedGoogle Scholar
• 21 Maule W F. Screening for colorectal cancer by nurse endoscopists.  N Engl J Med. 1994;  330 183-187
  CrossrefPubMedGoogle Scholar
• 22 Ashley O S, Nadel M, Ransohoff D F. Achieving quality in flexible sigmoidoscopy screening for colorectal cancer.  Am J Med. 2001;  111 643-653
  CrossrefPubMedGoogle Scholar
• 23 Proctor D D, Price J, Dunn K A. et al . Prospective evaluation of a teaching model to determine competency in performing flexible sigmoidoscopies.  Am J Gastroenterol. 1998;  93 1217-1221
  CrossrefPubMedGoogle Scholar
• 24 Holman J R, Marshall R C, Jordan B, Vogelman L. Technical competency in flexible sigmoidoscopy.  J Am Board Fam Pract. 2001;  14 424-429
  PubMedGoogle Scholar
• 25 Painter J, Saunders D B, Bell G D. et al . Depth of insertion at flexible sigmoidoscopy: implications for colorectal cancer screening and instrument design.  Endoscopy. 1999;  31 227-231
  Article in Thieme ConnectPubMedGoogle Scholar
• 26 Cutler A F, Pop A. Fifteen years later: colonoscopic retroflexion revisited.  Am J Gastroenterol. 1999;  94 1537-1538
  CrossrefPubMedGoogle Scholar
• 27 Shinya H, Cwern M, Karlstadt R. Colonoscopy: technique and training methods.  Surg Clin North Am. 1982;  62 869-876
  PubMedGoogle Scholar
• 28 Cotton P, Williams C B. Practical gastrointestinal endoscopy. 5th edn. Oxford; Blackwell Scientific 2003
  Google Scholar
• 29 Stranc M F, McDiarmid J G, Stranc L C. Video assessment of surgical technique.  Br J Plast Surg. 1991;  44 65-68
  CrossrefPubMedGoogle Scholar
• 30 Scott D J, Rege R V, Bergen P C. et al . Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation.  J Laparoendosc Adv Surg Tech A. 2000;  10 183-190
  CrossrefPubMedGoogle Scholar
• 31 Eubanks T R, Clements R H, Pohl D. et al . An objective scoring system for laparoscopic cholecystectomy.  J Am Coll Surg. 1999;  189 566-574
  CrossrefPubMedGoogle Scholar
• 32 Miller G E. The assessment of clinical skills/competence/performance.  Acad Med. 1990;  65 (9 Suppl) S63-S67
  CrossrefPubMedGoogle Scholar

W. Atkin, MPH, PhD

Cancer Research UK Colorectal Cancer Unit · St Mark’s Hospital

Watford Road · Harrow, HA1 3UJ · United Kingdom

Fax: +44-208-235-4277

Email: wendy.atkin@cancer.org.uk