Clin Endosc.  2024 Mar;57(2):217-225. 10.5946/ce.2023.145.

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

• Tiankanon K
• Karuehardsuwan J
• Aniwan S
• Mekaroonkamol P
• Sunthornwechapong P
• Navadurong H
• Tantitanawat K
• Mekritthikrai K
• Samutrangsi S
• Vateekul P
• Rerknimitr R

• KMID: 2553759
• DOI: http://doi.org/10.5946/ce.2023.145

Abstract

Background/Aims
This study aims to compare polyp detection performance of “Deep-GI,” a newly developed artificial intelligence (AI) model, to a previously validated AI model computer-aided polyp detection (CADe) using various false positive (FP) thresholds and determining the best threshold for each model.
Methods
Colonoscopy videos were collected prospectively and reviewed by three expert endoscopists (gold standard), trainees, CADe (CAD EYE; Fujifilm Corp.), and Deep-GI. Polyp detection sensitivity (PDS), polyp miss rates (PMR), and false-positive alarm rates (FPR) were compared among the three groups using different FP thresholds for the duration of bounding boxes appearing on the screen.
Results
In total, 170 colonoscopy videos were used in this study. Deep-GI showed the highest PDS (99.4% vs. 85.4% vs. 66.7%, p<0.01) and the lowest PMR (0.6% vs. 14.6% vs. 33.3%, p<0.01) when compared to CADe and trainees, respectively. Compared to CADe, Deep-GI demonstrated lower FPR at FP thresholds of ≥0.5 (12.1 vs. 22.4) and ≥1 second (4.4 vs. 6.8) (both p<0.05). However, when the threshold was raised to ≥1.5 seconds, the FPR became comparable (2 vs. 2.4, p=0.3), while the PMR increased from 2% to 10%.
Conclusions
Compared to CADe, Deep-GI demonstrated a higher PDS with significantly lower FPR at ≥0.5- and ≥1-second thresholds. At the ≥1.5-second threshold, both systems showed comparable FPR with increased PMR.

Keyword

Artificial intelligence; Colonoscopy; Detection; Polyps; Trainee

Figure

• 

  Fig. 1. Example of an adenomatous polyp detected by AI models. (A) A diminutive sessile polyp detected by the computer-aided polyp detection (CADe) model; (B) the same polyp detected by the Deep-GI model; (C) and (D) the alarm-tracing program detecting the bounding boxes of CADe and Deep-GI, respectively.

• 

  Fig. 2. An adenomatous polyp >1 cm (within the red box) that was missed by Deep-GI, computer-aided polyp detection (CADe), and the trainees.

• 

Reference

1. Virani S, Bilheem S, Chansaard W, et al. National and subnational population-based incidence of cancer in Thailand: assessing cancers with the highest burdens. Cancers (Basel). 2017; 9:108.

2. Siegel RL, Miller KD, Goding Sauer A, et al. Colorectal cancer statistics, 2020. CA Cancer J Clin. 2020; 70:145–164.

3. Zauber AG, Winawer SJ, O'Brien MJ, et al. Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med. 2012; 366:687–696.

4. Doubeni CA, Corley DA, Quinn VP, et al. Effectiveness of screening colonoscopy in reducing the risk of death from right and left colon cancer: a large community-based study. Gut. 2018; 67:291–298.

5. Rex DK, Schoenfeld PS, Cohen J, et al. Quality indicators for colonoscopy. Am J Gastroenterol. 2015; 110:72–90.

6. Corley DA, Jensen CD, Marks AR, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014; 370:1298–1306.

7. Rex DK. Colonoscopic withdrawal technique is associated with adenoma miss rates. Gastrointest Endosc. 2000; 51:33–36.

8. Barclay RL, Vicari JJ, Doughty AS, et al. Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med. 2006; 355:2533–2541.

9. Francis DL, Rodriguez-Correa DT, Buchner A, et al. Application of a conversion factor to estimate the adenoma detection rate from the polyp detection rate. Gastrointest Endosc. 2011; 73:493–497.

10. Boroff ES, Gurudu SR, Hentz JG, et al. Polyp and adenoma detection rates in the proximal and distal colon. Am J Gastroenterol. 2013; 108:993–999.

11. Niv Y. Polyp detection rate may predict adenoma detection rate: a meta-analysis. Eur J Gastroenterol Hepatol. 2018; 30:247–251.

12. Ng S, Sreenivasan AK, Pecoriello J, et al. Polyp detection rate correlates strongly with adenoma detection rate in trainee endoscopists. Dig Dis Sci. 2020; 65:2229–2233.

13. Aziz M, Fatima R, Dong C, et al. The impact of deep convolutional neural network-based artificial intelligence on colonoscopy outcomes: a systematic review with meta-analysis. J Gastroenterol Hepatol. 2020; 35:1676–1683.

14. Wang P, Berzin TM, Glissen Brown JR, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019; 68:1813–1819.

15. Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 2018; 155:1069–1078.

16. Repici A, Badalamenti M, Maselli R, et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology. 2020; 159:512–520.

17. Barua I, Vinsard DG, Jodal HC, et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy. 2021; 53:277–284.

18. Kudo SE, Misawa M, Mori Y, et al. Artificial intelligence-assisted system improves endoscopic identification of colorectal neoplasms. Clin Gastroenterol Hepatol. 2020; 18:1874–1881.

19. Mori Y, Bretthauer M. Addressing false-positive findings with artificial intelligence for polyp detection. Endoscopy. 2021; 53:941–942.

20. Misawa M, Kudo SE, Mori Y, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology. 2018; 154:2027–2029.

21. Holzwanger EA, Bilal M, Glissen Brown JR, et al. Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy. Endoscopy. 2021; 53:937–940.

22. Zhan W, Sun C, Wang M, et al. An improved Yolov5 real-time detection method for small objects captured by UAV. Soft Comput. 2022; 26:361–373.

23. Alagappan M, Brown JR, Mori Y, et al. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc. 2018; 10:239–249.

24. Vinsard DG, Mori Y, Misawa M, et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc. 2019; 90:55–63.

25. Wang P, Liu X, Berzin TM, et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol. 2020; 5:343–351.

26. Wang Z, Liang Z, Li L, et al. Reduction of false positives by internal features for polyp detection in CT-based virtual colonoscopy. Med Phys. 2005; 32:3602–3616.

27. Wang P, Xiao X, Glissen Brown JR, et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng. 2018; 2:741–748.

28. Weigt J, Repici A, Antonelli G, et al. Performance of a new integrated computer-assisted system (CADe/CADx) for detection and characterization of colorectal neoplasia. Endoscopy. 2022; 54:180–184.