Effect of AI on performance of endoscopists to detect Barrett neoplasia: A Randomized Tandem Trial

  • Background and study aims To evaluate the effect of an AI-based clinical decision support system (AI) on the performance and diagnostic confidence of endoscopists during the assessment of Barrett's esophagus (BE). Patients and Methods Ninety-six standardized endoscopy videos were assessed by 22 endoscopists from 12 different centers with varying degrees of BE experience. The assessment was randomized into two video sets: Group A (review first without AI and second with AI) and group B (review first with AI and second without AI). Endoscopists were required to evaluate each video for the presence of Barrett's esophagus-related neoplasia (BERN) and then decide on a spot for a targeted biopsy. After the second assessment, they were allowed to change their clinical decision and confidence level. Results AI had a standalone sensitivity, specificity, and accuracy of 92.2%, 68.9%, and 81.6%, respectively. Without AI, BE experts had an overall sensitivity, specificity, and accuracy of 83.3%, 58.1 and 71.5%, respectively. With AI, BE nonexperts showed a significant improvement in sensitivity and specificity when videos were assessed a second time with AI (sensitivity 69.7% (95% CI, 65.2% - 74.2%) to 78.0% (95% CI, 74.0% - 82.0%); specificity 67.3% (95% CI, 62.5% - 72.2%) to 72.7% (95 CI, 68.2% - 77.3%). In addition, the diagnostic confidence of BE nonexperts improved significantly with AI. Conclusion BE nonexperts benefitted significantly from the additional AI. BE experts and nonexperts remained below the standalone performance of AI, suggesting that there may be other factors influencing endoscopists to follow or discard AI advice.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Author:Michael MeinikheimORCiD, Robert MendelORCiD, Christoph PalmORCiDGND, Andreas Probst, Anna Muzalyova, Markus Wolfgang Scheppach, Sandra Nagl, Elisabeth Schnoy, Christoph RömmeleORCiD, Dominik Andreas Helmut Otto Schulz, Jakob Schlottmann, Friederike Prinz, David Rauber, Tobias RückertORCiD, Tomoaki MatsumuraORCiD, Glòria Fernández-Esparrach, Nasim Parsa, Michael F Byrne, Helmut Messmann, Alanna EbigboORCiD
Pubmed Id:https://pubmed.ncbi.nlm.nih.gov/38547927
Parent Title (English):Endoscopy
Publisher:Georg Thieme Verlag
Document Type:Article
Year of Completion:2024
Release Date:2024/04/25
Accepted Manuscript
Institutes:Fakultät Informatik und Mathematik
Fakultät Informatik und Mathematik / Regensburg Medical Image Computing (ReMIC)
research focus:Lebenswissenschaften und Ethik
OpenAccess Publikationsweg:Hybrid Open Access - OA-Veröffentlichung in einer Subskriptionszeitschrift/-medium
Licence (German):Creative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International