Human-Computer Interaction: Impact of Artificial Intelligence on the diagnostic confidence of endoscopists assessing videos of Barrett’s esophagus

  • Aims Human-computer interactions (HCI) may have a relevant impact on the performance of Artificial Intelligence (AI). Studies show that although endoscopists assessing Barrett’s esophagus (BE) with AI improve their performance significantly, they do not achieve the level of the stand-alone performance of AI. One aspect of HCI is the impact of AI on the degree of certainty and confidence displayed by the endoscopist. Indirectly, diagnostic confidence when using AI may be linked to trust and acceptance of AI. In a BE video study, we aimed to understand the impact of AI on the diagnostic confidence of endoscopists and the possible correlation with diagnostic performance. Methods 22 endoscopists from 12 centers with varying levels of BE experience reviewed ninety-six standardized endoscopy videos. Endoscopists were categorized into experts and non-experts and randomly assigned to assess the videos with and without AI. Participants were randomized in two arms: Arm A assessed videos first without AI and then with AI, while Arm B assessed videos in the opposite order. Evaluators were tasked with identifying BE-related neoplasia and rating their confidence with and without AI on a scale from 0 to 9. Results The utilization of AI in Arm A (without AI first, with AI second) significantly elevated confidence levels for experts and non-experts (7.1 to 8.0 and 6.1 to 6.6, respectively). Only non-experts benefitted from AI with a significant increase in accuracy (68.6% to 75.5%). Interestingly, while the confidence levels of experts without AI were higher than those of non-experts with AI, there was no significant difference in accuracy between these two groups (71.3% vs. 75.5%). In Arm B (with AI first, without AI second), experts and non-experts experienced a significant reduction in confidence (7.6 to 7.1 and 6.4 to 6.2, respectively), while maintaining consistent accuracy levels (71.8% to 71.8% and 67.5% to 67.1%, respectively). Conclusions AI significantly enhanced confidence levels for both expert and non-expert endoscopists. Endoscopists felt significantly more uncertain in their assessments without AI. Furthermore, experts with or without AI consistently displayed higher confidence levels than non-experts with AI, irrespective of comparable outcomes. These findings underscore the possible role of AI in improving diagnostic confidence during endoscopic assessment.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Author:D. A. Roser, Michael Meinikheim, Robert MendelORCiD, Christoph PalmORCiDGND, Andreas Probst, A. Muzalyova, Markus W. Scheppach, S. Nagl, Elisabeth Schnoy, Christoph Römmele, D. Schulz, Jakob Schlottmann, Friederike Prinz, David Rauber, Tobias RückertORCiD, T. Matsumura, G. Fernandez-Esparrach, N. Parsa, M. Byrne, Helmut Messmann, Alanna EbigboORCiD
Parent Title (English):Endoscopy
Publisher:Georg Thieme Verlag
Document Type:conference proceeding (presentation, abstract)
Year of first Publication:2024
Release Date:2024/04/26
Issue:S 02
First Page:79
Konferenzangabe:ESGE Days 2024, Berlin, 25.–27.04.2024
Institutes:Fakultät Informatik und Mathematik
research focus:Lebenswissenschaften und Ethik
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG