Single frame workflow recognition during endoscopic submucosal dissection (ESD) using artificial intelligence (AI)

  • Aims  Precise surgical phase recognition and evaluation may improve our understanding of complex endoscopic procedures. Furthermore, quality control measurements and endoscopy training could benefit from objective descriptions of surgical phase distributions. Therefore, we aimed to develop an artificial intelligence algorithm for frame-by-frame operational phase recognition during endoscopicAims  Precise surgical phase recognition and evaluation may improve our understanding of complex endoscopic procedures. Furthermore, quality control measurements and endoscopy training could benefit from objective descriptions of surgical phase distributions. Therefore, we aimed to develop an artificial intelligence algorithm for frame-by-frame operational phase recognition during endoscopic submucosal dissection (ESD). Methods  Full length ESD-videos from 31 patients comprising 6.297.782 single images were collected retrospectively. Videos were annotated on a frame-by-frame basis for the operational macro-phases diagnostics, marking, injection, dissection and bleeding. Further subphases were the application of electrical current, visible injection of fluid into the submucosal space and scope manipulation, leading to 11 phases in total. 4.975.699 frames (21 patients) were used for training of a video swin transformer using uniform frame sampling for temporal information. Hyperparameter tuning was performed with 897.325 further frames (6 patients), while 424.758 frames (4 patients) were used for validation. Results  The overall F1 scores on the test dataset for the macro-phases and all 11 phases were 0.96 and 0.90, respectively. The recall values for diagnostics, marking, injection, dissection and bleeding were 1.00, 1.00, 0.95, 0.96 and 0.93, respectively. Conclusions  The algorithm classified operational phases during ESD with high accuracy. A precise evaluation of phase distribution may allow for the development of objective quality metrics for quality control and training.show moreshow less

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Markus W. Scheppach, Danilo Weber Nunes, X. Arizi, David Rauber, Andreas Probst, Sandra Nagl, Christoph RömmeleORCiD, Christoph PalmORCiDGND, Helmut Messmann, Alanna EbigboORCiD
DOI:https://doi.org/10.1055/s-0045-1806324
Parent Title (English):Endoscopy
Publisher:Thieme
Place of publication:Stuttgart
Document Type:conference proceeding (presentation, abstract)
Language:English
Year of first Publication:2025
Release Date:2025/04/28
Volume:57
Issue:S 02
First Page:S511
Konferenzangabe:ESGE Days 2025
Institutes:Fakultät Informatik und Mathematik
Fakultät Informatik und Mathematik / Labor Regensburg Medical Image Computing (ReMIC)
Begutachtungsstatus:peer-reviewed
research focus:Lebenswissenschaften und Ethik
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG
Einverstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.