Artificial Intelligence (AI) – assisted vessel and tissue recognition during third space endoscopy (Smart ESD)

  • Clinical setting  Third space procedures such as endoscopic submucosal dissection (ESD) and peroral endoscopic myotomy (POEM) are complex minimally invasive techniques with an elevated risk for operator-dependent adverse events such as bleeding and perforation. This risk arises from accidental dissection into the muscle layer or through submucosal blood vessels as the submucosal cutting plane within the expanding resection site is not always apparent. Deep learning algorithms have shown considerable potential for the detection and characterization of gastrointestinal lesions. So-called AI – clinical decision support solutions (AI-CDSS) are commercially available for polyp detection during colonoscopy. Until now, these computer programs have concentrated on diagnostics whereas an AI-CDSS for interventional endoscopy has not yet been introduced. We aimed to develop an AI-CDSS („Smart ESD“) for real-time intra-procedural detection and delineation of blood vessels, tissue structures and endoscopic instruments during third-space endoscopic procedures. Characteristics of Smart ESD  An AI-CDSS was invented that delineates blood vessels, tissue structures and endoscopic instruments during third-space endoscopy in real-time. The output can be displayed by an overlay over the endoscopic image with different modes of visualization, such as a color-coded semitransparent area overlay, or border tracing (demonstration video). Hereby the optimal layer for dissection can be visualized, which is close above or directly at the muscle layer, depending on the applied technique (ESD or POEM). Furthermore, relevant blood vessels (thickness> 1mm) are delineated. Spatial proximity between the electrosurgical knife and a blood vessel triggers a warning signal. By this guidance system, inadvertent dissection through blood vessels could be averted. Technical specifications  A DeepLabv3+ neural network architecture with KSAC and a 101-layer ResNeSt backbone was used for the development of Smart ESD. It was trained and validated with 2565 annotated still images from 27 full length third-space endoscopic videos. The annotation classes were blood vessel, submucosal layer, muscle layer, electrosurgical knife and endoscopic instrument shaft. A test on a separate data set yielded an intersection over union (IoU) of 68%, a Dice Score of 80% and a pixel accuracy of 87%, demonstrating a high overlap between expert and AI segmentation. Further experiments on standardized video clips showed a mean vessel detection rate (VDR) of 85% with values of 92%, 70% and 95% for POEM, rectal ESD and esophageal ESD respectively. False positive measurements occurred 0.75 times per minute. 7 out of 9 vessels which caused intraprocedural bleeding were caught by the algorithm, as well as both vessels which required hemostasis via hemostatic forceps. Future perspectives  Smart ESD performed well for vessel and tissue detection and delineation on still images, as well as on video clips. During a live demonstration in the endoscopy suite, clinical applicability of the innovation was examined. The lag time for processing of the live endoscopic image was too short to be visually detectable for the interventionist. Even though the algorithm could not be applied during actual dissection by the interventionist, Smart ESD appeared readily deployable during visual assessment by ESD experts. Therefore, we plan to conduct a clinical trial in order to obtain CE-certification of the algorithm. This new technology may improve procedural safety and speed, as well as training of modern minimally invasive endoscopic resection techniques.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Author:Markus W. ScheppachORCiD, Robert MendelORCiD, Andreas Probst, Michael Meinikheim, Christoph PalmORCiDGND, Helmut Messmann, Alanna EbigboORCiD
Parent Title (German):Zeitschrift für Gastroenterologie
Publisher:Georg Thieme Verlag
Place of publication:Stuttgart
Document Type:conference proceeding (presentation, abstract)
Year of first Publication:2022
Release Date:2022/09/16
Tag:Artificial Intelligence; Endoscopy; Medical Image Computing
GND Keyword:Bildgebendes Verfahren; Medizin; Künstliche Intelligenz; Endoskopie
Article Number:e650
Konferenzangabe:Jahrestagung der Deutschen Gesellschaft für Gastroenterologie, Verdauungs- und Stoffwechselkrankheiten mit Sektion Endoskopie, 76, 2022, Hamburg
Institutes:Fakultät Informatik und Mathematik
Regensburg Center of Biomedical Engineering - RCBE
Regensburg Center of Health Sciences and Technology - RCHST
Fakultät Informatik und Mathematik / Regensburg Medical Image Computing (ReMIC)
research focus:Lebenswissenschaften und Ethik
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG