Overview Statistic: PDF-Downloads (blue) and Frontdoor-Views (gray)

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

  • Purpose Segmentation of surgical instruments in endoscopic video streams is essential for automated surgical scene understanding and process modeling. However, relying on fully supervised deep learning for this task is challenging because manual annotation occupies valuable time of the clinical experts. Methods We introduce a teacher–student learning approach that learns jointly from annotated simulation data and unlabeled real data to tackle the challenges in simulation-to-real unsupervised domain adaptation for endoscopic image segmentation. Results Empirical results on three datasets highlight the effectiveness of the proposed framework over current approaches for the endoscopic instrument segmentation task. Additionally, we provide analysis of major factors affecting the performance on all datasets to highlight the strengths and failure modes of our approach. Conclusions We show that our proposed approach can successfully exploit the unlabeled real endoscopic video frames and improve generalization performance over pure simulation-based training and the previous state-of-the-art. This takes us one step closer to effective segmentation of surgical instrument in the annotation scarce setting.

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics - number of accesses to the document
Author:Manish Sahu, Anirban Mukhopadhyay, Stefan ZachowORCiD
Document Type:Article
Parent Title (English):International Journal of Computer Assisted Radiology and Surgery
First Page:849
Last Page:859
Publisher:Springer Nature
Year of first publication:2021
ArXiv Id:http://arxiv.org/abs/arXiv:2103.01593
Honorary Mention: Machine Learning for Computer-Assisted Intervention (CAI) Award @IPCAI2021
Honorary Mention: Audience Award for Best Innovation @IPCAI2021
Licence (German):License LogoCreative Commons - CC BY - Namensnennung 4.0 International
Accept ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.