Enhancing generalization in zero-shot multi-label endoscopic instrument classification

  • Purpose Recognizing previously unseen classes with neural networks is a significant challenge due to their limited generalization capabilities. This issue is particularly critical in safety-critical domains such as medical applications, where accurate classification is essential for reliability and patient safety. Zero-shot learning methods address this challenge by utilizing additional semanticPurpose Recognizing previously unseen classes with neural networks is a significant challenge due to their limited generalization capabilities. This issue is particularly critical in safety-critical domains such as medical applications, where accurate classification is essential for reliability and patient safety. Zero-shot learning methods address this challenge by utilizing additional semantic data, with their performance relying heavily on the quality of the generated embeddings. Methods This work investigates the use of full descriptive sentences, generated by a Sentence-BERT model, as class representations, compared to simpler category-based word embeddings derived from a BERT model. Additionally, the impact of z-score normalization as a post-processing step on these embeddings is explored. The proposed approach is evaluated on a multi-label generalized zero-shot learning task, focusing on the recognition of surgical instruments in endoscopic images from minimally invasive cholecystectomies. Results The results demonstrate that combining sentence embeddings and z-score normalization significantly improves model performance. For unseen classes, the AUROC improves from 43.9% to 64.9%, and the multi-label accuracy from 26.1% to 79.5%. Overall performance measured across both seen and unseen classes improves from 49.3% to 64.9% in AUROC and from 37.3% to 65.1% in multi-label accuracy, highlighting the effectiveness of our approach. Conclusion These findings demonstrate that sentence embeddings and z-score normalization can substantially enhance the generalization performance of zero-shot learning models. However, as the study is based on a single dataset, future work should validate the method across diverse datasets and application domains to establish its robustness and broader applicability.show moreshow less

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Raphaela MaerklORCiD, Tobias RueckertORCiD, David RauberORCiD, Max GutbrodORCiD, Danilo Weber NunesORCiD, Christoph PalmORCiD
URN:urn:nbn:de:bvb:898-opus4-85674
DOI:https://doi.org/10.1007/s11548-025-03439-5
Parent Title (English):International Journal of Computer Assisted Radiology and Surgery
Publisher:Springer Nature
Document Type:Article
Language:English
Year of first Publication:2025
Publishing Institution:Ostbayerische Technische Hochschule Regensburg
Release Date:2025/11/11
Tag:Generalized zero-shot learning; Multi-label classification; Sentence embeddings; Surgical instruments; Z-score normalization
Volume:20
First Page:1577
Last Page:1587
Note:
Corresponding author der OTH Regensburg: Raphaela Maerkl
Institutes:Fakultät Informatik und Mathematik
Regensburg Center of Biomedical Engineering - RCBE
Regensburg Center of Health Sciences and Technology - RCHST
Fakultät Informatik und Mathematik / Labor Regensburg Medical Image Computing (ReMIC)
Begutachtungsstatus:peer-reviewed
Open Access Publication channel:Hybrid Open Access - OA-Veröffentlichung in einer Subskriptionszeitschrift/-medium
Corresponding author der OTH Regensburg
Funding:DEAL Springer Nature
research focus:Gesundheit und Soziales
Licence (German):Creative Commons - CC BY - Namensnennung 4.0 International
Einverstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.