Enhancing generalization in zero-shot multi-label endoscopic instrument classification
- Purpose
Recognizing previously unseen classes with neural networks is a significant challenge due to their limited generalization capabilities. This issue is particularly critical in safety-critical domains such as medical applications, where accurate classification is essential for reliability and patient safety. Zero-shot learning methods address this challenge by utilizing additional semanticPurpose
Recognizing previously unseen classes with neural networks is a significant challenge due to their limited generalization capabilities. This issue is particularly critical in safety-critical domains such as medical applications, where accurate classification is essential for reliability and patient safety. Zero-shot learning methods address this challenge by utilizing additional semantic data, with their performance relying heavily on the quality of the generated embeddings.
Methods
This work investigates the use of full descriptive sentences, generated by a Sentence-BERT model, as class representations, compared to simpler category-based word embeddings derived from a BERT model. Additionally, the impact of z-score normalization as a post-processing step on these embeddings is explored. The proposed approach is evaluated on a multi-label generalized zero-shot learning task, focusing on the recognition of surgical instruments in endoscopic images from minimally invasive cholecystectomies.
Results
The results demonstrate that combining sentence embeddings and z-score normalization significantly improves model performance. For unseen classes, the AUROC improves from 43.9% to 64.9%, and the multi-label accuracy from 26.1% to 79.5%. Overall performance measured across both seen and unseen classes improves from 49.3% to 64.9% in AUROC and from 37.3% to 65.1% in multi-label accuracy, highlighting the effectiveness of our approach.
Conclusion
These findings demonstrate that sentence embeddings and z-score normalization can substantially enhance the generalization performance of zero-shot learning models. However, as the study is based on a single dataset, future work should validate the method across diverse datasets and application domains to establish its robustness and broader applicability.…


| Author: | Raphaela MaerklORCiD, Tobias RueckertORCiD, David RauberORCiD, Max GutbrodORCiD, Danilo Weber NunesORCiD, Christoph PalmORCiD |
|---|---|
| URN: | urn:nbn:de:bvb:898-opus4-85674 |
| DOI: | https://doi.org/10.1007/s11548-025-03439-5 |
| Parent Title (English): | International Journal of Computer Assisted Radiology and Surgery |
| Publisher: | Springer Nature |
| Document Type: | Article |
| Language: | English |
| Year of first Publication: | 2025 |
| Publishing Institution: | Ostbayerische Technische Hochschule Regensburg |
| Release Date: | 2025/11/11 |
| Tag: | Generalized zero-shot learning; Multi-label classification; Sentence embeddings; Surgical instruments; Z-score normalization |
| Volume: | 20 |
| First Page: | 1577 |
| Last Page: | 1587 |
| Note: | Corresponding author der OTH Regensburg: Raphaela Maerkl |
| Institutes: | Fakultät Informatik und Mathematik |
| Regensburg Center of Biomedical Engineering - RCBE | |
| Regensburg Center of Health Sciences and Technology - RCHST | |
| Fakultät Informatik und Mathematik / Labor Regensburg Medical Image Computing (ReMIC) | |
| Begutachtungsstatus: | peer-reviewed |
| Open Access Publication channel: | Hybrid Open Access - OA-Veröffentlichung in einer Subskriptionszeitschrift/-medium |
| Corresponding author der OTH Regensburg | |
| Funding: | DEAL Springer Nature |
| research focus: | Gesundheit und Soziales |
| Licence (German): | Creative Commons - CC BY - Namensnennung 4.0 International |



