Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation
- Real-time computational speed and a high degree of precision are requirements for computer-assisted interventions. Applying a segmentation network to a medical video processing task can introduce significant inter-frame prediction noise. Existing approaches can reduce inconsistencies by including temporal information but often impose requirements on the architecture or dataset. This paper proposes a method to include temporal information in any segmentation model and, thus, a technique to improve video segmentation performance without alterations during training or additional labeling. With Motion-Corrected Moving Average, we refine the exponential moving average between the current and previous predictions. Using optical flow to estimate the movement between consecutive frames, we can shift the prior term in the moving-average calculation to align with the geometry of the current frame. The optical flow calculation does not require the output of the model and can therefore be performed in parallel, leading to no significant runtime penalty for our approach. We evaluate our approach on two publicly available segmentation datasets and two proprietary endoscopic datasets and show improvements over a baseline approach.
Author: | Robert MendelORCiD, Tobias Rueckert, Dirk WilhelmORCiD, Daniel RueckertORCiD, Christoph PalmORCiDGND |
---|---|
DOI: | https://doi.org/10.48550/arXiv.2403.03120 |
ArXiv Id: | http://arxiv.org/abs/arXiv:2403.03120 |
Document Type: | Preprint |
Language: | English |
Year of first Publication: | 2024 |
Release Date: | 2024/03/22 |
GND Keyword: | Deep Learning; Video; Segmentation |
Pagenumber: | 9 |
Institutes: | Fakultät Informatik und Mathematik |
Regensburg Center of Health Sciences and Technology - RCHST | |
Fakultät Informatik und Mathematik / Regensburg Medical Image Computing (ReMIC) | |
Kostenträger (Forschungsprojekt, Labor, Einrichtung etc.): | 2027701 |
research focus: | Lebenswissenschaften und Ethik |
Licence (German): | Creative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International |