Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation

  • Real-time computational speed and a high degree of precision are requirements for computer-assisted interventions. Applying a segmentation network to a medical video processing task can introduce significant inter-frame prediction noise. Existing approaches can reduce inconsistencies by including temporal information but often impose requirements on the architecture or dataset. This paper proposes a method to include temporal information in any segmentation model and, thus, a technique to improve video segmentation performance without alterations during training or additional labeling. With Motion-Corrected Moving Average, we refine the exponential moving average between the current and previous predictions. Using optical flow to estimate the movement between consecutive frames, we can shift the prior term in the moving-average calculation to align with the geometry of the current frame. The optical flow calculation does not require the output of the model and can therefore be performed in parallel, leading to no significant runtime penalty for our approach. We evaluate our approach on two publicly available segmentation datasets and two proprietary endoscopic datasets and show improvements over a baseline approach.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Author:Robert Mendel, Tobias Rueckert, Dirk WilhelmORCiD, Daniel RueckertORCiD, Christoph PalmORCiD
ArXiv Id:
Document Type:Preprint
Year of first Publication:2024
Release Date:2024/03/22
GND Keyword:Deep Learning; Video; Segmentation
Institutes:Fakultät Informatik und Mathematik
Regensburg Center of Health Sciences and Technology - RCHST
Fakultät Informatik und Mathematik / Regensburg Medical Image Computing (ReMIC)
Kostenträger (Forschungsprojekt, Labor, Einrichtung etc.):2027701
research focus:Lebenswissenschaften und Ethik
Licence (German):Creative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International