TY - GEN A1 - Tycowicz, Christoph von A1 - Ambellan, Felix A1 - Mukhopadhyay, Anirban A1 - Zachow, Stefan T1 - A Riemannian Statistical Shape Model using Differential Coordinates N2 - We propose a novel Riemannian framework for statistical analysis of shapes that is able to account for the nonlinearity in shape variation. By adopting a physical perspective, we introduce a differential representation that puts the local geometric variability into focus. We model these differential coordinates as elements of a Lie group thereby endowing our shape space with a non-Euclidian structure. A key advantage of our framework is that statistics in a manifold shape space become numerically tractable improving performance by several orders of magnitude over state-of-the-art. We show that our Riemannian model is well suited for the identification of intra-population variability as well as inter-population differences. In particular, we demonstrate the superiority of the proposed model in experiments on specificity and generalization ability. We further derive a statistical shape descriptor that outperforms the standard Euclidian approach in terms of shape-based classification of morphological disorders. T3 - ZIB-Report - 16-69 Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-61175 UR - https://opus4.kobv.de/opus4-zib/frontdoor/index/index/docId/6485 SN - 1438-0064 ER - TY - JOUR A1 - Sahu, Manish A1 - Mukhopadhyay, Anirban A1 - Szengel, Angelika A1 - Zachow, Stefan T1 - Addressing multi-label imbalance problem of Surgical Tool Detection using CNN JF - International Journal of Computer Assisted Radiology and Surgery N2 - Purpose: A fully automated surgical tool detection framework is proposed for endoscopic video streams. State-of-the-art surgical tool detection methods rely on supervised one-vs-all or multi-class classification techniques, completely ignoring the co-occurrence relationship of the tools and the associated class imbalance. Methods: In this paper, we formulate tool detection as a multi-label classification task where tool co-occurrences are treated as separate classes. In addition, imbalance on tool co-occurrences is analyzed and stratification techniques are employed to address the imbalance during Convolutional Neural Network (CNN) training. Moreover, temporal smoothing is introduced as an online post-processing step to enhance run time prediction. Results: Quantitative analysis is performed on the M2CAI16 tool detection dataset to highlight the importance of stratification, temporal smoothing and the overall framework for tool detection. Conclusion: The analysis on tool imbalance, backed by the empirical results indicates the need and superiority of the proposed framework over state-of-the-art techniques. Y1 - 2017 UR - https://link.springer.com/article/10.1007/s11548-017-1565-x U6 - https://doi.org/10.1007/s11548-017-1565-x N1 - Selected for final oral presentation VL - 12 IS - 6 SP - 1013 EP - 1020 PB - Springer ER - TY - JOUR A1 - von Tycowicz, Christoph A1 - Ambellan, Felix A1 - Mukhopadhyay, Anirban A1 - Zachow, Stefan T1 - An Efficient Riemannian Statistical Shape Model using Differential Coordinates JF - Medical Image Analysis N2 - We propose a novel Riemannian framework for statistical analysis of shapes that is able to account for the nonlinearity in shape variation. By adopting a physical perspective, we introduce a differential representation that puts the local geometric variability into focus. We model these differential coordinates as elements of a Lie group thereby endowing our shape space with a non-Euclidean structure. A key advantage of our framework is that statistics in a manifold shape space becomes numerically tractable improving performance by several orders of magnitude over state-of-the-art. We show that our Riemannian model is well suited for the identification of intra-population variability as well as inter-population differences. In particular, we demonstrate the superiority of the proposed model in experiments on specificity and generalization ability. We further derive a statistical shape descriptor that outperforms the standard Euclidean approach in terms of shape-based classification of morphological disorders. Y1 - 2018 U6 - https://doi.org/10.1016/j.media.2017.09.004 VL - 43 IS - 1 SP - 1 EP - 9 ER - TY - GEN A1 - Sahu, Manish A1 - Szengel, Angelika A1 - Mukhopadhyay, Anirban A1 - Zachow, Stefan T1 - Analyzing laparoscopic cholecystectomy with deep learning: automatic detection of surgical tools and phases T2 - 28th International Congress of the European Association for Endoscopic Surgery (EAES) N2 - Motivation: The ever-rising volume of patients, high maintenance cost of operating rooms and time consuming analysis of surgical skills are fundamental problems that hamper the practical training of the next generation of surgeons. The hospitals prefer to keep the surgeons busy in real operations over training young surgeons for obvious economic reasons. One fundamental need in surgical training is the reduction of the time needed by the senior surgeon to review the endoscopic procedures performed by the young surgeon while minimizing the subjective bias in evaluation. The unprecedented performance of deep learning ushers the new age of data-driven automatic analysis of surgical skills. Method: Deep learning is capable of efficiently analyzing thousands of hours of laparoscopic video footage to provide an objective assessment of surgical skills. However, the traditional end-to-end setting of deep learning (video in, skill assessment out) is not explainable. Our strategy is to utilize the surgical process modeling framework to divide the surgical process into understandable components. This provides the opportunity to employ deep learning for superior yet automatic detection and evaluation of several aspects of laparoscopic cholecystectomy such as surgical tool and phase detection. We employ ZIBNet for the detection of surgical tool presence. ZIBNet employs pre-processing based on tool usage imbalance, a transfer learned 50-layer residual network (ResNet-50) and temporal smoothing. To encode the temporal evolution of tool usage (over the entire video sequence) that relates to the surgical phases, Long Short Term Memory (LSTM) units are employed with long-term dependency. Dataset: We used CHOLEC 80 dataset that consists of 80 videos of laparoscopic cholecystectomy performed by 13 surgeons, divided equally for training and testing. In these videos, up to three different tools (among 7 types of tools) can be present in a frame. Results: The mean average precision of the detection of all tools is 93.5 ranging between 86.8 and 99.3, a significant improvement (p <0.01) over the previous state-of-the-art. We observed that less frequent tools like Scissors, Irrigator, Specimen Bag etc. are more related to phase transitions. The overall precision (recall) of the detection of all surgical phases is 79.6 (81.3). Conclusion: While this is not the end goal for surgical skill analysis, the development of such a technological platform is essential toward a data-driven objective understanding of surgical skills. In future, we plan to investigate surgeon-in-the-loop analysis and feedback for surgical skill analysis. Y1 - 2020 UR - https://academy.eaes.eu/eaes/2020/28th/298882/manish.sahu.analyzing.laparoscopic.cholecystectomy.with.deep.learning.html?f=listing%3D0%2Abrowseby%3D8%2Asortby%3D2 ER - TY - JOUR A1 - Mukhopadhyay, Anirban A1 - Bhandarkar, Suchendra T1 - Biharmonic Density Estimate - a scale space descriptor for 3D deformable surfaces JF - Pattern Analysis and Application Y1 - 2017 U6 - https://doi.org/10.1007/s10044-017-0610-2 SP - 1 EP - 13 ER - TY - CHAP A1 - Mukhopadhyay, Anirban A1 - Oksuz, Ilkay A1 - Bevilacqua, Marco A1 - Dharmakumar, Rohan A1 - Tsaftaris, Sotirios T1 - Data-Driven Feature Learning for Myocardial Segmentation of CP-BOLD MRI T2 - Functional Imaging and Modeling of the Heart N2 - Cardiac Phase-resolved Blood Oxygen-Level-Dependent (CP- BOLD) MR is capable of diagnosing an ongoing ischemia by detecting changes in myocardial intensity patterns at rest without any contrast and stress agents. Visualizing and detecting these changes require significant post-processing, including myocardial segmentation for isolating the myocardium. But, changes in myocardial intensity pattern and myocardial shape due to the heart’s motion challenge automated standard CINE MR myocardial segmentation techniques resulting in a significant drop of segmentation accuracy. We hypothesize that the main reason behind this phenomenon is the lack of discernible features. In this paper, a multi scale discriminative dictionary learning approach is proposed for supervised learning and sparse representation of the myocardium, to improve the myocardial feature selection. The technique is validated on a challenging dataset of CP-BOLD MR and standard CINE MR acquired in baseline and ischemic condition across 10 canine subjects. The proposed method significantly outperforms standard cardiac segmentation techniques, including segmentation via registration, level sets and supervised methods for myocardial segmentation. Y1 - 2015 U6 - https://doi.org/10.1007/978-3-319-20309-6_22 VL - 9126 SP - 189 EP - 197 PB - Springer ER - TY - CHAP A1 - Mukhopadhyay, Anirban A1 - Porikli, Fatih A1 - Bhandarkar, Suchendra T1 - Detection and Characterization of Intrinsic Symmetry of 3D Shapes T2 - Proceedings of IEEE International Conference on Pattern Recognition N2 - A comprehensive framework for detection and characterization of partial intrinsic symmetry over 3D shapes is proposed. To identify prominent symmetric regions which overlap in space and vary in form, the proposed framework is decoupled into a Correspondence Space Voting (CSV) procedure followed by a Transformation Space Mapping (TSM) procedure. In the CSV procedure, significant symmetries are first detected by identifying surface point pairs on the input shape that exhibit local similarity in terms of their intrinsic geometry while simultaneously maintaining an intrinsic distance structure at a global level. To allow detection of potentially overlapping symmetric shape regions, a global intrinsic distance-based voting scheme is employed to ensure the inclusion of only those point pairs that exhibit significant intrinsic symmetry. In the TSM procedure, the Functional Map framework is employed to generate the final map of symmetries between point pairs. The TSM procedure ensures the retrieval of the underlying dense correspondence map throughout the 3D shape that follows a particular symmetry. The TSM procedure is also shown to result in the formulation of a metric symmetry space where each point in the space represents a specific symmetry transformation and the distance between points represents the complexity between the corresponding transformations. Experimental results show that the proposed framework can successfully analyze complex 3D shapes that possess rich symmetries. Y1 - 2016 ER - TY - CHAP A1 - Oksuz, Ilkay A1 - Mukhopadhyay, Anirban A1 - Bevilacqua, Marco A1 - Dharmakumar, Rohan A1 - Tsaftaris, Sotirios T1 - Dictionary Learning Based Image Descriptor for Myocardial Registration of CP-BOLD MR T2 - Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015 N2 - Cardiac Phase-resolved Blood Oxygen-Level-Dependent (CP- BOLD) MRI is a new contrast agent- and stress-free imaging technique for the assessment of myocardial ischemia at rest. The precise registration among the cardiac phases in this cine type acquisition is essential for automating the analysis of images of this technique, since it can potentially lead to better specificity of ischemia detection. However, inconsistency in myocardial intensity patterns and the changes in myocardial shape due to the heart’s motion lead to low registration performance for state- of-the-art methods. This low accuracy can be explained by the lack of distinguishable features in CP-BOLD and inappropriate metric defini- tions in current intensity-based registration frameworks. In this paper, the sparse representations, which are defined by a discriminative dictionary learning approach for source and target images, are used to improve myocardial registration. This method combines appearance with Gabor and HOG features in a dictionary learning framework to sparsely represent features in a low dimensional space. The sum of squared differences of these distinctive sparse representations are used to define a similarity term in the registration framework. The proposed descriptor is validated on a challenging dataset of CP-BOLD MR and standard CINE MR acquired in baseline and ischemic condition across 10 canines. Y1 - 2015 U6 - https://doi.org/10.1007/978-3-319-24571-3_25 VL - 9350 SP - 205 EP - 213 PB - Springer ER - TY - CHAP A1 - Sahu, Manish A1 - Strömsdörfer, Ronja A1 - Mukhopadhyay, Anirban A1 - Zachow, Stefan T1 - Endo-Sim2Real: Consistency learning-based domain adaptation for instrument segmentation T2 - Proc. Medical Image Computing and Computer Assisted Intervention (MICCAI), Part III N2 - Surgical tool segmentation in endoscopic videos is an important component of computer assisted interventions systems. Recent success of image-based solutions using fully-supervised deep learning approaches can be attributed to the collection of big labeled datasets. However, the annotation of a big dataset of real videos can be prohibitively expensive and time consuming. Computer simulations could alleviate the manual labeling problem, however, models trained on simulated data do not generalize to real data. This work proposes a consistency-based framework for joint learning of simulated and real (unlabeled) endoscopic data to bridge this performance generalization issue. Empirical results on two data sets (15 videos of the Cholec80 and EndoVis'15 dataset) highlight the effectiveness of the proposed Endo-Sim2Real method for instrument segmentation. We compare the segmentation of the proposed approach with state-of-the-art solutions and show that our method improves segmentation both in terms of quality and quantity. Y1 - 2020 U6 - https://doi.org/https://doi.org/10.1007/978-3-030-59716-0_75 VL - 12263 PB - Springer Nature ER - TY - GEN A1 - Mukhopadhyay, Anirban A1 - Kumar, Arun A1 - Bhandarkar, Suchendra T1 - Joint Geometric Graph Embedding for Partial Shape Matching in Images T2 - IEEE Winter Conference on Applications of Computer Vision N2 - A novel multi-criteria optimization framework for matching of partially visible shapes in multiple images using joint geometric graph embedding is proposed. The proposed framework achieves matching of partial shapes in images that exhibit extreme variations in scale, orientation, viewpoint and illumination and also instances of occlusion; conditions which render impractical the use of global contour-based descriptors or local pixel-level features for shape matching. The proposed technique is based on optimization of the embedding distances of geometric features obtained from the eigenspectrum of the joint image graph, coupled with regularization over values of the mean pixel intensity or histogram of oriented gradients. It is shown to obtain successfully the correspondences denoting partial shape similarities as well as correspondences between feature points in the images. A new benchmark dataset is proposed which contains disparate image pairs with extremely challenging variations in viewing conditions when compared to an existing dataset [18]. The proposed technique is shown to significantly outperform several state-of-the-art partial shape matching techniques on both datasets. Y1 - 2016 SP - 1 EP - 9 PB - IEEE ET - IEEE Winter Conference on Applications of Computer Vision (WACV) ER -