Refine
Document Type
- Article (15)
- In Proceedings (11)
- Software (2)
- Book chapter (1)
- Research data (1)
Language
- English (30)
Has Fulltext
- no (30) (remove)
Keywords
We propose two graph neural network layers for graphs with features in a Riemannian manifold. First, based on a manifold-valued graph diffusion equation, we construct a diffusion layer that can be applied to an arbitrary number of nodes and graph connectivity patterns. Second, we model a tangent multilayer perceptron by transferring ideas from the vector neuron framework to our general setting. Both layers are equivariant with respect to node permutations and isometries of the feature manifold. These properties have been shown to lead to a beneficial inductive bias in many deep learning tasks. Numerical examples on synthetic data as well as on triangle meshes of the right hippocampus to classify Alzheimer's disease demonstrate the very good performance of our layers.
For decades, de Casteljau's algorithm has been used as a fundamental building block in curve and surface design and has found a wide range of applications in fields such as scientific computing, and discrete geometry to name but a few. With increasing interest in nonlinear data science, its constructive approach has been shown to provide a principled way to generalize parametric smooth curves to manifolds. These curves have found remarkable new applications in the analysis of parameter-dependent, geometric data. This article provides a survey of the recent theoretical developments in this exciting area as well as its applications in fields such as geometric morphometrics and longitudinal data analysis in medicine, archaeology, and meteorology.
Task-adapted image reconstruction methods using end-to-end trainable neural networks (NNs) have been proposed to optimize reconstruction for subsequent processing tasks, such as segmentation. However, their training typically requires considerable hardware resources and thus, only relatively simple building blocks, e.g. U-Nets, are typically used, which, albeit powerful, do not integrate model-specific knowledge.
In this work, we extend an end-to-end trainable task-adapted image reconstruction method for a clinically realistic reconstruction and segmentation problem of bone and cartilage in 3D knee MRI by incorporating statistical shape models (SSMs). The SSMs model the prior information and help to regularize the segmentation maps as a final post-processing step.
We compare the proposed method to a state-of-the-art (SOTA) simultaneous multitask learning approach for image reconstruction and segmentation (MTL) and to a complex SSMs-informed segmentation pipeline (SIS).
Our experiments show that the combination of joint end-to-end training and SSMs to further regularize the segmentation maps obtained by MTL highly improves the results, especially in terms of mean and maximal surface errors.
In particular, we achieve the segmentation quality of SIS and, at the same time, a substantial model reduction that yields a five-fold decimation in model parameters and a computational speedup of an order of magnitude.
Remarkably, even for undersampling factors of up to R=8, the obtained segmentation maps are of comparable quality to those obtained by SIS from ground-truth images.
Gesture recognition is a tool to enable novel interactions with different techniques and
applications, like Mixed Reality and Virtual Reality environments. With all the recent
advancements in gesture recognition from skeletal data, it is still unclear how well state-of-
the-art techniques perform in a scenario using precise motions with two hands. This
paper presents the results of the SHREC 2024 contest organized to evaluate methods
for their recognition of highly similar hand motions using the skeletal spatial coordinate
data of both hands. The task is the recognition of 7 motion classes given their spatial
coordinates in a frame-by-frame motion. The skeletal data has been captured using
a Vicon system and pre-processed into a coordinate system using Blender and Vicon
Shogun Post. We created a small, novel dataset with a high variety of durations in
frames. This paper shows the results of the contest, showing the techniques created
by the 5 research groups on this challenging task and comparing them to our baseline
method.
Predicting the future development of an anatomical shape from a single baseline observation is a challenging task. But it can be essential for clinical decision-making. Research has shown that it should be tackled in curved shape spaces, as (e.g., disease-related) shape changes frequently expose nonlinear characteristics. We thus propose a novel prediction method that encodes the whole shape in a Riemannian shape space. It then learns a simple prediction technique founded on hierarchical statistical modeling of longitudinal training data. When applied to predict the future development of the shape of the right hippocampus under Alzheimer's disease and to human body motion, it outperforms deep learning-supported variants as well as state-of-the-art.
The fact that the physical shapes of man-made objects are subject to overlapping influences—such as technological, economic, geographic, and stylistic progressions—holds great information potential. On the other hand, it is also a major analytical challenge to uncover these overlapping trends and to disentagle them in an unbiased way. This paper explores a novel mathematical approach to extract archaeological insights from ensembles of similar artifact shapes. We show that by considering all shape information in a find collection, it is possible to identify shape patterns that would be difficult to discern by considering the artifacts individually or by classifying shapes into predefined archaeological types and analyzing the associated distinguishing characteristics. Recently, series of high-resolution digital representations of artifacts have become available. Such data sets enable the application of extremely sensitive and flexible methods of shape analysis. We explore this potential on a set of 3D models of ancient Greek and Roman sundials, with the aim of providing alternatives to the traditional archaeological method of “trend extraction by ordination” (typology). In the proposed approach, each 3D shape is represented as a point in a shape space—a high-dimensional, curved, non-Euclidean space. Proper consideration of its mathematical properties reduces bias in data analysis and thus improves analytical power. By performing regression in shape space, we find that for Roman sundials, the bend of the shadow-receiving surface of the sundials changes with the latitude of the location. This suggests that, apart from the inscribed hour lines, also a sundial’s shape was adjusted to the place of installation. As an example of more advanced inference, we use the identified trend to infer the latitude at which a sundial, whose location of installation is unknown, was placed. We also derive a novel method for differentiated morphological trend assertion, building upon and extending the theory of geometric statistics and shape analysis. Specifically, we present a regression-based method for statistical normalization of shapes that serves as a means of disentangling parameter-dependent effects (trends) and unexplained variability. In addition, we show that this approach is robust to noise in the digital reconstructions of the artifact shapes.
We propose a generic spatiotemporal framework to analyze manifold-valued measurements, which allows for employing an intrinsic and computationally efficient Riemannian hierarchical model. Particularly, utilizing regression, we represent discrete trajectories in a Riemannian manifold by composite Bézier splines, propose a natural metric induced by the Sasaki metric to compare the trajectories, and estimate average trajectories as group-wise trends. We evaluate our framework in comparison to state-of-the-art methods within qualitative and quantitative experiments on hurricane tracks. Notably, our results demonstrate the superiority of spline-based approaches for an intensity classification of the tracks.
A Soft-Correspondence Approach to Shape-based Disease Grading with Graph Convolutional Networks
(2022)
Shape analysis provides principled means for understanding anatomical structures from medical images. The underlying notions of shape spaces, however, come with strict assumptions prohibiting the analysis of incomplete and/or topologically varying shapes. This work aims to alleviate these limitations by adapting the concept of soft correspondences. In particular, we present a graph-based learning approach for morphometric classification of disease states that is based on a generalized notion of shape correspondences in terms of functional maps. We demonstrate the performance of the derived classifier on the open-access ADNI database for differentiating normal controls and subjects with Alzheimer’s disease. Notably, our experiment shows that our approach can improve over state-of-the-art from geometric deep learning.
Large longitudinal studies provide lots of valuable information, especially in medical applications. A problem which must be taken care of in order to utilize their full potential is that of correlation between intra-subject measurements taken at different times. For data in Euclidean space this can be done with hierarchical models, that is, models that consider intra-subject and between-subject variability in two different stages. Nevertheless, data from medical studies often takes values in nonlinear manifolds. Here, as a first step, geodesic hierarchical models have been developed that generalize the linear ansatz by assuming that time-induced intra-subject variations occur along a generalized straight line in the manifold. However, this is often not the case (e.g., periodic motion or processes with saturation). We propose a hierarchical model for manifold-valued data that extends this to include trends along higher-order curves, namely Bézier splines in the manifold.
To this end, we present a principled way of comparing shape trends in terms of a functional-based Riemannian metric.
Remarkably, this metric allows efficient, yet simple computations by virtue of a variational time discretization requiring only the solution of regression problems.
We validate our model on longitudinal data from the osteoarthritis initiative, including classification of disease progression.
In many applications, geodesic hierarchical models are adequate for the study of temporal observations.
We employ such a model derived for manifold-valued data to Kendall's shape space.
In particular, instead of the Sasaki metric, we adapt a functional-based metric, which increases the computational efficiency and does not require the implementation of the curvature tensor. We propose the corresponding variational time discretization of geodesics
and employ the approach for longitudinal analysis of 2D rat skulls shapes as well as 3D shapes derived from an imaging study on osteoarthritis. Particularly, we perform hypothesis test and estimate the mean trends.
Data sets sampled in Lie groups are widespread, and as with multivariate data, it is important for many applications to assess the differences between the sets in terms of their distributions. Indices for this task are usually derived by considering the Lie group as a Riemannian manifold. Then, however, compatibility with the group operation is guaranteed only if a bi-invariant metric exists, which is not the case for most non-compact and non-commutative groups. We show here that if one considers an affine connection structure instead, one obtains bi-invariant generalizations of well-known dissimilarity measures: a Hotelling $T^2$ statistic, Bhattacharyya distance and Hellinger distance. Each of the dissimilarity measures matches its multivariate counterpart for Euclidean data and is translation-invariant, so that biases, e.g., through an arbitrary choice of reference, are avoided. We further derive non-parametric two-sample tests that are bi-invariant and consistent. We demonstrate the potential of these dissimilarity measures by performing group tests on data of knee configurations and epidemiological shape data. Significant differences are revealed in both cases.
3D shapes provide substantially more information than 2D images. However, the acquisition of 3D shapes is sometimes very difficult or even impossible in comparison with acquiring 2D images, making it necessary to derive the 3D shape from 2D images. Although this is, in general, a mathematically ill-posed problem, it might be solved by constraining the problem formulation using prior information. Here, we present a new approach based on Kendall’s shape space to reconstruct 3D shapes from single monocular 2D images. The work is motivated by an application to study the feeding behavior of the basking shark, an endangered species whose massive size and mobility render 3D shape data nearly impossible to obtain, hampering understanding of their feeding behaviors and ecology. 2D images of these animals in feeding position, however, are readily available. We compare our approach with state-of-the-art shape-based approaches both on human stick models and on shark head skeletons. Using a small set of training shapes, we show that the Kendall shape space approach is substantially more robust than previous methods and always results in plausible shapes. This is essential for the motivating application in which specimens are rare and therefore only few training shapes are available.
We present a novel approach for nonlinear statistical shape modeling that is invariant under Euclidean motion and thus alignment-free. By analyzing metric distortion and curvature of shapes as elements of Lie groups in a consistent Riemannian setting, we construct a framework that reliably handles large deformations. Due to the explicit character of Lie group operations, our non-Euclidean method is very efficient allowing for fast and numerically robust processing. This facilitates Riemannian analysis of large shape populations accessible through longitudinal and multi-site imaging studies providing increased statistical power. Additionally, as planar configurations form a submanifold in shape space, our representation allows for effective estimation of quasi-isometric surfaces flattenings. We evaluate the performance of our model w.r.t. shape-based classification of hippocampus and femur malformations due to Alzheimer's disease and osteoarthritis, respectively. In particular, we achieve state-of-the-art accuracies outperforming the standard Euclidean as well as a recent nonlinear approach especially in presence of sparse training data. To provide insight into the model's ability of capturing biological shape variability, we carry out an analysis of specificity and generalization ability.
Three-dimensional medical imaging enables detailed understanding of osteoarthritis structural status. However, there remains a vast need for automatic, thus, reader-independent measures that provide reliable assessment of subject-specific clinical outcomes. To this end, we derive a consistent generalization of the recently proposed B-score to Riemannian shape spaces. We further present an algorithmic treatment yielding simple, yet efficient computations allowing for analysis of large shape populations with several thousand samples. Our intrinsic formulation exhibits improved discrimination ability over its Euclidean counterpart, which we demonstrate for predictive validity on assessing risks of total knee replacement. This result highlights the potential of the geodesic B-score to enable improved personalized assessment and stratification for interventions.
Morphomatics is an open-source Python library for (statistical) shape analysis developed within the geometric data analysis and processing research group at Zuse Institute Berlin. It contains prototype implementations of intrinsic manifold-based methods that are highly consistent and avoid the influence of unwanted effects such as bias due to arbitrary choices of coordinates.
This paper presents the methods and results of the SHREC’21 contest on a dataset of cultural heritage (CH) objects. We present a dataset of 938 scanned models that have varied geometry and artistic styles. For the competition, we propose two challenges: the retrieval-by-shape challenge and the retrieval-by-culture challenge. The former aims at evaluating the ability of retrieval methods to discriminate cultural heritage objects by overall shape. The latter focuses on assessing the effectiveness of retrieving objects from the same culture. Both challenges constitute a suitable scenario to evaluate modern shape retrieval methods in a CH domain. Ten groups participated in the contest: thirty runs were submitted for the retrieval-by-shape task, and twenty-six runs were submitted for the retrieval-by-culture challenge. The results show a predominance of learning methods on image-based multi-view representations to characterize 3D objects. Nevertheless, the problem presented in our challenges is far from being solved. We also identify the potential paths for further improvements and give insights into the future directions of research.