Refine
Document Type
- Article (4)
- In Proceedings (2)
Language
- English (6)
Has Fulltext
- no (6)
Is part of the Bibliography
- no (6)
Institute
We present nonlinear formulations to Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) faithfully exploiting the isometric, conformal and equiareal deformation models. Existing work uses relaxations such as inextensibility or requires knowing the optic flow field around the correspondences, an impractical assumption. In contrast, the proposed formulations only require point correspondences and resolve all ambiguities using the notions of maximal depth and maximal isometry heuristics. We propose solution methods using Semi-Definite Programming (SDP) for all formulations. We show that straightforward SDP models conflict with the usual maximal depth heuristic and propose an adapted opposite-depth parameterisation demonstrating a lesser relaxation gap. Experimental results on many real-world benchmark datasets demonstrate superior accuracy over existing methods.
Camera pose is a very natural concept in 3D vision in the rigid setting. It is however much more difficult
to work with in deformable settings. Consequently, numerous deformable reconstruction methods simply ignore
camera pose. We analyse the concept of pose in deformable settings and prove that it is unconstrained with the
existing formulations, properly justifying the existing pose-less methods reconstructing structure only. We explain
this result intuitively by the impossibility to define an intrinsic coordinate frame to a general deforming object. The
proposed analysis uses the isometric deformation model and extends to the weaker models including conformality
and equiareality. We propose a novel prior to rescue camera pose estimation in deformable settings, which attributes
the deforming object’s dominant rigid-body motion to the camera. We show that adding this prior to any existing
formulation fully constrains camera pose and leads to elegant two-step solution methods, involving deformable
structure reconstruction using a base method in the first step, and absolute orientation or Procrustes analysis in
the second step. We derive the proposed approach for the template-based and template-less settings, respectively
implemented using Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) as base methods,
and validate them experimentally, showing that the computed pose is qualitatively and quantitatively plausible.
This article presents a new method for non-rigidly registering a 3D shape to 2D keypoints observed by a constellation of multiple cameras. Non-rigid registration of a 3D shape to observed 2D keypoints, i.e., Shape-from-Template (SfT), has been widely studied using single images, but SfT with information from multiple-cameras jointly opens new directions for extending the scope of known use-cases such as 3D shape registration in medical imaging and registration from hand-held cameras, to name a few. We represent such multi-camera setup with the generalised camera model; therefore any collection of perspective or orthographic cameras observing any deforming object can be registered. We propose multiple approaches for such SfT: the first approach where the corresponded keypoints lie on a direction vector from a known 3D point in space, the second approach where the corresponded keypoints lie on a direction vector from an unknown 3D point in space but with known orientation w.r.t some local reference frame, and a third approach where, apart from correspondences, the silhouette of the imaged object is also known. Together, these form the first set of solutions to the SfT problem with generalised cameras. The key idea behind SfT with generalised camera is the improved reconstruction accuracy from estimating deformed shape while utilising the additional information from the mutual constraints between multiple views of a deformed object. The correspondence-based approaches are solved with convex programming while the silhouette-based approach is an iterative refinement of the results from the convex solutions. We demonstrate the accuracy of our proposed methods on many synthetic and real data.
Uncertainty estimation and probabilistic skull shape reconstruction using bayesian neural networks
(2026)
3D shape reconstruction is an active area of research and a fundamental problem in computer vision, with growing applications in the medical domain, where it enables the recovery of missing or fine anatomical structures. Numerous approaches have been proposed for medical shape reconstruction, emphasizing accurate and anatomically plausible reconstructions. However, 3D shape reconstruction remains an inherently ill-posed problem, meaning (1) both neural network-based methods and conventional shape modeling approaches naturally introduce uncertainty in their predictions, and (2) multiple anatomically plausible reconstructions exist for a given partial or low-resolution input. While the uncertainty aspects have been widely explored in general computer vision, they remain relatively under-explored in the context of medical shape reconstruction. In this paper, we developed a 3D Bayesian U-Net and investigated its use for uncertainty estimation and probabilistic reconstructions across three key tasks: cranial reconstruction, facial reconstruction, and skull shape super-resolution. Our findings show that the Bayesian model is able to produce a range of anatomically plausible skull reconstructions while capturing natural skull variations arising from the learned weight uncertainty. Notably, these variations are primarily expressed through differences in bone thickness, which aligns with anatomical expectations, particularly relevant in real-world applications like cranial implant design. Additionally, we propose a principled framework to study the relationship between weight uncertainty and reconstruction uncertainty by analyzing the learned posterior distribution of the weights, demonstrating that our Bayesian U-Net achieves comparable reconstruction performance to a deterministic U-Net baseline while providing reliable uncertainty estimates. Our study also reveals a clear cross-task uncertainty pattern, where tasks with stronger structural constraints, like super-resolution, yield lower predictive uncertainty, while less constrained tasks, like facial reconstruction, result in higher uncertainty. Refer to the project page for more visual results https://git.zib.de/jli/uncertainty-aware-skull-reconstruction/.
We solve the problem of determining the pose of known shapes in R^3 from their unoccluded silhouettes. The pose is determined up to global optimality using a simple yet under-explored property of the area-of-silhouette: its continuity w.r.t trajectories in the rotation space. The proposed method utilises pre-computed silhouette-signatures, modelled as a response surface of the area-of-silhouettes. Querying this silhouette-signature response surface for pose estimation leads to a strong branching of the rotation search space, making resolution-guided candidate search feasible. Additionally, we utilise the aspect ratio of 2D ellipses fitted to projected silhouettes as an auxiliary global shape signature to accelerate the pose search. This combined strategy forms the first method to efficiently estimate globally optimal pose from just the silhouettes, without being guided by correspondences, for any shape, irrespective of its convexity and genus. We validate our method on synthetic and real examples, demonstrating significantly improved accuracy against comparable approaches.
Reconstructing the surfaces of deformable objects from correspondences between a 3D template and a 2D image is well studied under Shape-from-Template (SfT) methods; however, existing approaches break down when topological changes accompany the deformation. We propose a principled extension of SfT that enables reconstruction in the presence of such changes. Our approach is initialized with a classical SfT solution and iteratively adapts the template by partitioning its spatial domain so as to minimize an energy functional that jointly encodes physical plausibility and reprojection consistency. We demonstrate that the method robustly captures a wide range of practically relevant topological events including tears and cuts on bounded 2D surfaces, thereby establishing the first general framework for topological-change-aware SfT. Experiments on both synthetic and real data confirm that our approach consistently outperforms baseline methods.
