Multidimensional image morphing-fast image-based rendering of open 3D and VR environments

  • In recent years, the demand for interactive photorealistic three-dimensional (3D) environments has increased in various fields, including architecture, engineering, and entertainment. However, achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality (VR) remains challenging. Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering (IBR), which enables the exploration of spacious open environments in 3D and VR. Therefore, we introduce multimorphing, a novel rendering method based on the spatial data structure of 2D image patches, called the image graph. Using this approach, novel views can be rendered with up to six degrees of freedom using only a sparse set of views. The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information, and all relevant data for the output are extracted from the local morphing cells of the image graph. The detection of parallax image regions duringIn recent years, the demand for interactive photorealistic three-dimensional (3D) environments has increased in various fields, including architecture, engineering, and entertainment. However, achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality (VR) remains challenging. Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering (IBR), which enables the exploration of spacious open environments in 3D and VR. Therefore, we introduce multimorphing, a novel rendering method based on the spatial data structure of 2D image patches, called the image graph. Using this approach, novel views can be rendered with up to six degrees of freedom using only a sparse set of views. The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information, and all relevant data for the output are extracted from the local morphing cells of the image graph. The detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time. In addition, a GPU-based solution was presented to resolve exposure inconsistencies within a dataset, enabling seamless transitions of brightness when moving between areas with varying light intensities. Results Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high "VR-compatible" frame rates, even on mid-range and legacy hardware, respectively. While achieving adequate visual quality even for sparse datasets, it outperforms other IBR and current neural rendering approaches. Conclusions Using the correspondence-based decomposition of input images into morphing cells of 2D image patches, multidimensional image morphing provides high-performance novel view generation, supporting open 3D and VR environments. Nevertheless, the handling of morphing artifacts in the parallax image regions remains a topic for future research.show moreshow less

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Simon SeibtORCiD, Thomas Chang, Marc Erich LatoschikORCiD, Bastian Kuth, Bartosz von Rymon LipinskiORCiD
DOI:https://doi.org/10.1016/j.vrih.2023.06.007
Parent Title (English):Virtual Reality & Intelligent Hardware
Document Type:conference proceeding (article)
Language:English
Date of first Publication:2024/02/26
Reviewed:Begutachtet/Reviewed
Release Date:2025/05/26
Tag:Computer graphics; 3D real-time rendering; Computer vision; Image morphing; Virtual reality
Volume:7
Issue:2
Pagenumber:155-172
Konferenzangabe:Multidimensional image morphing-fast image-based rendering of open 3D and VR environments. Virtual Reality & Intelligent Hardware
institutes:Fakultät Informatik
Research Themes:Digitalisierung & Künstliche Intelligenz
Licence (German):Creative Commons - CC BY - Namensnennung 4.0 International
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.