The search result changed since you submitted your search request. Documents might be displayed in a different sort order.
  • search hit 37 of 128
Back to Result List

Partitioning Point Clouds and Meshes with Deep Learning

  • The overall objective of this dissertation is to enable a more efficient and effective point cloud and mesh partition for artists and 3D application developers. In this dissertation, 3D scans are assumed as the source data material of the 3D application development, reducing the manual and time-consuming modelling of virtual objects. Furthermore, the scanned data is assumed to be processed to a point cloud and reconstructed to a polygon mesh. The mesh has to be partitioned into the objects of interest to design specific interactions with a game engine. Interviews revealed that the partition is manually conducted on a mesh with a 3D manipulation software, which is time-consuming. The partition creation should be automated to increase efficiency and effectiveness. Freely available point cloud and mesh partition algorithms require an expert with appropriate programming skills and field knowledge, which makes them difficult to use. More precisely, the algorithms cannot be used in existing workflows as they are not implemented in a common graphical 3D manipulation software. Beneath these problems, the partition automation should work on real-world data and have a low runtime to raise efficiency. Different sub-research objectives were formulated from these problems and requirements, leading to novel approaches in the domains of: (a) sequential partition creation with deep reinforcement and imitation learning, (b) episodic partition creation with graph neural networks, (c) match-based reward calculation and (d) synthetic scene generation. One sub-research objective is the replacement of a human expert with an agent. In this context, a novel deep reinforcement learning (DRL) partition framework is presented. Experiments were conducted using this framework combined with the region growing algorithm and synthetic scenes created by a self-developed scene generator. The maximum reward could almost be achieved with a fine-tuned PointNet and by evaluating the wall and non-wall objects separately. This approach is not applicable to real-world scenes, which is necessary to achieve the efficiency and effectiveness objective. Therefore, another DRL partition approach is introduced, where an agent unifies superpoints in the so-called superpoint growing environment. The point cloud is divided into superpoints, which will be unified into the objects of interest by an agent. The experimental results show that this approach can be applied to real-world scenes. Beneath the application of DRL, an imitation learning approach was developed, increasing the agent’s performance in the superpoint growing environment. The runtime in the sequential superpoint growing environment is poor, as each union decision requires a neural network call. Hence, a further sub-research objective is to improve the runtime. An episodic environment was developed as a solution, only requiring one graph neural network call. Similarities between superpoints are estimated in this environment and passed to a union algorithm. The differences between two graph neural network architectures and two union algorithms were experimentally investigated. According to the results, calculating the superpoint similarities with a correlation of the embedded node features is more robust than the similarity estimation with a sigmoid activation function. The reward function, used in the DRL partition approaches, was realised by a matching procedure. As this function influences the partition quality, another sub-research objective is to investigate the differences between various match types. Matching functions from the literature were compared, and another match type was introduced. The usage of different match types in the learning process was experimentally evaluated. Although an agent gets more feedback with all match types, the best results (visual and in terms of the partition size) were achieved by only using first-order matches in the reward function. The synthetic scenes of the region growing approach lack realism as the lighting information is ignored, which can be important to train networks for the partition task. Therefore, a further sub-research objective is to develop a scene generator where the lighting is taken into account. After its development, the generated scenes were experimentally evaluated in a pre-training task. It turned out that the lighting information is important for a pre-training as larger accuracies were achieved. Furthermore, a faster convergence can be achieved with the pre-trained network instead of training a network on a target data set from scratch. Another sub-research objective targets the development of a usable partition interface. In this context, the Blender add-on OpenXtract was developed, containing five open-source point cloud partition algorithms. The partition algorithms were extended by approximating geodesic distances so that the edges of meshes are used. An experiment has shown that the extended algorithms produce larger accuracies, which is considered an increase in effectiveness. Moreover, unstructured interviews revealed that OpenXtract can improve the effectiveness and efficiency of the partition creation.

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar       frontdoor_oas    
Metadaten
Author:Marcel Tiator
URN:urn:nbn:de:hebis:ds114-opus4-4104
DOI:https://doi.org/10.48444/h_docs-pub-410
Subtitle (English):Striving for a more Efficient and Effective Process in the Development of 3D Applications
Publisher:Hochschule Darmstadt
Place of publication:Darmstadt
Referee:Alexander Gepperth, Stefanie Zollmann
Advisor:Paul Grimm, Christian Geiger
Document Type:Doctoral Thesis
Language:English
Year of Completion:2023
Publishing Institution:Hochschule Darmstadt
Granting Institution:Hochschule Darmstadt
Date of final exam:2023/10/11
Release Date:2023/12/06
Page Number:234
Institutes:Promotionszentrum Angewandte Informatik
OpenAIRE:OpenAIRE
Licence (German):License LogoCreative Commons - CC BY-SA - Namensnennung - Weitergabe unter gleichen Bedingungen 4.0 International