TY - THES A1 - Bhanderi, Savankumar T1 - Real-time semantic and instance segmentation of 3D radar point cloud for smart infrastructure-based road user detection N2 - The next generation 3 + 1D mmWave automotive Radar (Radio Detection and Ranging) sensors provide additional elevation information along with 2D object location and their relative radial velocity. They are proven to be robust against adverse weather and poor lighting conditions, while also having a long range sensing ability. Due to these advantages, they are widely used in research focusing on autonomous driving functions where scene awareness is a critical component in the decision-making process. However, most of existing literature suffers from one of two limitations: 1) the use of clustering-based approaches, which provide suboptimal performance, or 2) high inference time. Furthermore, the research involving the applicability of Radar in the emerging field of smart roadside perception is still limited. To bridge this gap, a novel roadside perception pipeline is proposed in this thesis, which is capable of real-time inference using automotive Radar point clouds. While deep learning has revolutionized image processing, its application to automotive Radar point cloud data, especially from smart roadside infrastructure units, remains under-explored. In this thesis, deep learning techniques are used for semantic and instance segmentation on 3 + 1D Radar point cloud data, targeting key road users such as person, bicycle, motorcycle, car, and bus. Additionally, the background class is also considered to account for the Radar clutter. Due to the real-time inference requirements of roadside perception tasks, the proposed pipeline is constructed with as few parameters as possible while also maintaining the desired performance. The main components of the pipeline are MLP (Multi Layer Perceptron) and the self-attention mechanism, which are used for semantic and instance segmentation respectively. Additionally, exploiting the static field of view of the sensor, a 3D background subtraction method is applied to Radar point clouds to further reduce processing time. Experimental results demonstrate a 95.35% F 1 -macro averaged score for semantic segmentation, and 91.03% mAP (mean Average Precision) at an IoU (Intersection over Union) threshold of 0.5 for instance segmentation on a test set. Furthermore, a COCO (Common Objects in Context) mAP of 80.01% is achieved by the method, indicating high performance across a range of overlaps. The complete system, including the neural network, achieves an impressive inference time of just 22.93 milliseconds on an edge device (Nvidia Jetson AGX Orin), yielding a frame rate of 43.61, with a memory requirement of less than 0.7 MB. Thus, it is suitable for real-time roadside perception in ITS (Intelligent Transportation System). Y1 - 2024 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-51450 CY - Ingolstadt ER -