TY - THES A1 - Brummel, Mattis T1 - Performance evaluation of state-of-the-art computer vision systems in the field of autonomous driving in dependency of optical parameters KW - Master KW - SoSe21 Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-37508 CY - Düsseldorf ER - TY - JOUR A1 - Brummel, Mattis A1 - Müller, Patrick A1 - Braun, Alexander T1 - Spatial precision and recall indices to assess the performance of instance segmentation algorithms JF - Electronic Imaging KW - Space-Variance KW - Computer Vision KW - IoU KW - image perturbations Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-41552 SN - 2470-1173 VL - 34 IS - 16 SP - 1 EP - 6 PB - Society for Imaging Science and Technology ER - TY - JOUR A1 - Müller, Patrick A1 - Brummel, Mattis A1 - Braun, Alexander T1 - Spatial recall index for machine learning algorithms JF - London Imaging Meeting N2 - We present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performance of ML-based algorithms from the optical quality over field, highlighting the need to take the spatial dimension into account when training ML-based algorithms, especially when looking forward to autonomous driving applications. Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-35657 SN - 0-89208-346-6 VL - 2021 IS - 1 SP - 58 EP - 62 PB - Society for Imaging Science and Technology ER -