TY - JOUR A1 - Krebs, Christian A1 - Müller, Patrick A1 - Braun, Alexander T1 - Impact of Windshield Optical Aberrations on Visual Range Camera Based Classification Tasks Performed by CNNs JF - London Imaging Meeting Y1 - 2021 SN - 0-89208-346-6 U6 - https://doi.org/10.2352/issn.2694-118X.2021.LIM-83 SN - 2694-118X VL - 2021 IS - 1 SP - 83 EP - 87 PB - Ingenta connect ER - TY - JOUR A1 - Müller, Patrick A1 - Brummel, Mattis A1 - Braun, Alexander T1 - Spatial recall index for machine learning algorithms JF - London Imaging Meeting N2 - We present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performance of ML-based algorithms from the optical quality over field, highlighting the need to take the spatial dimension into account when training ML-based algorithms, especially when looking forward to autonomous driving applications. Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-35657 SN - 0-89208-346-6 VL - 2021 IS - 1 SP - 58 EP - 62 PB - Society for Imaging Science and Technology ER - TY - CHAP A1 - Müller, Patrick A1 - Lehmann, Matthias A1 - Braun, Alexander ED - Kress, Bernard C. ED - Schelkens, Peter T1 - Optical quality metrics for image restoration T2 - Digital Optical Technologies 2019 N2 - Image restoration is a process used to remove blur (from different sources like object motion or aberrations) from images by either non-blind or blind-deconvolution. The metrics commonly used to quantify the restoration process are peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Often only a small sample of test images are used (like Lena or the camera guy). In optical design research PSNR and SSIM are not normally used, here image quality metrics based on linear system theory (e.g. modulation transfer function, MTF) are used to quantify optical errors like spherical or chromatic aberration. In this article we investigate how different image restoration algorithms can be quantified by applying image quality metrics. We start with synthetic image data that is used in camera test stands (e.g. Siemens star etc.), apply two different spatially variant degradation algorithms, and restore the original image by a direct method (Wiener filtering within sub-images), and by an iterative method (alternating direction method of multipliers, ADMM). Afterwards we compare the quality metrics (like MTF curves) for the original, the degraded and the restored image. As a first result we show that restoration algorithms sometimes fail in dealing with non-natural scenes, e.g. slanted-edge targets. Further, these first results indicate a correlation between degradation and restoration, i.e. the restoration algorithms are not capable of removing the optically relevant errors introduced by the degradation, a fact neither visible nor available from the PSNR values. We discuss the relevance in the context of the automotive industry, where image restoration may yield distinct advantages for camera-based applications, but testing methods rely on the used image quality metrics. Y1 - 2019 U6 - https://doi.org/10.1117/12.2528100 VL - Proceedings, Vol. 11062 CY - Munich ER - TY - JOUR A1 - Wohlers, Luis Constantin A1 - Müller, Patrick A1 - Braun, Alexander T1 - Original image noise reconstruction for spatially-varying filtered driving scenes JF - Electronic Imaging: Society for Imaging Science and Technology Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-41479 SN - 2470-1173 VL - 34 IS - 16 SP - 1 EP - 7 PB - Society for Imaging Science and Technology ER - TY - CHAP A1 - Müller, Patrick A1 - Braun, Alexander A1 - Keuper, Margret T1 - Impact of realistic properties of the point spread function on classification tasks to reveal a possible distribution shift T2 - NeurIPS 2022: Workshop on Distribution Shifts: Connecting Methods and Applications, December 3rd, 2022, New Orleans, USA KW - chromatic aberration KW - corruption KW - Deep Neural Networks KW - Distribution Shift KW - MTF Y1 - 2022 UR - https://openreview.net/forum?id=r7WJpE3oy0 PB - NeurIPS CY - New Orleans ER - TY - CHAP A1 - Müller, Patrick A1 - Braun, Alexander T1 - Simulating optical properties to access novel metrological parameter ranges and the impact of different model approximations T2 - 2022 IEEE International Workshop on Metrology for Automotive (MetroAutomotive), 4-6 July 2022 Y1 - 2022 SN - 978-1-6654-6689-9 U6 - https://doi.org/10.1109/MetroAutomotive54295.2022.9855079 SP - 133 EP - 138 PB - IEEE ER - TY - JOUR A1 - Müller, Patrick A1 - Braun, Alexander T1 - Local performance evaluation of AI-algorithms with the generalized spatial recall index JF - tm - Technisches Messen Y1 - 2023 U6 - https://doi.org/10.1515/teme-2023-0013 SN - 2196-7113 VL - 90 IS - 7-8 SP - 464 EP - 477 PB - De Gruyter ER - TY - JOUR A1 - Müller, Patrick A1 - Braun, Alexander T1 - MTF as a performance indicator for AI algorithms? JF - Electronic Imaging: Society for Imaging Science and Technology N2 - Abstract The modulation-transfer function (MTF) is a fundamental optical metric to measure the optical quality of an imaging system. In the automotive industry it is used to qualify camera systems for ADAS/AD. Each modern ADAS/AD system includes evaluation algorithms for environment perception and decision making that are based on AI/ML methods and neural networks. The performance of these AI algorithms is measured by established metrics like Average Precision (AP) or precision-recall-curves. In this article we research the robustness of the link between the optical quality metric and the AI performance metric. A series of numerical experiments were performed with object detection and instance segmentation algorithms (cars, pedestrians) evaluated on image databases with varying optical quality. We demonstrate with these that for strong optical aberrations a distinct performance loss is apparent, but that for subtle optical quality differences – as might arise during production tolerances – this link does not exhibit a satisfactory correlation. This calls into question how reliable the current industry practice is where a produced camera is tested end-of-line (EOL) with the MTF, and fixed MTF thresholds are used to qualify the performance of the camera-under-test. KW - AI performance metrics KW - Artificial Intelligence KW - Modulation Transfer Function (MTF) KW - Optical Quality KW - Perception Y1 - 2023 U6 - https://doi.org/10.2352/EI.2023.35.16.AVM-125 SN - 2470-1173 VL - 35 IS - 16 SP - 1 EP - 7 PB - Society for Imaging Science and Technology ER - TY - JOUR A1 - Brummel, Mattis A1 - Müller, Patrick A1 - Braun, Alexander T1 - Spatial precision and recall indices to assess the performance of instance segmentation algorithms JF - Electronic Imaging KW - Space-Variance KW - Computer Vision KW - IoU KW - image perturbations Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-41552 SN - 2470-1173 VL - 34 IS - 16 SP - 1 EP - 6 PB - Society for Imaging Science and Technology ER - TY - CHAP A1 - Müller, Patrick A1 - Braun, Alexander A1 - Keuper, Margret T1 - Classification robustness to common optical aberrations T2 - 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) KW - Maschinelles Sehen KW - Abbildungsfehler KW - Neuronales Netz KW - Benchmarking Y1 - 2023 U6 - https://doi.org/10.1109/ICCVW60793.2023.00391 SP - 3634 EP - 3645 PB - IEEE ER - TY - JOUR A1 - Molloy, Dara A1 - Müller, Patrick A1 - Deegan, Brian A1 - Mullins, Darragh A1 - Horgan, Jonathan A1 - Ward, Enda A1 - Jones, Edward A1 - Braun, Alexander A1 - Glavin, Martin T1 - Analysis of the Impact of Lens Blur on Safety-Critical Automotive Object Detection JF - IEEE Access N2 - Camera-based object detection is widely used in safety-critical applications such as advanced driver assistance systems (ADAS) and autonomous vehicle research. Road infrastructure has been designed for human vision, so computer vision, with RGB cameras, is a vital source of semantic information from the environment. Sensors, such as LIDAR and RADAR, are also often utilized for these applications; however, cameras provide a higher spatial resolution and color information. The spatial frequency response (SFR), or sharpness of a camera, utilized in object detection systems must be sufficient to allow a detection algorithm to localize objects in the environment over its lifetime reliably. This study explores the relationship between object detection performance and SFR. Six state-of-the-art object detection models are evaluated with varying levels of lens defocus. A novel raw image dataset is created and utilized, containing pedestrians and cars over a range of distances up to 100-m from the sensor. Object detection performance for each defocused dataset is analyzed over a range of distances to determine the minimum SFR necessary in each case. Results show that the relationship between object detection performance and lens blur is much more complex than previous studies have found due to lens field curvature, chromatic aberration, and astigmatisms. We have found that smaller objects are disproportionately impacted by lens blur, and different object detection models have differing levels of robustness to lens blur KW - DOAJ KW - Object detection KW - intelligent transportation system KW - autonomous vehicles KW - ADAS KW - lens blur Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-42999 SN - 2169-3536 VL - 12 SP - 3554 EP - 3569 PB - IEEE ER - TY - INPR A1 - Müller, Patrick A1 - Braun, Alexander A1 - Keuper, Margret T1 - Examining the Impact of Optical Aberrations to Image Classification and Object Detection Models N2 - Deep neural networks (DNNs) have proven to be successful in various computer vision applications such that models even infer in safety-critical situations. Therefore, vision models have to behave in a robust way to disturbances such as noise or blur. While seminal benchmarks exist to evaluate model robustness to diverse corruptions, blur is often approximated in an overly simplistic way to model defocus, while ignoring the different blur kernel shapes that result from optical systems. To study model robustness against realistic optical blur effects, this paper proposes two datasets of blur corruptions, which we denote OpticsBench and LensCorruptions. OpticsBench examines primary aberrations such as coma, defocus, and astigmatism, i.e. aberrations that can be represented by varying a single parameter of Zernike polynomials. To go beyond the principled but synthetic setting of primary aberrations, LensCorruptions samples linear combinations in the vector space spanned by Zernike polynomials, corresponding to 100 real lenses. Evaluations for image classification and object detection on ImageNet and MSCOCO show that for a variety of different pre-trained models, the performance on OpticsBench and LensCorruptions varies significantly, indicating the need to consider realistic image corruptions to evaluate a model's robustness against blur. KW - Maschinelles Lernen KW - Maschinelles Sehen KW - Tiefes neuronales Netz KW - Objekterkennung KW - Abbildungsfehler Y1 - 2025 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:due62-opus-55457 PB - arXiv ET - v1 ER - TY - JOUR A1 - Müller, Patrick A1 - Braun, Alexander A1 - Keuper, Margret T1 - Examining the Impact of Optical Aberrations to Image Classification and Object Detection Models JF - IEEE Transactions on Pattern Analysis and Machine Intelligence KW - Optical imaging KW - Lenses KW - Robustness KW - Kernel KW - Optical sensors Y1 - 2025 U6 - https://doi.org/10.1109/TPAMI.2025.3622234 SN - 0162-8828 SP - 1 EP - 15 PB - IEEE ER -