Spatial recall index for machine learning algorithms

  • We present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performanceWe present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performance of ML-based algorithms from the optical quality over field, highlighting the need to take the spatial dimension into account when training ML-based algorithms, especially when looking forward to autonomous driving applications.show moreshow less

Download full text files

  • This work is licensed under the Creative Commons Attribution 4.0 International License.

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Metadaten
Author:Patrick Müller, Mattis Brummel, Alexander BraunORCiD
open access (DINI-Set):open_access
Qualitätssicherung:peer reviewed
open access :Gold - Erstveröffentlichung mit Lizenzhinweis
Fachbereich/Einrichtung:Hochschule Düsseldorf / Fachbereich - Elektro- & Informationstechnik
Document Type:Article
Year of Completion:2021
Language of Publication:English
Publisher:Society for Imaging Science and Technology
Parent Title (English):London Imaging Meeting
Volume:2021
Issue:1
First Page:58
Last Page:62
URN:urn:nbn:de:hbz:due62-opus-35657
DOI:https://doi.org/10.2352/issn.2694-118X.2021.LIM-58
ISBN:0-89208-346-6
Dewey Decimal Classification:6 Technik, Medizin, angewandte Wissenschaften / 62 Ingenieurwissenschaften / 620 Ingenieurwissenschaften und zugeordnete Tätigkeiten
Licence (German):Creative Commons - CC BY - Namensnennung 4.0 International
Release Date:2022/01/31
Accept ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.