open_access
Refine
Year of publication
Document Type
- Article (15)
- Conference Proceeding (3)
- Preprint (2)
- Study Thesis (1)
Is part of the Bibliography
- yes (21) (remove)
Keywords
- Artificial Intelligence (3)
- DOAJ (2)
- Machine Learning (2)
- ADAS (1)
- AI performance metrics (1)
- Computer Vision (1)
- Computers and Society (1)
- DEAL (1)
- DFG Publikationskosten (1)
- Data Structures and Algorithms (1)
Department/institution
- Fachbereich - Elektro- & Informationstechnik (21) (remove)
We present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performance of ML-based algorithms from the optical quality over field, highlighting the need to take the spatial dimension into account when training ML-based algorithms, especially when looking forward to autonomous driving applications.
Spatial precision and recall indices to assess the performance of instance segmentation algorithms
(2022)
The importance of high data quality is increasing with the growing impact and distribution of ML systems and big data. Also, the planned AI Act from the European commission defines challenging legal requirements for data quality especially for the market introduction of safety relevant ML systems. In this paper, we introduce a novel approach that supports the data quality assurance process of multiple data quality aspects. This approach enables the verification of quantitative data quality requirements. The concept and benefits are introduced and explained on small example data sets. How the method is applied is demonstrated on the well-known MNIST data set based an handwritten digits.
The importance of high data quality is increasing with the growing impact and distribution of ML systems and big data. Also the planned AI Act from the European commission defines challenging legal requirements for data quality especially for the market introduction of safety relevant ML systems. In this paper we introduce a novel approach that supports the data quality assurance process of multiple data quality aspects. This approach enables the verification of quantitative data quality requirements. The concept and benefits are introduced and explained on small example data sets. How the method is applied is demonstrated on the well known MNIST data set based an handwritten digits.