Refine
Year of publication
- 2021 (9) (remove)
Document Type
Language
- English (9)
Publication reviewed
- begutachtet (9)
Keywords
- Advanced Driver Assistance Systems (3)
- Autoamted Driving (2)
- Automated Driving (2)
- Adadvanced driver assistance systems (1)
- Attribute (1)
- Automated vehicles (1)
- Data Analysis (1)
- Hardware-in-the-Loop (1)
- METAVI (1)
- MLPaSSAD (1)
Safeguarding and type approval of automated vehicles is a key enabler for their market launch in our complex traffic environment. Scenario-based testing by means of computer simulation is becoming increasingly important to cope with the enormous complexity and effort. However, there is a huge gap when assessing the safety of the virtual vehicle while the real vehicle will drive on the road. Simulation must be accompanied by model validation to ensure its credibility since errors and uncertainties are inherent in every model. Unfortunately, this is rarely addressed in the current literature. In this paper, a modular process is presented covering both model validation and safeguarding. It is characterized by the fact that it quantifies a large number of errors and uncertainties, represents them in the form of an error model, and ultimately integrates them into the safeguarding results. It is applied to a type-approval regulation for the lane-keeping behavior of a vehicle under various scenario conditions. The paper contains a thorough validation of the methodology itself by comparing its results with actual ground truth values. For this comparison, a binary classifier and confusion matrices are used that relate the binary type-approval decisions. The classifier demonstrates that the methodology of this paper identifies a systematic error of the simulation model across several safeguarding scenarios. Finally, the paper provides recommendations for alternative configurations of the modular methodology depending on different requirements.
Recently the trend of driving simulators with driver in the loop (DiL) integration in the development process has become more and more visible. BMW opened a completely new simulation center, Daimler and Toyota have already had theirs in operation for some years. The reasons are well known and are presented in conferences and written down in papers: Reducing development time, reducing prototypes, reducing costs and increasing overall performance and efficiency. The same benefits are promised in papers about pure simulation, but what is the actual benefit from a dynamic driving simulator? The investment and operating costs are very high and yet it is crucial to bring the driver in to the loop. Obviously, the maturity level or knowledge about pure simulation and understanding for human drivers are still insufficient for a major breakthrough of simulation in many fields of application. A driving simulator connects the real and virtual world, by which humans experience functions and characteristics subjectively. Crucial decisions are made on reliable subjective feedback and humans especially become part of the left arm of the V-model development process. The current approach is using the same models and simulation environments combined with a driving simulator transferring simple signals from visible into feelable. The disadvantage of closing open control loops with a driver is that the objective data and maneuver quality largely depends on the driver. Assuming the driver is well qualified, the interface must reflect the real driving experience. Otherwise, every system/component under test will be assessed with an offset, filter or error. Besides the visualization, the steering feel is the most important channel for the driver to properly control any lateral movement. The common approach is using a force feedback system. A look on a steering system illustrates the complexity to ensure good steering feel.
The proposed contribution addresses the growing need for systematic and efficient development methods for assisted and automated driving features. Active Lane Departure Warning (ALDW) systems with correcting steering interventions are the specific object of this research. Due to NCAP requirements ALDW systems are widespread in modern cars, but often lack drivers’ approval, as several related studies have shown. Causes are assumed to lie in unspecified attribute requirement metrics and missing target values. Based on typical ALDW usage scenarios on different road types, such as freeway, highway and country road, an advanced catalogue of subjective evaluation criteria was developed
and deployed in a comprehensive proband study with experts involving four distinct ALDW equipped passenger cars. Characteristic driving maneuvers with the intention to provoke controlled ALDW interventions complemented the subjective assessment with objective data. Statistically correlating derived objective key performance indicators with the respective subjective ratings established a link between drivers’ impressions and measurable system behavior. These dependencies allowed the conclusion of target values for a certain range of desired subjective ALDW characteristics fulfilling the initial requisites.
Today’s validation of driver assistance systems and automated driving functions in state-of-the-art vehicles, still takes place mostly on the proving ground or on the real road. But driving tests can only be performed with the finished vehicle and fully implemented function, which might be difficult in early stages of the development process. Those purely physical tests are also time consuming and lack reproducibility. Hence, the validation process is guided more and more by simulation tools to reduce the number of tests that have to be driven. The pure simulation offers benefits, such as the functions can be tested in early stages of the development without the need for a physical carrier vehicle. Also, the flexibility of the simulation is high, since all variations of driving scenarios, also safety critical, can be driven easily and much faster than real time. On the other side, compared to the physical test, the parameterization of supplemental models is sometimes difficult and the risk of errors due to model abstraction is high.
A major concern about advanced motion-based simulators is their level of fidelity i.e., how close the motion sensation in a simulator is to the one perceived in a real vehicle. In this study, we collect the assessment from an exceptional sample composed by n = 33 automotive industry experts who were asked to evaluate the fidelity in terms of steering, braking and speed. Given the subjective nature of our measure, we propose a censored-data Tobit regression model that accounts for this issue, thus providing more accurate estimations. Our results show that, on average, experts evaluated the steering actions close to the maximum level of fidelity. However, braking and speed were evaluated lower in realism, and in fact both diminished the overall fidelity judgement by up to 50%. Moreover, coefficients indicate that steering contributes more to the judgement of fidelity than braking and speed actions. Heterogeneity in the experts' responses and general implications are discussed.
This paper presents a detailed analysis and characterization of Subjective Assessment Indicators for evaluating manual as well as fully automatic parking maneuvers. Parking is a huge challenge for many drivers. With the introduction of autonomous driving, parking maneuver assistants are essential functional components. For the development of automatic parking assistants, a detailed characterization of a subjective evaluation is essential. The characterization analysis presented here is based on general Subjective Assessment Indicators, which cover the subjective overall performance of a parking maneuver on a customer-oriented level in as many facets as necessary. This paper shows meaningful characteristics of the individual Subjective Assessment Indicators validated in a driving study with 497 performed parking maneuvers. The study results reveal different degrees of intensity of the characterizations for the different driving maneuvers. Here, it is shown that the characterization of the Final Parking Position has different reference points for longitudinal and lateral parking maneuvers. Furthermore, it was shown that an additional characteristic ‘‘Driving-Off Behavior’’ is required for the evaluation of the Safety Feeling, but for Parking Comfort the ‘‘Lateral Acceleration’’ and for Dynamic Performance the ‘‘Distance Traveled’’ can be neglected. The characteristics described in this paper can be used for all parking maneuvers and vehicle types. It forms the basis for a complete evaluation and enables OEMs to apply their individual requirements in the development of parking assistants.
Parking – Evaluation of Manual and Automated Parking Maneuvers with Subjective Assessment Indicators
(2021)
In this paper, an analysis of a subjective evaluation of manually and automatically executed longitudinal and lateral parking maneuvers using Subjective Assessment Indicators is presented. With the introduction of autonomous driving, parking maneuver assistants are essential functional components. Driver assistance systems will only be accepted if they perform decisively better than the human driver. Whether the performance of such a system meets expectations is ultimately a subjective impression. For this reason, an analysis of the parking performance of humans and parking assistant systems is carried out based on a new innovative subjective evaluation method. This new subjective evaluation method is based on the so-called Subjective Assessment Indicators which cover the relevant areas of a parking maneuver but still do not reach a level of detail that makes evaluation unsuitable for customer. Using the new subjective evaluation method, a driving study was conducted with 21 participants and two different test vehicles. The participants evaluated both manual and fully automated longitudinal and lateral parking maneuvers purely digitally using an evaluation app. As the results of the study show, parking assistants still have notable deficits compared to human performance in some evaluation areas and show considerable potential for improvement. As the subjective evaluation method used is suitable for all parking maneuvers and vehicle types, the results of this and potentially further studies form the basis for determining Key Performance Indicators for parking maneuvers. This enables virtual development of automated parking systems, as a link can be established to subjective customer evaluations.
One of the most commonly used advanced driver assistance system is adaptive cruise control. Although many cars are equipped with such a driver assistance system, the development is still based on subjective evaluation indices. The design of adaptive cruise control could be more powerful and reach more customer acceptance when a design process with specific values to ensure good driving characteristics is used. Such an objective based process is still state of the art of classic chassis development, but typically not used for advanced driver assistance systems.
To achieve this, use case-based scenarios are deduced from properties. A scenario simulation is used to generate information about important KPIs and collect data to develop an automated data analysis tool. To correlate subjective KPIs with objective measurement values an inertial navigation system in each car with differential GPS (RTK) and a wireless connection between both cars is installed. With the aid of this measurement system differential speed, acceleration and distances are calculated in a precise way. It also contains a fully controllable target vehicle in longitudinal and lateral direction to generate manoeuvres with high reproducibility. An analysis algorithm automatically calculates KPIs and displays important diagrams after each test run. This leads to a complete system analysis test in two days for each car. The comparison of target KPIs and subjective evaluation criteria shows whether the system reached its goals or if improvements are necessary. For this purpose the project partners Porsche, University of Applied Sciences Kempten and MdynamiX have joined their forces.
Due to the rapid progress in the development of automated vehicles over the last decade, their market entry is getting closer. One of the remaining challenges is the safety assessment and type approval of automated vehicles, as conventional testing in the real world would involve an unmanageable mileage. Scenario-based testing using simulation is a promising candidate for overcoming this approval trap. Although the research community has recognized the importance of safeguarding in recent years, the quality of simulation models is rarely taken into account. Without investigating the errors and uncertainties of models, virtual statements about vehicle safety are meaningless. This paper describes a whole process combining model validation and safety assessment. It is demonstrated by means of an actual type-approval regulation that deals with the safety assessment of lane-keeping systems. Based on a thorough analysis of the current state-of-the-art, this paper introduces two approaches for selecting test scenarios. While the model validation scenarios are planned from scratch and focus on scenario coverage, the type-approval scenarios are extracted from measurement data based on a data-driven pipeline. The deviations between lane-keeping behavior in the real and virtual world are quantified using a statistical validation metric. They are then modeled using a regression technique and inferred from the validation experiments to the unseen virtual type-approval scenarios. Finally, this paper examines safety-critical lane crossings, taking into account the modeling errors. It demonstrates the potential of the virtual-based safeguarding process using exemplary simulations and real driving tests.