TY - CHAP A1 - Lipp, Stephan A1 - Elsner, Daniel A1 - Hutzelmann, Thomas A1 - Banescu, Sebastian A1 - Pretschner, Alexander A1 - Böhme, Marcel T1 - FuzzTastic: A Fine-grained, Fuzzer-agnostic Coverage Analyzer T2 - Proceedings, 2022 ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings N2 - Performing sound and fair fuzzer evaluations can be challenging, not only because of the randomness involved in fuzzing, but also due to the large number of fuzz tests generated. Existing evaluations use code coverage as a proxy measure for fuzzing effectiveness. Yet, instead of considering coverage of all generated fuzz inputs, they only consider the inputs stored in the fuzzer queue. However, as we show in this paper, this approach can lead to biased assessments due to path collisions. Therefore, we developed FuzzTastic, a fuzzeragnostic coverage analyzer that allows practitioners and researchers to perform uniform fuzzer evaluations that are not affected by such collisions. In addition, its time-stamped coverage-probing approach enables frequency-based coverage analysis to identify barely tested source code and to visualize fuzzing progress over time and across code. To foster further studies in this field, we make FuzzTastic, together with a benchmark dataset worth ~12 CPU-years of fuzzing, publicly available; the demo video can be found at https://youtu.be/Lm-eBx0aePA. UR - https://doi.org/10.1145/3510454.3516847 Y1 - 2022 UR - https://doi.org/10.1145/3510454.3516847 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-54624 SN - 978-1-4503-9223-5 SP - 75 EP - 79 PB - ACM CY - New York ER -