TY - GEN A1 - Herglotz, Christian A1 - Och, Hannah A1 - Meyer, Anna A1 - Ramasubbu, Geetha A1 - Eichermüller, Lena A1 - Kränzler, Matthias A1 - Brand, Fabian A1 - Fischer, Kristian A1 - Nguyen, Dat Thanh A1 - Regensky, Andy A1 - Kaup, André T1 - The Bjøntegaard Bible Why Your Way of Comparing Video Codecs May Be Wrong T2 - IEEE Transactions on Image Processing N2 - In this paper, we provide an in-depth assessment on the Bjøntegaard Delta. We construct a large data set of video compression performance comparisons using a diverse set of metrics including PSNR, VMAF, bitrate, and processing energies. These metrics are evaluated for visual data types such as classic perspective video, 360° video, point clouds, and screen content. As compression technology, we consider multiple hybrid video codecs as well as state-of-the-art neural network based compression methods. Using additional supporting points in-between standard points defined by parameters such as the quantization parameter, we assess the interpolation error of the Bjøntegaard-Delta (BD) calculus and its impact on the final BD value. From the analysis, we find that the BD calculus is most accurate in the standard application of rate-distortion comparisons with mean errors below 0.5 percentage points. For other applications and special cases, e.g., VMAF quality, energy considerations, or inter-codec comparisons, the errors are higher (up to 5 percentage points), but can be halved by using a higher number of supporting points. We finally come up with recommendations on how to use the BD calculus such that the validity of the resulting BD-values is maximized. Main recommendations are as follows: First, relative curve differences should be plotted and analyzed. Second, the logarithmic domain should be used for saturating metrics such as SSIM and VMAF. Third, BD values below a certain threshold indicated by the subset error should not be used to draw recommendations. Fourth, using two supporting points is sufficient to obtain rough performance estimates. KW - Measurement KW - Calculus KW - Codecs KW - Interpolation KW - Image coding KW - Distortion KW - Visualization Y1 - 2024 U6 - https://doi.org/10.1109/TIP.2023.3346695 SN - 1057-7149 SN - 1941-0042 IS - Volume 33 SP - 987 EP - 1001 ER - TY - GEN A1 - Herglotz, Christian A1 - Kränzler, Matthias A1 - Chu, Xixue A1 - François, Edouard A1 - He, Yong A1 - Kaup, André T1 - Extended Signaling Methods for Reduced Video Decoder Power Consumption Using Green Metadata T2 - IEEE Transactions on Circuits and Systems II: Express Briefs N2 - In this brief, we discuss one aspect of the latest MPEG standard edition on energy-efficient media consumption, also known as Green Metadata (ISO/IEC 232001-11), which is the interactive signaling for remote decoder-power reduction for peer-to-peer video conferencing. In this scenario, the receiver of a video, e.g., a battery-driven portable device, can send a dedicated request to the sender which asks for a video bitstream representation that is less complex to decode and process. Consequently, the receiver saves energy and extends operating times. We provide an overview on latest studies from the literature dealing with energy-saving aspects, which motivate the extension of the legacy Green Metadata standard. Furthermore, we explain the newly introduced syntax elements and verify their effectiveness by performing dedicated experiments. We show that the integration of these syntax elements can lead to dynamic energy savings of up to 90% for software video decoding and 80% for hardware video decoding, respectively. KW - Decoding KW - Streaming media KW - Syntactics KW - Metadata KW - Standards KW - Receivers KW - Software Y1 - 2024 U6 - https://doi.org/10.1109/TCSII.2023.3328405 SN - 1549-7747 SN - 1558-3791 VL - Volume 71 IS - Issue 3 SP - 1141 EP - 1145 ER - TY - GEN A1 - Ramasubbu, Geetha A1 - Kaup, André A1 - Herglotz, Christian T1 - Modeling the Energy Consumption of the HEVC Software Encoding Process using Processor events T2 - IEEE 26th International Workshop on Multimedia Signal Processing (MMSP) N2 - Developing energy-efficient video encoding algorithms is highly important due to the high processing complexities and, consequently, the high energy demand of the encoding process. To accomplish this, the energy consumption of the video encoders must be studied, which is only possible with a complex and dedicated energy measurement setup. This emphasizes the need for simple energy estimation models, which estimate the energy required for the encoding. Our paper investigates the possibility of estimating the energy demand of a HEVC software CPU-encoding process using processor events. First, we perform energy measurements and obtain processor events using dedicated profiling software. Then, by using the measured energy demand of the encoding process and profiling data, we build an encoding energy estimation model that uses the processor events of the ultrafast encoding preset to obtain the energy estimate for complex encoding presets with a mean absolute percentage error of 5.36% when averaged over all the presets. Additionally, we present an energy model that offers the possibility of obtaining energy distribution among various encoding sub-processes. energy models from literature. By using a unified evaluation framework we show how accurately the required decoding energy for different decoding systems can be approximated. We give thorough explanations on the model parameters and explain how the model variables are derived. To show the modeling capabilities in general, we test the estimation performance for different decoding software and hardware solutions, where we find that the proposed model outperforms the models from literature by reaching frame-wise mean estimation errors of less than 7% for software and less than 15% for hardware based systems. KW - Energy consumption KW - Analytical models KW - Software algorithms KW - Estimation KW - Energy measurement KW - Signal processing algorithms KW - Streaming media KW - Encoding KW - Software KW - Energy efficiency Y1 - 2024 SN - 979-8-3503-8725-4 SN - 979-8-3503-8726-1 U6 - https://doi.org/10.1109/MMSP61759.2024.10743858 SN - 2473-3628 SN - 2163-3517 PB - IEEE ER - TY - GEN A1 - Herglotz, Christian A1 - Kränzler, Matthias A1 - Xu, Bide A1 - Kaup, André T1 - Decoding Energy Optimization for Video Coding Using Model-Driven Gradient Descent T2 - IEEE 26th International Workshop on Multimedia Signal Processing (MMSP) N2 - Nowadays, a large part of the global energy consumption caused by video communications can be attributed to end-user devices such as smartphones, tablet PCs, and TV sets. In this paper, we present a method to increase the performance of an existing algorithm dedicated to reduce the end-user side energy consumption during video streaming. The algorithm, which is called decoding-energy-rate-distortion optimization (DERDO), exploits a decoding energy model during encoding and chooses coding modes in such a way that the software decoding energy is minimized. In this paper, we develop a dedicated gradient descent approach for DERDO that refines specific energy coefficients used for decoding energy modeling. We find that this approach boosts the performance of DERDO by increasing the energy savings by at least 5% with respect to standard DERDO. As a consequence, we observe decoding energy savings of more than 40% and more than 7% for practical encoder and decoder implementations of HEVC and H.264/AVC, respectively, when compared to standard encoding using classic rate-distortion optimization. KW - Energy consumption KW - Energy conservation KW - Software algorithms KW - Signal processing algorithms KW - Streaming media KW - Encoding KW - Software KW - Decoding KW - Optimization Y1 - 2024 SN - 979-8-3503-8725-4 SN - 979-8-3503-8726-1 U6 - https://doi.org/10.1109/MMSP61759.2024.10743503 SN - 2473-3628 SN - 2163-3517 ER - TY - GEN A1 - Rückert, Rainer A1 - Ullmann, Ingrid A1 - Herglotz, Christian A1 - Kaup, André A1 - Vossiek, Martin T1 - Data Compression for Close-Range Radar Imaging T2 - IEEE Transactions on Radar Systems N2 - The resolution of radar images is constantly increasing. As a result, radar images require more storage space, which is associated with increased costs. Therefore, it is advantageous to minimize the data size. In this paper, we present various compression methods for reducing the data size of radar images. Compression and decompression are performed in two scenarios. In the first scenario, the raw data are compressed and decompressed before the image is reconstructed. In the second scenario, the reconstructed image itself is compressed and decompressed. In both scenarios, the reconstructed radar image is compared with the original image. Due to its widespread use, High-Efficiency Video Coding (HEVC) is used as a state-of-the-art benchmark for both scenarios and compared with proprietary algorithms that combine lossy and lossless compression. A discrete Fourier transform–based compression algorithm from the automotive sector is used as another state-of-the-art benchmark. This is applied against our novel approaches, which are based on the discrete cosine transform, use of direct thresholding in the spatial domain, or are applied to the maximum intensity projection. With the exception of HEVC, all algorithms presented have in common that they perform lossy data processing in the first step and then use the Lempel–Ziv–Markov algorithm as a lossless compression step. To compare the compression ratios, we use various image- and video-specific metrics, such as the peak signal–to-noise ratio (PSNR), the similarity of speeded-up robust features, and the structural similarity index measure (SSIM). For a simple classification, we use Otsu’s method to examine the effects of compression on the images. The radar images are categorized into transparent and nontransparent based on the measurement objects. Depending on the application and the desired resolution, our approaches can achieve storage savings of up to 99.93 % compared to the uncompressed data with PSNR and SSIM values of 38.8 dB and 0.916, respectively. KW - Radar imaging KW - Radar KW - Image coding KW - Antenna measurements KW - Radar measurements KW - Image resolution KW - Data compression Y1 - 2024 U6 - https://doi.org/10.1109/TRS.2024.3387288 SN - 2832-7357 SP - 421 EP - 433 ER - TY - GEN A1 - Eichermüller, Lena A1 - Chaudhari, Gaurang A1 - Katsavounidis, Ioannis A1 - Lei, Zhijun A1 - Tmar, Hassene A1 - Herglotz, Christian A1 - Kaup, André T1 - Encoding Time and Energy Model for SVT-AV1 Based on Video Complexity T2 - ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - The share of online video traffic in global carbon dioxide emissions is growing steadily. To comply with the demand for video media, dedicated compression techniques are continuously optimized, but at the expense of increasingly higher computational demands and thus rising energy consumption at the video encoder side. In order to find the best trade-off between compression and energy consumption, modeling encoding energy for a wide range of encoding parameters is crucial. We propose an encoding time and energy model for SVT-AV1 based on empirical relations between the encoding time and video parameters as well as encoder configurations. Furthermore, we model the influence of video content by established content descriptors such as spatial and temporal information. We then use the predicted encoding time to estimate the required energy demand and achieve a prediction error of 19.6% for encoding time and 20.9% for encoding energy. KW - Energy consumption KW - Estimation error KW - Multicore processing KW - Computational modeling KW - Signal processing KW - Media KW - Encoding Y1 - 2024 SN - 979-8-3503-4485-1 SN - 979-8-3503-4486-8 U6 - https://doi.org/10.1109/ICASSP48485.2024.10446602 SN - 2379-190X SN - 1520-6149 SP - 3370 EP - 3374 PB - IEEE ER - TY - GEN A1 - Eichermüller, Lena A1 - Chaudhari, Gaurang A1 - Katsavounidis, Ioannis A1 - Lei, Zhijun A1 - Tmar, Hassene A1 - Herglotz, Christian A1 - Kaup, André T1 - SVT-AV1 Encoding Bitrate Estimation Using Motion Search Information T2 - 32nd European Signal Processing Conference (EUSIPCO) N2 - Enabling high compression efficiency while keeping encoding energy consumption at a low level, requires prioritization of which videos need more sophisticated encoding techniques. However, the effects vary highly based on the content, and information on how good a video can be compressed is required. This can be measured by estimating the encoded bitstream size prior to encoding. We identified the errors between estimated motion vectors from Motion Search, an algorithm that predicts temporal changes in videos, correlates well to the encoded bitstream size. Combining Motion Search with Random Forests, the encoding bitrate can be estimated with a Pearson correlation of above 0.96. KW - Bit rate KW - Estimation KW - Signal processing algorithms KW - Signal processing KW - Feature extraction KW - Size measurement KW - Encoding KW - Vectors KW - Random forests KW - Videos Y1 - 2024 SN - 978-9-4645-9361-7 SN - 979-8-3315-1977-3 SN - 2076-1465 SN - 2219-5491 SP - 937 EP - 941 PB - IEEE ER - TY - GEN A1 - Stürzenhofäcker, Teresa A1 - Kränzler, Matthias A1 - Herglotz, Christian A1 - Kaup, André T1 - Design Space Exploration at Frame-Level for Joint Decoding Energy and Quality Optimization in VVC T2 - 2024 32nd European Signal Processing Conference (EUSIPCO) N2 - In the pursuit of a reduced energy demand of VVC decoders, it was found that the coding tool configuration has a substantial influence on the bit rate efficiency and the decoding energy demand. The Advanced Design Space Exploration algorithm as proposed in the literature, can derive coding tool configurations that provide optimal trade-offs between rate and energy efficiency. Yet, some trade-off points in the design space cannot be reached with the state-of-the-art methodology, which defines coding tools for an entire bitstream. This work proposes a novel, granular adjustment of the coding tool usage in VVC. Consequently, the optimization algorithm is adjusted to explore coding tool configurations that operate on frame-level. Moreover, new optimization criteria are introduced to focus the search on specific bit rates. As a result, coding tool configurations are obtained which yield so far inaccessible trade-offs between bit rate efficiency and decoding energy demand for VVC-coded sequences. The proposed methodology extends the design space and enhances the continuity of the Pareto front. KW - Design methodology KW - Bit rate KW - Signal processing algorithms KW - Europe , Signal processing KW - Encoding KW - Energy efficiency KW - Decoding KW - Space exploration KW - Optimization Y1 - 2024 SN - 978-9-4645-9361-7 SN - 979-8-3315-1977-3 SN - 2076-1465 SN - 2219-5491 SP - 932 EP - 936 PB - IEEE ER - TY - GEN A1 - Eichermüller, Lena A1 - Chaudhari, Gaurang A1 - Katsavounidis, Ioannis A1 - Lei, Zhijun A1 - Tmar, Hassene A1 - Herglotz, Christian A1 - Kaup, Andre T1 - Encoding Time and Energy Model for SVT-AV1 based on Video Complexity T2 - arXiv - accepted for IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) N2 - The share of online video traffic in global carbon dioxide emissions is growing steadily. To comply with the demand for video media, dedicated compression techniques are continuously optimized, but at the expense of increasingly higher computational demands and thus rising energy consumption at the video encoder side. In order to find the best trade-off between compression and energy consumption, modeling encoding energy for a wide range of encoding parameters is crucial. We propose an encoding time and energy model for SVT-AV1 based on empirical relations between the encoding time and video parameters as well as encoder configurations. Furthermore, we model the influence of video content by established content descriptors such as spatial and temporal information. We then use the predicted encoding time to estimate the required energy demand and achieve a prediction error of 19.6 % for encoding time and 20.9 % for encoding energy. Y1 - 2024 U6 - https://doi.org/10.48550/arXiv.2401.16067 ER - TY - GEN A1 - Ramasubbu, Geetha A1 - Kaup, André A1 - Herglotz, Christian T1 - Towards Video Codec Performance Evaluation: A Rate-Energy-Distortion Perspective T2 - 16th International Conference on Quality of Multimedia Experience (QoMEX) N2 - The Bjøntegaard Delta rate (BD-rate) objectively assesses the coding efficiency of video codecs using the rate-distortion (R-D) performance but overlooks encoding energy, which is crucial in practical applications, especially for those on handheld devices. Although R-D analysis can be extended to incorporate encoding energy as energy-distortion (E-D), it fails to integrate all three parameters seamlessly. This work proposes a novel approach to address this limitation by introducing a 3D representation of rate, encoding energy, and distortion through surface fitting. In addition, we evaluate various surface fitting techniques based on their accuracy and investigate the proposed 3D representation and its projections. The overlapping areas in projections help in encoder selection and recommend avoiding the slow presets of the older encoders (x264, x265), as the recent encoders (x265, VVenC) offer higher quality for the same bitrate-energy performance and provide a lower rate for the same energy-distortion performance. KW - Performance evaluation KW - Video coding KW - Three-dimensional displays KW - Fitting KW - Rate-distortion KW - Rate distortion theory KW - Distortion Y1 - 2024 U6 - https://doi.org/10.1109/QoMEX61742.2024.10598269 SN - 2472-7814 SN - 2372-7179 SP - 96 EP - 99 ER - TY - GEN A1 - Kränzler, Matthias A1 - Herglotz, Christian A1 - Kaup, André T1 - A Comprehensive Review of Software and Hardware Energy Efficiency of Video Decoders T2 - Picture Coding Symposium (PCS) 2024 N2 - Energy and compression efficiency are two essential parts of modern video decoder implementations that have to be considered. This work comprehensively studies the following six video coding formats regarding compression and decoding energy efficiency: AVC, VP9, HEVC, AV1, VVC, and AVM. We first evaluate the energy demand of reference and optimized software decoder implementations. Furthermore, we consider the influence of the usage of SIMD instructions on those decoder implementations. We find that AV1 is a sweet spot for optimized software decoder implementations with an additional energy demand of 16.55% and bitrate savings of -43.95% compared to VP9. We furthermore evaluate the hardware decoding energy demand of four video coding formats. Thereby, we show that AV1 has energy demand increases by 117.50% compared to VP9. For HEVC, we found a sweet spot in terms of energy demand with an increase of 6.06% with respect to VP9. Relative to their optimized software counterparts, hardware video decoders reduce the energy consumption to less than 9% compared to software decoders KW - Energy consumption KW - Reviews KW - Switches KW - Energy efficiency KW - Software KW - Hardware KW - Encoding Y1 - 2024 U6 - https://doi.org/10.1109/PCS60826.2024.10566363 SN - 2472-7822 SN - 2330-7935 PB - IEEE ER - TY - GEN A1 - Herglotz, Christian A1 - Kränzler, Matthias A1 - Dai, Rui A1 - Kaup, André T1 - Complexity Metrics for VVC Decoder Power Reduction in Green Metadata T2 - Picture Coding Symposium (PCS) 2024 N2 - This paper discusses the new complexity metrics for VVC decoder power reduction introduced in the 3 rd edition of the Green Metadata standard. The standard defines dedicated syntax elements that represent the expected software decoding complexity with a high accuracy. Using a simple complexity model, which can be trained for any VVC decoder software implementation, the receiver of the video can estimate the processing complexity to decode the subsequent video segment. Afterwards, it can adjust the clock frequency of the processor to keep the real-time playback constraint. By reducing the frequency of the processor, a significant amount of energy is saved. In this paper, we present the syntax elements, their meaning, and show that they can accurately estimate the processing complexity of various software decoder implementations with errors below 11%. Furthermore, we present a processor-frequency-control algorithm and apply it to a development board performing VVC video decoding. Measurements reveal that the complexity metric signaling can lead to up to 30% of energy savings. KW - Measurement KW - Support vector machines KW - Accuracy KW - Metadata KW - Syntactics KW - Streaming media KW - Software Y1 - 2024 SN - 979-8-3503-5848-3 SN - 979-8-3503-5849-0 U6 - https://doi.org/10.1109/PCS60826.2024.10566419 SN - 2472-7822 SN - 2330-7935 SP - 1 EP - 5 PB - IEEE ER -